Skip to main content

Lessons Learned

Lessons Learned

CHECK OUT OUR MOST RELEVANT INCIDENT LISTINGS!

Disclaimer: The Lessons Learned Database includes the incidents that were voluntarily submitted. The database is not a comprehensive source for all incidents that have occurred.

Description
Characteristics: High Pressure (> 100 bar)
Contributing Factors:
Incident Date:
Severity: Incident
Leak: No
Ignition: No
Ignition Source:
Probable Cause: Equipment Failure
When Incident Discovered: During Operations
Lessons Learned: Proactive pre-operational training of host site personnel and first responders helped assure a measured response to this incident. First responders were familiar with location of local alarm panels, emergency stop actuation, and hydrogen storage. First responders will use site orientation presentation materials in post-incident training. Keep spare alarm equipment in stock to minimize repair time. Check hydrogen sensor readings monthly. Calibrate hydrogen sensors every 3 months.
Email (Primary):
Description
Characteristics:
Contributing Factors: Human Error
Damage and Injuries: Property Damage
Incident Date:
Severity: Incident
Leak: Yes
Ignition: No
Ignition Source:
Setting: Power Plant
When Incident Discovered: During Operations
Lessons Learned:
  1. Train personnel on delivery procedures and emphasize the safety aspects of hydrogen connections and disconnections, and verification of clearance for trailer movement prior to departure.
  2. Provide site-specific delivery procedures and possibly include a checklist as a reminder of key safety items prior to departure.
Email (Primary):
Description
Characteristics: High Pressure (> 100 bar)
Contributing Factors:
Damage and Injuries: None
Incident Date:
Severity: Incident
Leak: Yes
Ignition: No
Ignition Source:
Setting: Power Plant
Probable Cause: Fault in original weld
When Incident Discovered: During Operations
Lessons Learned:
  1. Defects in equipment, such as welds, may not be evident during initial proof testing and operation.
  2. When an equipment defect is found, check equipment for possible similar defects in other areas of the equipment.
Email (Primary):
Description
Characteristics: High Pressure (> 100 bar)
Contributing Factors: Situational Awareness
Damage and Injuries: None
Incident Date:
Severity: Non-Event
Leak: No
Ignition: No
Ignition Source:
Probable Cause: Deficiency in Procedures
When Incident Discovered: During Operations
Lessons Learned:

Manuals showing the correct connecting and disconnecting procedures were established for each fueling dispenser. Separate manuals were needed since the shape of the grip differs from dispenser to dispenser.

Email (Primary):
Description
Characteristics:
Damage and Injuries: Facility Closure, Property Damage
Incident Date:
Severity: Incident
Leak: Yes
Ignition: Yes
Ignition Source:
When Incident Discovered: During Maintenance
Lessons Learned:
  1. Provide additional retraining of operators. Curriculum should emphasize the proper response to high oxygen in the hydrogen gas.
  2. Evaluate current knowledge of operators. Operators should know proper response, such as contacting engineering or management support as needed, to evaluate any potentially dangerous process conditions they might observe.
  3. Revise interlock strategy. The plant control system was reviewed and enhanced with additional interlocks that automatically shut down the process and safely secure the hydrogen systems on certain pH levels and certain oxygen levels.
Email (Primary):
Description
Characteristics:
Contributing Factors: Incorrect Protocol/SOP
Damage and Injuries: None
Incident Date:
Severity: Near-Miss
Leak: No
Ignition: No
Ignition Source:
Setting: Laboratory
Equipment: Fuel Cell
Probable Cause: Deficiency in Procedures
When Incident Discovered: During Operations
Lessons Learned:

A step was added to the laboratory procedure in which all catalyst waste is thoroughly dampened with water and sealed in a plastic bag before being placed into a waste container.

Email (Primary):
Description
Characteristics: High Pressure (> 100 bar)
Damage and Injuries: None
Incident Date:
Severity: Incident
Leak: Yes
Ignition: Yes
Ignition Source:
When Incident Discovered: During Maintenance
Lessons Learned:

Safe work procedures will be prepared and followed. Hydrogen will be vented out of the system to create an inert atmosphere before working on system tubing and joints. The importance of purging hydrogen piping and equipment is discussed in the Lessons Learned Corner on this website.

Email (Primary):
Description
Characteristics:
Contributing Factors: Human Error
Damage and Injuries: None
Incident Date:
Severity: Incident
Leak: Yes
Ignition: No
Ignition Source:
Setting: Refinery
Equipment: Valve
When Incident Discovered: During Operations
Lessons Learned:
  1. Maintenance work scopes must be adhered to. If added work is approved, it must be tracked and reviewed to ensure that the equipment is inspected by operations personnel. Scheduled work and extra work should be listed and checked off for proper line-up.
  2. When closing valves, ensure that they are closed completely. Bull plugs must be installed tight enough so that they will not loosen as a result of contraction/expansion occurring during hot and cold cycling and/or compressor vibration. When closing a valve, a valve wrench must be used to synch up by using force to turn the valve handle a 1/4 turn. When installing a bull plug, a wrench must be used to ensure that the bull plug is tight.
  3. The recommissioning process used to reactivate the gas oil hydrotreater unit was modified to address bull plugs. Involve plant operations and health/safety personnel to develop an operational mechanical integrity checklist to ensure that all mechanical equipment components are addressed and then train personnel to use the checklist.
Email (Primary):
Description
Characteristics:
Contributing Factors:
Damage and Injuries: None
Incident Date:
Severity: Incident
Leak: Yes
Ignition: Yes
Ignition Source:
Equipment: Flexible Tubing
Probable Cause: Equipment Failure
When Incident Discovered: During Operations
Lessons Learned:
  • Unprotected hoses are susceptible to glass shards, abrasion, and burns during normal use. This leak was formed from a glass shard penetrating the tube wall.
  • Flexible hoses require protective sheaves to avoid cuts, abrasions and burns during normal use. Hydrogen supplied workstations require emergency shut off valves that are readily accessible by operators and are removed from the normal usage point of the gas. A request has been submitted to facilities engineering to select an appropriate valve. Once this has been done a follow-up request to facilities will be made to install such a valve. Such corrective action will be implemented plant-wide.
  • All existing hoses have been replaced. Weekly inspections during the units clean-up period has been requested of all glass workers. The hoses will be inspected for wear and burns. Flexible hoses on equipment have been replaced with stainless steel tubing where applicable.
Email (Primary):
Description
Characteristics:
Contributing Factors: Equipment Failure
Damage and Injuries: None
Incident Date:
Severity: Near-Miss
Leak: Yes
Ignition: No
Ignition Source:
Setting: Laboratory
Equipment: Valve
Probable Cause: Design Flaw
When Incident Discovered: During Maintenance
Lessons Learned:

This occurrence underscores the importance of appropriate design, equipment selection, and design review; the potential drawbacks of sharing utilities with other facilities; and the need to simplify [in this case, glove box atmosphere purification equipment/process] operations as much as possible.

Email (Primary):
Description
Characteristics:
Contributing Factors: Incorrect Protocol/SOP
Damage and Injuries: Environmental
Incident Date:
Severity: Incident
Leak: Yes
Ignition: Yes
Ignition Source:
Setting: Laboratory
Probable Cause: Design Flaw
When Incident Discovered: During Operations
Lessons Learned:
  1. There should have been greater awareness during the design and installation of the burn box regarding the static pressure limitations of the exhaust fan in relation to the anticipated static pressure buildup across the HEPA filter.
  2. Monitoring the pressure drop across the HEPA filter is not a sufficient or reliable indicator of ventilation system performance (air flow). An air flow monitor incorporating a low-level alarm would have provided a reliable indication of ventilation system performance.
Email (Primary):
Description
Characteristics: High Pressure (> 100 bar)
Damage and Injuries: None
Incident Date:
Severity: Incident
Leak: Yes
Ignition: No
Ignition Source:
Setting: Laboratory
Probable Cause: Equipment Failure
When Incident Discovered: During Operations
Lessons Learned:

This occurrence underscores the importance of completing a hazard analysis for each different process. Non-safety-related systems or equipment may be used for purposes other than their primary purpose so long as potential hazards are identified and engineering and administrative controls are applied to eliminate or minimize them. The filling of lecture bottles seemed to fall within the safety envelope previously established for high-pressure testing; however, the rupture of a cylinder revealed an unanticipated potential to release an overfilled gas cylinder to laboratory personnel.

Email (Primary):
Description
Characteristics:
Contributing Factors: Equipment Failure
Damage and Injuries: Minor Injury
Incident Date:
Severity: Incident
Leak: Yes
Ignition: Yes
Ignition Source: spark from quick-disconnect fitting striking floor of laboratory hood
Setting: Laboratory
Probable Cause: Design Flaw
When Incident Discovered: During Operations
Lessons Learned:

The risk of a serious fire occurring inside the chemistry laboratory hood from use of hydrogen gas and/or presence of a small quantity of hydrazine-hydrate was underestimated. Additional control measures were required to reduce the risk of fire and to ensure that consequences of an incipient fire were minimized.

Email (Primary):
Description
Characteristics:
Contributing Factors:
Damage and Injuries: None
Incident Date:
Severity: Incident
Leak: Uncertain
Ignition: No
Ignition Source:
Setting: Laboratory
Equipment: Vessel, Valve
Probable Cause: Equipment Failure
When Incident Discovered: During Maintenance
Lessons Learned:
  • Equipment that is designed to provide a safety barrier must be stringently tested upon installation.
  • A procedure requiring annual testing of both excess flow valves, which includes proper seating for closure and proper flow, has been developed.
Email (Primary):
Description
Characteristics:
Contributing Factors: Design Flaw
Damage and Injuries: None
Incident Date:
Severity: Incident
Leak: Yes
Ignition: No
Ignition Source:
Setting: Laboratory
Probable Cause: Equipment Failure
When Incident Discovered: During Operations
Lessons Learned:

All safety devices worked as designed thereby protecting the environment and laboratory personnel. Researchers involved in the experiment acted properly and with the parameters set forth in operational procedures.

Follow up: Stops have been added to the apparatus to prevent the valves from being opened too far and exposing the 0-ring seals to damage.

Email (Primary):
Description
Contributing Factors: Incorrect Protocol/SOP, Training Issue
Damage and Injuries: None
Incident Date:
Severity: Non-Event
Leak: No
Ignition: No
Ignition Source:
Equipment: Vessel
When Incident Discovered: During Inspection
Lessons Learned:

Whenever a new program or organization is created, management should ensure that program interfaces and new responsibilities are clearly defined. Effective program integration is necessary to ensure that all responsibilities and requirements are implemented. Facility safety is the focus of review on hazard evaluation documentation that is provided to the facility by the reindustrialization program.

Following receipt of hydrogen tank accident analysis and other data, develop an approved corrective action plan, identify applicable facilities "surrounding" the building, and review and update the authorization basis documents for those facilities, as necessary.

Email (Primary):
Description
Characteristics:
Damage and Injuries: None
Incident Date:
Severity: Incident
Leak: No
Ignition: Yes
Ignition Source: Electrical short
Setting: Laboratory
Equipment: Fuel Cell
When Incident Discovered: During Operations
Lessons Learned:

Procedures will be revised to include specific information about hydrogen sensors and alarm controls. Caution will be added about sparks and flames in the fuel cell operation area. Workers will be cautioned that shunt resistors may reach high temperatures, shunt resistors should be kept away from gas tubing and other electrical wiring but near the fan; and shunt resistors should be checked periodically to assure that a good connection is made, thus preventing overheating. The procedure will also be revised to specify the proper hierarchy of emergency notification.

Workers should think twice about blowing out a fire, even if it is perceived as small. Serious injury can result even if the fire is relatively small. Flammable gases such as hydrogen and oxygen could provide additional fuel. It is important to document changes to an experiment through a formal change control process. On-the-Job training should be provided in several modes of communication. An electronic mail notification should not be the only and primary method of communication.

Email (Primary):
Description
Characteristics:
Contributing Factors: Accountability, Communication, Human Error
Damage and Injuries: None
Incident Date:
Severity: Non-Event
Leak: No
Ignition: No
Ignition Source:
Equipment: Piping
When Incident Discovered: During Operations
Lessons Learned:

Management should ensure that a complete and thorough review of the previous work activities identified in the work package is performed at the pre-job briefing for the planned work. Work activities should be discontinued if any confusion exists concerning how to execute procedure steps. A process should be put in place to positively mark pipes that have been tested and retested to ensure proper identification when testing for acceptable levels of hydrogen.

Email (Primary):
Description
Characteristics:
Contributing Factors: Design Flaw
Damage and Injuries: None
Incident Date:
Severity: Near-Miss
Leak: Uncertain
Ignition: No
Ignition Source:
Equipment: Valve
Probable Cause: Equipment Failure
When Incident Discovered: During Operations
Lessons Learned:

This occurrence demonstrated the use of data by engineering to evaluate equipment problems. As the data changed, highlighting a problem with one-half inch PVC ball valves, the facility redesigned the valve extension and valve handle to prevent recurrence.

Valves for compressed gas service and for cryogenic liquid service are discussed in the Hydrogen Safety Best Practices Manual.

Email (Primary):
Description
Characteristics:
Contributing Factors: Decision Making, Human Error
Damage and Injuries: None
Incident Date:
Severity: Non-Event
Leak: No
Ignition: No
Ignition Source:
When Incident Discovered: During Operations
Lessons Learned:

This occurrence highlights the need for ensuring work is organized and planned in a manner that is practical and efficient. The primary lesson learned from this occurrence is that personnel must be cognizant of the impact of their actions on other shifts. Planning with thought on potential consequences is necessary to ensure work activities are performed safely and the appropriate actions are taken as required.

Email (Primary):