Artificial Intelligence (AI) and data-driven decisions based on Machine Learning (ML) are making an impact on an increasing number of industries. As these autonomous and self-learning systems become more and more responsible for making decisions that may ultimately affect the safety of personnel, assets, or the environment, the need to ensure the safe use of AI in systems will be crucial to safety management and operations.
Machine learning for high-risk and safety-critical applications is challenging, as there is a reduced tolerance for erroneous predictions due to potentially catastrophic consequences. The models which fit the data well are also often opaque, making them less falsifiable and difficult to trust. Moreover, there is usually a dearth of relevant data, and a proper treatment of uncertainty is essential, as DNV is not only concerned with what is likely to happen, but also with less likely events that may happen. However, there are some positives – namely, that there is often additional causal and physics-based knowledge available.