The ability to identify correlations in large amounts of data in a very short time makes artificial intelligence a valuable tool for decision-making. This also applies to the two AI systems that the team led by professors Andreas Metzger and Klaus Pohl has developed for business process management (BPM). One system allows very precise predictions about the course of business processes. In logistics, for example, it can help to detect impending delays in means of transport at an early stage. A second AI system gives recommendations on whether and when an ongoing business process should be adapted. If, for example, disruptions in rail traffic are predicted, the system can show when it is more cost-effective to reschedule transport from rail wagons to trucks.

But the high performance of AI for such types of decision support comes at a price: the knowledge that AI systems build up through machine learning is deeply embedded in thousands and thousands of links of artificial neurons in a neural network. Therefore, users cannot easily understand what the systems have learned and whether the learned connections make sense at all. "Companies should therefore not blindly trust the predictions and recommendations of AI systems," says Prof. Metzger. "We are researching how AI systems can be designed in such a way that they provide comprehensible explanations of their results without restrictions in their ability to conduct themselves."

Learning from Interpersonal Communication

To this end, the paluno team has developed techniques that provide insight into the internal behavior of AI systems. The development of these techniques follows cognitive science findings on human processes of explanation and understanding. It considers the fact that people are interested in counterfactual correlations. Accordingly, the computer-generated explanations indicate which other inputs to the AI would have led to a different result. In addition, the explanations are selective and focus on relevant events in the learning process of the AI system.

Since explanations often develop in dialogue, the team is currently working on interactive interfaces between humans and the AI system. Specifically, a chatbot like ChatGPT is to be developed, which will answer users' questions to make the internal workings of the AI systems understandable.

Research projects

The research work was mainly carried out within the framework of the recently completed EU project DataPorts. The AI systems developed were based on a data platform that suitably integrates and processes data from seaports. The current research work is being carried out in the current EU project Dynabic. Here, AI-based solutions are being developed to provide operators of critical infrastructures with decision support for proactive adaptation due to cyber attacks.

Publications

A. Metzger, T. Kley, and A. Palm, “Triggering proactive business process adaptations via online reinforcement learning,” in 18th Int’l Conference on Business Process Management (BPM 2020), Sevilla, Spain (virtual), Springer, 2020.

T. Huang, A. Metzger, and K. Pohl, “Counterfactual Explanations for Predictive Business Process Monitoring,” in 18th European Mediterranean & Middle Eastern Conference on Information Systems (EMCIS 2021), December 8-9, 2021, Online, Springer LNBIP 437, pp. 399-413

F. Feit, A. Metzger, K. Pohl, „Explaining Online Reinforcement Learning Decisions of Self-Adaptive Systems“, in 3rd Int’l Conference on Autonomic Computing and Self-Organizing Systems (ACSOS 2022), IEEE, 2022, pp. 51-60