4.6 Example problems

is unfortunately no. The chain of reasoning only tells us “how” a decision was made for a given input but not “why”. For example, knowing “how” is not sufficient to justify that the decision is made consistently, accurately, reliably, and validly. Thus, for a learning model to be truly transparent we need to know both “how” and “why”. Due to the high complexity of deep ML models, which often incorporate hundreds of fully (or partially) interconnected layers, a promising approach for increasing their transparency and interpretability is to provide justifications and insights for the decisions that can be gauged externally.

4.6 Example problems

Tangible example problems might include:

Exploring security and privacy attacks on ML models. An important direction for enhancing the security and privacy of ML algorithms is to reveal additional hidden vulnerabilities. Apart from the already established security/privacy attacks, such as model inversion, membership inference, model extraction and adversarial sample generation, additional effort is required to determine other possible threats. In addition, we need concepts and techniques to measure the vulnerability/robustness of ML.

Proposing generally applicable defence strategies. Another interesting direction is the development of generally applicable defences, more specifically, defences that can be applied to existing trained ML models without the need for retraining, which is a time-consuming process and would require large computational resources, or any modifications to their architecture/training algorithm, which would require significant manual intervention from experts in the field of AI and ML.

Applying vulnerable learning models in a secure way. One might say that perfectly secure ML is probably an illusion. Thus, instead of focusing on increasing their robustness, an alternative direction is focusing on how to apply them in such a secure way that exploiting those ML models becomes significantly harder.

Developing human-friendly interpretability techniques. This angle involves the development of systems and services that are able to provide humanfriendly explanations for the decisions of current state-of-the-art ML models. When we refer to “human-friendly explanations”, we mean justifications that are preferably simple enough for people who are not experts in the fields of AI and ML to understand.