Microsoft, MITRE Release Adversarial Machine Learning Threat Matrix

23 October 2020

Microsoft, MITRE Release Adversarial Machine Learning Threat Matrix

Microsoft and MITRE, in collaboration with a dozen other organizations, have developed a framework designed to help identify, respond to, and remediate attacks targeting machine learning (ML) systems.

Such attacks, Microsoft says, have increased significantly over the past four years, and are expected to continue evolving. Despite that, however, organizations have yet to come to terms with adversarial machine learning, Microsoft says.

In fact, a recent survey conducted by the tech giant among 28 organizations has revealed that most of them (25) don’t have the necessary tools to secure machine learning systems and are explicitly looking for guidance.

“We found that preparation is not just limited to smaller organizations. We spoke to Fortune 500 companies, governments, non-profits, and small and mid-sized organizations,” Microsoft says.

The Adversarial ML Threat Matrix, which Microsoft has released in collaboration with MITRE, IBM, NVIDIA, Airbus, Bosch, Deep Instinct, Two Six Labs, Cardiff University, the University of Toronto, PricewaterhouseCoopers, the Software Engineering Institute at Carnegie Mellon University, and the Berryville Institute of Machine Learning, is an industry-focused open framework that aims to address this issue.

The framework provides information on the techniques employed by adversaries when targeting ML systems and is primarily aimed at security analysts. Structured like the ATT&CK framework, the Adversarial ML Threat Matrix is based on observed attacks that have been vetted as effective against production ML systems.

Attacks targeting these systems are possible because of inherent limitations underlying ML algorithms and require a new approach to security and a shift in how cyber adversary behavior is modelled, to ensure the accurate reflection of emerging threat vectors, as well as the fast evolving adversarial machine learning attack lifecycle.

“MITRE has deep experience with technically complex multi-stakeholder problems. […] To succeed, we know we need to bring the experience of a community of analysts sharing real threat data and improving defenses. And for that to work, all the organizations and analysts involved need to be assured they have a trustworthy, neutral party who can aggregate these real-world incidents and maintain a level of privacy—and they have that in MITRE,” Charles Clancy, senior vice president and general manager of MITRE Labs, said.

The newly released framework is a first attempt at creating a knowledge base on the manner in which ML systems can be attacked and the partnering companies will modify it with input received from the security and machine learning community. Thus, the industry is encouraged to help fill the gaps, and to participate in discussions in this Google Group.

“This effort is aimed at security analysts and the broader security community: the matrix and the case studies are meant to help in strategizing protection and detection; the framework seeds attacks on ML systems, so that they can carefully carry out similar exercises in their organizations and validate the monitoring strategies,” Microsoft explains.