The Recognizing, Exploring, and Articulating Limitations in Machine Learning research tool (REAL ML) is a set of guided activities to help ML researchers recognize, explore, and articulate the limitations that arise in their research. The tool includes an instructional guide (pdf) and a worksheet (editable Word document) that you will find in this repository. It is recommended that you fill out the REAL ML worksheet while completing the activities outlined in REAL ML.

For more information on how this tool was created, you can read the full FAccT paper, REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Jessie J. Smith, Saleema Amershi, Solon Barocas, Hanna Wallach, and Jennifer Wortman Vaughan. REAL ML: Recognizing, Exploring, and Articulating Limitations of Machine Learning Research. In Proceedings of the 5th ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2022.

If you have questions about this tool, the paper, or the studies conducted to create this tool please feel free to reach out to the research team by email at realml [at] microsoft [dot] com