This blog post by Parul Pandey has a nice description of the resources available around the Google "What-if Tool". 

What-If Tool is an interactive visual tool that is designed to investigate the Machine Learning models. Abbreviated as WIT, it enables the understanding of a Classification or Regression model by enabling people to examine, evaluate, and compare machine learning models. Due to its user-friendly interface and less dependency on complex coding, everyone from a developer, a product manager, a researcher or a student can use it for their purpose.

Be sure to see the essay titled, "Playing with AI Fairness" by David Weinberger which talks about using the What-if tool environment to "try on five different types of fairness". I found it to be an accessible introduction to the different approaches to making real world decisions with ML tools.