Teaching Skin Tone Algorithms – Harvard Gazette
The Fitzpatrick scale has been in the open field for decades and has been adopted by the technology industry primarily for use in machine learning and artificial intelligence. Humans need to annotate images or videos to train machine learning algorithms for things like face detection.
“Instead of using this Fitzpatrick scale, which was not designed to classify various skin tones among different groups of people, you would use Monk’s skin tone scale, which gives you a more granular understanding” , said Monk. He chose 10 color points to better represent the range of skin tones, foregoing a wider range because more would start to create annotation issues. Extensive fieldwork on skin tone and colorism in the US and Brazil informed the final color selection.
Monk’s collaboration with Google began a few years ago when he was contacted by its Research Responsible AI team, and over time the work shifted to finding an alternative at the scale of Fitzpatrick, said the sociologist. It was then that the team learned that Monk had developed his own scale.
Google already uses the Monk Skin Tone Scale in its image search. When users search for makeup looks, for example, they can refine those searches by skin tone to find more relevant matches. The tech giant also used the scale to update its own Real Tone filters (an initiative it had previously started to address the skin tone issue) on its Pixel cameras so they could capture a wider and more realistic palette of shades.
“There have also been some improvements in search where people have complained that in the past if you search for certain images you only get certain types of people,” Monk said. “For example, if you search for ‘cute babies’, you might get a completely seamless set of babies that all look white. It’s not at all a very inclusive experience for people.
Google says the scale will be useful for checking the diversity of results for algorithm issues. “It’s been a long-standing problem, not just at Google, but across the tech industry as a whole,” Monk said.
Monk plans to publish research, testing the scale on a diverse group of people. He and the Google team conducted research together to validate the scale in the United States, which was an extension of his own work.
Developers who don’t intentionally design their products to work well across the skin tone continuum will have products that don’t perform as well, Monk added. The new scale is open source so that all companies around the world can make their products more inclusive.
“I hope other companies, not just Google, will take it upon themselves to audit and ensure that their products are designed to work equally well on the full range of skin tones, whether or not they use the Monk Skin Tone Scale”. Monk said. “I hope they do this job.”