https://mailchi.mp/edda78bd2a5a/the-weekly-gist-june-23-2023?e=d1e747d2d8

An in-depth piece co-published this week in New York Magazine chronicles the Kafkaesque working environment of artificial intelligence (AI) annotators, whose “unprecedented feat[s] of tedious and repetitive behavior” have enabled the AI boom. AI learns by training on massive, meticulously labeled datasets, but that training is not some high-tech, futuristic process.

Instead, AI companies of all kinds have secretively hired a vast, global web of millions of low-wage laborers—mostly in the global south—to annotate these large datasets according to precise, convoluted rules sets (one worker reported marking all the knees and elbows in pictures of crowds for 36 hours straight!). AI optimists predict this phase of labor-intensive annotation will pass once the bots advance to the point of automating the annotating process, but most use cases remain far from this goal. Turns out the AI systems are pretty poor students as well: for example, the models need to be explicitly taught the difference between a shirt, and the reflection of a shirt in a mirror, and a shirt on a hangar, and a shirt on a chair, and on and on. 

The Gist: If AI needs to be trained by thousands of low-skilled, low-wage workers to identify an elbow, how many doctors will it take to train the algorithms to accurately diagnose a CT scan as cancerous? Thanks to electronic health records, some of this annotation work has been built into the images, but the sensitivity and high price of patient data make it harder to assemble datasets large enough to power the training. 

While some remain hopeful that AI has the potential to eventually cure healthcare of Baumol’s cost disease, the path to that point will be paved by significant, tedious manual labor hours performed behind the scenes—adding substantial additional cost and slowing the advent of the long-predicted future of ubiquitous, super-intelligent bots.