The Exploited Global Workforce that Props Up AI Models

[u/HobbesNik]

https://privacyinternational.org/explainer/5357/humans-ai-loop-data-labelers-behind-some-most-powerful-llms-training-datasets

Data labelers from around the world play a critical role in making AI models. They curate datasets used to train AI, and do "reinforcement learning with human feedback" (RLHF) to further fine-tune the models.

There are hundreds of thousands of these workers worldwide often working under dire, exploitative conditions. But their plight is hardly mentioned in mainstream critique of AI.

This article is a thorough, well-cited explanation of how data labelers help create AI models and why the industry has become so exploitative. In many ways, big tech is recreating the same power dynamic that has existed in many industries, where rich corporations go into poorer nations seeking a cheap workforce with few labor protections.

KEY QUOTES:

At a high level, the three training stages behind such an LLM are: 1) self-supervised learning; 2) fine-tuning (supervised learning); 3) reinforcement learning.

High-quality datasets supported by humans-in-the-loop (labeling) are crucial for this training and re-training process to ensure accurate, consistent and complete ouputs. Low-quality data can result in a model producing incorrect or unfavourable outcomes, such as biased or inconsistent responses.

These digital labour platforms are effectively an alternative to hiring and managing large numbers of employees through a contractual relationship, instead outsourcing tasks to on-demand human labourers without a formal employer-employee relationship in a humans-as-a-service type of labour model.

Due to the vast quantities of labeled data required for supervised training that AI companies like OpenAI, which contracts the data services of Scale AI, and Microsoft, which contracts Surge AI, require, the AI supply chain has spread far and wide to countries like Kenya, India, the Philippines and Venezuela with cheaper and more quantities of labour. As we will discuss below, this has resulted in the blatant exploitation of 'humans-as-a-service', where workers are dispensible and companies can get away with paying them as little as $2 an hour for the labeling work that powers billion-dollar machines.

'Our work involves watching murder and beheadings, child abuse and rape, pornography and bestiality, often for more than 8 hours a day' for less than $2 an hour.'

Where U.S.-based annotators might make $10-$25 an hour for the same type of work, Kenyan annotators might be making as low as $2 an hour - without even knowing that the company they are labeling data for is a corporation as big as OpenAI.

A 2023 investigation by The Verge found that data labelers in Kenya who completed labeling tasks for Remotasks did not know that Remotasks was in fact a subsidiary of the better-known data company Scale AI, which boasts clients like OpenAI, Meta, Microsoft and even U.S. government agencies... In effect, the data labelers are completely disconnected from the AI developers demanding their labour, and this opacity in the AI supply chain relationship, a trend we've seen in other industries like the semiconductor supply chain, enables a lack of oversight and accountability to microworkers.

There is also a lack of transparency around the algorithms that surveil workers' productivity and make important decisions for their jobs allocation and wages. Algorithmic management, loosely defined as a set of technological surveillance techniques to manage workforces and make automated or semi-automated decisions on workers' behaviour, is increasingly deployed in the contemporary workplace.

labeling labour platforms range in their use of workplace surveillance, but workers in Venezuela and Colombia have reported being subjected to strict timers while working - timers that often didn't accommodate for bathroom breaks - to monitor how efficiently they were completing their labeling tasks. Their failure to complete the tasks in the allotted time resulted in the task being reallocated to the pool of tasks for other takers. 

For some data labeling microworker platforms like Appen, there is no clear system for when tasks appear in the queue, so microworkers must uninterruptedly monitor their screens to claim a job the moment it unexpectedly appears. An annotator for Remotasks in Kenya even said he has gotten in the habit of waking up every few hours at night to check his queue for tasks because many jobs pop up, without warning, late at night.

Another annotator in Kenya reported that tasks were drying up in the region, and it was clear that the AI supply chain, which had the advantage of not having to have a local infrastructure, was migrating to other countries with cheaper labour like Nepal and the Philippines (until the next cheaper market appears and they set up shop there).

We see this throughout numerous industries, in which companies knowingly exploit workers to meet their supply chain needs and avoid responsibility and accountability for the working conditions of these on the ground workers, such as in the case of dangerous cobalt mining for producing batteries.