Amazon Mechanical Turk put people inside the machine.

Seventy-three thousand products needed putting in the right categories. Channel Intelligence estimated the job would consume 600 working hours. Then it handed the problem to Amazon Mechanical Turk. Four days later, the crowd had finished. In Amazon’s 2009 account, the company reported an 85 percent cost saving. The striking detail was the pleasure of watching: someone kept logging in, and the work kept disappearing.

That little scene explains the attraction. A large queue became thousands of small invitations. Somewhere, someone could accept one. The requester did not need to assemble everybody in an office, arrange shifts or hire for a permanent role. It needed to make the next unit of work understandable.

The computer becomes the customer

Amazon launched Mechanical Turk on November 2, 2005, with a wonderfully sideways question. Usually, a person asks a computer to do something. What if a computer could ask a person? The launch announcement called the result “Artificial Artificial Intelligence.” The joke contained the product specification.

A requester posted a Human Intelligence Task, or HIT. A worker chose an assignment, completed it and submitted an answer. Amazon supplied the marketplace and payment machinery. Software could send work through an API and collect the results. The human contribution might be a category, a transcription or a judgment that two records referred to the same thing.

For a business drowning in untidy information, this was appealing plumbing. Researchers could recruit survey respondents. Developers could obtain annotations for machine learning. Content teams could ask people to evaluate material against instructions. The common problem was a backlog of decisions that required human attention, repeated many times.

The customers did not all want the same thing. Pinterest historically described building a human evaluation system to help assess its enormous collection of ideas. CastingWords used the crowd for transcription and grading. These were different purchases of attention: one helped evaluate what a system should show, another turned speech into text and checked the output. The platform supplied a common way to distribute both.

The answer needed another answer

The interesting complication arrived when researchers asked whether the results were any good. In a 2008 study, Rion Snow and colleagues examined five language tasks, including word similarity and emotional interpretation. Individual non-experts were noisier than experts. A cheap answer could still be an unreliable answer.

Pooling independent judgments changed the calculation. The researchers found strong agreement with expert labels across the tasks they tested. In their emotion experiment, averaging several non-expert responses often reached expert-level agreement. The result had boundaries: fear was an exception in the reported comparison. These were structured, learnable tasks, not a license to replace any specialist with enough strangers.

That distinction is useful well beyond MTurk. Repeating an ambiguous instruction can multiply the same misunderstanding. A buyer needs examples, a way to recognize mistakes and a decision about what disagreement means. The unit being purchased is an assignment; the useful product is a result someone can trust.

“Artificial Artificial Intelligence”

Amazon’s name for the idea, November 2005

Someone still had to write the instructions

The marketplace offered tools for this work. Its requester interface included templates, batch publishing and worker requirements. Buyers could ask for multiple assignments on an item and compare responses. Qualifications helped restrict access by relevant criteria. In 2011, Masters added a signal based on demonstrated worker performance.

Amazon’s best-practices guide advised keeping tasks simple, testing them and designating an owner. It also treated a requester’s reputation with workers as something worth building. This is an unusually practical detail: workers had choices, and the person offering the task had to make completing it worthwhile.

The portable method is straightforward. Start with a small batch. Watch for questions and unexpected answers. Fix confusing wording before expanding. Decide which errors require another judgment and which need an expert. Measure the cost of an accepted, useful result, including review time. A thousand quick clicks tell you little if the labels need doing again.

The price of a click, and the price of an hour

MTurk’s published fee schedule put a 20 percent platform charge on worker rewards and bonuses. HITs with ten or more assignments attracted another 20 percent on rewards. Masters and demographic qualifications could add charges. The requester selected the reward; Amazon charged for arranging the transaction.

For workers, the arithmetic was less tidy. A study released in 2017 recorded 2,676 people performing 3.8 million tasks and estimated median earnings at about $2 an hour. Searching, rejected work and unfinished submissions affected the calculation. That historical sample cannot describe every worker, but it exposes a weakness in the unit price: people spend time between paid clicks.

Scale made the exchange feel available on demand. In a 2012 interview, AWS described 500,000 workers in 190 countries, including students, retirees and stay-at-home mothers. Those were historical workforce figures, not a promise that half a million people were waiting for any particular task. Availability still depended on who qualified, who understood the instructions and who thought the reward justified the time.

Amazon described workers as independent contractors. Flexibility was part of the offer. So was the burden of finding worthwhile work. The culture of the marketplace depended on both sides making decisions, with very different amounts of control over the terms.

The marketplace ends. The question travels.

By 2017, AWS SDK support had made MTurk easier to connect to familiar developer tools. Its place in the market remained distinctive: a general-purpose work exchange where requesters designed the job. A managed annotation provider assumes more of that operational work. A research recruitment platform organizes access around participants and studies.

Today the choice is historical. Amazon permanently closed MTurk on September 30, 2026. Its closure FAQ gives October 30 for remaining approvals, bonuses and payment-information checks. Businesses still needing human input must choose another supplier, such as Prolific for research participation or Appen for AI data services, according to the work involved.

The lesson travels easily. When a project can be divided into clear, checkable decisions, distributed work can remove a formidable queue. When the work depends on deep context, confidential access or specialist judgment, the task needs a different design and workforce. Before buying the next thousand answers, try writing one question that a stranger can answer well.