Mechanical Turk
Description
J. NATHAN MATIAS:
"Mechanical Turk, which has no minimum wage, is a free market for digital labor. Only the collective decisions of workers and requesters determine wages and working conditions. Since Amazon provides no way for workers to rate employers, workers can’t always anticipate if they will be treated well or paid fairly. As a result, making a living on Mechanical Turk is a precarious venture, with few company policies and a mostly hands-off attitude from Amazon." (http://www.theatlantic.com/technology/archive/2015/06/the-tragedy-of-the-digital-commons/395129/)
How It Works
Moshe Z. Marvit:
"The genius of Mechanical Turk is in creating virtual assembly lines.
Here’s how it works: the employers (called “Requesters”) can be actual humans or a computer program running a script that automatically outsources any task it cannot perform to the crowd. The Requesters place microtasks (called “Human Intelligence Tasks,” or HITs) on Amazon’s Mechanical Turk website and offer non-negotiable contracts with a take-it-or-leave-it rate for each HIT. The Turkers (officially called “Providers”) perform only small microtasks over and over, rarely getting a glimpse of the whole. Using keyword searches for HITs, New York University professor Panagiotis Ipeirotis found that among the most numerous HITs were data collection, transcription, searches, tagging, content review, categorization and other similar tasks.
However, HITs can be any task that can be outsourced online. Many of the features of the Internet that we take for granted come from paid crowdworking. These include more efficient search results, transcriptions of audio and video, reliable local business information, marketing spam and other aspects of the Internet that just seem to be there. Crowdworkers also engage in tasks that can make certain individuals’ lives immeasurably better, such as providing near-instantaneous information on their surroundings for blind people with a smartphone and the right app.
A HIT typically includes a set of instructions, some quite involved, that one must read and understand before performing the task. The Requesters can set eligibility requirements. For example, they may restrict access to those with “Master Qualifications”—a designation Amazon awards at its discretion—that some have received after performing at least 50,000 HITs at an exceptionally high approval rate. Or the Requester may exclude certain workers, such as those from India, because in the world of online crowdworking, they are thought of as inferior workers.
Requesters can engage in such broad exclusions because the Civil Rights Act does not touch upon these workers. In fact, most of the hard-won worker protections of the twentieth century do not apply to Turkers. Some critics worry that even prohibitions on child labor are being flouted. This is because it is not clear how well some sites enforce requirements that users must be over 18 to perform HITs. Moreover, many crowdworkers are paid in gaming credits, which may be used to intentionally lure children into performing cheap labor. The payment offered for HITs ranges from nothing to a few dollars to payment in virtual currencies, with most HITs falling on the low end of the scale. As a result, it is estimated that the average wage of Turkers is approximately $2 an hour. For access to this unregulated labor pool, Amazon charges a 10 percent commission from the Requesters.
Crowdwork often takes place in the home, performed by people who are not otherwise employed. A few have argued that the low pay may be the result of longstanding gender inequities in society. The great majority of American Turkers are women, and there is some indication that many of them are careworkers, or caring for an elderly relative or young child.
...
Turkers are categorized as independent contractors, meaning that they are not legally entitled to minimum wage, overtime pay, workers’ compensation, unemployment insurance or the various other statutory protections that cover employees. The multi-page participation agreement that all Turkers must consent to before entering the site specifies that the Turker is neither an employee of the Requester nor of Mechanical Turk. Amazon defines its role as being limited to “the capacity of a payment processor in facilitating the transactions between Requesters and Providers” and claims that it is “not responsible for the actions of any Requester or Provider.” Its agreement warns: “As a Requester or Provider, you use the Site at your own risk.”
After being reminded by Amazon of its lack of legal liability, one aggrieved Turker was baffled at how the company could profit from conduct that it hosts while saying it has no liability: “That’s like saying someone is running a slave market on my property, and they’re paying me, but I have no responsibility." (http://www.thenation.com/article/178241/how-crowdworkers-became-ghosts-digital-machine?page=0,1)
Example
Moshe Z. Marvit:
1.
"Rachael Jones of Minnesota is an example of a careworker who Turks. She was trained to assist neurosurgeons, but after a neck injury and surgery, she could no longer work in that physically demanding field. In addition, Jones wanted to stay home and take care of her children while her partner worked outside the home. She is now a “very full-time” Turker and, after several years during which she earned three Masters Qualifications and performed 110,000 HITs, she has been able to make approximately $8 per hour. When asked for her thoughts on Turking, she responded, “I really love it.” She admits that “there are some times when it’s really hard, and you’re scrounging and looking and the HITs that pay just aren’t there.” She’d like the pay to be increased, but she also fears that any significant increase could destroy the world of crowdworking. If that were to happen, she is unsure what type of work she could do that would allow her to stay home with her children." (http://www.thenation.com/article/178241/how-crowdworkers-became-ghosts-digital-machine?page=0,1)
2.
"According to the limited Turker demographics available, Stephanie Costello’s story is all too common. Costello lives in a trailer at the edge of a desert town in the Southwest. She is 50 and has an associate’s degree in nursing, but she has been unable to find suitable work as a nurse. In 2007, Costello was working at a boring office job and, in slow periods, earning extra money by doing online surveys on Mechanical Turk. When she lost her job at the start of the 2008 recession, she took to Turking full time, often more. What started as a source of extra cash suddenly turned into her main source of income. According to the 2010 study, Costello’s situation may be representative of approximately one in eight Turkers in America, or one in five worldwide.
Costello describes full-time Turking as “feast or famine,” but years of Turking have diminished her view of the feast. In February 2013, she worked approximately sixty hours a week searching for and performing HITs and made approximately $150 per week—and that was the feast. The next month, she was unable to find as many “good-paying” HITs and earned only about $50 per week. She describes how she often stays up all night with the Mechanical Turk screen open, because when people post a good batch of HITs, they go quickly.
“Good-paying” has become a relative term. Costello refuses to work for 60 cents or even $1.20 an hour because those low amounts are “more undignified than begging.” However, at $2 per hour she starts to equivocate, and she admits that she often works for that wage. Even those who describe making decent money usually talk about earning $6 per hour, which is still below the federal minimum wage.
Costello’s story of being a full-time Turker who is barely holding on is all too familiar in the world of crowdworkers. However, Costello believes that Turking may one day change for the better, especially with worker organization." (http://www.thenation.com/article/178241/how-crowdworkers-became-ghosts-digital-machine?page=0,4)
Status
Demographics
Moshe Z. Marvit:
"Aside from the handful of companies that run crowdworking platforms, no one really knows who makes up the crowd. The most recent study of Turker demographics, in 2010, found that the vast majority of Turkers— 57 percent—were American, with Indians coming in second with 32 percent of the workforce. American Turkers tended to be highly educated, with 63 percent having college degrees, compared with the national average of 25 percent. They were young, with a median age of 30, and 69 percent were female. The crowd labor pool has grown considerably since Amazon created the platform, and the extreme low-wage market seems to rely on competition among the overabundance of workers." (http://www.thenation.com/article/178241/how-crowdworkers-became-ghosts-digital-machine?page=0,1)
History
Moshe Z. Marvit:
"Amazon created Mechanical Turk in 2005, a decade into its rise from online bookstore to massive virtual mall. Even in its early stages, Mechanical Turk was a “Jeff project,” meaning that Amazon CEO Jeff Bezos took a special interest in it and worked closely with the project managers. By that time, Bezos and his team had already perfected the model that has made Amazon a tech-age giant: catering to customers, undercutting competitors and treating workers poorly. With Mechanical Turk, they further refined this approach, but with a twist: instead of exploiting labor to sell goods to an avid public, they created a system to sell labor itself, cheaply, to perform many millions of microtasks.
The name Mechanical Turk was not simply a whimsical choice by Bezos. The original Mechanical Turk (also known simply as “the Turk”) was a seemingly groundbreaking invention: a chess-playing machine commissioned by the Empress Maria Theresa of Austria in the early days of the Industrial Revolution. In appearance, it was similar to the other automatons of the era—a cabinet with a mannequin inside, performing some function usually reserved for humans—but this one was dressed in “Oriental” garb, not unlike the fortune-telling machine in the 1980s movie Big. Its mystical, mysterious air was meant to appeal to Western Europe’s conception of the East; hence the name by which it came to be known.
The Mechanical Turk seemed to promise an age when automated devices could attain the heights of human intellectual achievement—and it did so exceptionally well, convincing observers and challengers for nearly a century that a machine could play chess. In reality, however, the Turk was nothing more than an elaborate ruse involving a man or woman sitting inside the cabinet with an internal chessboard. The machine could approximate the movements of a man playing chess, but it was necessary to have a person inside to do what the machine could not.
If it was merely a hoax, then the Mechanical Turk would have been forgotten as yet another eighteenth-century oddity. However, the device fit perfectly into the creeping belief—replete with excitement and anxiety—that mechanical labor (and maybe mechanical minds) could replace human labor and agency." (http://www.thenation.com/article/178241/how-crowdworkers-became-ghosts-digital-machine?page=0,1)
Discussion
Organizing Turkers
MIRANDA KATZ:
"The Turker workforce has proven particularly difficult to organize: MTurk magnifies the challenges of the gig economy, with its isolated workers spread across the globe and hidden behind usernames, performing minute tasks on a platform operated by a massive, wealthy corporation. MTurk is also one of the least consumer-facing corners of the gig economy—so while ethically minded customers have taken Uber, Handy, and the like to task for their treatment of workers, Amazon’s gig-work platform has largely managed to evade public scrutiny for its low pay and reported lack of transparency.
Turkers—Amazon says that there are “over 500,000” of them, but has not updated that number in years—have figured out how to make the best out of a less-than-ideal situation. They’ve created Chrome plugins to sort the good tasks from the bad; they’ve written programs to awaken themselves with a loud chime when a particularly high-paying task goes live in the middle of the night; and they’ve built forums where they can offer one another advice. Those forums have taken on a life of their own, leading to deep networks of support and friendship—and at least one Turker marriage." (https://www.wired.com/story/amazons-turker-crowd-has-had-enough/?mbid=social_tw_backchannel)
Labor issues, forums, and actions
Moshe Z. Marvit :
" Irani argues that one of the fundamental problems with Mechanical Turk and other crowdworking sites is that they are not neutral market-places, as the sites claim, but are built on an inherently exploitative model that privileges employers.
One example is the way the Turker has no leverage in the employment relationship—an imbalance that is particularly pronounced in the set-up that allows a Requester either to “accept” or “reject” a Turker's work. These terms belong in quotation marks because, as mentioned earlier, the Requester sees and retains the work product no matter which choice it makes. If the Requester “accepts” the work, then it gets to keep the work product and pays the Turker. If the Requester “rejects” the work, then it gets to keep the work product without paying the Turker. The Requester does not have to provide a reason for the rejection, and there is nothing a Turker can do to challenge the decision other than to inquire and hope for a response.
By “rejecting” work, the Requester not only deprives the Turker of payment, but also affects the Turker’s online reputation. Turkers are ostensibly anonymous, identified only by a long string of numbers and letters, so all they have is their Mechanical Turk reputation. Costello describes how, in the beginning, she did thousands of HITs of any type simply to build her approval rate, only to realize how easily it could be knocked back down. “If you have a 99.8 percent approval rating and then you work for some jack-wagon who rejects 500 of your HITs, you’re toast,” she says. “Because for every rejection, you have to get 100 HITs that are approved to get your rating back up. Do you know how long that takes? It can take months; it can take years.”
Out of this grind, online forums and worker sites such as Turkopticon, TurkerNation, mturkforum, CloudMeBaby and Reddit have become the primary locations for the crowdworkforce to talk about their working conditions. However, the anonymity of Turking combined with the sharp edge of the Internet have often made these forums as counterproductive to organizing as they are productive in navigating the world of crowdwork.
TurkerNation is one of the oldest forums, and many Turkers have complained about being arbitrarily banned from the site at the whim of the moderator, Spamgirl. (Full disclosure: in researching this article, I was banned from the site shortly after joining.) Spamgirl has described herself as “the Hoffa of the Turkers! Trying to help the people.” She allowed a thread about Turkopticon, then in its early stages, to be hosted on the site before taking it down for unknown reasons. Rachael Jones describes being banned for getting into a Twitter tangle with a TurkerNation moderator.
Other sites like mturkforum and Turkopticon are more friendly, but some have complained that there is little support from forum communities in trying to better conditions or organize. It’s not clear that any online forum can serve as the basis for workers to find common cause and demand better working conditions, especially when the workforce is anonymous and in flux—or, in the parlance of crowdwork boosters, “flexible.”
Some well-intentioned reformers envision magnanimous employers as the sole source of changes that will make crowdworkers’ lives better. This view was on full display at a conference last year when eight prominent researchers delivered an important paper titled “The Future of Crowd Work,” framed around the question, “Can we foresee a future crowd workplace in which we would want our children to participate?” Of the many ideas that came out of this paper, none included any type of worker self-help or organization.
That view is currently being challenged in what may turn out to be a seminal class-action lawsuit filed in federal court in California. Christopher Otey and Mary Greth, on behalf of themselves and “the World’s Largest Workforce,” are arguing that crowdworkers are not simply contractors, but employees of the crowdworking company. Otey and Greth performed “simple repetitive online tasks for the benefit of CrowdFlower.” For their efforts, they were paid less than the $2 to $3 per hour that the CEO of CrowdFlower has stated his workers make. Otey and Greth allege that workers earn between $1 and $2 per hour, and that some are compensated in online game credits, virtual money and enhanced video game capabilities.
The class may have as many as 4 million members, making it potentially one of the largest employment class actions in history. If the plaintiffs prevail, they will be categorized as employees of the crowdworking company rather than as contractors, and will therefore be entitled to the host of benefits and protections that employees receive. These include a minimum wage and overtime, protections from discrimination, a legally protected right to organize a union, workers compensation for injuries, unemployment compensation for layoffs, certain whistleblower protections and others.
The case has advanced past several of the many preliminary hurdles facing such an ambitious lawsuit; however, as this article goes to press, a settlement motion has been submitted for the judge’s approval. If the judge finds that the terms of the settlement are fair to all members of the class, then it is likely that another case will have to address whether the millions of crowdworkers are entitled to the protections and pay of employees.
Miriam Cherry, who was organizing an amicus brief by a group of labor law professors before the settlement motion, fears that a judge may not understand crowdwork and simply declare that the workers are contractors. However, she also fears that a judge or legislature will simply regulate crowdwork out of existence. “There’s a lot of ways that things can go right or things can go wrong, and it depends on cases like CrowdFlower to really tell us if they’re going to go right or if they’re going to go wrong,” Cherry says." (http://www.thenation.com/article/178241/how-crowdworkers-became-ghosts-digital-machine?page=0,4)
Digital Labor as a Tragedy of the Commons
J. NATHAN MATIAS:
"When the ecologist Garrett Hardin set to write his famous 1968 article on problems with “no technical solution,” “The Tragedy of The Commons,” he could have been describing the Charles. Hardin imagines open grazing areas managed by multiple herders who destroy their precious common when each rationally seeks to maximize personal gain. The problems of digital labor can also be interpreted through this tragedy. With a Mechanical Turk worker turnover rate of 69 percent every six months, requesters tend to seek the minimum price for someone’s labor, and workers compete for diminishing pay. With minimal accountability for the companies requesting work and limited intervention from Amazon, attractive stories of flexible, livable income from digital labor remain as partially true as Longfellow’s poetic image of the beautiful river Charles.
Academics advancing the idea of digital commons have tended to focus on how to prevent or regulate these problems—after they're identified. In Code and Other Laws of Cyberspace, Larry Lessig describes software design as a kind of regulation separate from top-down policies or community norms. Sixteen years after Lessig’s book, belief in the power of code and social psychology to shape successful online communities is widespread among the design teams who govern our digital lives. Their growing toolbox of design options is detailed in a recent law review article by James Grimmelman, who covers everything from banning and shaming to reputation and rewards. In this view, perhaps Mechanical Turk could become fairer if Amazon added the right buttons, set the right default wage, or changed its design to activate just the right motivations." (http://www.theatlantic.com/technology/archive/2015/06/the-tragedy-of-the-digital-commons/395129/)
Daemo, a worker-friendly alternative ?
MIRANDA KATZ:
"A new, worker-friendly platform, dubbed Daemo, is currently under development at Stanford’s Crowd Research Collective, and describes itself as a “self-governed crowdsourcing marketplace”—a utopia, of sorts, for Turkers who have long clamored for more agency. It’s a tall order for anyone—let alone a wonky team of academics—to seriously challenge one of the biggest companies in the world. But Amazon Mechanical Turk, by all accounts, has barely changed in its 12 years of existence. And that might just make it the perfect target for disruption.
...
Daemo takes aim at two of MTurk's most glaring flaws: pay and communication. On Daemo, tasks are expected to pay $10 an hour—higher than the federal minimum wage, though lower than that in states such as Massachusetts and Washington. (On MTurk, for comparison, Pew Research found that 91 percent of workers make less than $8 an hour.) “It’s tricky to institute these minimum wages without a kind of formal way of measuring work performance,” says Whiting. If workers earn a flat rate per hour, he explains, those who tear through tasks particularly quickly may feel it’s not worth their time, whereas requesters dealing with slower workers may feel they’re not getting their money’s worth. Daemo has settled on a policy of norm-setting, reminding requesters of the $10/hour expectation when they’re setting up their tasks and empowering the worker community to take action if that norm is ignored.
Every new task uploaded to Daemo triggers a new discussion topic on its forum, where workers can suggest improvements to the task’s setup, ask questions if they’re confused, and flag whether they think the pay is unfair. If enough people flag a given task, it’ll be removed. On MTurk, those lowball tasks linger—and are invariably snatched up by someone willing to work for pennies.
Daemo takes the issue of fair pay so seriously that its administrators would only allow me to speak with workers on the platform if I set up my interviews as “tasks” and compensated workers for their time. In the interest of learning whether workers like the platform as much as its creators hope—and with my editors’ signoff—I agreed to pay $10 to any worker who would speak with me for up to an hour (totaling $21.34). I found those who raised their hands to be measured in their praise of Daemo. They were delighted by its high pay and open communication; one worker happily noted that when he made a suggestion for how Daemo could improve its interface, the update came through within a few days. Yet they weren’t sure that Daemo could ramp up a steady enough supply of work to lure a critical mass of Turkers away from Amazon.
“It has to scale up ginormously—but I sure hope it does,” says Gina Bixby, who Bernstein recruited to Daemo from MTurk in May. “On Turk I am completely a number, and if I don’t do the work they know that a different number will. It doesn’t matter to them who does it. I actually feel like it does make a difference to Daemo.”
Daemo’s creators don’t hesitate to admit that it is a work in progress. More than 300 workers have completed some several thousand tasks on the platform—and as Daemo gets closer to launching publicly, the frequency with which new tasks are posted has ramped up—but it’s nowhere near the size of Mechanical Turk, which typically has half a million tasks available per day. Daemo also may never be able to attract what Whiting calls “lone wolf workers,” who aren’t interested in a community-oriented approach and who choose crowd work specifically because of its asocial nature. Meanwhile, Milland, the TurkerNation community manager, is skeptical of Daemo’s academic origins, and would rather see an MTurk competitor be fully worker-owned and operated—something that, she says, the community is raring to build.
It’s also not yet clear how Daemo’s model of transparency and collective governance will scale. At its current size, it's remarkably manageable: Workers are hand-selected, and Bernstein is able to talk to nearly every requester and explain the lay of the land. When I was unsure of whether I’d properly set up my interview task, Bernstein was quick to troubleshoot with me over email; when my task went live, he immediately introduced me to the community and vouched for my legitimacy." (https://www.wired.com/story/amazons-turker-crowd-has-had-enough/)
More Information
Bibliography
Stop citing this one: https://medium.com/@silberman/stop-citing-ross-et-al-2010-who-are-the-crowdworkers-b3b9b1e8d300#.blujriue3
- Fort, Karën, Gilles Adda, and K. Bretonnel Cohen. 2011. Amazon Mechanical Turk: gold mine or coal mine? Computational Linguistics 37(2): 413–420.
- Martin, David, Benjamin V. Hanrahan, Jacki O’Neill, and Neha Gupta. 2014. Being a Turker. Proc. CSCW ‘14: 224–235.
- Milland, Kristy (“spamgirl”). 2014. The myth of low cost, high quality on Amazon’s Mechanical Turk. Turker Nation, 30 Jan 2014.