This analysis explores some downsides to using Amazon Mechanical Turk or MTurk, a popular AI data collection and market survey platform. It also compares the top Amazon Mechanical Turk alternatives on the market.
Readers interested in MTurk alternatives usually fall under 3 categories; select yours to see relevant alternatives for your business. Explore:
- Alternatives for MTurk workers.
- Alternatives for MTurk requestors or customers.
- Survey participant recruitment platforms.
MTurk alternatives for workers
Here is a table comparing the alternatives based on criteria useful for workers.
Alternatives | Worker ratings (out of 5)* | Number of reviews* |
---|---|---|
4.4 | 2454 |
|
1.3 | 257 |
|
1.7 | 88 |
* The data was gathered from review platforms like Trustpilot because its reviews focus on workers.
Detailed analysis of alternatives for MTurk workers
This section compares MTurk alternatives based on worker reviews from the leading review platforms and AIMultiple’s analysis.
1. Clickworker
Clickworker provides a platform for completing microtasks for compensation. Key aspects for workers:
- Sign-up: Free and easy via computer or mobile app.
- Compensation: Payments are processed weekly through Payoneer or PayPal, though some report delays of up to 30 days.
2. Appen
Appen’s network includes over 1 million contributors. Key aspects for participants:
- Sign-up: Not available in languages like Spanish or French, though many find the process easy.
- Compensation: Payoneer is the sole payment option, with some noting the withdrawal limit is too high.
3. Telus International
Telus International claims a network of over 1 million contributors. Key aspects for workers:
- Sign-up: Requires personal information and multiple tests for onboarding.
- Compensation: Workers report the payment system as misleading, with less time to complete tasks and frequent payment delays.
MTurk alternatives for data collection requestors (Customers)
If you are interested in a quick comparison of MTurk’s alternatives, here is a table summarizing the data.
Companies | User Ratings* | Mobile application | ISO 27001 Certification | Code of Conduct |
---|---|---|---|---|
4.1 from | ✅ | ✅ | ✅ |
|
Appen | 4.4 from | ✅ | ✅ | ✅ |
Amazon Mechanical Turk | 4.1 from | ❌ | – | ❌ |
TELUS International | 4.7 from | ❌ | ❌ | ❌ |
- * Based on data from leading B2B review platforms.
- Companies are ranked based on the number of reviews.
Figure 1: Crowd size comparison of all companies
The crowd represents human workers in the company’s network who are registered for the micro projects:

Amazon Mechanical Turk review

Launched in 2005 and owned by Amazon, Amazon Mechanical Turk (MTurk) is a crowdsourcing platform that provides a marketplace for companies to outsource micro-tasks like AI training data collection, data entry, and image annotation.
MTurk’s key service is its platform, where clients specify the services they need and post them as online tasks. The crowd performs these tasks to earn extra money. These “Human Intelligence Tasks (HITs)” or micro-jobs can be:
- Data collection for developing and improving AI/ML models
- Data annotation/labeling/tagging
- Conducting online surveys
- Developing machine learning models
Pricing
- The client decides workers’ pay for each assignment.
- Amazon Mechanical Turk charges 20% on the worker’s pay and 20% extra on tasks with 10 or more assignments.
- MTurk charges an extra 5% off the pay for workers with a Master’s qualification. However, some clients complained that the quality of these workers’ output is not different from other workers’.
Why are we looking for Amazon Mechanical Turk alternatives?
This section highlights the drawbacks of using MTurk’s data collection platform.
- Lack of user-friendliness: Some survey service users claim that MTurk is not as user-friendly as its alternatives.
- Difficulty of communication: Data collection clients identified communication issues with the workers.
- Quality-related concerns:
- Studies identified quality issues in the data gathered by MTurkers.1
- A recent research estimated that 33-46% of MTurkers leveraged large language models in their responses.2
- Others hypothesized that quality issues could be due to the limited language abilities of the MTurk’s crowd.
- Other clients also identified issues in the language ability of the MTurk’s crowd.
- While the company claims3 to have a crowd of around half a million, studies show that there are around 100,0004 workers available on the platform.
- It was also identified that a large number of Mturk’s crowd uses Large language models and AI tools for text production tasks.5
For a more detailed Amazon Mechanical Turk review.
Detailed analysis of alternatives for MTurk customers

This section provides a detailed comparison of all the alternatives discussed in this article.
1. Clickworker
A popular MTurk alternative is Clickworker, which offers AI data solutions through a crowdsourcing platform. Its offerings include:
- Datasets for AI/ML models (Image, audio, video, document, text)
- Services are offered by a diverse crowd of over 4.5 million registered workers proficient in ~45 languages and more than 70 target markets.
- Sentiment analysis datasets
- SEO content & text creation services (product descriptions, etc.)
- Online surveys
- Their data collection platform includes an application
Pros and cons
- Reliable and user-friendly: Clients find the crowd “reliable” and the platform “easy to use.”
- Supports mobile and PC: Clickworker’s platform can be used both from mobile and PC, making it easier for customers to track the progress of their projects and making it easier for the contributors to work.
- High-quality output: Some clients found ‘quality’ to be a strong characteristic of Clickworker.
- Effective annotation but pricey: A client found its image annotation service effective and suggested that the pricing could be reduced.
- Pricing: The client determines workers’ pay for each assignment (Free recommendations on setting the rates are provided by the Clickworker). A free trial is offered.
Choose Clickworker to obtain diverse and custom datasets for your AI projects.
2. Appen
Another good alternative to Amazon Mechanical Turk is Appen, which also offers data collection and management services via a crowdsourcing model. For a more detailed Appen review.
Appen’s offerings include:
- Data collection service with datasets (Audio, image, video, text, sensory)
- Data solutions of all data types (image, video, audio, text).
- Data annotation and mode evaluation services are also provided.
Pros and cons
- Diverse network of contributors: The company offers a relatively more diverse crowd (130 countries and 180 languages, as claimed by the company). Clients can gain access to a larger crowd for projects that require diverse datasets.
- User-friendly with server downtime: While customers found its data processing service effective, they also faced server downtimes on several occasions.

- Low compensation: Workers at Appen also find the compensation rates low. According to some comments, the rates were as low as $2 an hour, which is much lower than the US minimum wage. This can be an issue for clients as they might not wish to work with a partner following unfair compensation practices.
- Weak financial situation: Our research identified that in 2022, Appen had a revenue decrease. These losses also impacted their customers.6
3. Telus International
Founded in 2005, Telus International also offers data solutions through a crowdsourcing site. Its offerings include:
- Scalable and custom datasets for developing and improving AI/ML solutions (video, image, audio, sensor-fusion, & text).
- Data annotation services
- Content summarization, formatting, and processing
- Data entry, analysis, and enrichment
- Dataset creation across multiple languages
Pros and cons
- More diverse and offer worker quality checks: A significantly more diverse crowd (500+ languages and dialects, as claimed by the company) compared to other alternatives like Appen. Offers secure data solutions with built-in validation and spot-checking of workers based on client’s requirements.
- Lack of reviews and pricing info: Even though the crowd is more diverse, the size (~1 million) is significantly smaller than other MTurk alternatives, such as Clickworker.
- Pricing: Information regarding prices is not available on the website. Fewer reviews are available online for a more detailed evaluation of the company.
MTurk alternatives for survey requestors (Customers)
If you are interested in MTurk’s alternatives for survey participant recruitment, check out the table below.
Vendors | # of reviews | Free trial | Survey respondent size | Languages offered | Multi-channel survey distribution | Customer support |
---|---|---|---|---|---|---|
Amazon Mechanical Turk | 28 | ❌ | 500K | Not provided | Web-based panel | Through e-mail |
68 | ✅ | 6M | 45 | Web-based panel, Android or iOS | 24/7 phone support |
|
Qualtrics CoreXM | 3,682 | ✅ | – | 50 | Web-based panel | 24/7 live support |
User Interviews | 826 | ✅ | 2.4M | English only | Web-based panel, Android or iOS | – |
Pollfish | 232 | ❌ | 250M | 50 | Web-based panel, Android or iOS | 24/7 live support |
SurveyMonkey Audience | 29,477 | ❌ | 175M | 16 | Web-based panel | 24/7 phone support |
- The number of reviews is gathered from leading B2B review platforms.
- Vendors are listed alphabetically with Amazon Mechanical Turk and reason by Clickworker (sponsored) at the top.
For more:
- Review of tools for survey participant recruitment.
- Complete list of survey participant recruitment services and survey tools.
FAQs
Why are we looking for MTurk alternatives?
Data quality and sourcing are pivotal elements in the development and refinement of AI models, directly influencing their efficiency and output. In this context, the selection of a data collection or generation partner, such as Amazon Mechanical Turk (MTurk), necessitates a thorough evaluation process.
Critical considerations include:
1. Assessing the data solutions and collection services provided by MTurk, ensuring they align with your project’s specific requirements, including the necessity for data mining, sentiment analysis, and diverse data entry tasks.
2. Evaluating the cost-effectiveness of MTurk’s offerings, to ascertain whether their pricing aligns with your budgetary constraints, potentially impacting your project’s extra cash and earn money goals.
3. Gauging customer feedback regarding MTurk’s services, which can offer insights into their reliability and the quality of their micro jobs and online tasks.
4. Confirming the availability of data in the requisite languages and dialects, crucial for projects targeting specific geographical regions or cultural groups.
5. Verifying that MTurk can supply the specific types of data needed for your project, whether it involves tagging objects, completing surveys, or watching videos for data analysis.
Why work with a data collection service?
Companies aiming to enhance their AI models find immense value in collaborating with data collection and annotation services like Appen, Clickworker, and Amazon Mechanical Turk alternatives.
These platforms offer comprehensive data solutions, including data annotation, labeling, and image annotation, essential for converting raw data into high-quality training datasets vital for sophisticated machine learning and neural networks.
With a focus on efficiency and user-friendliness, these services provide access to a variety of tasks, from micro-jobs and online tasks to complex data entry and data mining projects, which are crucial for developing effective AI technologies.
Further reading
External resources
- 1. Bauer, B., Larsen, K. L., Caulfield, N., Elder, D., Jordan, S., & Capron, D. (2020). Review of best practice recommendations for ensuring high-quality data with amazon’s mechanical turk. Accessed: 17/October/2023.
- 2. [2306.07899] Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks.
- 3. Demographics and Dynamics of Mechanical Turk Workers | Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining.
- 4. Welcome to the Amazon Mechanical Turk Requester User Interface Guide - Amazon Mechanical Turk.
- 5. [2306.07899] Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks.
- 6. Appen Financial report 2022 Page 34. Accessed: 03/Oct/2023.
Comments
Your email address will not be published. All fields are required.