Top 3 Appen Alternatives in 2024: Review-based Analysis

As businesses invest more in AI projects, the global interest in AI data services grows. Appen is one such service provider in the AI data space. However, recent news¹ has identified some downsides for businesses working with Appen, which could be the reason for its declining popularity.

Readers interested in Appen alternatives usually fall under 2 categories; select yours to see relevant alternatives for your business:

• Appen contributors: These are workers who wish to work for Appen’s alternatives. If you are a worker, click here.
• Clients or buyers: These are businesses that want to consider Appen’s alternatives as a buyer. If you are a client or buyer, click here.

This article provides an evaluation of Appen’s data services platform and highlights the top alternatives to Appen in the AI data collection services space.

Appen alternatives for contributors (Workers)

Here is a table comparing the alternatives based on criteria useful for workers.

Table 1. Alternatives comparison for workers

Appen alternatives comparison for contributors or workers

* Data gathered from Trustpilot since it has mostly worker reviews

** Data gathered from worker reviews. In some cases, workers reported payment delays.

In this section, we analyzed worker reviews from platforms such as G2, Trustradius, Trustpilot, and Capterra to identify reviews that can be useful from the worker’s point of view.

1. Clickworker

Clickworker offers a data collection & generation platform based on a crowdsourcing model. Clickworker sign-up for workers.

Here are some aspects for workers to consider:

Sign-up/onboarding

• Sign up for free through your computer or mobile phone. 
• Users found the signup process easy.²

Compensation

• Payments are promised to be processed on a weekly basis.³
• Payments are done via Payoneer or PayPal.
• Some state the payment process took around 30 days.⁴

2. Amazon Mechanical Turk

Amazon Mechanical Turk, or Mturk, offers a marketplace and a crowdsourcing platform for AI data service and other AI development tasks. MTurk worker sign-up.

Here are some aspects for workers to consider:

Sign-up/onboarding

• Workers faced sign-up calculations without reasoning.⁵
• MTurk support staff did not provide any support regarding the sign-up process for workers.⁶
• Most reviews regarding the sign-up process were negative.

Compensation

• While some workers found the pay rates decent, most found them low, and most stated that other platforms offer better pay.⁷
• Most reviews regarding compensation were negative.

3. Telus International

Telus International claims to operate with a network of over 1 million contributors in its crowdsourcing platform. Sign-up for Telus International’s AI community. 

Here are some aspects for workers to consider:

Sign-up/onboarding

• Some workers stated that the sign-up process requires unnecessary personal information.⁸
• Some workers gave multiple tests during the sign-up process.⁹

Compensation

• Workers found the payment system misleading. Time given to finish jobs is also said to be less, which reduces the hourly pay.¹⁰
• Several workers faced payment delays.¹¹

Appen alternatives for customers (Buyers)

If you are interested in a quick comparison of Appen’s alternatives, here is a table summarizing the data.

Table 2. Alternatives comparison for buyers or clients

* The data is based on B2B review platforms such as G2, Trustradius, and Capterra.

Notes:

• Companies are ranked based on the number of reviews.
• The table is created by using publically accessible and verifiable data.
• All vendors offer data collection and data annotation services

Appen review

Appen offers data collection and management services for each stage of the AI project lifecycle through a crowdsourcing model. Their data is mainly for developing, implementing, and improving AI-powered solutions such as computer vision (CV), facial image recognition, voice recognition solutions, etc.

Appen’s services/offerings

• Data collection service
• Based on a crowdsourcing model.
• Data collection of all data types (image, video, audio, text)
• Data annotation (audio, video, text, image annotation) and mode evaluation services

Appen evaluation

While looking at customer satisfaction reviews on G2 and other online sources, some downsides to working with Appen were identified. We have divided this section into downsides from the customers’ perspective and employee’s perspective. The employees refer to the workers that are a part of Appen’s network. It is important to know how the workers are treated in a company since it directly impacts the quality of the work and legal issues.

Client’s perspective

1. Weak financial situation

Our research identified that in 2022¹², Appen had a revenue decrease. These losses also impacted their customers.

Here is a screenshot from Appen’s financial report:

2. Focus on only larger customers

We also identified that over 80% of Appen’s revenues come from its top 5 biggest customers. This indicates that smaller customers might not receive priority over larger customers.

A screenshot from the report:

3. Lack of transparency

• Appen doesn’t offer a free trial, not giving the customer a chance to test their service before committing to it.
• The company also does not provide any details about the crowd’s demographics, qualifications, or diversity.

Worker’s perspective

1. Difficult-to-use UI 

Some sources describe the platform’s user interface as complicated to use. Workers also found invoicing difficult and issues regarding the unavailability of jobs¹³:

2. Low compensation

Workers from Appen’s network also find the compensation rates low. According to some comments, the rates were as low as $2 an hour¹⁴, which is much lower than the US minimum wage. This can be a problem for clients as they might not wish to work with a partner following unfair compensation practices.

For a more detailed review of Appen’s customer and user reviews, check out this article.

Detailed analysis of the alternatives

This section highlights alternatives to Appen and how they can cover some of the gaps. To evaluate the alternatives, we have analyzed the alternatives’ reviews and user ratings from various platforms such as G2, Trustpilot, and Gartner. We also considered the annual reports of the companies to gather details regarding their customers and financial performance. Additionally, we compared the size of the data collector network that each company has.

Size comparison of each alternative’s crowd 

The size of the crowd can be an important factor for data collection service providers since it determines the level of diversity they can offer in the dataset. See the Figure below for a crowd size comparison.

Figure 1. Crowd size comparison of all the alternatives and Appen

1. Clickworker

Clickworker is a data collection service provider that works with a crowdsourcing model. The company offers the following services:

• Training datasets for training machine learning models in multiple languages and target markets.
• Data processing services (Turning raw data into relevant and accurate data) and image annotation
• Sentiment analysis data for open-source tools
• SEO content & text creation services
• Data categorization and tagging
• Conducting surveys & web research

User ratings

This section provides the ratings from B2B review platforms such as G2.

• G2: 4.1 / 5
• Capterra: 4.3 / 5
• Trustpilot: 4.4 / 5

Pricing

The company offers the following details on pricing:

• Pricing is available for some data collection services
• Details on different packages
• No monthly license fee
• A free trial is available

Clickworker’s reviews

1. Comments regarding Clickworker’s performance:

• Clients find the crowd “reliable to work with” and the platform “easy to use”¹⁵.

2. Comments regarding the crowd:

• Largest network of workers amongst the alternatives
• Clickworker provides the following information regarding its crowd on its website:
  • Demographic distribution: How many of the contributors are located in which parts of the world
  • Gender distribution: How many people from the crowd are male/female/other genders?
  • Details regarding their education: How many people from the crowd are high-school graduates, university graduates, or even PhDs?
  • Language: The weightage of the spoken languages in the crowd.

One of the downsides that we identified with Clickworker was that their crowd covers fewer languages as compared to Appen (45).

2. Amazon Mechanical Turk

Amazon Mechanical Turk, or MTurk, is another training data platform that offers a crowdsourced marketplace for companies that need an online workforce to outsource small tasks (also called micro-tasks).

MTurk’s core offering is a platform or marketplace where the client can specify which service they require from the crowd and post it in the shape of an online task or job on the site. The crowd then performs these online tasks to earn extra money for themselves. These “human intelligence tasks (HITs)” or micro jobs can include:

• Data collection for developing and improving AI/ML models
• Data annotation/labeling/tagging 
• Conducting online surveys
• Developing machine learning models

User Ratings

• G2: 4.1 / 5
• Trustpilot: 2.0 / 5

Pricing

• The client decides workers’ pay for each assignment.
• Amazon Mechanical Turk charges 20% on the worker’s pay and 20% extra on micro-tasks with 10 or more assignments.
• A free trial is offered.

Amazon Mechanical Turk’s reviews

1. Scalable service:

• Clients can scale the workforce up or down based on demand, catering to fluctuating workloads and large-scale experiments.

2. Concerns regarding extra charges from customers:

• MTurk charges an extra 5% off the pay for workers with a Master’s qualification. However, some clients complained that the quality of the workers’ output is similar and not worth the extra charge¹⁶.

3. Concerns regarding user-friendliness:

• Some survey service users claim that MTurk is not as user-friendly¹⁷as its alternatives:

4. Concerns regarding communication:

• Data collection customers stated that it is very difficult to follow up with the workers who complete the micro-jobs¹⁸.

5. Quality issues:

• Studies identified quality issues in the research data gathered by MTurkers¹⁹.
• Others hypothesized that quality issues could be due to the limited language abilities of the MTurk’s crowd and that most workers were not proficient in English²⁰:

6. Smaller crowd size than claimed

• While the company claims to have a crowd of around half a million²¹, studies show that there are around only 100,000²² workers available on the platform.
• It was also identified²³ that a large number of Mturk’s workers use Large language models for text production tasks.

For a more detailed evaluation of Amazon Mechanical Turk, check out this article.

You may want to try alternatives to Amazon Mechanical Turk due to some of these drawbacks identified in customer reviews.

3. Telus International

Telus International is a Canadian company that offers data collection services through a crowdsourcing model. Its offerings include:

• Data collection for training machine learning models
• Data entry, analysis, and enrichment
• Content summarization, formatting, and processing
• Dataset creation across multiple languages

User ratings

• Gartner: 4.2 / 5

Telus International’s reviews

• Offers a similar crowd size to Appen
• Offers more language coverage (500+ languages and dialects)
• Even though pricing information was not available, reviews suggest that pricing is one of Telus International’s cons.
• Limited pricing details are available for some services from third-party sources²⁴.

You can also check our data-driven list of data collection companies to find the best option for your business needs.

Further reading

• Top 6 Data Collection Best Practices
• Crowdsourced AI Data Collection Benefits & Best Practices
• Data Annotation: What it is, Why it matters, and Implementations

If you need help finding a vendor or have any questions, feel free to contact us:

External resources

• 1. Hayden Field, (2023). Inside the turmoil at Appen, the former AI darling that’s reeling from executive exits, big losses. CNBC. Accessed: 06/September/2023
• 2. Review on the easy signup process for Clickworker. Capterra. Accessed: 10/November.2023.
• 3. Clickworker review regarding weekly payment promise. Capterra. Accessed: 10/November.2023.
• 4. Clickworker review regarding the long payment process. Capterra. Accessed: 10/November.2023.
• 5. MTurk review regarding sign-up. Trustpilot. Accessed: 14/November.2023.
• 6. MTurk review regarding lack of support during sign-up. Trustpilot. Accessed: 14/November.2023.
• 7. MTurk review regarding low pay rates. Trustpilot. Accessed: 14/November.2023.
• 8. Telus International review on sign-up process. Trustpilot. Accessed: 10/November.2023.
• 9. Telus international review on difficult onboarding tests. Trustpilot. Accessed: 10/November.2023.
• 10. Telus International review on misleading payment and less time to finish the jobs. Trustpilot. Accessed: 10/November.2023.
• 11. Telus International review on payment delays. Trustpilot. Accessed: 10/November.2023.
• 12. Appen Financial report 2022 Page 34. Accessed: 03/Oct/2023.
• 13. Appen review regarding difficult-to-use interface. G2. Accessed: 03/Oct.2023.
• 14. Appen negative review on low compensation. G2. Accessed: 03/Oct.2023.
• 15. Clickworker customer review on reliability and easy-to-use platform. G2. Accessed: 03/Oct/2023.
• 16. Customer review regarding low-quality work. G2. Accessed: 03/Oct/2023.
• 17. Customer review regarding difficult-to-use platform. G2. Accessed: 03/Oct/2023.
• 18. Customer review communication with workers at MTurk. G2. Accessed: 03/Oct/2023.
• 19. Bauer, B., Larsen, K.L., Caulfield, N., Elder, D., Jordan, S. and Capron, D., (2020). Review of best practice recommendations for ensuring high quality data with amazon’s mechanical turk. Accessed: 03/Oct/2023.
• 20. Customer review regarding language issues of Mturk. G2. Accessed: 03/Oct/2023
• 21. Amazon Mechanical Turk. Welcome to the Amazon Mechanical Turk Requester User Interface Guide. AWS. Accessed: 03/Oct/2023
• 22. Difallah, D., Filatova, E., & Ipeirotis, P. (2018, February). Demographics and dynamics of mechanical turk workers. In Proceedings of the eleventh ACM international conference on web search and data mining (pp. 135-143). Accessed: 03/Oct/2023.
• 23. Veselovsky, V., Ribeiro, M.H. and West, R., 2023. Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks. arXiv preprint Accessed: 03/Oct/2023.
• 24. TELUS International company profile. Clutch. Accessed: 03/Oct/2023