Appen's Transformation: From Manual to Automated Fraud Detection with AI/ML

Appen is a leading provider of high-quality training data for organizations that build effective AI systems at scale. The company uses crowd workers to label datasets to train ML and DL models. Their fraud detection system is used to ensure data quality by eliminating low-quality contributions. Appen's goal was to scale the number of distributed workers it could monitor per day to raise the bar for detecting and preventing malicious activity on the platform. They also aimed to reduce the amount of manual work done by distributed workers, to increase the speed and efficiency of data processing and to eliminate human error.

Appen, a leading provider of high-quality training data for AI systems, was facing a significant challenge in scaling its fraud detection mechanism. The company was using a partially automated but mostly manual system to detect and prevent malicious activity on their platform. This system, which relied on SQL and Python scripts, was not efficient enough to handle the increasing volume of work. Appen was struggling to monitor more than 50 jobs per day manually and considered hiring 20+ data analysts to keep up with the platform’s growth. The company needed a solution that would allow them to scale their fraud detection, increase the efficiency of their crowd workers, and attract new enterprise clients. The existing system also posed a challenge in terms of data quality, as it was prone to human error and could not efficiently eliminate low-quality contributions.

To address these challenges, Appen partnered with Provectus to design and build an automated, ML-powered fraud detection platform. The new platform featured a scalable SaaS architecture and a user-friendly GUI. Provectus designed and built data pipelines to streamline the process of labeling, annotating, categorizing, and moderating data. They also developed highly accurate ML/DL models using TensorFlow, which formed the AI core of the fraud detection solution. A web application was developed for the Appen team to efficiently manage data and alerts. The solution was fully automated and integrated for maximum efficiency and ease of use. Continuous monitoring was enabled using Prometheus and Grafana, providing complete visibility into the solution’s performance to Appen’s engineers, fraud analysts, and business stakeholders.

• The new ML-powered fraud detection platform brought about significant operational improvements for Appen. The platform enabled the team to process and handle 20x more jobs per day, with almost 97% of all jobs handled automatically. This increased efficiency led to a 25% reduction in scammer activity and a 5x drop in churned judgements. The platform also resulted in greater productivity and efficiency of staff, as Appen managed to avoid hiring 20+ data analysts, thereby considerably reducing operational costs. Most importantly, the new fraud detection platform helped Appen meet the customers’ requirements in regard to data quality and service efficiency. The changes made allowed the company to better satisfy existing and attract new enterprise clients, accelerating their global expansion.

• Appen managed to scale trust and security of the platform, which allowed them to manage 20x more jobs per day

• Appen's new platform detected 25% more scammers

• 97% of all jobs were handled automatically