Mercor

Founded by Brendan Foody in 2023

Hire specialized experts to train, evaluate, and benchmark artificial intelligence models

Pricing model

Free

Rating

★ Mixed Reviews

Time to value

Quick Setup (< 1 hour)

You can use Mercor to source vetted domain experts who train and benchmark artificial intelligence models. The service connects organizations with specialized professionals such as software engineers, physicians, accountants, and lawyers to evaluate model performance against real-world tasks. You can test your models using standardized professional benchmarks, gather expert feedback, and red-team safety protocols. Mercor also assists enterprises in anonymizing internal workflow data for model training and provides dedicated engineers to implement production-ready autonomous agents directly into existing business workflows.

What Mercor does

Source vetted domain experts for AI model evaluationsRun automated benchmark assessments on foundation modelsAnonymize enterprise records to prepare data for trainingConduct red-teaming sessions to find model failure modesCollect human feedback on complex reasoning outputsBuild reinforcement learning datasets for agent workflowsAccess a network of thousands of credentialed specialists across medicine, law, and engineeringEvaluate models using standardized APEX benchmarks across specialized professionsRun red-teaming exercises to identify safety and alignment risks in model outputsAnonymize sensitive internal workflow data by removing over 60 categories of personal identifiersDeploy forward-deployed engineers to integrate autonomous agents directly into internal toolsGenerate reinforcement learning environments based on complex, long-horizon tasks