HumanSignal

Founded 2019

Label, evaluate, and benchmark training data for machine learning models across teams

Pricing model

Free Tier, Paid

You can use HumanSignal to manage data labeling, evaluation, and quality assurance workflows for machine learning projects. The system allows you to build custom annotation interfaces for text, audio, video, sensor streams, and 3D models. You can capture human feedback through side-by-side model comparisons and custom evaluation rubrics to score model outputs accurately. It also supports ground truth benchmark creation, quality reviews, and red teaming workflows. Teams can either run the software within their own private cloud infrastructure or use professional human annotation services.

What HumanSignal does

Create custom data annotation templatesRun side-by-side model evaluation comparisonsLabel text, audio, images, and video filesTrack annotator agreement and review qualityEstablish ground truth benchmark datasetsDeploy labeling infrastructure in a private cloudBuild custom annotation interfaces for PDFs, audio, video, sensor data, and 3D modelsEvaluate model outputs with custom rubrics and side-by-side comparisonsCreate ground truth benchmark datasets to track model quality over timeDeploy on-premises or inside private cloud environments for data securityAI-assisted annotation to speed up labeling workflowsQuality review workflows and traceable workforce management