Improving Labeling Consistency with Detailed Constitutional Definitions and AI-Driven Evaluation

In three linesMethod to improve consistency in automated labeling pipelines for content moderation. Authors propose an AI-driven workflow where an LLM writes detailed per-category constitutions (harassment, hate speech, non-violent crime), then a frontier LLM interprets them to generate golden labels. Result: 57x reduction in cross-model inconsistency vs paragraph definitions.

Summary generated by Claude — human-verified