Skip to content
Live
OpenAI Deployment Safety·· yesterday

OpenAI CSAM Image Classifier Model Card

OpenAI CSAM Image Classifier Model Card

Brief overview

OpenAI has released the CSAM image classifier model card. The model is trained and evaluated using controlled, manually labeled image datasets that comply with U.S. laws to prevent child sexual exploitation. The accuracy of online evaluations is significantly lower than offline evaluations because the proportion of CSAM in the online evaluation set is much lower than that of the offline set, leading to a few false positives possibly exceeding the number of actual detections, thereby reducing accuracy.

Article

2. Model architecture, data, and training

2.2.2 Data handling

We trained and evaluated the classifier using access-controlled, human-labeled image datasets assembled in compliance with U.S. law for purposes of preventing the sexual exploitation of children. Sensitive material was stored and processed in controlled environments, with restricted access. Legal, privacy, security, and reviewer-wellness controls governed both training and data handling.

3.2.1 Understanding precision in online evaluation

At the same thresholds, precision was substantially lower in the online evaluation set than the offline held-out evaluation set. Precision is the share of flagged images that are true positives, and the held-out set contained a much higher proportion of CSAM than we observed in the online evaluation set. When CSAM is rare, even a small fraction of non-CSAM images flagged incorrectly can outnumber true detections. Those false positives increase the total number of flags without increasing the number of true positives, lowering precision.

The online estimates give us a better indication of the mix of flags a human reviewer would see on ChatGPT. Across several rounds of training, we used reviewed examples from earlier online evaluations to improve performance on the types of errors we had identified. Each round also surfaced new patterns of false positives, underscoring the value of reviewing flags from the current classifier.

Source: OpenAI Deployment Safety · deploymentsafety.openai.com