Q: Should I record problems that aren’t the model’s fault?
Q: Should I record problems that aren’t the model’s fault?
Any factors that make the product impractical should be recorded, including missing or damaged features unrelated to the model. Prioritize addressing issues rather than analyzing the causes of errors. For example, if customer service does not provide an order tracking link, it should be recorded even if the AI does not have this functionality. Identifying and prioritizing issues is the foundation of building evaluations, and some issues may be engineering or design flaws. Focusing on problems rather than root cause analysis helps write higher-quality annotations when viewing more data.
Yes. When reviewing interactions, write down anything that makes the product less useful. This includes missing or broken features that have nothing to do with the model. Additionally, don’t focus on why the error occurred, as that should only come after you prioritize which issues to fix.
For example, a support agent might tell a customer that an order has shipped without providing a tracking link. Even if your AI doesn’t have the ability to fetch a tracking link, record that problem. A prerequisite to building evals is to identify and prioritize which issues to fix through error analysis. Some of these issues may end up being engineering or design issues that don’t need an automated evaluator, but they are still important to fix!
Lastly, we’ve found that deferring root-cause analysis and focusing on problems allows you to write higher-quality annotations while looking at more data.
This article is part of our AI Evals FAQ, a collection of common questions (and answers) about LLM evaluation. View all FAQs or return to the homepage.
Source: Hamel Husain · hamel.dev