- The problem
- When an artificial intelligence system is unsure, ordinary training often rewards it for guessing because only a correct answer earns credit. That can leave people to catch mistakes, or let a bad action damage files, data, or other work.
- The idea
- The paper trains the system to choose among three outcomes: succeed, use a safe fallback, or make an error. Its training method is designed to keep learning even if the system starts by always choosing the fallback or making mistakes, and it learns when to back off from the task’s results rather than from special labels saying when to quit. For example, a math model can still answer problems it can solve while saying it does not know on ones it is likely to get wrong.
- What it got
- On a challenging math test, the method reduced wrong answers from 79.1% to 8.9%, while success fell by only 3.1 percentage points compared with training that rewards success alone.