When AWS released Nova 2 Lite with its three-tiered reasoning intensity settings, my immediate question was: how do these levels actually perform in production scenarios? I decided to run some real-world tests so other developers wouldn’t have to start from scratch.
The Test Scenario
I built an application that reconciles data tables using text string matching—specifically, linking entity names (like company subsidiaries) to their parent organizations. This requires the model to perform internet-grounded research to make intelligent matches, making it a good candidate for extended thinking capabilities.
Results (per 100 records, with 99% prompt caching)
| Reasoning Level | Estimated Cost | Accuracy |
|---|---|---|
| Low | $0.04 | 64% |
| Medium | $1.25 | 90% |
| High | $3.25 | 96% |
Key Takeaways
The low setting’s 64% accuracy effectively disqualifies it for most production use cases. The real decision point is between medium and high: you’re paying roughly 2.5x more for an additional 6 percentage points of accuracy. For many business applications, that 90% threshold at the medium level hits the sweet spot—accurate enough to be useful, economical enough to scale.
Your mileage will vary depending on your specific use case, but hopefully this gives you a starting framework for your own evaluation.
Leave a Reply