Model comparison · October 2, 2026

GPT models for code review

I compared seven GPT models as code reviewers. The question was simple: which ones find bugs without inventing problems, and what does that cost? Each reviewed the same four pull requests twice.

GPT-6.1 Sol found the most bugs, with fewer false positives and a lower estimated cost than GPT-6 Sol.

Bugs foundFalse positivesAvg. / four PRs

Model

Est. costUSD per PR

GPT-6.1 Sol
Bugs found per four pull requests, averaged over two passes:4
False positives per four pull requests, averaged over two passes:0.5
Estimated model cost per PR: $0.23–$0.25
GPT-6 Sol
Bugs found per four pull requests, averaged over two passes:3.5
False positives per four pull requests, averaged over two passes:1
Estimated model cost per PR: $0.31
GPT-6 Luna
Bugs found per four pull requests, averaged over two passes:1.5
False positives per four pull requests, averaged over two passes:0.5
Estimated model cost per PR: $0.013
GPT-5.6 Sol
Bugs found per four pull requests, averaged over two passes:2
False positives per four pull requests, averaged over two passes:0.5
Estimated model cost per PR: $0.53
GPT-5.6 Luna
Bugs found per four pull requests, averaged over two passes:0
False positives per four pull requests, averaged over two passes:1.5
Estimated model cost per PR: $0.028
GPT-5.5
Bugs found per four pull requests, averaged over two passes:2.5
False positives per four pull requests, averaged over two passes:1
Estimated model cost per PR: $0.68
GPT-5.4
Bugs found per four pull requests, averaged over two passes:0.5
False positives per four pull requests, averaged over two passes:2
Estimated model cost per PR: $0.38

Counts average two passes over four pull requests. False positives are bug reports ruled out by the code checks. Costs cover model usage only.

A small test with AI-assisted checks, not a general ranking. The PRs are private.

Results and method