The slowest benchmark in science
1.
It’s late 2025, and the field of machine learning is running out of benchmarks to beat.
Chess, Go, and tic-tac-toe have fought bravely and fallen. {fill later} has saturated long ago. {fill later} has also saturated. Even ominously named Humanity’s Last Exam is growing steadily upward, from X% last year to the current record of XX%.
But those are all easy. Let’s try to find benchmarks that can offer a real challenge.
Fusion comes to mind as a classic “always 30 years away” field. The progress in fusion is glacial:
- From JET 1997 Q=0.67 to NIF 2023 Q≈1.9: factor ≈