Unpublished draft

The slowest benchmark in science

1.

It’s late 2025, and the field of machine learning is running out of benchmarks to beat.

Chess, Go, and tic-tac-toe have fought bravely and fallen. {fill later} has saturated long ago. {fill later} has also saturated. Even ominously named Humanity’s Last Exam is growing steadily upward, from X% last year to the current record of XX%.

But those are all easy. Let’s try to find benchmarks that can offer a real challenge.

Fusion comes to mind as a classic “always 30 years away” field. The progress in fusion is glacial:

  • From JET 1997 Q=0.67 to NIF 2023 Q≈1.9: factor ≈