The most prestigious law school admissions discussion board in the world.
|
 |
 
|
Two studies show AI benchmarks vastly overstate AI abilities
| LathamTouchedMe | 03/16/26 |
| ....;..;...;;;.....;;......;; | 03/16/26 |
Poast new message in this thread
Date: March 16th, 2026 6:08 PM Author: LathamTouchedMe No doubt AI is groundbreaking. But maybe a little grounding is in order.
Carnegie Mellon study. AI benchmarks so narrowly defined that they only represent 7.6% of all occupational tasks. Benchmarks are disconnected from high-value labor tasks. https://x.com/rohanpaul_ai/status/2033450821850222811?s=46
Alibaba study. Tested code over course of 8 months. Vast majority broke down over time despite initially passing quality. |
Date: March 16th, 2026 6:11 PM Author: ....;..;...;;;.....;;......;; AI is going to be regarded as a joke pretty soon.
It basically has the same value as Excel |