The most prestigious law school admissions discussion board in the world.
|
 |
 
|
This critique that LLMs just "predict the next word" seems really dumb
Poast new message in this thread
Date: October 17th, 2026 9:40 PM Author: German pumo Is that how they are making more advancements in mathematical theory in a few weeks than humans have made in the last several decades? (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194615) |
Date: October 17th, 2026 9:45 PM Author: Apathetic prepper it's like saying a nuclear reactor "just heats up water." the AI-denialism is going to age as poorly as COVID-Zero hysteria (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194623) |
 |
Date: October 17th, 2026 9:49 PM Author: German pumo (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194626) |
|
Date: October 17th, 2026 10:48 PM Author: The Penis It's funny because for transformer models it isn't even true. next token prediction describes the training objective. that was the entire big thing with transformer computation that the head can integrate information from widely separated positions in the context. There is more to it than that too. The system develops its own representations and algorithms in order to satisfy the optimization objective, and the final system that emerges still is not fully understood. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194690) |
|
Date: October 17th, 2026 10:51 PM Author: Lucy Mercer (Safari) That is what people literally say about it so the critique is reasonable. It's just false. Its obviously not analyzing a photograph by predicting the next word. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194694) |
|
Date: October 17th, 2026 10:52 PM Author: censorship Yes it is (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194696) |
|
Date: October 17th, 2026 10:53 PM Author: Lucy Mercer (Safari) No it isn't (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194697) |
|
Date: October 17th, 2026 10:54 PM Author: listening intently as peter thiel stutters You're gonna want to sit down for this (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194698) |
|
Date: October 17th, 2026 10:55 PM Author: censorship (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194701) |
|
Date: October 17th, 2026 10:55 PM Author: Chill your Godhead Multimodal Large Language Models (MLLMs) analyze photos by converting visual data into a sequence of numerical tokens that the language model can process just like text. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194700) |
|
Date: October 17th, 2026 11:00 PM Author: The Penis yeah. so you could technically have a next token predictor that predicts the next word via visual processing. you hook up something that is actually next-token predicting like an autoregressive language model to a multi-modal architecture that does visual processing first. but it doesn't matter because transformer llms are not next token predictors. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194707) |
|
Date: October 17th, 2026 11:00 PM Author: censorship Yes they are (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194709) |
|
Date: October 17th, 2026 11:01 PM Author: The Penis That's objectively false. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194712) |
|
Date: October 17th, 2026 11:01 PM Author: censorship Ur wrong but ok (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194714) |
|
Date: October 17th, 2026 11:06 PM Author: The Penis no. the training objective is not the same thing as the underlying computation. in transformers multiple attention heads extract and combine different contextual information allowing the network to perform increasingly complex computations over the context. you are wrong. a transformer does not just predict the next token. It performs parallel, multilayer computations over contextual representations. read ATTENTION IS ALL YOU NEED nigga (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194724) |
|
Date: October 17th, 2026 11:07 PM Author: Chill your Godhead lol wait you're saying it doesn't predict the next token, it uses "parallel, multilayer computations over contextual representations" to predict the next token? (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194728) |
|
Date: October 17th, 2026 11:10 PM Author: listening intently as peter thiel stutters Holy shit
This Changes Everything
I'm all in on AI now |
|
Date: October 17th, 2026 11:10 PM Author: Chill your Godhead (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194736) |
|
Date: October 17th, 2026 11:11 PM Author: The Penis yeah that's what I'm saying. describing the output tells you almost nothing about the computation that produces it. A transformer's attention mechanisms process relationships across the context, and successive layers transform those representations. so yeah, it uses the parallel layered computations to predict the next token. like how a chess engine does complex computations to then choose it's next move. doesn't mean a chess ai is fundamentally a NEXT MOVE PREDICTOR (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194737) |
|
Date: October 17th, 2026 11:12 PM Author: Chill your Godhead its a next token predictor dude. that's literally what that means. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194739) |
|
Date: October 17th, 2026 11:16 PM Author: censorship (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194751) |
|
Date: October 17th, 2026 11:21 PM Author: The Penis if the computations that arrive at that prediction are increasingly layered and complex then what meaningful limitation is the reduction of "next token prediction" doing? its like saying a brain can't think because it's ultimately transmitting electrochemical signals. the sophistication of the overall computation matters. the difference between autocomplete style next token prediction algorithms and current models is enormous. the overall computational process is obviously what matters. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194776) |
|
Date: October 17th, 2026 11:23 PM Author: Chill your Godhead isnt the relevant comparison LLMs vs. e.g. diffusion models (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194781) |
|
Date: October 17th, 2026 11:23 PM Author: censorship U got btfo itt and exposed as a charlatan
Reminds me of Peterman w his crypto shit but I guess in the end he was vindicated for a while |
|
Date: October 17th, 2026 11:28 PM Author: The Penis okay whatever man (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194796) |
|
Date: October 17th, 2026 11:16 PM Author: listening intently as peter thiel stutters "Apropos is the story of the chess game between the invincible Capablanca and Charles Jaffe, pride of the East Side. Capa forgot he was invincible: he lost.
A reporter who was present asked the Cuban, “How far do you see ahead?” Capa replied impressively, “About ten moves”. Then the reporter went over to Jaffe: “How far do you see ahead?” Much to everyone’s surprise, the reply was, “Only one move”. This didn’t make sense. “How could a player who can see only one move ahead defeat another who can delve so deeply?”
Here Jaffe explained: “I see only one move ahead, but it's always the best move.” |
|
Date: October 17th, 2026 11:15 PM Author: censorship I'd explain it but it's like explaining a compiler to a 10 year old
Ur knowledge depth just isn't there |
|
Date: October 17th, 2026 11:25 PM Author: The Penis nah I think if you could do it you would (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194789) |
|
Date: October 17th, 2026 11:26 PM Author: Chill your Godhead the penis is smart he's just angry at dumb people itt (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194791) |
|
Date: October 17th, 2026 11:01 PM Author: Chill your Godhead Yes, Large Language Models (LLMs) are fundamentally built and trained on next-token prediction. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194713) |
|
Date: October 17th, 2026 11:07 PM Author: The Penis an optimization objective isn't the same thing as how the underlying computation ultimately works (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194726) |
|
Date: October 17th, 2026 11:08 PM Author: Chill your Godhead brother, (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194729) |
|
Date: October 17th, 2026 11:14 PM Author: The Penis when people say its a token predictor they are always using it in a way that makes it sound like that is an exhaustive description of it's computational abilities. its like reductionists who say brains are just neurons firing. the reduction smuggles in all sorts of other conclusions (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194742) |
|
Date: October 17th, 2026 11:16 PM Author: Chill your Godhead "predictor" is agnostic to method. it could be fucking linear regression or whatever hyper nerd shit you said above. (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194746) |
|
Date: October 17th, 2026 11:22 PM Author: The Penis it's mean to claude though to call it a dumb text predictor (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194780) |
|
Date: October 17th, 2026 11:24 PM Author: Chill your Godhead i dont use claude (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194783) |
|
Date: October 17th, 2026 11:50 PM Author: \"\'\"\'\"\'\'\"\"\" It’s a lame critique made by people who don’t stop and think what prediction actually is. prediction of a complex output stream requires reconstructing the hidden state of the system that generated it. human language is a compressed projection of human thought, logic, physical reality, and social intent, so predicting the next token at near-optimal accuracy forces a model to internalize the underlying generative rules of those domains. These models are basically being forced by brute force computation to learn everything about the world that humans talk about (and what they don’t, for multimodal LLMs that are just predicting raw pixels) (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194828) |
|
Date: October 17th, 2026 11:51 PM Author: The Penis (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194829) |
|
Date: October 17th, 2026 11:51 PM Author: Chill your Godhead sick (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194830) |
|
Date: October 17th, 2026 11:58 PM Author: hex these reductionist arguments are always a sign of a midwit (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194839) |
|
Date: October 17th, 2026 11:58 PM Author: Chill your Godhead a certain "chill" sort of not being a nerd all the time tp (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194840) |
|
Date: October 18th, 2026 12:04 AM Author: The Penis (http://www.autoadmit.com/thread.php?thread_id=5913613&forum_id=2#50194847) |