The most prestigious law school admissions discussion board in the world.
|
 |
 
|
Rate my PC build for large model local LLM
| Chest-beating Business Firm Elastic Band | 12/11/24 |
| crawly white property filthpig | 12/11/24 |
| Chest-beating Business Firm Elastic Band | 12/11/24 |
| crawly white property filthpig | 12/11/24 |
| Chest-beating Business Firm Elastic Band | 12/11/24 |
| Clear Hairraiser Tank Range | 12/11/24 |
| Chest-beating Business Firm Elastic Band | 12/11/24 |
| Clear Hairraiser Tank Range | 12/11/24 |
| https://imgur.com/a/o2g8xYK | 01/08/25 |
| greedy step-uncle's house | 12/11/24 |
| Chest-beating Business Firm Elastic Band | 12/11/24 |
| Sepia razzle partner hall | 12/11/24 |
| Talented hissy fit | 12/11/24 |
| maize area | 12/11/24 |
| Chest-beating Business Firm Elastic Band | 12/11/24 |
| plum zippy travel guidebook sneaky criminal | 12/11/24 |
| Chest-beating Business Firm Elastic Band | 12/11/24 |
| obsidian up-to-no-good whorehouse | 12/11/24 |
| plum zippy travel guidebook sneaky criminal | 12/11/24 |
| Sepia razzle partner hall | 12/11/24 |
| underhanded hell french chef | 12/11/24 |
| plum zippy travel guidebook sneaky criminal | 12/11/24 |
| underhanded hell french chef | 12/11/24 |
| Chest-beating Business Firm Elastic Band | 12/11/24 |
| plum zippy travel guidebook sneaky criminal | 12/11/24 |
| Beta brunch turdskin | 12/11/24 |
| Chest-beating Business Firm Elastic Band | 12/11/24 |
| Talented hissy fit | 12/11/24 |
| Excitant ruby generalized bond lettuce | 12/11/24 |
| Talented hissy fit | 12/12/24 |
| Thriller bespoke ape mood | 12/11/24 |
| Chest-beating Business Firm Elastic Band | 12/11/24 |
| red coiffed headpube point | 12/11/24 |
| Chest-beating Business Firm Elastic Band | 12/11/24 |
| red coiffed headpube point | 12/11/24 |
| Chest-beating Business Firm Elastic Band | 12/11/24 |
| red coiffed headpube point | 12/12/24 |
| know-it-all translucent degenerate azn | 12/12/24 |
| Chest-beating Business Firm Elastic Band | 12/12/24 |
| contagious mad cow disease preventive strike | 12/11/24 |
| Chest-beating Business Firm Elastic Band | 12/11/24 |
| contagious mad cow disease preventive strike | 12/11/24 |
| Chest-beating Business Firm Elastic Band | 12/11/24 |
| crawly white property filthpig | 12/11/24 |
| vibrant dull resort boistinker | 12/12/24 |
| Chest-beating Business Firm Elastic Band | 12/12/24 |
| crawly white property filthpig | 12/12/24 |
| know-it-all translucent degenerate azn | 12/12/24 |
| pitbulls eating your face in hell forever tp | 01/08/25 |
| imagine if the races were reversed | 01/08/25 |
Poast new message in this thread
Date: December 11th, 2024 1:25 AM Author: Chest-beating Business Firm Elastic Band AMD Ryzen 9 9950X GPU1: RTX 4090 GPU2: RTX A6000 128gb RAM 8TB NVMe 4.0 Windows 11 Pro |
 |
Date: December 11th, 2024 2:03 AM Author: crawly white property filthpig why not two 4090 (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48435788) |
|
Date: December 11th, 2024 2:12 AM Author: Chest-beating Business Firm Elastic Band My combo is more optimal for dual GPU since you can't onboard dual 4090s on 1 motherboard and eGPU throttles 50-70% of 4090 performance due to inherent bandwidth limitations with thunderbolt connection vs A6000 is optimized for low bandwidth and has 48gb VRAM (vs 24gb for 4090) to handle massive data sets
|
|
Date: December 11th, 2024 3:34 AM Author: crawly white property filthpig the CPU absolutely does nothing for LLM? (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48435877) |
|
Date: December 11th, 2024 5:14 AM Author: Chest-beating Business Firm Elastic Band CPU matters. That's why I got the best one (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48435907) |
|
Date: December 11th, 2024 8:22 AM Author: Clear Hairraiser Tank Range There's something called CPU offloading where some processing is handled by the CPU. But the kind of operations used in LLMs is different than the kind of operations at which CPUs excel. (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48436129) |
|
Date: December 11th, 2024 1:36 PM Author: Chest-beating Business Firm Elastic Band Do you think Apple's future M4 max/ultra cpu will be better at LLM? (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48437276) |
|
Date: December 11th, 2024 2:57 PM Author: Clear Hairraiser Tank Range no (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48437673) |
|
Date: January 8th, 2025 9:20 PM Author: https://imgur.com/a/o2g8xYK M1 Ultra is still plenty (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48533861) |
Date: December 11th, 2024 1:34 AM Author: greedy step-uncle's house i have similar but just for poasting (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48435773) |
|
Date: December 11th, 2024 2:14 AM Author: Chest-beating Business Firm Elastic Band (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48435795) |
|
Date: December 11th, 2024 3:05 PM Author: Sepia razzle partner hall (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48437693) |
|
Date: December 11th, 2024 9:07 PM Author: Talented hissy fit (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48438902) |
|
Date: December 11th, 2024 11:24 AM Author: maize area Where are you finding 4090s for sale that aren't jacked up 2x MSRP by scalpers? (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48436665) |
|
Date: December 11th, 2024 1:33 PM Author: Chest-beating Business Firm Elastic Band I got it months ago and just putting together now.
Are prices on 4090 still that bad? I thought 5090 and 5080 were about to drop any day now |
|
Date: December 11th, 2024 1:34 PM Author: plum zippy travel guidebook sneaky criminal For work? (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48437264) |
|
Date: December 11th, 2024 8:56 PM Author: Chest-beating Business Firm Elastic Band Yep (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48438874) |
|
Date: December 11th, 2024 1:35 PM Author: obsidian up-to-no-good whorehouse Is setting up a local LLM pretty easy? Does it need coding savvy? (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48437271) |
|
Date: December 11th, 2024 1:36 PM Author: plum zippy travel guidebook sneaky criminal really easy with ollama, https://ollama.com/ (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48437274) |
|
Date: December 11th, 2024 1:38 PM Author: Sepia razzle partner hall (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48437286) |
|
Date: December 11th, 2024 3:40 PM Author: underhanded hell french chef whats the benefit of running these locally vs just subscribing to claude or whatever
|
|
Date: December 11th, 2024 3:45 PM Author: plum zippy travel guidebook sneaky criminal the ones running locally are not as powerful and don't accept really long context windows. the main benefit is no limits to the number of requests and you can find uncensored models. (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48437832) |
|
Date: December 11th, 2024 3:55 PM Author: underhanded hell french chef does the hardware above make it practical to train your own? could you train on just on a giant dump of legal cases and get one superior at legal research at the expense of everything else, or is that even available better commercially at this point?
|
|
Date: December 11th, 2024 4:38 PM Author: Chest-beating Business Firm Elastic Band 1) Yes the above hardware can easily handle large models and is possibly even overkill for CRM, basic legal queries / prompts about client's file, and drafting complex legal briefs
2) Local llm with win10 Pro (eg, ollama, GPT4All, chat rtx) is the ONLY option for attorneys to do this offline given client data sensitivity
3) if you spent $10k on a fully spec'd out Mac Pro/Studio with 192gb unified ram it would not come close to this setup for local LLM in terms of speed and ability to handle large data and complex writing tasks |
|
Date: December 11th, 2024 5:23 PM Author: plum zippy travel guidebook sneaky criminal 0 chance you will be able to train your own LLM, these companies spend tens of millions $$$ on gpu hardware to train an LLM.
The best you can do locally is "retrieval augmented generation" that is specific to legal cases/research. |
|
Date: December 11th, 2024 1:37 PM Author: Beta brunch turdskin You don’t need this much power just to take notes/exams at NYU Law tax classes. (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48437283) |
|
Date: December 11th, 2024 8:48 PM Author: Chest-beating Business Firm Elastic Band 163 (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48438847) |
|
Date: December 11th, 2024 10:29 PM Author: Talented hissy fit (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48439148) |
|
Date: December 11th, 2024 10:32 PM Author: Excitant ruby generalized bond lettuce (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48439153) |
|
Date: December 12th, 2024 12:12 PM Author: Talented hissy fit (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48440746) |
|
Date: December 11th, 2024 4:41 PM Author: Thriller bespoke ape mood how much did it all come to? will a local llm be capable of translating from exotic languages? (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48438026) |
|
Date: December 11th, 2024 8:52 PM Author: Chest-beating Business Firm Elastic Band Under $5k but I didn't buy a6000 card yet
Those fuckers are expensive! |
|
Date: December 11th, 2024 5:25 PM Author: red coiffed headpube point how many watts does the PSU have? (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48438199) |
|
Date: December 11th, 2024 8:50 PM Author: Chest-beating Business Firm Elastic Band 170
Edit: this is the CPU watts |
|
Date: December 11th, 2024 9:06 PM Author: red coiffed headpube point that's a bit low, I don't think u actually have built a computer. (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48438901) |
|
Date: December 11th, 2024 9:15 PM Author: Chest-beating Business Firm Elastic Band Dark Side of the Moon 1350W PSU, Advanced Liquid Cooled CPU & Clear Side Panel Item number: 321-BIKW |
|
Date: December 12th, 2024 8:50 AM Author: red coiffed headpube point You are using an OEM Alienware part for you PSU? How does that make sense? (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48440037) |
|
Date: December 12th, 2024 8:54 AM Author: know-it-all translucent degenerate azn OP is growing a beard and changing monikres as we speak (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48440043) |
|
Date: December 12th, 2024 10:17 AM Author: Chest-beating Business Firm Elastic Band (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48440299) |
|
Date: December 11th, 2024 8:53 PM Author: contagious mad cow disease preventive strike I really want to put together my own local LLM setup too. Tagging this thread for later. 180 (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48438860) |
|
Date: December 11th, 2024 8:53 PM Author: Chest-beating Business Firm Elastic Band We should start a discord or something (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48438863) |
|
Date: December 11th, 2024 8:59 PM Author: contagious mad cow disease preventive strike I'm down, I'm still a total noob at this I haven't even started to do my own research on what hardware I'll need (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48438883) |
|
Date: December 11th, 2024 9:01 PM Author: Chest-beating Business Firm Elastic Band If you have an M2 or later MacBook/mini you can install ollama or lm studio right now on an external ssd and start fucking around (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48438889) |
|
Date: December 11th, 2024 10:31 PM Author: crawly white property filthpig can u show us what u can do with ur setup (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48439150) |
|
Date: December 12th, 2024 10:23 AM Author: vibrant dull resort boistinker What will you use it for? How is it better than just getting a subscription to Claude/ChatGPT? (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48440310) |
|
Date: December 12th, 2024 10:58 AM Author: Chest-beating Business Firm Elastic Band Re 1st question: scaling and streamlining CRM and customer service; reducing bottlenecks, automating as much of the client lifecycle as possible Re 2nd question: mainly compliance with HIPAA, SCA, ECPA, CPRA etc |
|
Date: December 12th, 2024 11:04 AM Author: crawly white property filthpig link to wat u can get from ur setup (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48440496) |
|
Date: December 12th, 2024 1:06 PM Author: know-it-all translucent degenerate azn automating as much of the client lifecycle as possible
tptptptp |
|
Date: January 8th, 2025 8:34 PM Author: pitbulls eating your face in hell forever tp Bump for 5090, how many are you getting OP? (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48533708) |
|
Date: January 8th, 2025 9:08 PM Author: imagine if the races were reversed Isn't it just easier to buy a Mac Studio or whatever it's called? Supposedly it's very good for private LLM stuff (http://www.autoadmit.com/thread.php?thread_id=5647806&forum_id=2#48533823) |