Late to this post, but shoot for and AMD Strix Halo or Nvidia Digits mini PC.
Prompt processing is just too slow on Apple, and the Nvidia/AMD backends are so much faster with long context.
Otherwise, your only sane option for 128K context in a server with a bunch of big GPUs.
Also… what model are you trying to use? You can fit Qwen coder 32B with like 70K context on a single 3090, but honestly its not good above 32K tokens anyway.
I wonder if those other “spammy” adblockers do precisely this. Insert affiliate links.
Doesn’t Brave already swap some ads for their own?
I interpreted this as “depression sure is great” at first.
…Still valid.
The context window is indeed the LLM’s memory.
…But its also muddy.
Many LLMs get ‘dumber’ and less attentive as their context windows grow, and OpenAI’s models just happen to be one of these. It’s awful close to the full 128K, even with the full GPT-4. Mistral models are also really bad at long context understanding while, conversely, I find that Google Gemini and Qwen 2.5 are really good close to their limits.
There are attempts to try and measure this performance objectively, like: https://github.com/NVIDIA/RULER
It’s still ephemeral, chats don’t change the underlying language model, but yes it’s interesting.
I know it’s a meme, but the idea that transformers models ‘remember’ anything is a common misconception.
They have zero memory. When you submit a prompt, it feeds your entire chat history as one big prompt and… forgets it immediately, with no impact on the model itself. It’s like its frozen in time, and copied, unfrozen, and thrown away every time it answers.
For all the talk of American Christian nationalism, it sure seems like “Christian Republicans” have been dropped like a rock.
Again, look at Trump’s cabinet. There’s lip service, but do you see many “Pence” kind of bible thumper characters there? Is there any groundswell of support for Pence here, who is clearly on their side? I sure don’t see it.
That’s so corporate. Everyone’s big idea boils down to being a service instead of a seller (because that’s theoretically more profitable), but it sucks.
He’ll probably show up at Ubisoft or EA, lol.
Or maybe even back at Microsoft to lead their handheld launch.
Off topic, but I am jealous of that handle.
The “piracy database” in question is: https://en.m.wikipedia.org/wiki/Library_Genesis
Screw scholarly journals and giant publishers for squeezing cash out of already-thin academics while being shitty gatekeepers, letting complete trash through while blocking others (among other things). They’re greedy, stagnant, exploitive middlemen.
Doesn’t make what Zuck\Facebook did OK, but I have zero sympathy for the monopolistic “victims” here, especially since it’s going to open weight models one can use for free.
Decep is right.
You’re basically implying the bad outweighs the good to the point that it won’t give them any ideas.
The bad may be very amplified now, but I think the TikTok crowd in particular and US history skews towards “majority good”