Consider the alternative option, we weaken equality before the law when dealing with non-Western entities who are bound by legal systems we consider on strategic grounds to produce inferior justice. Not only would this by definition be a weakening of our standards of justice, it creates an enormous moral hazard. Common law systems like the UK and US rely strongly on precedent, imagine the kind of unjust precedents which might be set if we do not apply equality before the law strictly. On top of that, having weakened standards of justice to frustrate foreign opponents, what is going to stop the political apparatus from weakening standards of justice again to frustrate domestic groups it doesn't like?

Even if it leads to disadvantage on the global stage, in my opinion the risks associated with weakening equality before the law far outweigh the benefits. A legal system with strong integrity is an asset, and it shouldn't be spent cheaply on short-term geopolitical gain.

If an American citizen murders a Chinese tourist in America, would you expect the American legal system to prosecute the murderer?

And should it be relevant if (hypothetically) China would not prosecute the murder of an American tourist by a Chinese citizen in China?

After all, _their_ AI is just trained on stolen IP, right? Unlike _ours_?

I still can't get over how irresponsible it is of _them_ to release these weights for free. That means _anyone_ could gain access to their AI model now.

At least _we_ make sure access to _ours_ is fully controlled by _us_.

Enterprises also need very stringent IP protections that you don't get through the subscriptions. Even the Anthropic-API per token license isn't great - sometimes you need to go to 3rd parties like AWS Bedrock to get guarantees regarding no multi-tenant GPUs.

This is expected if you're actively steering the agent, but what about running unattended tasks - perhaps multiple such tasks in parallel, such that you can stay engaged by "attending" to each of them in turn? Then you're not restricted to such a requirement, and can benefit from much slower (individually, not wrt. total throughput) inference streams.

> I currently have 192GB and 24GB VRAM

I'm not familiar with beellama, but this seems like it could be the perfect setup for Strata which people are saying good things about, including for running DS4.1F (at least quantized). The whole point of it is to effectively leverage heterogenous RAM/VRAM setups. Of course llama.cpp itself is doing a good job of trying to catch up, but they've yet to support efficient SSD streaming (mmap doesn't quite cut it) in the main branch.

This depends on how you plan to use the machine... but Qwen-flash-next is right at the top of what a 128GB Apple Silicon can do.

QFN with 4bit quantization will run "comfortably" with all weights in memory and a 128k context on a 128GB machine. On M5 Max w/ 128GB, that will give about 75 tok/sec (using oMLX with MTP enabled). It spits out text much faster than I can read.

However, this truly uses all the RAM. You'll have like 8-12GB left for the OS and apps, which isn't really enough to actually use the machine as a workstation while the model is loaded. But it would work fine as an inference server with Opencode running on your laptop. (Hence, "how you plan to use it?")

Alternatively, you can offload QFN's n-gram table to SSD (this is just a settings option in oMLX), which drops generation speed on the same machine to ~60-65 tok/sec (assuming the model lives on the internal SSD) and reduces memory usage to ~90GB with even a large context. That leaves plenty of RAM to use the Studio as your main workstation. Also, you may run much longer contexts because of the extra memory headroom.

Second alternatively, you can use oMLX's expert offloading feature to keep 50% of experts in memory (with n-gram offloading also). This reduces the RAM footprint to a little over 32GB + context. Offloading the experts drops generation speed to around 20-25 tok/sec on the same machine as above, but it would allow you to comfortably run on a 64GB machine, while still using it as a workstation.

I have tested the 50% expert offload case on an M1 Max / 64GB, and the inference speed is only a little slower (more like 18-20 tok/sec) than M5 Max / 128GB because in both cases the bottleneck is SSD read bandwidth and the GPU is spending a lot of time idle.

One last thing: GLM 5.3-flash doesn't use an n-gram table, so you can't use that trick to reduce memory footprint. You'd need to use expert offloading to make it fit a 128GB machine. Since it has over twice as many active parameters as QFN (which doubles the required SSD read bandwidth), my guess is that a 128GB Studio would not be much faster than your current setup for that model.

I wouldn't be surprised if RAM cartel or nvidia will attempt something like that, plus nvidia already talks about license-locking GPUs.

Also, again, IIRC, IBM hardware already works this way. There are almost no aftermarket when I last checked for them because it's all locked up and useless.