Hacker News

points by barrkel 4 days ago | 0 comments

I remain puzzled why anyone would want to do inference on a laptop if they have a home internet connection and a phone.

If you really want to do DIY inference, plug it in at home and set up tailscale so you can access it on the go.

No huge battery drain or toasty hot laptop required. Lower cost laptop. Lighter laptop. Longer battery life laptop. Win win win win.

[–] solyin 4 days ago

Not everyone grew up in a situation where home was a stable dependable place. Not everyone has a home to go to. If you have it with you, they can't mess with it or take it away from you when you're not there. If you grew up in a broken home, you would understand.

[–] barrkel 4 days ago

If the security of your home possessions is an issue, it should be addressed first before you spend thousands on a very stealable laptop that you need to carry everywhere.

[–] dirasieb 3 days ago

homeless RTX Spark owners, that's a new one

[–] solyin 3 days ago

You grow up, that mentality still stays with you. I know multi-millionaires that do the most broke ass poor people nonsense because they grew up poor.

[–] selimthegrim 3 days ago

How many grad students have you met?

[–] Gigachad 4 days ago

And those people are buying $10k laptops to run local AI?

[–] jeroenhd 4 days ago

Those people won't be buying the Spark laptop, but asking "why anyone would want to do inference on a laptop" is broader than what the article covers.

[–] c7b 4 days ago

Local AI is an expensive hobby. You're paying big money for small performance (so mainly for the feeling of "owning it", which is a great motivator). It can't match cloud services, not absolutely nor bang-for-buck, especially if you look a bit for cheaper and freebie options. As someone who loves and promotes local AI, in the situation you described my plea would be to use that money to improve your situation more directly (move out, get support,...). The cost of a laptop that can run somewhat decent models should buy you at least several months of therapy (@1 session/week).

[–] throw1234567891 3 days ago

You can't put a price of sending all your IP to a dubious third-party. Sorry. You argument apples vs oranges.

[–] Gigachad 3 days ago

The situation we are talking about here is someone with no fixed home or insecure housing. That kind of person likely isn’t even paying for cloud AI, let alone buying massively overpriced hardware to run it locally.

The market for someone who wants local AI in a laptop for $10k has to be tiny.

[–] c7b 3 days ago

If you want local AI, which I absolutely encourage you to, I agree with the OP. Get a (mini)-PC and set up a VPN. It's going to save you money compared to a laptop, short- (usually cheaper) and long-term (usually better upgradeability). You'll have more money for therapy, which I would also encourage you to consider if you come from a broken home, as the GP mentioned.

[–] throw1234567891 3 days ago

Sure, they pay you well at Anthropic or another OpenAI to go and ridicule people? Not everyone builds some irrelevant SaaS where everything is for sale. Some of us work with other people's data and treat it seriously.

Maybe you need a therapy. FOMO can be treated, too. Or you're just upset about Trump falling into irrelevance.

[–] Danox 4 days ago

The spark laptop in its present form is not very good, in fact, it’s disappointing.

[–] alerighi 4 days ago

Because maybe you don't want to send your private data to a company that could use it for whatever it wants?

[–] anonym29 3 days ago

GP isn't proposing using closed proprietary models over an API, they're proposing having local hardware serving open weight models at your house/apartment being remotely accessible through an overlay network like Tailscale. The idea being that local inference is very hungry for power and thermal budget, so you run the inference somewhere that power and thermal headroom is abundant (your house/apartment, on a tower, or dedicated unified memory boxes like DGX Spark, Strix Halo, Mac Studio, etc) and then remotely utilize that inference from a regular laptop. You still get all of the benefits of local, private AI inference, without having to expend a laptop's limited power and thermal budget on doing the inference on the laptop itself.

[–] barrkel 4 days ago

Can you explain what data is going where?

[–] entrope 3 days ago

Your prompts and whatever data they collect from your dev environment go to the inference provider, and typically the customer has neither control nor assurance over what the inference provider (re-)uses that data for.

[–] Orygin 3 days ago

Your prompts and data are going through your own VPN to your own inference hardware. You did not read the post apparently

[–] spaceywilly 3 days ago

On the contrary I think in a few years most inference will be done locally. The smaller models have been slowly catching up to frontier for many tasks, and chips are being developed to run large models on phones and laptops (according to Qualcomm CEO they want to run 10T weights on a mobile chip).

To me the simple answer to why is control. If I run a model locally I get to control the model params, the system prompt, all of it. And of course there is no quota. But probably most importantly I am not tied to paying some company a monthly subscription forever.

[–] KeplerBoy 4 days ago

The RTX Spark is also sold in a mac mini form factor. That's pretty well suited for home inference.

[–] barrkel 4 days ago

Exactly. Get yourself a DGX spark and plug it in at home.

I meanwhile, run my ridiculously OP workstation class hardware as a server - 9995wx, 768GiB, rtx pro 6000 w/ 96GiB vram - at home, and my Asus ZenBook is light, cool, and long-lived.

Runs GLM 5.3 flash at 35 tok/sec while consuming only 15 GiB VRAM, and I can run a lot more stuff on there alongside.

[–] jacquesm 3 days ago

That's slower than I would have expected for the RTX 6000, do you have figures for say Qwen 3.8 27B? That model fits completely on the card, that should push it a lot further.

[–] barrkel 2 days ago

I'm pretty certain I can get it to go faster by caching experts and whatnot (with experimental llama.cpp branches), but that would consume a lot more VRAM and it's fast enough as it is for playing around with, and I can leave it always loaded. It's like 50% CPU inference, with a few layers + router + kv cache on the GPU.

[–] jacquesm 2 days ago

Unused VRAM is a crime now, I heard ;)

[–] dlahoda 4 days ago

private memory.

unique local data and know how not shared with cloud.

[–] barrkel 4 days ago

What part of what I said implied data was going to the cloud?

[–] pixelpoet 4 days ago

I have been saying this for years, and the answer hits a nerve that gets you downvoted into oblivion: people prefer the aesthetics of laptops, simple as that. They will absolutely go to the grave arguing that they need the mobility etc, then just leave it plugged in. They simply prefer how laptops look, and they absolutely hate it when someone dares say it.

[–] monegator 4 days ago

I have a laptop for work. 95% of the time i'd prefer a tower or at least a mini PC with proper cooling, but that remaining 5% i have to be around with the laptop is enough. I don't like the idea of having to remember to sync the projects every time (especially while i'm in the middle of testing something new so it's not like i can just pull the latest commit or sync the latest good version of something)

[–] dumberquestions 4 days ago

If you do any travel whatsoever a laptop is usually not optional.

[–] DaSHacka 3 days ago

A laptop that is so beefy it can even run GLM 5.3 locally (while sounding like a jet engine, being as bulky as an old-school travelling briefcase, and having a 1.5 hour battery life) though?

GGP's point is just: keep the bulk of your compute at home, where airflow and energy is cheaper (in an opportunity-cost sense). Keep the laptop portable, cool, and light, with long battery life by offloading your own computing tasks to your own hardware.

[–] barrkel 4 days ago

To be absolutely crystal clear:

I am not saying don't use a laptop.

I am not saying you have to use cloud inference.

I am saying using a laptop for inference is dumb. If you want private inference:

- use a dgx spark or similar plugged in at home

- set up tailscale so you can connect from anywhere

- use your phone as a hotspot on the go

- work on your laptop, using tailscale to connect to your home inference server if you're on the go

This gets you everything except for the few hours you might be on an airplane.

[–] pixelpoet 3 days ago

100% agreed, and people saying they need a laptop are missing the point; I wasn't saying don't use laptops, either.

The simple point I'm always trying to make is that area vs volume is a fundamental physical difference regarding cooling, which is critical for high performance applications (that's why they're considering an RTX Spark in the first place, right?).

There's so much more I could say about my Strix Halo machine, which fits in the palm of my hand and I have a wonderful portable screen and keyboard+touchpad (and only cost me 2k with 128 GB fast memory, idles at 4w, amazing 32 thread Zen 5 CPU with AVX-512, ...), but it's either preaching to the choir or pissing against the wind.