[–] scoodah 1 day ago
[–] tptacek 1 day ago
[–] rvz 1 day ago
Another wave of vulnerabilities like this are yet to be discovered in Linux. We are just seeing how weak C and C++ based OSes and kernels are when put under hundreds of AI agents.
[0] https://openai.com/index/a-thousand-fewer-paths-to-root/
[–] imnotsuretheykn 20 hours ago
I’m not sure the people that caused this will be the one fixing it.
[–] fock 14 hours ago
[–] devmor 1 day ago
The only reason it wasn't seen as a problem was due to the effort and time it took to find and exploit them.
[–] amluto 1 day ago
[–] tptacek 1 day ago
[–] amluto 1 day ago
I assume they chose it as their target because Firecracker, like most Linux KVM userspace [0], maps guest memory into its own address space, giving an easy place to write gadgets.
My personal favorite thing to poke into pt_regs is an actual privileged (CPL0) context. [1] Returning from the kernel right back to a controlled kernel context is allowed by the CPU :)
[0] Until fairly recently this was mandatory. These days it’s possible to ask KVM to map something into the guest without having it in host userspace memory.
[1] Corrupt pt_regs so it executes user memory in kernel mode:
https://seclists.org/oss-sec/2015/q3/291
I think this may be my favorite of all exploits I’ve ever written. In fact, I often made a point of not writing interesting exploits for corruption bugs I found, but this one was too much fun to pass up.
[–] tptacek 11 hours ago
[–] monocasa 1 day ago
Which like, isn't really a proof of concept of anything being described.
"That was always allowed".
[–] tptacek 1 day ago
[–] monocasa 1 day ago
It just bothers me that the proof of concept starts with the assumption that host physical memory is mapped into guest physical memory at 2^39. If you can't talk about the actual exploit yet, I get it , but don't write a blog post with something like that called a "proof of concept" when the code shows no exploit, and you're obviously dancing around the actual novel exploit here in the chain. Just keep it in your pants until the disclosure window.
[–] whiatp 1 day ago
They mention in part 1 having a write primitive that can hit a 64 byte region, but never really explained how that worked. I'm guessing this was them being able to have checksum offload write TCP/UDP checksum values into the overhang region past the valid data. Maybe the memory map was set up that way, but obviously relies on getting really lucky with getting their page table in the next page after their packet.
Also it was never called out, but as far as I can tell their exploit should only be able to target the beginning of pages for reading/writing. In order to rug-pull the socket logic that reads the IP IHL field, the IP header content needs to be in a page mapped to the guest. The only useful way to use this attack is to have the region spill out of the guest page into whatever page comes next in the kernel's memory map. (Which also makes the mentioning of the skb linear header kind of confusing, since that would be in a completely unrelated kernel memory region, and just have the data pointers pointing into their data, which is what happens for zero-copy network drivers, which presumably was set up for the kvm network for them to retain guest control over the packet.)
[–] bulk88 19 hours ago
[–] vluft 1 day ago
[–] tptacek 1 day ago
[–] speedstyle 17 hours ago
[–] tptacek 11 hours ago
[–] sscaryterry 1 day ago
Exactly, its not a firecracker issue. Using firecracker without a jailer is like setting your password to "password".
[–] tclancy 1 day ago
[–] sscaryterry 1 day ago
[–] tptacek 1 day ago
This is a really good bug and nobody should downplay it.
[–] roughly 1 day ago
Also - re: the jailer, do you have more info on that being popped?
[–] tptacek 13 hours ago
[–] catlifeonmars 1 day ago
> They ran it without a jailer
And you said
> getting code execution behind the jailer is still effectively game-over
So I’m really confused, which is it?
[–] zobzu 1 day ago
the truth is there plenty more kvm escapes that will be found.
[–] tptacek 11 hours ago
[–] lrvick 1 day ago
Do virtualization with a tiny formally verified microkernel and have complex kernels like Linux run underneath instead of letting a huge, complex, and brittle linux kernel and vm ecosystem control the hypervisor layer.
[–] eddythompson80 1 day ago
This is different from KVM where your hypervisor runs in your Dom0/Root Partition.
[–] markasoftware 1 day ago
[–] eddythompson80 1 day ago
I was replying to OP saying it's the same "approach" as Xen, if your preference is "approach" based. It's fair though to point to Nova's smaller code base. It's becoming an important metric for me in general (how big is the code base). It's not a bullet proof thing, but having a smaller code base is a positive sign in 2026 for me.
[–] markasoftware 1 day ago
[–] eddythompson80 1 day ago
[–] ThePowerOfFuet 1 day ago
[–] tptacek 1 day ago
[–] lrvick 1 day ago
[–] tptacek 1 day ago
[–] lrvick 1 day ago
SeL4 is the closest contender on my radar atm.
[–] tptacek 1 day ago
And then, of course, L4s live in millions and millions of shipped products, which is a reason to consider L4 instead of something novel, but I'm not saying you should or shouldn't.
[–] lrvick 1 day ago
SeL4 is the only open, mature, and formally verified L4 solution I am aware of that can run on modern servers, but IIRC given vCPUs are not first class objects and any low level CPU virtualization features are handled in user space, it comes at a real performance cost making it only suitable for single-core or non-performance-critical applications.
Nova by contrast was built to handle the expensive and trust critical TPM and vCPU handling directly for high performance remotely attestable multi-core applications without having to reach for something as heavy as Xen, though formal verification is admittedly not as complete as SeL4 yet AFAICT, though lots of recent progress here.
Those are pretty big tradeoffs, but if you or anyone else knows other viable xen alternatives this general shape with a tiny easily-auditable footprint, please chime in!
[–] traceroute66 1 day ago
To save people the time:
"This code is experimental and not feature complete. If it breaks, you get to keep both pieces."
[–] lrvick 1 day ago
See his latest talk which represents a lot of impressive recent work: https://cfp.3mdeb.com/media/bsmconf/submissions/KW3SYT/resou...
I am working on a production OS using it as a foundation.
[–] zX41ZdbW 1 day ago
It hosts hundreds of database engines - many of them have vulnerabilities, and some of them, like Pandas, are basically open RCEs regardless.
We can PoC the Firecracker VM Escape on that service.
[–] gavinray 1 day ago
> "run SQL against 110+ databases"
This is one of the most useful web tools I've ever seen.I work on connectors for datasources (mostly DB's. Being able to quickly test things like sorting behavior and how the same function can be emulated across DB's would have made my life so much easier.
[–] theowaway213456 1 day ago
Does this mean you ran the exploit on your service and confirmed it worked? If so, why would you be advertising this, lol. Unless you've already patched?
[–] markasoftware 1 day ago
[–] YetAnotherNick 1 day ago
Couldn't find it. Care to provide link.
[–] tptacek 1 day ago
Crucially, this happens in the middle of packet processing, so the kernel code is forced to work with two different ideas of how long the IPv4 header is. All hell predictably breaks loose.
[–] catlifeonmars 1 day ago
[–] ytxm 1 day ago
https://github.com/torvalds/linux/commit/6785011f8b16abf40b1...
[–] sscaryterry 15 hours ago
[–] markasoftware 1 day ago
[–] denysvitali 1 day ago
(Doesn't HN usually deduplicate links? I'm confused)
[–] tptacek 1 day ago
[–] denysvitali 1 day ago
[–] clhodapp 1 day ago
How do people feel about the hardware isolation provided by microVMs versus the stronger end of software isolation techniques like gvisor?
Right now, we put a huge amount of stock in "hardware isolation". If that starts falling to agents to any substantial degree, I wonder whether we might end up leaning more on patchable software isolation, similar to the rise of software RAID (though, in userspace this time).
[–] oasisaimlessly 1 day ago
[–] eddythompson80 1 day ago
[–] nwilkens 1 day ago
Running our microVM's inside of a jailed zone, at least gives me some peace of mind.. It's far from perfect, as a zone escape could be lurking.. but at least it's a layer of defense.
[–] tptacek 1 day ago
[–] cyphar 1 day ago
[–] tptacek 1 day ago
[–] bulk88 19 hours ago
This so-called Firecracker exploit never explained how the Guest was able to write to the Host OS's Firecracker user mode process. If the write controlled by the guest to the host was that TCP/IP routing stack Exploit in the host OS's Linux kernel, this Firecracker Exploit is meaningless. The Guest already has unlimited control over the host OS.
I (and anyone) can do this exploit in 2-3 minutes using GDB or Visual Studio's C debugger tool. Freeze process, type new pointers into CPU context snap shot, click resume, Exploit done.
> The configuration used here had no jailer. Firecracker's production containment includes dropping privileges and restricting the process through namespaces and seccomp. A deployment using those controls requires a separate assessment of the privileges and resources available after compromising its VMM.
If I'm understanding all of this correctly, a properly jailed and configured firecracker instance would greatly reduce the blast radius of this escape, and I feel as though it would have been good for them to explain what jailing and dropping privileges would do to help. I'd hope most production firecracker deployments would be doing this already.
Not saying that this is any less severe and the root issue is not specific to Firecracker, but to a flaw in the kernel's network code -- they just used an insecure firecracker configuration to demonstrate it. However, I feel like some are taking this as a flaw with firecracker when its not. This is an issue in the kernel networking stack.