Hacker News
points by mnahkies 1 day ago | 0 comments
Damn, I've been building something very similar the past few weeks - haven't quite gotten to a public release yet.

Mines based on libvirt and uses cloud-init for creating the images, with shared golden images across projects.

Skimming your docs it wasn't totally clear to me if your images are built per project, or COW from shared toolchains.

Looks like you have a few features in there that I might take some inspiration from though, eg the proxying stuff.

Funny how many of us are trying to solve the same problem at the moment.

[–] mnahkies 1 day ago
One thing I'm struggling with at the moment which I may as well ask here.

Many of my projects use pnpm, which ideally uses hard links from a shared store of dependencies.

One of my goals is to reduce the disk overhead of VM based isolation, so I'd like a shared pnpm store across projects (and the same for maven, etc as well)

With https://virtio-fs.gitlab.io/ mounts, each mount inside the VM is a separate FS, and I can't hard link across them. I've tried playing around with ideas like bind mounting the directory structure from the host in a folder to have a single virtio-fs mount inside the VM - but the bind mounts still can't be hard linked across it seems.

I'm basically wondering if I'm missing something or just trying to do something that isn't possible.

All the host folders are on the same FS - but I don't want to organise the host system around the VM mounting needs, and I also don't want to mount extraneous directories into the VM (eg: simple solution could be just mount ~/projects but kinda self defeating)

(Fedora workstation host)

[–] genxy 1 day ago
Use a host fs that supports CoW.
[–] vindex10 1 day ago
add me as well. I learned about incus from HN ~a month ago, and was poking an idea of having a "development base image". The advantage compared to containers, is that you can hot-plug devices to the running instance: mounts, secrets, etc. incus also supports --vm and container mode with minimal difference in cli commands.

https://linuxcontainers.org/incus/

my main bottleneck was exactly in usability: how to make it least effort to spin up per use case and keep the set of operating scripts small - i didn't want to build an app for this, just shortcuts to use existing app.

[–] evertheylen 1 day ago
Add me to the list, although I use podman containers (https://github.com/evertheylen/probox). I find that the hard part is not restricting the container/VM, but to make it "easy" to do things in the container that require something protected. How will you commit? Where does your agent run, and can it access github? How will you deploy? How will you SSH to a server?

Happy to hear what other people are thinking. It seems like putting an intercepting HTTP proxy is getting popular in other projects.

[–] zenoprax 1 day ago
I usually commit outside the sandbox. Agents will see the git history and will emulate that but they can't sign the commits. If I authorize a series of commits I can just `git rebase -i HEAD~N` and adjust the messages and re-sign at the same time. With git hooks as a threat vector I'm considering just giving them read-only access to `.git` otherwise I can't safely go back in there without the sandbox.

I don't give it access to GH - it can read most things via API or web scraping fast enough and I manually run the commands I need outside the sandbox.

No need for SSH - it's running locally.

---

Given how many holes you're poking into the container I think you'd be better served with a real VM.

[–] mr_mitm 1 day ago
It feels like a fundamental problem. The more you restrict the agents, the less useful they become. The less you restrict them, the more dangerous they become.
[–] evertheylen 1 day ago
It makes sense, but a good system where you can say "give this VM access to repo X, branch Y, and server Z for the next 4 hours" without leaking any secrets into the VM would be a major improvement already.
[–] hollerith 1 day ago
Human coders and sys admins routinely make catastrophic mistakes, too, so it's not like we're facing a novel problem.
[–] mnahkies 1 day ago
On the git access part, I don't let my agent VMs have write access to GitHub currently. I do give them dedicated users on a self hosted gitea instance, and basically manage the blast radius through branch protection etc.

I think at a high level I'm trying to treat them as junior/intern employees where I give them enough permission to do their role, but not enough rope to hang themselves.

[–] evertheylen 1 day ago
I see. But then I have to debug some failing CI pipeline and the agent is quite good at going through the logs, so I give in and `gh auth login` anyway...
[–] mnahkies 1 day ago
I'm pretty sure you don't need to give it full access to accomplish that. It's possible to create scoped PAT etc. alternatively use a separate account. Ultimately your risk tolerance to decide
[–] lukax 1 day ago
You can have a separate GitHub account for your agent (GitHub ToS actually allows that since April) and that account can fork your repo and open PRs. This way it cannot do any real damage but can still push.

See this user for example: https://github.com/filippo-agent

[–] CharlieDigital 1 day ago

    > GitHub ToS actually allows that since April
Do you have more background on this? My team actually ran into some issues with GH when we tried this earlier this year (felt like it was around that same time).
[–] lukax 1 day ago
https://docs.github.com/en/site-policy/github-terms/github-t...

3. Account Requirements

We have a few simple rules for Accounts on GitHub's Service.

You must be a human to create an Account. Accounts registered by "bots" or other automated methods are not permitted. We do permit machine accounts:

A machine account is an Account set up by an individual human who accepts the Terms on behalf of the Account, provides a valid email address, and is responsible for its actions. A machine account is used exclusively for performing automated tasks. Multiple users may direct the actions of a machine account, but the owner of the Account is ultimately responsible for the machine's actions. You may maintain no more than one free machine account in addition to your free Personal Account.

One person or legal entity may maintain no more than one free Account (if you choose to control a machine account as well, that's fine, but it can only be used for running a machine).

[–] mr_mitm 1 day ago
Somebody shared this in another thread: https://pleasedonotescape.com/

I guess they can add all the sibling comments to the list, as well as mine: https://github.com/AdrianVollmer/Agent-Sandbox

(Python, Podman, Squid for network filtering)

[–] knowaveragejoe 1 day ago
It's a neat snapshot at the time it was creates but it's getting out of date.
[–] aforwardslash 1 day ago
Also been working on something similar, with qemu/kvm; the idea is to sandbox agents and limit resource usage per project. Its not production-ready yet, but the repo is public - https://github.com/blackshieldpt/catchpen
[–] dofm 1 day ago
> Mines based on libvirt and uses cloud-init for creating the images, with shared golden images across projects.

This sounds good. Cloud-init is still worth the time, I think (as I just said in another comment)

[–] sigbottle 1 day ago
Add me to the list lmao. It seems like this is pretty popular among folks
[–] sparsesignal 1 day ago
Ha, add me to the list. I built one with libvirt and cloud-init too, mostly focused on network containment (private/host networks are blocked) and VM hardening. There's also a check command that runs nmap from inside the VM to verify nothing private is reachable.

https://github.com/ertug/kvm-sandbox

[–] mnahkies 1 day ago
Heh, yeah also very similar. I don't strip devices like you are - I actually want the memory balloon functionality for example.

My working threat model for my projects is less about "malicious agent trys to escape VM" and more "well meaning agent got confused and screwed up" - the added protection of a VM is desirable, and it's useful for stuff like nested virtualization (agent working on improving the VM orchestration), makes containers inside the sandbox straightforward

I hadn't seen the stuff about filtering spoofed packets so I'll probably add that.