Simple PoC for Firecracker Build Isolation

Previously: Firecracker notes

My proof-of-concept for build isolation is really simple, and once you've got the toolchains (and dependencies) you theoretically don't even need to give the VM network access. I added an /etc/fstab entry to the root fs so that it auto-mounts /dev/vdb. I think this might cause problems if you tried to use the root FS without another filesystem, so I'd love a better idea here.

I then modified the filesystem to auto-start socat. It's already installed on the rootfs from Firecracker's CI, so I just made a systemd service, enabled it by making a symlink, and pointed it to a bash one-liner:

socat VSOCK-LISTEN:52,rcvbuf=1,fork EXEC:'bash -i',pty,stderr,setsid,sigint,sane

This lets you get a shell on the machine without sshing into it. Maybe running the build on machine start through a script would be better, but this seems more flexible. YMMV.

Pack up the squashfs into an ext4 filesystem and you're done with the rootfs.

Then I just made the workspace fs with:

# base the fs size on the workspace directory size, but make sure to leave a little extra
truncate -S 1G workspace.ext4
mkfs.ext4 -d ./workspace -F ./workspace.ext4

and added the fs to the vm config.

For getting shell, make sure you set up the vsock in the config.