a bit of everything

using-vms-instead-of-docker-for-local-dev-stack

@@ -0,0 +1,29 @@
+Throughout most of my career, the standard choice of tooling to run local dev stacks has been Docker/Podman. You make a compose file and conveniently start/stop services as needed.
+
+It surely works. But lately I've been slightly bothered by it. Many times I develop simple services that don't go through convoluted containerization and deployment patterns, such as Docker images + Helm + Kubernets. Instead, they simply end up as regular processes in a Debian box (typically a VM). What bothers me is that this made me end up with two quite different setups: I would use Docker for local dev, then bare metal deploys in production. This is annoying because you have relevant differences in how things run in each environment, which leaves you with no way to test locally certain deployment quirks. 
+
+You could thing for yourself: sure, but you can have a staging/testflight environment where you tests deploys as closely as possible as in production. That is certainly and option (and what I usually do nowadays), but why not do it already locally?
+
+To scratch my itch, I decided to experiment with a toy project I was fooling around. I decided I would come up with a way to run it locally with a fully isolated VM instead of the existint docker compose driven approach.
+
+## Tooling choice
+
+I decided to use KVM + QEMU + libvirt + inqus
+
+KVM (Kernel Virtual Machine) is a Linux kernel module. It's job is to expose CPU extensions to userspace, which allows the VMs you start to run CPU commands directly in the CPU, and not in an emulated way. Technically, you can run this stack without KVM, but then all CPU workload will be emulated and take many orders of magnitude longer to run, which sucks. One constraint to notice is that you can only leverage KVM if the VMs that you run have the same chip architecture as the host. Obviously, this is because you can't send, for instance, ARM operations to a RISC-V processor. QEMU can run (much slower) without KVM, and can also emulate other CPU architectures without it.
+
+QEMU (Quick EMUlator) is the actual Virtual Machine emulator. Technically, the VM running on your computer is a QEMU process. The process is taking care of simulating all of the physical components of the VM (CPU, RAM, disk, etc). QEMU holds the VM memory in the host RAM memory, handles CPU commands being passed to the host CPU (or emulates them itself if no KVM), and maps a virtual disk to actual disk space in one of the hosts disks. Overall, what QEMU does is to create virtual interfaces that emulate the real interfaces offered by the parts of a computer and then, behind the curtains, map those interfaces to backend processes that either do the thing on the host (writing to a QEMU disk will actually write to the host disk) or on a completely in-process simulation of the part (BIOS/UEFI firmware).  
+
+Now, to start a QEMU VM you need some initial state. Your options resemble pretty much the same you would face when working with a real computer. 
+
+Your first option is to grab an OS installer image and install the OS from scratch. This works just like it would if you were setting up a real computer. QEMU can emulate a CD/DVD reader and pretend the ISO is being read from it. This is an option, but rather painful if you plan on frequently recreating VMs. And it rarely gets used because there is a much better alternative.
+
+But if you had a computer, you could get up to speed much faster if you already had a drive with an OS installed and ready to run, and you simply plugged it to your motherboard and told it to boot from there, right? The same approach is what gets used the most when running QEMU VMs. There is this format called `qcow2`, which is basically a byte-per-byte clone of the disk of a VM. Large providers like Canonical will ship ready to use `qcow2`, which are pretty much the state of the OS after having been installed but before having done a first boot. Combined with `cloud-init`, these images let you spin a ready to use + minimally configured to your needs VMs fast without having to go through the OS install.
+
+You can also build your own `qcow2` images from a running VM, although making generic images that are useful in many situations has certain complexity. It is nevertheless a good option for backups or for some spots where you need to start a VM in a reproducible state repeteadly. And, if you're willing to learn how to do it, you can actually also build your own custom (as in, with whatever setup you need) yet generic (as in, stripped of host specific details that would give you headaches if you spin it repeteadly) images.
+
+Now, you can use QEMU directly in a standalone way, but the interfaces are rather raw. QEMU focuses heavily on doing its job (running a VM) well, and doesn't offer convenient tooling to make managing many VMs or automating VM management ergonomic. 
+
+To solve this, we add Incus to the stack. Incus is a management tool for Virtual Machines. It doesn't actually run the machines, that's QEMU's job. Incus runs a daemon on the host which can be interacted with via CLI.
+
+