Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split neonvm runtime tools out of VM image, supply it from neonvm-runner #1199

Open
sharnoff opened this issue Jan 8, 2025 · 0 comments
Open
Labels
a/tech_debt Area: related to tech debt c/autoscaling/neonvm Component: autoscaling: NeonVM

Comments

@sharnoff
Copy link
Member

sharnoff commented Jan 8, 2025

Problem description / Motivation

Currently (2025-01), vm-builder is responsible for adding the "neonvm runtime" to the VM image — this is the set of binaries and scripts responsible for management of the VM.

We have a number of issues resulting from the "neonvm runtime" being tied to the VM image, rather than anything else:

  1. Rollout of neonvm runtime changes is multi-step and slow: It requires cutting a release on this repo, bumping vm-builder in neondatabase/neon, and waiting for a compute release.
  2. We have no way of aligning the contents of the VM with our kernel image

There's also related issues from the input container image being used as the basis for the VM image's filesystem — in particular, that there's no isolation between that and the neonvm runtime, and so any work on the neonvm parts inside the VM must be very careful to not rely on anything outside the /neonvm directory. For example, we need to always set the path to just /neonvm/bin.

Implementation ideas

The internal "neonvm runtime" components should be packaged as a separate QCOW2 image (or similar), distributed as part of the neonvm-runner image, and do the initial boot before mounting what was the VM image. Programs from the VM image should be run by chroot'ing into the mount.

We may also want to configure sshd to chroot into the mount as well, because that's probably more likely to match existing expectations. Access via screen + tty can still work for things outside of that.

@sharnoff sharnoff added a/tech_debt Area: related to tech debt c/autoscaling/neonvm Component: autoscaling: NeonVM labels Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a/tech_debt Area: related to tech debt c/autoscaling/neonvm Component: autoscaling: NeonVM
Projects
None yet
Development

No branches or pull requests

1 participant