IMP

Appearance

How a GitHub Job Reaches a Runner ​

This page follows a queued GitHub Actions job. It separates three decisions that are easy to confuse:

  1. capacity: how many runner VMs Imp should create;
  2. placement: which Linux node should run a VM;
  3. job assignment: which registered runner GitHub chooses for the queued job.

Imp controls the first two. GitHub controls the third.

mermaid
flowchart TD
  queued[GitHub job is queued] --> demand[Demand reaches the runner pool]
  demand --> capacity[Imp chooses VM count]
  capacity --> placement[Imp chooses a Linux node]
  placement --> agent[Agent Pod starts Firecracker]
  agent --> register[Guest runner registers with GitHub]
  register --> assign[GitHub assigns a matching job]
  assign --> execute[Guest runs one job]
  execute --> remove[Runner exits; Imp cleans up]

Plain explanation: a queued job does not run inside a Pod. It can cause the pool to create a VM. Imp chooses where that VM runs. After the guest runner registers, GitHub decides whether that runner matches the queued job.

1. A queued job becomes demand ​

A GitHub Actions workflow enters GitHub's queue before Imp sees it. The current controller can receive demand in two ways:

  • Polling: for repository-scoped pools, it asks GitHub for queued workflow runs on a fixed interval.
  • Webhook mode: another component must validate the GitHub event and write the imp.dev/runner-demand annotation. The pool controller reads that annotation; it is not itself the inbound webhook server.

For organisation-scoped GitHub pools, the current queue-depth method returns zero because this implementation does not use GitHub GraphQL for organisation-wide queue depth. An organisation pool therefore needs another demand source, such as the webhook annotation or pre-warmed minIdle capacity.

2. Imp chooses how many VMs to create ​

The runner-pool controller starts at minIdle. It raises the desired count when a polling result or webhook annotation reports more waiting work. It never exceeds maxConcurrent, and one reconcile creates no more than scaleUpStep VMs.

That is capacity control. It does not select a particular GitHub workflow job.

3. Imp chooses a Linux node ​

Each new ImpVM is scheduled by Imp's controller, not by the Kubernetes Pod scheduler. The controller:

  1. lists nodes labelled imp/enabled=true;
  2. filters them by the VM's node selector, readiness, cordon state, pressure conditions, and tolerations;
  3. checks configured VM capacity when available;
  4. chooses the least-loaded eligible node;
  5. writes that node name to ImpVM.spec.nodeName and marks the VM scheduled.

The Imp agent is a Linux DaemonSet Pod. One agent runs on each enabled node. An agent ignores ImpVMs assigned to other nodes. The selected node's agent sees the scheduled VM and starts the Firecracker process locally.

Read How an ImpVM Runs on Linux for the boundary between the agent Pod and the microVM.

4. The guest runner registers ​

The agent turns the VM's OCI layers into an ext4 root filesystem, starts Firecracker with the selected kernel, root disk, CPU and memory values, and waits for the guest agent over VSOCK.

It reads the VM-owned JIT Secret and asks the guest agent to run /usr/local/bin/runner. The encoded JIT configuration travels as an execution environment variable rather than a command-line argument. The call waits for the runner process to return, then the agent deletes the one-time Secret.

The guest runner uses the JIT configuration to register with GitHub.

5. GitHub assigns the job ​

Once GitHub sees the runner, GitHub applies the workflow's own runner-group and label rules. That matching decision is outside Imp's controller.

The current Imp source does not prove that a registered runner received any particular queued job. A controlled unraid-lab run did prove one complete registration, assignment, and successful-completion path: GitHub assigned the job to the ephemeral Imp runner imp-runner-1788975945509078350 in omni-runner. See Demonstration Evidence for the workflow and job links, release digests, and the exact cleanup boundary.

6. The runner exits and the pool reconciles ​

The GitHub runner registration is ephemeral. The pool controller removes terminal runner VMs, while a successful VM may instead be retained as the pool's idle capacity. The agent observes the Firecracker process and handles the local stop path, including the VM socket files and networking resources.

A successful VM boot is therefore not the end of the story. The useful end state is: GitHub assigned a job, the guest completed it, the one-time setup disappeared, the ephemeral runner deregistered, and the pool returned to the required capacity. Depending on minIdle, the VM may remain as warm capacity instead of terminating.

Source map ​

  • Demand, caps, and VM creation: internal/controller/impvmrunnerpool_controller.go:92-184
  • Repository-only GitHub queue depth: internal/runner/github.go:175-187
  • Node choice: internal/controller/impvm_scheduler.go:102-246
  • Linux agent placement and VM start: charts/imp/templates/agent/daemonset.yaml:1-104, internal/agent/reconciler.go:91-267
  • Firecracker start: internal/agent/firecracker_driver.go:153-323, 473-530
  • Guest runner handoff: internal/agent/runnerlaunch/launcher.go:73-170

Related: From a Runner Pool to a Registered GitHub Runner and Demonstration Evidence.