A Timeout Won't Stop a Worker from Exhausting Memory

Work period: September 5, 2026.

A command can use up memory long before it runs out of time. That became concrete during work on my local agent tooling in September, after a full-project type check triggered a confirmed out-of-memory event.

The response was to put resource limits around the work the agent launched. The useful question became very specific: if one worker runs out of memory, can the rest of the session keep functioning?

Find every execution path

There was already background-worker isolation in the environment. The incident exposed why that wasn’t enough coverage. A foreground command could still exhaust the host.

The implementation covered foreground terminal commands, background pipes, background terminals, and local Python execution kernels. Each path needed to launch its work inside a systemd user scope. Leaving one ordinary execution path outside the policy would leave a gap in the resource budget.

The agent itself stayed outside the worker slice. That separation matters when the worker fails: the process managing the work needs enough room to report the failure and accept the next command.

Give the group a budget too

A per-worker limit answers how much one worker may consume. It doesn’t answer how much several workers may consume together.

The local policy set both individual and aggregate limits for memory, swap, CPU, and process counts. It also used conservative thread defaults. Those defaults help avoid accidental oversubscription, but they remain advisory. The operating-system limits provide the enforced boundary for ordinary child processes that inherit the scope.

The exact numbers belong to the machine and workload. Copying someone else’s memory budget without checking your own environment can leave too little room for useful work or too little headroom for the host. The part worth carrying over is the two-level structure: budget each worker and budget their shared parent.

The policy also refused an unlimited launch when limits were explicitly enabled but the required systemd isolation wasn’t available. Quietly continuing without the requested boundary would have made the setting misleading.

Make the failure small enough to test

Verification used a deliberately small allocation test. A worker with a 128 MiB memory limit and no swap was asked to make a fixed 256 MiB allocation. That exercised the failure boundary without repeating the original full-project workload.

The checks covered containment and recovery. After the constrained allocation failed, the next command had to succeed. The refusal case also checked that unavailable isolation did not silently turn into an unlimited run.

Four focused tests using real systemd passed. The affected regression run recorded 145 passes, no failures, and four Windows-only skips. Installed-runtime checks then read the actual cgroup limits across the foreground, background, terminal, and Python-kernel paths. That last step mattered: passing helper tests alone wouldn’t prove every installed execution path was using the policy.

Keep the claim the right size

This work addressed resource exhaustion in a local workflow. It did not create a hostile-code sandbox. Code running as the same user could deliberately use that user’s service-manager access to escape those scopes. The limits were useful for containing ordinary worker workloads within the intended launcher paths.

The September record also left the cause of subsequent Windows catastrophic-failure messages unconfirmed. A verified fix for one failure mechanism doesn’t explain every nearby symptom.

The implementation was locally activated at that checkpoint. Later runtime updates need their own verification; this post doesn’t establish which patches are installed today.

The recovery check is the part I keep coming back to. When a worker hits its limit, I want a clear failure and a session that can still do something useful. “The process stopped” is only half that test.