Skip to main content
A deployment that never comes up used to give you one sentence for every possible reason. A registry problem, a full disk, a driver mismatch, an out-of-memory kill, and a broken start command all arrived as the same unactionable message, because nothing ever read the worker’s own log. Now they do not. When a worker fails to start, RunInfra captures a bounded tail of that worker’s system and container log and classifies the failure from it. A cause is named only when the log actually evidences it.

​
Causes RunInfra can prove

Each of these is claimed only when the captured worker log contains the matching evidence, and each carries the action that resolves it.
The out-of-memory case is the one you can most often fix yourself. Either move up a GPU tier, or ask for quantization explicitly, which is otherwise off by default. See Lossless first.

​
When the cause is not proven

If the control plane shows a worker exited but the captured log contains none of the signatures above, the failure stays generic:
This is deliberate. A worker exiting tells you it stopped, not why it stopped. It never establishes a registry, disk, runtime, entrypoint, or memory cause on its own, so RunInfra does not pick the most likely-sounding one and present it as fact. A generic message you can retry is more useful than a confident wrong diagnosis that sends you to fix the wrong thing.
A generic failure is not a lower-quality answer than a named one. It is the same standard applied honestly: the product names a cause when it has proof and declines to when it does not.

​
What is captured, and what is not

The log tail exists to explain a failure, and it is bounded so it can never become a liability of its own:
  • Bounded by wall-clock time, by bytes read, and by characters retained.
  • Redacted line by line as it is read, before any truncation happens, so a secret cannot survive by sitting across a cutoff.
  • Never allowed to mask the failure. A malformed or unreadable log is dropped, and you still get the deployment error that triggered the capture.
Technical detail stays behind Technical details in the Deploy tab, scrubbed of internal infrastructure names, and is there for support conversations rather than everyday reading.

​
Failures caught before the deploy

Some causes are not deployment failures at all, because they are caught earlier. A model your workspace cannot read is refused when you accept the plan, before any GPU starts and before you are charged for one. See Model access.