Skip to main content

​
Optimization stack

TechniqueApplies whenBenefitAvailability
FP8 W8A8 dynamic precisionAda, Hopper, and Blackwell GPUsHigher image throughput when quality gates passAvailable on supported deployments
torch.compile reduce-overhead modeHopper or newer GPUs with large DiT modelsLower per-image latency on compatible pipelinesCore
Family-aware scheduler selectionFLUX, SD3, SDXL, and compatible familiesBetter speed and quality tradeoff than a generic schedulerAvailable on supported deployments
Step-distillation LoRANon-distilled FLUX-Dev, SDXL, and SD3 basesFewer denoising steps with quality guardrailsCore
TAESD and tiled VAEMemory-constrained image pipelinesLower memory pressure during decodeAvailable on supported deployments
Attention and feature-cache techniquesDiT workloads with compatible runtimesAdditional speedups after validationEvaluated when model and runtime support are available

​
Quality gates

MetricRequired result
FID deltaLess than 5.0 versus the baseline reference set
CLIP score deltaGreater than -0.02 versus the baseline
PickScoreGreater than 0.45
Image integrityOutput images parse successfully and match requested dimensions
Quality gates must pass before RunInfra treats a faster image-generation variant as deployable. If quality evidence is missing or fails, the dashboard keeps the safer baseline path. For backend lab evidence, image_semantic_alignment must come from an explicit measured semantic gate (qualityGate.measured=true). Numeric CLIP or semantic scores without that measured marker stay audit metadata and cannot support promotion.

​
How RunInfra applies image optimizations

DecisionWhat RunInfra checksRuntime effect
Dynamic precisionGPU family, model support, and quality resultsUses FP8 only when the measured candidate stays within quality limits
Compilation modeModel family, GPU support, and compile stabilityEnables reduce-overhead compilation only for compatible deployments
Scheduler selectionModel family and requested generation behaviorSelects a scheduler matched to FLUX, SD3, SDXL, or the detected family
Step distillationBase model family, plan tier, and quality gatesUses the validated step-distillation adapter for eligible non-distilled models
Decode memory controlsRequested image size and available GPU memoryApplies lower-memory decode paths when they are needed for reliability

​
Verification

  1. The optimization plan should explain which image techniques were accepted, skipped, or rejected.
  2. Benchmark results should include per-image latency, peak VRAM, and the relevant image-quality gates.
  3. Deployment settings should match the winning measured variant shown in the dashboard.
  4. The endpoint test should generate a valid image at the requested size before the deployment is treated as ready.

​
Rollback

Use customer overrides when you want a safer baseline or need to isolate a production issue: 
constraints.customerOverrides = {
  skipTechniques: ["compile"],
};

constraints.customerOverrides = {
  forceQuant: "none",
};

​
Availability notes

  • Image optimizations are shown as deployable only after the selected model, GPU, and runtime support the technique.
  • Step-distillation support depends on the base model family and the validated adapter for that family.
  • Multi-adapter image workloads may require a custom deployment review because not every runtime supports stacking multiple LoRA adapters efficiently.
  • Techniques under evaluation stay out of generated deployment snippets until RunInfra can carry their measured settings into the endpoint reliably.