Prompting best practices

You talk to RunInfra in plain language. The agent can act on a first message that carries four things: what you are building, which model, what to optimize for, and how much traffic you expect.

Name the model exactly

Type @ to mention a model and the agent resolves it to a Hugging Face id. A full owner/repo id always resolves. A description makes the agent choose for you.

Optimize @Qwen2.5-7B-Instruct for a summarization API. Priority is cost.

If you do not know which model you want, say what the workload is and ask:

Which model would you recommend for a low-cost translation API at 200 requests per minute?

Say limits as numbers

Constraints need no special syntax. Write them the way you would say them out loud.

Keep P99 latency under 150ms, cost under $0.001 per request, and the quality gate must pass.

Each one becomes a hard filter. A candidate that violates it is dropped before ranking, so a cheap but too slow result never reaches the top of your list. “Fast” is a mood. “Under 150ms P99” is a constraint the optimizer can respect.

Change things by asking

You never have to start over. Each message refines the same pipeline, and the agent keeps the whole conversation in context.

You:   Optimize meta-llama/Llama-3.1-8B-Instruct for a support chatbot. Priority is latency.
Agent: plans the run, prices it, and waits for you to accept

You:   Make cost the priority instead, keep P99 under 300ms.
Agent: re-plans against the new priority

You:   Deploy the winner.

A weak message and its fix

Weak: The latency is bad.Strong: P99 latency is 340ms, I need it under 200ms. Try a smaller model or a faster GPU, whichever costs less.The strong version gives the measured value, the target value, and the moves you will accept. The agent takes one step forward instead of asking you three questions.

A vague message does not fail, it stalls. The agent keeps asking clarifying questions until it has the four elements, so putting them in the first message is what saves the round trips.

Next steps

The guided prompt

The composer writes the sentence for you. You fill the model slot.

What a run does

How candidates get filtered, measured, and ranked.

Reading your result

What a published speedup means, and when one is refused.

Lossless first

Why nothing is quantized unless you ask or the arithmetic demands it.

Start here

How optimization works

Prompting best practices

Name the model exactly

Say limits as numbers

Change things by asking

Next steps

The guided prompt

What a run does

Reading your result

Lossless first

​Name the model exactly

​Say limits as numbers

​Change things by asking

​Next steps

The guided prompt

What a run does

Reading your result

Lossless first

Name the model exactly

Say limits as numbers

Change things by asking

Next steps