> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# LlamaIndex

> Use RunInfra as the LLM and embedding provider in any LlamaIndex pipeline.

LlamaIndex's `OpenAI` LLM class and `OpenAIEmbedding` class both accept a custom `api_base`. Point them at RunInfra.

## Install

```bash theme={"dark"}
pip install llama-index llama-index-llms-openai llama-index-embeddings-openai
```

## LLM

```python theme={"dark"}
import os
from llama_index.llms.openai import OpenAI

chat_model = os.environ.get("RUNINFRA_CHAT_MODEL")
if not chat_model:
    raise RuntimeError("Set RUNINFRA_CHAT_MODEL from GET /v1/models.")

llm = OpenAI(
    model=chat_model,
    api_base="https://api.runinfra.ai/v1",
    api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)

response = llm.complete("What is RunInfra?")
print(response.text)
```

## Embeddings

```python theme={"dark"}
import os
from llama_index.embeddings.openai import OpenAIEmbedding

embedding_model = os.environ.get("RUNINFRA_EMBEDDING_MODEL")
if not embedding_model:
    raise RuntimeError("Set RUNINFRA_EMBEDDING_MODEL from GET /v1/models.")

embed = OpenAIEmbedding(
    model=embedding_model,
    api_base="https://api.runinfra.ai/v1",
    api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)

vector = embed.get_text_embedding("Hello world")
```

Set `RUNINFRA_CHAT_MODEL` and `RUNINFRA_EMBEDDING_MODEL` to model IDs returned
by `GET /v1/models`.

## Full RAG example

```python theme={"dark"}
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings

Settings.llm = llm
Settings.embed_model = embed

documents = SimpleDirectoryReader("./docs").load_data()
index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine()
print(query_engine.query("How fast are RunInfra cold starts?"))
```

## Streaming

```python theme={"dark"}
for chunk in llm.stream_complete("Tell me a short story"):
    print(chunk.delta, end="", flush=True)
```

## Next steps

<Columns cols={2}>
  <Card title="LangChain" icon="link" href="/integrations/langchain">
    Same idea, different framework.
  </Card>

  <Card title="RAG cookbook" icon="database" href="/cookbook/rag">
    Raw RAG without a framework.
  </Card>

  <Card title="OpenAI compatibility" icon="plug" href="/tools-sdks/openai-compatibility">
    The contract powering this integration.
  </Card>

  <Card title="Embeddings API" icon="square-terminal" href="/api-reference/embeddings">
    Endpoint parameters and response shape.
  </Card>
</Columns>