> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# LangChain

> Use RunInfra as the LLM provider for any LangChain application. One-line config change.

LangChain's `ChatOpenAI` speaks the OpenAI REST API. Point it at RunInfra by overriding the base URL and key.

## Install

```bash theme={"dark"}
pip install langchain-openai
```

## Chat model

```python theme={"dark"}
import os
from langchain_openai import ChatOpenAI

chat_model = os.environ.get("RUNINFRA_CHAT_MODEL")
if not chat_model:
    raise RuntimeError("Set RUNINFRA_CHAT_MODEL from GET /v1/models.")

llm = ChatOpenAI(
    model=chat_model,
    openai_api_base="https://api.runinfra.ai/v1",
    openai_api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)

response = llm.invoke("What is RunInfra?")
print(response.content)
```

## Embeddings

```python theme={"dark"}
import os
from langchain_openai import OpenAIEmbeddings

embedding_model = os.environ.get("RUNINFRA_EMBEDDING_MODEL")
if not embedding_model:
    raise RuntimeError("Set RUNINFRA_EMBEDDING_MODEL from GET /v1/models.")

embeddings = OpenAIEmbeddings(
    model=embedding_model,
    openai_api_base="https://api.runinfra.ai/v1",
    openai_api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)

vectors = embeddings.embed_documents(["Hello", "World"])
```

## Streaming

```python theme={"dark"}
for chunk in llm.stream("Tell me a short story"):
    print(chunk.content, end="", flush=True)
```

## Tool calling via LangChain agents

```python theme={"dark"}
from langchain.agents import create_openai_tools_agent, AgentExecutor
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> str:
    """Get the current weather for a city."""
    return f"{city}: 21C, partly cloudy"

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful weather assistant."),
    ("human", "{input}"),
    ("placeholder", "{agent_scratchpad}"),
])

agent = create_openai_tools_agent(llm, [get_weather], prompt)
executor = AgentExecutor(agent=agent, tools=[get_weather])
result = executor.invoke({"input": "What's the weather in Paris?"})
print(result["output"])
```

## RAG with LangChain + RunInfra embeddings

```python theme={"dark"}
from langchain_community.vectorstores import FAISS
from langchain.chains import RetrievalQA

store = FAISS.from_texts(
    ["RunInfra cold starts under 2 seconds.", "RunInfra serves Llama, Qwen, Mistral..."],
    embeddings,
)
qa = RetrievalQA.from_chain_type(llm=llm, retriever=store.as_retriever(k=2))
print(qa.invoke({"query": "How fast are cold starts?"})["result"])
```

## Known gotchas

* Pass a model id returned by `GET /v1/models`. For multi-model pipelines, pass the alias you configured in chat.
* Streaming callbacks (`StreamingStdOutCallbackHandler`) work unchanged.
* LangChain retries use exponential backoff. Pair with `max_retries=3` and let the library handle 429s.

## Next steps

<Columns cols={2}>
  <Card title="LlamaIndex" icon="book-open" href="/integrations/llamaindex">
    Same OpenAI-base pattern for LlamaIndex.
  </Card>

  <Card title="OpenAI compatibility" icon="plug" href="/tools-sdks/openai-compatibility">
    The underlying contract.
  </Card>

  <Card title="RAG cookbook" icon="database" href="/cookbook/rag">
    Runnable end-to-end RAG example.
  </Card>

  <Card title="Tool calling cookbook" icon="wrench" href="/cookbook/tool-calling">
    Raw OpenAI tool loop (no framework).
  </Card>
</Columns>