Lambda Labs

Use Lambda’s GPU-powered inference for Llama and open-source models through Portkey.

Quick Start

Get started with Lambda Labs in under 2 minutes:

Python

from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @lambda provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@lambda/llama3.1-8b-instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @lambda provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@lambda/llama3.1-8b-instruct",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @lambda provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@lambda/llama3.1-8b-instruct",
    messages=[{"role": "user", "content": "Hello!"}]
)

print(response.choices[0].message.content)

import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @lambda provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@lambda/llama3.1-8b-instruct",
    messages: [{ role: "user", content: "Hello!" }]
})

console.log(response.choices[0].message.content)

# 1. Add @lambda provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@lambda/llama3.1-8b-instruct",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Add Provider in Model Catalog

Before making requests, add Lambda Labs to your Model Catalog:

1. Go to Model Catalog → Add Provider
2. Select Lambda Labs
3. Enter your Lambda API key
4. Name your provider (e.g., lambda)

Complete Setup Guide

See all setup options and detailed configuration instructions

---

Lambda Capabilities

Streaming

Stream responses for real-time output:

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@lambda")

stream = portkey.chat.completions.create(
    model="llama3.1-8b-instruct",
    messages=[{"role": "user", "content": "Tell me a story"}],
    stream=True
)

for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@lambda'
});

const stream = await portkey.chat.completions.create({
    model: 'llama3.1-8b-instruct',
    messages: [{ role: 'user', content: 'Tell me a story' }],
    stream: true
});

for await (const chunk of stream) {
    process.stdout.write(chunk.choices[0]?.delta?.content || '');
}

Function Calling

Use Lambda’s function calling capabilities:

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY", provider="@lambda")

tools = [{
    "type": "function",
    "function": {
        "name": "getWeather",
        "description": "Get the current weather",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City and state"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["location"]
        }
    }
}]

response = portkey.chat.completions.create(
    model="llama3.1-8b-instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What's the weather in Delhi?"}
    ],
    tools=tools,
    tool_choice="auto"
)

print(response.choices[0].message)

import Portkey from 'portkey-ai';

const portkey = new Portkey({
    apiKey: 'PORTKEY_API_KEY',
    provider: '@lambda'
});

const tools = [{
    type: "function",
    function: {
        name: "getWeather",
        description: "Get the current weather",
        parameters: {
            type: "object",
            properties: {
                location: { type: "string", description: "City and state" },
                unit: { type: "string", enum: ["celsius", "fahrenheit"] }
            },
            required: ["location"]
        }
    }
}];

const response = await portkey.chat.completions.create({
    model: "llama3.1-8b-instruct",
    messages: [
        { role: "system", content: "You are a helpful assistant." },
        { role: "user", content: "What's the weather in Delhi?" }
    ],
    tools,
    tool_choice: "auto"
});

console.log(response.choices[0].message);

---

Supported Endpoints and Parameters

Endpoint	Supported Parameters
/chat/completions	messages, max_tokens, temperature, top_p, stream, presence_penalty, frequency_penalty, tools, tool_choice
/completions	model, prompt, max_tokens, temperature, top_p, n, stream, logprobs, echo, stop, presence_penalty, frequency_penalty, best_of, logit_bias, user, seed, suffix

Check Lambda’s documentation for more details.

---

Next Steps

Gateway Configs

Add fallbacks, load balancing, and more

Observability

Monitor and trace your Lambda requests

Prompt Library

Manage and version your prompts

Metadata

Add custom metadata to requests

For complete SDK documentation:

SDK Reference

Complete Portkey SDK documentation