Databricks

Integrate Databricks Model Serving with Portkey’s AI Gateway

Portkey provides a robust and secure gateway to integrate various Large Language Models (LLMs) into applications, including Databricks Model Serving endpoints.

With Portkey, take advantage of features like fast AI gateway access, observability, prompt management, and more, while securely managing API keys through Model Catalog.

Provider Slug: databricks

Quick Start

Get Databricks working in 3 steps:

Python

from portkey_ai import Portkey

# 1. Install: pip install portkey-ai
# 2. Add @databricks provider in model catalog
# 3. Use it:

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.chat.completions.create(
    model="@databricks/databricks-meta-llama-3-1-70b-instruct",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)

import Portkey from 'portkey-ai'

// 1. Install: npm install portkey-ai
// 2. Add @databricks provider in model catalog
// 3. Use it:

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.chat.completions.create({
    model: "@databricks/databricks-meta-llama-3-1-70b-instruct",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)

from openai import OpenAI
from portkey_ai import PORTKEY_GATEWAY_URL

# 1. Install: pip install openai portkey-ai
# 2. Add @databricks provider in model catalog
# 3. Use it:

client = OpenAI(
    api_key="PORTKEY_API_KEY",  # Portkey API key
    base_url=PORTKEY_GATEWAY_URL
)

response = client.chat.completions.create(
    model="@databricks/databricks-meta-llama-3-1-70b-instruct",
    messages=[{"role": "user", "content": "Say this is a test"}]
)

print(response.choices[0].message.content)

import OpenAI from "openai"
import { PORTKEY_GATEWAY_URL } from "portkey-ai"

// 1. Install: npm install openai portkey-ai
// 2. Add @databricks provider in model catalog
// 3. Use it:

const client = new OpenAI({
    apiKey: "PORTKEY_API_KEY",  // Portkey API key
    baseURL: PORTKEY_GATEWAY_URL
})

const response = await client.chat.completions.create({
    model: "@databricks/databricks-meta-llama-3-1-70b-instruct",
    messages: [{ role: "user", content: "Say this is a test" }]
})

console.log(response.choices[0].message.content)

# 1. Add @databricks provider in model catalog
# 2. Use it:

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@databricks/databricks-meta-llama-3-1-70b-instruct",
    "messages": [
      { "role": "user", "content": "Say this is a test" }
    ]
  }'

Tip: You can also set provider="@databricks" in Portkey() and use just model="databricks-meta-llama-3-1-70b-instruct" in the request.

Add Provider in Model Catalog

1. Go to Model Catalog → Add Provider
2. Select Databricks
3. Choose existing credentials or create new by entering your Databricks personal access token and workspace name
4. Name your provider (e.g., databricks-prod)

Configuration Parameters

Parameter	Description	Required
apiKey	Databricks personal access token	Yes
databricksWorkspace	Databricks workspace name (used to construct the URL: https://<workspace>.cloud.databricks.com)	Yes

When using headers directly, pass the workspace name via the x-portkey-databricks-workspace header.

Complete Setup Guide →

See all setup options, code examples, and detailed instructions

---

Supported Endpoints

Endpoint	Support
/chat/completions	Supported
/completions	Supported
/embeddings	Supported
/responses	Supported
/messages	Supported

Supported Features

Feature	Support
Thinking/Reasoning	Supported via thinking and reasoning_effort parameters
Streaming	Supported

---

Embeddings

Generate embeddings using Databricks-hosted embedding models:

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.embeddings.create(
    model="@databricks/databricks-bge-large-en",
    input="The quick brown fox jumps over the lazy dog"
)

print(response.data[0].embedding[:5])

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.embeddings.create({
    model: "@databricks/databricks-bge-large-en",
    input: "The quick brown fox jumps over the lazy dog"
})

console.log(response.data[0].embedding.slice(0, 5))

curl https://api.portkey.ai/v1/embeddings \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@databricks/databricks-bge-large-en",
    "input": "The quick brown fox jumps over the lazy dog"
  }'

---

Responses API

Use the OpenAI Responses API format with Databricks models:

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.responses.create(
    model="@databricks/databricks-meta-llama-3-1-70b-instruct",
    input="Tell me a three sentence bedtime story about a unicorn."
)

print(response)

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.responses.create({
    model: "@databricks/databricks-meta-llama-3-1-70b-instruct",
    input: "Tell me a three sentence bedtime story about a unicorn."
})

console.log(response)

curl https://api.portkey.ai/v1/responses \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@databricks/databricks-meta-llama-3-1-70b-instruct",
    "input": "Tell me a three sentence bedtime story about a unicorn."
  }'

---

Messages API

Use the Anthropic Messages API format with Databricks models:

Python

from portkey_ai import Portkey

portkey = Portkey(api_key="PORTKEY_API_KEY")

response = portkey.with_options(
    headers={"x-portkey-anthropic-version": "2023-06-01"}
).post(
    "/v1/messages",
    body={
        "model": "@databricks/databricks-meta-llama-3-1-70b-instruct",
        "max_tokens": 250,
        "messages": [{"role": "user", "content": "Say this is a test"}]
    },
    cast_to=object
)

print(response)

import Portkey from 'portkey-ai'

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY"
})

const response = await portkey.post("/v1/messages", {
    body: {
        model: "@databricks/databricks-meta-llama-3-1-70b-instruct",
        max_tokens: 250,
        messages: [{ role: "user", content: "Say this is a test" }]
    },
    headers: { "x-portkey-anthropic-version": "2023-06-01" }
})

console.log(response)

curl https://api.portkey.ai/v1/messages \
  -H "Content-Type: application/json" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -d '{
    "model": "@databricks/databricks-meta-llama-3-1-70b-instruct",
    "max_tokens": 250,
    "messages": [
      { "role": "user", "content": "Say this is a test" }
    ]
  }'

---

Supported Models

Databricks Model Serving supports a variety of foundation models and custom endpoints:

• Meta Llama Models: databricks-meta-llama-3-1-70b-instruct, databricks-meta-llama-3-1-405b-instruct
• DBRX: databricks-dbrx-instruct
• Embedding Models: databricks-bge-large-en, databricks-gte-large-en
• Custom Endpoints: Any model deployed as a Databricks serving endpoint

For the complete list of available models, refer to the Databricks Foundation Model APIs documentation.

---

Next Steps

Add Metadata

Add metadata to your Databricks requests

Gateway Configs

Add gateway configs to your Databricks requests

Tracing

Trace your Databricks requests

Fallbacks

Setup fallback strategies with Databricks

For complete SDK documentation:

SDK Reference

Complete Portkey SDK documentation