🤖 Databricks MCP Server Template

Host Model Context Protocol (MCP) prompts and tools on Databricks Apps, enabling AI assistants like Claude to interact with your Databricks workspace through a secure, authenticated interface.

What is this?

This template lets you create an MCP server that runs on Databricks Apps. You can:

📝 Add prompts as simple markdown files in the prompts/ folder
🛠️ Create tools as Python functions that leverage Databricks SDK
🔐 Authenticate securely with OAuth through Databricks Apps
🚀 Deploy instantly to make your MCP server accessible to Claude

Think of it as a bridge between Claude and your Databricks workspace - you define what Claude can see and do, and this server handles the rest.

How it Works

Architecture Overview

┌─────────────┐       MCP Protocol      ┌──────────────────┐        OAuth         ┌─────────────────┐
│   Claude    │ ◄─────────────────────► │  dba-mcp-proxy   │ ◄──────────────────► │ Databricks App  │
│    CLI      │     (stdio/JSON-RPC)    │ (local process)  │    (HTTPS/SSE)      │  (MCP Server)   │
└─────────────┘                         └──────────────────┘                      └─────────────────┘
                                                ▲                                           │
                                                │                                           ▼
                                                └────────── Databricks OAuth ──────► Workspace APIs

Components

MCP Server (server/app.py): A FastAPI app with integrated MCP server that:
- Dynamically loads prompts from prompts/*.md files
- Exposes Python functions as MCP tools via @mcp_server.tool decorator
- Handles both HTTP requests and MCP protocol over Server-Sent Events
Prompts (prompts/): Simple markdown files where:
- Filename = prompt name (e.g., check_system.md → check_system prompt)
- First line with # = description
- File content = what gets returned to Claude
Local Proxy (dba_mcp_proxy/): Authenticates and proxies MCP requests:
- Handles Databricks OAuth authentication automatically
- Translates between Claude's stdio protocol and HTTP/SSE
- Works with both local development and deployed apps

🎬 Demo

This 10-minute video shows you how to set up and use a Databricks MCP server with Claude: https://www.youtube.com/watch?v=oKE59zgb6e0

This video demonstrates creating your own MCP server with a custom jobs interface in Claude.

Quick Start

Create Your Own MCP Server

Step 1: Use this template

Or use the GitHub CLI:

gh repo create my-mcp-server --template databricks-solutions/custom-mcp-databricks-app --private

Step 2: Clone and setup

# Clone your new repository
git clone https://github.com/YOUR-USERNAME/my-mcp-server.git
cd my-mcp-server

# Run the interactive setup
./setup.sh

This will:

Configure Databricks authentication
Set your MCP server name
Install all dependencies
Create your .env.local file

Step 3: Deploy with Claude

In Claude Code, run:

/setup-mcp

This will:

Deploy your MCP server to Databricks Apps
Configure the MCP integration
Show you available prompts and tools

Then restart Claude Code to use your new MCP server.

Add to Claude CLI

After deployment, add your MCP server to Claude:

# Set your Databricks configuration
export DATABRICKS_HOST="https://your-workspace.cloud.databricks.com"
export DATABRICKS_APP_URL="https://your-app.databricksapps.com"  # Get this from ./app_status.sh
export SERVER_NAME="your-server-name"  # This comes from config.yaml (set during ./setup.sh)

# Add your MCP server to Claude (user-scoped)
claude mcp add $SERVER_NAME --scope user -- \
  uvx --refresh --from git+ssh://[email protected]/YOUR-USERNAME/your-repo.git dba-mcp-proxy \
  --databricks-host $DATABRICKS_HOST \
  --databricks-app-url $DATABRICKS_APP_URL

Local Development

# Clone and setup
git clone <your-repo>
cd <your-repo>
./setup.sh

# Start dev server
./watch.sh

# Set your configuration for local testing
export DATABRICKS_HOST="https://your-workspace.cloud.databricks.com"
export DATABRICKS_APP_URL="http://localhost:8000"  # Local dev server

# Add to Claude for local testing
claude mcp add databricks-mcp-local --scope local -- \
  uvx --refresh --from git+ssh://[email protected]/YOUR-ORG/YOUR-REPO.git dba-mcp-proxy \
  --databricks-host $DATABRICKS_HOST \
  --databricks-app-url $DATABRICKS_APP_URL

Customization Guide

This template uses FastMCP, a framework that makes it easy to build MCP servers. FastMCP provides two main decorators for extending functionality:

@mcp_server.prompt - For registering prompts that return text
@mcp_server.tool - For registering tools that execute functions

Adding Prompts

The easiest way is to create a markdown file in the prompts/ directory:

# Get cluster information

List all available clusters in the workspace with their current status

The prompt will be automatically loaded with:

Name: filename without extension (e.g., get_clusters.md → get_clusters)
Description: first line after #
Content: entire file content

Alternatively, you can register prompts as functions in server/app.py:

@mcp_server.prompt(name="dynamic_status", description="Get dynamic system status")
async def get_dynamic_status():
    # This can include dynamic logic, API calls, etc.
    w = get_workspace_client()
    current_user = w.current_user.me()
    return f"Current user: {current_user.display_name}\nWorkspace: {DATABRICKS_HOST}"

We auto-load prompts/ for convenience, but function-based prompts are useful when you need dynamic content.

Adding Tools

Add a function in server/app.py using the @mcp_server.tool decorator:

@mcp_server.tool
def list_clusters(status: str = "RUNNING") -> dict:
    """List Databricks clusters by status."""
    w = get_workspace_client()
    clusters = []
    for cluster in w.clusters.list():
        if cluster.state.name == status:
            clusters.append({
                "id": cluster.cluster_id,
                "name": cluster.cluster_name,
                "state": cluster.state.name
            })
    return {"clusters": clusters}

Tools must:

Use the @mcp_server.tool decorator
Have a docstring (becomes the tool description)
Return JSON-serializable data (dict, list, str, etc.)
Accept only JSON-serializable parameters

Deployment

# Deploy to Databricks Apps
./deploy.sh

# Check status and get your app URL
./app_status.sh

Your MCP server will be available at https://your-app.databricksapps.com/mcp/

The app_status.sh script will show your deployed app URL, which you'll need for the DATABRICKS_APP_URL environment variable when adding the MCP server to Claude.

Authentication

Local Development: No authentication required
Production: OAuth is handled automatically by the proxy using your Databricks CLI credentials

Examples

Using with Claude

Once added, you can interact with your MCP server in Claude:

Human: What prompts are available?

Claude: I can see the following prompts from your Databricks MCP server:
- check_system: Get system information
- list_files: List files in the current directory
- ping_google: Check network connectivity

Sample Tool Usage

Human: Can you execute a SQL query to show databases?

Claude: I'll execute that SQL query for you using the execute_dbsql tool.

[Executes SQL and returns results]

Project Structure

├── server/                    # FastAPI backend with MCP server
│   ├── app.py                # Main application + MCP tools
│   └── routers/              # API endpoints
├── prompts/                  # MCP prompts (markdown files)
│   ├── check_system.md      
│   ├── list_files.md        
│   └── ping_google.md       
├── dba_mcp_proxy/           # MCP proxy for Claude CLI
│   └── mcp_client.py        # OAuth + proxy implementation
├── client/                  # React frontend (optional)
├── scripts/                 # Development tools
└── pyproject.toml          # Python package configuration

Advanced Usage

Environment Variables

Configure in .env.local:

DATABRICKS_HOST=https://your-workspace.cloud.databricks.com
DATABRICKS_TOKEN=your-token  # For local development
DATABRICKS_SQL_WAREHOUSE_ID=your-warehouse-id  # For SQL tools

Creating Complex Tools

Tools can access the full Databricks SDK:

@mcp_server.tool
def create_job(name: str, notebook_path: str, cluster_id: str) -> dict:
    """Create a Databricks job."""
    w = get_workspace_client()
    job = w.jobs.create(
        name=name,
        tasks=[{
            "task_key": "main",
            "notebook_task": {"notebook_path": notebook_path},
            "existing_cluster_id": cluster_id
        }]
    )
    return {"job_id": job.job_id, "run_now_url": f"{DATABRICKS_HOST}/#job/{job.job_id}"}

Testing Your MCP Server

This template includes comprehensive testing tools for validating MCP functionality at multiple levels.

Quick Verification

After adding the MCP server to Claude, verify it's working:

# List available prompts and tools
echo "What MCP prompts are available from databricks-mcp?" | claude

# Test a specific prompt
echo "Use the check_system prompt from databricks-mcp" | claude

Comprehensive Testing Suite

The claude_scripts/ directory contains 6 testing tools for thorough MCP validation:

Command Line Tests

# Test local MCP server (requires ./watch.sh to be running)
./claude_scripts/test_local_mcp_curl.sh      # Direct HTTP/curl tests with session handling
./claude_scripts/test_local_mcp_proxy.sh     # MCP proxy client tests

# Test remote MCP server (requires Databricks auth and deployment)
./claude_scripts/test_remote_mcp_curl.sh     # OAuth + HTTP tests with dynamic URL discovery
./claude_scripts/test_remote_mcp_proxy.sh    # Full end-to-end MCP proxy tests

Interactive Web UI Tests

# Launch MCP Inspector for visual testing (requires ./watch.sh for local)
./claude_scripts/inspect_local_mcp.sh        # Local server web interface
./claude_scripts/inspect_remote_mcp.sh       # Remote server web interface

MCP Inspector Features:

🖥️ Web-based interface for interactive MCP server testing
🔧 Visual tool execution with parameter input forms
📊 Real-time request/response monitoring
🐛 Protocol-level debugging and error inspection
📋 Complete tool and resource discovery

What Each Test Validates

Test Type	Authentication	Protocol	Session Management	Tool Discovery
curl tests	✅	✅	✅	✅
proxy tests	✅	✅	✅	✅
MCP Inspector	✅	✅	✅	✅

All tests dynamically discover app URLs and handle OAuth authentication automatically.

See claude_scripts/README.md for detailed documentation.

Troubleshooting

Authentication errors: Run databricks auth login to refresh credentials
MCP not found: Ensure the app is deployed and accessible
Tool errors: Check logs at https://your-app.databricksapps.com/logz
MCP connection issues:
- Check Claude logs: tail -f ~/Library/Logs/Claude/*.log
- Verify the proxy works: uvx --refresh --from git+ssh://... dba-mcp-proxy --help
- Test with echo pipe: echo "list your mcp commands" | claude

Cached version issues: If you get errors about missing arguments after an update:

# Clear uvx cache for this package
rm -rf ~/.cache/uv/git-v0/checkouts/*/
# Or clear entire uv cache
uv cache clean

Contributing

Fork the repository
Add your prompts and tools
Test locally with ./watch.sh
Submit a pull request

License

See LICENSE.md