3D MCP

Overview

3D-MCP is a universal implementation of the Model Context Protocol for 3D software. It creates a unified TypeScript interface for LLMs to interact with Blender, Maya, Unreal Engine, and other 3D applications through a single coherent API.

// LLMs use the same interface regardless of underlying 3D software
await tools.animation.createKeyframe({
  objectId: "cube_1",
  property: "rotation.x",
  time: 30,
  value: Math.PI/2
});

Core Philosophy & Design Decisions

3D-MCP is built on four interconnected architectural principles that together create a unified system for 3D content creation:

1. Entity-First Design: Well-defined domain entities form the foundation of all operations, enabling consistent data modeling across platforms
2. Type-Safe CRUD Operations: Automated generation of create, read, update, delete operations with complete type validation
3. Atomic Operation Layer: A minimal set of platform-specific implementations that handle fundamental operations
4. Composable Tool Architecture: Complex functionality built by combining atomic operations in a platform-agnostic way

This architecture creates a dependency inversion where platform-specific implementation details are isolated to atomic operations, while the majority of the codebase remains platform-independent.

┌─────────────────────────────────────────────────────────────────────────┐
│                             LLM / User API                              │
└───────────────────────────────────┬─────────────────────────────────────┘
                                    │ MCP Tool API
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                          Compound Operations                            │
│                                                                         │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────────────┐  │
│  │ Modeling Tools  │  │ Animation Tools │  │ Rigging Tools           │  │
│  └─────────────────┘  └─────────────────┘  └─────────────────────────┘  │
└───────────────────────────────────┬─────────────────────────────────────┘
                                    │ Implemented by
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                           Atomic Operations                             │
│                                                                         │
│  ┌─────────── Entity CRUD ────────────┐ ┌────────── Non-CRUD ─────────┐ │ 
│  │ create{Entity}s update{Entity}s ...│ │  select, undo, redo, etc.   │ │  
│  └────────────────────────────────────┘ └─────────────────────────────┘ │
└───────────────────────────────────┬─────────────────────────────────────┘
                                    │ Plug-in Server Request
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                       Platform-Specific Adapters                        │
│                                                                         │
│  ┌──── Blender ────┐  ┌────── Maya ─────┐  ┌─── Unreal Engine ────┐     │
│  │ createKeyframes │  │ createKeyframes │  │ createKeyframes      │     │    
│  └─────────────────┘  └─────────────────┘  └──────────────────────┘     │
└─────────────────────────────────────────────────────────────────────────┘

Why These Design Decisions?

Entity-First Design was chosen because:

• 3D applications use different object models but share core concepts (meshes, materials, animations)
• Zod schemas provide single-source-of-truth for validation, typing, and documentation
• Strong typing catches errors at compile time rather than runtime
• Rich metadata enables better AI understanding of domain objects

CRUD Operations as Foundation because:

• They map cleanly to what 3D applications need to do with entities
• Standardized patterns reduce cognitive overhead
• Auto-generation eliminates repetitive code using createCrudOperations
• Every entity automatically gets the same consistent interface

Atomic and Compound Tool Separation because:

• Only atomic tools need platform-specific implementation (~20% of codebase)
• Compound tools work across all platforms without modification (~80% of codebase)
• New platforms only need to implement atomic operations to gain all functionality
• Maintainable architecture with clear separation of concerns

Technical Architecture

1. Entity-Centric CRUD Architecture

The system's foundation is a rich type system of domain entities that generates CRUD operations:

// Define entities with rich metadata using Zod
export const Mesh = NodeBase.extend({
  vertices: z.array(Tensor.VEC3).describe("Array of vertex positions [x, y, z]"),
  normals: z.array(Tensor.VEC3).optional().describe("Array of normal vectors"),
  // ... other properties
});

// CRUD operations generated automatically from entity schemas
const entityCruds = createCrudOperations(ModelEntities);
// => Creates createMeshs, getMeshs, updateMeshs, deleteMeshs, listMeshs

// All operations preserve complete type information
await tool.createRigControls.execute({
  name: "arm_ctrl",          
  shape: "cube",            // TypeScript error if not a valid enum value
  targetJointIds: ["joint1"], // Must be string array
  color: [0.2, 0.4, 1],     // Must match Color schema format
  // IDE autocomplete shows all required/optional fields
});

Entity schemas provide:

• Schema Validation: Runtime parameter checking with detailed error messages
• Type Information: Complete TypeScript types for IDE assistance
• Documentation: Self-documenting API with descriptions
• Code Generation: Templates for platform-specific implementations

Entity Architecture Diagram

┌──────────────────────────────────────────────────────────────┐
│                Core Entity Definitions                       │
│                                                              │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐   │
│  │ BaseEntity  │  │ NodeBase    │  │ Other Core Entities │   │
│  └─────────────┘  └─────────────┘  └─────────────────────┘   │
└──────────────────────────────────────────────────────────────┘
                           ▲
                           │ extends
                           │
┌──────────────────────────────────────────────────────────────┐
│                Domain-Specific Entities                      │
│                                                              │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐   │
│  │ Model       │  │ Animation   │  │ Rigging             │   │
│  │ Entities    │  │ Entities    │  │ Entities            │   │
│  └─────────────┘  └─────────────┘  └─────────────────────┘   │
└──────────────────────────────────────────────────────────────┘
                           │
                           │ input to
                           ▼
┌──────────────────────────────────────────────────────────────┐
│                Automatic CRUD Generation                     │
│                                                              │
│  ┌─────────────────────────────────────────────────────────┐ │
│  │ createCrudOperations(Entities)                          │ │
│  └─────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘
                           │
                           │ generates
                           ▼
┌──────────────────────────────────────────────────────────────┐
│                     Atomic Operations                        │
│                                                              │
│  ┌─────────────────┐ ┌──────────────┐ ┌─────────────────┐    │
│  │ create{Entity}s │ │ get{Entity}s │ │ update{Entity}s │ .. │
│  └─────────────────┘ └──────────────┘ └─────────────────┘    │
└──────────────────────────────────────────────────────────────┘
                           │
                           │ foundation for
                           ▼
┌──────────────────────────────────────────────────────────────┐
│                    Compound Operations                       │
│                                                              │
│  ┌─────────────────────────────────────────────────────────┐ │
│  │ No need for platform-specific code. Use atomic ops only.│ │
│  └─────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘

2. Compound Tool Architecture

The system creates a clear separation between atomic and compound operations:

// From compounded.ts - Higher level operations composed from atomic operations
createIKFKSwitch: defineCompoundTool({
  // ...parameter and return definitions...
  execute: async (params) => {
    // Create IK chain using atomic operations
    const ikChainResult = await tool.createIKChains.execute({/*...*/});
    
    // Create control with full type-checking
    const ikControlResult = await tool.createRigControls.execute({
      name: `${switchName}_IK_CTRL`,
      shape: ikControlShape,  // Type-checked against schema
      targetJointIds: [jointIds[jointIds.length - 1]],
      color: ikColor,
      // ...other parameters
    });
    
    // Position the control at the end effector
    await tool.batchTransform.execute({/*...*/});
    
    // Create constraints to connect the system
    await tool.createConstraint.execute({/*...*/});
    
    // Return standardized response with created IDs
    return {
      success: true,
      switchControlId: switchControlResult.id,
      ikControlId: ikControlResult.id,
      fkControlIds,
      poleVectorId: poleVectorId || undefined,
    };
  }
})

This architecture provides several technical advantages:

1. Atomic Operations (~20% of the system):

   • Directly interact with platform APIs
   • Need platform-specific implementations
   • Focus on single entity operations (create, read, update, delete)
   • Form minimal implementation required for new platforms

2. Compound Operations (~80% of the system):

   • Built entirely from atomic operations
   • Zero platform-specific code
   • Implement higher-level domain concepts
   • Work on any platform without modification

Tool Composition Flow

┌─────────────────────────────────────────────────────────────────────────┐
│                        High-Level Tool Definition                       │
└──────────────────────────────────────┬──────────────────────────────────┘
                                       │
                                       ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                         Compound Tool Pattern                           │
│                                                                         │
│  ┌──────────────────────────────────────────────────────────────────┐   │
│  │ defineCompoundTool({                                             │   │
│  │   description: string,                                           │   │
│  │   parameters: zod.Schema,                                        │   │
│  │   returns: zod.Schema,                                           │   │
│  │   execute: async (params) => {                                   │   │
│  │     // Composed entirely from atomic operations                  │   │
│  │     await tool.atomicOperation1.execute({...});                  │   │
│  │     await tool.atomicOperation2.execute({...});                  │   │
│  │     return { success: true, ...results };                        │   │
│  │   }                                                              │   │
│  │ })                                                               │   │
│  └──────────────────────────────────────────────────────────────────┘   │
└───────────────────────────────────┬─────────────────────────────────────┘
                                    │ Plug-in Server Request
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                         Platform Adaptation                             │
│                                                                         │
│  ┌──────────────────────────┐  ┌─────────────────────────────────────┐  │
│  │ Blender Implementation   │  │ Maya Implementation                 │  │
│  │ of Atomic Operations     │  │ of Atomic Operations                │  │
│  └──────────────────────────┘  └─────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────────┘

Key files for the compound tool architecture:

• compounded.ts: Compound modeling tools
• compounded.ts: Compound animation tools
• compounded.ts: Compound rigging tools

3. Code Generation Pipeline

The system automatically generates platform-specific implementations from TypeScript definitions:

┌─────────────────┐     ┌────────────────────┐     ┌─────────────────────────┐
│ Entity Schemas  │     │ Schema             │     │ Platform-Specific Code  │
│ & Tools (TS)    │ ──> │ Extraction (TS)    │ ──> │ (Python/C++/etc.)       │
└─────────────────┘     └────────────────────┘     └─────────────────────────┘
      │                        │                             │
      │                        │                             │
      ▼                        ▼                             ▼
┌─────────────────┐     ┌────────────────────┐     ┌─────────────────────────┐
│ Type            │     │ Parameter          │     │ Implementation          │
│ Definitions     │     │ Validation         │     │ Templates               │
└─────────────────┘     └────────────────────┘     └─────────────────────────┘

Key aspects of the generation system:

• Entity Extraction: Analyzes Zod schemas to understand entity structure
• Parameter Mapping: Converts TypeScript types to platform-native types
• Validation Generation: Creates parameter validation in target languages
• Implementation Templates: Provides platform-specific code patterns

The codegen system is implemented in:

• plugin-codegen.ts: Main code generation script
• extract-schemas.ts: Extracts Zod schemas from TypeScript files into temporary JSON files.

4. Domain Organization

The system is organized into domains that mirror 3D content creation workflows:

• Core: Base entities and operations used across all domains
• Modeling: Mesh creation, editing, and topology operations
• Animation: Keyframes, curves, clips, and animation control
• Rigging: Skeletal systems, controls, and deformation
• Rendering: Materials, lights, and render settings

Each domain follows the same organizational pattern:

• entity.ts: Domain-specific entity definitions
• atomic.ts: Atomic operations for domain entities
• compounded.ts: Higher-level operations built from atomic tools

Domain Structure Diagram

packages/src/tool/
│
├── core/                  # Core shared components
│   ├── entity.ts          # Base entities all domains use
│   ├── utils.ts           # Shared utilities including CRUD generation
│   └── ...
│
├── model/                 # Modeling domain
│   ├── entity.ts          # Mesh, Vertex, Face, etc.
│   ├── atomic.ts          # Atomic modeling operations
│   ├── compounded.ts      # Higher-level modeling tools
│   └── ...
│
├── animation/             # Animation domain
│   ├── entity.ts          # Keyframe, AnimCurve, Clip, etc.
│   ├── atomic.ts          # Atomic animation operations
│   ├── compounded.ts      # Higher-level animation tools
│   └── ...
│
├── rig/                   # Rigging domain
│   ├── entity.ts          # Joint, IKChain, Control, etc.
│   ├── atomic.ts          # Atomic rigging operations
│   ├── compounded.ts      # Higher-level rigging tools
│   └── ...
│
└── rendering/             # Rendering domain
    ├── entity.ts          # Camera, Light, RenderSettings, etc.
    ├── atomic.ts          # Atomic rendering operations
    ├── compounded.ts      # Higher-level rendering tools
    └── ...

5. Entity-Centric CRUD Architecture

The system implements a sophisticated entity-centric approach where:

1. Entities as Domain Models: Each domain (modeling, animation, rigging) defines its core entities that represent its fundamental concepts. These are implemented as Zod schemas with rich type information.

2. CRUD as Foundation: Every entity automatically receives a complete set of CRUD operations (Create, Read, Update, Delete) through the createCrudOperations utility:

// Each domain starts with CRUD operations for all its entities
const entityCruds = createCrudOperations(ModelEntities);

const modelAtomicTools = {
  ...entityCruds,  // Foundation of all atomic tools
  // Domain-specific operations build on this foundation
}

1. Entity Reuse and Inheritance: Core entities defined in core/entity.ts are extended by domain-specific entities, promoting code reuse and consistent design across domains.

2. DDD-Inspired Architecture: The system follows Domain-Driven Design principles by organizing code around domain entities and aggregates rather than technical concerns.

This architecture provides several key benefits:

• Consistency: All entities have the same patterns for basic operations
• Reduced Boilerplate: CRUD operations are generated automatically
• Clear Organization: Tools are organized around domain entities
• Separation of Concerns: Each domain manages its own entities while sharing common patterns

The combination of rich entity models with automatic CRUD operations creates a robust foundation that simplifies development while maintaining flexibility for domain-specific operations.

Getting Started

# Install dependencies
bun install

# Run the server
bun run index.ts

# Extract schemas and generate plugins
bun run packages/scripts/plugin-codegen.ts

Development Workflow

1. Define Entities: Create or extend entity schemas in src/tool/<domain>/entity.ts
2. Generate CRUD: Use createCrudOperations to generate atomic operations
3. Create Compound Tools: Build higher-level operations from atomic tools
4. Generate Plugins: Run the code generator to create platform-specific implementations

Contributing

The architectural decisions in 3D-MCP make it uniquely extensible:

1. Add New Entities: Define new entities and automatically get CRUD operations
2. Add New Compound Tools: Combine existing atomic operations to create new functionality
3. Add New Platforms: Implement the atomic tool interfaces in a new plugin

See our contributing guide for more details on how to contribute.

---

3D-MCP: One API to rule all 3D software