Skip to content

ADR-0017: Automated Tool Documentation

Status

Accepted

Date: 2025-01-11

Context

MCP tools need comprehensive documentation for both human developers and AI agents. Documentation must be:

  • Accurate: Synchronized with implementation
  • Complete: Covers all tools and parameters
  • Discoverable: Easy to find and use
  • Multi-format: Serves different audiences (humans, agents, APIs)
  • Maintainable: Doesn't get out of sync with code

The challenge is balancing completeness with maintainability while serving multiple audiences.

Decision

We adopt a multi-layered documentation approach where code serves as the single source of truth, with automated extraction for discovery and reference documentation.

Documentation Layers

Layer 1: Python Docstrings (Primary Source of Truth)

Every tool has a comprehensive Google-style docstring:

@observe_tool
async def update_order_status(
    context: Context,
    order_id: int,
    status_code: str,
    comment: str | None = None,
    public: bool = False,
    email_customer: bool = True,
    email_additional: bool = True,
    confirm: bool = False,
) -> dict[str, Any]:
    """Change an order's status with user confirmation.

    🔴 HIGH-RISK OPERATION: Customer-visible state change. User confirmation required.

    Uses FastMCP elicitation to show a preview before applying the change.
    Call with ``confirm=False`` first to preview, then ``confirm=True`` to apply.

    **Workflow**:
    1. Validate the target status is a viable transition (``get_viable_statuses``).
    2. Call with ``confirm=False`` — returns a preview, no write.
    3. Call with ``confirm=True`` — elicits user approval, then writes.

    **Related Tools**:
    - ``get_viable_statuses`` — confirm target status is reachable
    - ``add_order_comment`` — record a comment without changing status

    **Related Resources**:
    - ``statuspro://statuses`` — look up status codes

    Args:
        context: FastMCP context for services and elicitation
        order_id: The StatusPro order id
        status_code: 8-char status code (e.g. ``"st000003"``)
        comment: Optional history comment
        public: Whether the comment is visible to the customer
        email_customer / email_additional: Which notifications to send
        confirm: Must be True to apply the change

    Returns:
        dict with ``confirmed``, ``success``, ``status_code`` fields

    Example:
        Request: {
            "order_id": 6110375248088,
            "status_code": "st000003",
            "comment": "Shipped via UPS",
            "email_customer": true,
            "confirm": false  # Preview mode
        }
        Returns: Preview showing the transition and notification plan
    """

Required Elements:

  • One-line summary
  • Risk indicator for destructive operations (🔴/🟡/🟢)
  • Workflow description
  • Related tools/resources
  • Full Args/Returns/Raises documentation
  • At least one example

Layer 2: Pydantic Field Descriptions

Model fields include rich descriptions that become part of the API schema:

class UpdateOrderStatusRequest(BaseModel):
    """Request to change an order's status."""

    order_id: int = Field(..., description="StatusPro order id")
    status_code: str = Field(..., description="8-char status code, e.g. 'st000003'")
    comment: str | None = Field(None, description="Optional history comment")
    public: bool = Field(False, description="Whether the comment is visible to the customer")
    email_customer: bool = Field(True, description="Send the customer a status email")
    email_additional: bool = Field(True, description="Send additional notification emails")
    confirm: bool = Field(
        False,
        description="If false, returns preview. If true, applies after user confirmation."
    )

Requirements:

  • Every field must have a description
  • Descriptions should include:
  • What the field represents
  • Expected format/values
  • Constraints or validation rules

Layer 3: Observability Decorators

The @observe_tool decorator automatically logs tool invocations, providing real-time documentation of usage patterns:

@observe_tool  # Logs: tool_invoked, tool_completed, tool_failed
@unpack_pydantic_params
async def my_tool(...):
    ...

Benefits:

  • Automatic timing/performance metrics
  • Error tracking
  • Usage pattern analysis
  • No manual logging code needed

Layer 4: Tool Metadata Generator

scripts/generate_tools_json.py auto-generates MCP tool metadata for discovery and Docker registry:

# Extracts:
# - Tool names from function definitions
# - Descriptions from first line of docstrings
# - Parameters from Pydantic models (via AST parsing)

$ python scripts/generate_tools_json.py -o tools.json --pretty

Output: tools.json for Docker MCP Registry submission

Layer 5: Template-Based Responses (Optional)

For tools with complex multi-format responses, externalized markdown templates provide consistent formatting:

from statuspro_mcp.templates import format_template

result = format_template(
    "order_verification_match",
    order_number="PO-2024-001",
    total_matches=15,
    ...
)

Templates live in statuspro_mcp_server/src/statuspro_mcp/templates/*.md

Documentation Standards

Docstring Style: Google Style (matches Python ecosystem conventions)

Field Descriptions: Complete sentences with punctuation

Examples: Real-world request/response pairs in docstrings

Cross-References: Link to related tools, resources, and ADRs

Consequences

Positive

  • Always Accurate: Code is documentation - can't get out of sync
  • Comprehensive: Multiple layers serve different needs
  • Discoverable: Tools are self-documenting via metadata
  • IDE Support: Docstrings and type hints provide rich IDE experience
  • Agent-Friendly: AI agents can understand tool capabilities from metadata
  • Maintainable: Single source of truth reduces duplication

Negative

  • Verbose Docstrings: Required detail makes functions longer
  • Upfront Work: Each tool requires comprehensive documentation
  • Template Maintenance: External templates need manual updates (if used)
  • Generator Maintenance: generate_tools_json.py must track code changes

Neutral

  • Convention Over Configuration: Strict standards reduce flexibility but increase consistency
  • Documentation Review: PRs require documentation review alongside code review
  • Template Opt-In: Templates are optional for tools with complex responses

Documentation Workflow

Adding a New Tool

  1. Write comprehensive docstring following Layer 1 standards
  2. Add field descriptions to all Pydantic models (Layer 2)
  3. Apply @observe_tool decorator (Layer 3)
  4. Run scripts/generate_tools_json.py to update metadata (Layer 4)
  5. (Optional) Create response template if needed (Layer 5)

Maintaining Documentation

  • Code Changes: Update docstrings and field descriptions
  • Examples: Keep example requests/responses current with API
  • Metadata: Regenerate tools.json before each release
  • Templates: Update templates when response format changes

Alternatives Considered

Alternative 1: External Documentation Only

Keep comprehensive docs in separate markdown files, minimal docstrings in code.

Why rejected:

  • Gets out of sync quickly
  • Harder to maintain
  • Reduces IDE support
  • Harder for agents to discover

Alternative 2: Minimal Docstrings

Brief one-line docstrings, detailed docs maintained separately.

Why rejected:

  • Reduces discoverability
  • IDE doesn't show full context
  • Agents can't understand tool capabilities
  • Requires context switching to understand tools

Alternative 3: Auto-Generated from OpenAPI

Generate all MCP documentation from OpenAPI spec.

Why rejected:

  • MCP tools don't map 1:1 to API endpoints
  • MCP tools provide higher-level workflows
  • Would lose tool-specific context and examples
  • OpenAPI spec doesn't capture elicitation patterns

Alternative 4: Template-First Approach

Require templates for all tool responses.

Why rejected:

  • Overkill for simple responses
  • More files to maintain
  • Templates needed only for complex multi-format responses
  • Pydantic models already provide structure

Implementation Status

Fully Implemented:

  • Layer 1: Comprehensive docstrings on all tools ✅
  • Layer 2: Field descriptions on all Pydantic models ✅
  • Layer 3: @observe_tool decorator on all tools ✅
  • Layer 4: scripts/generate_tools_json.py generator ✅

Deferred:

  • Layer 5 (response templates): not yet required — StatusPro tool responses are small and the default JSON surface is adequate.

Documentation Coverage:

  • 9 tools documented (5 read-only, 4 mutations)
  • All request/response models documented
  • Cross-references added to related tools/resources

Tools Documentation Examples

Read-Only Tool (list_orders):

  • 🟢 indicator (safe, no confirmation needed)
  • Example request/response
  • Related tools and resources
  • Performance characteristics

Destructive Tool (update_order_status):

  • 🔴 indicator (requires confirmation)
  • Elicitation workflow documented
  • Preview vs confirm modes explained
  • Safety patterns highlighted

Bulk Tool (bulk_update_order_status):

  • 🔴 indicator (up to 50 orders at once, 5/min rate limit)
  • 202 Accepted / async-queued response documented
  • Preview shows the full batch before confirmation

References