> ## Documentation Index
> Fetch the complete documentation index at: https://turnwise.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Python SDK Overview

> Use TurnWise programmatically with Python

# Python SDK

The TurnWise Python SDK allows you to run evaluations programmatically using your own LLM API keys, while automatically syncing results to the TurnWise platform.

## Why Use the SDK?

<CardGroup cols={2}>
  <Card title="Use Your Own API Keys" icon="key">
    Run evaluations locally with your OpenRouter or other LLM provider keys
  </Card>

  <Card title="Programmatic Control" icon="code">
    Integrate evaluations into your CI/CD pipelines and automation workflows
  </Card>

  <Card title="Cost Control" icon="dollar-sign">
    Pay only for LLM API calls, not for TurnWise compute resources
  </Card>

  <Card title="Automatic Sync" icon="sync">
    Results automatically sync to TurnWise web UI for visualization
  </Card>
</CardGroup>

## Key Features

* **Local Evaluation**: Run evaluations on your machine using your API keys
* **Template Variables**: Use `@VARIABLE` syntax for context-aware prompts
* **Multiple Levels**: Evaluate at conversation, message, or step level
* **Structured Outputs**: Support for text, number, checkbox, progress, and custom JSON schemas
* **Batch Processing**: Evaluate multiple conversations concurrently
* **Goal Extraction**: Automatically extracts user goals from conversations
* **Progress Tracking**: Real-time progress callbacks during evaluation

## Installation

```bash theme={null}
pip install turnwise-sdk
```

## Quick Start

```python theme={null}
import asyncio
from turnwise import TurnWiseClient, Metric, EvaluationLevel, OutputType

async def main():
    # Initialize client
    client = TurnWiseClient(
        turnwise_api_key="tw_your_api_key",      # From TurnWise platform
        openrouter_api_key="sk-or-your_key"      # From OpenRouter
    )

    # Verify connection
    await client.verify()

    # Define a metric
    metric = Metric(
        name="Response Helpfulness",
        prompt="""Evaluate how helpful this response is.

USER GOAL: @GOAL
RESPONSE: @CURRENT_MESSAGE.output

Score from 0.0 to 1.0.""",
        evaluation_level=EvaluationLevel.MESSAGE,
        output_type=OutputType.PROGRESS,
    )

    # Run evaluation
    results = await client.evaluate(
        dataset_id=1,
        metric=metric,
        progress_callback=lambda p: print(f"Progress: {p.completed}/{p.total}")
    )

    print(f"Evaluated {len(results)} items")
    await client.close()

asyncio.run(main())
```

## Architecture

The SDK consists of several components:

* **TurnWiseClient**: Main client for API communication and high-level evaluation
* **EvaluationOrchestrator**: Lower-level orchestrator for manual control
* **OpenRouterProvider**: LLM provider for making API calls
* **GoalAgent**: Extracts user goals from conversations
* **EvaluationAgent**: Executes individual evaluations

## Next Steps

<CardGroup cols={2}>
  <Card title="Installation & Setup" href="/sdk/installation">
    Get started with installation and configuration
  </Card>

  <Card title="Basic Usage" href="/sdk/basic-usage">
    Learn how to run your first evaluation
  </Card>

  <Card title="Advanced Usage" href="/sdk/advanced-usage">
    Explore advanced features and customization
  </Card>

  <Card title="API Reference" href="/sdk/api-reference">
    Complete API documentation
  </Card>
</CardGroup>