> ## Documentation Index
> Fetch the complete documentation index at: https://turnwise.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# API Reference

> Complete API documentation for the TurnWise Python SDK

# API Reference

## TurnWiseClient

Main client for interacting with TurnWise.

### Constructor

```python theme={null}
TurnWiseClient(
    turnwise_api_key: str,
    openrouter_api_key: str,
    turnwise_base_url: Optional[str] = None,
    default_model: str = "openai/gpt-4o-mini",
)
```

**Parameters:**

* `turnwise_api_key` (str): Your TurnWise API key
* `openrouter_api_key` (str): Your OpenRouter API key
* `turnwise_base_url` (Optional\[str]): Custom base URL (defaults to production)
* `default_model` (str): Default model for evaluations

### Methods

#### verify()

Verify API key and connection.

```python theme={null}
result = await client.verify()
# Returns: AuthVerifyResponse(valid=True, user_id="...")
```

#### list\_datasets()

List all datasets for the authenticated user.

```python theme={null}
datasets = await client.list_datasets()
# Returns: List[Dataset]
```

#### get\_conversations(dataset\_id)

Get all conversations for a dataset.

```python theme={null}
conversations = await client.get_conversations(dataset_id=1)
# Returns: List[Conversation]
```

#### get\_pipelines(dataset\_id)

Get all evaluation pipelines for a dataset.

```python theme={null}
pipelines = await client.get_pipelines(dataset_id=1)
# Returns: List[Pipeline]
```

#### register\_metric(dataset\_id, metric)

Register a metric in TurnWise.

```python theme={null}
result = await client.register_metric(
    dataset_id=1,
    metric=metric
)
# Returns: MetricCreateResponse(pipeline_id=1, node_id=1, name="...")
```

#### evaluate(dataset\_id, metric, ...)

Run evaluation on a dataset.

```python theme={null}
results = await client.evaluate(
    dataset_id: int,
    metric: Metric,
    max_concurrent: int = 3,
    auto_sync: bool = True,
    progress_callback: Optional[Callable] = None,
)
```

**Parameters:**

* `dataset_id` (int): Dataset ID to evaluate
* `metric` (Metric): Metric definition
* `max_concurrent` (int): Maximum concurrent evaluations
* `auto_sync` (bool): Automatically sync results to TurnWise
* `progress_callback` (Optional\[Callable]): Progress callback function

**Returns:** `List[EvaluationResult]`

#### sync\_results(results)

Manually sync evaluation results to TurnWise.

```python theme={null}
result = await client.sync_results(results)
# Returns: SyncResponse(synced_count=10, execution_id=1)
```

#### close()

Close the client and cleanup resources.

```python theme={null}
await client.close()
```

## Metric

Definition of an evaluation metric.

### Constructor

```python theme={null}
Metric(
    name: str,
    prompt: str,
    evaluation_level: EvaluationLevel,
    output_type: OutputType,
    description: Optional[str] = None,
    output_schema: Optional[Dict] = None,
    model_name: Optional[str] = None,
    aggregate_results: bool = False,
    node_id: Optional[int] = None,
    pipeline_id: Optional[int] = None,
)
```

**Parameters:**

* `name` (str): Metric name
* `prompt` (str): Evaluation prompt (supports `@VARIABLE` syntax)
* `evaluation_level` (EvaluationLevel): Level to evaluate at
* `output_type` (OutputType): Type of output
* `description` (Optional\[str]): Metric description
* `output_schema` (Optional\[Dict]): JSON schema for JSON output type
* `model_name` (Optional\[str]): Model to use (overrides default)
* `aggregate_results` (bool): Aggregate step results to message level
* `node_id` (Optional\[int]): Existing node ID (for reusing metrics)
* `pipeline_id` (Optional\[int]): Pipeline ID (for reusing metrics)

## EvaluationLevel

Enumeration of evaluation levels.

```python theme={null}
class EvaluationLevel(str, Enum):
    CONVERSATION = "conversation"
    MESSAGE = "message"
    STEP = "step"
```

## OutputType

Enumeration of output types.

```python theme={null}
class OutputType(str, Enum):
    TEXT = "text"
    NUMBER = "number"
    CHECKBOX = "checkbox"
    PROGRESS = "progress"
    JSON = "json"
```

## EvaluationResult

Result of an evaluation.

### Properties

* `entity_id` (int): ID of evaluated entity
* `entity_type` (str): Type of entity ("conversation", "message", "step")
* `result` (Any): Raw evaluation result
* `metadata` (Dict): Additional metadata

### Methods

#### get\_score()

Extract score from result.

```python theme={null}
score = result.get_score()
# Returns: Optional[float] (for PROGRESS output type)
```

## EvaluationOrchestrator

Lower-level orchestrator for manual evaluation control.

### Constructor

```python theme={null}
EvaluationOrchestrator(
    llm_provider: LLMProvider,
    default_model: str,
    extract_goals: bool = True,
)
```

**Parameters:**

* `llm_provider` (LLMProvider): LLM provider instance
* `default_model` (str): Default model name
* `extract_goals` (bool): Whether to extract goals from conversations

### Methods

#### evaluate\_conversation(conversation, metric)

Evaluate a single conversation.

```python theme={null}
results = await orchestrator.evaluate_conversation(
    conversation=conversation,
    metric=metric,
)
# Returns: List[EvaluationResult]
```

## OpenRouterProvider

LLM provider for OpenRouter API.

### Constructor

```python theme={null}
OpenRouterProvider(
    api_key: str,
    base_url: str = "https://openrouter.ai/api/v1",
)
```

## Models

### Dataset

```python theme={null}
class Dataset(BaseModel):
    id: int
    name: str
    description: Optional[str]
    conversation_count: int
```

### Conversation

```python theme={null}
class Conversation(BaseModel):
    id: int
    name: Optional[str]
    messages: List[Message]
    agents: Optional[List[Agent]]
```

### Message

```python theme={null}
class Message(BaseModel):
    id: int
    role: str
    output: Optional[str]
    steps: Optional[List[Step]]
```

### Step

```python theme={null}
class Step(BaseModel):
    id: int
    thinking: Optional[str]
    tool_call: Optional[Dict]
    tool_result: Optional[Dict]
    output_content: Optional[str]
    output_structured: Optional[Dict]
```

## Utility Functions

### setup\_logging()

Configure logging for the SDK.

```python theme={null}
from turnwise import setup_logging

setup_logging(level="INFO")
```

## Error Handling

### TurnWiseAPIError

Exception raised for TurnWise API errors.

```python theme={null}
try:
    await client.verify()
except TurnWiseAPIError as e:
    print(f"API Error: {e.status_code} - {e.message}")
```

## Next Steps

<CardGroup cols={2}>
  <Card title="Basic Usage" href="/sdk/basic-usage">
    Learn how to use the SDK
  </Card>

  <Card title="Advanced Usage" href="/sdk/advanced-usage">
    Explore advanced features
  </Card>
</CardGroup>