> ## Documentation Index
> Fetch the complete documentation index at: https://docs.apus.network/llms.txt
> Use this file to discover all available pages before exploring further.

# DCV Inference Architecture

> Understanding the APUS DCV Inference Service architecture

This document outlines the architecture and technical implementation of the **DCV (Deterministic, Confidential, Verification)** inference module for HyperBEAM. It focuses on the system design, the integration of the inference backend, and the mechanism for GPU TEE attestation.

## Architecture Overview

The DCV inference system enables HyperBEAM nodes to provide OpenAI-compatible AI inference services while cryptographically proving that the execution occurred on a verified NVIDIA GPU within a Trusted Execution Environment (TEE).

### System Components

1. **HyperBEAM Node (Erlang)**: The core node software.
   * `dev_inference`: Orchestrates the request flow, manages the backend process, and handles attestation logic.
   * `dev_sev_gpu`: Interacts with the NVIDIA GPU TEE to generate and verify attestation tokens.
   * `hb_http`: Enhanced with Server-Sent Events (SSE) support for streaming responses.
2. **Inference Backend (Python)**: A lightweight proxy wrapper around **SGLang**, a high-performance inference framework.
3. **Hardware**: NVIDIA GPUs with TEE support (e.g., H100).

### High-Level Data Flow

```mermaid theme={null}
graph TD
    Client[Client / User] -->|HTTP POST /v1/chat/completions| Node[HyperBEAM Node]

    subgraph "HyperBEAM Node"
        Node -->|Forward Request| Proxy[Inference Device]
        Node -->|Request Attestation| TEE[GPU TEE Module]
    end

    subgraph "Inference Backend"
        Proxy -->|OpenAI-Compatible API| SGLang[Deterministic Backend]
        SGLang -->|SGLang API| Engine[SGLang Engine]
        Engine -->|CUDA| GPU[NVIDIA GPU]
    end

    Engine -->|Tokens| Proxy
    Proxy -->|Stream| Node
    TEE -->|Attestation Token| Node
    Node -->|Response + Proof| Client
```

## Key Concepts

### Server-Sent Events (SSE) for Streaming

To support the interactive nature of LLMs, we implemented **Server-Sent Events (SSE)** within the `hb_http` module. This allows the node to stream generated tokens to the client in real-time, rather than waiting for the full generation to complete.

* **Implementation**: The Erlang HTTP server uses chunked transfer encoding to push data frames (`data: ...`) to the client as they are received from the backend.
* **Protocol**: Follows the standard EventStream format, compatible with standard OpenAI client libraries.

### GPU TEE Attestation

The "Confidential" and "Verification" aspects of DCV are achieved through TEE attestation. This ensures that the inference result was generated by a specific, trusted hardware environment and has not been tampered with.

#### Attestation Flow

The following sequence describes how a verifiable inference request is processed:

```mermaid theme={null}
sequenceDiagram
    participant Client
    participant Node as HyperBEAM Node
    participant Backend as Inference Backend
    participant TEE as GPU TEE Module

    Client->>Node: POST /v1/chat/completions (tee: true)
    Node->>Backend: Forward Request
    Backend->>Node: Return Response (Complete or Stream)

    Note over Node: Calculate Hash(Request + Response + Timestamp)

    Node->>TEE: Generate Attestation(Nonce=Hash)
    TEE-->>Node: Return Attestation Token (Signed Quote)

    Node->>Client: Return Response + Attestation Token
```

## API Compatibility

The inference module exposes an API compatible with the OpenAI Chat Completions API. This allows developers to use existing SDKs and tools with minimal changes.

### Supported Endpoints

* `POST /v1/chat/completions`: Standard chat interface.
* `POST /v1/completions`: Legacy completion interface.
* `GET /health`: System health check.

### Supported Parameters

The backend supports standard parameters passed through to the underlying SGLang engine, including but not limited to:

* `model`: ID of the model to use.
* `messages`: List of chat messages (system, user, assistant).
* `temperature`: Sampling temperature.
* `max_tokens`: Maximum number of tokens to generate.
* `stream`: Boolean to enable SSE streaming.
* `top_p`, `frequency_penalty`, `presence_penalty`.