# System Architecture

TruePortAI is a **Defensible AI Gateway** — an enterprise-grade, multi-tenant platform that sits between your applications and commercial LLM providers. It provides real-time governance, complete audit trails, ML-powered violation detection, and full data sovereignty.

---

## C4 Level 1 — Context Diagram

```mermaid
C4Context
    title TruePortAI — System Context

    Person(devUser, "Developer / AI Engineer", "Sends AI requests via SDK or HTTP")
    Person(adminUser, "Platform Admin", "Manages tenants, billing, system config")
    Person(tenantAdmin, "Tenant Admin", "Manages API keys, policies, team members")

    System(trueportai, "TruePortAI Platform", "AI Gateway, Governance & Audit Engine")

    System_Ext(openai, "OpenAI API", "GPT-4o, GPT-3.5-turbo")
    System_Ext(anthropic, "Anthropic API", "Claude 3.5, Claude 3")
    System_Ext(google, "Google Gemini API", "Gemini 1.5 Pro / Flash")
    System_Ext(azure, "Azure OpenAI", "Enterprise GPT deployments")
    System_Ext(s3, "AWS S3 / Azure Blob", "Tenant-owned raw interaction storage")
    System_Ext(smtp, "SMTP / SES", "Email notifications and alerts")
    System_Ext(stripe, "Stripe", "Billing and subscriptions")

    Rel(devUser, trueportai, "Routes LLM requests through", "HTTPS/REST")
    Rel(tenantAdmin, trueportai, "Configures keys, policies, views analytics", "HTTPS")
    Rel(adminUser, trueportai, "Manages tenants, billing, system settings", "HTTPS")

    Rel(trueportai, openai, "Proxies requests to", "HTTPS")
    Rel(trueportai, anthropic, "Proxies requests to", "HTTPS")
    Rel(trueportai, google, "Proxies requests to", "HTTPS")
    Rel(trueportai, azure, "Proxies requests to", "HTTPS")
    Rel(trueportai, s3, "Streams raw logs to", "HTTPS/S3 API")
    Rel(trueportai, smtp, "Sends alerts via", "SMTP/TLS")
    Rel(trueportai, stripe, "Manages billing via", "HTTPS")
```

---

## C4 Level 2 — Container Diagram

```mermaid
graph TB
    subgraph "Client Layer"
        APP["Client Application<br/>(any OpenAI-compatible SDK)"]
        BROWSER["Browser<br/>(Angular Dashboard)"]
        WWW["Public Website<br/>(Next.js)"]
    end

    subgraph "CDN / Edge"
        CF["AWS CloudFront<br/>Global CDN + TLS termination"]
    end

    subgraph "trueportai-services  |  AWS Lambda + Python FastAPI"
        GW["AI Gateway / Proxy<br/>Route: /trueportai-services/{provider}/..."]
        TSAPI["TruePortAI REST API<br/>Route: /trueportai-services/api/v1"]
        GWMW["Gateway Middleware<br/>Auth · RateLimit · AuditLog"]
        CB["Circuit Breaker<br/>(Redis-backed)"]
        RL["Rate Limiter<br/>(Redis fixed-window)"]
    end

    subgraph "platform-backend  |  AWS Lambda + Python FastAPI"
        AUTH["Auth Service<br/>JWT · OTP · Password Reset"]
        ORGAPI["Organization API<br/>Tenants · Members · Invites"]
        CFGAPI["Config API<br/>Hierarchical Config Mgmt"]
        SUBAPI["Subscriptions & Catalog<br/>Plans · Products · Billing"]
        STOR["Storage Service<br/>S3 · Azure · GCP abstraction"]
        ADMINAPI["Admin API<br/>Platform-level management"]
        MAILSVC["Mail Service<br/>SMTP · Templates"]
    end

    subgraph "trueportai-analytics  |  On-Premise / VPC Python"
        INGST["S3 Event Listener<br/>(Trigger / Polling)"]
        MLPIPE["ML Pipeline<br/>PII · Bias · Injection · Exfil"]
        VIOWRT["Violation Writer<br/>MongoDB + Alert Publisher"]
    end

    subgraph "Data Layer"
        MONGODB[("MongoDB Atlas<br/>saas_platform_core<br/>trueport_ai")]
        REDIS[("Redis<br/>Rate limits · Circuit state")]
        S3STORE[("AWS S3 / Azure Blob<br/>Raw interaction JSON")]
    end

    subgraph "External LLMs"
        OPENAI["OpenAI API"]
        ANTHROPIC["Anthropic API"]
        GOOGLE["Google Gemini API"]
        AZURE_AI["Azure OpenAI"]
    end

    APP --> CF
    BROWSER --> CF
    WWW --> CF

    CF --> GW
    CF --> TSAPI
    CF --> AUTH
    CF --> ORGAPI

    GW --> GWMW
    GWMW --> RL
    GWMW --> CB
    CB --> OPENAI & ANTHROPIC & GOOGLE & AZURE_AI
    GWMW --> S3STORE
    GWMW --> MONGODB

    TSAPI --> MONGODB

    AUTH --> MONGODB
    ORGAPI --> MONGODB
    CFGAPI --> MONGODB
    SUBAPI --> MONGODB
    STOR --> S3STORE
    MAILSVC --> SMTP_EXT["SMTP / SES"]

    S3STORE --> INGST
    INGST --> MLPIPE
    MLPIPE --> VIOWRT
    VIOWRT --> MONGODB
    VIOWRT --> MAILSVC

    RL --> REDIS
    CB --> REDIS

    style GW fill:#7B68EE,color:#fff
    style MLPIPE fill:#FF6B6B,color:#fff
    style MONGODB fill:#50C878,color:#fff
    style REDIS fill:#FF9500,color:#fff
    style S3STORE fill:#4A90D9,color:#fff
```

---

## Solution Repositories

| Repository | Technology | Deployment | Responsibility |
|---|---|---|---|
| `platform-backend` | Python 3.11 · FastAPI · Uvicorn | AWS Lambda (Zappa) | Auth, RBAC, Billing, Mail, Config |
| `trueportai-services` | Python 3.11 · FastAPI · Uvicorn | AWS Lambda (Zappa) | AI Gateway Proxy, Key Mgmt, Usage Analytics API |
| `trueportai-analytics` | Python 3.11 · HuggingFace · Triton | On-Premise / VPC GPU Node | ML violation detection pipeline |
| `trueportai-ui` | Angular 17+ | AWS S3 + CloudFront | AI Governance dashboard (SPA) |
| `platform-ui` | Angular 17+ | AWS S3 + CloudFront | Platform admin dashboard (SPA) |
| `trueportai-www` | Next.js 14+ | AWS S3 + CloudFront (Static Export) | Public website + this documentation |

---

## Implementation Stack

| Layer | Technology | Purpose |
|---|---|---|
| API Runtime | Python 3.11 + FastAPI + Uvicorn | Async REST APIs |
| ASGI → Lambda | Mangum (AWS Lambda adapter) | Serverless hosting |
| ODM (Async) | Motor + Beanie | MongoDB async driver |
| Password Hashing | Argon2-cffi | Credential security |
| Token Auth | python-jose (HS256 JWT) | Session management |
| Proxy HTTP | httpx (async) | Upstream LLM forwarding |
| Rate Limiting | Redis (fixed-window) | Per-key throttling |
| Circuit Breaker | Redis (state machine) | Provider resilience |
| Frontend | Angular 17 + TailwindCSS | Platform dashboards |
| Public Site | Next.js 14 + TailwindCSS | Marketing + docs |
| Database | MongoDB Atlas (M10+) | Operational data |
| Storage | AWS S3 / Azure Blob | Audit log archive |
| CDN | AWS CloudFront | Global edge delivery |
| ML Inference | NVIDIA Triton Inference Server | GPU-accelerated models |
| ML Models | HuggingFace Transformers | RoBERTa, DeBERTa, DistilBERT |

---

## Key Workflows

### 1. AI Request Flow

```mermaid
flowchart LR
    A[Client App] -->|"POST /trueportai-services/openai/v1/chat/completions\nx-api-key: sk-..."| B[CloudFront]
    B --> C[TruePortAI Gateway]
    C --> D{API Key\nValid?}
    D -->|No| E[401 Unauthorized]
    D -->|Yes| F{Rate Limit\nOK?}
    F -->|Exceeded| G[429 Too Many Requests]
    F -->|OK| H{Circuit\nBreaker Open?}
    H -->|Open| I[503 Service Unavailable]
    H -->|Closed| J[Forward to Provider]
    J --> K[LLM Provider API]
    K --> L[Response Stream]
    L --> M[Return to Client]
    L --> N([Async: Log to MongoDB])
    L --> O([Async: Stream to S3])
    O --> P([Analytics Engine triggers])
```

### 2. Violation Detection Pipeline

```mermaid
flowchart TD
    A[Raw JSON in S3] -->|S3 Event / Polling| B[Analytics Ingest]
    B --> C[Deserialize Payload]
    C --> D{Policy Rules\nMatch?}
    D -->|Yes| E[Instant Violation Record]
    C --> F[ML Pipeline — Parallel]
    F --> G[PII/PHI NER\nRoBERTa]
    F --> H[Bias / Toxicity\nDeBERTa-v3]
    F --> I[Prompt Injection\nDistilBERT + Heuristics]
    F --> J[Exfil Guard\nRegex + Entropy]
    G & H & I & J --> K{Any\nViolations?}
    K -->|No| L[Update log: clean=true]
    K -->|Yes| M[Write Violation Records\nto MongoDB]
    M --> N[Send Alert Email\nvia Platform Mail API]
    M --> O[Update Dashboard\nMetrics]
```

### 3. Authentication & Tenant Selection Flow

```mermaid
flowchart LR
    A[User Login] --> B[POST /auth/login]
    B --> C{Credentials\nValid?}
    C -->|No| D[401]
    C -->|Yes| E[Global JWT Issued\nsub = user_id]
    E --> F[GET /tenants\nList user's organizations]
    F --> G[User Selects Tenant]
    G --> H[POST /auth/select-tenant]
    H --> I[Scoped JWT Issued\nsub, tid, role, env]
    I --> J[Access Platform APIs\nwith scoped token]
    J --> K{Need env change?}
    K -->|Yes| L[POST /auth/switch-environment]
    L --> M[New scoped token\nfor dev / staging / prod]
```

---

## Multi-Tenancy Architecture

TruePortAI uses a **logical multi-tenancy** model: all tenants share the same database cluster but are isolated via `tenant_id` fields and middleware-enforced scoping.

```mermaid
graph TD
    subgraph "Request enters"
        REQ[Incoming Request]
    end

    subgraph "TenantContextMiddleware"
        M1{Skip global\n/auth routes?}
        M2{JWT has\ntid claim?}
        M3{Subdomain\npresent?}
        M4{X-Tenant-Slug\nheader?}
        M5[Resolve Tenant\nfrom DB]
        M6[Attach tenant_id\nto request.state]
        M7[403 — Tenant\nUnresolved]
    end

    REQ --> M1
    M1 -->|Yes| PASS[Pass through]
    M1 -->|No| M2
    M2 -->|Yes| M5
    M2 -->|No| M3
    M3 -->|Yes| M5
    M3 -->|No| M4
    M4 -->|Yes| M5
    M4 -->|No| M7
    M5 --> M6
```

**Tenant context resolution priority**:
1. JWT `tid` claim (production flows)
2. HTTP subdomain (e.g., `acme.trueportai.com`)
3. `X-Tenant-Slug` header (local/dev override)

---

## Security Architecture

```mermaid
graph TD
    subgraph "Authentication"
        AT1[Email + Password → JWT]
        AT2[OTP / Magic Code → JWT]
        AT3[Password Reset → Temp Token → JWT]
        AT4[OAuth2 → Google / LinkedIn / GitHub]
    end

    subgraph "Authorization (RBAC)"
        AZ1[Platform Admin — System-wide]
        AZ2[Tenant Owner — Full tenant control]
        AZ3[Tenant Admin — Manage members & keys]
        AZ4[Member — Use keys, view own analytics]
        AZ5[Viewer — Read-only analytics]
    end

    subgraph "Gateway Security"
        GS1[API Key Validation sk-...]
        GS2[Rate Limiting 60 RPM default]
        GS3[Circuit Breaker 5-failure threshold]
        GS4[PII Redaction on response]
        GS5[Prompt Injection Block]
    end

    subgraph "Data Security"
        DS1[Argon2 password hashing]
        DS2[Short UUID — no ObjectId exposure]
        DS3[Tenant-isolated storage paths]
        DS4[Encrypted storage credentials]
        DS5[HTTPS everywhere CloudFront TLS]
    end
```

---

## Reliability Patterns

### Circuit Breaker State Machine

```mermaid
stateDiagram-v2
    [*] --> CLOSED : Service starts
    CLOSED --> OPEN : 5 consecutive failures
    OPEN --> HALF_OPEN : 30s timeout expires
    HALF_OPEN --> CLOSED : Request succeeds
    HALF_OPEN --> OPEN : Request fails
    CLOSED --> CLOSED : Success resets counter
```

### Provider Failover Strategy

```
Primary Provider → [Circuit Breaker] → Secondary Provider → [Circuit Breaker] → Error Response
```

Each provider has an independent circuit breaker key in Redis:
- `circuit_breaker:openai:state`
- `circuit_breaker:anthropic:state`
- `circuit_breaker:google:state`
- `circuit_breaker:azure:state`

### Rate Limiting Algorithm

**Fixed Window** per API key per minute:

```
Redis Key: rate_limit:{api_key}:{YYYY-MM-DD-HH-MM}
Redis Ops: INCR → if count > limit → 429; else EXPIRE 60s
```