jwuthri-cloud/Tracely-ai

选用指南

AI 根据 README 整理 · 2026/10/19 09:36:42

适用场景

适用于需要监控 AI 代理生产轨迹、自动检测失败并阻止回归的团队，特别是希望将生产失败转化为回归测试的场景。

查看 README 依据

--- ## Why another agent-observability tool? Because observability stops at the dashboard. You can see that your agent broke — then what? Every eval tool asks you to **hand-author a dataset**: sit down, invent questions, write ideal answers, keep them current as the product changes. That dataset is a guess about what might break. Production already handed you the real thing: a trace of the exact run that failed, with the exact input, the exact tool calls, the exact model responses. > **The recorded run _is_ the test.** Tracely freezes that trace into a hermetic regression case and

> replays it on every PR. Everything else — quality scores, failure clusters, suggested fixes, CI > verdicts, trends, alerts — is **derived from the trace**. There are no hand-authored datasets. | | Dataset-first tools | Tracely | |---|---|---| | Where tests come from | You write them | Promoted from real failing traces | | Fidelity to production | A guess | The exact failing run, byte for byte | | Cost to replay in CI | Live model calls | **$0** — recorded tool/LLM fixtures | | What happens on regression | A dashboard number moves | **The PR is blocked** |

使用方式

通过 Docker Compose 一键启动整个产品，或使用本地开发模式；通过 SDK 发送轨迹，并通过 CLI 命令和 GitHub Action 实现 CI 门禁。

查看 README 依据

| **Ops** | Beat self-check (queue depth, worker liveness, ingest freshness) + `/health/queue`, optional Sentry, optional trace quota + Stripe billing | [`guides/DEPLOY.md`](guides/DEPLOY.md) | Near-term plan: [design/part2-tracely/11-prd-next-steps.md](design/part2-tracely/11-prd-next-steps.md) · Long-term roadmap: [10-mvp-and-roadmap.md](design/part2-tracely/10-mvp-and-roadmap.md) --- ## Quickstart **Prerequisites:** Docker + Docker Compose. (For local dev also [uv](https://docs.astral.sh/uv/) and Node 20+ / pnpm.) ### Everything in Docker — the whole product, populated ```bash

git clone https://github.com/jwuthri-cloud/Tracely-ai && cd Tracely docker compose --profile demo up -d --build --wait open http://localhost:3001 ``` That brings up ClickHouse, Postgres, Redis and MinIO, runs every migration, seeds the default project and ingest key (`tracely_dev_key`), then populates **traces, clusters, cases and gates** — so the app opens with the screenshots above rather than an empty shell. ```bash docker compose down # stop (add -v to wipe data) ```

A promoted production failure becomes a regression test that **blocks the PR** that reintroduces it. All you need is your ingest key (it identifies your workspace) and your Tracely API URL. Three ways to wire it, depending on how your CI can reach your agent. <details open> <summary><b>Option A — let Tracely call your agent</b> (no agent code in CI, any language)</summary> Register your agent's HTTP endpoint once, author [scenarios](https://doc.tracely-ai.com/product/scenarios) — multi-turn conversations, or an adversarial goal a red-team model improvises against — and Tracely

部署要求

需要 Docker 和 Docker Compose；本地开发需要 uv 和 Node 20+ / pnpm。

查看 README 依据

| **Ops** | Beat self-check (queue depth, worker liveness, ingest freshness) + `/health/queue`, optional Sentry, optional trace quota + Stripe billing | [`guides/DEPLOY.md`](guides/DEPLOY.md) | Near-term plan: [design/part2-tracely/11-prd-next-steps.md](design/part2-tracely/11-prd-next-steps.md) · Long-term roadmap: [10-mvp-and-roadmap.md](design/part2-tracely/10-mvp-and-roadmap.md) --- ## Quickstart **Prerequisites:** Docker + Docker Compose. (For local dev also [uv](https://docs.astral.sh/uv/) and Node 20+ / pnpm.) ### Everything in Docker — the whole product, populated ```bash

已知限制

Celery worker 不支持热重载，修改后需要重启服务。

查看 README 依据

Host ports default to web **:3001** and backend **:8000**; remap with `TRACELY_WEB_PORT` / `TRACELY_BACKEND_PORT`. `backend`/`worker`/`frontend` run off source volume-mounts, so most edits need only `docker compose restart <svc>` — **except** the Celery worker, which doesn't hot-reload. ### Local dev (hot reload) ```bash cp .env.example .env make infra-up # clickhouse, postgres, redis, minio make install # uv sync + pnpm install make migrate # ClickHouse DDL + Alembic (Postgres) make seed # default project + ingest key → tracely_dev_key