Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .claude/rules/sim-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ contracts, and demonstrated regressions.
| Suffix | Needs | Run with | In CI |
|--------|-------|----------|-------|
| `*.test.ts(x)` | nothing; global mocks from `vitest.setup.ts` | `bun run --cwd <workspace> test` | `test` jobs |
| `*.integration.ts` | real PostgreSQL (`TEST_DATABASE_URL`), Redis (`TEST_REDIS_URL`; optional locally, required in CI) | `bun run --cwd <workspace> test --mode integration` | `integration` jobs, by glob (sharded per provisioning path) |
| `*.integration.ts` | real PostgreSQL (`TEST_DATABASE_URL`) and Redis (`TEST_REDIS_URL`: required by `apps/sim` and `apps/realtime`; `packages/db` runs without it locally and requires it in CI) | `bun run --cwd <workspace> test --mode integration` | `integration` jobs, by glob (sharded per provisioning path) |
| `*.live.test.ts` | provider APIs, hosted sandboxes, local runtimes, or sibling checkouts | `bun run --cwd apps/sim test --mode live <file>` | never |
| `apps/desktop/e2e/*.spec.ts` | the packaged Electron app | Playwright | desktop E2E workflow |
| `apps/sim/scripts/test-*-e2e.ts` | a running app over HTTP | its `package.json` script when one exists (`bun run test:scim:e2e`; `test:workflow-version-compare:e2e` adds `--no-env-file`), else `bun scripts/test-<suite>-e2e.ts` from apps/sim | `e2e` jobs (`.github/scripts/http-e2e.sh`) |
Expand All @@ -46,7 +46,10 @@ contracts, and demonstrated regressions.
`vi.mock` in the file.
- `TEST_DATABASE_URL` must be loopback and its database name must contain a `test` segment
(`sim_test`); `TEST_REDIS_URL` must be loopback. `packages/db/testing/test-infrastructure.ts` owns
those checks. Isolate with a unique schema or generated IDs, and clean up in `afterAll`.
those checks. `apps/sim` integration setup runs `FLUSHDB` before every file, so its
`TEST_REDIS_URL` must also select a non-zero database (`redis://127.0.0.1:6379/15`); never
point it at a Redis holding data you want to keep. Isolate with a unique schema or generated
IDs, and clean up in `afterAll`.
- Integration files run one at a time against one shared database. A new `*.integration.ts` is
picked up by CI with no workflow change, and the run writes the JSON report configured in
`vitest.shared.ts`, which CI uploads. Never exclude a failing suite from the run in
Expand Down
7 changes: 5 additions & 2 deletions .cursor/rules/sim-testing.mdc
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ contracts, and demonstrated regressions.
| Suffix | Needs | Run with | In CI |
|--------|-------|----------|-------|
| `*.test.ts(x)` | nothing; global mocks from `vitest.setup.ts` | `bun run --cwd <workspace> test` | `test` jobs |
| `*.integration.ts` | real PostgreSQL (`TEST_DATABASE_URL`), Redis (`TEST_REDIS_URL`; optional locally, required in CI) | `bun run --cwd <workspace> test --mode integration` | `integration` jobs, by glob (sharded per provisioning path) |
| `*.integration.ts` | real PostgreSQL (`TEST_DATABASE_URL`) and Redis (`TEST_REDIS_URL`: required by `apps/sim` and `apps/realtime`; `packages/db` runs without it locally and requires it in CI) | `bun run --cwd <workspace> test --mode integration` | `integration` jobs, by glob (sharded per provisioning path) |
| `*.live.test.ts` | provider APIs, hosted sandboxes, local runtimes, or sibling checkouts | `bun run --cwd apps/sim test --mode live <file>` | never |
| `apps/desktop/e2e/*.spec.ts` | the packaged Electron app | Playwright | desktop E2E workflow |
| `apps/sim/scripts/test-*-e2e.ts` | a running app over HTTP | its `package.json` script when one exists (`bun run test:scim:e2e`; `test:workflow-version-compare:e2e` adds `--no-env-file`), else `bun scripts/test-<suite>-e2e.ts` from apps/sim | `e2e` jobs (`.github/scripts/http-e2e.sh`) |
Expand All @@ -43,7 +43,10 @@ contracts, and demonstrated regressions.
`vi.mock` in the file.
- `TEST_DATABASE_URL` must be loopback and its database name must contain a `test` segment
(`sim_test`); `TEST_REDIS_URL` must be loopback. `packages/db/testing/test-infrastructure.ts` owns
those checks. Isolate with a unique schema or generated IDs, and clean up in `afterAll`.
those checks. `apps/sim` integration setup runs `FLUSHDB` before every file, so its
`TEST_REDIS_URL` must also select a non-zero database (`redis://127.0.0.1:6379/15`); never
point it at a Redis holding data you want to keep. Isolate with a unique schema or generated
IDs, and clean up in `afterAll`.
- Integration files run one at a time against one shared database. A new `*.integration.ts` is
picked up by CI with no workflow change, and the run writes the JSON report configured in
`vitest.shared.ts`, which CI uploads. Never exclude a failing suite from the run in
Expand Down
14 changes: 7 additions & 7 deletions .github/scripts/http-e2e.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
#
# Usage: http-e2e.sh <scim|cli|stop-after|desktop-inbox|assistant-actions|organization-search|search-permissions|browser-extension|browser-extension-native|browser-extension-navigation|browser-extension-composer|browser-extension-access|project-files|markdown-viewer|workflow-code-ui|mobile|seo> (run from apps/sim)
#
# The job provides DATABASE_URL, BETTER_AUTH_SECRET and ENCRYPTION_KEY; each group sets the rest of
# The job provides DATABASE_URL, BETTER_AUTH_SECRET, ENCRYPTION_KEY and a Redis on 127.0.0.1:6379
# (Redis is required, so every app and realtime server gets REDIS_URL); each group sets the rest of
# its app's environment here. Reports and server logs land in $RUNNER_TEMP/e2e.
#
# Under `next dev` the first request cold-compiles the app under Turbopack, which takes 60-185s on
Expand All @@ -23,6 +24,7 @@ set -euo pipefail
group=${1:?usage: http-e2e.sh <scim|cli|stop-after|desktop-inbox|assistant-actions|organization-search|search-permissions|browser-extension|browser-extension-native|browser-extension-navigation|browser-extension-composer|browser-extension-access|project-files|markdown-viewer|workflow-code-ui|mobile|seo>}
report_dir="$RUNNER_TEMP/e2e"
ready_timeout_seconds=600
export REDIS_URL=${REDIS_URL:-redis://127.0.0.1:6379}
mkdir -p "$report_dir"

server_pid=''
Expand Down Expand Up @@ -204,6 +206,7 @@ case "$group" in
export PEOPLE_E2E_SCREENSHOT_DIR="$report_dir/people-ui-report.json.artifacts"
SCIM_E2E_BASE_URL="$local_app_url" \
SCIM_E2E_DATABASE_URL="$DATABASE_URL" \
SCIM_E2E_REDIS_URL="$REDIS_URL" \
SCIM_E2E_AUTH_SECRET="$BETTER_AUTH_SECRET" \
SCIM_E2E_REPORT_PATH="$report_dir/scim-e2e-report.json" \
bun run test:scim:e2e
Expand Down Expand Up @@ -238,8 +241,7 @@ case "$group" in
;;

# The search suites serve their own fixtures in-process and need no app. They share this group
# because they finish in seconds. Self-hosted without billing: hosted billing admits runs through
# Redis, which the CLI app is not given.
# because they finish in seconds. Self-hosted, so runs are not admitted through hosted billing.
cli)
for search in google-content lucid zoom google-meet; do
report_var="SEARCH_$(echo "$search" | tr 'a-z-' 'A-Z_')_REPORT_PATH"
Expand All @@ -262,7 +264,7 @@ case "$group" in
bun run test:cli-run-latency:e2e
;;

# Self-hosted: hosted billing admits a run only through a Redis usage reservation.
# Self-hosted, so runs are not admitted through hosted billing's usage reservation.
stop-after)
export NEXT_PUBLIC_FORCE_HOSTED=false
export CRON_SECRET=stop-after-e2e-local-cron-secret
Expand All @@ -278,7 +280,6 @@ case "$group" in
;;

desktop-inbox|assistant-actions|organization-search|search-permissions)
export REDIS_URL=redis://127.0.0.1:6379
export NEXT_PUBLIC_FORCE_HOSTED=false
export COPILOT_TOOL_PERMISSIONS_ENABLED=true
export KNOWLEDGE_MEMBER_ACCESS=true CREDENTIAL_GROUPS=true ACCESS_CONTROL_ENABLED=true
Expand Down Expand Up @@ -330,7 +331,6 @@ case "$group" in
browser-extension-composer) extension_group=composer ;;
browser-extension-access) extension_group=access ;;
esac
export REDIS_URL=redis://127.0.0.1:6379
export NEXT_PUBLIC_FORCE_HOSTED=false
export KNOWLEDGE_MEMBER_ACCESS=true CREDENTIAL_GROUPS=true ACCESS_CONTROL_ENABLED=true
export SIM_LINKS=true
Expand Down Expand Up @@ -358,7 +358,7 @@ JS

project-files)
PROJECT_FILES_E2E_ADMIN_DATABASE_URL=postgresql://postgres:postgres@127.0.0.1:5432/postgres \
PROJECT_FILES_E2E_REDIS_URL=redis://127.0.0.1:6379 \
PROJECT_FILES_E2E_REDIS_URL="$REDIS_URL" \
PROJECT_FILES_E2E_BASE_URL=http://127.0.0.1:3050 \
PROJECT_FILES_E2E_RELAY_URL=http://127.0.0.1:3052 \
PROJECT_FILES_E2E_REPORT_PATH="$report_dir/project-files/orchestration.json" \
Expand Down
16 changes: 13 additions & 3 deletions .github/workflows/checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -122,7 +122,7 @@ jobs:
--health-retries 10
env:
TEST_DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_test
TEST_REDIS_URL: redis://127.0.0.1:6379
TEST_REDIS_URL: redis://127.0.0.1:6379/15
DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_test
BETTER_AUTH_SECRET: oauth-postgres-ci-secret-at-least-32-characters
NEXT_PUBLIC_APP_URL: https://test.sim.ai
Expand Down Expand Up @@ -236,9 +236,9 @@ jobs:
--health-interval 5s
--health-timeout 5s
--health-retries 10
# Desktop inbox, browser extension, and Project Files use Redis.
# Redis is required: every app server the suites start connects to it.
redis:
image: ${{ (matrix.group == 'desktop-inbox' || matrix.group == 'assistant-actions' || matrix.group == 'organization-search' || matrix.group == 'search-permissions' || startsWith(matrix.group, 'browser-extension-') || matrix.group == 'project-files') && 'redis:8.2-alpine' || '' }}
image: redis:8.2-alpine
ports:
- 6379:6379
options: >-
Expand Down Expand Up @@ -408,6 +408,16 @@ jobs:
--health-interval 5s
--health-timeout 5s
--health-retries 10
# Redis is required: the app server refuses to start without it.
redis:
image: redis:8.2-alpine
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 5s
--health-timeout 5s
--health-retries 10
env:
DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/sim_test
BETTER_AUTH_SECRET: http-e2e-ci-secret-at-least-32-characters
Expand Down
12 changes: 12 additions & 0 deletions .github/workflows/docs-negotiation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -334,6 +334,17 @@ jobs:
--health-interval 5s
--health-timeout 5s
--health-retries 10
# The platform consumer runs the app's docs search in-process, and its provider admission
# needs Redis.
search-redis:
image: ${{ (matrix.group == 'http' || matrix.group == 'platform') && 'redis:8.2-alpine' || '' }}
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 5s
--health-timeout 5s
--health-retries 10
env:
DATABASE_URL: postgresql://docs_test:docs_test@127.0.0.1:1/docs_test
DOCS_E2E_BASE_URL: http://127.0.0.1:3001
Expand All @@ -348,6 +359,7 @@ jobs:
DOCS_SAMPLES_SYNTAX_LANGUAGES: javascript,python,bash,go,java,csharp
DOCS_SAMPLES_HTTP_BASE_URL: http://127.0.0.1:3001
DOCS_PROVIDER_E2E_DATABASE_URL: postgresql://docs_test:docs_test@127.0.0.1:5432/docs_search_test
DOCS_PROVIDER_E2E_REDIS_URL: redis://127.0.0.1:6379
DOCS_PROVIDER_E2E_BASE_URL: http://127.0.0.1:3001
DOCS_PROVIDER_E2E_REPORT_PATH: ${{ github.workspace }}/apps/docs/test-results/platform-consumer/report.json
steps:
Expand Down
1 change: 0 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,6 @@ npx sim-setup add email
npx sim-setup add storage
npx sim-setup add sandbox
npx sim-setup add jobs
npx sim-setup add cache
npx sim-setup add knowledge
npx sim-setup add chat
npx sim-setup add llm
Expand Down
7 changes: 4 additions & 3 deletions apps/desktop/e2e/desktop-tools-live-sim.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,10 @@ import {
* what the user or the model would observe: the result the model is resumed with, what landed in
* the workspace, what Sim persisted, and which requests reached Sim.
*
* Sim runs the background executor wherever it has Redis, as it does here. Most tests cover the
* chat view running desktop tools itself, as on an install without Redis: the proxy answers the
* app's registration as such an install does, so the app stays dormant and no turn binds to it.
* Sim runs the background executor for a registered desktop app, as it does here. Most tests cover
* the chat view running desktop tools itself: the proxy answers the app's registration with the
* executor off (as servers predating required Redis did), so the app stays dormant and no turn
* binds to it.
* The tests marked as running in the background let Sim's own answer through.
*/

Expand Down
2 changes: 1 addition & 1 deletion apps/docs/MACHINE_ACCESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ The suite exercises real HTTP and the official MCP client, including discovery p

The PostgreSQL suite uses a new, empty, loopback database and no embedding key. It verifies real keyword search, citation freshness, graceful database failure, repeated index publication, retirement and failed-publication preservation. It refuses databases containing existing tables. CI provisions its own `pgvector/pgvector:pg17` service and uploads the report with the other docs checks.

The platform consumer harness also uses an empty loopback database and the running docs build. From `apps/sim`, run `bun --no-env-file scripts/test-docs-search-provider-e2e.ts` with `DOCS_PROVIDER_E2E_DATABASE_URL`, `DOCS_PROVIDER_E2E_BASE_URL` and `DOCS_PROVIDER_E2E_REPORT_PATH`. Its default mode uses the existing Azure endpoint configuration with a local HTTP fixture, deterministic vectors and the real PostgreSQL provider-admission store. It verifies the platform's direct-table search, scope exclusions, VFS reads and public citations, with no paid requests. This verifies the transport and consumer contract, not semantic relevance.
The platform consumer harness also uses an empty loopback database and the running docs build. From `apps/sim`, run `bun --no-env-file scripts/test-docs-search-provider-e2e.ts` with `DOCS_PROVIDER_E2E_DATABASE_URL`, `DOCS_PROVIDER_E2E_REDIS_URL` (a loopback Redis), `DOCS_PROVIDER_E2E_BASE_URL` and `DOCS_PROVIDER_E2E_REPORT_PATH`. Its default mode uses the existing Azure endpoint configuration with a local HTTP fixture, deterministic vectors and the real Redis provider-admission store. It verifies the platform's direct-table search, scope exclusions, VFS reads and public citations, with no paid requests. This verifies the transport and consumer contract, not semantic relevance.

Live provider verification is opt-in: set `DOCS_PROVIDER_E2E_ALLOW_LIVE=1` and supply `OPENAI_API_KEY`, using a fresh database named `docs_search_provider_test`. The harness limits publication to the introduction page, rejects plans above 15 chunks, and sends at most one batch with up to three attempts. The consumer makes at most one query embedding request, for a total cap of four provider requests. Provider failure stops the run and remains a failed report. The default fixture mode also accepts the empty `docs_search_test` database used by the PostgreSQL suite. Both modes remove their tables when finished.

Expand Down
6 changes: 3 additions & 3 deletions apps/docs/content/docs/platform/self-hosting/architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Sim runs an app, a realtime server, PostgreSQL, Redis, and a scheduler. This pag
| **realtime** | `ghcr.io/simstudioai/realtime` | 3002 | In-flight collaboration state | Yes |
| **migrations** | `ghcr.io/simstudioai/migrations` | — | One-shot schema and data migrations | Yes |
| **postgresql** | `pgvector/pgvector:pg17` | 5432 | Durable application data | Yes |
| **redis** | `redis:7-alpine` | 6379 | Cache, coordination, and live events | Bundled by default; required across replicas |
| **redis** | `redis:7-alpine` | 6379 | Rate limits, coordination, and live events | Yes (bundled by default) |
| **cron** | `ghcr.io/simstudioai/cron` (Compose) / `curlimages/curl` (Kubernetes) | — | None | Required for scheduled work |
| **pii** | `ghcr.io/simstudioai/pii` | 5001 | None | Optional |
| **ollama** | `ollama/ollama` | 11434 | Model cache | Optional |
Expand All @@ -36,7 +36,7 @@ The Bun Socket.IO server carries collaborative editing, live execution updates,
Realtime shares PostgreSQL and `BETTER_AUTH_SECRET` with the app, using Better Auth's shared-database-session pattern. It authenticates the same accounts without a separate login.

<Callout type="warn">
More than one realtime replica requires `REDIS_URL`. The Socket.IO Redis adapter carries events between replicas; without it, members connected to different pods do not see each other's edits.
Realtime requires `REDIS_URL`, pointing at the same Redis as the app. The Socket.IO Redis adapter carries events between replicas, so members connected to different pods see each other's edits.
</Callout>

A realtime server must exist for editing and live updates. Set `realtime.enabled: false` in Helm only when `app.env.SOCKET_SERVER_URL` points to a realtime instance you run separately.
Expand All @@ -55,7 +55,7 @@ PostgreSQL holds workflows, runs, logs, accounts, organizations, credentials, kn

### Redis

Redis carries live events and stores shared coordination state. Some stores fall back to PostgreSQL or process-local state, but cross-pod collaboration has no local fallback. CLI API-key pairing also requires Redis, even with one replica; OAuth login uses a separate path. See [Redis](/platform/self-hosting/redis) for the full fallback matrix.
Redis is required, even with one replica. It holds rate-limit buckets, one-time passcodes, and shared coordination state, and carries live events between processes. There is no PostgreSQL or process-local fallback, and the app refuses to start without `REDIS_URL`. See [Redis](/platform/self-hosting/redis) for everything it backs.

### Scheduler

Expand Down
6 changes: 3 additions & 3 deletions apps/docs/content/docs/platform/self-hosting/docker.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ curl -fsS http://localhost:3000/api/health
| `simstudio` | 3000 | App; 8G memory limit. |
| `realtime` | 3002 | WebSocket service; 1G memory limit. |
| `db` | Not published | PostgreSQL 17 with pgvector. |
| `redis` | Not published | Shared coordination, pub/sub, and caches. |
| `redis` | Not published | Required. Rate limits, shared coordination, pub/sub, and caches. |
| `cron` | None | Calls background job endpoints. |
| `migrations` | None | Applies migrations once, then exits. |

Expand Down Expand Up @@ -238,14 +238,14 @@ Invoke commands as `bunx sim-setup <command>` unless the package is installed gl
| `reset` | Confirms before archiving env files and removing managed Compose/dev data. On Helm it uninstalls the release; PostgreSQL StatefulSet PVCs remain and must be deleted separately to erase data. |
| `config` | Reports configured capabilities and integrations. |
| `doctor` | Checks env files and reachable services. `--fix` writes supported repairs; `--json` emits structured findings. |
| `add <feature>` | Configures `email`, `storage`, `sandbox`, `jobs`, `cache`, `knowledge`, `knowledge-embeddings`, `chat`, `llm`, or `integration <slug>`. Requires one writable configuration; refuses ambiguous destinations and Helm releases. |
| `add <feature>` | Configures `email`, `storage`, `sandbox`, `jobs`, `knowledge`, `knowledge-embeddings`, `chat`, `llm`, or `integration <slug>`. Requires one writable configuration; refuses ambiguous destinations and Helm releases. |
| `desktop` | Prints this deployment's installer/server links. `--url <url>` skips discovery; `--no-open` skips the browser-opening prompt. |

`--dir <path>` works with every command and scopes standalone lifecycle commands to that installation.

For Compose, `reset` archives `.env` beside the Compose file. For dev, Kubernetes, or no detected install, it archives `apps/sim/.env`, `apps/realtime/.env`, `packages/db/.env`, and root `.env`. Preserve any backups before intentionally removing volumes.

`add` writes beside the Compose file or to `apps/sim/.env` on a source checkout. Recreate containers that read the change: `cache` changes `REDIS_URL` for both app and realtime. For a source checkout, mirror shared values into `apps/realtime/.env`. `restart` alone does not apply changed container environment. Desktop discovery refuses configurations naming different origins; pass `--url` to choose.
`add` writes beside the Compose file or to `apps/sim/.env` on a source checkout. Recreate containers that read the change. To point both app and realtime at a different Redis, set `REDIS_URL` in `.env` and recreate both containers. For a source checkout, mirror shared values into `apps/realtime/.env`. `restart` alone does not apply changed container environment. Desktop discovery refuses configurations naming different origins; pass `--url` to choose.

## Back up and update [#commands]

Expand Down
Loading
Loading