# Sidetrail

[Sidetrail](https://sidetrail.app/) is an app to create and share "trails". Create sequential paths with 2-24 stops, walk through them one step at a time, and share them with others.

Built on [AT](https://atproto.com/) protocol: trails, walks, and completions are stored in users' repositories and synced via Jetstream.

## Architecture

Three services:

- **Next.js app** — Web frontend + API. Reads from PostgreSQL, writes to users' ATProto repos via OAuth.
- **Ingester** — Subscribes to [Jetstream](https://docs.bsky.app/blog/jetstream) and indexes `app.sidetrail.*` records into PostgreSQL.
- **Realtime** — Rebroadcasts Jetstream events to browser clients for real-time updates on some pages.

## Local Development

### Prerequisites

- Node.js 22.16.0+
- PostgreSQL
- Redis

### Setup

```bash
# Install dependencies
npm install

# Start Redis
brew services start redis

# Create the database
createdb sidetrail

# Copy environment file and configure DATABASE_URL and REDIS_URL
cp .env.example .env

# Push database schema
npm run db:push

# Populate the database from the live network (see Syncing below)
npm run sync

# Start all services (run in separate terminals)
npm run dev           # Next.js app on :3000
npm run dev:ingester  # Jetstream ingester
npm run dev:realtime  # Realtime server
```

Then open `127.0.0.1:3000` (it has to be `127.0.0.1`, not `localhost`).

### Syncing

The ingester only indexes records as they stream past on Jetstream, and
Jetstream's replay window is only a few days. Any fresh database — or one
whose ingester was down for longer than the window — has drifted from the
network.

`npm run sync` reconciles: it asks the relay which repos contain
`app.sidetrail.*` records, fetches them from each user's PDS, and makes the
index mirror the lexicon-valid records currently on the network — adding
what's missing and pruning what was deleted or doesn't validate. Repos it
fails to reach are left untouched. Idempotent, safe to rerun anytime
(including after changing a lexicon).

```bash
npm run sync                            # local DB (.env)
DATABASE_URL=postgres://... npm run sync  # any other DB
```

### Environment Variables

See `.env.example`. For local development, `DATABASE_URL` and `REDIS_URL` are required.

Production requires:

- `PUBLIC_URL` - Your app's public URL
- `PRIVATE_KEY_ES256` - OAuth signing key (generate with `node scripts/gen-jwk.mjs`)
- `COOKIE_SECRET` - Session encryption (32+ chars)

## Lexicons

Sidetrail uses three ATProto record types:

- `app.sidetrail.trail` - Trail with embedded stops
- `app.sidetrail.walk` - User's current walk state (deleted on completion)
- `app.sidetrail.completion` - Permanent completion record

See [lexicons/LEXICONS.md](lexicons/LEXICONS.md) for details.

## Testing

```bash
# Create test database (one-time)
psql -c "CREATE DATABASE sidetrail_test"

npm test
```

## Scripts

```bash
npm run dev            # Start Next.js dev server
npm run dev:ingester   # Start Jetstream ingester
npm run dev:realtime   # Start realtime server

npm run build          # Production build
npm run db:push        # Push schema to database
npm run db:studio      # Open Drizzle Studio

npm run test           # Run tests
npm run check          # Lint + typecheck
npm run sync           # Sync the index with the live network
```

## Deployment

Uses [Railway](https://railway.app/) with three services. One-time setup:

```bash
railway link --project sidetrail
```

Then deploy with:

```bash
npm run deploy:app
npm run deploy:ingester
npm run deploy:realtime
npm run deploy:all      # all three, sequentially
```

## License

MIT
