Stitch any CI into Tangled
1# tack - Connect Tangled to your CI
2
3Tack is a custom [Tangled](https://tangled.org) spindle that runs
4CI on alternate providers and reports their results back
5to Tangled using standard ATProto records so they show up natively
6in Tangled's UI.
7
8## What it does
9
10Tack is a drop-in alternative to the stock `spindle` runner. You run
11`tack` and [register it using the standard UI](https://tangled.org/settings/spindles).
12
13Instead of executing workflows in local containers, tack translates each
14Tangled pipeline trigger into a 3rd party CI build, and reports build state
15back to Tangled using the existing `sh.tangled.pipeline.status` wire format.
16
17This makes even 3rd party CIs integrate first class into Tangled so their
18status, counts, etc. can show up inline in things like pull requests.
19
20```
21 sh.tangled.pipeline
22 Jetstream ───────────────────────▶ tack
23 │
24 │ Create Build
25 ▼
26 Buildkite
27 │
28 │ webhooks
29 ▼
30 tack ──── /events (WebSocket) ────▶ Tangled appview
31 sh.tangled.pipeline.status
32```
33
34```sh
35go run . -addr :8080
36```
37
38## Configuration
39
40Core configuration controls how tack talks to Tangled. Provider-specific
41configuration (e.g. Buildkite) lives in its own section below.
42
43### Required
44
45| Env var | Description |
46| ---------------- | ----------------------------------------------------------- |
47| `TACK_HOSTNAME` | This spindle's hostname (matches `sh.tangled.repo.spindle`) |
48| `TACK_OWNER_DID` | DID of the spindle operator |
49
50### Optional
51
52| Env var | Description |
53| -------------------- | -------------------------------------------------------- |
54| `TACK_LISTEN_ADDR` | HTTP listen address (default `:8080`) |
55| `TACK_DB_PATH` | Local SQLite path (default `tack.db`) |
56| `TACK_JETSTREAM_URL` | Tangled Jetstream WebSocket URL |
57| `TACK_DEV` | Use `ws://` for knot event-streams (any non-empty value) |
58
59When no provider is configured, tack runs an in-process fake provider
60that's useful for exercising the jetstream → knot → `/events` flow
61locally without a real CI account.
62
63## Buildkite
64
65[Buildkite](https://buildkite.com) is the primary provider tack
66supports today. In Buildkite mode, every Tangled pipeline trigger
67fans out into one Buildkite build per workflow on the pipeline that
68workflow names; build state flows back to Tangled via Buildkite's
69notification webhooks.
70
71### How it fits together
72
73```
74 sh.tangled.pipeline Buildkite
75 trigger record ──▶ tack ──▶ Create Build ─┐
76 │
77 /webhooks/buildkite ◀──── notification ◀─────┘
78 │
79 ▼
80 sh.tangled.pipeline.status (broadcast on /events)
81```
82
83* **Spawn:** for each workflow on a pipeline trigger, tack POSTs to
84 `/v2/organizations/<org>/pipelines/<slug>/builds`. Both `<org>` and
85 `<slug>` come from the workflow's YAML body (see
86 [Configuring your workflows](#configuring-your-workflows)).
87* **Track:** tack persists the resulting `(build_uuid → knot, rkey,
88 workflow)` mapping in its local SQLite store so it can later
89 resolve incoming webhooks back to the originating Tangled
90 pipeline.
91* **Report:** Buildkite delivers `build.*` events to
92 `POST /webhooks/buildkite`. tack authenticates each request,
93 translates the Buildkite state into a Tangled status, and
94 broadcasts a `sh.tangled.pipeline.status` record on `/events`.
95
96### Setting up Buildkite
97
98These steps happen once on the Buildkite side, before tack can talk
99to it.
100
101#### 1. Create one or more pipelines
102
103Each Tangled workflow targets exactly one Buildkite pipeline by
104slug. There's no requirement that pipelines map 1:1 to workflows —
105many users point every workflow at a single pipeline whose
106`pipeline.yml` does `pipeline upload some-file-${TACK_WORKFLOW}.yml`,
107keeping all the per-workflow logic in the repo rather than in
108Buildkite's UI.
109
110In your Buildkite org, **Pipelines → New pipeline**:
111
112* Repository: any URL (the agent only needs to be able to clone it).
113* Steps: a minimal `pipeline upload` is usually enough — tack passes
114 the workflow name through `$TACK_WORKFLOW` so you can branch on
115 it.
116
117Note the pipeline slug from the URL
118(`https://buildkite.com/<org>/<pipeline-slug>`); your workflow YAML
119will reference it.
120
121#### 2. Create an API access token
122
123Tack uses a single API token to create builds, list jobs, and fetch
124logs. Generate one at
125<https://buildkite.com/user/api-access-tokens> with these scopes:
126
127| Scope | Used for |
128| ------------------- | ------------------------------------------------- |
129| `read_organizations`| Sanity-checking the configured org slug |
130| `write_builds` | `POST .../builds` when a Tangled trigger arrives |
131| `read_builds` | Resolving build → jobs for the `/logs` endpoint |
132| `read_build_logs` | Streaming job logs back to the Tangled appview |
133
134Restrict the token to the specific organization(s) tack will spawn
135into.
136
137#### 3. Configure a notification webhook
138
139Builds report their state back to tack through Buildkite's
140notification service.
141
142In your Buildkite org, **Settings → Notification Services → Add →
143Webhook**:
144
145* **Webhook URL:** `https://<your-tack-host>/webhooks/buildkite`
146* **Token / Secret:** any high-entropy string. You'll set the same
147 value in `TACK_BUILDKITE_WEBHOOK_SECRET`.
148* **Events:** `build.scheduled`, `build.running`, `build.finished`
149 (job-level events are ignored).
150* **Pipelines:** the pipelines tack will fire builds on.
151
152Buildkite supports two header schemes for authenticating webhooks;
153tack supports both:
154
155| Header scheme | `TACK_BUILDKITE_WEBHOOK_MODE` | Notes |
156| ----------------------- | ----------------------------- | -------------------------------------------- |
157| `X-Buildkite-Token` | `token` (default) | Secret is sent verbatim in the header |
158| `X-Buildkite-Signature` | `signature` | HMAC-SHA256 of `<timestamp>.<body>`; safer |
159
160Pick `signature` if the notification setting offers it — it doesn't
161expose the secret on the wire.
162
163### Configuring tack
164
165Setting `TACK_BUILDKITE_TOKEN` is the master switch that puts tack
166into Buildkite mode. The other variables in this section are then
167required.
168
169| Env var | Description |
170| ------------------------------- | ------------------------------------------------------------------------------ |
171| `TACK_BUILDKITE_TOKEN` | Buildkite API token (enables Buildkite mode) |
172| `TACK_BUILDKITE_ORG` | Default Buildkite organization slug (workflows may override via YAML) |
173| `TACK_BUILDKITE_WEBHOOK_SECRET` | Shared secret for `/webhooks/buildkite` auth |
174| `TACK_BUILDKITE_WEBHOOK_MODE` | `token` (default) or `signature` — must match the notification service |
175
176The pipeline a workflow runs against is **not** an environment
177variable. It lives inside the workflow YAML so each repo can target
178its own pipeline without an operator round-trip.
179
180### Configuring your workflows
181
182A Tangled workflow's `raw` body is parsed by tack as YAML. Only
183`pipeline` is required — every other field is an optional override
184or extension of what the trigger metadata already provides:
185
186```yaml
187# Required: which Buildkite pipeline this workflow fires.
188pipeline: my-pipeline-slug
189
190# Optional: org override. Defaults to TACK_BUILDKITE_ORG. The API
191# token must have access to whichever org you target.
192org: another-org
193
194# Optional: human-readable build message (default: "tangled: <name>").
195message: "Custom build message"
196
197# Optional: pin the commit/branch tack would otherwise derive from
198# the trigger. Useful for manual triggers (which carry no commit).
199commit: abcdef0123
200branch: main
201
202# Optional: extra env + meta_data merged on top of tack's defaults
203# (see "What tack injects into every build" below).
204env:
205 CUSTOM_VAR: value
206meta_data:
207 custom-key: value
208
209# Optional: forwarded verbatim to the Buildkite create-build API.
210clean_checkout: true
211ignore_pipeline_branch_filters: true # default: true
212author:
213 name: "Author Name"
214 email: "author@example.com"
215```
216
217When the trigger is a pull request, tack auto-populates Buildkite's
218`pull_request_base_branch` from the PR target so step-level branch
219filters work without extra config.
220
221#### What tack injects into every build
222
223Regardless of what the workflow YAML adds on top, tack always
224provides the following so your Buildkite pipeline can recover the
225Tangled identity of the build:
226
227| Channel | Key | Value |
228| ----------- | -------------------- | ---------------------------------------- |
229| `env` | `TACK_KNOT` | knot hostname the pipeline came from |
230| `env` | `TACK_PIPELINE_RKEY` | rkey of the originating pipeline record |
231| `env` | `TACK_WORKFLOW` | workflow name (typically a YAML filename) |
232| `env` | `TACK_WORKFLOW_RAW` | the workflow's raw YAML body |
233| `meta_data` | `tack:knot` | same as `TACK_KNOT` |
234| `meta_data` | `tack:pipeline_rkey` | same as `TACK_PIPELINE_RKEY` |
235| `meta_data` | `tack:workflow` | same as `TACK_WORKFLOW` |
236
237A common pattern is for the Buildkite pipeline's root step to do a
238`pipeline upload` against a workflow-specific YAML file based on
239`$TACK_WORKFLOW`, e.g.:
240
241```yaml
242# Buildkite pipeline.yml
243steps:
244 - label: ":pipeline: dispatch ${TACK_WORKFLOW}"
245 command: "buildkite-agent pipeline upload .buildkite/${TACK_WORKFLOW}"
246```