Monorepo for Tangled
tangled.org
1---
2title: Tangled docs
3author: The Tangled Contributors
4date: 21 Sun, Dec 2025
5abstract: |
6 Tangled is a decentralized code hosting and collaboration
7 platform. Every component of Tangled is open-source and
8 self-hostable. [tangled.org](https://tangled.org) also
9 provides hosting and CI services that are free to use.
10
11 There are several models for decentralized code
12 collaboration platforms, ranging from ActivityPub’s
13 (Forgejo) federated model, to Radicle’s entirely P2P model.
14 Our approach attempts to be the best of both worlds by
15 adopting the AT Protocol—a protocol for building decentralized
16 social applications with a central identity
17
18 Our approach to this is the idea of “knots”. Knots are
19 lightweight, headless servers that enable users to host Git
20 repositories with ease. Knots are designed for either single
21 or multi-tenant use which is perfect for self-hosting on a
22 Raspberry Pi at home, or larger “community” servers. By
23 default, Tangled provides managed knots where you can host
24 your repositories for free.
25
26 The appview at tangled.org acts as a consolidated "view"
27 into the whole network, allowing users to access, clone and
28 contribute to repositories hosted across different knots
29 seamlessly.
30---
31
32# Quick start guide
33
34## Login or sign up
35
36You can [login](https://tangled.org) by using your AT Protocol
37account. If you are unclear on what that means, simply head
38to the [signup](https://tangled.org/signup) page and create
39an account. By doing so, you will be choosing Tangled as
40your account provider (you will be granted a handle of the
41form `user.tngl.sh`).
42
43In the AT Protocol network, users are free to choose their account
44provider (known as a "Personal Data Service", or PDS), and
45login to applications that support AT accounts.
46
47You can think of it as "one account for all of the atmosphere"!
48
49If you already have an AT account (you may have one if you
50signed up to Bluesky, for example), you can login with the
51same handle on Tangled (so just use `user.bsky.social` on
52the login page).
53
54## Add an SSH key
55
56Once you are logged in, you can start creating repositories
57and pushing code. Tangled supports pushing git repositories
58over SSH.
59
60First, you'll need to generate an SSH key if you don't
61already have one:
62
63```bash
64ssh-keygen -t ed25519 -C "foo@bar.com"
65```
66
67When prompted, save the key to the default location
68(`~/.ssh/id_ed25519`) and optionally set a passphrase.
69
70Copy your public key to your clipboard:
71
72```bash
73# on X11
74cat ~/.ssh/id_ed25519.pub | xclip -sel c
75
76# on wayland
77cat ~/.ssh/id_ed25519.pub | wl-copy
78
79# on macos
80cat ~/.ssh/id_ed25519.pub | pbcopy
81```
82
83Now, navigate to 'Settings' -> 'Keys' and hit 'Add Key',
84paste your public key, give it a descriptive name, and hit
85save.
86
87## Create a repository
88
89Once your SSH key is added, create your first repository:
90
911. Hit the green `+` icon on the topbar, and select
92 repository
932. Enter a repository name
943. Add a description
954. Choose a knotserver to host this repository on
965. Hit create
97
98Knots are self-hostable, lightweight Git servers that can
99host your repository. Unlike traditional code forges, your
100code can live on any server. Read the [Knots](TODO) section
101for more.
102
103## Configure SSH
104
105To ensure Git uses the correct SSH key and connects smoothly
106to Tangled, add this configuration to your `~/.ssh/config`
107file:
108
109```
110Host tangled.org
111 Hostname tangled.org
112 User git
113 IdentityFile ~/.ssh/id_ed25519
114 AddressFamily inet
115```
116
117This tells SSH to use your specific key when connecting to
118Tangled and prevents authentication issues if you have
119multiple SSH keys.
120
121Note that this configuration only works for knotservers that
122are hosted by tangled.org. If you use a custom knot, refer
123to the [Knots](TODO) section.
124
125## Push your first repository
126
127Initialize a new Git repository:
128
129```bash
130mkdir my-project
131cd my-project
132
133git init
134echo "# My Project" > README.md
135```
136
137Add some content and push!
138
139```bash
140git add README.md
141git commit -m "Initial commit"
142git remote add origin git@tangled.org:user.tngl.sh/my-project
143git push -u origin main
144```
145
146That's it! Your code is now hosted on Tangled.
147
148## Migrating an existing repository
149
150Moving your repositories from GitHub, GitLab, Bitbucket, or
151any other Git forge to Tangled is straightforward. You'll
152simply change your repository's remote URL. At the moment,
153Tangled does not have any tooling to migrate data such as
154GitHub issues or pull requests.
155
156First, create a new repository on tangled.org as described
157in the [Quick Start Guide](#create-a-repository).
158
159Navigate to your existing local repository:
160
161```bash
162cd /path/to/your/existing/repo
163```
164
165You can inspect your existing Git remote like so:
166
167```bash
168git remote -v
169```
170
171You'll see something like:
172
173```bash
174origin git@github.com:username/my-project.git (fetch)
175origin git@github.com:username/my-project.git (push)
176```
177
178Update the remote URL to point to tangled:
179
180```bash
181git remote set-url origin git@tangled.org:user.tngl.sh/my-project
182```
183
184Verify the change:
185
186```bash
187git remote -v
188```
189
190You should now see:
191
192```bash
193origin git@tangled.org:user.tngl.sh/my-project (fetch)
194origin git@tangled.org:user.tngl.sh/my-project (push)
195```
196
197Push all your branches and tags to Tangled:
198
199```bash
200git push -u origin --all
201git push -u origin --tags
202```
203
204Your repository is now migrated to Tangled! All commit
205history, branches, and tags have been preserved.
206
207## Mirroring a repository to Tangled
208
209If you want to maintain your repository on multiple forges
210simultaneously, for example, keeping your primary repository
211on GitHub while mirroring to Tangled for backup or
212redundancy, you can do so by adding [multiple remotes](https://git-scm.com/docs/git-push#_remotes).
213
214You can configure your local repository to push to both
215Tangled and, say, GitHub. You may already have the following
216setup:
217
218```bash
219$ git remote -v
220origin git@github.com:username/my-project.git (fetch)
221origin git@github.com:username/my-project.git (push)
222```
223
224Now add Tangled as an additional push URL to the same
225remote:
226
227```bash
228git remote set-url --add --push origin git@tangled.org:user.tngl.sh/my-project
229```
230
231You also need to re-add the original URL as a push
232destination (Git will now use the original URL to fetch only):
233
234```bash
235git remote set-url --add --push origin git@github.com:username/my-project.git
236```
237
238Verify your configuration:
239
240```bash
241$ git remote -v
242origin git@github.com:username/my-project.git (fetch)
243origin git@tangled.org:user.tngl.sh/my-project (push)
244origin git@github.com:username/my-project.git (push)
245```
246
247Notice that there's one fetch URL (the primary remote) and
248two push URLs. Now, whenever you push, Git will
249automatically push to both remotes:
250
251```bash
252git push origin main
253```
254
255This single command pushes your `main` branch to both GitHub
256and Tangled simultaneously.
257
258To push all branches and tags:
259
260```bash
261git push origin --all
262git push origin --tags
263```
264
265If you prefer more control over which remote you push to,
266you can maintain separate remotes:
267
268```bash
269git remote add github git@github.com:username/my-project.git
270git remote add tangled git@tangled.org:user.tngl.sh/my-project
271```
272
273Then push to each explicitly:
274
275```bash
276git push github main
277git push tangled main
278```
279
280# Hosting websites on Tangled
281
282You can serve static websites directly from your git repositories on
283Tangled. If you've used GitHub Pages or Codeberg Pages, this should feel
284familiar.
285
286## Overview
287
288Every user gets a sites domain. If you signed up through Tangled's own
289PDS (`tngl.sh`), your sites domain is automatically
290`<your-handle>.tngl.sh` no setup needed. Otherwise, you can claim a
291`<subdomain>.tngl.io` domain from your settings.
292
293You can serve multiple sites per domain:
294
295- One **index site** served at the root of your domain (e.g.
296 `alice.tngl.sh`)
297- Any number of **sub-path sites** served under the repository name
298 (e.g. `alice.tngl.sh/my-project`)
299
300## Claiming a domain
301
302If you don't have a `tngl.sh` handle, you need to claim a domain before
303publishing sites:
304
3051. Go to **Settings → Sites**
3062. Enter a subdomain (e.g. `alice` to claim `alice.tngl.io`)
3073. Click **claim**
308
309You can only hold one domain at a time. Releasing a domain puts it in a
31030-day cooldown before anyone else can claim it.
311
312## Configuring a site for a repository
313
3141. Navigate to your repository
3152. Go to **Settings → Sites**
3163. Choose a **branch** to deploy from
3174. Set the **deploy directory** — the path within the repository
318 containing your `index.html`. Use `/` for the root, or a subdirectory
319 like `/docs` or `/public`
3205. Choose the **site type**:
321 - **Index site** — served at the root of your domain (e.g.
322 `alice.tngl.sh`)
323 - **Sub-path site** — served under the repository name (e.g.
324 `alice.tngl.sh/my-project`)
3256. Click **save**
326
327The site will be deployed automatically. You can see the status of your
328previous deploys in the **Recent Deploys** section at the bottom of the
329page.
330
331Sites are redeployed automatically on every push to the configured
332branch.
333
334## Custom domains
335
336Tangled currently doesn't support custom domains for sites. This will be
337added in a future update.
338
339## Deploy directory
340
341The deploy directory is the path within your repository that Tangled
342serves as the site root. It must contain an `index.html`.
343
344| Deploy directory | Result |
345|---|---|
346| `/` | Serves the repository root |
347| `/docs` | Serves the `docs/` subdirectory |
348| `/public` | Serves the `public/` subdirectory |
349
350Directories are served with automatic `index.html` resolution -- a
351request to `/about` will serve `/about/index.html` if it exists.
352
353## Site types
354
355| Type | URL |
356|---|---|
357| Index site | `alice.tngl.sh` |
358| Sub-path site | `alice.tngl.sh/my-project` |
359
360Only one repository can be the index site for a given domain at a time.
361If another repository already holds the index site, you will see a
362notice in the settings and only the sub-path option will be available.
363
364## Deploy triggers
365
366A deployment is triggered automatically when:
367
368- You push to the configured branch
369- You change the site configuration (branch, deploy directory, or site
370 type)
371
372## Disabling a site
373
374To stop serving a site, go to **Settings → Sites** in your repository
375and click **Disable**. This removes the site configuration and stops
376serving the site. The deployed files are also deleted from storage.
377
378Releasing your domain from **Settings → Sites** at the account level
379will disable all sites associated with it and delete their files.
380
381
382# Knot self-hosting guide
383
384So you want to run your own knot server? Great! Here are a few prerequisites:
385
3861. A server of some kind (a VPS, a Raspberry Pi, etc.). Preferably running a Linux distribution of some kind.
3872. A (sub)domain name. People generally use `knot.example.com`.
3883. A valid SSL certificate for your domain.
389
390## NixOS
391
392Refer to the [knot
393module](https://tangled.org/tangled.org/core/blob/master/nix/modules/knot.nix)
394for a full list of options. Sample configurations:
395
396- [The test VM](https://tangled.org/tangled.org/core/blob/master/nix/vm.nix#L85)
397- [@pyrox.dev/nix](https://tangled.org/pyrox.dev/nix/blob/d19571cc1b5fe01035e1e6951ec8cf8a476b4dee/hosts/marvin/services/tangled.nix#L15-25)
398
399## Docker
400
401Refer to
402[@tangled.org/knot-docker](https://tangled.org/@tangled.org/knot-docker).
403Note that this is community maintained.
404
405## Manual setup
406
407First, clone this repository:
408
409```
410git clone https://tangled.org/@tangled.org/core
411```
412
413Then, build the `knot` CLI. This is the knot administration
414and operation tool. For the purpose of this guide, we're
415only concerned with these subcommands:
416
417- `knot server`: the main knot server process, typically
418 run as a supervised service
419- `knot guard`: handles role-based access control for git
420 over SSH (you'll never have to run this yourself)
421- `knot keys`: fetches SSH keys associated with your knot;
422 we'll use this to generate the SSH
423 `AuthorizedKeysCommand`
424
425```
426cd core
427export CGO_ENABLED=1
428go build -o knot ./cmd/knot
429```
430
431Next, move the `knot` binary to a location owned by `root` --
432`/usr/local/bin/` is a good choice. Make sure the binary itself is also owned by `root`:
433
434```
435sudo mv knot /usr/local/bin/knot
436sudo chown root:root /usr/local/bin/knot
437```
438
439This is necessary because SSH `AuthorizedKeysCommand` requires [really
440specific permissions](https://stackoverflow.com/a/27638306). The
441`AuthorizedKeysCommand` specifies a command that is run by `sshd` to
442retrieve a user's public SSH keys dynamically for authentication. Let's
443set that up.
444
445```
446sudo tee /etc/ssh/sshd_config.d/authorized_keys_command.conf <<EOF
447Match User git
448 AuthorizedKeysCommand /usr/local/bin/knot keys -o authorized-keys
449 AuthorizedKeysCommandUser nobody
450EOF
451```
452
453Then, reload `sshd`:
454
455```
456sudo systemctl reload ssh
457```
458
459Next, create the `git` user. We'll use the `git` user's home directory
460to store repositories:
461
462```
463sudo adduser git
464```
465
466Create `/home/git/.knot.env` with the following, updating the values as
467necessary. The `KNOT_SERVER_OWNER` should be set to your
468DID, you can find your DID in the [Settings](https://tangled.sh/settings) page.
469
470```
471KNOT_REPO_SCAN_PATH=/home/git
472KNOT_SERVER_HOSTNAME=knot.example.com
473APPVIEW_ENDPOINT=https://tangled.org
474KNOT_SERVER_OWNER=did:plc:foobar
475KNOT_SERVER_INTERNAL_LISTEN_ADDR=127.0.0.1:5444
476KNOT_SERVER_LISTEN_ADDR=127.0.0.1:5555
477```
478
479If you run a Linux distribution that uses systemd, you can
480use the provided service file to run the server. Copy
481[`knotserver.service`](https://tangled.org/tangled.org/core/blob/master/systemd/knotserver.service)
482to `/etc/systemd/system/`. Then, run:
483
484```
485systemctl enable knotserver
486systemctl start knotserver
487```
488
489The last step is to configure a reverse proxy like Nginx or Caddy to front your
490knot. Here's an example configuration for Nginx:
491
492```
493server {
494 listen 80;
495 listen [::]:80;
496 server_name knot.example.com;
497
498 location / {
499 proxy_pass http://localhost:5555;
500 proxy_set_header Host $host;
501 proxy_set_header X-Real-IP $remote_addr;
502 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
503 proxy_set_header X-Forwarded-Proto $scheme;
504 }
505
506 # wss endpoint for git events
507 location /events {
508 proxy_set_header X-Forwarded-For $remote_addr;
509 proxy_set_header Host $http_host;
510 proxy_set_header Upgrade websocket;
511 proxy_set_header Connection Upgrade;
512 proxy_pass http://localhost:5555;
513 }
514 # additional config for SSL/TLS go here.
515}
516
517```
518
519Remember to use Let's Encrypt or similar to procure a certificate for your
520knot domain.
521
522You should now have a running knot server! You can finalize
523your registration by hitting the `verify` button on the
524[/settings/knots](https://tangled.org/settings/knots) page. This simply creates
525a record on your PDS to announce the existence of the knot.
526
527### Custom paths
528
529(This section applies to manual setup only. Docker users should edit the mounts
530in `docker-compose.yml` instead.)
531
532Right now, the database and repositories of your knot lives in `/home/git`. You
533can move these paths if you'd like to store them in another folder. Be careful
534when adjusting these paths:
535
536- Stop your knot when moving data (e.g. `systemctl stop knotserver`) to prevent
537 any possible side effects. Remember to restart it once you're done.
538- Make backups before moving in case something goes wrong.
539- Make sure the `git` user can read and write from the new paths.
540
541#### Database
542
543As an example, let's say the current database is at `/home/git/knotserver.db`,
544and we want to move it to `/home/git/database/knotserver.db`.
545
546Copy the current database to the new location. Make sure to copy the `.db-shm`
547and `.db-wal` files if they exist.
548
549```
550mkdir /home/git/database
551cp /home/git/knotserver.db* /home/git/database
552```
553
554In the environment (e.g. `/home/git/.knot.env`), set `KNOT_SERVER_DB_PATH` to
555the new file path (_not_ the directory):
556
557```
558KNOT_SERVER_DB_PATH=/home/git/database/knotserver.db
559```
560
561#### Repositories
562
563As an example, let's say the repositories are currently in `/home/git`, and we
564want to move them into `/home/git/repositories`.
565
566Create the new folder, then move the existing repositories (if there are any):
567
568```
569mkdir /home/git/repositories
570# move all DIDs into the new folder; these will vary for you!
571mv /home/git/did:plc:wshs7t2adsemcrrd4snkeqli /home/git/repositories
572```
573
574In the environment (e.g. `/home/git/.knot.env`), update `KNOT_REPO_SCAN_PATH`
575to the new directory:
576
577```
578KNOT_REPO_SCAN_PATH=/home/git/repositories
579```
580
581Similarly, update your `sshd` `AuthorizedKeysCommand` to use the updated
582repository path:
583
584```
585sudo tee /etc/ssh/sshd_config.d/authorized_keys_command.conf <<EOF
586Match User git
587 AuthorizedKeysCommand /usr/local/bin/knot keys -o authorized-keys -git-dir /home/git/repositories
588 AuthorizedKeysCommandUser nobody
589EOF
590```
591
592Make sure to restart your SSH server!
593
594#### MOTD (message of the day)
595
596To configure the MOTD used ("Welcome to this knot!" by default), edit the
597`/home/git/motd` file:
598
599```
600printf "Hi from this knot!\n" > /home/git/motd
601```
602
603Note that you should add a newline at the end if setting a non-empty message
604since the knot won't do this for you.
605
606## Secure Mode
607
608Secure Mode isolates each `git` subprocess to the repository it is
609operating on, using two mechanisms:
610
611- **Linux Landlock** restricts the filesystem paths the subprocess
612 can access -- it can only read/write its own repository and the
613 system directories it needs to run.
614- **UID isolation** runs each subprocess as a virtual UID assigned
615 to the repository owner, so that repositories belonging to
616 different owners are isolated from each other at the OS level
617 even if Landlock were somehow bypassed.
618
619Secure Mode requires:
620
621- Linux kernel >= 5.19 (Landlock V2). This is the minimum needed
622 for `git push` to work, because receive-pack's quarantine
623 migration uses cross-directory rename which requires the
624 Landlock `REFER` access right (added in V2). Kernels 5.13-5.18
625 support Landlock V1 and clones will work, but pushes will fail
626 with cross-device link errors. On kernels without any Landlock
627 support (< 5.13), the sandbox call is a no-op: UID isolation
628 still applies but no filesystem restriction is enforced.
629- `CAP_SETUID`, `CAP_SETGID`, and `CAP_CHOWN` available to the
630 knot process. The NixOS module grants these automatically; for
631 manual setups see the `setcap` step below.
632
633### NixOS
634
635Add `server.secureMode = true;` to your knot module configuration:
636
637```nix
638services.tangled.knot = {
639 server.secureMode = true;
640 # ... other options
641};
642```
643
644The NixOS module handles everything else automatically:
645
646- Grants the required capabilities to the knot service via
647 `AmbientCapabilities` in the systemd unit.
648- Installs a capability-bearing wrapper at
649 `/run/wrappers/bin/knot` via `security.wrappers`, so that
650 SSH-invoked git operations (pushes) also run under the correct
651 UID without requiring the service to run as root.
652- Runs `knot migrate-isolation` at service start to chown
653 existing repositories to their virtual UIDs.
654
655### Manual setup
656
657**Step 1.** Grant the required capabilities to the knot binary.
658This allows the knot process to switch to virtual UIDs at runtime
659without running as root. You will need to repeat this step
660whenever the binary is updated.
661
662```
663sudo setcap cap_setuid,cap_setgid,cap_chown+eip /usr/local/bin/knot
664```
665
666**Step 2.** Run the migration tool to assign virtual UIDs to all
667existing repositories and set their filesystem permissions. This
668must be run as root:
669
670```
671sudo knot migrate-isolation \
672 --git-dir /home/git \
673 --db /home/git/knotserver.db \
674 --internal-api 127.0.0.1:5444
675```
676
677You can re-run this at any time with `--force` to reapply
678permissions (e.g. after a manual repair or after updating the
679binary).
680
681**Step 2a.** Ensure the home directory is traversable by
682non-group users. Git subprocesses run as virtual UIDs that are
683not in the git group, and they need to resolve
684`$HOME/.config/git/config` to load the global config:
685
686```
687sudo chmod o+x /home/git
688```
689
690This adds only the execute bit, not read -- the virtual UIDs can
691traverse to known paths but cannot list directory contents.
692
693**Step 3.** Enable Secure Mode in your environment file:
694
695```
696KNOT_SERVER_SECURE_MODE=true
697```
698
699Or pass it as a flag:
700
701```
702knot server --secure-mode
703```
704
705**Step 4.** Regenerate the `AuthorizedKeysCommand` with the
706`-secure-mode` flag. This causes `knot keys` to emit guard
707command lines that include `-secure-mode`, so SSH pushes also
708get UID isolation:
709
710```
711sudo tee /etc/ssh/sshd_config.d/authorized_keys_command.conf <<EOF
712Match User git
713 AuthorizedKeysCommand /usr/local/bin/knot keys \
714 -o authorized-keys -secure-mode
715 AuthorizedKeysCommandUser nobody
716EOF
717```
718
719Reload `sshd` after making this change.
720
721> **Note:** the server will refuse to start in Secure Mode if any
722> repositories have not yet been isolation-migrated. Re-run
723> `migrate-isolation` if you see this error.
724
725## Troubleshooting
726
727If you run your own knot, you may run into some of these
728common issues. You can always join the
729[IRC](https://web.libera.chat/#tangled) or
730[Discord](https://chat.tangled.org/) if this section does
731not help.
732
733### Unable to push
734
735If you are unable to push to your knot or repository:
736
7371. First, ensure that you have added your SSH public key to
738 your account
7392. Check to see that your knot has synced the key by running
740 `knot keys`
7413. Check to see if git is supplying the correct private key
742 when pushing: `GIT_SSH_COMMAND="ssh -v" git push ...`
7434. Check to see if `sshd` on the knot is rejecting the push
744 for some reason: `journalctl -xeu ssh` (or `sshd`,
745 depending on your machine). These logs are unavailable if
746 using docker.
7475. Check to see if the knot itself is rejecting the push,
748 depending on your setup, the logs might be in one of the
749 following paths:
750 - `/tmp/knotguard.log`
751 - `/home/git/log`
752 - `/home/git/guard.log`
753
754# Self-hosting an appview
755
756The appview is the web frontend and indexer for a Tangled
757instance. It ingests events from the AT Protocol firehose,
758indexes repositories and social data, and serves the web UI.
759Running your own appview lets you host a fully independent
760Tangled instance scoped to your own users, knots, and
761content.
762
763## NixOS
764
765Refer to the [appview
766module](https://tangled.org/tangled.org/core/blob/master/nix/modules/appview.nix)
767for the full list of options. A minimal NixOS configuration:
768
769```nix
770services.tangled.appview = {
771 enable = true;
772 package = pkgs.appview;
773
774 appviewHost = "git.example.com";
775 appviewName = "My Forge";
776 dbPath = "/var/lib/appview/appview.db";
777
778 environmentFile = "/etc/appview.env";
779 # secrets in the environment file:
780 # TANGLED_COOKIE_SECRET
781 # TANGLED_OAUTH_CLIENT_SECRET
782 # TANGLED_OAUTH_CLIENT_KID
783};
784```
785
786## Project mode
787
788Project mode collapses the URL namespace so the appview
789behaves like a single-project forge rather than a
790multi-user platform. When enabled:
791
792- `/{repo}` is served as `/{user}/{repo}`, where `{user}`
793 is a configured project user (a handle or DID).
794- The home page and global timeline are disabled; `/`
795 serves the project user's profile page instead.
796- The `/signup` route and all signup CTAs are hidden.
797- The sites (static site hosting) settings are hidden.
798
799Enable it in the NixOS module:
800
801```nix
802services.tangled.appview = {
803 enable = true;
804 package = pkgs.appview;
805
806 project = {
807 enable = true;
808 user = "anirudh.fi"; # handle or DID of the project owner
809 };
810};
811```
812
813Or via environment variables:
814
815```bash
816TANGLED_PROJECT_MODE=true
817TANGLED_PROJECT_USER=anirudh.fi
818```
819
820All other routes (settings, notifications, login, search,
821issues, pull requests, pipelines) continue to work as
822normal. Existing `/{user}/{repo}` URLs remain valid and
823need not be updated.
824
825## Caveats
826
827The appview builds its index by consuming the AT Protocol
828Jetstream firehose from the point it starts. It does **not**
829backfill historical data on first run, so repositories,
830users, and social data that existed before your instance
831started will not appear until they produce new events on the
832network (a push, a new issue, a profile edit, etc.).
833
834There is currently no first-party tool to perform a full
835network backfill.
836
837## Custom templates
838
839The appview UI is built from HTML templates embedded in the
840binary at build time. You can override individual templates
841by providing a custom templates directory whose structure
842mirrors `appview/pages/templates/`. Files present in the
843custom directory replace the defaults; everything else
844falls back to the originals.
845
846Point to your custom directory in the NixOS module:
847
848```nix
849services.tangled.appview = {
850 enable = true;
851 package = pkgs.appview;
852
853 project = {
854 enable = true;
855 user = "anirudh.fi";
856 templatesDir = ./custom-templates;
857 };
858};
859```
860
861Your `custom-templates/` directory only needs to contain
862the files you want to override. For example, to replace the
863footer:
864
865```
866custom-templates/
867 layouts/
868 fragments/
869 footerMinimal.html
870```
871
872For local development, copy your custom templates on top of
873the defaults and run with `TANGLED_DEV=true` for live
874reload:
875
876```bash
877cp -rfv custom-templates/* appview/pages/templates/
878TANGLED_DEV=true nix run .#watch-appview
879```
880
881## Custom CSS
882
883For small CSS additions, override `layouts/base.html` in
884your custom templates directory and add a `<style>` block
885after the stylesheet link:
886
887```html
888<link rel="stylesheet" href="/static/tw.css?{{ cssContentHash }}" type="text/css" />
889<style>
890 /* your overrides */
891 :root { --brand: #6366f1; }
892</style>
893```
894
895For full Tailwind customisation, override the static files
896Nix package to run Tailwind with your own `input.css`:
897
898```nix
899services.tangled.appview = {
900 package = pkgs.appview.override {
901 appview-static-files = pkgs.appview-static-files.overrideAttrs (old: {
902 buildCommand = old.buildCommand + ''
903 cat ${./extra.css} >> $out/tw.css
904 '';
905 });
906 };
907};
908```
909
910# Spindles
911
912## Pipelines
913
914Spindle workflows allow you to write CI/CD pipelines in a
915simple format. They're located in the `.tangled/workflows`
916directory at the root of your repository, and are defined
917using YAML.
918
919A workflow has a set of common fields that apply no matter
920which engine you pick:
921
922- [Trigger](#trigger): A **required** field that defines
923 when a workflow should be triggered.
924- [Engine](#engine): A **required** field that defines which
925 engine a workflow should run on.
926- [Clone options](#clone-options): An **optional** field
927 that defines how the repository should be cloned.
928- [Environment](#environment): An **optional** field that
929 allows you to define environment variables.
930- [Steps](#steps): An **optional** field that allows you to
931 define what steps should run in the workflow.
932
933On top of these, each engine has its own options for things
934like dependencies and images. See [Engines](#engines) for
935the per-engine fields.
936
937### Trigger
938
939The first thing to add to a workflow is the trigger, which
940defines when a workflow runs. This is defined using a `when`
941field, which takes in a list of conditions. Each condition
942has the following fields:
943
944- `event`: This is a **required** field that defines when
945 your workflow should run. It's a list that can take one or
946 more of the following values:
947 - `push`: The workflow should run every time a commit is
948 pushed to the repository.
949 - `pull_request`: The workflow should run every time a
950 pull request is made or updated.
951 - `manual`: The workflow can be triggered manually.
952- `branch`: Defines which branches the workflow should run
953 for. If used with the `push` event, commits to the
954 branch(es) listed here will trigger the workflow. If used
955 with the `pull_request` event, updates to pull requests
956 targeting the branch(es) listed here will trigger the
957 workflow. This field has no effect with the `manual`
958 event. Supports glob patterns using `*` and `**` (e.g.,
959 `main`, `develop`, `release-*`). Either `branch` or `tag`
960 (or both) must be specified for `push` events.
961- `tag`: Defines which tags the workflow should run for.
962 Only used with the `push` event - when tags matching the
963 pattern(s) listed here are pushed, the workflow will
964 trigger. This field has no effect with `pull_request` or
965 `manual` events. Supports glob patterns using `*` and `**`
966 (e.g., `v*`, `v1.*`, `release-**`). Either `branch` or
967 `tag` (or both) must be specified for `push` events.
968
969For example, if you'd like to define a workflow that runs
970when commits are pushed to the `main` and `develop`
971branches, or when pull requests that target the `main`
972branch are updated, or manually, you can do so with:
973
974```yaml
975when:
976 - event: ["push", "manual"]
977 branch: ["main", "develop"]
978 - event: ["pull_request"]
979 branch: ["main"]
980```
981
982You can also trigger workflows on tag pushes. For instance,
983to run a deployment workflow when tags matching `v*` are
984pushed:
985
986```yaml
987when:
988 - event: ["push"]
989 tag: ["v*"]
990```
991
992You can even combine branch and tag patterns in a single
993constraint (the workflow triggers if either matches):
994
995```yaml
996when:
997 - event: ["push"]
998 branch: ["main", "release-*"]
999 tag: ["v*", "stable"]
1000```
1001
1002### Engine
1003
1004Next is the engine on which the workflow should run, defined
1005using the **required** `engine` field. The currently
1006supported engines are:
1007
1008- `nixery`: This uses an instance of
1009 [Nixery](https://nixery.dev) to run steps, which allows
1010 you to add [dependencies](#dependencies) from
1011 Nixpkgs (https://github.com/NixOS/nixpkgs). You can
1012 search for packages on https://search.nixos.org, and
1013 there's a pretty good chance the package(s) you're looking
1014 for will be there.
1015 See [Nixery engine](#nixery-engine).
1016- `microvm`: Runs the whole workflow inside its own
1017 microVM. Has configuration features for NixOS images
1018 that will let you enable services, do Docker-in-VM, etc.
1019 See [microVM engine](#microvm-engine).
1020
1021Example:
1022
1023```yaml
1024engine: "nixery"
1025```
1026
1027Each engine also adds its own workflow fields (dependencies,
1028images, services, and so on). These are documented under
1029[Engines](#engines).
1030
1031### Clone options
1032
1033When a workflow starts, the first step is to clone the
1034repository. You can customize this behavior using the
1035**optional** `clone` field. It has the following fields:
1036
1037- `skip`: Setting this to `true` will skip cloning the
1038 repository. This can be useful if your workflow is doing
1039 something that doesn't require anything from the
1040 repository itself. This is `false` by default.
1041- `depth`: This sets the number of commits, or the "clone
1042 depth", to fetch from the repository. For example, if you
1043 set this to 2, the last 2 commits will be fetched. By
1044 default, the depth is set to 1, meaning only the most
1045 recent commit will be fetched, which is the commit that
1046 triggered the workflow.
1047- `submodules`: If you use Git submodules
1048 (https://git-scm.com/book/en/v2/Git-Tools-Submodules)
1049 in your repository, setting this field to `true` will
1050 recursively fetch all submodules. This is `false` by
1051 default.
1052
1053The default settings are:
1054
1055```yaml
1056clone:
1057 skip: false
1058 depth: 1
1059 submodules: false
1060```
1061
1062### Environment
1063
1064The `environment` field allows you define environment
1065variables that will be available throughout the entire
1066workflow. **Do not put secrets here, these environment
1067variables are visible to anyone viewing the repository. You
1068can add secrets for pipelines in your repository's
1069settings.**
1070
1071Example:
1072
1073```yaml
1074environment:
1075 GOOS: "linux"
1076 GOARCH: "arm64"
1077 NODE_ENV: "production"
1078 MY_ENV_VAR: "MY_ENV_VALUE"
1079```
1080
1081By default, the following environment variables are set:
1082
1083- `CI` - Always set to `true` to indicate a CI environment
1084- `TANGLED_PIPELINE_ID` - The AT URI of the current pipeline
1085- `TANGLED_PIPELINE_KIND` - One of `push`, `pull_request` or
1086 `manual`
1087- `TANGLED_REPO_KNOT` - The repository's knot hostname
1088- `TANGLED_REPO_DID` - The DID of the repository owner
1089- `TANGLED_REPO_NAME` - The name of the repository
1090- `TANGLED_REPO_DEFAULT_BRANCH` - The default branch of the
1091 repository
1092- `TANGLED_REPO_URL` - The full URL to the repository
1093
1094These variables are only available when the pipeline is
1095triggered by a push:
1096
1097- `TANGLED_REF` - The full git reference (e.g.,
1098 `refs/heads/main` or `refs/tags/v1.0.0`)
1099- `TANGLED_REF_NAME` - The short name of the reference
1100 (e.g., `main` or `v1.0.0`)
1101- `TANGLED_REF_TYPE` - The type of reference, either
1102 `branch` or `tag`
1103- `TANGLED_SHA` - The commit SHA that triggered the pipeline
1104- `TANGLED_COMMIT_SHA` - Alias for `TANGLED_SHA`
1105
1106These variables are only available when the pipeline is
1107triggered by a pull request:
1108
1109- `TANGLED_PR_SOURCE_BRANCH` - The source branch of the pull
1110 request
1111- `TANGLED_PR_TARGET_BRANCH` - The target branch of the pull
1112 request
1113- `TANGLED_PR_SOURCE_SHA` - The commit SHA of the source
1114 branch
1115
1116### Steps
1117
1118The `steps` field allows you to define what steps should run
1119in the workflow. It's a list of step objects, each with the
1120following fields:
1121
1122- `name`: This field allows you to give your step a name.
1123 This name is visible in your workflow runs, and is used to
1124 describe what the step is doing.
1125- `command`: This field allows you to define a command to
1126 run in that step. The step is run in a Bash shell, and the
1127 logs from the command will be visible in the pipelines
1128 page on the Tangled website. Any dependencies you added in
1129 your engine's section (see [Engines](#engines)) will be
1130 available to use here.
1131- `environment`: Similar to the global
1132 [environment](#environment) config, this **optional**
1133 field is a key-value map that allows you to set
1134 environment variables for the step. **Do not put secrets
1135 here, these environment variables are visible to anyone
1136 viewing the repository. You can add secrets for pipelines
1137 in your repository's settings.**
1138
1139Example:
1140
1141```yaml
1142steps:
1143 - name: "Build backend"
1144 command: "go build"
1145 environment:
1146 GOOS: "darwin"
1147 GOARCH: "arm64"
1148 - name: "Build frontend"
1149 command: "npm run build"
1150 environment:
1151 NODE_ENV: "production"
1152```
1153
1154## Engines
1155
1156The common fields above apply to every workflow. Each engine
1157then adds its own fields on top. Pick an engine with the
1158[`engine`](#engine) field and use the matching section below.
1159
1160### Nixery engine
1161
1162#### Dependencies
1163
1164When you're running a workflow you'll usually need additional
1165dependencies. The `dependencies` field lets you define which
1166dependencies to get, and from where. It's a key-value map,
1167with the key being the registry to fetch dependencies from,
1168and the value being the list of dependencies to fetch.
1169
1170The registry URL syntax can be found [on the nix
1171manual](https://nix.dev/manual/nix/2.18/command-ref/new-cli/nix3-registry-add).
1172
1173Say you want to fetch Node.js and Go from `nixpkgs`, and a
1174package called `my_pkg` you've made from your own registry
1175at your repository at
1176`https://tangled.org/@example.com/my_pkg`. You can define
1177those dependencies like so:
1178
1179```yaml
1180dependencies:
1181 # nixpkgs
1182 nixpkgs:
1183 - nodejs
1184 - go
1185 # unstable
1186 nixpkgs/nixpkgs-unstable:
1187 - bun
1188 # custom registry
1189 git+https://tangled.org/@example.com/my_pkg:
1190 - my_pkg
1191```
1192
1193Now these dependencies are available to use in your
1194workflow!
1195
1196#### Complete nixery workflow
1197
1198```yaml
1199# .tangled/workflows/build.yml
1200
1201when:
1202 - event: ["push", "manual"]
1203 branch: ["main", "develop"]
1204 - event: ["pull_request"]
1205 branch: ["main"]
1206
1207engine: "nixery"
1208
1209# using the default values
1210clone:
1211 skip: false
1212 depth: 1
1213 submodules: false
1214
1215dependencies:
1216 # nixpkgs
1217 nixpkgs:
1218 - nodejs
1219 - go
1220 # custom registry
1221 git+https://tangled.org/@example.com/my_pkg:
1222 - my_pkg
1223
1224environment:
1225 GOOS: "linux"
1226 GOARCH: "arm64"
1227 NODE_ENV: "production"
1228 MY_ENV_VAR: "MY_ENV_VALUE"
1229
1230steps:
1231 - name: "Build backend"
1232 command: "go build"
1233 environment:
1234 GOOS: "darwin"
1235 GOARCH: "arm64"
1236 - name: "Build frontend"
1237 command: "npm run build"
1238 environment:
1239 NODE_ENV: "production"
1240```
1241
1242If you want another example of a workflow, you can look at
1243the one [Tangled uses to build the
1244project](https://tangled.org/@tangled.org/core/blob/master/.tangled/workflows/build.yml).
1245
1246### microVM engine
1247
1248#### Image
1249
1250A workflow picks the image to boot with the top-level `image`
1251field:
1252
1253```yaml
1254engine: microvm
1255image: nixos
1256```
1257
1258There are two flavours of images:
1259
1260- **NixOS images** (e.g. `nixos`): the whole guest is built
1261 with Nix, so you can configure it from the workflow file
1262 itself. The `dependencies`, `services`, `virtualisation`,
1263 `registry` and `caches` fields below are all understood
1264 here, and the guest builds and activates that configuration
1265 before any of your steps run.
1266- **Non-NixOS images** (e.g. `alpine`): there's no NixOS to
1267 configure, so the workflow-level config fields above have
1268 no effect. You still get a full machine to run steps in.
1269
1270The available image names depend on what the spindle operator
1271has installed. `nixos` and `alpine` are examples. If `image`
1272is omitted, the spindle's configured default image is used.
1273
1274#### Dependencies
1275
1276On the microVM engine, `dependencies` is a flat list of
1277packages that are made available to every step. This field
1278only applies to **NixOS images**; for other images you can
1279use the package manager included in a step.
1280
1281The guest builds a [`nix develop`](https://nix.dev/manual/nix/2.18/command-ref/new-cli/nix3-develop)-style
1282devshell from your dependencies and uses it for each step,
1283so you can, for example, add `pkg-config` and `openssl` and
1284have the `openssl-sys` crate while compiling a Rust project
1285just work.
1286
1287A bare name like `go` is looked up in nixpkgs. You can also
1288point at any flake with the `flakeref#attr` syntax, so
1289`github:nixos/nixpkgs#hello` pulls `hello` straight out of
1290that flake.
1291
1292```yaml
1293dependencies:
1294 - go
1295 - github:nixos/nixpkgs#hello
1296```
1297
1298#### Registry
1299
1300The `registry` field remaps flake references, the same way
1301`nix registry` does. This lets you pin or alias the flakes
1302used by `dependencies`.
1303
1304For example, pin `nixpkgs` to `nixos-unstable` so that the
1305bare `go` above resolves from unstable, and alias your own
1306flake so you can use `myflake#tool` in `dependencies`:
1307
1308```yaml
1309registry:
1310 nixpkgs: github:nixos/nixpkgs/nixos-unstable
1311 myflake: github:me/x
1312```
1313
1314#### Caches
1315
1316The `caches` field is a map of Nix binary cache URL to its
1317trusted public key. These are fed into the spindle's read
1318proxy, so the guest can substitute prebuilt paths from them
1319instead of building everything from scratch.
1320
1321```yaml
1322caches:
1323 https://nix-community.cachix.org: "nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs="
1324```
1325
1326#### Services and virtualisation
1327
1328The `services` and `virtualisation` fields are passed straight
1329through to NixOS. Anything you could write under
1330`services.*` or `virtualisation.*` in a NixOS configuration,
1331you can write here, and it's brought up before any of your
1332steps run.
1333
1334As a convenience, `true` works as shorthand for
1335`.enable = true` anywhere an `enable` option exists (e.g.
1336`virtualisation.docker: true`).
1337
1338```yaml
1339services:
1340 postgresql:
1341 enable: true
1342 ensureDatabases: ["spindle-workflow"]
1343 ensureUsers:
1344 - name: spindle-workflow
1345 ensureDBOwnership: true
1346
1347virtualisation:
1348 docker: true
1349```
1350
1351#### Recipes
1352
1353##### Lint, test and build a Node project
1354
1355```yaml
1356when:
1357 - event: ["push", "pull_request"]
1358 branch: ["main"]
1359
1360engine: microvm
1361image: nixos
1362
1363dependencies:
1364 - pnpm
1365
1366steps:
1367 - name: "Install dependencies"
1368 command: pnpm install --frozen-lockfile
1369 - name: "Lint and test"
1370 command: |
1371 pnpm run lint
1372 pnpm test
1373 - name: "Build"
1374 command: pnpm run build
1375```
1376
1377##### Check formatting
1378
1379```yaml
1380when:
1381 - event: ["push", "pull_request"]
1382 branch: ["main"]
1383
1384engine: microvm
1385image: alpine # slimmer image for checking the formatting
1386
1387steps:
1388 - name: "Install go"
1389 command: apk add go
1390 - name: "Check formatting"
1391 command: test -z $(gofmt -l .)
1392```
1393
1394##### Build a Rust project that links OpenSSL
1395
1396```yaml
1397when:
1398 - event: ["push", "pull_request"]
1399 branch: ["main"]
1400
1401engine: microvm
1402image: nixos
1403
1404dependencies:
1405 - gcc
1406 - cargo
1407 - rustc
1408 - clippy
1409 - rustfmt
1410 - pkg-config # exports PKG_CONFIG_PATH for the libraries below
1411 - openssl # the C library + headers openssl-sys links against
1412
1413steps:
1414 - name: "Check formatting"
1415 command: cargo fmt --check
1416 - name: "Clippy"
1417 command: cargo clippy --all-targets -- -D warnings
1418 - name: "Test"
1419 command: cargo test --all
1420 - name: "Release build"
1421 command: cargo build --release
1422```
1423
1424##### Run migrations and integration tests against PostgreSQL
1425
1426```yaml
1427when:
1428 - event: ["push", "pull_request"]
1429 branch: ["main"]
1430
1431engine: microvm
1432image: nixos
1433
1434environment:
1435 DATABASE_URL: "postgresql:///spindle-workflow?host=/run/postgresql"
1436
1437dependencies:
1438 - gcc
1439 - cargo
1440 - rustc
1441 - pkg-config
1442 - openssl
1443 - sqlx-cli
1444
1445services:
1446 postgresql:
1447 enable: true
1448 # has to be same name as the user for peer auth to work automatically
1449 ensureDatabases: ["spindle-workflow"]
1450 ensureUsers:
1451 - name: spindle-workflow
1452 ensureDBOwnership: true
1453
1454steps:
1455 - name: "Run migrations"
1456 command: sqlx migrate run
1457 - name: "Integration tests"
1458 command: cargo test --all
1459```
1460
1461##### Build and push a Docker image on tag
1462
1463```yaml
1464when:
1465 - event: ["push"]
1466 tag: ["v*"]
1467
1468engine: microvm
1469image: nixos
1470
1471virtualisation:
1472 docker: true
1473
1474steps:
1475 - name: "Build and push to ghcr.io"
1476 command: |
1477 set -euo pipefail
1478
1479 echo "$REGISTRY_TOKEN" | docker login ghcr.io -u "$REGISTRY_USER" --password-stdin
1480 image="ghcr.io/$REGISTRY_USER/myapp:$TANGLED_REF_NAME"
1481
1482 docker build -t "$image" -t "ghcr.io/$REGISTRY_USER/myapp:latest" .
1483 docker push "$image"
1484 docker push "ghcr.io/$REGISTRY_USER/myapp:latest"
1485```
1486
1487##### Deploy to Cloudflare Workers on tag
1488
1489```yaml
1490# .tangled/workflows/deploy.yml
1491when:
1492 - event: ["push"]
1493 tag: ["v*"]
1494
1495engine: microvm
1496image: nixos
1497
1498dependencies:
1499 - pnpm
1500
1501steps:
1502 - name: "Install dependencies"
1503 command: pnpm install --frozen-lockfile
1504 - name: "Deploy worker"
1505 # `wrangler` picks up `CLOUDFLARE_API_TOKEN` from the env.
1506 # set it under **Settings → Secrets**.
1507 command: pnpm exec wrangler deploy
1508```
1509
1510##### Publish a release artifact
1511
1512```yaml
1513when:
1514 - event: ["push"]
1515 tag: ["v*"] # trigger on versions
1516
1517engine: microvm
1518image: nixos
1519
1520dependencies:
1521 - go
1522
1523steps:
1524 - name: "Build release binary"
1525 command: |
1526 mkdir -p dist
1527 CGO_ENABLED=0 go build -trimpath -ldflags "-s -w" -o dist/myapp ./cmd/myapp
1528
1529 - name: "Publish artifact record"
1530 command: |
1531 set -euo pipefail
1532 # change this if you're not on `tngl.sh`
1533 PDS="https://tngl.sh"
1534 # also update this to your handle or did
1535 ATP_IDENTIFIER="user.tngl.sh"
1536 ARTIFACT_PATH="dist/myapp"
1537 ARTIFACT_NAME="myapp"
1538
1539 # set `ATP_APP_PASSWORD` under **Settings → Secrets**
1540 session=$(curl -fsS -X POST "$PDS/xrpc/com.atproto.server.createSession" \
1541 -H "Content-Type: application/json" \
1542 -d "{\"identifier\":\"$ATP_IDENTIFIER\",\"password\":\"$ATP_APP_PASSWORD\"}")
1543 jwt=$(echo "$session" | jq -r .accessJwt)
1544 did=$(echo "$session" | jq -r .did)
1545
1546 # upload the binary as a blob
1547 blob=$(curl -fsS -X POST "$PDS/xrpc/com.atproto.repo.uploadBlob" \
1548 -H "Authorization: Bearer $jwt" \
1549 -H "Content-Type: application/octet-stream" \
1550 --data-binary @"$ARTIFACT_PATH")
1551
1552 # note that this requires an annotated tag (`git tag -a v1.0.0 -m ...`)
1553 tag_hash=$(git rev-parse "$TANGLED_REF_NAME^{tag}")
1554 tag_bytes=$(printf '%s' "$tag_hash" | xxd -r -p | base64 | tr -d '=')
1555
1556 # the sh.tangled.repo.artifact record for your artifact
1557 record=$(jq -n \
1558 --arg did "$did" \
1559 --arg tag "$tag_bytes" \
1560 --arg name "$ARTIFACT_NAME" \
1561 --arg repo "$TANGLED_REPO_URL" \
1562 --arg created "$(date -Iseconds)" \
1563 --argjson blob "$(echo "$blob" | jq .blob)" '{
1564 repo: $did,
1565 collection: "sh.tangled.repo.artifact",
1566 validate: false,
1567 record: {
1568 "$type": "sh.tangled.repo.artifact",
1569 tag: {"$bytes": $tag},
1570 name: $name,
1571 repo: $repo,
1572 artifact: $blob,
1573 createdAt: $created
1574 }
1575 }')
1576
1577 # create the record on the PDS
1578 curl -fsS -X POST "$PDS/xrpc/com.atproto.repo.createRecord" \
1579 -H "Authorization: Bearer $jwt" \
1580 -H "Content-Type: application/json" \
1581 -d "$record"
1582```
1583
1584## Self-hosting guide
1585
1586### Prerequisites
1587
1588- Go
1589- For the **nixery** engine: Docker (or Podman with Docker
1590 compatibility enabled).
1591- For the **microVM** engine: a Linux host with KVM, plus the
1592 microVM host dependencies described in [Running microVM
1593 workflows](#running-microvm-workflows).
1594
1595### Configuration
1596
1597Spindle is configured using environment variables. The following environment variables are available:
1598
1599- `SPINDLE_SERVER_LISTEN_ADDR`: The address the server listens on (default: `"0.0.0.0:6555"`).
1600- `SPINDLE_SERVER_DB_PATH`: The path to the SQLite database file (default: `"spindle.db"`).
1601- `SPINDLE_SERVER_HOSTNAME`: The hostname of the server (required).
1602- `SPINDLE_SERVER_JETSTREAM_ENDPOINT`: The endpoint of the Jetstream server (default: `"wss://jetstream1.us-west.bsky.network/subscribe"`).
1603- `SPINDLE_SERVER_DEV`: A boolean indicating whether the server is running in development mode (default: `false`).
1604- `SPINDLE_SERVER_OWNER`: The DID of the owner (required).
1605- `SPINDLE_SERVER_LOG_DIR`: The directory to store workflow logs (default: `"/var/log/spindle"`).
1606- `SPINDLE_SERVER_DOCKER_SOCKET`: Path to Docker socket to expose to invoked Spindle containers (default: `""`).
1607- `SPINDLE_PIPELINES_NIXERY`: The Nixery URL (default: `"nixery.tangled.sh"`).
1608- `SPINDLE_PIPELINES_WORKFLOW_TIMEOUT`: The default workflow timeout (default: `"5m"`).
1609
1610For the microVM engine, the following are also available
1611(prefix `SPINDLE_MICROVM_PIPELINES_`):
1612
1613- `SPINDLE_MICROVM_PIPELINES_IMAGE_DIR`: Directory containing
1614 microVM images (**required** to use the engine). See
1615 [Running microVM workflows](#running-microvm-workflows).
1616- `SPINDLE_MICROVM_PIPELINES_DEFAULT_IMAGE`: Image used when a
1617 workflow doesn't set `image` (default: `"nixos-x86_64"`).
1618- `SPINDLE_MICROVM_PIPELINES_OVERLAY_DIR`: Where per-workflow
1619 temporary disks are created (default: the system temp dir).
1620- `SPINDLE_MICROVM_PIPELINES_ENABLE_KVM`: Use KVM hardware
1621 acceleration (default: `true`). Without KVM, guests fall
1622 back to slow software emulation.
1623- `SPINDLE_MICROVM_PIPELINES_WORKFLOW_TIMEOUT`: Default
1624 workflow timeout (default: `"5m"`).
1625
1626Optional resource limits (a value of `0` disables that
1627limit). The limits cap usage across all running microVM
1628workflows:
1629
1630- `SPINDLE_MICROVM_PIPELINES_MAX_TOTAL_MEMORY_MIB`
1631- `SPINDLE_MICROVM_PIPELINES_MAX_TOTAL_VCPUS`
1632- `SPINDLE_MICROVM_PIPELINES_MAX_TOTAL_DISK_MIB`
1633
1634Optional cgroup enforcement:
1635
1636- `SPINDLE_MICROVM_PIPELINES_ENABLE_CGROUPS`: Place each
1637 workflow's QEMU and slirp4netns in a per-workflow cgroup=
1638 (default: `false`).
1639- `SPINDLE_MICROVM_PIPELINES_CGROUP_PARENT`: Parent cgroup;
1640 `self` resolves the spindle service's own cgroup (default:
1641 `"self"`).
1642- `SPINDLE_MICROVM_PIPELINES_CGROUP_PIDS_MAX`: Max processes
1643 per workflow cgroup (default: `4096`).
1644- `SPINDLE_MICROVM_PIPELINES_CGROUP_SWAP_MAX_MIB`: Max swap
1645 per workflow cgroup (default: `0`, no swap).
1646- `SPINDLE_MICROVM_PIPELINES_CGROUP_SUPERVISOR_MEMORY_MIN_MIB`:
1647 Memory protected for spindle itself so it isn't OOM-killed
1648 before the workflows (default: `512`).
1649
1650To push paths built inside microVMs back to a shared Nix
1651cache (and read from it), configure the cache (prefix
1652`SPINDLE_NIX_CACHE_`):
1653
1654- `SPINDLE_NIX_CACHE_READ_URLS`: Comma-separated binary cache
1655 URLs the guest reads from.
1656- `SPINDLE_NIX_CACHE_TRUSTED_PUBLIC_KEYS`: Comma-separated
1657 trusted public keys for those caches.
1658- `SPINDLE_NIX_CACHE_UPLOAD_URL`: Cache URL that paths built
1659 in the guest are uploaded to.
1660
1661### Running spindle
1662
16631. **Set the environment variables.** For example:
1664
1665 ```shell
1666 export SPINDLE_SERVER_HOSTNAME="your-hostname"
1667 export SPINDLE_SERVER_OWNER="your-did"
1668 ```
1669
16702. **Build the Spindle binary.**
1671
1672 ```shell
1673 cd core
1674 go mod download
1675 go build -o cmd/spindle/spindle cmd/spindle/main.go
1676 ```
1677
16783. **Create the log directory.**
1679
1680 ```shell
1681 sudo mkdir -p /var/log/spindle
1682 sudo chown $USER:$USER -R /var/log/spindle
1683 ```
1684
16854. **Run the Spindle binary.**
1686
1687 ```shell
1688 ./cmd/spindle/spindle
1689 ```
1690
1691Spindle will now start, connect to the Jetstream server, and begin processing pipelines.
1692
1693### Running microVM workflows
1694
1695The microVM engine needs a few extra things on the host, and
1696it needs images to boot.
1697
1698#### Host dependencies
1699
1700microVM workflows depend on a handful of host tools and
1701devices. spindle checks for the ones an image needs right
1702before it launches, so a missing dependency surfaces as a
1703clear error. You'll need:
1704
1705- `qemu`: the runner. The QEMU binary for the image's arch
1706 must be present (e.g. `qemu-system-x86_64`).
1707- `mkfs.ext4` (from `e2fsprogs`): to format the per-workflow
1708 writable volumes.
1709- [`slirp4netns`](https://github.com/rootless-containers/slirp4netns#install),
1710 `ip` (from `iproute2`), `mount` and `unshare` (from `util-linux`):
1711 used to sandbox guest networking.
1712- `/dev/kvm`: for hardware acceleration (unless you disable
1713 KVM with `SPINDLE_MICROVM_PIPELINES_ENABLE_KVM=false`).
1714- `/dev/vhost-vsock`: the guest agent talks to spindle over
1715 vsock.
1716
1717On NixOS, the [spindle
1718module](https://tangled.org/tangled.org/core/blob/master/nix/modules/spindle.nix)
1719puts `qemu`, `e2fsprogs`, `slirp4netns`, `iproute2` and
1720`util-linux` on the service's `PATH` for you.
1721
1722#### Building images
1723
1724Images are built with Nix. The flake exposes packages for the
1725two stock images (use the `-tarball` prefixed ones for a gzipped
1726tarball you can copy to another host):
1727
1728```shell
1729# a NixOS image
1730nix build .#spindle-nixos-image
1731# an Alpine image
1732nix build .#spindle-alpine-image
1733```
1734
1735#### Installing images
1736
1737Spindle looks for images in
1738`SPINDLE_MICROVM_PIPELINES_IMAGE_DIR`. An image is resolved by
1739the name a workflow puts in its `image` field, matched
1740literally against what's on disk:
1741
17421. a directory `<name>/` containing a `spec.json` (next to the
1743 kernel/initrd/store-disk), or
17442. a flat `<name>.json` self-contained spec.
1745
1746Resolution depends only on the name and what's on disk, never
1747on the host doing the resolving, so the same workflow resolves
1748to the same image on every spindle. If you keep multiple
1749arches side by side, you can name them `<name>-<arch>` (e.g.
1750`nixos-x86_64`, `alpine-aarch64`); the suffix is just part of
1751the name. To make a name like `nixos` work if you are hosting
1752multiple arches, you can use symlinks.
1753
1754On NixOS, you'll most likely want to use `systemd.tmpfiles.rules`
1755to set these up declaratively.
1756
1757## Architecture
1758
1759Spindle is a small CI runner service. Here's a high-level overview of how it operates:
1760
1761- Listens for [`sh.tangled.spindle.member`](/lexicons/spindle/member.json) and
1762 [`sh.tangled.repo`](/lexicons/repo.json) records on the Jetstream.
1763- When a new repo record comes through (typically when you add a spindle to a
1764 repo from the settings), spindle then resolves the underlying knot and
1765 subscribes to repo events (see:
1766 [`sh.tangled.pipeline`](/lexicons/pipeline.json)).
1767- The spindle engine then handles execution of the pipeline, with results and
1768 logs beamed on the spindle event stream over WebSocket
1769
1770### The engines
1771
1772Spindle has two execution backends, picked per-workflow with
1773the [`engine`](#engine) field:
1774
1775- **nixery**: executes each step in a fresh Docker container
1776 (Podman works too, if Docker compatibility is enabled so
1777 that `/run/docker.sock` is created), with state persisted
1778 across steps within the `/tangled/workspace` directory. The
1779 base image for the container is constructed on the fly using
1780 [Nixery](https://nixery.dev), which is/rhandy for caching
1781 layers for frequently used packages.
1782- **microvm**: runs the whole workflow inside its own
1783 microVM, supporting different images, with extra
1784 configuration for NixOS images (e.g. services in workflow file)
1785 See the [engine
1786 README](https://tangled.org/tangled.org/core/blob/master/spindle/engines/microvm/README.md)
1787 for the architecture in depth.
1788
1789The pipeline manifest is [specified here](https://docs.tangled.org/spindles.html#pipelines).
1790
1791## Secrets with openbao
1792
1793This document covers setting up spindle to use OpenBao for secrets
1794management via OpenBao Proxy instead of the default SQLite backend.
1795
1796### Overview
1797
1798Spindle now uses OpenBao Proxy for secrets management. The proxy handles
1799authentication automatically using AppRole credentials, while spindle
1800connects to the local proxy instead of directly to the OpenBao server.
1801
1802This approach provides better security, automatic token renewal, and
1803simplified application code.
1804
1805### Installation
1806
1807Install OpenBao from Nixpkgs:
1808
1809```bash
1810nix shell nixpkgs#openbao # for a local server
1811```
1812
1813### Setup
1814
1815The setup process can is documented for both local development and production.
1816
1817#### Local development
1818
1819Start OpenBao in dev mode:
1820
1821```bash
1822bao server -dev -dev-root-token-id="root" -dev-listen-address=127.0.0.1:8201
1823```
1824
1825This starts OpenBao on `http://localhost:8201` with a root token.
1826
1827Set up environment for bao CLI:
1828
1829```bash
1830export BAO_ADDR=http://localhost:8200
1831export BAO_TOKEN=root
1832```
1833
1834#### Production
1835
1836You would typically use a systemd service with a
1837configuration file. Refer to
1838[@tangled.org/infra](https://tangled.org/@tangled.org/infra)
1839for how this can be achieved using Nix.
1840
1841Then, initialize the bao server:
1842
1843```bash
1844bao operator init -key-shares=1 -key-threshold=1
1845```
1846
1847This will print out an unseal key and a root key. Save them
1848somewhere (like a password manager). Then unseal the vault
1849to begin setting it up:
1850
1851```bash
1852bao operator unseal <unseal_key>
1853```
1854
1855All steps below remain the same across both dev and
1856production setups.
1857
1858#### Configure openbao server
1859
1860Create the spindle KV mount:
1861
1862```bash
1863bao secrets enable -path=spindle -version=2 kv
1864```
1865
1866Set up AppRole authentication and policy:
1867
1868Create a policy file `spindle-policy.hcl`:
1869
1870```hcl
1871# Full access to spindle KV v2 data
1872path "spindle/data/*" {
1873 capabilities = ["create", "read", "update", "delete"]
1874}
1875
1876# Access to metadata for listing and management
1877path "spindle/metadata/*" {
1878 capabilities = ["list", "read", "delete", "update"]
1879}
1880
1881# Allow listing at root level
1882path "spindle/" {
1883 capabilities = ["list"]
1884}
1885
1886# Required for connection testing and health checks
1887path "auth/token/lookup-self" {
1888 capabilities = ["read"]
1889}
1890```
1891
1892Apply the policy and create an AppRole:
1893
1894```bash
1895bao policy write spindle-policy spindle-policy.hcl
1896bao auth enable approle
1897bao write auth/approle/role/spindle \
1898 token_policies="spindle-policy" \
1899 token_ttl=1h \
1900 token_max_ttl=4h \
1901 bind_secret_id=true \
1902 secret_id_ttl=0 \
1903 secret_id_num_uses=0
1904```
1905
1906Get the credentials:
1907
1908```bash
1909# Get role ID (static)
1910ROLE_ID=$(bao read -field=role_id auth/approle/role/spindle/role-id)
1911
1912# Generate secret ID
1913SECRET_ID=$(bao write -f -field=secret_id auth/approle/role/spindle/secret-id)
1914
1915echo "Role ID: $ROLE_ID"
1916echo "Secret ID: $SECRET_ID"
1917```
1918
1919#### Create proxy configuration
1920
1921Create the credential files:
1922
1923```bash
1924# Create directory for OpenBao files
1925mkdir -p /tmp/openbao
1926
1927# Save credentials
1928echo "$ROLE_ID" > /tmp/openbao/role-id
1929echo "$SECRET_ID" > /tmp/openbao/secret-id
1930chmod 600 /tmp/openbao/role-id /tmp/openbao/secret-id
1931```
1932
1933Create a proxy configuration file `/tmp/openbao/proxy.hcl`:
1934
1935```hcl
1936# OpenBao server connection
1937vault {
1938 address = "http://localhost:8200"
1939}
1940
1941# Auto-Auth using AppRole
1942auto_auth {
1943 method "approle" {
1944 mount_path = "auth/approle"
1945 config = {
1946 role_id_file_path = "/tmp/openbao/role-id"
1947 secret_id_file_path = "/tmp/openbao/secret-id"
1948 }
1949 }
1950
1951 # Optional: write token to file for debugging
1952 sink "file" {
1953 config = {
1954 path = "/tmp/openbao/token"
1955 mode = 0640
1956 }
1957 }
1958}
1959
1960# Proxy listener for spindle
1961listener "tcp" {
1962 address = "127.0.0.1:8201"
1963 tls_disable = true
1964}
1965
1966# Enable API proxy with auto-auth token
1967api_proxy {
1968 use_auto_auth_token = true
1969}
1970
1971# Enable response caching
1972cache {
1973 use_auto_auth_token = true
1974}
1975
1976# Logging
1977log_level = "info"
1978```
1979
1980#### Start the proxy
1981
1982Start OpenBao Proxy:
1983
1984```bash
1985bao proxy -config=/tmp/openbao/proxy.hcl
1986```
1987
1988The proxy will authenticate with OpenBao and start listening on
1989`127.0.0.1:8201`.
1990
1991#### Configure spindle
1992
1993Set these environment variables for spindle:
1994
1995```bash
1996export SPINDLE_SERVER_SECRETS_PROVIDER=openbao
1997export SPINDLE_SERVER_SECRETS_OPENBAO_PROXY_ADDR=http://127.0.0.1:8201
1998export SPINDLE_SERVER_SECRETS_OPENBAO_MOUNT=spindle
1999```
2000
2001On startup, spindle will now connect to the local proxy,
2002which handles all authentication automatically.
2003
2004### Production setup for proxy
2005
2006For production, you'll want to run the proxy as a service:
2007
2008Place your production configuration in
2009`/etc/openbao/proxy.hcl` with proper TLS settings for the
2010vault connection.
2011
2012### Verifying setup
2013
2014Test the proxy directly:
2015
2016```bash
2017# Check proxy health
2018curl -H "X-Vault-Request: true" http://127.0.0.1:8201/v1/sys/health
2019
2020# Test token lookup through proxy
2021curl -H "X-Vault-Request: true" http://127.0.0.1:8201/v1/auth/token/lookup-self
2022```
2023
2024Test OpenBao operations through the server:
2025
2026```bash
2027# List all secrets
2028bao kv list spindle/
2029
2030# Add a test secret via the spindle API, then check it exists
2031bao kv list spindle/repos/
2032
2033# Get a specific secret
2034bao kv get spindle/repos/your_repo_path/SECRET_NAME
2035```
2036
2037### How it works
2038
2039- Spindle connects to OpenBao Proxy on localhost (typically
2040 port 8200 or 8201)
2041- The proxy authenticates with OpenBao using AppRole
2042 credentials
2043- All spindle requests go through the proxy, which injects
2044 authentication tokens
2045- Secrets are stored at
2046 `spindle/repos/{sanitized_repo_path}/{secret_key}`
2047- Repository paths like `did:plc:alice/myrepo` become
2048 `did_plc_alice_myrepo`
2049- The proxy handles all token renewal automatically
2050- Spindle no longer manages tokens or authentication
2051 directly
2052
2053### Troubleshooting
2054
2055**Connection refused**: Check that the OpenBao Proxy is
2056running and listening on the configured address.
2057
2058**403 errors**: Verify the AppRole credentials are correct
2059and the policy has the necessary permissions.
2060
2061**404 route errors**: The spindle KV mount probably doesn't
2062exist—run the mount creation step again.
2063
2064**Proxy authentication failures**: Check the proxy logs and
2065verify the role-id and secret-id files are readable and
2066contain valid credentials.
2067
2068**Secret not found after writing**: This can indicate policy
2069permission issues. Verify the policy includes both
2070`spindle/data/*` and `spindle/metadata/*` paths with
2071appropriate capabilities.
2072
2073Check proxy logs:
2074
2075```bash
2076# If running as systemd service
2077journalctl -u openbao-proxy -f
2078
2079# If running directly, check the console output
2080```
2081
2082Test AppRole authentication manually:
2083
2084```bash
2085bao write auth/approle/login \
2086 role_id="$(cat /tmp/openbao/role-id)" \
2087 secret_id="$(cat /tmp/openbao/secret-id)"
2088```
2089
2090# Webhooks
2091
2092Webhooks allow you to receive HTTP POST notifications when events occur in your repositories. This enables you to integrate Tangled with external services, trigger CI/CD pipelines, send notifications, or automate workflows.
2093
2094## Overview
2095
2096Webhooks send HTTP POST requests to URLs you configure whenever specific events happen. Currently, Tangled supports push events, with more event types coming soon.
2097
2098## Configuring webhooks
2099
2100To set up a webhook for your repository:
2101
21021. Navigate to your repository
21032. Go to **Settings → Hooks**
21043. Click **new webhook**
21054. Configure your webhook:
2106 - **Payload URL**: The endpoint that will receive the webhook POST requests
2107 - **Secret**: An optional secret key for verifying webhook authenticity (leave blank to send unsigned webhooks)
2108 - **Events**: Select which events trigger the webhook (currently only push events)
2109 - **Active**: Toggle whether the webhook is enabled
2110
2111## Webhook payload
2112
2113### Push
2114
2115When a push event occurs, Tangled sends a POST request with a JSON payload of the format:
2116
2117```json
2118{
2119 "after": "7b320e5cbee2734071e4310c1d9ae401d8f6cab5",
2120 "before": "c04ddf64eddc90e4e2a9846ba3b43e67a0e2865e",
2121 "pusher": {
2122 "did": "did:plc:hwevmowznbiukdf6uk5dwrrq"
2123 },
2124 "ref": "refs/heads/main",
2125 "repository": {
2126 "clone_url": "https://tangled.org/did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo",
2127 "created_at": "2025-09-15T08:57:23Z",
2128 "description": "an example repository",
2129 "fork": false,
2130 "full_name": "did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo",
2131 "html_url": "https://tangled.org/did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo",
2132 "name": "some-repo",
2133 "open_issues_count": 5,
2134 "owner": {
2135 "did": "did:plc:hwevmowznbiukdf6uk5dwrrq"
2136 },
2137 "ssh_url": "ssh://git@tangled.org/did:plc:hwevmowznbiukdf6uk5dwrrq/some-repo",
2138 "stars_count": 1,
2139 "updated_at": "2025-09-15T08:57:23Z"
2140 }
2141}
2142```
2143
2144## HTTP headers
2145
2146Each webhook request includes the following headers:
2147
2148- `Content-Type: application/json`
2149- `User-Agent: Tangled-Hook/<short-sha>` — User agent with short SHA of the commit
2150- `X-Tangled-Event: push` — The event type
2151- `X-Tangled-Hook-ID: <webhook-id>` — The webhook ID
2152- `X-Tangled-Delivery: <uuid>` — Unique delivery ID
2153- `X-Tangled-Signature-256: sha256=<hmac>` — HMAC-SHA256 signature (if secret configured)
2154
2155## Verifying webhook signatures
2156
2157If you configured a secret, you should verify the webhook signature to ensure requests are authentic. For example, in Go:
2158
2159```go
2160package main
2161
2162import (
2163 "crypto/hmac"
2164 "crypto/sha256"
2165 "encoding/hex"
2166 "io"
2167 "net/http"
2168 "strings"
2169)
2170
2171func verifySignature(payload []byte, signatureHeader, secret string) bool {
2172 // Remove 'sha256=' prefix from signature header
2173 signature := strings.TrimPrefix(signatureHeader, "sha256=")
2174
2175 // Compute expected signature
2176 mac := hmac.New(sha256.New, []byte(secret))
2177 mac.Write(payload)
2178 expected := hex.EncodeToString(mac.Sum(nil))
2179
2180 // Use constant-time comparison to prevent timing attacks
2181 return hmac.Equal([]byte(signature), []byte(expected))
2182}
2183
2184func webhookHandler(w http.ResponseWriter, r *http.Request) {
2185 // Read the request body
2186 payload, err := io.ReadAll(r.Body)
2187 if err != nil {
2188 http.Error(w, "Bad request", http.StatusBadRequest)
2189 return
2190 }
2191
2192 // Get signature from header
2193 signatureHeader := r.Header.Get("X-Tangled-Signature-256")
2194
2195 // Verify signature
2196 if signatureHeader != "" && verifySignature(payload, signatureHeader, yourSecret) {
2197 // Webhook is authentic, process it
2198 processWebhook(payload)
2199 w.WriteHeader(http.StatusOK)
2200 } else {
2201 http.Error(w, "Invalid signature", http.StatusUnauthorized)
2202 }
2203}
2204```
2205
2206## Delivery retries
2207
2208Webhooks are automatically retried on failure:
2209
2210- **3 total attempts** (1 initial + 2 retries)
2211- **Exponential backoff** starting at 1 second, max 10 seconds
2212- **Retried on**:
2213 - Network errors
2214 - HTTP 5xx server errors
2215- **Not retried on**:
2216 - HTTP 4xx client errors (bad request, unauthorized, etc.)
2217
2218### Timeouts
2219
2220Webhook requests timeout after 30 seconds. If your endpoint needs more time:
2221
22221. Respond with 200 OK immediately
22232. Process the webhook asynchronously in the background
2224
2225## Example integrations
2226
2227### Discord notifications
2228
2229```javascript
2230app.post("/webhook", (req, res) => {
2231 const payload = req.body;
2232
2233 fetch("https://discord.com/api/webhooks/...", {
2234 method: "POST",
2235 headers: { "Content-Type": "application/json" },
2236 body: JSON.stringify({
2237 content: `New push to ${payload.repository.full_name}`,
2238 embeds: [
2239 {
2240 title: `${payload.pusher.did} pushed to ${payload.ref}`,
2241 url: payload.repository.html_url,
2242 color: 0x00ff00,
2243 },
2244 ],
2245 }),
2246 });
2247
2248 res.status(200).send("OK");
2249});
2250```
2251
2252# Migrating knots and spindles
2253
2254Sometimes, non-backwards compatible changes are made to the
2255knot/spindle XRPC APIs. If you host a knot or a spindle, you
2256will need to follow this guide to upgrade. Typically, this
2257only requires you to deploy the newest version.
2258
2259This document is laid out in reverse-chronological order.
2260Newer migration guides are listed first, and older guides
2261are further down the page.
2262
2263## Upgrading to v1.15.0-alpha
2264
2265With v1.15.0-alpha, a knot itself owns its members and
2266per-repo collaborators directly. Previously this data was sourced from
2267PDS records (`sh.tangled.knot.member` and `sh.tangled.repo.collaborator`)
2268that the appview and the knot both read off the firehose.
2269The knot is now the source of truth and serves them over XRPC instead:
2270
2271- `sh.tangled.knot.addMember`, `sh.tangled.knot.removeMember`, `sh.tangled.knot.listMembers`
2272- `sh.tangled.repo.addCollaborator`, `sh.tangled.repo.removeCollaborator`, `sh.tangled.repo.listCollaborators`
2273
2274Until your knot is upgraded, the appview keeps reading its
2275members and collaborators from the old firehose-sourced records.
2276Upgrade to move your knot onto knot-owned access control.
2277
2278- Upgrade to the latest tag (v1.15.0 or above)
2279- Head to the [knot dashboard](https://tangled.org/settings/knots) and
2280 hit the "retry" button to verify your knot
2281
2282## Upgrading to v1.14.0-alpha
2283
2284Starting with v1.14.0-alpha, the fully knot uses the repoDID as its
2285canonical handle for repositories. This unlocks repository
2286renames from the appview UI and changes the wire format for
2287the following lexicons (`sh.tangled.repo.pull`, `sh.tangled.repo.collaborator`,
2288`sh.tangled.repo.issue`, `sh.tangled.git.refUpdate`).
2289
2290Knots that have not been upgraded may silently drop new push
2291events, pull requests, issues, and collaborator invites for
2292repositories they host until upgraded. So upgrade please!!!
2293
2294- Upgrade to the latest tag (v1.14.0 or above)
2295- Head to the [knot dashboard](https://tangled.org/settings/knots) and
2296 hit the "retry" button to verify your knot
2297
2298## Upgrading to v1.13.0-alpha
2299
2300Starting with v1.13.0-alpha, every repository on a knot is
2301assigned a DID. This makes repositories stable across
2302renames and transfers.
2303
2304When you upgrade your knot to this version, the server will
2305automatically mint DIDs for all existing repositories on
2306startup. This is a one-time process and you may see
2307additional log output during the first boot as DIDs are
2308assigned.
2309
2310- Upgrade to the latest tag (v1.13.0 or above)
2311- Head to the [knot dashboard](https://tangled.org/settings/knots) and
2312 hit the "retry" button to verify your knot
2313
2314## Upgrading from v1.8.x
2315
2316After v1.8.2, the HTTP API for knots and spindles has been
2317deprecated and replaced with XRPC. Repositories on outdated
2318knots will not be viewable from the appview. Upgrading is
2319straightforward however.
2320
2321For knots:
2322
2323- Upgrade to the latest tag (v1.9.0 or above)
2324- Head to the [knot dashboard](https://tangled.org/settings/knots) and
2325 hit the "retry" button to verify your knot
2326
2327For spindles:
2328
2329- Upgrade to the latest tag (v1.9.0 or above)
2330- Head to the [spindle
2331 dashboard](https://tangled.org/settings/spindles) and hit the
2332 "retry" button to verify your spindle
2333
2334## Upgrading from v1.7.x
2335
2336After v1.7.0, knot secrets have been deprecated. You no
2337longer need a secret from the appview to run a knot. All
2338authorized commands to knots are managed via [Inter-Service
2339Authentication](https://atproto.com/specs/xrpc#inter-service-authentication-jwt).
2340Knots will be read-only until upgraded.
2341
2342Upgrading is quite easy, in essence:
2343
2344- `KNOT_SERVER_SECRET` is no more, you can remove this
2345 environment variable entirely
2346- `KNOT_SERVER_OWNER` is now required on boot, set this to
2347 your DID. You can find your DID in the
2348 [settings](https://tangled.org/settings) page.
2349- Restart your knot once you have replaced the environment
2350 variable
2351- Head to the [knot dashboard](https://tangled.org/settings/knots) and
2352 hit the "retry" button to verify your knot. This simply
2353 writes a `sh.tangled.knot` record to your PDS.
2354
2355If you use the nix module, simply bump the flake to the
2356latest revision, and change your config block like so:
2357
2358```diff
2359 services.tangled.knot = {
2360 enable = true;
2361 server = {
2362- secretFile = /path/to/secret;
2363+ owner = "did:plc:foo";
2364 };
2365 };
2366```
2367
2368# Bobbin
2369
2370Bobbin is an API appview for Tangled records. It serves XRPC
2371endpoints for `sh.tangled.*`, with it you can get repos,
2372issues, pulls, comments, follows, stars, labels, pipelines,
2373and profiles. It is read-only, there is no auth, since that
2374should all be handled direct-to-PDS and knot respectively.
2375
2376**Bobbin has no permanent storage**.
2377
2378It is only a glorified edge index, in the graph theory
2379sense. Additionally it has a record cache, re-filled on
2380demand. All other data that Bobbin serves comes live from
2381PDSes & knots.
2382
2383## What Bobbin needs
2384
2385The way that Bobbin is able to pull off being
2386so stateless is by moving state upstream.
2387Primarily it depends on an instance of
2388[Hydrant](https://tangled.org/did:plc:6v3ul2ptnqctyxwkz5ti4amn)
2389, which is the service that gives an event stream
2390for Bobbin to quickly backfill from on every restart.
2391Backfilling ought to take less than a couple of minutes
2392maximum. If the upstream instance of Hydrant fails
2393while Bobbin is live, its list/count endpoints stop
2394advancing and report a stale cursor. Single-lookups
2395will continue working, due to the second dependency:
2396[Slingshot](https://tangled.org/did:plc:c7mc2fn47ihdihul4vjwsuy3/tree/main/slingshot).
2397Slingshot fetches individual records & resolves identities.
2398If the upstream instance of Slingshot fails, single-lookups
2399will fail with a `502` error. There are some aggregation
2400endpoints that use Slingshot for hydrating, which will also
2401fail.
2402
2403A soft dependency that ought to exist for Bobbin to operate
2404correctly is simply the plethora of knots that are out
2405there, that Bobbin talks to directly for git data and, for
2406knots at v1.15+, members & collaborators.
2407
2408## Building Bobbin
2409
2410Bobbin is under [Tangled's core monorepo, under bobbin/](https://tangled.org/did:plc:j5hmlfdrwkvtxm7cjmu7j2is/tree/master/bobbin).
2411Here's an easy local debug-build:
2412
2413```sh
2414cargo build -p bobbin
2415```
2416
2417Bobbin loves being in a container. When using
2418`bobbin/containerfiles/bobbin.Containerfile`, it runs `cargo
2419build --release --bin bobbin --package bobbin` within a
2420little Debian runtime, exposing port 8090.
2421
2422## Configuration
2423
2424The best way to configure Bobbin is via a toml config file.
2425There's an `example.toml` in [Bobbin's subdir](https://tangled.org/did:plc:j5hmlfdrwkvtxm7cjmu7j2is/blob/master/bobbin/example.toml).
2426Every value is overridable by a `BOBBIN_*` env var.
2427The load order is env, then `--config <path>`, then
2428`/etc/bobbin/config.toml`, then built-in defaults.
2429
2430Load and check a config without starting the server:
2431
2432```sh
2433bobbin --config config.toml validate
2434```
2435
2436Minimal config is the two upstream URLs. The hydrant URL
2437takes `ws://` or `wss://`. An `http://` or `https://`
2438URL is rewritten to the matching websocket scheme at
2439connection-time.
2440
2441```toml
2442[server]
2443binds = ["127.0.0.1:8090"]
2444
2445# Loopback-only & can leave empty to disable debug introspection.
2446debug_bind = "127.0.0.1:8091"
2447
2448[hydrant]
2449url = "https://hydrant.example.com"
2450
2451[slingshot]
2452url = "https://slingshot.example.com"
2453```
2454
2455> 🦪 Lewis
2456>
2457> At time of writing, we (Tangled) don't host public
2458> instances of Hydrant or Slingshot. You will have to
2459> find public instances or spin these up yourself! :P
2460
2461Take a gander in the project's example.toml for an
2462exhaustive list of things to configure.
2463
2464You will discover fun things such as a configurable adaptive
2465loop that watches the cgroup memory limit & throttles heavy
2466requests under pressure. It only works if it detects a
2467cgroup limit is present. The config for that is in the
2468`[backpressure]` block of the config template.
2469
2470## Running Bobbin
2471
2472Start the server using a config toml:
2473
2474```bash
2475bobbin --config config.toml
2476```
2477Bobbin wakes up in a cold sweat and immediately gets to
2478work:
24791. It binds its listeners, connects to the Hydrant stream
2480 in the background.
24812. It serves requests from the first
2482 moment it's alive, even before the Hydrant stream connects
2483 or finishes catching up. Having a cold Hydrant itself
2484 costs only latency and approximate counts.
2485
2486## The API
2487
2488**Single lookups** take a record's AT-URI.
2489
2490- `getRepo` takes the repo URI:
2491
2492```sh
2493curl "$BOBBIN/xrpc/sh.tangled.repo.getRepo?repo=at://did:plc:boltless/sh.tangled.repo/squid"
2494```
2495```json
2496{
2497 "uri": "at://did:plc:boltless/sh.tangled.repo/squid",
2498 "cid": "bafyrei...",
2499 "value": { "$type": "sh.tangled.repo", "knot": "knot1.tangled.sh", "description": "...", "createdAt": "..." }
2500}
2501```
2502
2503- `getProfile` takes the full profile record URI, so a bare
2504 handle or DID will not resolve:
2505
2506```sh
2507curl "$BOBBIN/xrpc/sh.tangled.actor.getProfile?actor=at://did:plc:boltless/sh.tangled.actor.profile/self"
2508```
2509
2510- If Slingshot cannot serve the record, the response is `502`:
2511
2512```json
2513{ "error": "UpstreamFailed", "message": "upstream unavailable: ..." }
2514```
2515
2516**Aggregation** endpoints come in `list*` and `count*` pairs,
2517each with a `*By` sibling, and require a `subject` query param.
2518
2519- `listRepos` and `countRepos` key on the owner DID:
2520
2521```sh
2522curl "$BOBBIN/xrpc/sh.tangled.repo.countRepos?subject=did:plc:boltless"
2523```
2524```json
2525{ "count": 7, "distinctAuthors": 1 }
2526```
2527
2528```sh
2529curl "$BOBBIN/xrpc/sh.tangled.repo.listRepos?subject=did:plc:boltless&limit=3"
2530```
2531```json
2532{ "items": [ { "uri": "at://did:plc:boltless/sh.tangled.repo/squid", "cid": "bafyrei...", "value": { } } ], "cursor": null }
2533```
2534
2535- Bobbin validates the subject per collection. Here a repo URI
2536 is passed where a bare DID is required, so the call returns a
2537 `400`:
2538
2539```sh
2540curl "$BOBBIN/xrpc/sh.tangled.graph.listFollows?subject=at://did:plc:boltless/sh.tangled.repo/squid"
2541```
2542```json
2543{ "error": "InvalidRequest", "message": "invalid request: subject must be a bare did, got at-uri with collection sh.tangled.repo" }
2544```
2545
2546**Search** is a single endpoint over an in-mem full-text
2547index:
2548
2549```sh
2550curl "$BOBBIN/xrpc/sh.tangled.search.query?q=tangled&limit=2"
2551```
2552```json
2553{ "hits": [ { "uri": "at://...", "cid": "...", "nsid": "sh.tangled.repo", "score": 27.1, "value": { } } ], "cursor": null }
2554```
2555
2556**Git data** such as blob, tree, diff, log, and archive proxies
2557straight to the repo's knot, streamed back without caching.
2558
2559## Coverage and warm-up
2560
2561- While the edge index is catching up from Hydrant,
2562 the aggregation count is a lower bound & may still climb.
2563- One endpoint reports how far along the backfill it is:
2564
2565```sh
2566curl "$BOBBIN/xrpc/sh.tangled.bobbin.getCoverage"
2567```
2568
2569While warming up:
2570
2571```json
2572{ "ready": false, "eventsProcessed": 45588, "lastCursor": 51658 }
2573```
2574
2575Once caught up, Bobbin flips to ready:
2576
2577```json
2578{ "ready": true, "eventsProcessed": 106085, "lastCursor": 116527 }
2579```
2580
2581If starting up Hydrant for the first time, Hydrant itself
2582will take a decent while (a couple of hours) to backfill
2583from PDSes. Hydrant stores its backfill on disk. Bobbin
2584restart reaches `ready` in minutes by replaying event from
2585an already-populated Hydrant. If your Hydrant is new, expect
2586Bobbin to backfill in that same couple of hours that Hydrant
2587takes.
2588
2589## Loose ends and not-gonna-impl
2590
2591- **No coverage signal for per-knot rosters yet.**
2592 Coverage tracks the hydrant stream only. A v1.15 knot
2593 that is unreachable serves a stale or empty member set
2594 with nothing to flag it.
2595- **Knot eventstream fan-out isn't pooled.**
2596 Bobbin opens one websocket per v1.15
2597 knot on top of the hydrant subscription. A network with
2598 thousands of knots wants pooling or a shared subscription.
2599- **No sequential issue or PR numbers.** bobbin returns rkeys,
2600 not `#42` style ids like the web appview. A client
2601 deriving a display number does it from creation order. But
2602 why bother? rkeys are the IDs.
2603
2604# Hacking on Tangled
2605
2606We highly recommend [installing
2607Nix](https://nixos.org/download/) (the package manager)
2608before working on the codebase. The Nix flake provides a lot
2609of helpers to get started and most importantly, builds and
2610dev shells are entirely deterministic.
2611
2612To set up your dev environment:
2613
2614```bash
2615nix develop
2616```
2617
2618Non-Nix users can look at the `devShell` attribute in the
2619`flake.nix` file to determine necessary dependencies.
2620
2621## Running the appview
2622
2623The appview requires Redis and OAuth JWKs. Start these
2624first, before launching the appview itself.
2625
2626```bash
2627# OAuth JWKs should already be set up by the Nix devshell:
2628echo $TANGLED_OAUTH_CLIENT_SECRET
2629z42ty4RT1ovnTopY8B8ekz9NuziF2CuMkZ7rbRFpAR9jBqMc
2630
2631echo $TANGLED_OAUTH_CLIENT_KID
26321761667908
2633
2634# if not, you can set it up yourself:
2635goat key generate -t P-256
2636Key Type: P-256 / secp256r1 / ES256 private key
2637Secret Key (Multibase Syntax): save this securely (eg, add to password manager)
2638 z42tuPDKRfM2mz2Kv953ARen2jmrPA8S9LX9tRq4RVcUMwwL
2639Public Key (DID Key Syntax): share or publish this (eg, in DID document)
2640 did:key:zDnaeUBxtG6Xuv3ATJE4GaWeyXM3jyamJsZw3bSPpxx4bNXDR
2641
2642# the secret key from above
2643export TANGLED_OAUTH_CLIENT_SECRET="z42tuP..."
2644
2645# Run Redis in a new shell to store OAuth sessions
2646redis-server
2647```
2648
2649The Nix flake exposes a few `app` attributes (run `nix
2650flake show` to see a full list of what the flake provides),
2651one of the apps runs the appview with the `air`
2652live-reloader:
2653
2654```bash
2655TANGLED_DEV=true nix run .#watch-appview
2656
2657# TANGLED_DB_PATH might be of interest to point to
2658# different sqlite DBs
2659
2660# in a separate shell, you can live-reload tailwind
2661nix run .#watch-tailwind
2662```
2663
2664## Running knots and spindles
2665
2666An end-to-end knot setup requires setting up a machine with
2667`sshd`, `AuthorizedKeysCommand`, and a Git user, which is
2668quite cumbersome. So the Nix flake provides a
2669`nixosConfiguration` to do so.
2670
2671<details>
2672 <summary><strong>macOS users will have to set up a Nix Builder first</strong></summary>
2673
2674In order to build Tangled's dev VM on macOS, you will
2675first need to set up a Linux Nix builder. The recommended
2676way to do so is to run a [`darwin.linux-builder`
2677VM](https://nixos.org/manual/nixpkgs/unstable/#sec-darwin-builder)
2678and to register it in `nix.conf` as a builder for Linux
2679with the same architecture as your Mac (`linux-aarch64` if
2680you are using Apple Silicon).
2681
2682If you're on nix-darwin, you can simply add
2683
2684```
2685nix.linux-builder.enable = true;
2686```
2687
2688to your host's `configuration.nix`.
2689
2690Alternatively, you can use any other method to set up a
2691Linux machine with Nix installed that you can `sudo ssh`
2692into (in other words, root user on your Mac has to be able
2693to ssh into the Linux machine without entering a password)
2694and that has the same architecture as your Mac. See
2695[remote builder
2696instructions](https://nix.dev/manual/nix/2.28/advanced-topics/distributed-builds.html#requirements)
2697for how to register such a builder in `nix.conf`.
2698
2699> WARNING: If you'd like to use
2700> [`nixos-lima`](https://github.com/nixos-lima/nixos-lima) or
2701> [Orbstack](https://orbstack.dev/), note that setting them up so that `sudo
2702ssh` works can be tricky. It seems to be [possible with
2703> Orbstack](https://github.com/orgs/orbstack/discussions/1669).
2704
2705</details>
2706
2707To begin, grab your DID from http://localhost:3000/settings.
2708Then, set `TANGLED_VM_KNOT_OWNER` and
2709`TANGLED_VM_SPINDLE_OWNER` to your DID. You can now start a
2710lightweight NixOS VM like so:
2711
2712```bash
2713nix run --impure .#vm
2714
2715# type `poweroff` at the shell to exit the VM
2716```
2717
2718This starts a knot on port 6444, a spindle on port 6555
2719with `ssh` exposed on port 2222.
2720
2721Once the services are running, head to
2722http://localhost:3000/settings/knots and hit "Verify". It should
2723verify the ownership of the services instantly if everything
2724went smoothly.
2725
2726You can push repositories to this VM with this ssh config
2727block on your main machine:
2728
2729```bash
2730Host nixos-shell
2731 Hostname localhost
2732 Port 2222
2733 User git
2734 IdentityFile ~/.ssh/my_tangled_key
2735```
2736
2737Set up a remote called `local-dev` on a git repo:
2738
2739```bash
2740git remote add local-dev git@nixos-shell:user/repo
2741git push local-dev main
2742```
2743
2744The above VM should already be running a spindle on
2745`localhost:6555`. Head to http://localhost:3000/settings/spindles and
2746hit "Verify". You can then configure each repository to use
2747this spindle and run CI jobs.
2748
2749Of interest when debugging spindles:
2750
2751```
2752# Service logs from journald:
2753journalctl -xeu spindle
2754
2755# CI job logs from disk:
2756ls /var/log/spindle
2757
2758# Debugging spindle database:
2759sqlite3 /var/lib/spindle/spindle.db
2760
2761# litecli has a nicer REPL interface:
2762litecli /var/lib/spindle/spindle.db
2763```
2764
2765If for any reason you wish to disable either one of the
2766services in the VM, modify [nix/vm.nix](/nix/vm.nix) and set
2767`services.tangled.spindle.enable` (or
2768`services.tangled.knot.enable`) to `false`.
2769
2770# Contribution guide
2771
2772## Commit guidelines
2773
2774We follow a commit style similar to the Go project. Please keep commits:
2775
2776- **atomic**: each commit should represent one logical change
2777- **descriptive**: the commit message should clearly describe what the
2778 change does and why it's needed
2779
2780### Message format
2781
2782```
2783<service/top-level directory>/<affected package/directory>: <short summary of change>
2784
2785Optional longer description can go here, if necessary. Explain what the
2786change does and why, especially if not obvious. Reference relevant
2787issues or PRs when applicable. These can be links for now since we don't
2788auto-link issues/PRs yet.
2789```
2790
2791Here are some examples:
2792
2793```
2794appview/state: fix token expiry check in middleware
2795
2796The previous check did not account for clock drift, leading to premature
2797token invalidation.
2798```
2799
2800```
2801knotserver/git/service: improve error checking in upload-pack
2802```
2803
2804### General notes
2805
2806- PRs get merged "as-is" (fast-forward)—like applying a patch-series
2807 using `git am`. At present, there is no squashing—so please author
2808 your commits as they would appear on `master`, following the above
2809 guidelines.
2810- If there is a lot of nesting, for example "appview:
2811 pages/templates/repo/fragments: ...", these can be truncated down to
2812 just "appview: repo/fragments: ...". If the change affects a lot of
2813 subdirectories, you may abbreviate to just the top-level names, e.g.
2814 "appview: ..." or "knotserver: ...".
2815- Keep commits lowercased with no trailing period.
2816- Use the imperative mood in the summary line (e.g., "fix bug" not
2817 "fixed bug" or "fixes bug").
2818- Try to keep the summary line under 72 characters, but we aren't too
2819 fussed about this.
2820- Follow the same formatting for PR titles if filled manually.
2821- Don't include unrelated changes in the same commit.
2822- Avoid noisy commit messages like "wip" or "final fix"—rewrite history
2823 before submitting if necessary.
2824
2825## Code formatting
2826
2827We use a variety of tools to format our code, and multiplex them with
2828[`treefmt`](https://treefmt.com). All you need to do to format your changes
2829is run `nix run .#fmt` (or just `treefmt` if you're in the devshell).
2830
2831## Proposals for bigger changes
2832
2833Small fixes like typos, minor bugs, or trivial refactors can be
2834submitted directly as PRs.
2835
2836For larger changes—especially those introducing new features, significant
2837refactoring, or altering system behavior—please open a proposal first. This
2838helps us evaluate the scope, design, and potential impact before implementation.
2839
2840Create a new issue titled:
2841
2842```
2843proposal: <affected scope>: <summary of change>
2844```
2845
2846In the description, explain:
2847
2848- What the change is
2849- Why it's needed
2850- How you plan to implement it (roughly)
2851- Any open questions or tradeoffs
2852
2853We'll use the issue thread to discuss and refine the idea before moving
2854forward.
2855
2856## Developer Certificate of Origin (DCO)
2857
2858We require all contributors to certify that they have the right to
2859submit the code they're contributing. To do this, we follow the
2860[Developer Certificate of Origin
2861(DCO)](https://developercertificate.org/).
2862
2863By signing your commits, you're stating that the contribution is your
2864own work, or that you have the right to submit it under the project's
2865license. This helps us keep things clean and legally sound.
2866
2867To sign your commit, just add the `-s` flag when committing:
2868
2869```sh
2870git commit -s -m "your commit message"
2871```
2872
2873This appends a line like:
2874
2875```
2876Signed-off-by: Your Name <your.email@example.com>
2877```
2878
2879We won't merge commits if they aren't signed off. If you forget, you can
2880amend the last commit like this:
2881
2882```sh
2883git commit --amend -s
2884```
2885
2886If you're submitting a PR with multiple commits, make sure each one is
2887signed.
2888
2889For [jj](https://jj-vcs.github.io/jj/latest/) users, you can run the following command
2890to make it sign off commits in the tangled repo:
2891
2892```shell
2893# Safety check, should say "No matching config key..."
2894jj config list templates.commit_trailers
2895# The command below may need to be adjusted if the command above returned something.
2896jj config set --repo templates.commit_trailers "format_signed_off_by_trailer(self)"
2897```
2898
2899Refer to the [jujutsu
2900documentation](https://jj-vcs.github.io/jj/latest/config/#commit-trailers)
2901for more information.
2902
2903# Troubleshooting guide
2904
2905## Login issues
2906
2907Owing to the distributed nature of OAuth on AT Protocol, you
2908may run into issues with logging in. If you run a
2909self-hosted PDS:
2910
2911- You may need to ensure that your PDS is timesynced using
2912 NTP:
2913 - Enable the `ntpd` service
2914 - Run `ntpd -qg` to synchronize your clock
2915- You may need to increase the default request timeout:
2916 `NODE_OPTIONS="--network-family-autoselection-attempt-timeout=500"`
2917
2918## Empty punchcard
2919
2920For Tangled to register commits that you make across the
2921network, you need to setup one of following:
2922
2923- The committer email should be a verified email associated
2924 to your account. You can add and verify emails on the
2925 settings page.
2926- Or, the committer email should be set to your account's
2927 DID: `git config user.email "did:plc:foobar"`. You can find
2928 your account's DID on the settings page
2929
2930## Commit is not marked as verified
2931
2932Presently, Tangled only supports SSH commit signatures.
2933
2934To sign commits using an SSH key with git:
2935
2936```
2937git config --global gpg.format ssh
2938git config --global user.signingkey ~/.ssh/tangled-key
2939```
2940
2941To sign commits using an SSH key with jj, add this to your
2942config:
2943
2944```
2945[signing]
2946behavior = "own"
2947backend = "ssh"
2948key = "~/.ssh/tangled-key"
2949```
2950
2951## Self-hosted knot issues
2952
2953If you need help troubleshooting a self-hosted knot, check
2954out the [knot troubleshooting
2955guide](/knot-self-hosting-guide.html#troubleshooting).