Deploying the gitweb browser

How to build and deploy the gitweb service to the upspin VM (instance-2). It is a single Go binary (page templates + static assets embedded via go:embed) that runs as a docker-compose service alongside the existing upspin stack, exactly like vault and markdown.

Architecture at a glance

• Image: gcr.io/cosmocr-at-upspin/gitweb:latest
• Host: instance-2 (GCP, us-central1-a, COS 121), static IP 34.28.234.94
• TLS host: git.cosmocr.at — the container terminates its own TLS with a certbot cert; nginx SNI-routes git.cosmocr.at → gitweb:8446. (DNS is a *.cosmocr.at wildcard → 34.28.234.94, so no per-host record is needed.)
• Runs as upspin user git@cosmocr.at. It reads any bare repo whose Access file grants it read,list; push write-back additionally needs create,write,delete for git@cosmocr.at on the repo. The demo repo is git@cosmocr.at/demo.git; third parties expose a repo by granting read,list: git@cosmocr.at.
• Config & keys live on the persistent data disk (not in the image):
  • /mnt/disks/data/gitweb/config — upspin config (secrets: /gitweb/secrets)
  • /mnt/disks/data/gitweb/secrets/{public,secret}.upspinkey
  • /mnt/disks/data/letsencrypt/live/git.cosmocr.at/ — TLS cert
• Authoritative infra sources (in the control repo):
  • upspin/vm/cloud-init-cos121.yaml — the VM's cloud-init (embeds the docker-compose + nginx.conf used on boot). Edit here first, and it is only live once pushed to the instance metadata user-data key (see "If you changed config" — cloud-init re-applies the metadata on every boot and overwrites the /var/upspin config files, so anything not in the metadata is reverted on the next reboot).
  • upspin/vm/docker-compose.vault.yml — rendered from the cloud-init for local reference; the live /var/upspin/docker-compose.yml is written from the metadata on boot (all of vault, markdown, and gitweb live in it).
  • upspin/nginx/nginx.conf — canonical nginx config (the SNI map).
  • upspin/vm/deploy-git.sh — first-time end-to-end deploy script.
  • upspin/secrets/git/ — the browser's upspin config + keys (git-ignored). Note the workstation-side secrets are under secrets/git/, but the on-VM service + data dir are named gitweb (matching the compose service).

Prerequisites

• docker locally (for --platform linux/amd64 builds) and gcloud configured for project cosmocr-at-upspin. (The Dockerfile is COPY-only over a cross-compiled CGO_ENABLED=0 GOOS=linux GOARCH=amd64 binary, so the amd64 image builds on an arm workstation with no qemu.)
• SSH to the VM. The deploy script defaults to ~/.ssh/google_compute_engine; override with DEPLOY_SSH_KEY to use a passphrase-less key:

  DEPLOY_SSH_KEY=$HOME/.ssh/nopass_claude_code ./deploy-git.sh
  

• The git@cosmocr.at upspin user must exist on the keyserver and be in the keyserver's Writers. (One-time; already done — see upspin/doc/provision_user.md.)

Routine redeploy (code change)

This is the common case. Because the templates + assets are embedded, any change needs a rebuilt image.

1. Build + push the image from the gitweb repo:

   make push
   

   (= CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build, docker build --platform linux/amd64, tag, docker push.)

2. Pull + recreate the container on the VM:

   # authenticate the Docker daemon to gcr.io
   sudo env HOME=/var/upspin /bin/sh /var/upspin/auth.sh
   
   # pull the new image
   sudo docker run --rm \
     -v /var/run/docker.sock:/var/run/docker.sock \
     -v /var/upspin/.docker:/root/.docker \
     -v /var/upspin:/var/upspin -w /var/upspin \
     docker/compose:1.29.2 -p patrick pull gitweb
   
   # recreate just the gitweb container (rm + create; see "compose recreate" note)
   sudo docker rm -f patrick_gitweb_1
   sudo docker run --rm \
     -v /var/run/docker.sock:/var/run/docker.sock \
     -v /var/upspin/.docker:/root/.docker \
     -v /var/upspin:/var/upspin -w /var/upspin \
     docker/compose:1.29.2 -p patrick up -d --no-recreate gitweb
   

   (Re-running deploy-git.sh also works and is idempotent; the cert step is a no-op once the cert exists — but see the certbot gotcha below.)

3. Verify:

   sudo docker logs --tail 5 patrick_gitweb_1        # "gitweb: serving on :8446"
   curl -sI https://git.cosmocr.at/git@cosmocr.at/demo.git/   # 200
   

If you changed config, the compose service, or nginx

This covers any change to the compose services (incl. restart: policies), the nginx routing/SNI map, or the upspin config — i.e. anything that lives in cloud-init-cos121.yaml.

⚠️ The instance metadata is the source of truth, not the files on the VM. On every boot, cloud-init re-applies the VM's user-data metadata and overwrites /var/upspin/docker-compose.yml and /var/upspin/nginx/nginx.conf. A change you only sudo cp onto the VM (or nginx -s reload) works until the next reboot, then silently reverts. This is exactly how the vault service once vanished on its first reboot: it was added to the running VM and to the repo, but never pushed to the metadata. Always push the metadata.

1. Edit + push the metadata (authoritative — survives reboots):

# edit upspin/vm/cloud-init-cos121.yaml in the control repo, then from that repo:
gcloud compute instances add-metadata instance-2 --zone us-central1-a \
  --metadata-from-file user-data=upspin/vm/cloud-init-cos121.yaml

Mirror the same edit into upspin/vm/docker-compose.vault.yml and upspin/nginx/nginx.conf so the local reference copies stay identical. The push takes effect on the next reboot; verify it landed (run on the VM):

curl -s -H "Metadata-Flavor: Google" \
  http://metadata.google.internal/computeMetadata/v1/instance/attributes/user-data \
  | grep -c 'gitweb:'      # expect a non-zero count

2. Apply now without waiting for a reboot (optional, for a live change): copy the rendered files up and reload/recreate only the affected pieces:

# control/upspin/vm/docker-compose.vault.yml -> /var/upspin/docker-compose.yml
# control/upspin/nginx/nginx.conf            -> /var/upspin/nginx/nginx.conf
sudo docker exec patrick_nginx_1 nginx -t && sudo docker exec patrick_nginx_1 nginx -s reload
# recreate any container whose service/config changed (rm + up; see note below)

These hot-apply the change immediately; step 1 is what makes it durable. Do both — they are not alternatives.

First-time setup (new VM / from scratch)

Use control/upspin/vm/deploy-git.sh, which does it all idempotently: stages the config + keys onto the data disk, issues the TLS cert (certbot webroot via the running nginx), updates nginx + compose, pulls the image, recreates nginx (for the SNI route) and starts gitweb, then verifies. Manual outline:

1. DNS: covered by the *.cosmocr.at wildcard → 34.28.234.94 (no per-host record needed).
2. Config + keys onto the data disk: /mnt/disks/data/gitweb/config + /mnt/disks/data/gitweb/secrets/*.upspinkey (keys 0600, root-owned).
3. TLS cert (nginx must be serving :80 for the ACME challenge):

   sudo docker run --rm \
     -v /mnt/disks/data/certbot-www:/var/www/certbot:rw \
     -v /mnt/disks/data/letsencrypt:/etc/letsencrypt:rw \
     certbot/certbot certonly --webroot --webroot-path=/var/www/certbot \
       --register-unsafely-without-email --agree-tos --no-eff-email \
       -d git.cosmocr.at
   

4. nginx route and the gitweb compose service (already in cloud-init-cos121.yaml): SNI route git.cosmocr.at gitweb:8446;.
5. Recreate nginx, then start gitweb (commands as in "Routine redeploy").

Before the first run, make sure the infra edits are in cloud-init-cos121.yaml and pushed to the metadata (see "If you changed config"): the gitweb: compose service and the git.cosmocr.at gitweb:8446; line in the nginx map.

Gotchas / troubleshooting

• Config on the VM reverts on reboot if it isn't in the instance metadata. cloud-init re-applies user-data on every boot and overwrites the /var/upspin config files. A sudo cp / nginx -s reload only holds until the next reboot. Push every infra change with gcloud compute instances add-metadata … --metadata-from-file user-data=cloud-init-cos121.yaml (see "If you changed config"). Symptom of a miss: after a reboot the service is gone from docker-compose.yml (compose logs Found orphan containers …) or nginx returns no host in upstream "".

• certbot step must stay non-interactive. The certonly invocation runs with --keep-until-expiring -n, so on a re-run where the cert already exists it's a silent no-op instead of prompting 1: Keep / 2: Renew. Without those flags it would block on stdin and die with EOFError, aborting the script (set -e) before the pull/recreate steps — don't remove them. (deploy-markdown.sh and deploy-vault.sh carry the same flags.)

• Compose command: is flags only. The image ENTRYPOINT is already /bin/gitweb; do not repeat /bin/gitweb in the compose command:. If you do, Go's flag parser stops at that leading positional and silently ignores all flags (including -tls_cert), so it serves plain HTTP → clients get a TLS "wrong version number" through nginx.

• Compose recreate KeyError: ContainerConfig. docker-compose 1.29.2 on COS's Docker errors on in-place recreate. Always docker rm -f <container> first, then up -d --no-recreate. (This is also why a redeploy with a new image must docker rm -f patrick_gitweb_1 — up alone with --no-recreate keeps the old container/image.)

• nginx network aliases (hairpin NAT). GCP does not let a VM reach its own external IP, so the gitweb container can't talk to upspin.cosmocr.at / key.cosmocr.at via the public IP. The shared nginx service carries those as network aliases, so the gitweb upspin client resolves them to nginx and routes back in via SNI. Don't remove them.

• Reading other users' ee-encrypted files. A read,list grant lets gitweb list and fetch a repo, but an ee-packed (encrypted) object is only readable if a key was wrapped for git@cosmocr.at — the owner must run upspin share -fix -r <repo> after granting. plain/eeintegrity objects render with just the grant. Objects gitweb can't decrypt surface as 404.

• TLS cert. nginx must be serving :80 for the certbot webroot ACME challenge; the cert lives at /mnt/disks/data/letsencrypt/live/git.cosmocr.at/ and the container hot-reloads it via the certReloader (no restart on renewal).

• Restart policy is unless-stopped (like every service in the stack). A docker.service drop-in gates the daemon on the data-disk mount, so it can't start against an unmounted disk. The store makes no upspin calls until the first request, so there's no startup readiness race.