| docs/FR-large-push-400.md |
| | # FR: pushes over ~1 MiB fail with an undiagnosable `400 bad request` |
| | |
| - | Status: open — root cause narrowed, not yet identified |
| + | Status: **fixed** (2026-10-18). Root cause identified; see Resolution. |
| | Filed: 2026-09-19 |
| | Observed on: `git.cosmocr.at` (deployed), pushing `rpg137@gmail.com/einkdashboards.git` |
| | Reproduced on: `git@cosmocr.at/demo.git` with a 3 MiB payload (see below) |
| + | |
| + | ## Resolution (2026-10-18) |
| + | |
| + | **Root cause: gitweb answered git's probe request with 400.** When a push is |
| + | larger than `http.postBuffer`, git (`remote-curl.c`, `probe_rpc`) first sends a |
| + | receive-pack POST whose entire body is a flush-pkt, `0000` |
| + | (`Content-Length: 4`). It checks the endpoint (and auth) that way before |
| + | streaming the real body chunked, and it needs a `200` back. gitweb passed the |
| + | probe to `DecodeReceivePack`, which fails on a request with no commands, and |
| + | the handler returned a bare `400`. git gave up before sending any of the pack. |
| + | |
| + | That explains every observation below. Pack size alone triggers it, because |
| + | the probe is only sent over `postBuffer`. Ancestry makes no difference. Raising |
| + | `postBuffer` makes the push work, because then there is no probe. Chunked |
| + | decoding tests fine, because the pack never got as far as being decoded. |
| + | `Expect: 100-continue` (hypothesis 2) is not involved. A `GIT_CURL_VERBOSE` |
| + | trace shows the failing request is the 4-byte POST. |
| + | |
| + | **Fix:** |
| + | |
| + | - `githttp.DecodeReceivePack` reads the first 4 bytes and returns `ErrProbe` |
| + | when they are a flush. Reading them can't block, because git sends the whole |
| + | first pkt-line before it waits. A real push starts with a command. |
| + | - The handler answers `ErrProbe` with `200` and an empty result. The probe |
| + | runs after authentication and authorization, as before, so a non-writer is |
| + | still refused at the probe. |
| + | - Decode failures are now logged (`gitweb: receive-pack decode …`). A body |
| + | over the cap gets `413` with the limit named; anything else gets `400` with |
| + | the reason. |
| + | |
| + | **Tests:** |
| + | |
| + | - `TestCloneAndPush/large_push_over_http.postBuffer` pushes 3 MiB of random |
| + | data with real git at `http.postBuffer=1048576` (the default) and re-clones |
| + | it. It failed with HTTP 400 before the fix. |
| + | - `…/large_push_as_reader_denied` checks that the probe is still authorized. |
| + | - `TestDecodeReceivePackProbe`, `TestDecodeReceivePackProbeGzip` and |
| + | `TestDecodeReceivePackChunked` (the regression test proposed below) cover |
| + | the decode path. |
| + | |
| + | The investigation notes below are kept as filed. |
| | |
| | ## Symptom |
| | |
| | Pushing a repository whose pack exceeds git's default `http.postBuffer` (1 MiB) |
| | fails: |
| | |
| | ``` |
| | $ git push --force origin master multi-dashboard |
| | error: RPC failed; HTTP 400 curl 22 The requested URL returned error: 400 |
| | fatal: the remote end hung up unexpectedly |
| | Everything up-to-date |
| | ``` |
| | |
| | `git fetch` against the same repo, over the same connection, with the same |
| | credentials, succeeds immediately beforehand — so this is not auth, TLS, |
| | routing, or repo existence. |
| | |
| | The workaround is client-side: |
| | |
| | ```bash |
| | git config http.postBuffer 524288000 # 500 MiB |
| | ``` |
| | |
| | After which the identical push succeeds. Every user has to discover and apply |
| | this per-repo, and nothing in the server's response hints at it. |
| | |
| | ## What is actually wrong |
| | |
| | **The primary defect is that the failure is unattributable.** In |
| | `internal/server/githttp.go:140-143`: |
| | |
| | ```go |
| | case gitReceivePack: |
| | req, err := githttp.DecodeReceivePack(r) |
| | if err != nil { |
| | http.Error(w, "400 bad request", http.StatusBadRequest) |
| | return |
| | } |
| | ``` |
| | |
| | Every possible decode failure — malformed pkt-line, truncated body, a pack |
| | go-git cannot resolve, a body over `maxBody`, a read error mid-stream — |
| | collapses into the same opaque string, and **`err` is discarded without being |
| | logged**. Every other branch in this handler logs (`log.Printf("gitweb: |
| | upload-pack %s: %v", ...)`); this one does not. |
| | |
| | Consequence: the server operator cannot tell a client-side protocol error from |
| | a server-side bug, and the client is told nothing actionable. Diagnosing the |
| | incident above took roughly an hour and still did not reach a root cause. |
| | |
| | ## What has been ruled out |
| | |
| | Investigated and eliminated, so nobody repeats the work: |
| | |
| | | Hypothesis | Verdict | |
| | |---|---| |
| | | nginx rejecting the body | **No.** nginx does SNI passthrough (`docs/ARCHITECTURE.md:53`); it routes TCP and never parses HTTP, so `client_max_body_size` cannot apply. | |
| | | `maxBody` cap in `requestBody` | **No.** 256 MiB (`internal/githttp/githttp.go:34`); the failing pack was ~5.7 MiB. | |
| | | `MaxRepoBytes` | **No.** 256 MiB default (`main.go:57`), and it is enforced after decode. | |
| | | Server read timeout cutting off a slow upload | **No.** Only `ReadHeaderTimeout` is set (`main.go:112`); there is no `ReadTimeout`. | |
| | | **Chunked transfer-encoding decoding** | **No — tested.** `DecodeReceivePack` handles a chunked body correctly, at both minimal size and 6 MiB. See the test below. | |
| | |
| | That last row matters: the obvious theory ("gitweb can't parse chunked |
| | requests") is **false**, and an FR that asserted it would have sent the fix in |
| | the wrong direction. The chunked path decodes; something else about a real |
| | large push does not. |
| | |
| | ## Controlled reproduction (2026-09-19) |
| | |
| | Reproduced on `git@cosmocr.at/demo.git` — a 688 KiB repo unrelated to the |
| | original incident — using two branches identical except for ancestry, each |
| | carrying a 3 MiB incompressible payload (random data; zeros would deflate away |
| | and never exceed `postBuffer` on the wire). |
| | |
| | | Branch | Common ancestor with `main` | Pack | `http.postBuffer` | Result | |
| | |---|---|---|---|---| |
| | | `size-only` | **yes** | 3.00 MiB | default (1 MiB) | **HTTP 400** | |
| | | `no-ancestor` (orphan) | no | 3.00 MiB | default (1 MiB) | **HTTP 400** | |
| | | `size-only` | yes | 3.00 MiB | 500 MiB | **succeeded** | |
| | |
| | Client error is byte-identical to the original incident: |
| | |
| | ``` |
| | error: RPC failed; HTTP 400 curl 22 The requested URL returned error: 400 |
| | fatal: the remote end hung up unexpectedly |
| | ``` |
| | |
| | **Conclusion: pack size alone is sufficient.** The `size-only` branch shares an |
| | ancestor with `main`, so no external delta bases and no thin-pack resolution |
| | are involved, and it fails anyway. Ancestry makes no difference — both shapes |
| | fail identically, and the same objects push cleanly once the client stops using |
| | chunked encoding. |
| | |
| | This demotes the thin-pack hypothesis below. The trigger is a pack exceeding |
| | `http.postBuffer`, and the fix belongs in request handling rather than the pack |
| | resolution path. |
| | |
| | Also observed in the same session: a **force-push rewinding `main` succeeded** |
| | at the default `postBuffer` (it sends a ref update and no new objects). Ref |
| | update handling is therefore not implicated; only the large-pack path fails. |
| | |
| | ## Remaining hypotheses |
| | |
| | In rough order of likelihood, after the reproduction above: |
| | |
| | 1. ~~**Thin packs.**~~ **Ruled out** by the controlled reproduction — a push |
| | sharing an ancestor, with no external delta bases, fails identically. |
| | 2. **`Expect: 100-continue` handling.** Now the leading suspect. curl adds the |
| | header for large streamed bodies; Go sends `100 Continue` only when the |
| | handler first reads the body, so any path that writes a response before |
| | reading would strand the client. This fits the evidence: it is specific to |
| | the large/streamed request path and independent of pack contents. |
| | 3. **Deployed binary drift.** `HEAD` here is `5444040` (2026-07-12) on branch |
| | `sso-browser-login`. Whether the running service matches this source is |
| | unconfirmed — the bug may already be fixed in tree. Worth eliminating |
| | cheaply before any code change, since the unit tests in this repo show the |
| | decode path handling chunked bodies correctly. |
| | |
| | ## Proposed change |
| | |
| | ### 1. Log the error (do this first, independently of any fix) |
| | |
| | ```go |
| | req, err := githttp.DecodeReceivePack(r) |
| | if err != nil { |
| | log.Printf("gitweb: receive-pack decode %s: %v", repoPath, err) |
| | http.Error(w, "400 bad request", http.StatusBadRequest) |
| | return |
| | } |
| | ``` |
| | |
| | One line. It converts this class of incident from guesswork into a log grep, |
| | and it is what tells us which of the hypotheses above is real. |
| | |
| | ### 2. Return a useful body to the client |
| | |
| | git surfaces the response body on `remote:` lines. Distinguish the cases that |
| | have distinct remedies — at minimum, separate "body too large" (413, with the |
| | limit named) from "malformed request" (400). A client seeing |
| | `remote: pack exceeds 256 MiB limit` needs no server log to self-serve. |
| | |
| | ### 3. Fix the underlying failure |
| | |
| | Only once (1) identifies it. If it is thin packs, the fix belongs in the pack |
| | resolution path, not in request handling — and no amount of `postBuffer` tuning |
| | is the right answer for users. |
| | |
| | ## How to test |
| | |
| | ### Regression test for the chunked path (passes today — keep it that way) |
| | |
| | Add to `internal/githttp/githttp_test.go`. This is the test that disproved the |
| | original theory; it is worth keeping so the chunked path does not silently |
| | regress: |
| | |
| | ```go |
| | // TestDecodeReceivePackChunked drives a chunked body through a real |
| | // http.Server, which is what git sends once a request exceeds |
| | // http.postBuffer (default 1 MiB). |
| | func TestDecodeReceivePackChunked(t *testing.T) { |
| | const ( |
| | oldOID = "0000000000000000000000000000000000000000" |
| | newOID = "1111111111111111111111111111111111111111" |
| | ) |
| | cmd := fmt.Sprintf("%s %s refs/heads/master\x00report-status\n", oldOID, newOID) |
| | var body bytes.Buffer |
| | fmt.Fprintf(&body, "%04x%s", len(cmd)+4, cmd) |
| | body.WriteString("0000") |
| | body.WriteString("PACK") |
| | body.Write([]byte{0, 0, 0, 2, 0, 0, 0, 0}) |
| | body.Write(bytes.Repeat([]byte{0xAB}, 6<<20)) // past postBuffer |
| | body.Write(make([]byte, 20)) |
| | |
| | errCh := make(chan error, 1) |
| | srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { |
| | if r.ContentLength != -1 { |
| | t.Errorf("ContentLength = %d, want -1 (chunked)", r.ContentLength) |
| | } |
| | req, err := DecodeReceivePack(r) |
| | if err == nil && req.Packfile != nil { |
| | _, err = io.Copy(io.Discard, req.Packfile) |
| | } |
| | errCh <- err |
| | })) |
| | defer srv.Close() |
| | |
| | hreq, _ := http.NewRequest(http.MethodPost, srv.URL+"/git-receive-pack", |
| | io.NopCloser(bytes.NewReader(body.Bytes()))) |
| | hreq.ContentLength = -1 // force chunked |
| | resp, err := http.DefaultClient.Do(hreq) |
| | if err != nil { |
| | t.Fatalf("request failed: %v", err) |
| | } |
| | defer resp.Body.Close() |
| | if err := <-errCh; err != nil { |
| | t.Fatalf("chunked body rejected: %v", err) |
| | } |
| | } |
| | ``` |
| | |
| | Run: `go test ./internal/githttp/ -run Chunked -v` |
| | |
| | ### End-to-end reproduction (the one that actually fails) |
| | |
| | A unit test cannot produce a real git pack. This recipe reproduces it on the |
| | demo repo in under a minute; it is the one used for the results above. |
| | |
| | ```bash |
| | git clone https://git.cosmocr.at/git@cosmocr.at/demo.git /tmp/repro |
| | cd /tmp/repro |
| | |
| | # Push to a scratch ref with an EXPLICIT refspec. This clone ships |
| | # push.default=upstream, so a bare `git push origin <branch>` targets the |
| | # branch's upstream (main) rather than creating a same-named branch — |
| | # that mistake polluted main once already. |
| | git checkout -b fr-repro |
| | |
| | # Incompressible payload: zeros would deflate away and never exceed |
| | # postBuffer on the wire. |
| | mkdir -p fr-repro && head -c 3145728 /dev/urandom > fr-repro/payload-3mb.bin |
| | git add fr-repro && git commit -qm "FR-large-push-400 repro payload" |
| | |
| | git config --unset http.postBuffer 2>/dev/null # ensure the 1 MiB default |
| | |
| | # Expected: HTTP 400. |
| | GIT_CURL_VERBOSE=1 git push origin fr-repro:refs/heads/fr-repro 2>&1 | tee /tmp/push.log |
| | |
| | # With the logging change in place, the server log now names the cause: |
| | sudo docker logs --tail 50 patrick_gitweb_1 | grep 'receive-pack decode' |
| | |
| | # Control: identical objects, non-chunked request. Expected: success. |
| | git -c http.postBuffer=524288000 push origin fr-repro:refs/heads/fr-repro |
| | |
| | # Clean up the scratch ref. |
| | git push origin :refs/heads/fr-repro |
| | ``` |
| | |
| | To also confirm ancestry is irrelevant, repeat with `git checkout --orphan` |
| | instead of `git checkout -b`; both shapes fail identically. |
| | |
| | `GIT_CURL_VERBOSE=1` shows whether the request went out chunked and whether |
| | `Expect: 100-continue` was used — which settles hypothesis 2 directly. |
| | |
| | ### Acceptance criteria |
| | |
| | - A failing push writes a specific, actionable reason to the server log. |
| | - The client receives a response body distinguishing "too large" from |
| | "malformed". |
| | - A >1 MiB push succeeds at the default `http.postBuffer`, with no client-side |
| | configuration, for both normal and orphan history. |
| | - `go test ./...` green, chunked regression test included. |
| | |
| | ## Impact |
| | |
| | Low frequency, high friction. It only bites repos over ~1 MiB, but when it does |
| | the failure is opaque, the fix is non-obvious, and the workaround has to be |
| | repeated per-repo per-clone. Of the repos currently hosted, `einkdashboards` |
| | (5.7 MiB) trips it; `mtadash` (328 KiB) does not, which is why this went |
| | unnoticed until now. |
| internal/githttp/githttp.go |
| | // Package githttp implements the git "smart HTTP" wire protocol on top of |
| | // go-git's server transport sessions. go-git ships the upload-pack/receive-pack |
| | // session logic and the pkt-line/packp encoders, but not a net/http handler, so |
| | // this package supplies the framing that stock `git` expects: |
| | // |
| | // GET <repo>/info/refs?service=git-upload-pack -> ref advertisement |
| | // GET <repo>/info/refs?service=git-receive-pack -> ref advertisement |
| | // POST <repo>/git-upload-pack -> pack negotiation (clone/fetch) |
| | // POST <repo>/git-receive-pack -> ref updates (push) |
| | // |
| | // Only protocol v0 is implemented. A client that sends "Git-Protocol: version=2" |
| | // gets no v2 response and transparently falls back to v0. go-git's server does |
| | // not support shallow (--depth) clones. |
| | package githttp |
| | |
| | import ( |
| | "bytes" |
| | "compress/gzip" |
| | "context" |
| + | "errors" |
| | "fmt" |
| | "io" |
| | "net/http" |
| | |
| | "github.com/go-git/go-git/v5/plumbing" |
| | "github.com/go-git/go-git/v5/plumbing/format/pktline" |
| | "github.com/go-git/go-git/v5/plumbing/protocol/packp" |
| | "github.com/go-git/go-git/v5/plumbing/storer" |
| | "github.com/go-git/go-git/v5/plumbing/transport" |
| | "github.com/go-git/go-git/v5/plumbing/transport/server" |
| | ) |
| | |
| | // maxBody bounds a POST body (the packfile for push, or the negotiation for |
| | // fetch) to guard against a malicious or runaway client. |
| | const maxBody = 256 << 20 // 256 MiB |
| | |
| | // Service identifies a git smart-HTTP service. |
| | type Service string |
| | |
| | const ( |
| | UploadPack Service = "git-upload-pack" // clone / fetch |
| | ReceivePack Service = "git-receive-pack" // push |
| | ) |
| | |
| | // fixedLoader is a go-git transport loader that always serves one storer, |
| | // ignoring the requested endpoint. gitweb resolves the repo before dispatch, so |
| | // each request already has its storer. |
| | type fixedLoader struct{ s storer.Storer } |
| | |
| | func (l fixedLoader) Load(*transport.Endpoint) (storer.Storer, error) { return l.s, nil } |
| | |
| | func newUploadSession(s storer.Storer) (transport.UploadPackSession, error) { |
| | return server.NewServer(fixedLoader{s}).NewUploadPackSession(&transport.Endpoint{}, nil) |
| | } |
| | |
| | // NewReceivePackSession returns a receive-pack session bound to s. It is |
| | // exported so the push write-back path (which must run the session against a |
| | // private, writable storer) can reuse the same construction. |
| | func NewReceivePackSession(s storer.Storer) (transport.ReceivePackSession, error) { |
| | return server.NewServer(fixedLoader{s}).NewReceivePackSession(&transport.Endpoint{}, nil) |
| | } |
| | |
| | // InfoRefs writes the ref advertisement for svc over s, in smart-HTTP form: |
| | // the "# service=<svc>" banner and a flush, followed by the pkt-line ref list |
| | // (with capabilities). The caller must not have written a body yet. |
| | func InfoRefs(w http.ResponseWriter, s storer.Storer, svc Service) error { |
| | var ar *packp.AdvRefs |
| | var err error |
| | switch svc { |
| | case UploadPack: |
| | sess, e := newUploadSession(s) |
| | if e != nil { |
| | return e |
| | } |
| | defer sess.Close() |
| | ar, err = sess.AdvertisedReferencesContext(context.Background()) |
| | case ReceivePack: |
| | sess, e := NewReceivePackSession(s) |
| | if e != nil { |
| | return e |
| | } |
| | defer sess.Close() |
| | ar, err = sess.AdvertisedReferencesContext(context.Background()) |
| | default: |
| | return fmt.Errorf("githttp: unknown service %q", svc) |
| | } |
| | if err != nil { |
| | return err |
| | } |
| | |
| | w.Header().Set("Content-Type", fmt.Sprintf("application/x-%s-advertisement", svc)) |
| | w.Header().Set("Cache-Control", "no-cache") |
| | |
| | enc := pktline.NewEncoder(w) |
| | if err := enc.Encodef("# service=%s\n", svc); err != nil { |
| | return err |
| | } |
| | if err := enc.Flush(); err != nil { |
| | return err |
| | } |
| | return ar.Encode(w) |
| | } |
| | |
| | // negotiation is the tail of an upload-pack request: the "have" lines the |
| | // client is offering, and whether it closed the request with "done". |
| | type negotiation struct { |
| | haves []plumbing.Hash |
| | done bool |
| | } |
| | |
| | // decodeNegotiation reads the have/done section that follows the upload |
| | // request. UploadRequest.Decode stops at the flush-pkt ending the want section, |
| | // so this part of the body is still unread; without it req.Haves stays empty |
| | // and every fetch re-sends the full history. |
| | func decodeNegotiation(r io.Reader) (negotiation, error) { |
| | var n negotiation |
| | sc := pktline.NewScanner(r) |
| | for sc.Scan() { |
| | line := bytes.TrimSuffix(sc.Bytes(), []byte("\n")) |
| | switch { |
| | case len(line) == 0: |
| | // flush-pkt: end of this batch of haves. |
| | case bytes.Equal(line, []byte("done")): |
| | n.done = true |
| | return n, nil |
| | case bytes.HasPrefix(line, []byte("have ")): |
| | text := string(bytes.TrimPrefix(line, []byte("have "))) |
| | h := plumbing.NewHash(text) |
| | if h.IsZero() { |
| | return n, fmt.Errorf("malformed have line %q", line) |
| | } |
| | n.haves = append(n.haves, h) |
| | } |
| | } |
| | return n, sc.Err() |
| | } |
| | |
| | // ServeUploadPack handles a POST git-upload-pack: it decodes the client's |
| | // negotiation from the (possibly gzip-compressed) body, runs upload-pack over |
| | // s, and streams the packfile response. |
| | // |
| | // Over stateless HTTP the client negotiates across several requests. It offers |
| | // its haves in batches — the first holds 16 — and ends a batch that is not its |
| | // last with a flush-pkt and no "done". In that round it is reading ACK/NAK |
| | // pkt-lines to decide what to offer next, so a packfile must not be sent: the |
| | // client would read the literal "PACK" where a pkt-line length belongs and fail |
| | // with "protocol error: bad line length character: PACK". We answer such a |
| | // round with a bare NAK; the commit we share is acknowledged only once the |
| | // client sends "done", which is also when the pack goes out. |
| | func ServeUploadPack(w http.ResponseWriter, r *http.Request, s storer.Storer) error { |
| | body, err := requestBody(r) |
| | if err != nil { |
| | return err |
| | } |
| | defer body.Close() |
| | |
| | req := packp.NewUploadPackRequest() |
| | if err := req.Decode(body); err != nil { |
| | return err |
| | } |
| | neg, err := decodeNegotiation(body) |
| | if err != nil { |
| | return err |
| | } |
| | // Only the haves we actually hold may be passed on. The session walks them |
| | // to build the exclusion set and fails on any object it cannot find, and a |
| | // client with unpushed local commits legitimately offers some we lack. |
| | // Excluding everything reachable from the rest is what keeps a fetch from |
| | // re-sending history the client already has. |
| | common := commonHaves(s, neg.haves) |
| | req.Haves = common |
| | |
| | if !neg.done { |
| | // The client is still negotiating, so no packfile yet -- but the answer |
| | // must be NAK even when we do share a commit. We advertise no multi_ack, |
| | // and in that mode a plain ACK does not mean "noted": it means |
| | // negotiation is over and the pack follows. git acts on it immediately, |
| | // sending a bare "done" request carrying no haves and reading that |
| | // response as a raw packfile, so an ACK here costs both the negotiated |
| | // haves and the fetch, which fails with "protocol error: bad pack |
| | // header". The common base is acknowledged on the "done" round instead. |
| | setUploadPackHeaders(w) |
| | return pktline.NewEncoder(w).Encode([]byte("NAK\n")) |
| | } |
| | |
| | sess, err := newUploadSession(s) |
| | if err != nil { |
| | return err |
| | } |
| | defer sess.Close() |
| | |
| | resp, err := sess.UploadPack(r.Context(), req) |
| | if err != nil { |
| | return err |
| | } |
| | // The client has finished negotiating, so this ACK names the base the pack |
| | // was cut against rather than cutting negotiation short. Without it |
| | // find_common returns "no common commits" and git warns on every fetch. |
| | if len(common) > 0 { |
| | resp.ACKs = []plumbing.Hash{common[0]} |
| | } |
| | |
| | setUploadPackHeaders(w) |
| | return resp.Encode(w) |
| | } |
| | |
| | // commonHaves returns the haves present in s, in the order the client offered |
| | // them — git sends its newest commits first, so the first entry is the most |
| | // recent commit the two sides share. |
| | func commonHaves(s storer.Storer, haves []plumbing.Hash) []plumbing.Hash { |
| | var common []plumbing.Hash |
| | for _, h := range haves { |
| | if _, err := s.EncodedObject(plumbing.AnyObject, h); err == nil { |
| | common = append(common, h) |
| | } |
| | } |
| | return common |
| | } |
| | |
| | func setUploadPackHeaders(w http.ResponseWriter) { |
| | w.Header().Set("Content-Type", fmt.Sprintf("application/x-%s-result", UploadPack)) |
| | w.Header().Set("Cache-Control", "no-cache") |
| | } |
| | |
| + | // ErrProbe is returned by DecodeReceivePack for git's probe request: a POST |
| + | // whose whole body is a flush-pkt ("0000"). git sends it before a request too |
| + | // large for http.postBuffer, to check that the endpoint works (and that it is |
| + | // authorized) before streaming the real body chunked. It must be answered |
| + | // with 200 and an empty result, or git abandons the push. |
| + | var ErrProbe = errors.New("receive-pack probe") |
| + | |
| | // DecodeReceivePack parses a POST git-receive-pack body into a reference-update |
| | // request. The returned request's Packfile is a streaming reader over the rest |
| | // of the body; the caller runs the receive-pack session (see the push package) |
| - | // and then calls WriteReceivePackResult. |
| + | // and then calls WriteReceivePackResult. It returns ErrProbe for git's |
| + | // flush-only probe request. |
| | func DecodeReceivePack(r *http.Request) (*packp.ReferenceUpdateRequest, error) { |
| | body, err := requestBody(r) |
| | if err != nil { |
| | return nil, err |
| | } |
| + | // The first pkt-line is always sent in full before the client waits for |
| + | // anything, so reading its 4-byte length can't block. A real push starts |
| + | // with a command; only the probe starts with a flush. |
| + | var head [4]byte |
| + | if _, err := io.ReadFull(body, head[:]); err != nil { |
| + | body.Close() |
| + | return nil, fmt.Errorf("read receive-pack request: %w", err) |
| + | } |
| + | if string(head[:]) == "0000" { |
| + | body.Close() |
| + | return nil, ErrProbe |
| + | } |
| + | body = prefixedBody{Reader: io.MultiReader(bytes.NewReader(head[:]), body), Closer: body} |
| + | |
| | // Note: body is not closed here; req.Packfile streams from it and is closed |
| | // by the receive-pack session. |
| | req := packp.NewReferenceUpdateRequest() |
| | if err := req.Decode(body); err != nil { |
| | body.Close() |
| | return nil, err |
| | } |
| | // A pack follows the commands only when at least one ref is created or |
| | // updated; a delete-only push sends none (same rule as stock |
| | // receive-pack). Decode leaves the body reader in req.Packfile regardless, |
| | // and go-git's session unpacks whenever Packfile is non-nil, failing on |
| | // the packless stream ("empty packfile") — so nil it out. Detection must |
| - | // come from the commands, not from probing the body: a read would block, |
| - | // because the client transmits nothing further until it sees our response. |
| + | // come from the commands, not from probing the body past them: a read |
| + | // would block, because the client transmits nothing further until it sees |
| + | // our response. |
| | needPack := false |
| | for _, cmd := range req.Commands { |
| | if cmd.Action() != packp.Delete { |
| | needPack = true |
| | break |
| | } |
| | } |
| | if !needPack { |
| | body.Close() |
| | req.Packfile = nil |
| | } |
| | return req, nil |
| | } |
| | |
| | // WriteReceivePackResult writes the report-status for a completed push. A nil |
| | // status (client did not request report-status) writes nothing. |
| | func WriteReceivePackResult(w http.ResponseWriter, rs *packp.ReportStatus) error { |
| | w.Header().Set("Content-Type", fmt.Sprintf("application/x-%s-result", ReceivePack)) |
| | w.Header().Set("Cache-Control", "no-cache") |
| | if rs == nil { |
| | return nil |
| | } |
| | return rs.Encode(w) |
| | } |
| | |
| | // requestBody returns the request body, transparently decompressing it when the |
| | // client sent Content-Encoding: gzip (stock git gzips larger request bodies), |
| | // and capping the amount read. |
| | func requestBody(r *http.Request) (io.ReadCloser, error) { |
| | var body io.ReadCloser = http.MaxBytesReader(nil, r.Body, maxBody) |
| | if r.Header.Get("Content-Encoding") == "gzip" { |
| | zr, err := gzip.NewReader(body) |
| | if err != nil { |
| | body.Close() |
| | return nil, err |
| | } |
| | return gzipBody{zr: zr, under: body}, nil |
| | } |
| | return body, nil |
| + | } |
| + | |
| + | // prefixedBody puts bytes already read back in front of a body, keeping the |
| + | // original's Close. |
| + | type prefixedBody struct { |
| + | io.Reader |
| + | io.Closer |
| | } |
| | |
| | // gzipBody couples a gzip reader with the underlying capped body so both are |
| | // closed together. |
| | type gzipBody struct { |
| | zr *gzip.Reader |
| | under io.ReadCloser |
| | } |
| | |
| | func (g gzipBody) Read(p []byte) (int, error) { return g.zr.Read(p) } |
| | func (g gzipBody) Close() error { |
| | err := g.zr.Close() |
| | if cerr := g.under.Close(); err == nil { |
| | err = cerr |
| | } |
| | return err |
| | } |
| internal/githttp/githttp_test.go |
| | package githttp |
| | |
| | import ( |
| | "bytes" |
| | "compress/gzip" |
| + | "errors" |
| + | "fmt" |
| | "io" |
| + | "net/http" |
| | "net/http/httptest" |
| | "strings" |
| | "testing" |
| | |
| | "github.com/go-git/go-git/v5/storage/memory" |
| | ) |
| | |
| | func TestInfoRefsBanner(t *testing.T) { |
| | rec := httptest.NewRecorder() |
| | if err := InfoRefs(rec, memory.NewStorage(), UploadPack); err != nil { |
| | t.Fatalf("InfoRefs: %v", err) |
| | } |
| | body := rec.Body.String() |
| | // pkt-line: 4-hex length ("001e") + "# service=git-upload-pack\n", then a |
| | // flush-pkt "0000". |
| | const want = "001e# service=git-upload-pack\n0000" |
| | if !strings.HasPrefix(body, want) { |
| | t.Fatalf("advertisement prefix = %q, want prefix %q", body[:min(len(body), 40)], want) |
| | } |
| | if ct := rec.Header().Get("Content-Type"); ct != "application/x-git-upload-pack-advertisement" { |
| | t.Errorf("Content-Type = %q", ct) |
| | } |
| | if cc := rec.Header().Get("Cache-Control"); cc != "no-cache" { |
| | t.Errorf("Cache-Control = %q", cc) |
| | } |
| | } |
| | |
| | func TestInfoRefsReceivePackCapabilities(t *testing.T) { |
| | rec := httptest.NewRecorder() |
| | if err := InfoRefs(rec, memory.NewStorage(), ReceivePack); err != nil { |
| | t.Fatalf("InfoRefs: %v", err) |
| | } |
| | body := rec.Body.String() |
| | if !strings.HasPrefix(body, "001f# service=git-receive-pack\n0000") { |
| | t.Fatalf("receive-pack banner wrong: %q", body[:min(len(body), 40)]) |
| | } |
| | // The advertisement must offer the capabilities stock git needs to push and |
| | // delete branches, and to receive a status report. |
| | for _, cap := range []string{"report-status", "delete-refs"} { |
| | if !strings.Contains(body, cap) { |
| | t.Errorf("advertisement missing capability %q", cap) |
| | } |
| | } |
| | } |
| | |
| | func TestRequestBodyGzip(t *testing.T) { |
| | var gz bytes.Buffer |
| | zw := gzip.NewWriter(&gz) |
| | const payload = "0009done\n" |
| | if _, err := zw.Write([]byte(payload)); err != nil { |
| | t.Fatal(err) |
| | } |
| | zw.Close() |
| | |
| | r := httptest.NewRequest("POST", "/x/git-upload-pack", bytes.NewReader(gz.Bytes())) |
| | r.Header.Set("Content-Encoding", "gzip") |
| | |
| | body, err := requestBody(r) |
| | if err != nil { |
| | t.Fatalf("requestBody: %v", err) |
| | } |
| | defer body.Close() |
| | got, err := io.ReadAll(body) |
| | if err != nil { |
| | t.Fatalf("read: %v", err) |
| | } |
| | if string(got) != payload { |
| | t.Fatalf("gunzipped body = %q, want %q", got, payload) |
| | } |
| | } |
| + | |
| + | // receivePackBody is a minimal create-branch request followed by payload. |
| + | func receivePackBody(payload []byte) []byte { |
| + | const ( |
| + | oldOID = "0000000000000000000000000000000000000000" |
| + | newOID = "1111111111111111111111111111111111111111" |
| + | ) |
| + | cmd := fmt.Sprintf("%s %s refs/heads/master\x00report-status\n", oldOID, newOID) |
| + | var body bytes.Buffer |
| + | fmt.Fprintf(&body, "%04x%s", len(cmd)+4, cmd) |
| + | body.WriteString("0000") |
| + | body.Write(payload) |
| + | return body.Bytes() |
| + | } |
| + | |
| + | func TestDecodeReceivePackProbe(t *testing.T) { |
| + | r := httptest.NewRequest("POST", "/git-receive-pack", strings.NewReader("0000")) |
| + | if _, err := DecodeReceivePack(r); !errors.Is(err, ErrProbe) { |
| + | t.Fatalf("probe: err = %v, want ErrProbe", err) |
| + | } |
| + | |
| + | // A real request still decodes, with its first pkt-line intact after the |
| + | // 4-byte peek. |
| + | r = httptest.NewRequest("POST", "/git-receive-pack", bytes.NewReader(receivePackBody([]byte("PACKDATA")))) |
| + | req, err := DecodeReceivePack(r) |
| + | if err != nil { |
| + | t.Fatalf("real request: %v", err) |
| + | } |
| + | if len(req.Commands) != 1 || req.Commands[0].Name != "refs/heads/master" { |
| + | t.Fatalf("commands = %+v", req.Commands) |
| + | } |
| + | rest, _ := io.ReadAll(req.Packfile) |
| + | if string(rest) != "PACKDATA" { |
| + | t.Errorf("pack stream = %q", rest) |
| + | } |
| + | |
| + | // Empty and truncated bodies are errors, not probes. |
| + | for _, body := range []string{"", "00"} { |
| + | r = httptest.NewRequest("POST", "/git-receive-pack", strings.NewReader(body)) |
| + | if _, err := DecodeReceivePack(r); err == nil || errors.Is(err, ErrProbe) { |
| + | t.Errorf("body %q: err = %v", body, err) |
| + | } |
| + | } |
| + | } |
| + | |
| + | func TestDecodeReceivePackProbeGzip(t *testing.T) { |
| + | var gz bytes.Buffer |
| + | zw := gzip.NewWriter(&gz) |
| + | zw.Write([]byte("0000")) |
| + | zw.Close() |
| + | r := httptest.NewRequest("POST", "/git-receive-pack", &gz) |
| + | r.Header.Set("Content-Encoding", "gzip") |
| + | if _, err := DecodeReceivePack(r); !errors.Is(err, ErrProbe) { |
| + | t.Fatalf("gzipped probe: err = %v, want ErrProbe", err) |
| + | } |
| + | } |
| + | |
| + | // TestDecodeReceivePackChunked drives a chunked body through a real |
| + | // http.Server, which is what git sends once a request exceeds |
| + | // http.postBuffer (default 1 MiB). Kept from docs/FR-large-push-400.md. |
| + | func TestDecodeReceivePackChunked(t *testing.T) { |
| + | var pack bytes.Buffer |
| + | pack.WriteString("PACK") |
| + | pack.Write([]byte{0, 0, 0, 2, 0, 0, 0, 0}) |
| + | pack.Write(bytes.Repeat([]byte{0xAB}, 6<<20)) // past postBuffer |
| + | pack.Write(make([]byte, 20)) |
| + | body := receivePackBody(pack.Bytes()) |
| + | |
| + | errCh := make(chan error, 1) |
| + | srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { |
| + | if r.ContentLength != -1 { |
| + | t.Errorf("ContentLength = %d, want -1 (chunked)", r.ContentLength) |
| + | } |
| + | req, err := DecodeReceivePack(r) |
| + | if err == nil && req.Packfile != nil { |
| + | _, err = io.Copy(io.Discard, req.Packfile) |
| + | } |
| + | errCh <- err |
| + | })) |
| + | defer srv.Close() |
| + | |
| + | hreq, _ := http.NewRequest(http.MethodPost, srv.URL+"/git-receive-pack", io.NopCloser(bytes.NewReader(body))) |
| + | hreq.ContentLength = -1 // force chunked |
| + | resp, err := http.DefaultClient.Do(hreq) |
| + | if err != nil { |
| + | t.Fatalf("request failed: %v", err) |
| + | } |
| + | resp.Body.Close() |
| + | if err := <-errCh; err != nil { |
| + | t.Fatalf("chunked body rejected: %v", err) |
| + | } |
| + | } |
| internal/server/clone_push_test.go |
| | package server |
| | |
| | import ( |
| + | "bytes" |
| + | "crypto/rand" |
| | "crypto/sha256" |
| | "encoding/hex" |
| | "fmt" |
| | "net/http" |
| | "net/http/httptest" |
| | "net/url" |
| | "os" |
| | "os/exec" |
| | "path/filepath" |
| | "strings" |
| | "testing" |
| | |
| | "gitweb/internal/store" |
| | ) |
| | |
| | // gitEnv returns a hermetic environment for invoking git in tests: no prompts, |
| | // no system/global config, no credential helpers, and an isolated HOME. |
| | func gitEnv(t *testing.T) []string { |
| | return append(os.Environ(), |
| | "HOME="+t.TempDir(), |
| | "GIT_TERMINAL_PROMPT=0", |
| | "GIT_CONFIG_NOSYSTEM=1", |
| | "GIT_ASKPASS=true", |
| | "GIT_AUTHOR_NAME=T", "GIT_AUTHOR_EMAIL=t@e.com", |
| | "GIT_COMMITTER_NAME=T", "GIT_COMMITTER_EMAIL=t@e.com", |
| | ) |
| | } |
| | |
| | func runGit(t *testing.T, env []string, dir string, args ...string) (string, error) { |
| | t.Helper() |
| | c := exec.Command("git", args...) |
| | c.Dir = dir |
| | c.Env = env |
| | out, err := c.CombinedOutput() |
| | return string(out), err |
| | } |
| | |
| | func mustGit(t *testing.T, env []string, dir string, args ...string) string { |
| | t.Helper() |
| | out, err := runGit(t, env, dir, args...) |
| | if err != nil { |
| | t.Fatalf("git %s: %v\n%s", strings.Join(args, " "), err, out) |
| | } |
| | return out |
| | } |
| | |
| | // makeBareRepo builds a bare repo at ns/<repoPath> with one commit, and writes |
| | // an Access file with the given content beside it. |
| | func makeBareRepo(t *testing.T, env []string, ns, repoPath, access string) { |
| | t.Helper() |
| | work := t.TempDir() |
| | mustGit(t, env, work, "init", "-q", "-b", "main") |
| | if err := os.WriteFile(filepath.Join(work, "README.md"), []byte("# hello\n"), 0o644); err != nil { |
| | t.Fatal(err) |
| | } |
| | mustGit(t, env, work, "add", "-A") |
| | mustGit(t, env, work, "commit", "-qm", "initial") |
| | |
| | dst := filepath.Join(ns, filepath.FromSlash(repoPath)) |
| | if err := os.MkdirAll(filepath.Dir(dst), 0o755); err != nil { |
| | t.Fatal(err) |
| | } |
| | mustGit(t, env, work, "clone", "-q", "--bare", work, dst) |
| | if err := os.WriteFile(filepath.Join(dst, "Access"), []byte(access), 0o644); err != nil { |
| | t.Fatal(err) |
| | } |
| | } |
| | |
| | // writeTokens writes a gitweb tokens file granting the given plaintext tokens to |
| | // user (by storing their SHA-256 hashes). |
| | func writeTokens(t *testing.T, ns, user string, tokens ...string) { |
| | t.Helper() |
| | dir := filepath.Join(ns, user, "gitweb") |
| | if err := os.MkdirAll(dir, 0o755); err != nil { |
| | t.Fatal(err) |
| | } |
| | var b strings.Builder |
| | for _, tok := range tokens { |
| | h := sha256.Sum256([]byte(tok)) |
| | fmt.Fprintf(&b, "%s token\n", hex.EncodeToString(h[:])) |
| | } |
| | if err := os.WriteFile(filepath.Join(dir, "tokens"), []byte(b.String()), 0o644); err != nil { |
| | t.Fatal(err) |
| | } |
| | } |
| | |
| | // cloneURL builds an http clone URL, optionally embedding Basic-auth creds. |
| | func cloneURL(base, repoPath, user, token string) string { |
| | u, _ := url.Parse(base) |
| | if user != "" { |
| | u.User = url.UserPassword(user, token) |
| | } |
| | u.Path = "/" + repoPath |
| | return u.String() |
| | } |
| | |
| | func TestCloneAndPush(t *testing.T) { |
| | if _, err := exec.LookPath("git"); err != nil { |
| | t.Skip("git not available") |
| | } |
| | env := gitEnv(t) |
| | ns := t.TempDir() |
| | |
| | const owner = "u@d.com" |
| | const writer = "w@d.com" |
| | const reader = "r@d.com" |
| | |
| | makeBareRepo(t, env, ns, owner+"/public.git", "read,list: all\n*: "+owner+"\n") |
| | makeBareRepo(t, env, ns, owner+"/private.git", |
| | "*: "+owner+"\nread,list,write,create,delete: "+writer+"\nread,list: "+reader+"\n") |
| | writeTokens(t, ns, writer, "writer-token") |
| | writeTokens(t, ns, reader, "reader-token") |
| | |
| | web := os.DirFS("../../web") |
| | cacheDir := t.TempDir() |
| | srv, err := New(Config{Store: store.NewLocalStore(ns), Web: http.FS(web), CacheDir: cacheDir}) |
| | if err != nil { |
| | t.Fatal(err) |
| | } |
| | ts := httptest.NewServer(srv.Handler()) |
| | defer ts.Close() |
| | |
| | t.Run("anon clone public", func(t *testing.T) { |
| | dst := filepath.Join(t.TempDir(), "pub") |
| | if out, err := runGit(t, env, ".", "clone", "-q", cloneURL(ts.URL, owner+"/public.git", "", ""), dst); err != nil { |
| | t.Fatalf("clone public failed: %v\n%s", err, out) |
| | } |
| | if _, err := os.Stat(filepath.Join(dst, "README.md")); err != nil { |
| | t.Fatalf("cloned repo missing README: %v", err) |
| | } |
| | }) |
| | |
| | t.Run("anon clone private denied", func(t *testing.T) { |
| | dst := filepath.Join(t.TempDir(), "priv") |
| | if out, err := runGit(t, env, ".", "clone", "-q", cloneURL(ts.URL, owner+"/private.git", "", ""), dst); err == nil { |
| | t.Fatalf("anonymous clone of private repo succeeded, want failure\n%s", out) |
| | } |
| | }) |
| | |
| | t.Run("authed clone private", func(t *testing.T) { |
| | dst := filepath.Join(t.TempDir(), "priv-ok") |
| | u := cloneURL(ts.URL, owner+"/private.git", writer, "writer-token") |
| | if out, err := runGit(t, env, ".", "clone", "-q", u, dst); err != nil { |
| | t.Fatalf("authed clone failed: %v\n%s", err, out) |
| | } |
| | }) |
| | |
| | t.Run("wrong token denied", func(t *testing.T) { |
| | dst := filepath.Join(t.TempDir(), "priv-bad") |
| | u := cloneURL(ts.URL, owner+"/private.git", writer, "nope") |
| | if out, err := runGit(t, env, ".", "clone", "-q", u, dst); err == nil { |
| | t.Fatalf("clone with wrong token succeeded, want failure\n%s", out) |
| | } |
| | }) |
| | |
| | t.Run("push as writer", func(t *testing.T) { |
| | dst := filepath.Join(t.TempDir(), "push") |
| | u := cloneURL(ts.URL, owner+"/private.git", writer, "writer-token") |
| | mustGit(t, env, ".", "clone", "-q", u, dst) |
| | if err := os.WriteFile(filepath.Join(dst, "new.txt"), []byte("added\n"), 0o644); err != nil { |
| | t.Fatal(err) |
| | } |
| | mustGit(t, env, dst, "add", "-A") |
| | mustGit(t, env, dst, "commit", "-qm", "add new.txt") |
| | if out, err := runGit(t, env, dst, "push", "-q", "origin", "main"); err != nil { |
| | t.Fatalf("push failed: %v\n%s", err, out) |
| | } |
| | // A fresh clone must see the pushed commit. |
| | fresh := filepath.Join(t.TempDir(), "fresh") |
| | mustGit(t, env, ".", "clone", "-q", u, fresh) |
| | if _, err := os.Stat(filepath.Join(fresh, "new.txt")); err != nil { |
| | t.Fatalf("pushed file not visible after re-clone: %v", err) |
| | } |
| | }) |
| | |
| | t.Run("incremental push (thin pack)", func(t *testing.T) { |
| | dst := filepath.Join(t.TempDir(), "thin") |
| | u := cloneURL(ts.URL, owner+"/private.git", writer, "writer-token") |
| | mustGit(t, env, ".", "clone", "-q", u, dst) |
| | // A file large enough that git deltas the second version against the |
| | // first. On the second push that base blob is not in the pack (the |
| | // server already has it), so git sends a thin pack with a REF_DELTA — |
| | // the case that used to fail with "reference delta not found". |
| | lines := strings.Repeat("a line of text that compresses and deltas well\n", 200) |
| | if err := os.WriteFile(filepath.Join(dst, "big.txt"), []byte(lines), 0o644); err != nil { |
| | t.Fatal(err) |
| | } |
| | mustGit(t, env, dst, "add", "-A") |
| | mustGit(t, env, dst, "commit", "-qm", "add big.txt") |
| | mustGit(t, env, dst, "push", "-q", "origin", "main") |
| | |
| | if err := os.WriteFile(filepath.Join(dst, "big.txt"), []byte(lines+"one changed line\n"), 0o644); err != nil { |
| | t.Fatal(err) |
| | } |
| | mustGit(t, env, dst, "add", "-A") |
| | mustGit(t, env, dst, "commit", "-qm", "append to big.txt") |
| | if out, err := runGit(t, env, dst, "push", "origin", "main"); err != nil { |
| | t.Fatalf("incremental push failed: %v\n%s", err, out) |
| | } |
| | |
| | fresh := filepath.Join(t.TempDir(), "fresh-thin") |
| | mustGit(t, env, ".", "clone", "-q", u, fresh) |
| | got, err := os.ReadFile(filepath.Join(fresh, "big.txt")) |
| | if err != nil { |
| | t.Fatal(err) |
| | } |
| | if string(got) != lines+"one changed line\n" { |
| | t.Error("re-clone does not have the incrementally pushed content") |
| | } |
| | }) |
| | |
| | t.Run("push branch create and delete", func(t *testing.T) { |
| | dst := filepath.Join(t.TempDir(), "branch") |
| | u := cloneURL(ts.URL, owner+"/private.git", writer, "writer-token") |
| | mustGit(t, env, ".", "clone", "-q", u, dst) |
| | mustGit(t, env, dst, "push", "-q", "origin", "main:refs/heads/scratch") |
| | // Deleting a ref sends no packfile at all; the server must not choke |
| | // on the empty pack stream. |
| | if out, err := runGit(t, env, dst, "push", "origin", ":refs/heads/scratch"); err != nil { |
| | t.Fatalf("delete push failed: %v\n%s", err, out) |
| | } |
| | refs := mustGit(t, env, dst, "ls-remote", "origin") |
| | if strings.Contains(refs, "scratch") { |
| | t.Errorf("scratch branch still on remote:\n%s", refs) |
| + | } |
| + | }) |
| + | |
| + | // Over http.postBuffer, git first sends a probe POST whose body is just a |
| + | // flush-pkt ("0000") and only streams the real (chunked) pack after the |
| + | // probe gets a 200. Answering the probe with 400 used to fail every push |
| + | // over ~1 MiB (docs/FR-large-push-400.md). |
| + | t.Run("large push over http.postBuffer", func(t *testing.T) { |
| + | dst := filepath.Join(t.TempDir(), "large") |
| + | u := cloneURL(ts.URL, owner+"/private.git", writer, "writer-token") |
| + | mustGit(t, env, ".", "clone", "-q", u, dst) |
| + | payload := make([]byte, 3<<20) // incompressible, so the wire size stays > 1 MiB |
| + | if _, err := rand.Read(payload); err != nil { |
| + | t.Fatal(err) |
| + | } |
| + | if err := os.WriteFile(filepath.Join(dst, "payload.bin"), payload, 0o644); err != nil { |
| + | t.Fatal(err) |
| + | } |
| + | mustGit(t, env, dst, "add", "-A") |
| + | mustGit(t, env, dst, "commit", "-qm", "add 3 MiB payload") |
| + | // Pin git's default postBuffer (1 MiB) so the test doesn't depend on |
| + | // the environment. |
| + | if out, err := runGit(t, env, dst, "-c", "http.postBuffer=1048576", "push", "origin", "main"); err != nil { |
| + | t.Fatalf("large push failed: %v\n%s", err, out) |
| + | } |
| + | fresh := filepath.Join(t.TempDir(), "fresh-large") |
| + | mustGit(t, env, ".", "clone", "-q", u, fresh) |
| + | got, err := os.ReadFile(filepath.Join(fresh, "payload.bin")) |
| + | if err != nil || !bytes.Equal(got, payload) { |
| + | t.Fatalf("re-clone lacks the large payload (err=%v, %d bytes)", err, len(got)) |
| + | } |
| + | }) |
| + | |
| + | t.Run("large push as reader denied", func(t *testing.T) { |
| + | // The probe must still be authorized: a reader gets refused at the |
| + | // probe, before any pack is sent. |
| + | dst := filepath.Join(t.TempDir(), "large-ro") |
| + | u := cloneURL(ts.URL, owner+"/private.git", reader, "reader-token") |
| + | mustGit(t, env, ".", "clone", "-q", u, dst) |
| + | payload := make([]byte, 2<<20) |
| + | rand.Read(payload) |
| + | if err := os.WriteFile(filepath.Join(dst, "ro.bin"), payload, 0o644); err != nil { |
| + | t.Fatal(err) |
| + | } |
| + | mustGit(t, env, dst, "add", "-A") |
| + | mustGit(t, env, dst, "commit", "-qm", "reader large") |
| + | if out, err := runGit(t, env, dst, "-c", "http.postBuffer=1048576", "push", "origin", "main"); err == nil { |
| + | t.Fatalf("reader large push succeeded, want failure\n%s", out) |
| | } |
| | }) |
| | |
| | t.Run("push copies cleaned up", func(t *testing.T) { |
| | // Every push materializes a private push-* dir under the cache root and |
| | // must remove it when done. |
| | children, err := os.ReadDir(cacheDir) |
| | if err != nil { |
| | t.Fatal(err) |
| | } |
| | for _, ch := range children { |
| | if strings.HasPrefix(ch.Name(), "push-") { |
| | t.Errorf("leftover push dir %s in cache root", ch.Name()) |
| | } |
| | } |
| | }) |
| | |
| | t.Run("push as reader denied", func(t *testing.T) { |
| | dst := filepath.Join(t.TempDir(), "push-ro") |
| | u := cloneURL(ts.URL, owner+"/private.git", reader, "reader-token") |
| | mustGit(t, env, ".", "clone", "-q", u, dst) |
| | if err := os.WriteFile(filepath.Join(dst, "sneaky.txt"), []byte("x\n"), 0o644); err != nil { |
| | t.Fatal(err) |
| | } |
| | mustGit(t, env, dst, "add", "-A") |
| | mustGit(t, env, dst, "commit", "-qm", "sneak") |
| | if out, err := runGit(t, env, dst, "push", "-q", "origin", "main"); err == nil { |
| | t.Fatalf("reader push succeeded, want 403 failure\n%s", out) |
| | } |
| | }) |
| | } |
| internal/server/githttp.go |
| | package server |
| | |
| | import ( |
| + | "errors" |
| + | "fmt" |
| | "log" |
| | "net/http" |
| | "path" |
| | "strings" |
| | |
| | "gitweb/internal/githttp" |
| | "gitweb/internal/gitrepo" |
| | ) |
| | |
| | // gitKind identifies which git smart-HTTP endpoint a request hit. |
| | type gitKind int |
| | |
| | const ( |
| | gitInfoRefs gitKind = iota |
| | gitUploadPack |
| | gitReceivePack |
| | ) |
| | |
| | // gitReq describes a matched git smart-HTTP request. |
| | type gitReq struct { |
| | kind gitKind |
| | service githttp.Service // the effective service (upload-pack or receive-pack) |
| | urlPath string // URL path with the git suffix stripped, e.g. "/u@d.com/x.git" |
| | } |
| | |
| | // gitEndpoint recognizes the three git smart-HTTP endpoints by method and path |
| | // suffix. It returns ok=false for ordinary browse requests. |
| | func gitEndpoint(r *http.Request) (gitReq, bool) { |
| | p := r.URL.Path |
| | switch { |
| | case r.Method == http.MethodGet && strings.HasSuffix(p, "/info/refs"): |
| | svc := r.URL.Query().Get("service") |
| | switch githttp.Service(svc) { |
| | case githttp.UploadPack, githttp.ReceivePack: |
| | return gitReq{gitInfoRefs, githttp.Service(svc), strings.TrimSuffix(p, "/info/refs")}, true |
| | } |
| | // info/refs without a smart service param: fall through to browse (this |
| | // also lets someone view a repo's literal info/refs file). |
| | return gitReq{}, false |
| | case r.Method == http.MethodPost && strings.HasSuffix(p, "/git-upload-pack"): |
| | return gitReq{gitUploadPack, githttp.UploadPack, strings.TrimSuffix(p, "/git-upload-pack")}, true |
| | case r.Method == http.MethodPost && strings.HasSuffix(p, "/git-receive-pack"): |
| | return gitReq{gitReceivePack, githttp.ReceivePack, strings.TrimSuffix(p, "/git-receive-pack")}, true |
| | } |
| | return gitReq{}, false |
| | } |
| | |
| | // handleGitHTTP serves a git smart-HTTP request: authenticate, resolve the repo, |
| | // authorize (read for clone/fetch, write for push), then run the protocol. |
| | func (s *Server) handleGitHTTP(w http.ResponseWriter, r *http.Request, ep gitReq) { |
| | user, clean, ok := parseUpspinPath(ep.urlPath) |
| | if !ok { |
| | s.notFound(w) |
| | return |
| | } |
| | |
| | authUser, ok := s.authenticate(w, r) |
| | if !ok { |
| | return // 401 already written |
| | } |
| | |
| | repoPath, rest, isRepo, err := gitrepo.Locate(s.store, user, clean) |
| | if err != nil { |
| | s.storeErr(w, err) |
| | return |
| | } |
| | // Endpoints live only at a repo root. |
| | if !isRepo || rest != "" { |
| | s.denyRead(w, r, authUser) |
| | return |
| | } |
| | |
| | writing := ep.service == githttp.ReceivePack |
| | if writing { |
| | // Push always requires an authenticated, authorized writer. |
| | if authUser == "" { |
| | s.challenge(w) |
| | return |
| | } |
| | can, err := s.authz.CanWrite(authUser, repoPath) |
| | if err != nil { |
| | s.serverErr(w, err) |
| | return |
| | } |
| | if !can { |
| | // The caller is known; distinguish "can't read" (hide it) from |
| | // "can read but not write" (explicit 403). |
| | readable, rerr := s.authz.CanRead(authUser, repoPath) |
| | if rerr != nil { |
| | s.serverErr(w, rerr) |
| | return |
| | } |
| | if !readable { |
| | s.notFound(w) |
| | return |
| | } |
| | http.Error(w, "403 forbidden: no write access", http.StatusForbidden) |
| | return |
| | } |
| | } else { |
| | can, err := s.authz.CanRead(authUser, repoPath) |
| | if err != nil { |
| | s.serverErr(w, err) |
| | return |
| | } |
| | if !can { |
| | s.denyRead(w, r, authUser) |
| | return |
| | } |
| | } |
| | |
| | switch ep.kind { |
| | case gitInfoRefs: |
| | storer, release, err := s.repos.GetStorer(repoPath) |
| | if err != nil { |
| | s.storeErr(w, err) |
| | return |
| | } |
| | defer release() |
| | if err := githttp.InfoRefs(w, storer, ep.service); err != nil { |
| | log.Printf("gitweb: info/refs %s: %v", repoPath, err) |
| | } |
| | case gitUploadPack: |
| | // upload-pack streams the packfile from the storer for the whole |
| | // response; the deferred release keeps the repo's on-disk backing |
| | // alive until ServeUploadPack returns. |
| | storer, release, err := s.repos.GetStorer(repoPath) |
| | if err != nil { |
| | s.storeErr(w, err) |
| | return |
| | } |
| | defer release() |
| | if err := githttp.ServeUploadPack(w, r, storer); err != nil { |
| | log.Printf("gitweb: upload-pack %s: %v", repoPath, err) |
| | } |
| | case gitReceivePack: |
| | req, err := githttp.DecodeReceivePack(r) |
| + | if errors.Is(err, githttp.ErrProbe) { |
| + | // git checking the endpoint before streaming a large push. |
| + | if err := githttp.WriteReceivePackResult(w, nil); err != nil { |
| + | log.Printf("gitweb: receive-pack probe %s: %v", repoPath, err) |
| + | } |
| + | return |
| + | } |
| | if err != nil { |
| - | http.Error(w, "400 bad request", http.StatusBadRequest) |
| + | log.Printf("gitweb: receive-pack decode %s: %v", repoPath, err) |
| + | var tooBig *http.MaxBytesError |
| + | if errors.As(err, &tooBig) { |
| + | http.Error(w, fmt.Sprintf("push too large: the limit is %d MiB", tooBig.Limit>>20), |
| + | http.StatusRequestEntityTooLarge) |
| + | return |
| + | } |
| + | http.Error(w, "malformed receive-pack request: "+err.Error(), http.StatusBadRequest) |
| | return |
| | } |
| | rs, err := s.pusher.ReceivePack(r.Context(), repoPath, req) |
| | if err != nil { |
| | log.Printf("gitweb: receive-pack %s: %v", repoPath, err) |
| | if rs == nil { |
| | s.serverErr(w, err) |
| | return |
| | } |
| | // rs carries per-ref failure detail; report it to the client. |
| | } |
| | if err := githttp.WriteReceivePackResult(w, rs); err != nil { |
| | log.Printf("gitweb: receive-pack result %s: %v", repoPath, err) |
| | } |
| | } |
| | } |
| | |
| | // cleanRest cleans a URL remainder into an upspin path suffix beginning with |
| | // "/". path.Clean collapses any ".." so it cannot escape the user's tree. |
| | func cleanRest(rest string) string { |
| | return path.Clean("/" + rest) |
| | } |