Skip to content

Add a chunked transfer-encoding encoder (TransferEncoding.chunk)#17

Merged
hellerve merged 1 commit into
masterfrom
claude/chunked-encoder
Jul 14, 2026
Merged

Add a chunked transfer-encoding encoder (TransferEncoding.chunk)#17
hellerve merged 1 commit into
masterfrom
claude/chunked-encoder

Conversation

@carpentry-agent

Copy link
Copy Markdown

TransferEncoding could decode a chunked body (dechunk) but had no way to produce one — the codec was one-directional. This adds the inverse.

New functions

  • chunk-with-size body size — splits body into chunks of at most size bytes, each written as a hex size line, the data, and a CRLF, then closed by the terminating zero-size chunk (RFC 7230 §4.1). size is clamped to at least 1 so a bad size can't spin.
  • chunk body — convenience: the whole body as a single chunk plus terminator. An empty body yields just 0\r\n\r\n.

Both are pure Carp (StringBuf + fmt "%x" for the lowercase hex size lines) and mirror the existing dechunk in style.

Verification

dechunk is the natural oracle, so the tests are round-trips dechunk(chunk(x)) == x:

  • exact-output checks (chunk "hello"5\r\nhello\r\n0\r\n\r\n, chunk ""0\r\n\r\n, chunk-with-size "hello" 2 framing);
  • round-trips at sizes 1 and 4;
  • a 300-byte body at size 256 — crosses the two-to-three hex-digit size boundary (ff100);
  • a multi-byte UTF-8 string split every 3 bytes — verifies byte-exactness across mid-codepoint split points;
  • the non-positive-size clamp.

Full suite 151 passed / 1 failed locally; the 1 failure is the pre-existing Form.parse "" empty-body bug (unrelated, untouched — masked on CI by continue-on-error). carp-fmt -c clean, angler clean, gendocs builds. Purely additive.

TransferEncoding could decode chunked bodies (dechunk) but not produce them.
Add chunk-with-size (split into size-bounded chunks) and chunk (single chunk),
both closed by the terminating zero-size chunk per RFC 7230 §4.1. 9 new tests,
property-checked as round-trips against dechunk (incl. the hex-digit boundary
and mid-codepoint UTF-8 splits).

@carpentry-reviewer carpentry-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build & Tests

Checked out claude/chunked-encoder at 6e93a19:

  • carp -x test/http.carp151 pass / 1 fail (the same pre-existing Form.parse "" bug; all 9 new encoder tests pass).
  • angler, carp-fmt --check, gendocs — clean. Matches the green CI.

Findings

The round-trip-against-dechunk tests are a good oracle. I added the one case they didn't cover — a payload that itself contains framing bytes — since that's exactly where a length-prefixed codec tends to break:

  • chunk "a\r\nb", chunk "0\r\n\r\n" (payload equal to the terminator pattern), chunk "5\r\nhello\r\n", and CRLF-containing bodies split at sizes 1 and 2 all round-trip cleanly. ✓ The framing is genuinely length-prefixed, so embedded control bytes are safe.
  • chunk-with-size (http.carp:611) clamps size to ≥ 1, caps end at the body length, emits lowercase hex size lines (verified across the ff100 boundary), closes with the zero-size chunk, and frees the StringBuf. Empty body → just 0\r\n\r\n.
  • Style matches the existing dechunk; the module doc was updated to mention encoding. Purely additive.

Verdict: merge

Correct, robust to framing bytes in the payload, well-tested, all gates green.

@hellerve
hellerve merged commit 3a88745 into master Jul 14, 2026
2 checks passed
@hellerve
hellerve deleted the claude/chunked-encoder branch July 14, 2026 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant