Skip to content

Implement Put Block From URL - #2681

Open
gaul wants to merge 1 commit into
Azure:mainfrom
gaul:stage-block-from-url
Open

Implement Put Block From URL#2681
gaul wants to merge 1 commit into
Azure:mainfrom
gaul:stage-block-from-url

Conversation

@gaul

@gaul gaul commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Stage the block by fetching the copy source over loopback so that SAS authentication, x-ms-source-range, and the x-ms-source-if-* conditions are enforced by the existing download path, then persist it through the same extent flow as Put Block. Only sources on the same Azurite instance are supported, matching copyFromURL. The response carries the MD5 of the staged content and source condition failures return 412 SourceConditionNotMet as on the real service.

Validated with the blockblob test suite and end to end with S3Proxy's native multipart part copy, which previously fell back to streamed emulation on Azurite's 501.

Copilot AI review requested due to automatic review settings July 25, 2026 22:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for Block Blob Stage Block From URL (Put Block From URL) in Azurite by downloading the source content via the existing blob download path (so SAS/range/source conditions can be applied) and persisting the staged data through the normal extent + uncommitted-block flow.

Changes:

  • Implements stageBlockFromURL in BlockBlobHandler by fetching the source via HTTP and staging it as an uncommitted block.
  • Introduces a new SourceConditionNotMet (412) storage error for unmet source conditional headers during staging.
  • Expands the block blob API test suite with stageBlockFromURL coverage (range, full copy, unmet condition, missing source).

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
tests/blob/apis/blockblob.test.ts Adds tests validating stageBlockFromURL behavior (ranges, full copy, 412 on unmet source condition, 404 on missing source).
src/blob/handlers/BlockBlobHandler.ts Implements the stageBlockFromURL handler: validates input, downloads source data, persists it, and returns MD5.
src/blob/errors/StorageErrorFactory.ts Adds getSourceConditionNotMet() (412) error factory helper.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +277 to +282
const blobCtx = new BlobStorageContext(context);
const accountName = blobCtx.account!;
const containerName = blobCtx.container!;
const blobName = blobCtx.blob!;
const date = blobCtx.startTime!;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment on lines +322 to +338
const sourceConditions = options.sourceModifiedAccessConditions || {};
if (sourceConditions.sourceIfMatch !== undefined) {
headers["if-match"] = sourceConditions.sourceIfMatch;
}
if (sourceConditions.sourceIfNoneMatch !== undefined) {
headers["if-none-match"] = sourceConditions.sourceIfNoneMatch;
}
if (sourceConditions.sourceIfModifiedSince !== undefined) {
headers["if-modified-since"] = new Date(
sourceConditions.sourceIfModifiedSince
).toUTCString();
}
if (sourceConditions.sourceIfUnmodifiedSince !== undefined) {
headers["if-unmodified-since"] = new Date(
sourceConditions.sourceIfUnmodifiedSince
).toUTCString();
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment thread src/blob/handlers/BlockBlobHandler.ts Outdated
Comment on lines +303 to +314
const currentServer = blobCtx.request!.getHeader("Host") || "";
if (currentServer !== url.host) {
this.logger.error(
`BlockBlobHandler:stageBlockFromURL() Source ${url} is not on the same Azurite instance as target account ${accountName}`,
context.contextId
);
throw StorageErrorFactory.getCannotVerifyCopySource(
context.contextId!,
404,
"The specified resource does not exist"
);
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copilot AI review requested due to automatic review settings July 25, 2026 22:51
@gaul

gaul commented Jul 25, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the review in aa00b52:

  1. Content-Length — the handler now rejects a nonzero Content-Length with 400 InvalidHeaderValue, since Put Block From URL carries no request body, with a test issuing a raw request with a body.
  2. x-ms-source-if-tags — deliberately not forwarded, with a comment in the handler explaining why: unlike the Copy Blob operations, Put Block From URL has no source tags condition. The field appears in the shared SourceModifiedAccessConditions TypeScript interface, but blockBlobStageBlockFromURLOperationSpec has no Parameters.sourceIfTags, matching the service REST contract, so the deserializer never populates it for this operation.
  3. SSRF via Host — the loopback fetch is no longer built from the caller-supplied URL. The handler takes the actual bound address and port from the request's connection socket and fetches scheme://127.0.0.1:<localPort> plus only the caller's path and query, so a forged Host header can no longer point the fetch at an arbitrary URL. The same-instance Host comparison remains for the 404-on-foreign-source behavior that copyFromURL has.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Comment on lines +338 to +341
const scheme = "encrypted" in rawRequest.socket ? "https" : "http";
const pinnedUrl =
`${scheme}://127.0.0.1:${rawRequest.socket.localPort}` +
`${url.pathname}${url.search}`;

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment on lines +346 to +348
if (options.sourceRange !== undefined) {
headers.range = options.sourceRange;
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copilot AI review requested due to automatic review settings July 25, 2026 22:59
@gaul

gaul commented Jul 25, 2026

Copy link
Copy Markdown
Contributor Author

Second round addressed in 381b126: the source fetch now pins to the local address the request arrived on (bracketing IPv6 literals) instead of hard-coded 127.0.0.1, so non-loopback --blobHost binds work; and malformed x-ms-source-range values now fail with 400 InvalidHeaderValue up front instead of silently staging the whole source, with tests for both the malformed-range shapes and the reversed-bounds case.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comment thread src/blob/handlers/BlockBlobHandler.ts Outdated

// Fetch the source range over loopback so that SAS authentication,
// range handling, and source conditions reuse the download path.
const headers: { [key: string]: string } = {};

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copilot AI review requested due to automatic review settings July 25, 2026 23:05
@gaul

gaul commented Jul 25, 2026

Copy link
Copy Markdown
Contributor Author

Addressed in 8847aec: the pinned loopback fetch now sends the original source URL host as the Host header — so product-style sources resolve their account through blobStorageContext.middleware exactly as a direct request would — while the TCP connection stays pinned to the server's bound address. Added an end-to-end test where both the destination request and the copy source use product-style account.localhost URLs.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

Comment thread src/blob/handlers/BlockBlobHandler.ts Outdated
Comment on lines +312 to +313
const currentServer = blobCtx.request!.getHeader("Host") || "";
if (currentServer !== url.host) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@gaul

gaul commented Jul 25, 2026

Copy link
Copy Markdown
Contributor Author

Addressed in e665103: the same-instance comparison now lowercases the client-supplied Host header before comparing against the already-lowercased URL host, with a mixed-case Host regression test.

Copilot AI review requested due to automatic review settings July 25, 2026 23:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated no new comments.

Stage the block by fetching the copy source over loopback so that SAS
authentication, x-ms-source-range, and the x-ms-source-if-* conditions
are enforced by the existing download path, then persist it through
the same extent flow as Put Block.  Only sources on the same Azurite
instance are supported, matching copyFromURL.  The response carries
the MD5 of the staged content and source condition failures return
412 SourceConditionNotMet as on the real service.

Validated with the blockblob test suite and end to end with S3Proxy's
native multipart part copy, which previously fell back to streamed
emulation on Azurite's 501.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gaul
gaul force-pushed the stage-block-from-url branch from e665103 to d2654cb Compare July 25, 2026 23:15
@gaul

gaul commented Jul 25, 2026

Copy link
Copy Markdown
Contributor Author

Squashed after AI review succeeded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants