Skip to content

Enable S3 client to upload binary files#76

Merged
fatimarahman merged 3 commits into
mainfrom
s3-parquet-upload
Jul 21, 2026
Merged

Enable S3 client to upload binary files#76
fatimarahman merged 3 commits into
mainfrom
s3-parquet-upload

Conversation

@fatimarahman

@fatimarahman fatimarahman commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

This is needed for DE-661. Binary data (including parquet files) cannot currently be sent through upload_file because of the encoding step (bytes can't be re-encoded safely)

Changes made:

  • Prior to S3 upload, encodes content based on data type (if str -> encodes, keeps as-is otherwise)
  • Changed all single quotes to double

@fatimarahman
fatimarahman merged commit 3e689c5 into main Jul 21, 2026
5 checks passed
@fatimarahman
fatimarahman deleted the s3-parquet-upload branch July 21, 2026 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants