Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow using fragment and query characters in S3 locations #19296

Conversation

homar
Copy link
Member

@homar homar commented Oct 6, 2023

Description

Remove restrictions on the s3 path
https://trinodb.slack.com/archives/CP1MUNEUX/p1696396755060829

Release notes

(x) Release notes are required, with the following suggested text:

# Hive, Delta, Iceberg, Hudi
* Allow using `#` and `?` characters in S3 locations. ({issue}`19296`)

@cla-bot cla-bot bot added the cla-signed label Oct 6, 2023
@homar homar force-pushed the homar/remove_check_against_valid_characters_in_location_class branch from 2201194 to 0ba6b3f Compare October 6, 2023 18:37
@github-actions github-actions bot added the hive Hive connector label Oct 6, 2023
@homar homar force-pushed the homar/remove_check_against_valid_characters_in_location_class branch 3 times, most recently from 3f8edf1 to 3d52fc5 Compare October 9, 2023 09:47
@homar homar marked this pull request as ready for review October 9, 2023 13:07
Copy link
Contributor

@guptashailesh92 guptashailesh92 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for the fix.

@homar homar requested a review from electrum October 9, 2023 14:10
@@ -61,9 +61,6 @@ public static Location of(String location)
checkArgument(!location.isEmpty(), "location is empty");
checkArgument(!location.isBlank(), "location is blank");

checkArgument(location.indexOf('#') < 0, "Fragment is not allowed in a file system location: %s", location);
checkArgument(location.indexOf('?') < 0, "URI query component is not allowed in a file system location: %s", location);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there anything else preventing us from using such paths correctly?
like some urlencode somewhere in delta?

would it make sense to have end-to-end tests with Delta, Hive, Iceberg?

@homar homar force-pushed the homar/remove_check_against_valid_characters_in_location_class branch 2 times, most recently from ff4de3d to 0f73f16 Compare October 9, 2023 19:49
Copy link
Member

@electrum electrum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change the commit message to

Allow using fragment and query characters in S3 locations

No need to put "Trino" in commit titles as everything is about Trino

@electrum electrum changed the title Remove checks that prevent trino from using valid s3 paths Allow using fragment and query characters in S3 locations Oct 9, 2023
@homar homar force-pushed the homar/remove_check_against_valid_characters_in_location_class branch 3 times, most recently from 6713227 to 702bb3a Compare October 10, 2023 14:23
@homar homar force-pushed the homar/remove_check_against_valid_characters_in_location_class branch from 702bb3a to 637d5d6 Compare October 10, 2023 16:53
@electrum electrum merged commit 979e54a into trinodb:master Oct 10, 2023
61 checks passed
@github-actions github-actions bot added this to the 429 milestone Oct 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed hive Hive connector
Development

Successfully merging this pull request may close these issues.

5 participants