Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add partition column to the Iceberg's File metadata table #17528

Closed
wants to merge 3 commits into from

Conversation

osscm
Copy link
Contributor

@osscm osscm commented May 17, 2023

Description

Adding partition column to the file metadata table.
Table has to be partitioned to access the partition column.

Underlying Iceberg DataFile/ContentFile has partition data, Spark supports and provide that information.
This information is required while debugging data issues.

ref: https://github.com/apache/iceberg/blob/master/api/src/main/java/org/apache/iceberg/ContentFile.java#L52

Additional context and related issues

fixes: #17148

Sample query

SELECT content,partition FROM "tab$files"

Release notes

( ) Release notes are required, with the following suggested text:

Partition column is added to the File metadata table.

# Section
* Fix some things. ({issue}`17148`)

@cla-bot cla-bot bot added the cla-signed label May 17, 2023
@osscm osscm requested a review from mosabua May 17, 2023 00:11
@github-actions github-actions bot added the iceberg Iceberg connector label May 17, 2023
@mosabua mosabua requested review from findepi and alexjo2144 May 17, 2023 04:57
@mosabua
Copy link
Member

mosabua commented May 17, 2023

Can @findepi and @alexjo2144 please have a first look?

@osscm osscm force-pushed the file-with-partition branch 2 times, most recently from 2ba97c7 to ab22fde Compare May 23, 2023 23:35
@ebyhr
Copy link
Member

ebyhr commented Jul 12, 2023

Could you rebase on master to resolve conflicts?

@mosabua
Copy link
Member

mosabua commented Jul 17, 2023

Could you rebase @osscm so that @ebyhr and others can review

@osscm
Copy link
Contributor Author

osscm commented Jul 26, 2023 via email

@osscm osscm force-pushed the file-with-partition branch from 9eb59c2 to 1e07f28 Compare July 28, 2023 09:32
@osscm osscm force-pushed the file-with-partition branch 2 times, most recently from 60bd438 to 776c47f Compare July 29, 2023 07:30
@osscm osscm force-pushed the file-with-partition branch from 776c47f to fffdc50 Compare July 30, 2023 17:08
@osscm
Copy link
Contributor Author

osscm commented Jul 31, 2023

Could you rebase on master to resolve conflicts?

@ebyhr rebased, now you can review please.

@osscm
Copy link
Contributor Author

osscm commented Aug 17, 2023

Could you rebase on master to resolve conflicts?

@ebyhr rebased, now you can review please.

can someone please help in reviewing this PR, thanks!

cc @ebyhr or @findinpath or @mosabua

@findepi findepi changed the title Added partition column to the Iceberg's File metadata table Add partition column to the Iceberg's File metadata table Oct 16, 2023
Copy link

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

@github-actions github-actions bot added the stale label Jan 15, 2024
@mosabua
Copy link
Member

mosabua commented Jan 15, 2024

Could you help out here @findepi @ebyhr ?

@github-actions github-actions bot removed the stale label Jan 16, 2024
Copy link

github-actions bot commented Feb 7, 2024

This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua

@github-actions github-actions bot added the stale label Feb 7, 2024
Copy link

Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time.

@github-actions github-actions bot closed this Feb 29, 2024
@osscm osscm reopened this May 1, 2024
@github-actions github-actions bot removed the stale label May 1, 2024
@mosabua mosabua added the stale-ignore Use this label on PRs that should be ignored by the stale bot so they are not flagged or closed. label May 2, 2024
@ebyhr
Copy link
Member

ebyhr commented Nov 12, 2024

Superseded by #24102

@ebyhr ebyhr closed this Nov 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed iceberg Iceberg connector stale-ignore Use this label on PRs that should be ignored by the stale bot so they are not flagged or closed.
Development

Successfully merging this pull request may close these issues.

Add partition column/details to Iceberg's Files table
3 participants