Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Derive aws sdk retry count from request count #24606

Merged
merged 1 commit into from
Jan 2, 2025

Conversation

ljw9111
Copy link
Contributor

@ljw9111 ljw9111 commented Dec 31, 2024

Description

For retry count metric for Trino S3 file system, we are more interested in RetryCount than HttpClientRetryCount. But Trino was only tracking HttpClientRetryCount and not RetryCount.

Definition from aws java sdk v1 github is as follows
RetryCount - Number of retries of AWS SDK sending a request to AWS.
HttpClientRetryCount - Number of retries of the underlying http client library in sending a request to AWS.

And RetryCount count should be derived from RequestCount (i.e. RetryCount = RequestCount - 1) as in here

Additional context and related issues

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(O) Release notes are required, with the following suggested text:

## Hive
* Fix reporting of AWS SDK client retry count metrics to report both client level logical retries and lower level HTTP client retries separately. Before the change, only tracking HttpClientRetryCount was collected and exposed via JMX as "awsretrycount" on the relevant JMX tables. Now, "awsretrycount" represents the logical retries and the HTTP client retries are reported as "awshttpclientretrycount" on tables such as, e.g.: "io.trino.hdfs.s3:name=hive,type=trinos3filesystem" or "trino.plugin.hive.metastore.glue:name=hive,type=gluehivemetastore". ({issue}`24606`)

@cla-bot cla-bot bot added the cla-signed label Dec 31, 2024
@github-actions github-actions bot added the hive Hive connector label Dec 31, 2024
@ljw9111 ljw9111 requested a review from pettyjamesm December 31, 2024 20:12
@pettyjamesm
Copy link
Member

The code change LGTM, can we catalog the JMX tables that are affected and include the column names in the release notes section on the PR?

@pettyjamesm pettyjamesm merged commit c06f1e1 into trinodb:master Jan 2, 2025
59 checks passed
@github-actions github-actions bot added this to the 469 milestone Jan 2, 2025
@ljw9111 ljw9111 deleted the aws-sdk-metric-fix branch January 2, 2025 17:33
@ljw9111 ljw9111 self-assigned this Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed hive Hive connector
Development

Successfully merging this pull request may close these issues.

2 participants