Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for pagination at controller level. #1420

Conversation

DavideSegullo
Copy link
Contributor

@DavideSegullo DavideSegullo commented Apr 26, 2024

Changelist

This pr is a continuation of PR #1244.
In particular, it modifies the controllers of fills, trades, and transfers to support paging in REST services.

Test Plan

Tests are delivered with the features.
We have not tested the environment with a full live (development) environment, but have performed unit tests.
We encourage you to perform full (integration) testing before merging.

Author/Reviewer Checklist

  • If this PR has changes that result in a different app state given the same prior state and transaction list, manually add the state-breaking label.
  • If the PR has breaking postgres changes to the indexer add the indexer-postgres-breaking label.
  • If this PR isn't state-breaking but has changes that modify behavior in PrepareProposal or ProcessProposal, manually add the label proposal-breaking.
  • If this PR is one of many that implement a specific feature, manually label them all feature:[feature-name].
  • If you wish to for mergify-bot to automatically create a PR to backport your change to a release branch, manually add the label backport/[branch-name].
  • Manually add any of the following labels: refactor, chore, bug.

Summary by CodeRabbit

  • New Features

    • Introduced pagination parameters (page, pageSize, totalResults, offset) to API endpoints for fills, trades, and transfers.
    • Enhanced API responses to include pagination details.
  • Bug Fixes

    • Ensured trades and fills are fetched in descending order by createdAtHeight and correctly paginated.
  • Documentation

    • Updated API documentation to reflect new pagination parameters and response fields.
  • Tests

    • Added comprehensive test cases to verify pagination functionality for fills, trades, and transfers.

Copy link
Contributor

coderabbitai bot commented Apr 26, 2024

Walkthrough

The changes across various files in the indexer services primarily introduce and enhance pagination features in API endpoints. New test cases ensure functionality, while controllers have been updated to handle pagination parameters. Additionally, the API documentation and response types have been adjusted to reflect these updates, ensuring a consistent and scalable user experience when interacting with the system's data retrieval features.

Changes

Files Summary
.../fills-controller.test.ts, .../trades-controller.test.ts, .../transfers-controller.test.ts Added test cases for pagination in fetching fills, trades, and transfers.
.../api-documentation.md, .../src/types.ts Updated API documentation and response types to include new pagination fields.
.../fills-controller.ts, .../trades-controller.ts, .../transfers-controller.ts Enhanced controller methods to support pagination parameters and updated response structures.
.../src/lib/validation/schemas.ts Introduced a new pagination schema for validating API request parameters.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Out of diff range and nitpick comments (20)
indexer/services/comlink/public/api-documentation.md (20)

Line range hint 14-14: Consider adding a more detailed description for the GetAddress endpoint to enhance clarity.


Line range hint 188-188: Add a space after the colon for better readability in the parameters table of the GetSubaccount endpoint.


Line range hint 313-313: Add a space after the colon in the parameters table of the GetParentSubaccount endpoint for consistency.


Line range hint 448-448: Insert a space after the colon in the parameters table of the GetAssetPositions endpoint to improve the formatting.


Line range hint 525-525: Insert a space after the colon in the parameters table of the GetAssetPositionsForParentSubaccount endpoint for uniformity across the document.


Line range hint 603-603: Add a space after the colon in the parameters table of the GetCandles endpoint to enhance the readability of the table.


Line range hint 611-611: Ensure there is a space between the number and the unit in the enumerated values for the resolution parameter to adhere to standard formatting practices.


Line range hint 769-769: Add a space after the colon in the parameters table of the GetFills endpoint to maintain consistency in the document's formatting.


Line range hint 871-871: Insert a space after the colon in the parameters table of the GetFillsForParentSubaccount endpoint to keep the formatting consistent throughout the document.


Line range hint 1032-1032: Add a space after the colon in the parameters table of the GetTradingRewards endpoint for better readability.


Line range hint 1107-1107: Insert a space after the colon in the parameters table of the GetHistoricalFunding endpoint to improve the clarity of the table.


Line range hint 1186-1186: Add a space after the colon in the parameters table of the GetHistoricalPnl endpoint to enhance the document's uniformity.


Line range hint 1271-1271: Insert a space after the colon in the parameters table of the GetAggregations endpoint for consistent formatting across the document.


Line range hint 1629-1629: Add a space after the colon in the parameters table of the ListOrdersForParentSubaccount endpoint to maintain a uniform appearance in the documentation.


Line range hint 1906-1906: Ensure there is a space after the colon in the parameters table of the ListPerpetualMarkets endpoint to keep the formatting consistent.


Line range hint 2023-2023: Insert a space after the colon in the parameters table of the ListPositions endpoint for better readability and consistency.


Line range hint 2123-2123: Add a space after the colon in the parameters table of the ListPositionsForParentSubaccount endpoint to enhance the clarity of the table.


2362-2362: Ensure there is a space after the colon in the parameters table of the GetTrades endpoint to maintain uniformity in the document's formatting.


Line range hint 2447-2447: Insert a space after the colon in the parameters table of the GetTransfers endpoint for consistent formatting across the document.


Line range hint 2541-2541: Add a space after the colon in the parameters table of the GetTransfersForParentSubaccount endpoint to enhance the document's readability.

@@ -77,6 +88,7 @@ class FillsController extends Controller {
? createdBeforeOrAtHeight.toString()
: undefined,
createdBeforeOrAt,
page,
},
[QueryableField.LIMIT],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to specify a default ordering. Can you add
{ orderBy: [[FillColumns.eventId, Ordering.ASC]], },?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -141,6 +162,7 @@ class FillsController extends Controller {
? createdBeforeOrAtHeight.toString()
: undefined,
createdBeforeOrAt,
page,
},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. Can you add
{ orderBy: [[FillColumns.eventId, Ordering.ASC]], },?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -62,6 +68,7 @@ class TradesController extends Controller {
? createdBeforeOrAtHeight.toString()
: undefined,
createdBeforeOrAt,
page,
},
[QueryableField.LIQUIDITY, QueryableField.CLOB_PAIR_ID, QueryableField.LIMIT],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to specify a default ordering. Can you add
{ orderBy: [[FillColumns.eventId, Ordering.ASC]], },?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -77,6 +81,7 @@ class TransfersController extends Controller {
? createdBeforeOrAtHeight.toString()
: undefined,
createdBeforeOrAt,
page,
},
[QueryableField.LIMIT],
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you change the ordering to be by eventId desc here? To make transfer ordering deterministic if they occurred at the same height.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -165,6 +179,7 @@ class TransfersController extends Controller {
? createdBeforeOrAtHeight.toString()
: undefined,
createdBeforeOrAt,
page,
},
[QueryableField.LIMIT],
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you change the ordering to be by eventId desc here? To make transfer ordering deterministic if they occurred at the same height.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits Files that changed from the base of the PR and between 6ea2dbf and 2fd8d09.
Files selected for processing (3)
  • indexer/services/comlink/src/controllers/api/v4/fills-controller.ts (16 hunks)
  • indexer/services/comlink/src/controllers/api/v4/trades-controller.ts (8 hunks)
  • indexer/services/comlink/src/controllers/api/v4/transfers-controller.ts (14 hunks)
Additional Context Used
Path-based Instructions (3)
indexer/services/comlink/src/controllers/api/v4/trades-controller.ts (1)

Pattern **/**: Do not include a walkthrough. For all comments on line changes that are longer than 15 lines, add a prefix to the comment of what lines you are commenting on (for example: "The changes from lines 100 to 200...". Add your comment to be on the last 15 lines of the chunk rather than adding your comment to the entire chunk.

indexer/services/comlink/src/controllers/api/v4/transfers-controller.ts (1)

Pattern **/**: Do not include a walkthrough. For all comments on line changes that are longer than 15 lines, add a prefix to the comment of what lines you are commenting on (for example: "The changes from lines 100 to 200...". Add your comment to be on the last 15 lines of the chunk rather than adding your comment to the entire chunk.

indexer/services/comlink/src/controllers/api/v4/fills-controller.ts (1)

Pattern **/**: Do not include a walkthrough. For all comments on line changes that are longer than 15 lines, add a prefix to the comment of what lines you are commenting on (for example: "The changes from lines 100 to 200...". Add your comment to be on the last 15 lines of the chunk rather than adding your comment to the entire chunk.

Additional comments not posted (12)
indexer/services/comlink/src/controllers/api/v4/trades-controller.ts (4)

28-28: Import statement for CheckPaginationSchema looks good.


50-50: Addition of page query parameter in getTrades method is appropriate.


Line range hint 59-76: The changes to the findAll method call to include pagination parameters and ordering are correct and necessary for pagination support.


83-85: Inclusion of pagination fields (pageSize, totalResults, offset) in the response object is appropriate.

indexer/services/comlink/src/controllers/api/v4/transfers-controller.ts (4)

33-33: Import statement for CheckPaginationSchema looks good.


65-65: Addition of page query parameter in getTransfers method is appropriate.


Line range hint 71-94: The changes to the findAllToOrFromSubaccountId method call to include pagination parameters and ordering are correct and necessary for pagination support.


142-144: Inclusion of pagination fields (pageSize, totalResults, offset) in the response object is appropriate.

indexer/services/comlink/src/controllers/api/v4/fills-controller.ts (4)

37-37: Import statement for CheckPaginationSchema looks good.


65-65: Addition of page query parameter in getFills method is appropriate.


Line range hint 79-96: The changes to the findAll method call to include pagination parameters and ordering are correct and necessary for pagination support.


116-118: Inclusion of pagination fields (pageSize, totalResults, offset) in the response object is appropriate.

@DavideSegullo DavideSegullo requested a review from dydxwill May 16, 2024 09:06
@dydxwill dydxwill merged commit 1aa9e26 into dydxprotocol:main May 16, 2024
11 checks passed
@dydxwill
Copy link
Contributor

https://github.com/Mergifyio backport release/indexer/v5.x

Copy link
Contributor

mergify bot commented May 17, 2024

backport release/indexer/v5.x

✅ Backports have been created

mergify bot pushed a commit that referenced this pull request May 17, 2024
* feat: ✨ add pagination for getFills

* feat: ✨ add pagination support for getTrades

* feat: ✨ add pagination support for getTransfers

* test(comlink): ✅ add trades controller pagination tests

* test(comlink): ✅ add transfers controller pagination tests

* test(comlink): ✅ add fills controller pagination tests

* feat: ✨ add pagination check schema

* test(comlink): ✅ update pagination tests

* feat: ✨ add default orderBy for pagination

(cherry picked from commit 1aa9e26)
dydxwill pushed a commit that referenced this pull request May 17, 2024
* feat: ✨ add pagination for getFills

* feat: ✨ add pagination support for getTrades

* feat: ✨ add pagination support for getTransfers

* test(comlink): ✅ add trades controller pagination tests

* test(comlink): ✅ add transfers controller pagination tests

* test(comlink): ✅ add fills controller pagination tests

* feat: ✨ add pagination check schema

* test(comlink): ✅ update pagination tests

* feat: ✨ add default orderBy for pagination

(cherry picked from commit 1aa9e26)

Co-authored-by: Davide Segullo <davidesegullo@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

3 participants