Skip to content

Commit

Permalink
[IND-399] Improve comments, remove SQL function that wasn't being use…
Browse files Browse the repository at this point in the history
…d, fix some stylistic choices in case statements.
  • Loading branch information
lcwik committed Nov 9, 2023
1 parent 9ed26bd commit 81c7d43
Show file tree
Hide file tree
Showing 32 changed files with 114 additions and 161 deletions.
13 changes: 0 additions & 13 deletions indexer/services/ender/__tests__/scripts/scripts.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -134,19 +134,6 @@ describe('SQL Function Tests', () => {
));
});

it.each([
{ transactionIndex: 5 } as IndexerTendermintEvent,
{
blockEvent: IndexerTendermintEvent_BlockEvent.BLOCK_EVENT_BEGIN_BLOCK,
} as IndexerTendermintEvent,
{
blockEvent: IndexerTendermintEvent_BlockEvent.BLOCK_EVENT_END_BLOCK,
} as IndexerTendermintEvent,
])('dydx_event_to_transaction_index (%s)', async (event: IndexerTendermintEvent) => {
const result = await getSingleRawQueryResultRow(`SELECT dydx_event_to_transaction_index('${JSON.stringify(event)}') AS result;`);
expect(result).toEqual(indexerTendermintEventToTransactionIndex(event));
});

it.each([
Long.fromNumber(1_000_000_000, true),
Long.fromNumber(1_000_000_000, false),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ const scripts: string[] = [
'dydx_market_modify_handler.sql',
'dydx_market_price_update_handler.sql',
'dydx_event_id_from_parts.sql',
'dydx_event_to_transaction_index.sql',
'dydx_from_jsonlib_long.sql',
'dydx_from_protocol_order_side.sql',
'dydx_from_protocol_time_in_force.sql',
Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
/**
Returns the market status (https://github.com/dydxprotocol/v4-chain/blob/ea4f6895a73627aaa9bc5e21eed1ba51313b1ce4/indexer/packages/postgres/src/types/perpetual-market-types.ts#L60)
from the clob pair status (https://github.com/dydxprotocol/v4-chain/blob/ea4f6895a73627aaa9bc5e21eed1ba51313b1ce4/proto/dydxprotocol/indexer/protocol/v1/clob.proto#L157).
The conversion is equivalent to https://github.com/dydxprotocol/v4-chain/blob/ea4f6895a73627aaa9bc5e21eed1ba51313b1ce4/indexer/packages/postgres/src/lib/protocol-translations.ts#L351.
Returns the market status (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/packages/postgres/src/types/perpetual-market-types.ts#L60)
from the clob pair status (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/proto/dydxprotocol/indexer/protocol/v1/clob.proto#L157).
The conversion is equivalent to https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/packages/postgres/src/lib/protocol-translations.ts#L351.
Parameters:
- status: the ClobPairStatus (https://github.com/dydxprotocol/v4-chain/blob/ea4f6895a73627aaa9bc5e21eed1ba51313b1ce4/proto/dydxprotocol/indexer/protocol/v1/clob.proto#L157)
- status: the ClobPairStatus (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/proto/dydxprotocol/indexer/protocol/v1/clob.proto#L157)
*/
CREATE OR REPLACE FUNCTION dydx_clob_pair_status_to_market_status(status jsonb)
RETURNS text AS $$
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ BEGIN
transaction_idx := dydx_tendermint_event_to_transaction_index(event);
event_id := dydx_event_id_from_parts(CAST(block_height AS int), transaction_idx, CAST(event->>'eventIndex' AS int));

INSERT INTO tendermint_events ("id", "blockHeight", "transactionIndex", "eventIndex")
VALUES (event_id, block_height::bigint, transaction_idx, CAST(event->>'eventIndex' AS int))
RETURNING to_jsonb(tendermint_events.*) INTO inserted_event;
INSERT INTO tendermint_events ("id", "blockHeight", "transactionIndex", "eventIndex")
VALUES (event_id, block_height::bigint, transaction_idx, CAST(event->>'eventIndex' AS int))
RETURNING to_jsonb(tendermint_events.*) INTO inserted_event;

RETURN inserted_event;
END;
Expand Down
14 changes: 7 additions & 7 deletions indexer/services/ender/src/scripts/dydx_deleveraging_handler.sql
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,18 @@
Parameters:
- block_height: the height of the block being processing.
- block_time: the time of the block being processed.
- event_data: The 'data' field of the IndexerTendermintEvent (https://github.com/dydxprotocol/v4-proto/blob/8d35c86/dydxprotocol/indexer/indexer_manager/event.proto#L25)
- event_data: The 'data' field of the IndexerTendermintEvent (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/proto/dydxprotocol/indexer/indexer_manager/event.proto#L25)
converted to JSON format. Conversion to JSON is expected to be done by JSON.stringify.
- event_index: The 'event_index' of the IndexerTendermintEvent.
- transaction_index: The transaction_index of the IndexerTendermintEvent after the conversion that takes into
account the block_event (https://github.com/dydxprotocol/indexer/blob/cc70982/services/ender/src/lib/helper.ts#L33)
account the block_event (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/services/ender/src/lib/helper.ts#L41)
- transaction_hash: The transaction hash corresponding to this event from the IndexerTendermintBlock 'tx_hashes'.
Returns: JSON object containing fields:
- liquidated_fill: The created liquidated fill in fill-model format (https://github.com/dydxprotocol/indexer/blob/cc70982/packages/postgres/src/models/fill-model.ts).
- offsetting_fill: The created offsetting fill in fill-model format (https://github.com/dydxprotocol/indexer/blob/cc70982/packages/postgres/src/models/fill-model.ts).
- perpetual_market: The perpetual market for the deleveraging in perpetual-market-model format (https://github.com/dydxprotocol/indexer/blob/cc70982/packages/postgres/src/models/perpetual-market-model.ts).
- liquidated_perpetual_position: The updated liquidated perpetual position in perpetual-position-model format (https://github.com/dydxprotocol/indexer/blob/cc70982/packages/postgres/src/models/perpetual-position-model.ts).
- offsetting_perpetual_position: The updated offsetting perpetual position in perpetual-position-model format (https://github.com/dydxprotocol/indexer/blob/cc70982/packages/postgres/src/models/perpetual-position-model.ts).
- liquidated_fill: The created liquidated fill in fill-model format (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/packages/postgres/src/models/fill-model.ts).
- offsetting_fill: The created offsetting fill in fill-model format (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/packages/postgres/src/models/fill-model.ts).
- perpetual_market: The perpetual market for the deleveraging in perpetual-market-model format (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/packages/postgres/src/models/perpetual-market-model.ts).
- liquidated_perpetual_position: The updated liquidated perpetual position in perpetual-position-model format (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/packages/postgres/src/models/perpetual-position-model.ts).
- offsetting_perpetual_position: The updated offsetting perpetual position in perpetual-position-model format (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/packages/postgres/src/models/perpetual-position-model.ts).
*/
CREATE OR REPLACE FUNCTION dydx_deleveraging_handler(
block_height int, block_time timestamp, event_data jsonb, event_index int, transaction_index int,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
Parameters:
- block_height: the height of the block being processing.
- transaction_index: The transaction_index of the IndexerTendermintEvent after the conversion that takes into
account the block_event (https://github.com/dydxprotocol/indexer/blob/cc70982/services/ender/src/lib/helper.ts#L33)
account the block_event (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/services/ender/src/lib/helper.ts#L41)
- event_index: The 'event_index' of the IndexerTendermintEvent.
*/
CREATE OR REPLACE FUNCTION dydx_event_id_from_parts(block_height int, transaction_index int, event_index int) RETURNS bytea AS $$
Expand All @@ -14,7 +14,7 @@ BEGIN
|| is the byte string concatenation operator.
transactionIndex is -2 for BEGIN_BLOCK events, and -1 for END_BLOCK events. Increment by 2 to ensure result is >= 0.
See https://github.com/dydxprotocol/indexer/blob/6aafb97/packages/postgres/src/stores/tendermint-event-table.ts#L33
See https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/packages/postgres/src/stores/tendermint-event-table.ts#L34
*/
RETURN int4send(block_height) || int4send(transaction_index + 2) || int4send(event_index);
END;
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,14 +1,12 @@
/**
Converts the 'Side' enum from the IndexerOrder protobuf (https://github.com/dydxprotocol/v4-proto/blob/437f6d8/dydxprotocol/indexer/protocol/v1/clob.proto#L56)
Converts the 'Side' enum from the IndexerOrder protobuf (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/proto/dydxprotocol/indexer/protocol/v1/clob.proto#L56)
to the 'OrderSide' enum in postgres.
*/
CREATE OR REPLACE FUNCTION dydx_from_protocol_order_side(order_side jsonb) RETURNS text AS $$
BEGIN
CASE order_side
WHEN '1'::jsonb THEN
RETURN 'BUY';
ELSE
RETURN 'SELL';
WHEN '1'::jsonb THEN RETURN 'BUY'; /** SIDE_BUY */
ELSE RETURN 'SELL';
END CASE;
END;
$$ LANGUAGE plpgsql IMMUTABLE PARALLEL SAFE;
Original file line number Diff line number Diff line change
@@ -1,20 +1,18 @@
/**
Converts the TimeInForce field from an IndexerOrder proto (https://github.com/dydxprotocol/v4-proto/blob/437f6d8/dydxprotocol/indexer/protocol/v1/clob.proto#L95)
Converts the TimeInForce field from an IndexerOrder proto (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/proto/dydxprotocol/indexer/protocol/v1/clob.proto#L94)
to a TimeInForce enum in postgres.
Raise an exception if the input TimeInForce enum is not in the known enum values for TimeInForce.
*/
CREATE OR REPLACE FUNCTION dydx_from_protocol_time_in_force(tif jsonb) RETURNS text AS $$
BEGIN
CASE tif
-- Default behavior with UNRECOGNIZED = GTT (Good-Til-Time)
WHEN '-1'::jsonb THEN RETURN 'GTT';
-- Default behavior with TIME_IN_FORCE_UNSPECIFIED = GTT (Good-Til-Time)
WHEN '0'::jsonb THEN RETURN 'GTT';
WHEN '1'::jsonb THEN RETURN 'IOC';
WHEN '2'::jsonb THEN RETURN 'POST_ONLY';
WHEN '3'::jsonb THEN RETURN 'FOK';
WHEN '-1'::jsonb THEN RETURN 'GTT'; /** Default behavior with UNRECOGNIZED = GTT (Good-Til-Time) */
WHEN '0'::jsonb THEN RETURN 'GTT'; /** Default behavior with TIME_IN_FORCE_UNSPECIFIED = GTT (Good-Til-Time) */
WHEN '1'::jsonb THEN RETURN 'IOC'; /** TIME_IN_FORCE_IOC */
WHEN '2'::jsonb THEN RETURN 'POST_ONLY'; /** TIME_IN_FORCE_POST_ONLY */
WHEN '3'::jsonb THEN RETURN 'FOK'; /** TIME_IN_FORCE_FILL_OR_KILL */
ELSE RAISE EXCEPTION 'Unexpected TimeInForce from protocol %', tif;
END CASE;
END CASE;
END;
$$ LANGUAGE plpgsql IMMUTABLE PARALLEL SAFE;
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
/**
Converts a JSON.stringify byte array representing a SerializableInt
(https://github.com/dydxprotocol/v4/blob/f77b074/dtypes/serializable_int.go#L57) to a numeric. Note that the
underlying SerializableInt encoding format uses the big.Int GobEncoding
(https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/protocol/dtypes/serializable_int.go#L84) to a numeric.
Note that the underlying SerializableInt encoding format uses the big.Int GobEncoding
(https://github.com/golang/go/blob/886fba5/src/math/big/intmarsh.go#L18)
which is represented as [versionAndSignByte bigEndianByte0 bigEndianByte1 ... bigEndianByte2]
byte array.
Expand Down
6 changes: 3 additions & 3 deletions indexer/services/ender/src/scripts/dydx_funding_handler.sql
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@
Parameters:
- block_height: the height of the block being processing.
- block_time: the time of the block being processed.
- event_data: The 'data' field of the IndexerTendermintEvent (https://github.com/dydxprotocol/v4-proto/blob/8d35c86/dydxprotocol/indexer/indexer_manager/event.proto#L25)
- event_data: The 'data' field of the IndexerTendermintEvent (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/proto/dydxprotocol/indexer/indexer_manager/event.proto#L25)
converted to JSON format. Conversion to JSON is expected to be done by JSON.stringify.
- event_index: The 'event_index' of the IndexerTendermintEvent.
- transaction_index: The transaction_index of the IndexerTendermintEvent after the conversion that takes into
account the block_event (https://github.com/dydxprotocol/indexer/blob/cc70982/services/ender/src/lib/helper.ts#L33)
account the block_event (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/services/ender/src/lib/helper.ts#L41)
Returns: JSON object containing fields:
- perpetual_markets: A mapping from perpetual market id to the associated perpetual market in perpetual-market-model format (https://github.com/dydxprotocol/indexer/blob/cc70982/packages/postgres/src/models/perpetual-market-model.ts).
- perpetual_markets: A mapping from perpetual market id to the associated perpetual market in perpetual-market-model format (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/packages/postgres/src/models/perpetual-market-model.ts).
- errors: An array containing an error string (or NULL if no error occurred) for each FundingEventUpdate.
*/
CREATE OR REPLACE FUNCTION dydx_funding_handler(
Expand Down
39 changes: 23 additions & 16 deletions indexer/services/ender/src/scripts/dydx_get_order_status.sql
Original file line number Diff line number Diff line change
@@ -1,20 +1,27 @@
/**
* The obvious case is if totalFilled >= size, then the order status should always be `FILLED`.
* The difficult case is if totalFilled < size after a fill, then we need to keep the following
* cases in mind:
* 1. Stateful Orders - All cancelations are on-chain events, so the will be `OPEN`. The
* CanceledOrdersCache does not store any stateful orders and we never send
* BEST_EFFORT_CANCELED notifications for stateful orders.
* 2. Short-term FOK - FOK orders can never be `OPEN`, since they don't rest on the orderbook, so
* totalFilled cannot be < size. By the end of the block, the order will be filled, so we mark
* it as `FILLED`.
* 3. Short-term IOC - Protocol guarantees that an IOC order will only ever be filled in a single
* block, so status should be `CANCELED`.
* 4. Short-term Limit & Post-only - If the order is in the CanceledOrdersCache, then it should be
* set to the corresponding CanceledOrderStatus, otherwise `OPEN`.
* @param isCanceled - if the order is in the CanceledOrderCache, always false for liquidiation
* orders
*/
Computes the order status given a set of order parameters.
The obvious case is if totalFilled >= size, then the order status should always be `FILLED`.
The difficult case is if totalFilled < size after a fill, then we need to keep the following
cases in mind:
1. Stateful Orders - All cancelations are on-chain events, so the will be `OPEN`. The
CanceledOrdersCache does not store any stateful orders and we never send
BEST_EFFORT_CANCELED notifications for stateful orders.
2. Short-term FOK - FOK orders can never be `OPEN`, since they don't rest on the orderbook, so
totalFilled cannot be < size. By the end of the block, the order will be filled, so we mark
it as `FILLED`.
3. Short-term IOC - Protocol guarantees that an IOC order will only ever be filled in a single
block, so status should be `CANCELED`.
4. Short-term Limit & Post-only - If the order is in the CanceledOrdersCache, then it should be
set to the corresponding CanceledOrderStatus, otherwise `OPEN`.
Parameters:
- total_filled - The amount the order is filled.
- size - The order size.
- order_canceled_status - The status of the order.
- order_flags - The flags of the order.
Returns the order status.
*/
CREATE OR REPLACE FUNCTION dydx_get_order_status(total_filled numeric, size numeric, order_canceled_status text, order_flags bigint, time_in_force text)
RETURNS text AS $$
BEGIN
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
/**
Returns the weighted average between two prices.
Note that since division is used the scale of the resulting number if limited to 20 which matches the division
Note that since division is used the scale of the resulting number is limited to 20 which matches the division
precision (DP) of the https://mikemcl.github.io/big.js/ library.
Parameters:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,21 @@
- field: the field storing the order to process.
- block_height: the height of the block being processing.
- block_time: the time of the block being processed.
- event_data: The 'data' field of the IndexerTendermintEvent (https://github.com/dydxprotocol/v4-proto/blob/8d35c86/dydxprotocol/indexer/indexer_manager/event.proto#L25)
- event_data: The 'data' field of the IndexerTendermintEvent (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/proto/dydxprotocol/indexer/indexer_manager/event.proto#L25)
converted to JSON format. Conversion to JSON is expected to be done by JSON.stringify.
- event_index: The 'event_index' of the IndexerTendermintEvent.
- transaction_index: The transaction_index of the IndexerTendermintEvent after the conversion that takes into
account the block_event (https://github.com/dydxprotocol/indexer/blob/cc70982/services/ender/src/lib/helper.ts#L33)
account the block_event (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/services/ender/src/lib/helper.ts#L41)
- transaction_hash: The transaction hash corresponding to this event from the IndexerTendermintBlock 'tx_hashes'.
- fill_liquidity: The liquidity for the fill record.
- fill_type: The type for the fill record.
- usdc_asset_id: The USDC asset id.
Returns: JSON object containing fields:
- order: The updated order in order-model format (https://github.com/dydxprotocol/indexer/blob/cc70982/packages/postgres/src/models/order-model.ts).
- order: The updated order in order-model format (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/packages/postgres/src/models/order-model.ts).
Only returned if field == 'makerOrder'.
- fill: The updated fill in fill-model format (https://github.com/dydxprotocol/indexer/blob/cc70982/packages/postgres/src/models/fill-model.ts).
- perpetual_market: The perpetual market for the order in perpetual-market-model format (https://github.com/dydxprotocol/indexer/blob/cc70982/packages/postgres/src/models/perpetual-market-model.ts).
- perpetual_position: The updated perpetual position in perpetual-position-model format (https://github.com/dydxprotocol/indexer/blob/cc70982/packages/postgres/src/models/perpetual-position-model.ts).
- fill: The updated fill in fill-model format (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/packages/postgres/src/models/fill-model.ts).
- perpetual_market: The perpetual market for the order in perpetual-market-model format (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/packages/postgres/src/models/perpetual-market-model.ts).
- perpetual_position: The updated perpetual position in perpetual-position-model format (https://github.com/dydxprotocol/v4-chain/blob/9ed26bd/indexer/packages/postgres/src/models/perpetual-position-model.t).
*/
CREATE OR REPLACE FUNCTION dydx_liquidation_fill_handler_per_order(
field text, block_height int, block_time timestamp, event_data jsonb, event_index int, transaction_index int,
Expand Down
Loading

0 comments on commit 81c7d43

Please sign in to comment.