-
Notifications
You must be signed in to change notification settings - Fork 124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[CT-708] Indexer track e2e latency #1237
[CT-708] Indexer track e2e latency #1237
Conversation
WalkthroughThe recent updates across various services and packages aim to improve metrics, timing calculations, and message handling. Changes include introducing a new constant for stats sampling, refining timing metrics, enhancing message timestamp handling, and incorporating message headers in processing calls. Additionally, there is a significant refactoring related to time functionalities, introducing a utility to convert protobuf timestamps to JavaScript Date objects, with corresponding test adjustments. Changes
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review Status
Actionable comments generated: 0
Configuration used: CodeRabbit UI
Files selected for processing (6)
- indexer/services/socks/src/lib/message-forwarder.ts (3 hunks)
- indexer/services/vulcan/src/handlers/handler.ts (2 hunks)
- indexer/services/vulcan/src/handlers/order-place-handler.ts (7 hunks)
- indexer/services/vulcan/src/handlers/order-remove-handler.ts (11 hunks)
- indexer/services/vulcan/src/handlers/order-update-handler.ts (3 hunks)
- indexer/services/vulcan/src/lib/on-message.ts (2 hunks)
Additional comments: 24
indexer/services/vulcan/src/handlers/handler.ts (3)
- 7-7: The import of
IHeaders
fromkafkajs
is correctly added to support the new functionality of passing headers through the handler methods. This aligns with the PR's objective to enhance end-to-end latency tracking.- 19-19: The abstract method
handle
now correctly accepts an additional parameterheaders
of typeIHeaders
. This change is necessary for threading through headers in various components of the indexer services, as mentioned in the PR objectives.- 22-23: The
handleUpdate
method has been updated to accept and pass theheaders
parameter to thehandle
method. This ensures that headers are correctly propagated through the system, allowing for enhanced tracking of message time since received.indexer/services/vulcan/src/lib/on-message.ts (3)
- 65-75: The addition of tracking for
message_time_since_received
using theoriginalMessageTimestamp
from the message headers is a significant improvement. It aligns with the PR's goal to enhance the precision of latency tracking within the indexer's infrastructure. However, ensure that themessage_received_timestamp
header is consistently set across all messages to avoid potentialNaN
results fromDate.now() - Number(undefined)
.- 101-101: The conditional check for
originalMessageTimestamp !== undefined
before calculating and recording the timing statistics is a good practice. It ensures that only messages with a validoriginalMessageTimestamp
header contribute to the latency metrics.- 101-101: The use of
message.headers ?? {}
as a fallback when headers arenull
is a prudent measure to ensure that thehandleUpdate
method call does not fail due to missing headers. This change supports the PR's objective by allowing for the inclusion ofmessage_received_timestamp
from the headers in statistics, even when headers are not present.indexer/services/socks/src/lib/message-forwarder.ts (3)
- 6-6: The introduction of the
STATS_NO_SAMPLING
constant is a relevant addition for adjusting the calculation of time metrics. This constant likely serves to ensure that certain statistics are always recorded without sampling, which is crucial for accurate latency tracking.- 99-109: Adding a check for
originalMessageTimestamp
before updating stats is a critical enhancement. This ensures that latency metrics are only updated when theoriginalMessageTimestamp
is available, contributing to more accurate and meaningful statistics.- 142-147: Refining timing calculations with distinct start times for different operations, as seen in the
forwardMessage
method, is a good practice. It allows for more granular tracking of operation durations, aligning with the PR's goal to enhance latency tracking.indexer/services/vulcan/src/handlers/order-update-handler.ts (3)
- 27-27: The modification to include
IHeaders
in the import statement is correctly implemented to support the new functionality of passing headers through thehandle
method. This change is essential for threading through headers in the order update handling process.- 54-54: The
handle
method now correctly accepts an additionalheaders
parameter of typeIHeaders
. This is a necessary change to ensure that headers are threaded through the system, allowing for enhanced tracking of message time since received.- 174-176: Including the
message_received_timestamp
from the headers when sending a message is a crucial enhancement. It ensures that the latency tracking mechanism can accurately account for the time since the message was received, aligning with the PR's objectives.indexer/services/vulcan/src/handlers/order-place-handler.ts (6)
- 33-33: The inclusion of
IHeaders
in the import statement is correctly implemented to support the new functionality of passing headers through thehandle
andsendCachedOrderUpdate
methods. This change is essential for threading through headers in the order placement handling process.- 54-54: The
handle
method now correctly accepts an additionalheaders
parameter of typeIHeaders
. This is a necessary change to ensure that headers are threaded through the system, allowing for enhanced tracking of message time since received.- 139-139: The
sendCachedOrderUpdate
method now correctly takesheaders: IHeaders
as a parameter. This ensures that when cached order updates are sent, they include the necessary headers for latency tracking.- 148-150: Including the
message_received_timestamp
from the headers when creating a subaccount message is a crucial enhancement. It ensures that the latency tracking mechanism can accurately account for the time since the message was received, aligning with the PR's objectives.- 164-166: Including the
message_received_timestamp
from the headers when creating an orderbook message is a crucial enhancement. It ensures that the latency tracking mechanism can accurately account for the time since the message was received, aligning with the PR's objectives.- 347-349: Including the
message_received_timestamp
from the headers when sending a cached order update is a crucial enhancement. It ensures that the latency tracking mechanism can accurately account for the time since the message was received, aligning with the PR's objectives.indexer/services/vulcan/src/handlers/order-remove-handler.ts (6)
- 71-71: The method
handle
now accepts an additional parameterheaders
of typeIHeaders
. This change aligns with the PR's objective to enhance e2e latency tracking by threading through headers in various components. Ensure that all calls to this method have been updated accordingly to pass theheaders
parameter.- 134-134: The method
handleStatefulOrderCancelation
has been updated to include theheaders
parameter. This is consistent with the PR's goal of improving latency tracking. Verify that theheaders
are being used effectively within the method to track latency-related metrics.- 138-138: The method
handleOrderRemoval
now requires theheaders
parameter. This modification supports the PR's aim of enhancing latency measurement. Confirm that the inclusion ofheaders
is leveraged for latency tracking purposes throughout the method's implementation.- 248-248: The call to
updateOrderbook
withinhandleStatefulOrderCancelation
now passes theheaders
parameter. This change is part of the effort to thread headers through the system for improved latency tracking. Ensure that theupdateOrderbook
method utilizes theheaders
effectively for latency-related calculations or logging.- 333-333: Similarly, in
handleOrderRemoval
, theupdateOrderbook
method is called with theheaders
parameter. This adjustment is in line with the PR's objectives. Check that theheaders
are being used appropriately withinupdateOrderbook
to enhance latency tracking.- 350-350: The
updateOrderbook
method now includes theheaders
parameter. This modification supports the overarching goal of the PR to improve latency tracking by threading headers through various components. Verify that theheaders
are utilized effectively within the method for latency measurement or logging.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we also add unit tests?
@@ -86,7 +98,7 @@ export async function onMessage(message: KafkaMessage): Promise<void> { | |||
const handler: Handler = new (getHandler(update))!( | |||
getTransactionHashFromHeaders(message.headers), | |||
); | |||
await handler.handleUpdate(update); | |||
await handler.handleUpdate(update, message.headers ?? {}); | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we also emit a stat here comparing to message_received_timestamp ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we already do that here in the same code block right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
that's before handling the message though
@@ -125,12 +139,12 @@ export class MessageForwarder { | |||
return; | |||
} | |||
|
|||
const start: number = Date.now(); | |||
const startForwardMessage: number = Date.now(); | |||
this.forwardMessage(messageToForward); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function seems to send out websocket messages in batches for certain websocket topics. Can we pipe the initial timestamp all the way to the sendMessage
function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you mean we should add a message_received_timestamp field to
export interface OutgoingMessage {
type: OutgoingMessageType;
connection_id: string;
message_id: number;
}
```?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nvm, the batches happen at periodic intervals so not necessary to pipe it through.
Does it make more sense to have the stat be end - Number(message.headers?.message_received_timestamp)
(after calling forwardMessage
) rather than start - Number(message.headers?.message_received_timestamp)
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review Status
Actionable comments generated: 0
Configuration used: CodeRabbit UI
Files selected for processing (2)
- indexer/services/socks/src/lib/message-forwarder.ts (3 hunks)
- indexer/services/vulcan/src/lib/on-message.ts (2 hunks)
Files skipped from review as they are similar to previous changes (2)
- indexer/services/socks/src/lib/message-forwarder.ts
- indexer/services/vulcan/src/lib/on-message.ts
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review Status
Actionable comments generated: 0
Configuration used: CodeRabbit UI
Files selected for processing (2)
- indexer/services/vulcan/src/handlers/order-place-handler.ts (7 hunks)
- indexer/services/vulcan/src/lib/on-message.ts (2 hunks)
Files skipped from review as they are similar to previous changes (2)
- indexer/services/vulcan/src/handlers/order-place-handler.ts
- indexer/services/vulcan/src/lib/on-message.ts
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review Status
Actionable comments generated: 1
Configuration used: CodeRabbit UI
Files selected for processing (6)
- indexer/packages/v4-protos/src/index.ts (1 hunks)
- indexer/packages/v4-protos/src/utils.ts (1 hunks)
- indexer/services/ender/tests/helpers/indexer-proto-helpers.ts (2 hunks)
- indexer/services/ender/src/lib/helper.ts (3 hunks)
- indexer/services/vulcan/tests/handlers/order-remove-handler.test.ts (23 hunks)
- indexer/services/vulcan/tests/helpers/constants.ts (1 hunks)
Additional comments: 24
indexer/packages/v4-protos/src/utils.ts (2)
- 3-4: Constants
MILLIS_IN_NANOS
andSECONDS_IN_MILLIS
are correctly defined and named, providing clear intent for time conversion purposes.- 5-11: The
protoTimestampToDate
function correctly converts a protobuf Timestamp to a JavaScript Date object. Consider adding a comment explaining the conversion process for future maintainability.indexer/packages/v4-protos/src/index.ts (1)
- 19-19: The addition of the export statement for the 'utils' module is correctly implemented, enabling other parts of the application to utilize the utilities defined in
utils.ts
.indexer/services/ender/__tests__/helpers/indexer-proto-helpers.ts (1)
- 53-53: The update in imports for
protoTimestampToDate
, moving it to@dydxprotocol-indexer/v4-protos
, is a good practice. It centralizes utility functions, reducing code duplication and improving maintainability.indexer/services/vulcan/__tests__/handlers/order-remove-handler.test.ts (20)
- 8-10: The addition of
defaultTime
from../helpers/constants
is a good practice for ensuring consistent time values across tests. This helps in creating predictable and reproducible test outcomes.- 141-143: The introduction of
defaultKafkaHeaders
withmessage_received_timestamp
derived fromdefaultTime
is a solid approach to simulate realistic Kafka headers in test scenarios. This ensures that tests accurately reflect the handling of headers in production code.- 191-194: It's commendable to see thorough testing of failure scenarios, ensuring that the system behaves as expected when encountering invalid updates. This contributes significantly to the robustness of the order removal process.
- 209-212: Testing for early return conditions, such as being unable to find an order in Redis, is crucial for ensuring the resilience of the system. This test case effectively verifies that the system can gracefully handle missing data scenarios.
- 233-236: Similar to the previous comment, testing for the absence of a perpetual market and ensuring the system's graceful handling of such a scenario is vital for maintaining system integrity under various conditions.
- 341-344: The comprehensive testing of successful order removal across different order types and conditions demonstrates a deep understanding of the system's requirements. This ensures that the order removal process is well-tested for a variety of scenarios.
- 494-497: Testing for the
BEST_EFFORT_CANCELED
status in order removal scenarios is important for ensuring that the system can handle different cancellation statuses appropriately. This adds another layer of robustness to the order management system.- 631-634: Ensuring that the orderbook levels cache is not affected when an order is removed but was not resting on the book is a critical aspect of maintaining data integrity. This test case effectively validates that behavior.
- 772-775: Testing for scenarios where the total filled quantity exceeds the order's quantums and ensuring that the orderbook level is not incorrectly increased is crucial for maintaining accurate market data. This test case effectively addresses that concern.
- 1053-1056: Logging errors when an order cannot be found in Postgres during stateful cancellation is essential for debugging and operational monitoring. This test ensures that such errors are correctly logged, aiding in system maintenance.
- 1104-1107: Sending subaccount websocket messages for orders not found in Redis during stateful cancellation scenarios is an important aspect of ensuring that clients are informed about order status changes. This test case validates that functionality.
- 1207-1210: The removal of stateful orders that are not resting on the book and ensuring that the orderbook levels cache is not affected is crucial for data accuracy. This test case effectively verifies that the system behaves as expected in such scenarios.
- 1327-1330: Successfully removing stateful orders that are resting on the book and correctly updating the orderbook levels cache is vital for maintaining market integrity. This test case thoroughly validates that process.
- 1474-1477: Ensuring that the orderbook level is not incorrectly increased when the total filled quantity exceeds the order's quantums in stateful cancellation scenarios is important for data accuracy. This test case addresses that concern effectively.
- 1599-1602: Successfully removing expired orders and ensuring that the orderbook levels cache is correctly updated is crucial for the integrity of market data. This test case effectively validates the handling of indexer-expired orders.
- 1733-1736: Testing for the successful removal of fully filled expired orders without sending unnecessary websocket messages is important for system efficiency. This test case ensures that the system behaves as expected in such scenarios.
- 1798-1801: Handling errors gracefully when the latest block cannot be found during the expiry verification of indexer-expired orders is crucial for system reliability. This test case ensures that such errors are correctly logged and handled.
- 1868-1871: Properly handling scenarios where an order cannot be found during the expiry verification of indexer-expired orders is important for system integrity. This test case ensures that the system logs such occurrences and exits gracefully.
- 1944-1947: Ensuring that long-term orders are not incorrectly processed during the expiry verification of indexer-expired orders is crucial for maintaining data accuracy. This test case validates that the system correctly identifies and logs such scenarios.
- 2009-2012: Testing for scenarios where an order is not actually expired during the expiry verification of indexer-expired orders is important for avoiding unnecessary cancellations. This test case ensures that the system correctly identifies such cases and exits gracefully.
export const defaultTime: Timestamp = { | ||
seconds: Long.fromValue(Math.floor(defaultDateTime.toSeconds()), true), | ||
nanos: (defaultDateTime.toMillis() % SECONDS_IN_MILLIS) * MILLIS_IN_NANOS, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The calculation of nanos
in defaultTime
might not be correct. It seems to multiply the remainder of milliseconds by MILLIS_IN_NANOS
, which would result in an incorrect value for nanoseconds. The correct approach should be to take the remainder of milliseconds (after converting to seconds) and then multiply by MILLIS_IN_NANOS
to convert it to nanoseconds.
- nanos: (defaultDateTime.toMillis() % SECONDS_IN_MILLIS) * MILLIS_IN_NANOS,
+ nanos: (defaultDateTime.toMillis() % SECONDS_IN_MILLIS) * 1000,
Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.
export const defaultTime: Timestamp = { | |
seconds: Long.fromValue(Math.floor(defaultDateTime.toSeconds()), true), | |
nanos: (defaultDateTime.toMillis() % SECONDS_IN_MILLIS) * MILLIS_IN_NANOS, | |
export const defaultTime: Timestamp = { | |
seconds: Long.fromValue(Math.floor(defaultDateTime.toSeconds()), true), | |
nanos: (defaultDateTime.toMillis() % SECONDS_IN_MILLIS) * 1000, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@@ -64,7 +64,7 @@ describe('onMessage', () => { | |||
await onMessage(message); | |||
|
|||
expect(handler).toHaveBeenCalledTimes(1); | |||
expect(handleUpdateMock).toHaveBeenCalledWith(update); | |||
expect(handleUpdateMock).toHaveBeenCalledWith(update, message.headers ?? {}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The update to include message.headers ?? {}
in the test assertion is a good practice, ensuring that the function is tested with the expected headers or a fallback to an empty object if headers are undefined. This aligns well with the PR objectives.
However, consider adding a specific test case to explicitly cover scenarios where message.headers
is undefined, to ensure robustness.
Would you like assistance in crafting such a test case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no it is not the ?? operator sucks and promotes bad coding practices
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jonfung-dydx, I understand your concerns regarding the nullish coalescing operator and its impact on coding practices. It's important to align with coding standards that best suit the project's needs and preferences.
As an alternative to using the ??
operator, you could explicitly check for undefined
headers and provide a default value in such cases. This approach can maintain code clarity and robustness without relying on the ??
operator. Here's a modified version of the assertion:
expect(handleUpdateMock).toHaveBeenCalledWith(update, message.headers ? message.headers : {});
Or, if the project's style guide permits, using a default parameter in the function definition could also serve the purpose, ensuring headers
always have a default value when not provided.
I hope this suggestion aligns better with your coding practices. If there's anything more I can help with, please let me know.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review Status
Actionable comments generated: 0
Configuration used: CodeRabbit UI
Files selected for processing (1)
- indexer/services/vulcan/tests/lib/on-message.test.ts (1 hunks)
Files skipped from review as they are similar to previous changes (1)
- indexer/services/vulcan/tests/lib/on-message.test.ts
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review Status
Actionable comments generated: 0
Configuration used: CodeRabbit UI
Files selected for processing (5)
- indexer/services/vulcan/tests/handlers/order-place-handler.test.ts (3 hunks)
- indexer/services/vulcan/tests/handlers/order-remove-handler.test.ts (24 hunks)
- indexer/services/vulcan/tests/helpers/constants.ts (1 hunks)
- indexer/services/vulcan/tests/helpers/helpers.ts (3 hunks)
- indexer/services/vulcan/tests/helpers/websocket-helpers.ts (1 hunks)
Path instructions used (3)
indexer/services/vulcan/__tests__/helpers/websocket-helpers.ts (1)
/
Do not include a walkthrough. For all comments on line changes that are longer than 15 lines, add a prefix to the comment of what lines you are commenting on (for example: "The changes from lines 100 to 200...". Add your comment to be on the last 15 lines of the chunk rather than adding your comment to the entire chunk.indexer/services/vulcan/__tests__/helpers/helpers.ts (1)
/
Do not include a walkthrough. For all comments on line changes that are longer than 15 lines, add a prefix to the comment of what lines you are commenting on (for example: "The changes from lines 100 to 200...". Add your comment to be on the last 15 lines of the chunk rather than adding your comment to the entire chunk.indexer/services/vulcan/__tests__/handlers/order-place-handler.test.ts (1)
/
Do not include a walkthrough. For all comments on line changes that are longer than 15 lines, add a prefix to the comment of what lines you are commenting on (for example: "The changes from lines 100 to 200...". Add your comment to be on the last 15 lines of the chunk rather than adding your comment to the entire chunk.
Files skipped from review as they are similar to previous changes (2)
- indexer/services/vulcan/tests/handlers/order-remove-handler.test.ts
- indexer/services/vulcan/tests/helpers/constants.ts
Additional comments (4)
indexer/services/vulcan/__tests__/helpers/websocket-helpers.ts (1)
- 8-18: The addition of
expectedHeaders
parameter and its validation against the headers in theProducerRecord
is correctly implemented and aligns with the PR's objective to enhance end-to-end latency tracking. Good job on ensuring thorough testing of the new functionality.indexer/services/vulcan/__tests__/helpers/helpers.ts (2)
- 27-27: The assignment of
defaultKafkaHeaders
tomessage.headers
inhandleInitialOrderPlace
is correctly implemented, ensuring consistent headers for Kafka messages in tests. This aligns well with the PR's objective to enhance latency tracking.- 41-41: The assignment of
defaultKafkaHeaders
tomessage.headers
inhandleOrderUpdate
is correctly implemented, ensuring consistent headers for Kafka messages in tests. This consistency is crucial for accurately testing the new latency tracking functionality.indexer/services/vulcan/__tests__/handlers/order-place-handler.test.ts (1)
- 193-204: > 📝 NOTE
This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [66-200]
The import and usage of
defaultKafkaHeaders
to set headers for Kafka messages in test setups are correctly implemented. This ensures consistency in message handling and aligns well with the PR's objective to enhance latency tracking. Good job on ensuring thorough testing of the new functionality.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Review Status
Actionable comments generated: 0
Configuration used: CodeRabbit UI
Files selected for processing (3)
- indexer/services/ender/tests/helpers/indexer-proto-helpers.ts (2 hunks)
- indexer/services/vulcan/tests/handlers/order-place-handler.test.ts (3 hunks)
- indexer/services/vulcan/src/handlers/order-place-handler.ts (7 hunks)
Files skipped from review as they are similar to previous changes (3)
- indexer/services/ender/tests/helpers/indexer-proto-helpers.ts
- indexer/services/vulcan/tests/handlers/order-place-handler.test.ts
- indexer/services/vulcan/src/handlers/order-place-handler.ts
This reverts commit 60b94df.
* fwd through message times * use the var i made * post processing stat emission * post-forwarding timestamp * pass through event type from vulcan * event type to stat emissions * test fix function calls * WIP WIP WIP * fix tests * unused import * test that kafka messages are threaded
* Add subaccountNumber to PerpetualPositionResponseObject (#1274) Signed-off-by: Shrenuj Bansal <[email protected]> * [CT-712] send updates for both normal order matches and liquidation (#1280) * Remove volatile market (#1263) * [SKI-21] Bump slinky version to v0.3.1 (#1275) * Bump slinky version * Add cp number method * clean up socks logging (#1285) * [CT-681] fix liquidated side and offsetting side for indexer delevera… (#1284) * [CT-681] fix liquidated side and offsetting side for indexer deleveraging events * fix test * fix test * fix lint * [TRA-105] Add API for parent subaccount perpetual positions (#1282) Signed-off-by: Shrenuj Bansal <[email protected]> * Use sample rate with stream destroyed stats. (#1294) * Revert "[CT-708] Indexer track e2e latency (#1237)" (#1292) This reverts commit 60b94df. * Fix swagger generation makefile command / regen swagger docs (#1299) * pull dydx fork to generate swagger properly * remove the print * remove vault constants (#1293) * Remove custom ping message from socks (#1301) * Add subaccountNumber to the OrderResponseObject (#1296) Signed-off-by: Shrenuj Bansal <[email protected]> * sample more metrics (#1304) * [OTE-256] Add upgrade handler to initialize OI during upgrade handler (#1302) * Add upgrade handler to initialize OI during upgrade handler * nits * Fix lib.ErrorLogWithError: properly pass in args (#1306) * fix broken tests (#1312) * Explicitly close websockets on errors (#1290) * Increase the number of allowed connections to 8000 (#1317) * [TRA-104] Add parentSubaccountNumber API for orders (#1313) Signed-off-by: Shrenuj Bansal <[email protected]> * Improve Slinky logs to prevent unnecessary logs (#1289) * [SKI-26]: Prevent funding index update with no oracle prices from (#1321) halting indexer * Skip equity tier limit check in PlaceShortTermOrder (#1318) * Skip equity tier limit check in PlaceShortTermOrder * remove tests * Add comment * fix lint (#1323) --------- Signed-off-by: Shrenuj Bansal <[email protected]> Co-authored-by: shrenujb <[email protected]> Co-authored-by: jayy04 <[email protected]> Co-authored-by: Eric Warehime <[email protected]> Co-authored-by: vincentwschau <[email protected]> Co-authored-by: Jonathan Fung <[email protected]> Co-authored-by: Tian <[email protected]> Co-authored-by: Teddy Ding <[email protected]> Co-authored-by: roy-dydx <[email protected]> Co-authored-by: Christopher-Li <[email protected]>
* [CT-708] Indexer track e2e latency (#1237) * fwd through message times * use the var i made * post processing stat emission * post-forwarding timestamp * pass through event type from vulcan * event type to stat emissions * test fix function calls * WIP WIP WIP * fix tests * unused import * test that kafka messages are threaded * pass through message headers verbatim * test logs for on message * short term order event types
* [CT-708] Indexer track e2e latency (#1237) * fwd through message times * use the var i made * post processing stat emission * post-forwarding timestamp * pass through event type from vulcan * event type to stat emissions * test fix function calls * WIP WIP WIP * fix tests * unused import * test that kafka messages are threaded * pass through message headers verbatim * test logs for on message * short term order event types (cherry picked from commit 4daa11d) # Conflicts: # indexer/services/socks/src/lib/message-forwarder.ts
vulcan -> socks. Thread through the kafka headers.
Socks, emit the e2e latency.
Vulcan, emit the e2e latency.