You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
trade aggregations is implemented within horizon, it is intertwined into ingestion, db, and api. horizon ingestion must spend extra time doing trade aggregation tasks after each live ledger is ingested and during reingestion ranges to keep the trade agg tables fresh, this adds overhead to ingestion processing times and also introduces complexities for parallel worker ingestion such as #5127.
What would you like to see?
refactor trade aggregations for horizon benefits, there are two options from least to most effort:
change the trade aggregation bucket to align per ledger rather than a unit of time that could span ledgers as that prevents parallel worker reingestion on contiguous ranges - see comment here
pull out trade aggregation from horizon and package as a standalone microservice, I think this becomes a viable option to consider when pre-compiled remote tx-meta, #4911 becomes available, the trade agg microservice can follow stream topology pattern of running as a processor using the tx-meta as input stream and derive the trade agg data models from there, no traditional ingestion needed.
What alternatives are there?
The text was updated successfully, but these errors were encountered:
What problem does your feature solve?
trade aggregations is implemented within horizon, it is intertwined into ingestion, db, and api. horizon ingestion must spend extra time doing trade aggregation tasks after each live ledger is ingested and during reingestion ranges to keep the trade agg tables fresh, this adds overhead to ingestion processing times and also introduces complexities for parallel worker ingestion such as #5127.
What would you like to see?
refactor trade aggregations for horizon benefits, there are two options from least to most effort:
What alternatives are there?
The text was updated successfully, but these errors were encountered: