Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

copy fix: arbitrum network cost estimate #466

Merged
merged 1 commit into from
Jan 5, 2024

Conversation

jsy1218
Copy link
Member

@jsy1218 jsy1218 commented Jan 4, 2024

  • What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
    This PR fixes 2 bugs that causes inflated network cost (gas fee) estimate for Arbitrum.

  • What is the current behavior? (You can also link to an open issue here)
    Smart Order Router is inflating the Arbitrum fee due to double counting L1 fee and a incorrect unit conversion

  • What is the new behavior (if this is a feature change)?

    1. Removed double counting of L1 fee in calculateGasUsed logic
    2. Converted per byte fee into per gas fee by dividing 16
  • Other information:
    See discussions in fix: arbitrum network cost estimate #464 for original PR. This PR is created because it runs within https://github.com/Uniswap/smart-order-router, which contains all the repo secrets to successfully run all the tests.

@jsy1218 jsy1218 requested a review from a team as a code owner January 4, 2024 17:38
@jsy1218 jsy1218 changed the title fix: arbitrum network cost estimate copy fix: arbitrum network cost estimate Jan 4, 2024
@jsy1218
Copy link
Member Author

jsy1218 commented Jan 4, 2024

@Uniswap/routing @gzeoneth I did some additional local CLI testing to see the improvements:

  1. Before the fix, if I issue a USDT -> WETH swap at block 167093243
    1. with no simuation, the gas estimate is 131000: https://app.warp.dev/block/QQFGICDJvpkwmg33bPaQpC. Note this resembles what's actually happening in prod, when user swaps on Uniswap web interface.
    2. with simulation, the gas estimate is 1300760: https://app.warp.dev/block/7m9yKFvSRKWc2KtzYl8t6r. Note this resembles what's actually happening in prod, when user swaps on Uniswap mobile wallet.
  2. After the fix, if I issue a USDT -> WETH swap at block 167093243
    1. with no simuation, the gas estimate is 358000: https://app.warp.dev/block/wsLNAnc1COTEer4LNprtuy.
    2. with simulation, the gas estimate is 1678101: https://app.warp.dev/block/6VToHXNROUEW3pDEDhSK49.

I think we should ship this PR, and have couple more fixes as follow-up items (Although some fixes require deeper thinking):

  1. When estimating the L1 calldata posting cost, right now SOR uses the non-compressed calldata bytesize https://github.com/Uniswap/smart-order-router/blob/main/src/util/gas-factory-helpers.ts#L187-L198. Arbitrum uses Brotli algorithm to compress, and Optimism uses Zlib. I think it's reasonable to assume every major rollup chain will compress the calldata that's going to get posted to L1 (even post Danksharding), so it makes sense to take into account the compression ratio within SOR, when estimating the L1 calldata posting cost.
  2. Right now SOR uses OVM_GasPriceOracle as a reference to calculate L1 fee gas. For a more accurate L1 fee estimate, I think we should instead rely on Arbitrum's RPC gasEstimateL1Component. One caveat is that when SOR calculates L1 fee gas, as part of estimateGasCost, it should avoid making additional RPC calls. This makes me think that SOR needs to somehow replicate the Nitro's GO implementation of GasEstimateL1Component. A bigger question though - since Arbitrum charges L2 gas for L1 calldata posting cost, it actually makes more sense to just compute the L1 + L2 gas estimate together for Arbitrum in SOR. However the current SOR gas model relies on the assumption that all rollup chains charges L2 gas & L1 gas separately. Also Uniswap interface doesn't show a breakdown of L1 + L2 fee breakdown for the network cost. To address bigger question, it requires more thinking.
  3. (this is somewhat difficult) Arbitrum charges L2 gas for L1 calldata posting cost. V3HeuristicGasModelFactory separates the calculation into calculateL1GasFees and estimateGasCost. The reason why 2.i. still has significantly underestimated gas estimate is due to this. In getBestSwapRouteBy, estimatedGasUsed is calculated using the estimateGasCost heuristic only for all L2s. But for Arbitrum, it should also use calculateL1GasFees, so that the final estimatedGasUsed will be much less underestimated. Fixing this one requires more thoughts, because it might impact the best swap route selection.

@gzeoneth
Copy link
Contributor

gzeoneth commented Jan 5, 2024

The big difference between the gas estimate with/wout simulation is due to the SOR

  • without simulation: print estimatedGasUsed that represent execution gas
  • with simulation: print estimatedGasUsed that represent total gas = calldata + execution gas
    but the USD amount is actually much closer since L1 calldata is added into the USD amount in both path

The difference between pre/post fix with simulation is that SOR is picking another route, due to to previously wrong math inflated the USD amount that led to a suboptimal route being picked.

@gzeoneth
Copy link
Contributor

gzeoneth commented Jan 5, 2024

re follow-up items:

  1. Treating Arbitrum as normal evm chain (use eth_gasEstimate / simulation) would simplify things a lot, although it might not always be possible to estimate due to the need of approval override. In that case you can use gasEstimateL1Component or if you want to save a rpc call, see f819134.
  2. To factor in the calldata l2 gas for Arbiturm, we can return an extra value from the gas helper, see 187a309

Obviously those change are probably not production ready. I also found it quite confusing to adapt to SOR's L2 gas assumption without creating api breaking changes. Also noticed:

  1. GasData is assumed to be static, tho it is dynamic and if requestBlockNumber is a historical block the estimate will be off. Ideally it should fetch the GasData from an archive node for the specific block requested.
  2. Receipent is not included in the calldata when finding the best route, this might cause the algo to underestimate 20 bytes per hop.
  3. There was an additional overhead that is only applicable to Optimism but not Arbitrum, in Arbitrum the perL2TxFee covers that. It is refactored away in f819134.

That said, with these 2 commit I am getting a pretty close estimate in terms of gas and USD

$ ./bin/cli quote --tokenIn 0xfd086bc7cd5c481dcc9c85ebe478a1c0b69fcbb9 --tokenOut 0x82af49447d8a07e3bd95bd0d56f35241523fbab1 --amount 20000 --exactIn --minSplits 1 --router alpha --chainId 42161 --recipient 0xb38e8c17e38363af6ebdcb3dae12e0243582891d --simulate
Best Route:
[V3] 80.00% = USDT -- 0.05% [0x641C00A822e8b671738d32a431a4Fb6074E5c79d] --> WETH, [V3] 20.00% = USDT -- 0.01% [0x8c9D230D45d6CfeE39a6680Fb7CB7E8DE7Ea8E71] --> USDC -- 0.05% [0xC31E54c7a869B9FcBEcc14363CF510d1c41fa443] --> WETH
        Raw Quote Exact In:
                8.91
        Gas Adjusted Quote In:
                8.91

Gas Used Quote Token: 0.000137
Gas Used USD: 0.308263
Calldata: 0x3593564c000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000a0000000000000000000000000000000000000000000000000000009184e72a000000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000001600000000000000000000000000000000000000000000000000000000000000100000000000000000000000000b38e8c17e38363af6ebdcb3dae12e0243582891d00000000000000000000000000000000000000000000000000000003b9aca0000000000000000000000000000000000000000000000000005e426f91ab1eef8800000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000002bfd086bc7cd5c481dcc9c85ebe478a1c0b69fcbb90001f482af49447d8a07e3bd95bd0d56f35241523fbab10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000120000000000000000000000000b38e8c17e38363af6ebdcb3dae12e0243582891d00000000000000000000000000000000000000000000000000000000ee6b2800000000000000000000000000000000000000000000000000178ff9163bd1095f00000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000042fd086bc7cd5c481dcc9c85ebe478a1c0b69fcbb9000064ff970a61a04b1ca14834a43f5de4533ebddb5cc80001f482af49447d8a07e3bd95bd0d56f35241523fbab1000000000000000000000000000000000000000000000000000000000000
Value: 0x00

  blockNumber: "167333306"
  estimatedGasUsed: "1374404"
  gasPriceWei: "100000000"
  simulationStatus: 2
Total ticks crossed: 3

$ ./bin/cli quote --tokenIn 0xfd086bc7cd5c481dcc9c85ebe478a1c0b69fcbb9 --tokenOut 0x82af49447d8a07e3bd95bd0d56f35241523fbab1 --amount 20000 --exactIn --minSplits 1 --router alpha --chainId 42161 --recipient 0xb38e8c17e38363af6ebdcb3dae12e0243582891d
Best Route:
[V3] 55.00% = USDT -- 0.01% [0x8c9D230D45d6CfeE39a6680Fb7CB7E8DE7Ea8E71] --> USDC -- 0.05% [0xC31E54c7a869B9FcBEcc14363CF510d1c41fa443] --> WETH, [V3] 45.00% = USDT -- 0.05% [0x641C00A822e8b671738d32a431a4Fb6074E5c79d] --> WETH
        Raw Quote Exact In:
                8.91
        Gas Adjusted Quote In:
                8.91

Gas Used Quote Token: 0.000132
Gas Used USD: 0.297242
Calldata: 0x3593564c000000000000000000000000000000000000000000000000000000000000006000000000000000000000000000000000000000000000000000000000000000a0000000000000000000000000000000000000000000000000000009184e72a000000000000000000000000000000000000000000000000000000000000000000200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000000000000000000001800000000000000000000000000000000000000000000000000000000000000120000000000000000000000000b38e8c17e38363af6ebdcb3dae12e0243582891d000000000000000000000000000000000000000000000000000000028fa6ae0000000000000000000000000000000000000000000000000040c8910b2f1a665d00000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000042fd086bc7cd5c481dcc9c85ebe478a1c0b69fcbb9000064ff970a61a04b1ca14834a43f5de4533ebddb5cc80001f482af49447d8a07e3bd95bd0d56f35241523fbab10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000b38e8c17e38363af6ebdcb3dae12e0243582891d0000000000000000000000000000000000000000000000000000000218711a00000000000000000000000000000000000000000000000000350162788d03f0a700000000000000000000000000000000000000000000000000000000000000a00000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000002bfd086bc7cd5c481dcc9c85ebe478a1c0b69fcbb90001f482af49447d8a07e3bd95bd0d56f35241523fbab1000000000000000000000000000000000000000000
Value: 0x00

  blockNumber: "167333164"
  estimatedGasUsed: "1325266"
  gasPriceWei: "100000000"
Total ticks crossed: 3

https://www.tdly.co/shared/simulation/c236561e-1bbc-40b4-987f-f436ca6b642e

But agree, we can ship this PR first and fix the rest later. This PR already reduced a magnitude of the error of USD fee estimate.

Copy link
Contributor

@cgkol cgkol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets give it a go ✅

@jsy1218 jsy1218 merged commit c147528 into main Jan 5, 2024
13 checks passed
@jsy1218 jsy1218 deleted the jsy1218/fix-arb-gasEstimateUsed-field branch January 5, 2024 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants