Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: refactor fetcher to extend kona's fetcher instead of copying all of its logic #1

Merged
merged 9 commits into from
Dec 17, 2024

Conversation

samlaf
Copy link
Collaborator

@samlaf samlaf commented Dec 14, 2024

Currently our fetcher is copy pasted from kona's fetcher with an added 20ish lines to support eigenda hints.

This PR refactors hokulea's fetcher into an ExtendedFetcher struct which wraps kona's fetcher instead, which simplifies the code and makes it easy to inherit any code modifications to kona's logic by simply bumping the client's version.

There has to be a better way to architect this wrapping... see TODO comments in the code.

Tests

I tried running this against the devnet the same way we did in our kona-fork repo (I also added steps in README in this PR), but it doesn't work (even the current main branch). How did you run this @bxue-l2 ?

[...]
2024-12-14T20:36:02.417350Z ERROR frame-queue: Failed to parse frames from data.
2024-12-14T20:36:02.417355Z  WARN pipeline: Attributes queue step failed: Temporary(NotEnoughData)
2024-12-14T20:36:02.417357Z  WARN client_derivation_driver: Failed to step derivation pipeline: Temporary(NotEnoughData)
2024-12-14T20:36:02.417360Z  INFO batch-queue: Deriving next batch for epoch: 11739
2024-12-14T20:36:02.417374Z  WARN client_derivation_driver: Failed to step derivation pipeline: Critical(EndOfSource)
2024-12-14T20:36:02.417376Z  WARN client: Exhausted data source; Halting derivation and using current safe head.
2024-12-14T20:36:02.417378Z  INFO client: Derivation complete, reached L2 safe head.
2024-12-14T20:36:02.417381Z ERROR client: Failed to validate L2 block #64388 with output root 0x266831f1627fb6f7373406d1c181edbb6633d607ce723eb0efb3d33b7648c18c
2024-12-14T20:36:02.417445Z  INFO hokulea_host: Preimage server and client program have joined.
error: Recipe `run-client-native` failed with exit code 1
error: Recipe `run-client-native-against-devnet` failed with exit code 1

@samlaf samlaf requested a review from bxue-l2 December 14, 2024 20:47
@samlaf samlaf mentioned this pull request Dec 15, 2024
Copy link
Collaborator

@bxue-l2 bxue-l2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this change

/// Set the last hint to be received.
pub fn hint(&mut self, hint: &str) {
trace!(target: "fetcher", "Received hint: {hint}");
self.last_hint = Some(hint.to_string());
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since we use self.fetcher.get_preimage(key), 10 lines below, we should pass the last_hint into the inner fetcher.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh snap that's a good point... kona's fetcher's hint is no longer updated now right?
Hmm not sure how to best structure this then, we can talk tomorrow.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the logic quite a bit, see: a7723ee

@bxue-l2
Copy link
Collaborator

bxue-l2 commented Dec 15, 2024

Looking at alloy vs op-alloy, it looks that when they add a new type inside enum, the whole enum is replicated

@samlaf samlaf force-pushed the chore--extend-konas-fetcher-instead-of-forking branch from 848e10e to 31e02bf Compare December 17, 2024 03:32
- Logic was previously routing to both inner fetcher and eigenda fetcher, which was a bug
- Renamed ExtendedFetcher -> FetcherWithEigenDASupport
- Also tested against devnet successfully
result = self.get_preimage_altda(key) => result,
result = self.fetcher.get_preimage(key) => result,
match self.last_eigenda_hint.as_ref() {
Some(hint) => self.get_preimage_altda(key, hint).await,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sense

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name change get_preimage_eigend?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

}
}

async fn get_preimage_altda(&self, key: B256) -> Result<Vec<u8>> {
trace!(target: "extended fetcher", "Pre-image requested. Key: {key}");
async fn get_preimage_altda(&self, key: B256, hint: &str) -> Result<Vec<u8>> {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change the name into get_preimage_eigenda?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


if hint_type == ExtendedHintType::AltDACommitment {
if hint_type == ExtendedHintType::EigenDACommitment {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add else part that panic! when accidentally get other hint

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_ => {
self.fetcher.hint(hint);
// get_preimage will fetch from the underlying fetcher when last_eigenda_hint = None
self.last_eigenda_hint = None;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just to confirm last_hint is a private field, so it is safe to set last_eigenda_hint to be None

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct

@samlaf samlaf merged commit 63e19d9 into master Dec 17, 2024
5 checks passed
@samlaf samlaf deleted the chore--extend-konas-fetcher-instead-of-forking branch December 17, 2024 21:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants