SDK with Stwo + Legacy Support (#311)

* Baby steps towards porting over sdk.

* Sketch out updated traits.

* More work on new traits.

* More prep work.

* Move macros inboard to SDK and do a lot more prep.

* Incremental work towards building with nova.

* More work on Nova translation.

* Get Nova set.

* Get building with serialization.

* Make some lifetimes easier to manage.

* Minor.

* Work on examples.

* Add legacy examples.

* Get example building working.

* Start working on HyperNova

* Work around strange issue with implementation.

* Get hypernova example working.

* Port over Jolt.

* Format.

* Some trait cleaning up.

* More formatting.

* Even more formatting.

* Fix dependency.

* Fix traits and some implementations

* Minor.

* Try to fix CI error.

* Fix.

* Clippy.

* Clean up .cargo.

* Fix CI.

* Update integration tests.

* Start to work on Stwo integration.

* More work.

* More cleaning up of components.

* A lot of cleanup + redesign work.

* Add log capturing.

* More redesign of log handling.

* Big format.

* More work towards final version.

* Finish out compilation.

* Minor fixes and improvements.

* Extend use of InternalView for tests.

* Fix change.

* Fixes and formatting.

* Sketch out verification interfaces

* A lot of cleanup.

* More fixing.

* Getting there.

* Finish verification.

* Almost there...

* Get building.

* Some clippy.

* Finish clippy.

* Format.

* Fixing tests.

* Format.

* Get building.

* Formatting

* Minor.

* More.

* Somehow even more.

* Oh please make it stop...

* Improved logging elements.

* Fix tests.

* Fix

* More formatting.

* Start to work on examples.

* Get first example working.

* Get all examples working via COBS.

* Clippy

* Format.

* Update sdk/macros/Cargo.toml

Co-authored-by: Ben Hoberman <bhoberman@users.noreply.github.com>

* Minor.

* Fix testing infrastructure.

* Format.

* Move utility functions.

* Format.

* Fix cargo config.

* Fix branch.

* Fix no input examples.

* Getting legacy examples working...

* Add feature gating.

* Port over minimal cli with host program setup.

* Update README.

* Format.

* Update header.

* Typo.

* More formatting.

* A few last fixes.

* Fix doctest.

* Temp improve main readme.

* Spruce up readme a bit more.

* Remove some todos.

* Add some clarifying comments.

* Carry raw logs.

---------

Co-authored-by: Ben Hoberman <bhoberman@users.noreply.github.com>

.github/workflows/ci.yml

@@ -52,8 +52,9 @@ jobs:
 
       - name: Run `cargo check`
         run: |
-          cargo check --all-features --all-targets --workspace
-          cargo check --all-features --examples --workspace
+          cargo check --all-features --all-targets --workspace --exclude example
+          cargo check --all-features --all-targets --examples --workspace --exclude example
+          cargo check --package example --target riscv32i-unknown-none-elf
           cd prover-benches && cargo check --benches --workspace
 
       - name: Run `cargo clippy`

Cargo.toml

@@ -7,9 +7,12 @@ members = [
     "precompiles",
     "common",
     "core",
+    "sdk",
     "examples",
+    "examples/legacy",
     "tests/testing-framework",
     "benchmarks",
+    "cli",
 ]
 default-members = ["runtime", "prover", "vm", "precompiles", "common", "core"]
 exclude = ["prover-benches"]

README.md

@@ -1,18 +1,12 @@
 # The Nexus zkVM
 
 <div align="left">
-    <a href="https://t.me/nexus_zkvm">
-        <img src="https://img.shields.io/endpoint?color=neon&logo=telegram&label=chat&url=https%3A%2F%2Fmogyo.ro%2Fquart-apis%2Ftgmembercount%3Fchat_id%3Dnexus_zkvm"/></a>
     <a href="https://github.com/nexus-xyz/nexus-zkvm/graphs/contributors">
         <img src="https://img.shields.io/github/contributors/nexus-xyz/nexus-zkvm.svg"></a>
-    <a href="https://twitter.com/NexusLabsHQ">
+    <a href="https://x.com/NexusLabs">
         <img src="https://img.shields.io/badge/Twitter-black?logo=x&logoColor=white"/></a>
     <a href="https://nexus.xyz">
         <img src="https://img.shields.io/static/v1?label=Stage&message=Alpha&color=2BB4AB"/></a>
-    <a href="https://github.com/nexus-xyz/nexus-zkvm/blob/main/LICENSE-MIT">
-        <img src="https://img.shields.io/badge/license-MIT-blue"/></a>
-    <a href="https://github.com/nexus-xyz/nexus-zkvm/blob/main/LICENSE-APACHE">
-        <img src="https://img.shields.io/badge/license-APACHE-blue"/></a>
 </div>
 
 <p align="center">
@@ -21,8 +15,6 @@
   </p>
 </p>
 
-The Nexus zkVM is a modular, extensible, open-source, and highly-parallelized zkVM, designed to run at *a trillion CPU cycles proved per second* given enough machine power.
+Built with [Stwo](https://github.com/starkware-libs/stwo) by [StarkWare](https://starkware.co/blog/starkware-new-proving-record/).
 
-## Examples
-
-Check out the [prover examples](./prover/README.md#Examples).
+*The Nexus zkVM is in an experimental stage and is not currently recommended for production use.*

benchmarks/src/runner.rs

@@ -142,14 +142,7 @@ pub fn run_benchmark<T>(
 
     // Measure verification.
     let (start_time, initial_self, initial_children) = phase_start();
-    verify(
-        proof,
-        view.get_program_memory(),
-        view.get_initial_memory(),
-        view.get_exit_code(),
-        view.get_public_output(),
-    )
-    .unwrap();
+    verify(proof, &view).unwrap();
     let (
         verification_duration,
         verification_user_time,

cli/Cargo.toml

@@ -0,0 +1,36 @@
+[package]
+name = "cargo-nexus"
+authors = { workspace = true }
+version = { workspace = true }
+edition = { workspace = true }
+homepage = { workspace = true }
+repository = { workspace = true }
+publish = { workspace = true }
+keywords = { workspace = true }
+categories = { workspace = true }
+default-run = "cargo-nexus"
+
+[lib]
+name = "nexus_cli"
+path = "src/lib.rs"
+
+[[bin]]
+name = "cargo-nexus"
+path = "src/bin/cargo-nexus.rs"
+
+[dependencies]
+anyhow = "1.0"
+dotenvy = "0.15.7"
+tracing = { version = "0.1", default-features = false, features = ["std"] }
+tracing-subscriber = { version = "0.3", default-features = false, features = [
+    "fmt",
+    "ansi",
+    "chrono",
+    "env-filter",
+] }
+
+cargo_metadata = "0.18.1"
+clap.workspace = true
+
+nexus-core = { path = "../core" }
+nexus-progress-bar = { path = "./progress-bar" }

cli/progress-bar/Cargo.toml

@@ -0,0 +1,14 @@
+[package]
+name = "nexus-progress-bar"
+edition.workspace = true
+version.workspace = true
+authors.workspace = true
+homepage.workspace = true
+repository.workspace = true
+keywords.workspace = true
+categories.workspace = true
+publish.workspace = true
+
+[dependencies]
+anyhow = "1.0"
+superconsole = "0.2"

cli/progress-bar/src/action.rs

@@ -0,0 +1,44 @@
+use super::component::FmtDuration;
+
+pub(crate) struct Action {
+    pub iter: usize,
+    pub iter_num: usize,
+
+    pub step_header: &'static str,
+    pub step_trailing: Box<dyn Fn(usize) -> String + Send>,
+    pub loading_bar_header: Option<&'static str>,
+
+    pub completion_header: &'static str,
+    pub completion_trailing: Box<dyn Fn(FmtDuration) -> String + Send>,
+}
+
+impl Action {
+    pub(crate) fn show_progress(&self) -> bool {
+        self.loading_bar_header.is_some()
+    }
+
+    pub(crate) fn next_iter(&mut self) {
+        let next_iter = self.iter + 1;
+        assert!(next_iter <= self.iter_num);
+
+        self.iter = next_iter;
+    }
+
+    pub(crate) fn is_finished(&self) -> bool {
+        self.iter == self.iter_num
+    }
+}
+
+impl Default for Action {
+    fn default() -> Self {
+        Self {
+            iter: 0,
+            iter_num: 1,
+            step_header: "",
+            step_trailing: Box::new(|_step| String::new()),
+            loading_bar_header: None,
+            completion_header: "Finished",
+            completion_trailing: Box::new(|elapsed| format!("in {elapsed}")),
+        }
+    }
+}

cli/progress-bar/src/component/loading.rs

@@ -0,0 +1,69 @@
+use std::{cell::RefCell, time::Duration};
+
+use superconsole::{style::Stylize, Component, Dimensions, DrawMode, Line, Lines, Span};
+
+use crate::action::Action;
+
+const WIDTH: usize = "=======>                  ".len() - 1;
+
+pub struct LoadingBar<'a> {
+    pub time_spent: Duration,
+
+    action: &'a RefCell<Action>,
+}
+
+impl Component for LoadingBar<'_> {
+    fn draw_unchecked(&self, _: Dimensions, _: DrawMode) -> anyhow::Result<Lines> {
+        let action = self.action.borrow();
+
+        let res = if !action.is_finished() {
+            if !action.show_progress() {
+                return Ok(Lines::new());
+            }
+            let iteration = action.iter;
+            let total = action.iter_num;
+
+            let heading_span = Span::new_styled(
+                action
+                    .loading_bar_header
+                    .unwrap_or_default()
+                    .to_owned()
+                    .cyan()
+                    .bold(),
+            )?;
+
+            let percentage = iteration as f64 / total as f64;
+            let amount = (percentage * WIDTH as f64).ceil() as usize;
+
+            let loading_bar = format!(
+                " [{test:=>bar_amt$}{empty:padding_amt$}] {}/{}: ...",
+                iteration,
+                total,
+                test = ">",
+                empty = "",
+                bar_amt = amount,
+                padding_amt = WIDTH - amount,
+            );
+            let loading = Span::new_unstyled(loading_bar)?;
+            Line::from_iter([heading_span, loading])
+        } else {
+            let elapsed = self.time_spent;
+
+            let heading_span = Span::new_styled(action.completion_header.to_owned().blue().bold())?;
+            let completion_span = Span::new_unstyled((action.completion_trailing)(elapsed.into()))?;
+
+            Line::from_iter([heading_span, Span::padding(1), completion_span])
+        };
+
+        Ok(Lines(vec![res]))
+    }
+}
+
+impl<'a> LoadingBar<'a> {
+    pub fn new(action: &'a RefCell<Action>) -> Self {
+        Self {
+            action,
+            time_spent: Duration::ZERO,
+        }
+    }
+}

cli/progress-bar/src/component/mod.rs

@@ -0,0 +1,89 @@
+use std::{cell::RefCell, fmt, ops::Deref, time};
+
+use superconsole::{Component, Dimensions, DrawMode, Lines};
+
+use crate::action::Action;
+
+mod loading;
+mod timer;
+
+#[derive(Debug)]
+pub struct FmtDuration(time::Duration);
+
+impl From<time::Duration> for FmtDuration {
+    fn from(d: time::Duration) -> Self {
+        Self(d)
+    }
+}
+
+impl Deref for FmtDuration {
+    type Target = time::Duration;
+
+    fn deref(&self) -> &Self::Target {
+        &self.0
+    }
+}
+
+impl fmt::Display for FmtDuration {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        let secs = self.as_secs();
+        if secs == 0 {
+            write!(f, "{}ms", self.as_millis())
+        } else if secs >= 60 {
+            write!(f, "{}min{}s", secs / 60, secs % 60)
+        } else {
+            let secs = self.as_secs_f32();
+            write!(f, "{:.1}s", secs)
+        }
+    }
+}
+
+pub struct Compositor<'a> {
+    pub timer: timer::Timer<'a>,
+    pub loading_bar: loading::LoadingBar<'a>,
+}
+
+impl Component for Compositor<'_> {
+    fn draw_unchecked(&self, dimensions: Dimensions, mode: DrawMode) -> anyhow::Result<Lines> {
+        let mut lines = self.timer.draw_unchecked(dimensions, mode)?;
+        lines.pad_lines_left(2);
+
+        let action = self.timer.action.borrow();
+        let is_finished = action.is_finished();
+        if !action.show_progress() && !is_finished {
+            return Ok(lines);
+        }
+        let mut loading = self.loading_bar.draw_unchecked(dimensions, mode)?;
+
+        let step_len = action.step_header.len();
+        let loading_len = if is_finished {
+            action.completion_header.len()
+        } else {
+            action.loading_bar_header.unwrap_or_default().len()
+        };
+
+        let padding = if step_len >= loading_len {
+            step_len - loading_len + 2
+        } else {
+            2usize.saturating_sub(loading_len - step_len)
+        };
+        loading.pad_lines_left(padding);
+
+        lines.0.extend(loading.0);
+
+        Ok(lines)
+    }
+}
+
+impl<'a> Compositor<'a> {
+    pub fn new(action: &'a RefCell<Action>) -> Self {
+        Self {
+            timer: timer::Timer::new(action),
+            loading_bar: loading::LoadingBar::new(action),
+        }
+    }
+
+    pub fn finalize(&mut self) -> anyhow::Result<Lines> {
+        self.draw_unchecked(Dimensions::default(), DrawMode::Final)
+    }
+}