Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: compilation cache #43

Merged
merged 3 commits into from
Dec 5, 2023
Merged

feat: compilation cache #43

merged 3 commits into from
Dec 5, 2023

Conversation

mhmd-azeez
Copy link
Collaborator

This PR is related to extism/extism#605

PS D:\dylibso\go-sdk> go test -benchmem -run=^$ -bench .
goos: windows
goarch: amd64
pkg: github.com/extism/go-sdk
cpu: 13th Gen Intel(R) Core(TM) i7-1365U
BenchmarkInitialize/noop-12                         385           3051739 ns/op         2942236 B/op       2751 allocs/op
BenchmarkInitializeWithCache/noop-12                2112            504269 ns/op         1686493 B/op       1554 allocs/op

Without using compilation cache: 3ms
with compilation cache: 0.5ms

Since we're already exposing wazero.RuntimeConfig, I think we just need to mention it in the README

@mhmd-azeez mhmd-azeez changed the title feat: add two benchmarks for initialization feat: compilation cache Nov 29, 2023
@nilslice
Copy link
Member

Great work! I'm curious what the lift would be pre 1.0 to actually change the SDK API to make it so we don't actually expose any wazero functions or package info -- instead we add some configuration functions to our API that set thing in wazero.

Just as we don't expose anything wasmtime related in the Rust SDK.. if it's a massive change that would also be good to know.

@mhmd-azeez
Copy link
Collaborator Author

mhmd-azeez commented Nov 30, 2023

@nilslice Currently we are exposing these wazero APIs:

  • RuntimeConfig
    • WithCoreFeatures: ❌
    • WithMemoryLimitPages: ✅
    • WithMemoryCapacityFromMax: ❌
    • WithDebugInfoEnabled: ❌
    • WithCompilationCache: ❌
    • WithCustomSections: ❌
    • WithCloseOnContextDone: ❓ (If timeout is configured, this will automatically be enabled)
  • ModuleConfig
    • WithArgs: ❌
    • WithEnv: ❌
    • WithFS: ✅
    • WithFSConfig: ✅ (wazero allows readonly mounting too)
    • WithName: ✅
    • WithStartFunctions ✅ (we have our own module initialization logic)
    • WithStderr: ❌
    • WithStdin: ❌
    • WithStdout: ❌
    • WithWalltime: ❌
    • WithSysWalltime: ❌
    • WithNanotime: ❌
    • WithSysNanotime: ❌
    • WithNanosleep: ❌
    • WithOsyield: ❌
    • WithSysNanosleep: ❌
    • WithRandSource: ❌
  • api.ValueType
  • api.EncodeXX/api.DecodeXX
  • api.Memory

It all comes down to how many of these we want to implement for 1.0, maybe we can start with a minimal number of these implemented and then add them as people ask for them.

I think I can squeeze these in before December 4th:

  • Our own RuntimeConfig struct: configure caching
  • Our own ModuleConfig struct: configure WithStderr, WithStdin, WithSysWalltime, and WithStdout
  • Our own api.ValueType enum
  • Our own api.EncodeXX/api.DecodeXX functions

Let me know if there is anything else you think is important

@mhmd-azeez mhmd-azeez merged commit c14b7a3 into main Dec 5, 2023
3 checks passed
@mhmd-azeez mhmd-azeez deleted the benchmark-init branch December 5, 2023 18:56
mhmd-azeez added a commit that referenced this pull request Dec 5, 2023
…eXX (#44)

Currently we are exposing these wazero APIs:
-
[`RuntimeConfig`](https://github.com/tetratelabs/wazero/blob/d39d84505ad8021b5ba47bbc9dd55911b9b0406f/config.go#L40-L171)
-
[`ModuleConfig`](https://github.com/tetratelabs/wazero/blob/d39d84505ad8021b5ba47bbc9dd55911b9b0406f/config.go#L444-L638)
-
[`api.ValueType`](https://github.com/tetratelabs/wazero/blob/d39d84505ad8021b5ba47bbc9dd55911b9b0406f/api/wasm.go#L83-L110)
-
[`api.EncodeXX/api.DecodeXX`](https://github.com/tetratelabs/wazero/blob/d39d84505ad8021b5ba47bbc9dd55911b9b0406f/api/wasm.go#L693-L755)
-
[`api.Memory`](https://github.com/tetratelabs/wazero/blob/d39d84505ad8021b5ba47bbc9dd55911b9b0406f/api/wasm.go#L557-L673)

The idea of this PR is hide away wazero from normal usage. If the user
doesn't need any advanced configuration or functionality, then they
shouldn't have to know about Wazero. That's why we are abstracting away
`api.EncodeXX/api.DecodeXX` and `api.ValueType` since they will be used
whenever you write a host function.

`RuntimeConfig` and `ModuleConfig` are about giving the user fine grain
control of the underyling configurations. We expose some of the
functionality as part of the manifest as well, [see this for a more
detailed
breakdown](#43 (comment)).

And `api.Memory` is for write-through access of the underyling memory.

The only thing I am a bit worried about is
[`RuntimeConfig.WithCloseOnContextDone`](https://github.com/tetratelabs/wazero/blob/5796897f37852bf2042cd63959ce4ff673c4366e/config.go#L170C2-L170C24)
because by default wazero sets it to false, which might be very
confusing. We do set it to true if the user configures a timeout though.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants