Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial draft of the registry developer guide #38

Merged
merged 23 commits into from
Feb 17, 2025
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b4c2237
initial draft of the registry developer guide
alexcasalboni Feb 5, 2025
be26706
Merge branch 'main' into registry
alexcasalboni Feb 5, 2025
eb0e45e
fix typo
alexcasalboni Feb 5, 2025
32024d4
fix some minor typos
alexcasalboni Feb 5, 2025
fd28977
feat: add first set of registry FAQs
alexcasalboni Feb 5, 2025
bfa81e2
update brew install command
alexcasalboni Feb 6, 2025
01cba24
fix 2 typos
alexcasalboni Feb 10, 2025
7b50625
Merge branch 'main' into registry
alexcasalboni Feb 10, 2025
31ca9c4
Update CLI help output (to be updated once final)
alexcasalboni Feb 10, 2025
49609db
chore: update deps
alexcasalboni Feb 10, 2025
ddb4eab
move foundational concepts to its own page, add guide sub-sections, u…
alexcasalboni Feb 10, 2025
4aab256
chore: update deps
alexcasalboni Feb 11, 2025
9991a1d
feat: update CLI output, add reference implementations
alexcasalboni Feb 11, 2025
8c1f999
proofreading and improving flow
alexcasalboni Feb 12, 2025
5a36a3d
add manifest file section, add after-push instructions (archive/delet…
alexcasalboni Feb 12, 2025
40ab980
add curl/brew installation comments and udpate hints
alexcasalboni Feb 12, 2025
d77f891
add beta disclaimers everywhere, add push guided UX, clarify public/p…
alexcasalboni Feb 13, 2025
9c8333c
some typos, add test events sub-section
alexcasalboni Feb 13, 2025
9378aff
rename folder to just registry, update website links
alexcasalboni Feb 14, 2025
508225a
Merge branch 'main' into registry
alexcasalboni Feb 14, 2025
0044598
minor copyediting and fix broken links
alexcasalboni Feb 17, 2025
92203e5
another round of copyediting and minor updates
alexcasalboni Feb 17, 2025
9160d16
Add links to new GitHub repositories in OSS page
alexcasalboni Feb 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions mint.json
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,16 @@
"group": "Services",
"pages": [
"services/overview",
{
"group": "Component Registry",
"icon": "rectangle-history-circle-plus",
"iconType": "solid",
"pages": [
"services/component-registry/overview",
"services/component-registry/developer-guide",
"services/component-registry/faqs"
]
},
{
"group": "Data Collection",
"icon": "chart-simple",
Expand Down
204 changes: 204 additions & 0 deletions services/component-registry/developer-guide.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,204 @@
---
title: Developer Guide
description: Step-by-step guide for component developers
icon: rectangle-terminal
---

import EdgeeSdk from '/snippets/edgee-sdk.mdx';

<EdgeeSdk />

Welcome to Edgee! This guide will cover how to create a new component locally,
compile it to WebAssembly, and push it to the Edgee Component Registry.

## Foundational concepts
The following concepts are a useful starting points before you start coding.

#### Reverse proxy
Edgee acts as a [reverse proxy](https://en.wikipedia.org/wiki/Reverse_proxy) in front of a website, intercepting HTTP requests and running business
logic on top of edge networks and CDNs such as Fastly and Cloudflare; the main proxy interacts with WebAssembly components
to implement features such as data collection for analytics, warehousing, and attribution data.

#### WebAssembly and language support
[WebAssembly (Wasm in short)](https://webassembly.org/) allows developers to implement a component in a programming language such as Rust or JavaScript
and compile it to a binary file; each component runs in a sandboxed environment with no access to network or filesystem (by default)
and implements a specific interface that the Edgee proxy will use to invoke a component's business logic.

You can use WebAssembly as compile target with [many different languages](https://component-model.bytecodealliance.org/language-support.html)
and the resulting binary file is language-agnostic and cross-platform. Today Edgee supports and provides tooling for
Rust, Go, Python, JavaScript, C#, and C. Even though the local development experience with the Edgee CLI is quite similar across languages,
it's worth noting that compiled languages such as Rust, Go, and C result in pretty small and efficient Wasm files (within hundreds of KBs),
while interpreted languages such as Python and JavaScript result in much heavier Wasm files (20MB+).

#### Component types and WIT
Today Edgee supports only one component type: `data_collection`. That includes components
for web analytics such as
[Google Analytics](/components/data-collection/google-analytics) or [Amplitude](/components/data-collection/amplitude),
for attribution/conversion such as
[Meta Conversions API](/components/data-collection/meta-capi) or [LinkedIn Conversions API](/components/data-collection/linkedin-capi),
and for data warehousing destinations such as Amazon S3 or Amazon Firehose.

We're quickly expanding to new component types such as `consent_mapping`
to integrate with CMPs (Consent Management Platforms) and `ab_testing` to integrate with A/B testing frameworks and services at the edge.

In Wasm, interfaces are defined with [WIT (Wasm Interface Type)](https://component-model.bytecodealliance.org/design/wit.html),
a definition language to define types and world. You can find [Edgee's WIT files on GitHub](https://github.com/edgee-cloud/edgee-wit).
The convention is to store them in each component project under a `wit/` folder and with a `wit/deps.toml` file to define your WIT dependencies
Under the hood, the Edgee CLI is using [wit-deps](https://github.com/bytecodealliance/wit-deps) to keep the WIT dependencies up-to-date. When a new WIT file is released,
simply update the `wit/deps.toml` file with the new version and `wit-deps` will update the WIT dependencies. Depending on the programming language,
you might need to re-generate the WIT bindings for your code to run and compile properly.

#### Edgee CLI and APIs
You can use the [Edgee CLI](https://github.com/edgee-cloud/edgee) to create and build a Wasm component locally with commands such as
`edgee component new` and `edgee component build`.
When the component is ready, the Edgee CLI lets you push it to the Edgee Component Registry as a public or private component
under your organization's account, with `edgee component push`. Under the hood, the CLI interacts with the
[Edgee API](/api-reference/introduction) and its goal is to simplify the local development experience across all supported languages.

## How to create a new Edgee component

#### Install and configure the Edgee CLI

First of all, you need to install the Edgee CLI:

<CodeGroup>

```bash curl
$ curl https://install.edgee.cloud | sh
```

```bash brew
$ brew tap edgee-cloud/edgee
$ brew install edgee
```

</CodeGroup>

Second, log in with your Edgee account's token (you can [create one here](https://www.edgee.cloud/me/settings/tokens)):

```bash
$ edgee login
Enter Edgee API token (press Ctrl+R to toggle input display): ****
```

Quickly confirm that the API is working properly with this token:

```bash
$ edgee whoami
Logged in as:
ID: XYZ-XYZ-DYZ
Name: Your name
Email: your@email.com
```

And if you need assistance about a CLI command:

```bash
$ edgee help
Usage: edgee [OPTIONS] <COMMAND>
Commands:
login Log in to the Edgee Console
whoami Print currently login informations
serve Run the Edgee server [aliases: server]
test TBD
check TBD
```

#### Create a new component

Next, you can create a new component project and choose your preferred programming language.
This will create a new local folder with the sample code and tooling you need to get started quickly.

```bash
$ edgee component create
Choose a name: my-new-component
Choose a language from the sample projects below: example-rust-component
```

Now you can enter the new folder and start implementing your business logic.

In short, a data collection component needs to implement this WIT interface:

```text
page: func(e: event, settings: dict) -> result<edgee-request, string>;
track: func(e: event, settings: dict) -> result<edgee-request, string>;
user: func(e: event, settings:dict) -> result<edgee-request, string>;
```

The corresponding method is invoked when a new `page`, `track` or `user` event is sent for data collection
by the Edgee proxy. These methods receive the incoming event object and a set of settings, and the expected
output is an `edgee-request` object, which looks like this:

```text
record edgee-request {
method: http-method,
url: string,
headers: dict,
forward-client-headers: bool,
body: string,
}
```

<Note>
Note: an Edgee component doesn't perform any HTTP call on its own because it runs in a sandboxed environment with no access
to network or file system. The inteded purpose of a component is to instruct the proxy on which HTTP request to perform
to fullfil its data collection duties.
</Note>

For example, the Google Analytics component returns an `edgee-request` object that points to the official
`https://www.google-analytics.com/g/collect` endpoint, with the correct querystring parameters.
Similarly, the Amazon S3 component points to `https://{bucket}.s3.amazonaws.com` with the correct sigv4 authentication headers.

In order to make your Edgee component reusable by other developers and organizations via the Edgee Component Registry,
you can define a set of input `settings`, containing all the API credentials or dynamic
information you need to craft the correct `edgee-request` object. For example, the Google Analytics component only expects a
`ga_measurement_id` setting, while the Segment component expects `segment_project_id` and `segment_write_key`.

You can configure the expected settings for your component in the local manifest file (TODO add more details about this).

#### Build and test locally

When the implementation is ready, you can build your component locally using the CLI:

```bash
$ edgee component build
```

The output of this command will be a new .wasm file in the current folder.

You can also customize the behavior of the build command in the manifest file, such as changing the target file name
and the default build script (TODO add more details about this).

Before pushing your component to the Edgee Component Registry, it's highly recommended to validate and test the .wasm file locally:

```bash
# validate local .wasm file
$ edgee component check

# run the Wasm file with test events
$ edgee component test \
--event-type page \
--settings "setting1=value1,setting1=value2"
```

The test command will run your local .wasm file with a sample event and the provided settings.
If no `--event-type` is provided, it will run the .wasm file with all event types (page, track, and user).
This is particularly useful to make sure your component is behaving as expected from the proxy point of view, in addition to your unit tests.


#### Push the component

When you're ready to publish your component, it's time to push it using the Edgee CLI:

```bash
$ edgee component push
```

The component will be pushed to the registry under your Edgee Organization.
In case your Edgee user is part of multiple Organizations, you'll be prompted to choose one.

Congratulations, your component is now public on `https://www.edgee.cloud/{organization}/{component}`.

In case you prefer to keep your component private, the CLI will ask you to confirm the component's visibility during the push command
(TODO add more details about this)

59 changes: 59 additions & 0 deletions services/component-registry/faqs.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
title: FAQs
description: Here's everything you need to know to get the most out of the Edgee Component Registry as a component developer.
icon: message-question
---

import EdgeeSdk from '/snippets/edgee-sdk.mdx';

<EdgeeSdk />


<AccordionGroup>
<Accordion title="What is an edge component??">
An edge component is a simple piece of business logic that runs at the edge, just before an HTML page is served to a visitor's browser.
Depending on the page content, a component may invoke 3rd-party APIs, edit the HTML response, or manipulate HTTP headers.
</Accordion>

<Accordion title="What tooling is available for local development and testing?">
You can use the Edgee CLI to build and publish new components from your laptop.
The Edgee CLI also acts as the reverse proxy you can use to serve and test components locally before pushing to the registry.
</Accordion>

<Accordion title="What programming languages can I use to develop a new component?">
You can use all programming languages that compile to Wasm, with WASI and component support,
such as C, C#, Go, JavaScript, Python, and Rust.
</Accordion>

<Accordion title="How do I keep a component private? And how do I make it public?">
When creating and pushing a component for the first time, it will be set to private by default.
You can easily make it public when pushing a new component to the registry, either via CLI or using the web console.
Please keep in mind that public components cannot be unpublished (only deleted), so make sure you're 100% comfortable with
your component being publicly visible when pushing.
</Accordion>

<Accordion title="Does my public component need to be open source?">
All Edgee-native components are public and open source and we'd love to encourage all component developers to keep their
projects open source as well. This allows you to build trust and encourages other developers to get involved with bugfixing
and other maintenance tasks. That said, we understand there might be special cases where organizations are more comfortable with
publishing closed-source components and the Edgee platforms supports that too.
</Accordion>

<Accordion title="Do I need an Edgee account to create and publish components?">
Yes, components are bound to your Edgee account or your organization's Edgee account.
On the other hand, Edgee Open Source Edition users can use the Edgee Component Registry to pull and add public components
to their proxy configuration as well.
</Accordion>

<Accordion title="Can I use the Edgee Component Registry for free?">
Yes, the Edgee Component Registry is free of charge.
With Edgee's managed service, you only get charged for the events and requests generated by your projects.
</Accordion>

<Accordion title="How do I use components published by other developers in my Edgee project?">
You can easily add a component to your projects from a public component's page, simply click “Add to my project”.
Alternatively, you can visit your project's dashboard and visit the “Registry” tab to list all public and private components available to you.
Last but not least, you could use the [Edgee API](/api-reference/introduction) to automate every step, from project creation to component integration.
</Accordion>

</AccordionGroup>
39 changes: 39 additions & 0 deletions services/component-registry/overview.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
---
title: Registry Overview
description: Build and publish WebAssembly components to the edge
icon: chart-mixed
---

import EdgeeSdk from '/snippets/edgee-sdk.mdx';

<EdgeeSdk />


The Edgee Component Registry lets developers publish, share, and reuse components powered by WebAssembly on Edgee.

In addition to built-in components for popular analytics APIs such as Google Analytics, now you can build and
publish custom components to the Edgee Component Registry and use them in your Edgee project, or even share it publicly.

Think of this as the "Docker Hub for Wasm components".

## How does it work?

There are many ways to interact with the Edgee Component Registry, depending on your use case. Let's review some common examples:

1. You are an open source developer and want to create a new **public component** for everybody to use in their Edgee project.
2. You are a organization with first-party APIs or special requirements and want to create a new **private component** for you or your
customers to use in your Edgee project.
3. You are an **Edgee Open Source Edition** user and want to pull **public components** from the registry into your Edgee deployment.

In all cases, the typical workflow consists in using the Edgee open source tools to develop, build, push,
or pull components from/to the registry.

For example, imagine you are a large organization and want to use Edgee for your first-party analytics solutions or to integrate
with additional tools and APIs not natively supported by Edgee. You'd use the Edgee CLI to create a new component locally,
push it to the registry as a private component, and add it to your Edgee projects without sharing it with anyone.

On the other hand, imagine you are an Edgee partner or API provider and want to make your offering available to all Edgee customers.
You'd use the Edgee CLI to create a new component locally, push it to the registry as a public component under your organization's account,
and enable developers to add your component to their Edgee projects seamlessly.

If you want to dive deeper into the technical details, check out the [Developer Guide](./developer-guide) and [FAQs](./faqs).