-
Notifications
You must be signed in to change notification settings - Fork 8
Golem Unlimited Roadmap
Currently, Golem Unlimited supports running:
- Native binaries (Linux, macOS, Windows)
- Docker images.
Golem Unlimited Provider should contain a QEMU-based tool called gVMkit for running arbitrary images containing applications that need distributed computing power.
Most computers in typical home and office networks run on Windows or on macOS, where Docker is a large download. A QEMU-based gVMkit would be an order of magnitude smaller and run at a similar speed (this was tested by us). Even on Linux, Docker is a larger download.
Amazon Firecracker is a lightweight virtualization technology, but it requires Intel CPUs (with planned Arm and AMD support) and Linux as a host OS; QEMU runs on Windows PCs and on Macs.
| Binary Files | Docker Runtime | gVMkit | Firecracker | |
|---|---|---|---|---|
| Runtime Size | Run Natively | Large | Small | Small |
| Memory Usage | Native | Memory Shared Between Instances | Each Instance Allocates Separate Memory | Each Instance Allocates Separate Memory |
| Runtime Speed | Native | Typically Nearly Native | Typically Nearly Native | Typically Nearly Native |
| Supports Popular OSes and CPUs | Yes, Separate Binaries | Yes | Yes | Only Linux and Intel CPUs |
Golem Unlimited Provider should support authentication using keys. An app should authenticate itself using an API call before starting a new session and before uploading an image.
Authenticated apps can have multiple permissions. They can be authorized to run selected images on selected providers. Golem Unlimited Hub should keep and verify the permissions.
| App Permission | Description |
|---|---|
| Allow To Run On Providers | A list of provider nodes that can be added to the app session |
| Allow Images | A list of a hashes (e.g. SHA-3) of allowed images |
| Allow Images Signed By | A list of public keys of third parties that can provide allowed images |
Sometimes it is necessary to start a new high-priority task while providers are running some low-priority task. The hub should stop or suspend low-priority task until the high-priority tasks are completed.
The web user interface of Golem Unlimited Hub should display information (disk and CPU usage, running tasks etc.) related to the nodes that are connected to it.
Golem Unlimited Hub should have a new API that returns usage statistics for Golem Unlimited Providers that are connected to the hub. The user can use this information e.g. to add only unused providers to the session.
Specification:
| URL | Returned Value |
|---|---|
| /peers/{nodeId}/inf/storage | { gib = ..., } |
| /peers/{nodeId}/inf/cpu | { architecture = "x86", bit = [32,64], cores = ..., threads = ... } |
| /peers/{nodeId}/inf/mem | { gib = ..., total = ... } |
| /peers/{nodeId}/inf/os | { name = ... } |
Golem Unlimited Provider should allow creating virtual network containing all provider nodes, so that they can communicate with each other even if their external IP addresses change.
If gVMkit is used, all input and output from the runtime environment should be routed to other nodes in the virtual network. The process running the VM should only see this virtual network.
QEMU has many network-related options, e.g. -netdev socket, -netdev tap, -netdev l2tpv3. More info:
https://qemu.weilnetz.de/doc/qemu-doc.html#Network-options
The -netdev socket QEMU option with mcast= param can be used to create a network that connects several QEMU VMs. It uses a UDP multicast socket to create a communication bus for other VMs in the network.