generated from opentensor/bittensor-subnet-template
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
16 changed files
with
110 additions
and
2,077 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,17 +1,70 @@ | ||
|
||
<div align="center"> | ||
|
||
# Storage Subnet | ||
``` | ||
█████████ █████ █████ | ||
███░░░░░███ ░░███ ░░███ | ||
░███ ░░░ ███████ ██████ ████████ ░███████ | ||
░░█████████ ░░░███░ ███░░███░░███░░███ ░███░░███ | ||
░░░░░░░░███ ░███ ░███ ░███ ░███ ░░░ ░███ ░███ | ||
███ ░███ ░███ ███░███ ░███ ░███ ░███ ░███ | ||
░░█████████ ░░█████ ░░██████ █████ ████████ | ||
░░░░░░░░░ ░░░░░ ░░░░░░ ░░░░░ ░░░░░░░░ | ||
The Decentralized Object Storage Subnet | ||
``` | ||
|
||
</div> | ||
|
||
## Get Started | ||
Storb is a decentralized object storage subnet built on the Bittensor network. It aims to be a distributed, fault-tolerant, and efficient digital storage solution. | ||
|
||
```sh | ||
# Initialise repo | ||
uv venv --python 3.12 | ||
uv sync | ||
``` | ||
## Features | ||
|
||
- **Decentralization**: Utilizes a network of nodes to store data redundantly. | ||
|
||
- **Erasure Coding**: Enhances data reliability and storage efficiency by fragmenting and distributing data across multiple nodes, allowing reconstruction even if some fragments are lost. | ||
|
||
- **Incentivized Storage**: Storb leverages the power of the Bittensor. The subnet rewards miners for contributing reliable and responsive storage resources, with validators ensuring data integrity. Bittensor serves as *the* ideal incentive layer for this. | ||
|
||
For an overview of how the subnet works [see here](docs/overview.md) | ||
|
||
## Installation | ||
|
||
Follow these steps to set up a miner or validator node: | ||
|
||
1. **Clone the Repository**: | ||
|
||
```bash | ||
git clone https://github.com/fr34kcoders/storb.git | ||
cd storb | ||
``` | ||
|
||
w. **Set Up Virtual Environment**: | ||
|
||
Ensure you have [`uv` installed](https://docs.astral.sh/uv/getting-started/installation/). Once you do, run the following commands: | ||
|
||
```bash | ||
uv venv --python 3.12 | ||
source ./venv/bin/activate | ||
uv sync | ||
uv pip install -e . | ||
``` | ||
|
||
3. **Configure and run Node**: | ||
- [**Miner**](docs/miner.md) | ||
- [**Validator**](docs/validator.md) | ||
|
||
|
||
## Contributing | ||
|
||
We welcome contributions to enhance Storb. Please fork the repository and submit a pull request with your improvements. | ||
|
||
## License | ||
|
||
This repository is licensed under the [MIT License](./LICENSE). | ||
This project is licensed under the MIT License. See the [LICENSE](https://github.com/fr34kcoders/storb/blob/main/LICENSE) file for details. | ||
|
||
## Contact | ||
|
||
For questions or support, please open an issue in this repository or contact the maintainers on the Bittensor discord server. | ||
|
||
--- |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# Running a Miner | ||
|
||
Miners are the backbone of Storb. They are responsible for storing and serving pieces (of files) to validators, and by proxy to the end user. | ||
|
||
### Testnet | ||
#### Running | ||
``` | ||
python neurons/miner.py --netuid 269 --subtensor.network test --wallet.name MINER_WALLET --wallet.hotkey MINER_HOTKEY --logging.debug --axon.port AXON_PORT | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
# Storb - An Overview | ||
|
||
# TLDR | ||
Basically, the subnet can be used as shown below: | ||
![overview](../assets/overview.png) | ||
|
||
# Scoring Mechanism | ||
![scoring](../assets/weight-scoring.png) | ||
Scoring is made up of multiple components: | ||
- **Latency**: Miners are scored based on how quickly they respond to storage and retrieval requests. | ||
- **Response Rate**: Reliable miners are the name of the game. The less a miner responds to storage and retrieval requests the lower it is scored. | ||
- **PDP Challenge**: The subnet uses an efficient proof of data possession technique inspired by [Ateniese et al.](https://dl.acm.org/doi/10.1145/1315245.1315318) to ensure miners actually store the data they are asked to keep. | ||
|
||
Please read [neurons/validator.py](../neurons/validator.py) for more information on how the scoring mechanism works. | ||
|
||
# Chunking and Piecing | ||
Files are split into erasure-coded chunks, and subsequently split into pieces and stored across various miners for redundancy | ||
![chunk](../assets/chunk.png) | ||
|
||
# DHT for File Metadata | ||
File metadata - which is useful for querying miners for pieces, and, eventually, reconstructing files - is replicated and stored across neurons in the subnet in the form of a DHT. | ||
![metadata](../assets/metadata.png) |
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.