Skip to content

Commit

Permalink
feat: updated docs + README
Browse files Browse the repository at this point in the history
  • Loading branch information
Shr1ftyy committed Dec 24, 2024
1 parent d739015 commit 1d5492d
Show file tree
Hide file tree
Showing 16 changed files with 110 additions and 2,077 deletions.
69 changes: 61 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,70 @@

<div align="center">

# Storage Subnet
```
█████████ █████ █████
███░░░░░███ ░░███ ░░███
░███ ░░░ ███████ ██████ ████████ ░███████
░░█████████ ░░░███░ ███░░███░░███░░███ ░███░░███
░░░░░░░░███ ░███ ░███ ░███ ░███ ░░░ ░███ ░███
███ ░███ ░███ ███░███ ░███ ░███ ░███ ░███
░░█████████ ░░█████ ░░██████ █████ ████████
░░░░░░░░░ ░░░░░ ░░░░░░ ░░░░░ ░░░░░░░░
The Decentralized Object Storage Subnet
```

</div>

## Get Started
Storb is a decentralized object storage subnet built on the Bittensor network. It aims to be a distributed, fault-tolerant, and efficient digital storage solution.

```sh
# Initialise repo
uv venv --python 3.12
uv sync
```
## Features

- **Decentralization**: Utilizes a network of nodes to store data redundantly.

- **Erasure Coding**: Enhances data reliability and storage efficiency by fragmenting and distributing data across multiple nodes, allowing reconstruction even if some fragments are lost.

- **Incentivized Storage**: Storb leverages the power of the Bittensor. The subnet rewards miners for contributing reliable and responsive storage resources, with validators ensuring data integrity. Bittensor serves as *the* ideal incentive layer for this.

For an overview of how the subnet works [see here](docs/overview.md)

## Installation

Follow these steps to set up a miner or validator node:

1. **Clone the Repository**:

```bash
git clone https://github.com/fr34kcoders/storb.git
cd storb
```

w. **Set Up Virtual Environment**:

Ensure you have [`uv` installed](https://docs.astral.sh/uv/getting-started/installation/). Once you do, run the following commands:

```bash
uv venv --python 3.12
source ./venv/bin/activate
uv sync
uv pip install -e .
```

3. **Configure and run Node**:
- [**Miner**](docs/miner.md)
- [**Validator**](docs/validator.md)


## Contributing

We welcome contributions to enhance Storb. Please fork the repository and submit a pull request with your improvements.

## License

This repository is licensed under the [MIT License](./LICENSE).
This project is licensed under the MIT License. See the [LICENSE](https://github.com/fr34kcoders/storb/blob/main/LICENSE) file for details.

## Contact

For questions or support, please open an issue in this repository or contact the maintainers on the Bittensor discord server.

---
Binary file added assets/chunk.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/metadata.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/overview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added assets/weight-scoring.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
9 changes: 9 additions & 0 deletions docs/miner.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Running a Miner

Miners are the backbone of Storb. They are responsible for storing and serving pieces (of files) to validators, and by proxy to the end user.

### Testnet
#### Running
```
python neurons/miner.py --netuid 269 --subtensor.network test --wallet.name MINER_WALLET --wallet.hotkey MINER_HOTKEY --logging.debug --axon.port AXON_PORT
```
22 changes: 22 additions & 0 deletions docs/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Storb - An Overview

# TLDR
Basically, the subnet can be used as shown below:
![overview](../assets/overview.png)

# Scoring Mechanism
![scoring](../assets/weight-scoring.png)
Scoring is made up of multiple components:
- **Latency**: Miners are scored based on how quickly they respond to storage and retrieval requests.
- **Response Rate**: Reliable miners are the name of the game. The less a miner responds to storage and retrieval requests the lower it is scored.
- **PDP Challenge**: The subnet uses an efficient proof of data possession technique inspired by [Ateniese et al.](https://dl.acm.org/doi/10.1145/1315245.1315318) to ensure miners actually store the data they are asked to keep.

Please read [neurons/validator.py](../neurons/validator.py) for more information on how the scoring mechanism works.

# Chunking and Piecing
Files are split into erasure-coded chunks, and subsequently split into pieces and stored across various miners for redundancy
![chunk](../assets/chunk.png)

# DHT for File Metadata
File metadata - which is useful for querying miners for pieces, and, eventually, reconstructing files - is replicated and stored across neurons in the subnet in the form of a DHT.
![metadata](../assets/metadata.png)
244 changes: 0 additions & 244 deletions docs/running_on_mainnet.md

This file was deleted.

Loading

0 comments on commit 1d5492d

Please sign in to comment.