Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implementing logic #7

Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
0dbf21f
feat: implement enabling shield
Oct 30, 2024
9648ecb
fix: fixes after code review
Nov 5, 2024
6f321c7
feat: shield logic finished
Nov 7, 2024
23fccc4
fix: code review fixes
Nov 8, 2024
290fc3d
feat: add serialization logic
Nov 9, 2024
1c69b99
feat: logic finished
Nov 11, 2024
6b4abe1
feat: encryption manager refactored
Nov 11, 2024
34a9cf7
feat: add memory implementation for some managers
Nov 11, 2024
168de86
feat: finish memory implementations for all managers along with full …
Nov 12, 2024
8f975a7
fix: finish tests for shield with needed fixes (there is still proble…
Nov 12, 2024
7f1873c
feat: back to EncryptionManager using ECIES library
Nov 13, 2024
4d7fba3
fix: added hash to manifest file to verify if content is the same
Nov 13, 2024
9802e2d
feat: implemented Route53AddressManager
Nov 14, 2024
16603b6
feat: implemented SQLAlchemyMinerShieldStateManager
Nov 14, 2024
6b24602
feat: implemented S3ManifestManager
Nov 15, 2024
82e330c
feat: added integration test
Nov 15, 2024
968f045
feat: filling AwsAddressManager, work in progress
Nov 22, 2024
935c2e6
feat: added possibility of specyfing IP in AwsAddressManager
Nov 22, 2024
6ba1675
feat: filling AwsAddressManager - creating ELB is working
Nov 22, 2024
2ac5dd7
feat: filling AwsAddressManager - creating HostedZone during ELB crea…
Nov 24, 2024
097b2de
feat: filling AwsAddressManager - creating alias DNS entry in Route53
Nov 24, 2024
dc62a30
feat: finished AwsAddressManager - added creating firewall
Nov 25, 2024
c9d3b75
feat: back to passing hosted_zone_id to AwsAddressManager
Dec 5, 2024
845ec8a
fix: tiny code review fixes
Dec 6, 2024
44ebe33
Add AWSClientFactory
grzegorz-leszczynski-reef Dec 25, 2024
840941e
Use pytest fixture in TestAddressManager
grzegorz-leszczynski-reef Dec 25, 2024
54de75f
Simplify code after CR
grzegorz-leszczynski-reef Dec 25, 2024
d2bc4a4
Upgrade handling AWS API errors
grzegorz-leszczynski-reef Dec 26, 2024
030cba1
Add project dependencies
grzegorz-leszczynski-reef Dec 26, 2024
5e27248
Apply ruff linter fixes
grzegorz-leszczynski-reef Dec 26, 2024
78ad8f2
Add MinerShieldFactory
grzegorz-leszczynski-reef Dec 27, 2024
4af7d75
Implement Validator
grzegorz-leszczynski-reef Dec 31, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,25 +30,27 @@ The goal of this project is to implement a distributed and decentralized system
- Validators can request the connection information of miners from the subtensor network. This information is validated and
decrypted locally using the validator's private key.

## Communication Flow
## Basic Communication Flow

<!--
@startuml ./assets/diagrams/CommunicationFlow
participant Validator
participant Miner
participant GitHub
participant AddressManager
participant Storage
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
participant Storage
database Storage

participant Bittensor
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
participant Bittensor
database Bittensor

Validator -> Validator: Generate Validator key-pair
Validator -> GitHub: Publish public key along with HotKey
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Validator -> GitHub: Publish public key along with HotKey
Validator -> Bittensor: Publish public key along with HotKey

GitHub -> Miner: Fetch Validator info
Miner -> Miner: Encrypt Miner IP/Domain with Validator public key
Miner -> Storage: Add/update file with encrypted IPs/Domains for Validators
GitHub -> Miner: Fetch new Validator info
Miner -> AddressManager: Generate new address
Miner -> Miner: Encrypt generated address with Validator public key
Miner -> Storage: Update file with encrypted addresses for Validators
Miner -> Bittensor: Publish file location
Bittensor -> Validator: Fetch file location
Storage -> Validator: Fetch Miner file
Validator -> Validator: Decrypt Miner file entry encrypted for given Validator
Validator -> Miner: Send request using decrypted Miner IP/Domain
Validator -> Miner: Send request using decrypted Miner address
@enduml
-->

Expand Down
12 changes: 7 additions & 5 deletions assets/diagrams/CommunicationFlow.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
15 changes: 9 additions & 6 deletions bt_ddos_shield/address.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,24 +29,27 @@ def __init__(self, address_id, address_type: AddressType, address: str, port: in
self.address = address
self.port = port

def __repr__(self):
return f"Address(id={self.address_id}, type={self.address_type}, address={self.address}:{self.port})"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return f"Address(id={self.address_id}, type={self.address_type}, address={self.address}:{self.port})"
return f"Address(id={self.address_id}, address={self.address}:{self.port})"


@abstractmethod
def encrypt(self) -> bytes:
def serialize(self) -> bytes:
"""
Encrypts address data.
Serialize address data.

Returns:
bytes: Encrypted address data.
bytes: Serialized address data.
"""
pass

@classmethod
@abstractmethod
def decrypt(cls, encrypted_data: bytes) -> Address:
def deserialize(cls, serialized_data: bytes) -> 'Address':
"""
Create address from encrypted address data.
Deserialize address data.

Args:
encrypted_data: Encrypted address data.
serialized_data: Data serialized before by serialize method.

Returns:
Address: Created address.
Expand Down
18 changes: 17 additions & 1 deletion bt_ddos_shield/blockchain_manager.py
Original file line number Diff line number Diff line change
@@ -1,17 +1,33 @@
from abc import ABC, abstractmethod

from bt_ddos_shield.utils import Hotkey


class AbstractBlockchainManager(ABC):
"""
Abstract base class for manager handling publishing blocks to blockchain.
"""

@abstractmethod
def publish(self, data: bytes):
def put(self, hotkey: Hotkey, data: bytes):
"""
Puts data to blockchain.

Args:
hotkey: Hotkey of user for whom we are putting data.
data: Data.
"""
pass

@abstractmethod
def get(self, hotkey: Hotkey) -> bytes:
"""
Gets data from blockchain.

Args:
hotkey: Hotkey of user to get data from.

Returns:
data: Last block of data put by user.
"""
pass
20 changes: 17 additions & 3 deletions bt_ddos_shield/event_processor.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
from abc import ABC, abstractmethod
from dataclasses import dataclass
import traceback


@dataclass
class Event:
class MinerShieldEvent:
"""
Class describing event, which happened in the shield.
"""
Expand All @@ -12,17 +13,30 @@ class Event:
exception: Exception = None # Exception which caused the event.


class AbstractEventProcessor(ABC):
class AbstractMinerShieldEventProcessor(ABC):
"""
Abstract base class for processor handling events generated by shield.
"""

@abstractmethod
def add_event(self, event: Event):
def add_event(self, event: MinerShieldEvent):
"""
Add new event to be handled by processor.

Args:
event: Event to add.
"""
pass


class PrintingMinerShieldEventProcessor(AbstractMinerShieldEventProcessor):
"""
Event processor which logs events to console.
"""

def add_event(self, event: MinerShieldEvent):
if event.exception is not None:
print(f"MinerShieldEvent: {event.event_description}\nException happened:")
print(traceback.format_exc())
else:
print(f"MinerShieldEvent: {event.event_description}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This prints, not logs, but I also think it is generally assembled wrong and won't work. No worries, you'll fix it when you start running it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is class mostly (and maybe only) for unit tests. But I change name to PrintingMinerShieldEventProcessor, because probably in the future there will be LoggingMinerShieldEventProcessor. Also it works - just run unit test.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And if we ever print that, we'd rather use stderr.
traceback.print_exc() can be a sufficient routine for that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed it already in future commits. But now I'm printing exception captured inside MinerShieldEvent. Probably not, but there is possibility, that it wouldn't be last exception thrown and print_exc calls sys.exception() inside to get the last one.

4 changes: 2 additions & 2 deletions bt_ddos_shield/manifest_manager.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from abc import ABC, abstractmethod

from bt_ddos_shield.address import Address
from bt_ddos_shield.miner_shield import Hotkey
from bt_ddos_shield.utils import Hotkey


class AbstractManifestManager(ABC):
andreea-popescu-reef marked this conversation as resolved.
Show resolved Hide resolved
Expand Down Expand Up @@ -31,6 +31,6 @@ def _put_file(self, data: bytes) -> Address:
data: File contents.

Returns:
Address: Address where file is put.
Address: Address where file was put.
"""
pass
Loading