Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add neuron certificate discovery #2

Open
wants to merge 208 commits into
base: staging
Choose a base branch
from
Open

Conversation

andreea-popescu-reef
Copy link
Collaborator

@andreea-popescu-reef andreea-popescu-reef commented Mar 29, 2024

Requirements for Adding, Changing, or Removing a Feature

subtensor migrated to version that handles certificate discovery

Description of the Change

Adds the ability to send neuron certificate and receive other neuron's certificates. This change will enable setting up mutual tls between neurons

serve_axon function takes an additional optional parameter certificate
if certificate is present the serve_axon_tls rpc call will be called on the subtensor and the neuron's certificate will be discoverable through the subtensor. The function will behave the same other than the possible storing of certificate.

get_neuron_certificate(netuid, neuron_uid) will fetch from the subtensor a specific neuron's certificate.

Alternate Designs

save the certificate as a field in the axon.
This can be implemented either:

  • on subtensor level, extend the axon to have a certificate field in both bittensor library and subtensor logic. Then, the certificate will just be passed along with the axon data in current serve_axon, get_neuron_info and other function. However this would break compatibility for multiple rpc calls encoding and would need patching.
  • on just bittensor library level, the bittensor library axon model would have a certificate field that would be stripped out on rpc calls to the subtensor, and similarly added as needed from separate rpc calls to the subtensor. However this obscures the axon's real data and adds complexity.

Possible Drawbacks

none?

Verification Process

TODO

Release Notes

  • tls certificates can now be passed via the serve_axon function and fetched via the get_neuron_certificate function

@andreea-popescu-reef andreea-popescu-reef force-pushed the encrypt branch 3 times, most recently from ecf52d3 to 308a886 Compare April 4, 2024 13:22
@andreea-popescu-reef andreea-popescu-reef changed the title wip add neuron certificate discovery Apr 4, 2024
@@ -30,6 +30,8 @@
U16_MAX = 65535
U64_MAX = 18446744073709551615

Certificate = str

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it be bytes?

@@ -44,6 +47,8 @@ def serve_extrinsic(
Endpoint host port i.e., ``192.122.31.4``.
port (int):
Endpoint port number i.e., ``9221``.
certificate (str):
Certificate.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs way more text

@@ -30,6 +30,8 @@
U16_MAX = 65535
U64_MAX = 18446744073709551615

Certificate = str

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can make a new type that inherits from str but is not str, so that mypy or something will be able to tell the difference between str and Certificate(str)

Comment on lines 1435 to 1922
if call_params["certificate"] is None:
del call_params["certificate"]
call_function = "serve_axon"
else:
call_function = "serve_axon_tls"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels like a hack. I'm not sure about this, lets ask OTF

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it was needed for backwards compatibility. A new method was added to substrate, instead of modifying the existing one.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's sad that the placeholders they left in case of stuff like this can't be used https://github.com/backend-developers-ltd/subtensor-fork/blob/development/pallets/subtensor/src/serving.rs#L30

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to validate that the provided certificate length is correct before calling substrate.

@andreea-popescu-reef andreea-popescu-reef force-pushed the encrypt branch 2 times, most recently from 3bed339 to 467bcf9 Compare November 20, 2024 10:18
andreea-popescu-reef and others added 25 commits November 20, 2024 21:42
…ests-for-async-registration

Async unittests for `bittensor/core/extrinsics/async_registration.py`
# Conflicts:
#	bittensor/core/extrinsics/async_registration.py
#	bittensor/core/extrinsics/registration.py
E2E tests - Increasing Subtensor coverage (Pt 1)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants