Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Master #17

Merged
merged 65 commits into from
Dec 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
d06f99d
Add cm api client to dependencies, pin python 3.11
Dec 3, 2024
8378e07
Update pre commit branch name check
Dec 3, 2024
9973fdb
Add file for release notes
Dec 3, 2024
e2e6783
Read version from toml file
Dec 3, 2024
8cd804b
Test package version, adds layer to prevent accidental versioning
Dec 3, 2024
1fe713e
Add License
Dec 3, 2024
4120ea0
Add basic badges. Not functional since repo is private
Dec 3, 2024
442f832
Merge pull request #5 from coinmetrics/3-finish-code-skeleton
hscott-yuma Dec 3, 2024
e967889
Merge pull request #4 from coinmetrics/1-cm-api-dependency
hscott-yuma Dec 3, 2024
2d1f497
First update to cm_utils. Added ReferenceRateUSD
umocm Dec 3, 2024
7dd0a43
Trivial Addition to cm_utils
umocm Dec 3, 2024
c84f4e9
Take 3 Testing Pull Request rule enforcement
umocm Dec 3, 2024
bae5ce6
First attempt at CI workflow
Dec 3, 2024
d67f02e
Merge branch 'dev' into 2-github-actions
Dec 3, 2024
6cb85d3
Run pre commit on all files
Dec 3, 2024
506ba3c
Merge pull request #9 from coinmetrics/cm_utils_update_2
hscott-yuma Dec 4, 2024
87c3f56
Merge pull request #8 from coinmetrics/2-github-actions
hscott-yuma Dec 4, 2024
4559373
Run pre-commit on this file
Dec 4, 2024
f4a8d05
Merge pull request #11 from coinmetrics/10-fix-pipeline
Dec 4, 2024
6ecdd83
organized parameters in Makefile
hscott-yuma Dec 5, 2024
c1b016c
moved to more advanced loop handler
hscott-yuma Dec 5, 2024
3d87eb6
--network flag not appropriate for the parser
hscott-yuma Dec 5, 2024
8b428e0
--network flag not appropriate for the parser
hscott-yuma Dec 5, 2024
7df5c8f
missing arg in loop handler
hscott-yuma Dec 5, 2024
121d418
reduced vpermit limit to 2 in makefile
hscott-yuma Dec 5, 2024
b16f89d
unnecessary asyncio lock blocks runtime
hscott-yuma Dec 5, 2024
4b68f67
linters
hscott-yuma Dec 5, 2024
b8cb6ba
invalid ip error on weight setting
hscott-yuma Dec 5, 2024
c34f9f2
moved from chain_endpoint to network
hscott-yuma Dec 5, 2024
a497a27
changed how network is handled in config
hscott-yuma Dec 5, 2024
9313a46
typo
hscott-yuma Dec 5, 2024
603c9fa
reverted some changes
hscott-yuma Dec 5, 2024
6aa5ed1
fixed Makefile error
hscott-yuma Dec 5, 2024
64c486a
added try block for when node-query fails
hscott-yuma Dec 5, 2024
db6e44d
missing network info from subtensor call
hscott-yuma Dec 5, 2024
3b08678
set block update to before weight setting code in set_weights loop
hscott-yuma Dec 5, 2024
fff5587
added reinitializing subtensor to resync metagraph code
hscott-yuma Dec 5, 2024
781080f
reduced set_weights and resync_metagraph cadence
hscott-yuma Dec 5, 2024
2d616c3
trying to catch and restart websocket closed exceptions
hscott-yuma Dec 6, 2024
8390195
Make iso8601 string format compatible with CM API
Dec 6, 2024
02bf5f9
Implement naive base miner
Dec 6, 2024
813cc6c
added return_exceptions to asyncio gather
hscott-yuma Dec 6, 2024
c9677f1
moved error handling to weight_setter
hscott-yuma Dec 6, 2024
0b3467e
removed node_query function
hscott-yuma Dec 6, 2024
ca2b7b3
typo
hscott-yuma Dec 6, 2024
4a4cb8c
simplified block handling code
hscott-yuma Dec 6, 2024
ed937b4
now updates current block on calls to set_weights
hscott-yuma Dec 6, 2024
9573841
resets blocks_since_last_update on successful set weights
hscott-yuma Dec 6, 2024
2908dea
now pulls weights_rate_limit from subnet hyperparams instead of setti…
hscott-yuma Dec 6, 2024
855cd94
typo
hscott-yuma Dec 6, 2024
790e359
missed assignment in utils function
hscott-yuma Dec 6, 2024
756fda1
unused variable removed
hscott-yuma Dec 6, 2024
64d4841
added retry wrapper to get_block calls in set_weights
hscott-yuma Dec 6, 2024
d4c58ec
added delay functionality to func_with_retry
hscott-yuma Dec 6, 2024
ef5f00b
missing variable
hscott-yuma Dec 6, 2024
189062b
changed print statements to bt.logging statements
hscott-yuma Dec 6, 2024
6942445
Merge pull request #14 from coinmetrics/12-dummy-base-miner
hscott-yuma Dec 6, 2024
63e284c
Merge pull request #13 from coinmetrics/Update-Makefile-for-testnet
Dec 6, 2024
1bdf196
initial api integration into reward.py function
hscott-yuma Dec 9, 2024
cf13fc1
turned base miner into a forward function for importing
hscott-yuma Dec 9, 2024
3d21396
fixed iso_to_datetime function to handle z formatted strings
hscott-yuma Dec 9, 2024
52e3901
added new function to handle cm-specific timestamp formatting
hscott-yuma Dec 9, 2024
c0dd328
changed challenge prediction field from list to float
hscott-yuma Dec 9, 2024
0189c99
Merge pull request #15 from coinmetrics/6-cm-api-integration
Dec 9, 2024
77e85b7
Merge pull request #16 from coinmetrics/dev
hscott-yuma Dec 9, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
name: Cache Build

on:
workflow_call:
inputs:
command:
required: true
type: string
name:
required: true
type: string

jobs:
reusable-build:
name: ${{ inputs.name }}
runs-on: ubuntu-latest
steps:

#------------------------------------------------
# check-out repo and set-up python
#------------------------------------------------
- name: Check out repository
uses: actions/checkout@v4
- name: Set up python
uses: actions/setup-python@v5
with:
python-version: '3.11'

#------------------------------------------------
# ----- install & configure poetry -----
#------------------------------------------------
- name: Load cached Poetry installation
id: cached-poetry
uses: actions/cache@v4
with:
path: ~/.local # the path depends on the OS
key: poetry-0 # increment to reset cache

- name: Install Poetry
if: steps.cached-poetry.outputs.cache-hit != 'true'
uses: snok/install-poetry@v1
with:
version: latest
virtualenvs-create: true
virtualenvs-in-project: true
virtualenvs-path: .venv
installer-parallel: true

# If cache was loaded, we must redo configuration
- name: Configure poetry
if: steps.cached-poetry.outputs.cache-hit == 'true'
run: |
poetry config virtualenvs.create true
poetry config virtualenvs.in-project true
poetry config virtualenvs.path .venv

#------------------------------------------------
# Load cached venv if exists
#------------------------------------------------
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v4
with:
path: .venv
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ hashFiles('**/poetry.lock') }}

#------------------------------------------------
# Install dependencies if cache does not exist
#------------------------------------------------
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root

#------------------------------------------------
# Install your root project
#------------------------------------------------
- name: Install project
run: poetry install --no-interaction

#------------------------------------------------
# Run custom command(s) within venv
#------------------------------------------------
- name: Run commands
run: ${{ inputs.command }}
71 changes: 71 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
name: CI

on:
pull_request:
push:
branches: [master, staging, dev]

jobs:
#----------------------------------------------
# Build Environment
#----------------------------------------------
build:
name: Build
uses: ./.github/workflows/build.yml
with:
name: Cache
command: |
poetry run python -m pip list
poetry run python --version
poetry --version
poetry run echo "Build successful"

#----------------------------------------------
# Run Linters
#----------------------------------------------
lint-black:
name: Linter
needs: build
uses: ./.github/workflows/build.yml
with:
name: Black
command: poetry run python -m black --check .
lint-isort:
name: Linter
needs: build
uses: ./.github/workflows/build.yml
with:
name: Isort
command: poetry run python -m isort --check-only .
lint-mypy:
name: Linter
needs: build
if: false # This condition ensures the job is never executed
uses: ./.github/workflows/build.yml
with:
name: Mypy
command: poetry run python -m mypy --verbose 0 .
lint-flake8:
name: Linter
needs: build
uses: ./.github/workflows/build.yml
with:
name: Flake8
command: poetry run python -m flake8 .

#----------------------------------------------
# Run Tests
#----------------------------------------------
test-unittest:
name: Tests
needs: [
lint-black,
lint-isort,
lint-mypy,
lint-flake8,
]
if: ${{ always() }} # will run the tests regardless of linting success
uses: ./.github/workflows/build.yml
with:
name: Unittests
command: poetry run pytest tests/
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ repos:
language: system
pass_filenames: false
always_run: true
args: [--branch, main, --branch, staging, --branch, dev]
args: [--branch, master, --branch, staging, --branch, dev]
- id: trailing-whitespace
name: trim trailing whitespace
description: trims trailing whitespace.
Expand Down
1 change: 1 addition & 0 deletions .python-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
3.11
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2023 Opentensor

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
55 changes: 18 additions & 37 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,7 +1,19 @@
network = ws://127.0.0.1:9944
netuid = 1
## Network Parameters ##
finney = wss://entrypoint-finney.opentensor.ai:443
testnet = wss://test.finney.opentensor.ai:443
locanet = ws://127.0.0.1:9944

testnet_netuid = 256
localnet_netuid = 1
logging_level = trace # options= ['info', 'debug', 'trace']
coldkey = cm-owner

netuid = $(testnet_netuid)
network = $(testnet)

## User Parameters
coldkey = default
validator_hotkey = validator
miner_hotkey = miner

metagraph:
btcli subnet metagraph --netuid $(netuid) --subtensor.chain_endpoint $(network)
Expand All @@ -16,32 +28,23 @@ validator:
python start_validator.py \
--neuron.name validator \
--wallet.name $(coldkey) \
--wallet.hotkey validator \
--wallet.hotkey $(validator_hotkey) \
--subtensor.chain_endpoint $(network) \
--axon.port 30335 \
--netuid $(netuid) \
--logging.level $(logging_level)

validator2:
python start_validator.py \
--neuron.name validator2 \
--wallet.name $(coldkey) \
--wallet.hotkey validator2 \
--subtensor.chain_endpoint $(network) \
--axon.port 30339 \
--netuid $(netuid) \
--logging.level $(logging_level)

miner:
python start_miner.py \
--neuron.name miner \
--wallet.name $(coldkey) \
--wallet.hotkey miner \
--wallet.hotkey $(miner_hotkey) \
--subtensor.chain_endpoint $(network) \
--axon.port 30336 \
--netuid $(netuid) \
--logging.level $(logging_level) \
--timeout 16 \
--vpermit_tao_limit 2 \
--forward_function forward

miner2:
Expand All @@ -55,25 +58,3 @@ miner2:
--logging.level $(logging_level) \
--timeout 16 \
--forward_function forward_bad

miner3:
python start_miner.py \
--neuron.name miner3 \
--wallet.name $(coldkey) \
--wallet.hotkey miner3 \
--subtensor.chain_endpoint $(network) \
--axon.port 30338 \
--netuid $(netuid) \
--logging.level $(logging_level) \
--timeout 16 \
--forward_function forward

setup_local:
btcli wallet faucet --wallet.name $(coldkey) --subtensor.chain_endpoint $(network) ;\
btcli subnet create --wallet.name $(coldkey) --subtensor.chain_endpoint $(network) ;\
btcli subnet register \
--wallet.name $(coldkey) \
--wallet.hotkey validator \
--netuid $(netuid)
--subtensor.chain_endpoint $(network) ;\
btcli stake add --wallet.name $(coldkey) --wallet.hotkey validator --amount 1024 ;\
25 changes: 14 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@

| | |
| :-: | :-: |
Badges here?
| **Status** | <img src="https://img.shields.io/github/v/release/coinmetrics/precog?label=Release" height="25"/> <img src="https://img.shields.io/github/actions/workflow/status/coinmetrics/precog/ci.yml?label=Build" height="25"/> <br> <a href="https://github.com/pre-commit/pre-commit" target="_blank"> <img src="https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white&label=Pre-Commit" height="25"/> </a> <a href="https://github.com/psf/black" target="_blank"> <img src="https://img.shields.io/badge/code%20style-black-000000.svg?label=Code%20Style" height="25"/> </a> <br> <img src="https://img.shields.io/github/license/coinmetrics/precog?label=License" height="25"/> |
| **Activity** | <img src="https://img.shields.io/github/commit-activity/m/coinmetrics/precog?label=Commit%20Activity" height="25"/> <img src="https://img.shields.io/github/commits-since/coinmetrics/precog/latest/dev?label=Commits%20Since%20Latest%20Release" height="25"/> <br> <img src="https://img.shields.io/github/release-date/coinmetrics/precog?label=Latest%20Release%20Date" height="25"/> <img src="https://img.shields.io/github/last-commit/coinmetrics/precog/dev?label=Last%20Commit" height="25"/> <br> <img src="https://img.shields.io/github/contributors/coinmetrics/precog?label=Contributors" height="25"/> |
| **Compatibility** | <img src="https://img.shields.io/badge/dynamic/toml?url=https%3A%2F%2Fraw.githubusercontent.com%2Fcoinmetrics%2Fprecog%2Frefs%2Fheads%2Fdev%2Fpyproject.toml&query=%24.tool.poetry.dependencies.python&logo=python&label=Python&logoColor=yellow" height="25"/> <img src="https://img.shields.io/badge/dynamic/toml?url=https%3A%2F%2Fraw.githubusercontent.com%2Fcoinmetrics%2Fprecog%2Frefs%2Fheads%2Fdev%2Fpyproject.toml&query=%24.tool.poetry.dependencies.bittensor&prefix=v&label=Bittensor" height="25"/> <br> <img src="https://img.shields.io/badge/dynamic/toml?url=https%3A%2F%2Fraw.githubusercontent.com%2Fcoinmetrics%2Fprecog%2Frefs%2Fheads%2Fdev%2Fpyproject.toml&query=%24.tool.poetry.dependencies.coinmetrics-api-client&prefix=v&label=coinmetrics-api-client" height="25"/> |
| **Social** | <a href="" target="_blank"> <img src="https://img.shields.io/website?url=https%3A%2F%2Fcharts.coinmetrics.io%2Fcrypto-data%2F&up_message=CoinMetrics&label=Website" height="25"/> </a> |


</div>
Expand All @@ -20,8 +23,8 @@ Badges here?
- [Running a Miner](#running-a-miner)
- [Running a Validator](#running-a-validator)
- [About the Rewards Mechanism](#about-the-rewards-mechanism)
- [Compute Requirements](#compute-requirements)
- [Roadmap](#roadmap)
- [Compute Requirements](#compute-requirements)
- [License](#license)

---
Expand Down Expand Up @@ -56,7 +59,7 @@ cd precog

Create and source a python virtual environment:
```
python3 -m venv
python3 -m venv
source .venv/bin/activate
```

Expand All @@ -75,23 +78,23 @@ Start by editing the Makefile with you wallet and network information.

### Running a Miner
TODO: write this \
Base miner:
Base miner:
1. Run the command:
```
make miner
```
make miner
```

Custom miner:
1. Write a custom forward function stored in precog/miners/your_function.py
1. Write a custom forward function stored in precog/miners/your_function.py
- This function should handle how the miner responds to requests from the validator
- Within the function, synapse.predictions and synapse.interval should be set.
- See [forward.py](https://github.com/coinmetrics/precog/blob/master/precog/miners/forward.py) for an example
- See [forward.py](https://github.com/coinmetrics/precog/blob/master/precog/miners/forward.py) for an example
2. Add a command to Makefile.
- copy the miner command and rename it (e.g. miner_custom) in Makefile
- replace the --forward_function argument with your_function
3. Run the Command:
```
make miner_custom
```
make miner_custom
```


Expand Down Expand Up @@ -119,7 +122,7 @@ TODO: update these
This repository is licensed under the MIT License.
```text
# The MIT License (MIT)
# Copyright © 2024 Foundry Digital LLC
# Copyright © 2024 CoinMetrics LLC

# Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated
# documentation files (the “Software”), to deal in the Software without restriction, including without limitation
Expand Down
8 changes: 8 additions & 0 deletions docs/Release Notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
Release Notes
=============

1.0.0
-----
Release on
- ...
- ...
Loading
Loading