Add Code Linter Action for push and pull requests (#20)

* Add super-linter github action and fix existing issues

* fix linter issues for dotenv-linter

* Update Dockerfile and hadolint config file

* Fix markdownlint issues and remove natural language linting

* fix pylint and flake8 linting issues

* Disable validate all after resolving all lint issues

https://github.com/dting/staketaxcsv/actions/runs/1680632730

.github/linters/.hadolint.yaml

@@ -0,0 +1,6 @@
+---
+##########################
+## Hadolint config file ##
+##########################
+ignored:
+  - DL3008

.github/linters/.isort.cfg

@@ -0,0 +1,4 @@
+[settings]
+profile=black
+default_section=FIRSTPARTY
+line_length=120

.github/workflows/linter.yml

@@ -0,0 +1,44 @@
+#################################
+#################################
+## Super Linter GitHub Actions ##
+#################################
+#################################
+name: Lint Code Base
+
+on:
+  push:
+  pull_request:
+    branches: [main]
+
+jobs:
+  build:
+    name: Lint Code Base
+    # Set the agent to run on
+    runs-on: ubuntu-latest
+
+    ##################
+    # Load all steps #
+    ##################
+    steps:
+      ##########################
+      # Checkout the code base #
+      ##########################
+      - name: Checkout Code
+        uses: actions/checkout@v2
+        with:
+          # Full git history is needed to get a proper list of changed files within `super-linter`
+          fetch-depth: 0
+
+      ################################
+      # Run Linter against code base #
+      ################################
+      - name: Lint Code Base
+        uses: github/super-linter@v4
+        env:
+          VALIDATE_ALL_CODEBASE: false
+          VALIDATE_JSCPD: false
+          VALIDATE_NATURAL_LANGUAGE: false
+          VALIDATE_PYTHON_BLACK: false
+          VALIDATE_PYTHON_MYPY: false
+          DEFAULT_BRANCH: main
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.isort.cfg

@@ -0,0 +1,4 @@
+[settings]
+profile=black
+default_section=FIRSTPARTY
+line_length=120

Dockerfile

@@ -3,14 +3,15 @@ FROM python:3.9.9-slim
 
 ENV APPHOME=/staketaxcsv
 
-RUN apt-get update
-RUN apt-get install -y curl bzip2
-RUN mkdir -p ${APPHOME}
+RUN apt-get update && apt-get install --no-install-recommends -y curl bzip2 \
+    && apt-get clean \
+    && rm -rf /var/lib/apt/lists/* \
+    && mkdir -p ${APPHOME}
 
 # install pip
 WORKDIR $APPHOME
 COPY ./requirements.txt .
-RUN pip3 install -r requirements.txt
+RUN pip3 install --no-cache-dir -r requirements.txt
 
 # copy rest
 COPY . .

README.md

@@ -1,66 +1,67 @@
 
 # staketaxcsv
 
-  * Python repo to create blockchain CSVs for Terra (LUNA), Solana (SOL), Cosmos (ATOM),
-    and Osmosis (OSMO) blockchains.
-  * CSV codebase for https://stake.tax
-  * Community contribution and PRs are most welcome, especially to fix/support new types of 
-    protocols/transactions.
+* Python repository to create blockchain CSVs for Terra (LUNA), Solana (SOL), Cosmos (ATOM),
+  and Osmosis (OSMO) blockchains.
+* CSV codebase for <https://stake.tax>
+* Community contribution and PRs are most welcome, especially to fix/support new types of
+  protocols/transactions.
 
 # Usage
 
-  * Same arguments apply for report_terra.py (LUNA), report_sol.py (SOL), report_atom.py (ATOM),
-    report_osmo.py (OSMO):
-    ```
-    # Load environment variables from sample.env (add to ~/.bash_profile or ~/.bashrc to avoid doing every time)
-    set -o allexport
-    source sample.env
-    set +o allexport
-    
-    cd src
-    
-    # Create default CSV
-    python3 report_terra.py <wallet_address>
-    
-    # Create all CSV formats (i.e. koinly, cointracking, etc.)
-    python3 report_terra.py <wallet_address> --format all
-    
-    # Show CSV result for single transaction (great for development/debugging)
-    python3 report_terra.py <wallet_address> --txid <txid>
-    ```
-    
+* Same arguments apply for report_terra.py (LUNA), report_sol.py (SOL), report_atom.py (ATOM),
+  report_osmo.py (OSMO):
+
+  ```sh
+  # Load environment variables from sample.env (add to ~/.bash_profile or ~/.bashrc to avoid doing every time)
+  set -o allexport
+  source sample.env
+  set +o allexport
+
+  cd src
+
+  # Create default CSV
+  python3 report_terra.py <wallet_address>
+
+  # Create all CSV formats (i.e. koinly, cointracking, etc.)
+  python3 report_terra.py <wallet_address> --format all
+
+  # Show CSV result for single transaction (great for development/debugging)
+  python3 report_terra.py <wallet_address> --txid <txid>
+  ```
+
 # Install
 
   1. Install python 3.9 ([one way](README_reference.md#installing-python-39-on-macos))
   2. Install pip packages
-     ```
+
+     ```sh
      pip3 install -r requirements.txt
      ```
 
 # Docker
 
 See [Docker](README_reference.md#docker) to alternatively install/run in docker container.
 
-
 # Contributing Code
 
-  * Code style follows `pep8`.  This can be tested with `pycodestyle`.
-  * Providing a sample txid will expedite a pull request (email support@stake.tax, 
-    DM @staketax, etc.):
-    ```
-    # For a given txid, your PR (most commonly) should print different output before/after:
-    python3 report_terra.py <wallet_address> --txid <txid>
-    ```
+* Code style follows `pep8`.  This can be tested with `pycodestyle`.
+* Providing a sample txid will expedite a pull request (email support@stake.tax,
+  DM @staketax, etc.):
+
+  ```sh
+  # For a given txid, your PR (most commonly) should print different output before/after:
+  python3 report_terra.py <wallet_address> --txid <txid>
+  ```
 
 # Reference
 
 See [README_reference.md](README_reference.md):
 
-  * [Code Style](README_reference.md#code-style)
-  * [Unit Tests](README_reference.md#unit-tests)
-  * [Docker](README_reference.md#docker)
-  * [Ideal Configuration](README_reference.md#ideal-configuration)
-    * [RPC Node Settings](README_reference.md#rpc-node-settings)
-    * [DB Cache](README_reference.md#db-cache)
-  * [Installing python 3.9.9 on MacOS](README_reference.md#installing-python-39-on-macos)
-  
+* [Code Style](README_reference.md#code-style)
+* [Unit Tests](README_reference.md#unit-tests)
+* [Docker](README_reference.md#docker)
+* [Ideal Configuration](README_reference.md#ideal-configuration)
+  * [RPC Node Settings](README_reference.md#rpc-node-settings)
+  * [DB Cache](README_reference.md#db-cache)
+* [Installing python 3.9.9 on macOS](README_reference.md#installing-python-39-on-macos)

README_reference.md

@@ -1,21 +1,22 @@
 
 # Reference Notes
 
-  * Random notes, hopefully helpful on occasion.  Probably not helpful on first look.
+* Random notes, hopefully helpful on occasion.  Probably not helpful on first look.
 
-
 # Code Style
-  * pep8
-  * To run python code style check:
-    ```
-    # (some configuration in setup.cfg)
-    pycodestyle
-    ```
+
+* pep8
+* To run python code style check:
+
+  ```sh
+  # (some configuration in setup.cfg)
+  pycodestyle
+  ```
 
 # Unit Tests
 
 You may notice a lack of unit tests in this codebase.  Though tests exist, I omitted them because they rely on extensive
-use of real world wallet data.  For the sake of all users' privacy, I do not include these tests.  I'm open to 
+use of real world wallet data.  For the sake of all users' privacy, I do not include these tests.  I'm open to
 ideas for alternatives, since obviously this is non-optimal.
 
 # Docker
@@ -32,7 +33,6 @@ docker run --platform linux/amd64 -it --volume $PWD:/staketaxcsv staketaxcsv bas
 # See README usage section to run script(s)
 ```
 
-
 # Ideal Configuration
 
 Default code was made to work out of the box.  These are changes that require manual
@@ -41,46 +41,48 @@ default version.
 
 ## RPC Node settings
 
-  * Default `sample.env` points to public RPC nodes.  This generally works, up to a point.
-  * Edit/uncomment `sample.env` to change to point to more reliable private RPC node(s).
-    * Examples for private RPC nodes (Figment, Quicknode) are included.
+* Default `sample.env` points to public RPC nodes.  This generally works, up to a point.
+* Edit/uncomment `sample.env` to change to point to more reliable private RPC node(s).
+  * Examples for private RPC nodes (Figment, Quicknode) are included.
 
 ## DB Cache
 
 Use of a database for caching is ideal to speed up certain RPC queries (especially SOL).  Here is
 the script usage to enable caching:
 
-  ```
-  cd src
-  
-  # --cache flag requires working implementation of Cache class (common/cache.py)
-  python3 report_terra.py <wallet_address> --cache
-  ```
+```sh
+cd src
+
+# --cache flag requires working implementation of Cache class (common/cache.py)
+python3 report_terra.py <wallet_address> --cache
+```
 
 To enable --cache, you must configure an aws connection for the boto3 code found in src/common/Cache.py:
-  * One method: `aws configure`
-    * See here to install AWS CLI: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
-    * See here to use `aws configure`: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-quickstart.html
-
+
+* One method: `aws configure`
+  * See here to install AWS CLI: <https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html>
+  * See here to use `aws configure`: <https://docs.aws.amazon.com/cli/latest/userguide/getting-started-quickstart.html>
+
 Alternatively, you may implement your own Cache class (common/cache.py).
 
-# Installing python 3.9 on MacOS
-
-  * Personal method--google is probably better 
-
-    ```
-    # Install brew (see https://brew.sh/)
-    
-    # python 3.9.9
-    brew install openssl readline sqlite3 xz zlib
-    brew install pyenv
-    pyenv install 3.9.9
-    
-    # use virtualenv
-    brew install virtualenv
-    virtualenv -p ~/.pyenv/versions/3.9.9/bin/python3.9 env
-    source env/bin/activate
-    
-    # install pip packages (same as README.md)
-    pip3 install -r requirements.txt
-    ```
+# Installing python 3.9 on macOS
+
+* Personal method--google is probably better
+
+  ```sh
+  # Install brew (see https://brew.sh/)
+
+  # python 3.9.9
+  brew install openssl readline sqlite3 xz zlib
+  brew install pyenv
+  pyenv install 3.9.9
+
+  # use virtualenv
+  brew install virtualenv
+  virtualenv -p ~/.pyenv/versions/3.9.9/bin/python3.9 env
+  source env/bin/activate
+
+  # install pip packages (same as README.md)
+  pip3 install -r requirements.txt
+  ```
+

sample.env

@@ -12,8 +12,7 @@ TERRA_LCD_NODE=https://lcd.terra.dev  # this public LCD node may have rate-limit
 #TERRA_LCD_NODE=https://columbus-5--lcd--full.datahub.figment.io/apikey/<TERRA_FIGMENT_KEY>
 
 ### OPTIONAL ##################################################################################################
-TERRA_FIGMENT_KEY=""
-
+# TERRA_FIGMENT_KEY=<key>
 
 ### OTHER #####################################################################################################
 STAGE=local

src/atom/ProgressAtom.py

@@ -1,6 +1,6 @@
-
 import logging
 import time
+
 from atom.config_atom import localconfig
 
 SECONDS_PER_PAGE = 15.0

src/atom/api_lcd.py

@@ -1,9 +1,10 @@
-
 import logging
-import time
 import math
+import time
+
 import requests
 from settings_csv import ATOM_NODE
+
 LIMIT = 50
 
 

src/atom/make_tx.py

@@ -1,7 +1,5 @@
-
-
-from common.make_tx import make_transfer_out_tx, make_transfer_in_tx, make_reward_tx
 from atom.constants import CUR_ATOM
+from common.make_tx import make_reward_tx, make_transfer_in_tx, make_transfer_out_tx
 
 
 def make_transfer_receive_tx(txinfo, received_amount, received_currency=None):