Add Taking Snapshot document (#31)

* Add Taking Snapshot document * Update format * Update MDs, L2 Token Airdrop -> Migration Airdrop * Update according to PR comments * Add description to MD * Updated format and the excluded-address part
LiskHQ · May 8, 2024 · cf46b97 · cf46b97
1 parent c9e216e
commit cf46b97
Show file tree

Hide file tree

Showing 3 changed files with 179 additions and 14 deletions.
diff --git a/README.md b/README.md
@@ -54,12 +54,21 @@ docker build -t lisk-claim-backend -f ./docker/claim-backend/Dockerfile .
 ## Documentation
 
 - [Tech Design](./documentation/Tech_Design.md) - A comprehensive document, including:
+
   - Project Background
   - Requirement of the project
   - Components of the project
   - Technical specification
   - Design
 
+- [Taking Snapshot](./documentation/Taking_Snapshot.md) - A detailed documentation for:
+
+  - Obtaining Lisk v4 Snapshot
+  - Data preparation for Merkle Tree Generation
+  - Generate Merkle Tree for
+    - Lisk Token Migration
+    - Migration Airdrop
+
 ## Contributing
 
 If you find any issues or have suggestions for improvements,

diff --git a/documentation/Taking_Snapshot.md b/documentation/Taking_Snapshot.md
@@ -0,0 +1,97 @@
+# Taking Snapshot & Merkle Tree Generation
+
+## Background
+
+This document focuses on converting a Lisk Snapshot to Merkle Tree using `tree-builder`.
+The Merkle Trees are used for the Token Migration and future airdrops.
+
+## Merkle Trees
+
+There are 2 planned Merkle Trees for Lisk:
+
+1. **Lisk Token Migration** - The main Merkle Tree to let users migrate their LSK Token from Lisk L1 to LSK L2 Network (L2 Chain).
+2. **Migration Airdrop** - A Merkle Tree that rewards LSK holders for migrating from Lisk L1 to Lisk L2
+
+## Pre-Requisite
+
+- Node v18
+- yarn
+
+## Preparation
+
+1. Download and extract blockchain data of block [24,823,618](https://snapshots.lisk.com/mainnet/blockchain-24823618.db.tar.gz) from [Lisk Snapshots](https://snapshots.lisk.com/mainnet/). Blockchain data will be available soon after the block height has reached.
+
+   ```
+   # Estimated Available Date: 21 May, 2024 (0800 CET)
+   curl https://snapshots.lisk.com/mainnet/blockchain-24823618.db.tar.gz -o ./blockchain.db.tar.gz
+
+   # OR download the latest snapshot available
+   # curl https://snapshots.lisk.com/mainnet/blockchain.db.tar.gz -o ./blockchain.db.tar.gz
+
+   tar -zxvf ./blockchain.db.tar.gz
+   ```
+
+2. Clone and install `lisk-token-claim`.
+   ```
+   git clone git@github.com:LiskHQ/lisk-token-claim.git && cd lisk-token-claim
+   yarn && yarn build
+   ```
+3. Navigate to `tree-builder`.
+   ```
+   cd packages/tree-builder
+   ```
+
+## Generate Merkle Tree
+
+After running the following command, `accounts.json`, `merkle-root.json`, `merkle-tree-result.json` and `merkle-tree-result-detailed.json` will be generated.
+The descriptions of the above files can be found at [Tech Design](./Tech_Design.md) and [Tree Builder README](../packages/tree-builder/README.md).
+
+### Lisk Token Migration
+
+Generate merkle tree for Lisk Token Migration.
+
+| Flag                    | Description                                                                                                                                                                  | Required | Default            |
+| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ------------------ |
+| db-path                 | Database path, where your state.db is located. If following the preparation stage above, db-path would be `../../../`                                                        | True     |                    |
+| output-path             | Destination path of the merkle tree                                                                                                                                          | False    | `./data`           |
+| token-id                | Token ID, use default for mainnet LSK Token                                                                                                                                  | False    | `0000000000000000` |
+| excluded-addresses-path | File Path of List of addresses excluded from airdrop. Exact addresses to be exlucded from Migration has been stored in `lisk-token-claim/data/mainnet/exclude_addresses.txt` | False    | `""`               |
+
+```
+# Create a separate folder to store Merkle Tree for Migration
+mkdir -p ./migration
+
+./bin/run.js generate-merkle-tree \
+--db-path=../../../ \
+--output-path=./migration \
+--token-id=0000000000000000 \
+--excluded-addresses-path=../../data/mainnet/exclude_addresses.txt
+```
+
+### Migration Airdrop
+
+Generate merkle tree for Migration Airdrop.
+
+| Flag                    | Description                                                                                                                                                                        | Required | Default            |
+| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ------------------ |
+| db-path                 | Database path, where your state.db is located                                                                                                                                      | True     |                    |
+| output-path             | Destination path of the merkle tree                                                                                                                                                | False    | `./data`           |
+| token-id                | Token ID, use default for mainnet LSK Token                                                                                                                                        | False    | `0000000000000000` |
+| cutoff                  | Minimal amount of LSK required to participate in the migration airdrop                                                                                                             | False    | `50`               |
+| whale-cap               | Cap on the LSK amount of a single Lisk L1 account to be used for the airdrop computation                                                                                           | False    | `250000`           |
+| airdrop-percent         | The airdrop amount is equal to the given percentage of LSK balance, after whale cap and cutoff are applied                                                                         | False    | `10`               |
+| excluded-addresses-path | File Path of List of addresses excluded from airdrop. Exact addresses to be exlucded from Airdrop has been stored in `lisk-token-claim/data/mainnet/exclude_airdrop_addresses.txt` | False    | `""`               |
+
+```
+# Create a separate folder to store Merkle Tree for Airdrop
+mkdir -p ./airdrop-migration
+
+./bin/run.js generate-airdrop-merkle-tree \
+--db-path=../../../ \
+--output-path=./airdrop-migration \
+--token-id=0000000000000000 \
+--cutoff 50 \
+--whale-cap 250000 \
+--airdrop-percent 10 \
+--excluded-addresses-path=../../data/mainnet/excluded_airdrop_addresses.txt
+```
diff --git a/packages/tree-builder/README.md b/packages/tree-builder/README.md
@@ -5,34 +5,63 @@ This library is a command-line service that builds a Merkle tree from a snapshot
 ## Run
 
 ```
-$ cd packages/tree-builder
-$ ./bin/run.js generate-merkle-tree --db-path <value> [--output-path <value>] [--token-id <value>]
+cd packages/tree-builder
+
+# Lisk Token Migration
+./bin/run.js generate-merkle-tree --db-path <value> [--output-path <value>] [--token-id <value>] [excluded-addresses-path <value>]
+
+# Migration Airdrop
+./bin/run.js generate-airdrop-merkle-tree --db-path <value> [--output-path <value>] [--token-id <value>] [--cutoff <value>] [--whale-cap <value>] [--airdrop-percent <value>] [--excluded-addresses-path <value>]
 ```
 
 ## Files
 
-| Name                                               | Description                                                                                                                                                                           | Generated By                                              |
-| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------- |
-| `/data/<network>/accounts.json`                    | Stores addresses, balances, and multisig details(If any) per account after a snapshot is taken, addresses must be sorted in ascending order. Will be used for MerkleTree computation. | Snapshot                                                  |
-| `/data/<network>/merkle-tree-result-detailed.json` | Stores MerkleRoot, and leaves for each account. Will be used for examination by 3rd Party or public, also used by Claim Backend API.                                                  | `$ ./bin/run.js generate-merkle-tree --network=<network>` |
-| `/data/<network>/merkle-tree-result.json`          | Stores MerkleRoot, and leaves for each account. A lightweight version of `merkle-tree-result-detailed.json`. Will be used for testing of Claim Contract.                              | `$ ./bin/run.js generate-merkle-tree --network=<network>` |
-| `/data/<network>/merkle-root.json`                 | Stores MerkleRoot only. Will be used for deployment of Claim Contract.                                                                                                                | `$ ./bin/run.js generate-merkle-tree --network=<network>` |
+| Name                                               | Description                                                                                                                                                                           | Generated By                                            |
+| -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------- |
+| `/data/<network>/accounts.json`                    | Stores addresses, balances, and multisig details(If any) per account after a snapshot is taken, addresses must be sorted in ascending order. Will be used for MerkleTree computation. | Snapshot                                                |
+| `/data/<network>/merkle-tree-result-detailed.json` | Stores MerkleRoot, and leaves for each account. Will be used for examination by 3rd Party or public, also used by Claim Backend API.                                                  | `./bin/run.js generate-merkle-tree --network=<network>` |
+| `/data/<network>/merkle-tree-result.json`          | Stores MerkleRoot, and leaves for each account. A lightweight version of `merkle-tree-result-detailed.json`. Will be used for testing of Claim Contract.                              | `./bin/run.js generate-merkle-tree --network=<network>` |
+| `/data/<network>/merkle-root.json`                 | Stores MerkleRoot only. Will be used for deployment of Claim Contract.                                                                                                                | `./bin/run.js generate-merkle-tree --network=<network>` |
 
 ## Merkle Leaf
 
-Each leaf will be encoded as ABI-format, in the following order:
+Each leaf will be encoded as ABI-format, structure of Merkle Tree may vary and depends on usage
+
+### Lisk Token Migration
 
 ```
 LSK_ADDRESS_IN_HEX: bytes20
 BALANCE_IN_BEDDOWS: uint64
 NUMBER_OF_SIGNATURES: uint32
 MANDATORY_KEYS: bytes32[]
 OPTIONAL_KEYS: bytes32[]
+```
+
+If the address is not a multisig address, `NUMBER_OF_SIGNATURES` would be `0,` `MANDATORY_KEYS` and `OPTIONAL_KEYS` be `[]`
+
+### Migration Airdrop
+
+```
+LSK_ADDRESS_IN_HEX: bytes20
+BALANCE_IN_WEI: uint256
+```
+
+Note that Balance is represented in Wei(2\*\*18) and in `uint256` format.
+
+## Params
 
-P.S. If the address is not a multisig address, NUMBER_OF_SIGNATURES would be 0, MANDATORY_KEYS and OPTIONAL_KEYS be []
+For both `Lisk Token Migration` and `Migration Airdrop`, a `merkle-root.json` will be generated.
+
+```
+merkle-root.json:
+{
+  merkleRoot: string;
+}
 ```
 
-### Params
+For other files, refer to the table below:
+
+### Lisk Token Migration
 
 ```
 accounts.json:
@@ -74,13 +103,43 @@ merkle-tree-result.json:
   }[];
 }
 # `address` is a reserved in solidity, hence `b32Address` here
+```
 
-merkle-root.json:
+### Migration Airdrop
+
+```
+accounts.json:
+{
+    lskAddress: string;
+    balanceWei: number;
+}
+
+merkle-tree-result-detailed.json:
+{
+    merkleRoot: string;
+    leaves: {
+        lskAddress: string;
+        address: string;
+        balanceWei: number;
+        hash: string;
+        proof: string[];
+    }[];
+}
+
+merkle-tree-result.json:
 {
   merkleRoot: string;
+  leaves: {
+    b32Address: string;
+    balanceWei: number;
+    proof: string[];
+  }[];
 }
+```
 
-# Only used at example
+### # Only used at example
+
+```
 signatures.json:
 {
   message: string;
@@ -95,7 +154,7 @@ signatures.json:
 ## _Demo/Testing Purpose Only_
 
 ```
-$ ./bin/run.js example [--amountOfLeaves <value>] [--recipient <value>]
+./bin/run.js example [--amountOfLeaves <value>] [--recipient <value>]
 
 # FLAGS
 #   --amountOfLeaves=<value>  [default: 100] Amount of leaves in the tree