Skip to content

Commit

Permalink
Merge pull request #30
Browse files Browse the repository at this point in the history 
add trace file results
  • Loading branch information
@liuyang2006
liuyang2006 authored May 8, 2021
2 parents 321276b + a0b07d2 commit 85c20b1
Show file tree
Hide file tree
Showing 4 changed files with 81 additions and 2 deletions.
2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Guppy software >= 4.2.2 from [ONT (Oxford Nanopore Technologies) website](https:

### Benchmarking reports on our HPC using [Nextflow](https://www.nextflow.io/)

We construct a benchmarking data contains reads from 800 to 8,000 reads for NA19240, and monitoring job running timeline and resource usage on our HPC, reports generated nextflow are: [Report](https://github.com/liuyangzzu/nanome/blob/master/docs/reports.pdf) and [Timeline](https://github.com/liuyangzzu/nanome/blob/master/docs/timeline.pdf).
We construct a benchmarking data contains reads from 800 to 8,000 reads for NA19240, and monitoring job running timeline and resource usage on our HPC, reports generated nextflow are: [trace file](https://github.com/liuyangzzu/nanome/blob/master/docs/nanome.pipeline_trace.txt), [Report](https://github.com/liuyangzzu/nanome/blob/master/docs/reports.pdf) and [Timeline](https://github.com/liuyangzzu/nanome/blob/master/docs/timeline.pdf).

Our HPC hardware specifications are as follows:
* CPU: Intel(R) Xeon(R) Gold 6136 CPU @ 3.00GHz
Expand Down
2 changes: 1 addition & 1 deletion docs/Usage.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,6 @@ nextflow run main.nf -profile winter \
-config conf/benchmarking.config
```

Reports runing time and memory usage by [Nextflow](https://www.nextflow.io/) utilities. Please refer to the [reports](https://github.com/liuyangzzu/nanome/blob/master/docs/reports.pdf) and [timeline](https://github.com/liuyangzzu/nanome/blob/master/docs/timeline.pdf) of benchmarking results on our HPC.
Reports runing time and memory usage by [Nextflow](https://www.nextflow.io/) utilities. Please refer to the [trace file](https://github.com/liuyangzzu/nanome/blob/master/docs/nanome.pipeline_trace.txt), [reports](https://github.com/liuyangzzu/nanome/blob/master/docs/reports.pdf) and [timeline](https://github.com/liuyangzzu/nanome/blob/master/docs/timeline.pdf) of benchmarking results on our HPC.

We will update more examples here within a short time.
79 changes: 79 additions & 0 deletions docs/nanome.pipeline_trace.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
task_id hash native_id name status exit submit duration realtime %cpu peak_rss peak_vmem rchar wchar
2 bd/061901 79597 EnvCheck (EnvCheck) COMPLETED 0 2021-04-17 18:28:29.759 34.2s 4.2s 63.6% 40.8 MB 279.8 MB 26.6 MB 9.7 KB
1 48/6c1f81 79598 UntarBenchmarking (UntarBench) COMPLETED 0 2021-04-17 18:28:29.820 2m 4s 1m 30s 81.5% 3.5 MB 246.6 MB 18.1 GB 18.5 GB
14 12/4f799c 79607 Basecall (MB800K) COMPLETED 0 2021-04-17 18:30:34.696 1m 59s 1m 39s 45.1% 910.7 MB 12.8 GB 496.5 MB 347.3 MB
3 db/712e4e 79600 Basecall (MB1600K) COMPLETED 0 2021-04-17 18:30:34.374 2m 45s 2m 28s 53.2% 984.4 MB 12.9 GB 885.4 MB 710.6 MB
22 b5/e40fd7 79617 Megalodon (MB800K) COMPLETED 0 2021-04-17 18:30:35.188 2m 49s 2m 29s 261.5% 269.3 GB 326 GB 4.1 GB 867.1 MB
4 f5/0eb645 79599 Basecall (MB2400K) COMPLETED 0 2021-04-17 18:30:34.319 3m 20s 2m 59s 62.7% 1 GB 12.9 GB 1.2 GB 1000.8 MB
5 0b/214180 79610 Megalodon (MB1600K) COMPLETED 0 2021-04-17 18:30:34.876 3m 24s 3m 7s 332.0% 274.7 GB 330.3 GB 4.9 GB 1.7 GB
6 e4/ccb41d 79609 Megalodon (MB2400K) COMPLETED 0 2021-04-17 18:30:34.834 3m 44s 3m 23s 367.8% 277.5 GB 334.5 GB 5.6 GB 2.5 GB
8 18/8dd041 79602 Basecall (MB3200K) COMPLETED 0 2021-04-17 18:30:34.468 3m 50s 3m 28s 68.7% 1.1 GB 12.9 GB 1.5 GB 1.3 GB
7 fd/bc258c 79601 Basecall (MB4000K) COMPLETED 0 2021-04-17 18:30:34.423 4m 10s 3m 51s 73.9% 1.2 GB 14.6 GB 1.8 GB 1.6 GB
15 b2/b69ceb 79611 Megalodon (MB3200K) COMPLETED 0 2021-04-17 18:30:34.912 4m 10s 3m 48s 390.8% 278.9 GB 336.1 GB 6.4 GB 3.3 GB
16 78/d243cb 79612 Megalodon (MB4000K) COMPLETED 0 2021-04-17 18:30:34.979 4m 19s 4m 454.3% 279.2 GB 335.5 GB 7.1 GB 4.1 GB
10 0c/37a680 79604 Basecall (MB4800K) COMPLETED 0 2021-04-17 18:30:34.570 4m 44s 4m 24s 85.0% 1.2 GB 13.1 GB 2.3 GB 2 GB
9 4a/002289 79603 Basecall (MB5600K) COMPLETED 0 2021-04-17 18:30:34.515 5m 9s 4m 49s 89.1% 1.3 GB 13.1 GB 2.6 GB 2.4 GB
12 c9/f4b287 79606 Basecall (MB6400K) COMPLETED 0 2021-04-17 18:30:34.654 5m 24s 5m 7s 90.3% 1.3 GB 14.7 GB 2.9 GB 2.5 GB
11 5b/8d040b 79605 Basecall (MB7200K) COMPLETED 0 2021-04-17 18:30:34.610 5m 49s 5m 29s 93.4% 1.3 GB 13.2 GB 3.2 GB 2.9 GB
13 c4/bf52d5 79608 Basecall (MB8000K) COMPLETED 0 2021-04-17 18:30:34.742 6m 9s 5m 48s 100.0% 1.4 GB 14.8 GB 3.6 GB 3.2 GB
17 75/dddfda 79613 Megalodon (MB4800K) COMPLETED 0 2021-04-17 18:30:35.025 6m 13s 5m 53s 433.4% 278.9 GB 337.8 GB 8.1 GB 5.1 GB
25 76/88fe82 79621 Nanopolish (MB800K) COMPLETED 0 2021-04-17 18:32:34.418 4m 25s 4m 3s 150.8% 11.3 GB 14.1 GB 3.4 GB 108.2 MB
19 13/e15755 79614 Megalodon (MB6400K) COMPLETED 0 2021-04-17 18:30:35.061 6m 54s 6m 37s 406.5% 281.3 GB 337.3 GB 9.4 GB 6.5 GB
18 48/ceb0e4 79615 Megalodon (MB5600K) COMPLETED 0 2021-04-17 18:30:35.108 6m 59s 6m 41s 413.2% 281.9 GB 337.9 GB 8.9 GB 5.9 GB
21 0b/721bba 79618 Megalodon (MB8000K) COMPLETED 0 2021-04-17 18:30:35.224 7m 29s 7m 12s 409.1% 282.1 GB 337 GB 11 GB 8.2 GB
23 50/b33707 79619 Resquiggle (MB800K) COMPLETED 0 2021-04-17 18:32:34.346 5m 40s 5m 20s 589.3% 198.1 GB 212.8 GB 4.7 GB 1.7 GB
20 b9/aff932 79616 Megalodon (MB7200K) COMPLETED 0 2021-04-17 18:30:35.148 7m 49s 7m 29s 401.4% 280 GB 335.4 GB 10.2 GB 7.4 GB
26 97/cf958b 79623 Nanopolish (MB1600K) COMPLETED 0 2021-04-17 18:33:19.552 7m 5s 6m 27s 136.1% 11.3 GB 14.1 GB 3.8 GB 221.2 MB
27 1c/dc2831 79622 Resquiggle (MB1600K) COMPLETED 0 2021-04-17 18:33:19.512 8m 55s 8m 22s 670.8% 201.9 GB 217.4 GB 6.5 GB 3.4 GB
29 f7/d92d41 79626 Nanopolish (MB2400K) COMPLETED 0 2021-04-17 18:33:54.853 11m 5s 10m 33s 138.9% 11.3 GB 14.1 GB 4.1 GB 310.1 MB
33 fa/6da8b5 79629 Nanopolish (MB3200K) COMPLETED 0 2021-04-17 18:34:25.162 12m 9s 11m 39s 114.7% 11.3 GB 14.1 GB 4.6 GB 415.7 MB
30 8c/371879 79625 Resquiggle (MB2400K) COMPLETED 0 2021-04-17 18:33:54.805 13m 12m 31s 691.4% 203.4 GB 219 GB 8 GB 4.9 GB
52 4f/4f5924 79649 QCStep COMPLETED 0 2021-04-17 18:36:48.156 10m 21s 253ms 11.4% 1.6 MB 110.8 MB 13.6 MB 13.5 MB
54 af/7aad68 79650 Tombo (MB800K) COMPLETED 0 2021-04-17 18:38:14.642 10m 15s 1m 9s 99.5% 2 GB 44 GB 21.1 MB 176 KB
36 27/05b308 79632 Nanopolish (MB4000K) COMPLETED 0 2021-04-17 18:34:45.206 15m 49s 15m 10s 135.5% 11.3 GB 14.1 GB 4.9 GB 505.8 MB
56 59/f39223 79652 MgldnCombine COMPLETED 0 2021-04-17 18:38:26.969 12m 27s 2.9s 79.0% 2.2 MB 224.9 MB 640.4 MB 640.3 MB
32 19/b2e35b 79628 Resquiggle (MB3200K) COMPLETED 0 2021-04-17 18:34:25.128 17m 49s 17m 16s 710.6% 204.1 GB 220.1 GB 9.7 GB 6.6 GB
37 80/efc23d 79633 Resquiggle (MB4000K) COMPLETED 0 2021-04-17 18:34:45.240 18m 24s 17m 45s 703.0% 205.5 GB 221 GB 11.2 GB 8 GB
57 7a/b5c887 79654 Tombo (MB1600K) COMPLETED 0 2021-04-17 18:42:14.593 11m 5s 1m 101.0% 3.7 GB 46.3 GB 22.2 MB 257.3 KB
40 b6/39a07c 79636 Nanopolish (MB4800K) COMPLETED 0 2021-04-17 18:35:21.361 19m 58s 18m 23s 123.2% 11.4 GB 14.2 GB 5.4 GB 635 MB
61 f4/d64c55 79657 Tombo (MB3200K) COMPLETED 0 2021-04-17 18:52:14.579 4m 35s 57.8s 99.5% 2.4 GB 47 GB 22.2 MB 276.2 KB
64 5e/5fc0a0 79659 Tombo (MB4000K) COMPLETED 0 2021-04-17 18:53:09.544 4m 50s 1m 6s 100.1% 2.4 GB 47 GB 22.6 MB 237.1 KB
60 73/b64f1e 79656 Tombo (MB2400K) COMPLETED 0 2021-04-17 18:46:54.610 11m 20s 4m 26s 376.2% 4.2 GB 225.2 GB 2.5 GB 537.1 MB
45 bc/dd244b 79640 Nanopolish (MB6400K) COMPLETED 0 2021-04-17 18:36:03.400 23m 46s 21m 51s 113.9% 12.3 GB 14.1 GB 6.1 GB 807.8 MB
39 bd/e69803 79634 Resquiggle (MB4800K) COMPLETED 0 2021-04-17 18:35:21.286 24m 43s 24m 8s 714.1% 204.5 GB 220.2 GB 13.2 GB 9.9 GB
42 83/4cab12 79638 Nanopolish (MB5600K) COMPLETED 0 2021-04-17 18:35:47.558 25m 22s 24m 14s 118.3% 12.1 GB 14.2 GB 5.9 GB 745.3 MB
65 d3/15f54a 79661 Tombo (MB4800K) COMPLETED 0 2021-04-17 19:00:04.585 1m 20s 1m 4s 100.8% 2.5 GB 47.2 GB 23 MB 278.6 KB
48 c6/6ca810 79644 Nanopolish (MB7200K) COMPLETED 0 2021-04-17 18:36:29.142 26m 10s 24m 12s 113.5% 12.7 GB 14.1 GB 6.5 GB 918.1 MB
55 82/fb15b4 79651 DeepSignal (MB800K) COMPLETED 0 2021-04-17 18:38:14.680 25m 50s 15m 25s 752.2% 26.5 GB 37.1 GB 4.4 GB 496.6 MB
43 42/5b66f3 79639 Resquiggle (MB5600K) COMPLETED 0 2021-04-17 18:35:47.589 30m 52s 29m 16s 718.0% 205.7 GB 221.9 GB 14.9 GB 11.5 GB
46 e0/c58fdd 79642 Resquiggle (MB6400K) COMPLETED 0 2021-04-17 18:36:03.467 31m 46s 29m 29s 723.6% 205.6 GB 221.7 GB 16 GB 12.6 GB
69 63/dc945c 79666 Tombo (MB6400K) COMPLETED 0 2021-04-17 19:07:49.641 1m 25s 1m 4s 100.3% 2.7 GB 47.4 GB 23.3 MB 248.2 KB
47 9e/9c5b34 79643 Resquiggle (MB7200K) COMPLETED 0 2021-04-17 18:36:29.112 36m 40s 34m 49s 724.5% 206.6 GB 221.6 GB 17.8 GB 14.4 GB
68 6e/fadbff 79664 Tombo (MB5600K) COMPLETED 0 2021-04-17 19:06:39.645 7m 10s 6m 42s 466.0% 29.8 GB 275.4 GB 5.4 GB 1.9 GB
51 77/7b3c1e 79648 Nanopolish (MB8000K) COMPLETED 0 2021-04-17 18:36:48.105 37m 51s 27m 56s 113.2% 11.9 GB 14.1 GB 7 GB 1 GB
73 b6/8b86fa 79669 NplshCombine COMPLETED 0 2021-04-17 19:14:39.623 39.9s 2.5s 71.5% 2.2 MB 224.9 MB 437.1 MB 429.1 MB
71 94/475415 79668 Tombo (MB7200K) COMPLETED 0 2021-04-17 19:13:09.654 7m 20s 7m 7s 518.7% 13.5 GB 549.4 GB 6.8 GB 3.2 GB
53 19/b1e42a 79647 Resquiggle (MB8000K) COMPLETED 0 2021-04-17 18:36:48.060 48m 11s 39m 48s 719.2% 207.3 GB 223.2 GB 19.6 GB 16.1 GB
58 1b/e770de 79653 DeepSignal (MB1600K) COMPLETED 0 2021-04-17 18:42:14.553 47m 45s 38m 37s 775.3% 7.8 GB 18 GB 5.3 GB 1006.9 MB
59 f7/94341c 79655 DeepSignal (MB2400K) COMPLETED 0 2021-04-17 18:46:54.563 43m 45s 37m 22s 766.9% 8.3 GB 19 GB 6.1 GB 1.4 GB
75 11/c8c894 79671 Tombo (MB8000K) COMPLETED 0 2021-04-17 19:24:59.678 8m 7m 48s 530.6% 14.5 GB 551.7 GB 7.6 GB 3.8 GB
76 b6/e6f322 79672 TomboCombine COMPLETED 0 2021-04-17 19:32:59.685 24.8s 916ms 15.2% 6 MB 220.2 MB 232.3 MB 232.3 MB
24 4b/322727 79620 DeepMod (MB800K) COMPLETED 0 2021-04-17 18:32:34.383 1h 1m 1h 37s 441.3% 13.7 GB 20.4 GB 9.2 GB 3.2 GB
62 ef/79cdfc 79658 DeepSignal (MB3200K) COMPLETED 0 2021-04-17 18:52:14.618 1h 3m 40s 57m 33s 784.2% 8.8 GB 19.6 GB 7.1 GB 1.9 GB
28 a3/a4d61a 79624 DeepMod (MB1600K) COMPLETED 0 2021-04-17 18:33:19.588 1h 26m 30s 1h 25m 53s 606.5% 18.9 GB 27.4 GB 18.2 GB 6.3 GB
66 27/5ae3e9 79662 DeepSignal (MB4800K) COMPLETED 0 2021-04-17 19:00:04.624 1h 17m 40s 1h 17m 23s 767.7% 9.8 GB 20.6 GB 8.9 GB 2.8 GB
63 11/c5f6e4 79660 DeepSignal (MB4000K) COMPLETED 0 2021-04-17 18:53:09.581 1h 26m 20s 1h 21m 6s 769.0% 9.2 GB 19.4 GB 7.8 GB 2.3 GB
67 ae/bb589c 79663 DeepSignal (MB5600K) COMPLETED 0 2021-04-17 19:06:39.608 1h 38m 50s 1h 38m 23s 776.4% 10.3 GB 21 GB 9.8 GB 3.3 GB
70 10/e7750b 79665 DeepSignal (MB6400K) COMPLETED 0 2021-04-17 19:07:49.600 1h 45m 20s 1h 45m 786.9% 10.7 GB 21.5 GB 10.4 GB 3.6 GB
72 ef/6dfca5 79667 DeepSignal (MB7200K) COMPLETED 0 2021-04-17 19:13:09.609 1h 48m 40s 1h 48m 30s 769.9% 11.3 GB 22.1 GB 11.5 GB 4.2 GB
31 d6/1769f5 79627 DeepMod (MB2400K) COMPLETED 0 2021-04-17 18:33:54.892 2h 42m 35s 2h 42m 8s 713.2% 27.7 GB 39 GB 27.1 GB 9.3 GB
34 69/779323 79630 DeepMod (MB3200K) COMPLETED 0 2021-04-17 18:34:25.198 2h 44m 55s 2h 44m 24s 752.4% 29.3 GB 42.5 GB 36.2 GB 12.5 GB
74 e3/067c33 79670 DeepSignal (MB8000K) COMPLETED 0 2021-04-17 19:24:59.632 2h 3m 30s 2h 3m 20s 771.2% 11.9 GB 22.5 GB 12.4 GB 4.6 GB
77 4f/4e625f 79673 DpSigCombine COMPLETED 0 2021-04-17 21:28:29.950 39.9s 2.2s 17.8% 6 MB 220.3 MB 600.7 MB 600.6 MB
35 57/acff4f 79631 DeepMod (MB4000K) COMPLETED 0 2021-04-17 18:34:45.173 3h 6m 30s 3h 6m 17s 750.9% 38.2 GB 53.6 GB 36.7 GB 12.8 GB
38 55/034820 79635 DeepMod (MB4800K) COMPLETED 0 2021-04-17 18:35:21.325 3h 51m 54s 3h 50m 53s 767.6% 47.1 GB 67.6 GB 45.7 GB 15.9 GB
44 b3/6a8855 79641 DeepMod (MB6400K) COMPLETED 0 2021-04-17 18:36:03.433 4h 14m 2s 4h 12m 4s 789.9% 66.9 GB 94.8 GB 63.5 GB 22.1 GB
41 7e/9e899f 79637 DeepMod (MB5600K) COMPLETED 0 2021-04-17 18:35:47.529 4h 25m 3s 4h 24m 21s 774.9% 62.5 GB 87.7 GB 54.7 GB 19.1 GB
49 13/8cd21d 79645 DeepMod (MB7200K) COMPLETED 0 2021-04-17 18:36:29.174 5h 17m 46s 5h 13m 45s 785.5% 75.7 GB 107.9 GB 72.6 GB 25.2 GB
50 41/7792a8 79646 DeepMod (MB8000K) COMPLETED 0 2021-04-17 18:36:48.009 5h 41m 57s 5h 36m 22s 796.6% 87.1 GB 124.5 GB 72.9 GB 25.5 GB
78 4f/a82320 79674 DpmodCombine COMPLETED 0 2021-04-18 00:18:45.343 4m 10s 3m 47s 309.8% 12.4 GB 17.2 GB 20.1 GB 11.3 GB
Binary file added docs/reports2.pdf
View file
Binary file not shown.

0 comments on commit 85c20b1

Please sign in to comment.