-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
e2cc6d9
commit 8a178b3
Showing
1 changed file
with
69 additions
and
69 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,111 +1,111 @@ | ||
| # BenchmarkMockNet Results | ||
|
|
||
| ## Official Run: 11/07/2023 | ||
| ## Official Run: 12/04/2024 | ||
|
|
||
| | Framework | Version | | ||
| |-----------|---------| | ||
| | FakeItEasy | 8.0.0 | | ||
| | JustMock | 2023.3.1011.155 | | ||
| | Moq | 4.20.69 | | ||
| | NSubstitute | 5.1.0 | | ||
| | FakeItEasy | 8.3.0 | | ||
| | JustMock | 2024.4.1113.345 | | ||
| | Moq | 4.20.72 | | ||
| | NSubstitute | 5.3.0 | | ||
| | PCLMock | 5.1.3 | | ||
| | Rocks | 7.1.4 | | ||
| | Rocks | 9.0.0 | | ||
|
|
||
| ### Construction | ||
|
|
||
| This test simply creates an `IThing` object to test using the given framework | ||
|
|
||
| | Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio | | ||
| |------------- |-------------:|-------------:|------------:|---------:|--------:|-------:|-------:|----------:|------------:| | ||
| | Stub | 31.34 ns | 211.41 ns | 11.59 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 | | ||
| | FakeItEasy | 4,979.65 ns | 30,856.32 ns | 1,691.34 ns | 167.74 | 52.16 | 0.1500 | 0.0100 | 2962 B | 123.42 | | ||
| | JustMockLite | 48,256.19 ns | 35,224.70 ns | 1,930.78 ns | 1,719.42 | 739.33 | 0.9100 | 0.0100 | 17033 B | 709.71 | | ||
| | Moq | 4,585.05 ns | 19,632.53 ns | 1,076.13 ns | 157.27 | 51.49 | 0.1100 | - | 2088 B | 87.00 | | ||
| | NSubstitute | 5,473.06 ns | 29,137.72 ns | 1,597.14 ns | 185.43 | 56.33 | 0.2700 | - | 5176 B | 215.67 | | ||
| | PCLMock | 155.87 ns | 1,080.57 ns | 59.23 ns | 5.03 | 0.70 | - | - | 144 B | 6.00 | | ||
| | Rocks | 146.48 ns | 1,090.11 ns | 59.75 ns | 4.69 | 0.62 | - | - | 144 B | 6.00 | | ||
| | Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Allocated | Alloc Ratio | | ||
| |------------- |-------------:|-------------:|-------------:|-------------:|---------:|--------:|-------:|----------:|------------:| | ||
| | Stub | 21.92 ns | 141.09 ns | 7.734 ns | 23.70 ns | 1.11 | 0.53 | - | 24 B | 1.00 | | ||
| | FakeItEasy | 3,980.41 ns | 46,194.31 ns | 2,532.066 ns | 3,304.76 ns | 201.05 | 137.51 | 0.0300 | 2938 B | 122.42 | | ||
| | JustMockLite | 28,136.37 ns | 66,462.03 ns | 3,643.008 ns | 26,047.76 ns | 1,421.17 | 538.17 | 0.2000 | 16928 B | 705.33 | | ||
| | Moq | 4,539.87 ns | 54,657.79 ns | 2,995.978 ns | 3,959.12 ns | 229.31 | 161.10 | 0.0200 | 2088 B | 87.00 | | ||
| | NSubstitute | 4,610.34 ns | 58,992.60 ns | 3,233.584 ns | 3,338.53 ns | 232.87 | 171.24 | 0.0600 | 5064 B | 211.00 | | ||
| | PCLMock | 81.27 ns | 727.25 ns | 39.863 ns | 80.54 ns | 4.10 | 2.36 | - | 144 B | 6.00 | | ||
| | Rocks | 114.28 ns | 649.43 ns | 35.597 ns | 116.96 ns | 5.77 | 2.65 | - | 128 B | 5.33 | | ||
|
|
||
| ### Return | ||
|
|
||
| A mock object is created, a method is setup to a return and `int` value, and that method is called | ||
|
|
||
| | Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio | | ||
| |------------- |--------------:|--------------:|------------:|----------:|---------:|-------:|-------:|----------:|------------:| | ||
| | Stub | 28.90 ns | 212.68 ns | 11.66 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 | | ||
| | FakeItEasy | 8,969.13 ns | 38,381.23 ns | 2,103.80 ns | 343.22 | 134.31 | 0.2600 | 0.0100 | 4982 B | 207.58 | | ||
| | JustMockLite | 280,723.96 ns | 105,038.74 ns | 5,757.53 ns | 11,205.96 | 5,574.96 | 1.8500 | 1.8400 | 34629 B | 1,442.88 | | ||
| | Moq | 7,620.79 ns | 26,993.24 ns | 1,479.59 ns | 294.42 | 121.71 | 0.2100 | - | 3952 B | 164.67 | | ||
| | NSubstitute | 9,560.75 ns | 32,758.64 ns | 1,795.61 ns | 369.85 | 153.98 | 0.4000 | 0.0100 | 7648 B | 318.67 | | ||
| | PCLMock | 3,186.99 ns | 15,562.04 ns | 853.01 ns | 120.68 | 43.98 | 0.1000 | - | 1912 B | 79.67 | | ||
| | Rocks | 822.79 ns | 2,695.01 ns | 147.72 ns | 31.05 | 9.61 | 0.0300 | - | 680 B | 28.33 | | ||
| | Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Gen2 | Allocated | Alloc Ratio | | ||
| |------------- |--------------:|--------------:|-------------:|--------------:|----------:|---------:|-------:|-------:|-------:|----------:|------------:| | ||
| | Stub | 20.00 ns | 140.37 ns | 7.694 ns | 22.06 ns | 1.13 | 0.61 | - | - | - | 24 B | 1.00 | | ||
| | FakeItEasy | 6,557.53 ns | 80,952.12 ns | 4,437.259 ns | 4,157.54 ns | 372.11 | 277.92 | 0.0500 | 0.0100 | 0.0100 | 5055 B | 210.62 | | ||
| | JustMockLite | 187,100.74 ns | 153,098.21 ns | 8,391.830 ns | 182,385.53 ns | 10,617.18 | 4,325.82 | 0.4100 | 0.4000 | - | 34713 B | 1,446.38 | | ||
| | Moq | 7,216.51 ns | 98,008.56 ns | 5,372.180 ns | 4,807.78 ns | 409.51 | 327.76 | 0.0400 | - | - | 3952 B | 164.67 | | ||
| | NSubstitute | 7,425.12 ns | 106,521.54 ns | 5,838.805 ns | 4,063.17 ns | 421.34 | 351.45 | 0.0900 | - | - | 7536 B | 314.00 | | ||
| | PCLMock | 4,153.44 ns | 22,762.50 ns | 1,247.689 ns | 4,494.09 ns | 235.69 | 115.91 | 0.0200 | - | - | 1800 B | 75.00 | | ||
| | Rocks | 428.98 ns | 1,934.62 ns | 106.043 ns | 387.88 ns | 24.34 | 11.33 | - | - | - | 376 B | 15.67 | | ||
|
|
||
| ### EmptyReturn | ||
|
|
||
| A mock object is created, with no method setup, and an method returning an `int` is called | ||
|
|
||
| | Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio | | ||
| |------------- |-------------:|-------------:|------------:|---------:|---------:|-------:|-------:|----------:|------------:| | ||
| | Stub | 28.71 ns | 204.15 ns | 11.19 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 | | ||
| | FakeItEasy | 5,890.26 ns | 29,446.18 ns | 1,614.04 ns | 223.10 | 80.85 | 0.2000 | 0.0100 | 3833 B | 159.71 | | ||
| | JustMockLite | 60,504.16 ns | 41,721.72 ns | 2,286.91 ns | 2,398.37 | 1,130.18 | 1.1100 | 0.1200 | 20858 B | 869.08 | | ||
| | Moq | 5,204.82 ns | 20,856.75 ns | 1,143.23 ns | 200.98 | 83.37 | 0.1300 | - | 2456 B | 102.33 | | ||
| | NSubstitute | 7,862.23 ns | 31,442.13 ns | 1,723.45 ns | 299.91 | 110.04 | 0.3100 | - | 5872 B | 244.67 | | ||
| | PCLMock | 3,256.73 ns | 13,948.61 ns | 764.57 ns | 124.13 | 45.95 | 0.1000 | - | 1912 B | 79.67 | | ||
| | Rocks | 789.88 ns | 1,424.53 ns | 78.08 ns | 30.92 | 13.42 | 0.0300 | - | 680 B | 28.33 | | ||
| | Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio | | ||
| |------------- |-------------:|--------------:|-------------:|-------------:|---------:|--------:|-------:|-------:|----------:|------------:| | ||
| | Stub | 20.40 ns | 145.48 ns | 7.974 ns | 22.37 ns | 1.14 | 0.62 | - | - | 24 B | 1.00 | | ||
| | FakeItEasy | 4,966.61 ns | 61,618.52 ns | 3,377.519 ns | 3,695.11 ns | 277.18 | 208.70 | 0.0400 | - | 3809 B | 158.71 | | ||
| | JustMockLite | 35,981.92 ns | 126,940.79 ns | 6,958.053 ns | 31,982.32 ns | 2,008.13 | 899.01 | 0.2400 | 0.1100 | 20641 B | 860.04 | | ||
| | Moq | 4,953.10 ns | 67,781.09 ns | 3,715.310 ns | 3,397.27 ns | 276.43 | 223.42 | 0.0200 | - | 2456 B | 102.33 | | ||
| | NSubstitute | 5,676.17 ns | 80,205.68 ns | 4,396.344 ns | 3,175.17 ns | 316.78 | 262.26 | 0.0600 | - | 5760 B | 240.00 | | ||
| | PCLMock | 3,964.90 ns | 23,672.27 ns | 1,297.557 ns | 4,529.38 ns | 221.28 | 112.94 | 0.0200 | - | 1800 B | 75.00 | | ||
| | Rocks | 338.55 ns | 1,928.35 ns | 105.699 ns | 307.14 ns | 18.89 | 9.49 | - | - | 376 B | 15.67 | | ||
|
|
||
| ### EmptyMethod | ||
|
|
||
| A mock object is created, with no method setup, and a method with no return value is called | ||
|
|
||
| | Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio | | ||
| |------------- |-------------:|-------------:|------------:|---------:|---------:|-------:|-------:|----------:|------------:| | ||
| | Stub | 28.84 ns | 206.64 ns | 11.33 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 | | ||
| | FakeItEasy | 6,006.92 ns | 27,867.68 ns | 1,527.52 ns | 226.54 | 78.82 | 0.2000 | 0.0100 | 3785 B | 157.71 | | ||
| | JustMockLite | 60,569.59 ns | 41,286.68 ns | 2,263.06 ns | 2,397.36 | 1,142.45 | 1.1100 | 0.1300 | 20748 B | 864.50 | | ||
| | Moq | 4,646.49 ns | 22,813.25 ns | 1,250.47 ns | 175.53 | 63.52 | 0.1300 | - | 2432 B | 101.33 | | ||
| | NSubstitute | 6,980.76 ns | 31,131.66 ns | 1,706.43 ns | 266.23 | 103.37 | 0.3100 | - | 5872 B | 244.67 | | ||
| | PCLMock | 3,090.80 ns | 15,101.93 ns | 827.79 ns | 116.08 | 39.15 | 0.1000 | - | 1872 B | 78.00 | | ||
| | Rocks | 713.38 ns | 1,572.38 ns | 86.19 ns | 27.55 | 10.94 | 0.0300 | - | 672 B | 28.00 | | ||
| | Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio | | ||
| |------------- |-------------:|--------------:|-------------:|-------------:|---------:|--------:|-------:|-------:|----------:|------------:| | ||
| | Stub | 20.33 ns | 143.43 ns | 7.862 ns | 22.16 ns | 1.13 | 0.61 | - | - | 24 B | 1.00 | | ||
| | FakeItEasy | 5,279.97 ns | 66,195.26 ns | 3,628.386 ns | 4,074.53 ns | 294.48 | 221.78 | 0.0400 | - | 3785 B | 157.71 | | ||
| | JustMockLite | 35,621.65 ns | 129,000.99 ns | 7,070.980 ns | 31,545.01 ns | 1,986.74 | 879.19 | 0.2400 | 0.1100 | 20529 B | 855.38 | | ||
| | Moq | 4,416.85 ns | 59,264.63 ns | 3,248.494 ns | 3,016.04 ns | 246.34 | 194.93 | 0.0200 | - | 2432 B | 101.33 | | ||
| | NSubstitute | 5,235.58 ns | 70,096.90 ns | 3,842.248 ns | 3,138.10 ns | 292.01 | 230.69 | 0.0600 | - | 5760 B | 240.00 | | ||
| | PCLMock | 3,746.25 ns | 24,150.26 ns | 1,323.757 ns | 3,999.61 ns | 208.94 | 108.40 | 0.0200 | - | 1784 B | 74.33 | | ||
| | Rocks | 294.88 ns | 1,695.87 ns | 92.956 ns | 261.91 ns | 16.45 | 8.18 | - | - | 328 B | 13.67 | | ||
|
|
||
| ### OneParameter | ||
|
|
||
| A mock object is created, with no method setup, and a method with no return value requiring an `int` parameter is called | ||
|
|
||
| | Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio | | ||
| |------------- |--------------:|--------------:|------------:|---------:|---------:|-------:|-------:|----------:|------------:| | ||
| | Stub | 28.64 ns | 202.43 ns | 11.10 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 | | ||
| | FakeItEasy | 6,006.30 ns | 29,564.27 ns | 1,620.52 ns | 228.04 | 82.64 | 0.2000 | 0.0100 | 3922 B | 163.42 | | ||
| | JustMockLite | 60,026.49 ns | 42,608.40 ns | 2,335.51 ns | 2,380.53 | 1,110.35 | 1.1300 | 0.1200 | 21220 B | 884.17 | | ||
| | Moq | 4,844.69 ns | 22,506.18 ns | 1,233.64 ns | 184.21 | 66.53 | 0.1300 | - | 2464 B | 102.67 | | ||
| | NSubstitute | 7,445.89 ns | 32,069.41 ns | 1,757.83 ns | 283.07 | 99.70 | 0.3100 | - | 5904 B | 246.00 | | ||
| | PCLMock | 234,568.49 ns | 114,493.45 ns | 6,275.77 ns | 9,373.17 | 4,580.23 | 0.5600 | 0.5500 | 10584 B | 441.00 | | ||
| | Rocks | 936.82 ns | 5,065.72 ns | 277.67 ns | 33.92 | 4.93 | 0.0400 | - | 792 B | 33.00 | | ||
| | Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio | | ||
| |------------- |--------------:|--------------:|-------------:|--------------:|---------:|---------:|-------:|-------:|----------:|------------:| | ||
| | Stub | 19.85 ns | 134.17 ns | 7.355 ns | 22.55 ns | 1.13 | 0.59 | - | - | 24 B | 1.00 | | ||
| | FakeItEasy | 5,037.68 ns | 63,019.22 ns | 3,454.296 ns | 3,660.07 ns | 286.12 | 214.21 | 0.0400 | - | 3921 B | 163.38 | | ||
| | JustMockLite | 35,918.35 ns | 116,894.03 ns | 6,407.356 ns | 32,404.15 ns | 2,040.05 | 878.56 | 0.2500 | 0.1100 | 21001 B | 875.04 | | ||
| | Moq | 4,395.96 ns | 59,496.97 ns | 3,261.230 ns | 2,943.79 ns | 249.68 | 198.03 | 0.0200 | - | 2464 B | 102.67 | | ||
| | NSubstitute | 5,292.97 ns | 70,211.33 ns | 3,848.520 ns | 3,145.33 ns | 300.62 | 234.89 | 0.0600 | - | 5792 B | 241.33 | | ||
| | PCLMock | 126,630.98 ns | 82,679.22 ns | 4,531.927 ns | 124,237.18 ns | 7,192.25 | 2,870.92 | 0.1200 | 0.1100 | 10478 B | 436.58 | | ||
| | Rocks | 535.07 ns | 2,286.96 ns | 125.356 ns | 484.05 ns | 30.39 | 13.77 | - | - | 392 B | 16.33 | | ||
|
|
||
| ### Callback | ||
|
|
||
| A mock object is created, a method is setup with a callback, and that method is called | ||
|
|
||
| | Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio | | ||
| |------------- |--------------:|-------------:|------------:|---------:|---------:|-------:|-------:|----------:|------------:| | ||
| | Stub | 36.04 ns | 211.35 ns | 11.58 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 | | ||
| | FakeItEasy | 9,730.53 ns | 41,163.62 ns | 2,256.32 ns | 284.24 | 81.51 | 0.2700 | 0.0100 | 5167 B | 215.29 | | ||
| | JustMockLite | 314,033.86 ns | 71,902.79 ns | 3,941.23 ns | 9,480.39 | 3,595.73 | 1.8300 | 1.8200 | 34287 B | 1,428.62 | | ||
| | Moq | 7,799.30 ns | 25,730.45 ns | 1,410.37 ns | 229.49 | 68.42 | 0.2200 | - | 4112 B | 171.33 | | ||
| | NSubstitute | 9,171.73 ns | 31,745.34 ns | 1,740.07 ns | 269.03 | 76.95 | 0.3600 | - | 6896 B | 287.33 | | ||
| | PCLMock | 3,867.43 ns | 18,210.21 ns | 998.16 ns | 112.21 | 30.07 | 0.1100 | - | 2080 B | 86.67 | | ||
| | Rocks | 777.72 ns | 1,593.74 ns | 87.36 ns | 22.97 | 6.67 | 0.0400 | - | 760 B | 31.67 | | ||
| | Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio | | ||
| |------------- |--------------:|--------------:|-------------:|--------------:|---------:|---------:|-------:|-------:|----------:|------------:| | ||
| | Stub | 23.80 ns | 137.75 ns | 7.551 ns | 26.43 ns | 1.09 | 0.47 | - | - | 24 B | 1.00 | | ||
| | FakeItEasy | 6,435.55 ns | 78,626.44 ns | 4,309.781 ns | 4,097.05 ns | 293.70 | 202.50 | 0.0600 | 0.0100 | 5247 B | 218.62 | | ||
| | JustMockLite | 165,805.10 ns | 124,869.53 ns | 6,844.520 ns | 161,891.83 ns | 7,566.92 | 2,491.64 | 0.4100 | 0.4000 | 34689 B | 1,445.38 | | ||
| | Moq | 7,293.69 ns | 99,574.81 ns | 5,458.031 ns | 4,672.58 ns | 332.87 | 250.64 | 0.0400 | - | 4112 B | 171.33 | | ||
| | NSubstitute | 6,361.13 ns | 84,003.99 ns | 4,604.542 ns | 3,733.59 ns | 290.31 | 212.81 | 0.0800 | - | 6784 B | 282.67 | | ||
| | PCLMock | 3,973.67 ns | 30,900.74 ns | 1,693.774 ns | 4,320.88 ns | 181.35 | 91.80 | 0.0200 | - | 1992 B | 83.00 | | ||
| | Rocks | 398.71 ns | 2,197.96 ns | 120.478 ns | 350.16 ns | 18.20 | 7.76 | - | - | 416 B | 17.33 | | ||
|
|
||
| ### Verify | ||
|
|
||
| Verifies that the method was called | ||
|
|
||
| | Method | Mean | Error | StdDev | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio | | ||
| |------------- |--------------:|-------------:|------------:|---------:|---------:|-------:|-------:|----------:|------------:| | ||
| | Stub | 33.64 ns | 201.12 ns | 11.02 ns | 1.00 | 0.00 | - | - | 24 B | 1.00 | | ||
| | FakeItEasy | 9,186.06 ns | 40,565.27 ns | 2,223.52 ns | 287.89 | 84.23 | 0.2900 | 0.0100 | 5489 B | 228.71 | | ||
| | JustMockLite | 134,882.54 ns | 39,368.39 ns | 2,157.91 ns | 4,375.22 | 1,685.94 | 2.1500 | 0.5300 | 40260 B | 1,677.50 | | ||
| | Moq | 8,506.66 ns | 25,250.99 ns | 1,384.09 ns | 269.44 | 82.73 | 0.2100 | - | 4072 B | 169.67 | | ||
| | NSubstitute | 10,575.58 ns | 38,199.33 ns | 2,093.83 ns | 333.07 | 97.84 | 0.3900 | - | 7464 B | 311.00 | | ||
| | PCLMock | 4,121.90 ns | 16,560.56 ns | 907.74 ns | 129.45 | 37.61 | 0.1400 | - | 2736 B | 114.00 | | ||
| | Rocks | 1,027.11 ns | 1,942.11 ns | 106.45 ns | 32.66 | 10.03 | 0.0400 | - | 800 B | 33.33 | | ||
| | Method | Mean | Error | StdDev | Median | Ratio | RatioSD | Gen0 | Gen1 | Allocated | Alloc Ratio | | ||
| |------------- |-------------:|--------------:|------------:|-------------:|---------:|---------:|-------:|-------:|----------:|------------:| | ||
| | Stub | 28.69 ns | 294.01 ns | 16.12 ns | 25.00 ns | 1.24 | 0.88 | - | - | 24 B | 1.00 | | ||
| | FakeItEasy | 6,885.35 ns | 90,069.79 ns | 4,937.03 ns | 4,270.25 ns | 297.10 | 244.85 | 0.0600 | 0.0100 | 5567 B | 231.96 | | ||
| | JustMockLite | 79,479.13 ns | 158,326.48 ns | 8,678.41 ns | 74,514.44 ns | 3,429.49 | 1,642.31 | 0.4700 | 0.4600 | 39810 B | 1,658.75 | | ||
| | Moq | 7,394.65 ns | 102,095.03 ns | 5,596.17 ns | 4,507.46 ns | 319.08 | 272.93 | 0.0400 | - | 4016 B | 167.33 | | ||
| | NSubstitute | 6,855.15 ns | 79,735.93 ns | 4,370.60 ns | 4,365.06 ns | 295.80 | 225.81 | 0.0800 | - | 7272 B | 303.00 | | ||
| | PCLMock | 4,701.80 ns | 46,629.21 ns | 2,555.90 ns | 5,118.33 ns | 202.88 | 141.05 | 0.0300 | - | 2560 B | 106.67 | | ||
| | Rocks | 863.52 ns | 3,578.30 ns | 196.14 ns | 769.67 ns | 37.26 | 19.18 | - | - | 640 B | 26.67 | | ||
|
|