Added more matrix benchmarks #34
Conversation
|
There are a quite a few methods named |
|
I sure did. So used to pyunit! |
|
Good. Running this against SymPy master I get: Are those failures expected? |
benchmarks/solve.py
Outdated
|
|
||
| # every test will be based of a submatrix of this matrix | ||
| big_mat = Matrix([[3, 8, 10, 5, 10, 7, 10, 10, 8, 6], [10, 9, 3, 7, 10, 1, 4, 2, 8, 1], [5, 9, 9, 0, 2, 10, 5, 9, 3, 9], [1, 8, 0, 7, 8, 8, 0, 4, 1, 10], [6, 5, 3, 0, 3, 4, 6, 1, 10, 5], [7, 10, 8, 9, 10, 7, 2, 8, 3, 2], [10, 8, 5, 10, 3, 5, 10, 4, 2, 3], [8, 4, 10, 9, 1, 9, 7, 4, 8, 6], [6, 2, 4, 1, 1, 0, 1, 3, 1, 9], [9, 2, 6, 10, 9, 4, 10, 2, 1, 8]]) | ||
| symbol_locations = [(2, 2), (1, 9), (0, 0), (0, 7), (9, 1), (6, 9), (8, 9), (4, 0), (3, 8), (3, 2), (2, 8), (1, 8), (5, 3), (5, 9), (6, 4), (5, 5), (7, 9), (5, 1), (1, 0), (3, 3), (7, 1), (2, 5), (1, 5), (4, 4), (4, 2), (7, 3), (3, 4), (6, 6), (9, 5), (1, 6), (9, 0), (3, 1), (0, 4), (8, 3), (2, 3), (3, 9), (9, 6), (4, 8), (9, 3), (8, 0), (6, 7), (5, 7), (8, 6), (3, 6), (4, 5), (1, 2), (9, 8), (7, 4), (8, 8), (6, 1), (0, 3), (4, 7), (7, 0), (9, 7), (5, 4), (7, 6), (2, 6), (3, 7), (3, 5), (1, 4), (5, 0), (4, 9), (7, 8), (6, 8), (2, 1), (9, 2), (3, 0), (7, 7), (2, 7), (2, 0), (8, 1), (7, 5), (4, 3), (1, 3), (9, 9), (0, 6), (4, 1), (5, 8), (8, 4), (0, 8), (2, 4), (9, 4), (7, 2), (1, 7), (6, 3), (6, 5), (5, 2), (6, 0), (0, 1), (8, 2), (2, 9), (8, 5), (0, 2), (0, 9), (8, 7), (4, 6), (0, 5), (1, 1), (6, 2), (5, 6)] |
There was a problem hiding this comment.
could you split up these two long lines?
|
It is expected that for |
|
I see, the only worry I have is that the default timeout is 60 seconds. That means that merging this PR as-is would drastically increase the time spent per commit in the benchmark. We have had a loose goal of trying to stay under 1 second per benchmark, see: But we've also discussed having a set of slow benchmarks, which we wouldn't run on each commit, but maybe every 500th commit, and then use the bisecting function of |
|
Increasing the cache size might speed things up. The default value is
1000. You could try 10000 and see what happens. The size is controlled
through the environment variable SYMPY_CACHE_SIZE
…On Mon, Apr 3, 2017 at 3:40 PM, Björn Dahlgren ***@***.***> wrote:
I see, the only worry I have is that the default timeout is 60 seconds.
That means that merging this PR as-is would drastically increase the time
spent per commit in the benchmark.
We have had a loose goal of trying to stay under 1 second per benchmark,
see:
#8 <#8>
But we've also discussed having a set of slow benchmarks, which we
wouldn't run on each commit, but maybe every 500th commit, and then use the
bisecting function of asv to find regressions.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#34 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADuV_h4Om1uRJSHiOdRb-Dga8lrTSeR6ks5rsWdhgaJpZM4MxEZu>
.
|
|
I like the idea of a slow benchmark run more infrequently. I'm working on some optimized code to bring some of these computations to < 1s. The trouble is, things slow down really fast in the matrix library with For things like determinants, size ~ Could the speed tests be set to timeout after 2 seconds? That seems like it would get a lot of good data and when the algorithms improve and they suddenly jump below that threshold, they'd suddenly start showing up. |
|
@siefkenj We can absolutely change the timeout to 2 "travis-seconds" (I think they use Xeon processors on google compute enginge). Looking at the output from the most recently merged PR we have these timings on Travis: In that case, would you mind creating a new folder e.g. @pbrady thanks for pointing out |
|
@bjodah Is the Travis error related to my commit or is it something else? |
|
Forgot to say, we need a folder |
|
Ok, so this line needs to be updated: |
|
Great, thanks! |
This adds several more benchmarks of matrix algorithms including
rank,rref,det, as well as matrix multiplication and matrix addition.