Continuous benchmarking #51
Conversation
petervdonovan
force-pushed
the
continuous-benchmarking
branch
3 times, most recently
from
8f4cb40 to
4d1150a
Compare
petervdonovan
force-pushed
the
continuous-benchmarking
branch
2 times, most recently
from
5dc3ec6 to
8ed049e
Compare
|
There might be a little bit of meaningless noise in this PR while I try to setup GitHub Actions. There is an "unsubscribe" button in Github (on the right) for anyone who does not want useless messages. |
petervdonovan
force-pushed
the
continuous-benchmarking
branch
2 times, most recently
from
a1e7191 to
b661909
Compare
petervdonovan
force-pushed
the
continuous-benchmarking
branch
from
b661909 to
eae4e02
Compare
petervdonovan
force-pushed
the
continuous-benchmarking
branch
6 times, most recently
from
06fc272 to
26751d4
Compare
petervdonovan
force-pushed
the
continuous-benchmarking
branch
from
99e6597 to
8c71526
Compare
petervdonovan
force-pushed
the
continuous-benchmarking
branch
from
8c71526 to
ceab594
Compare
petervdonovan
force-pushed
the
continuous-benchmarking
branch
from
bcacdfe to
3ac8ecf
Compare
petervdonovan
force-pushed
the
continuous-benchmarking
branch
from
5c25d0c to
dd98802
Compare
|
The only thing holding this PR back right now is that the performance alerts are not appearing anymore, and I cannot figure out why not. Edit: Resolved! |
petervdonovan
force-pushed
the
continuous-benchmarking
branch
from
8f4a38d to
5f89f93
Compare
petervdonovan
force-pushed
the
continuous-benchmarking
branch
from
f67442f to
3812932
Compare
|
It seems we're incredibly close with this one. Is this ready to merge yet? |
|
I think it might actually be fine to merge this. The other PR has to get moved, this seems like it can just get merged here. I still need to set up the self-hosted runner, but it seems imprudent to wait for the backlog of tasks to clear. |
|
I keep being reminded of this PR, which is still potentially useful and which is still on the brink of being completed. After letting it sit for a while, I am beginning to get the feeling that hooking benchmarks into CI just might not be the way to go. CI has been a constant time-drain, and it will additionally become a security hazard if/when we link it up with our own servers. We may eventually decide that the most time-saving option in the long run is to focus on making sophisticated tests and benchmarks easy to run manually, on a server that we can just manually ssh into when necessary. |
|
If we run benchmarks for feature branches on request and regularly (during times that CI is less busy) on |
| jobs: | ||
| benchmark: | ||
| name: Run C Benchmarks | ||
| runs-on: ubuntu-latest #FIXME: change to self-hosted after russel is set up. |
There was a problem hiding this comment.
| runs-on: ubuntu-latest #FIXME: change to self-hosted after russel is set up. | |
| runs-on: Linux |
This PR adds the continuous benchmarking. It currently runs with Github hosted runners; we are in the process of setting up russell as self-hosted runner to run the benchmarks. This PR comes with another PR in the benchmarks repo that makes some modifications to the runner script.