-
Notifications
You must be signed in to change notification settings - Fork 47
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce automated UI testing based on asciinema #117
Introduce automated UI testing based on asciinema #117
Conversation
0cd5138
to
5ecce0f
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Automated visual inspection in CI? Wow! That's awesome! I find this really great, but at the same time I am a bit worried this could cause some friction for contributions that legitimately change the rendering. I understand in such a case one could push the change, wait for the CI to fail, download the generated artifacts, commit the actual images as the expected ones, and push again. It would be a bit cumbersome though.
Could one instead just run record.sh
to get the expected images? I can't find the agg
command on the Ubuntu repos, is there an alternative?
@edgar-bonet glad you like the idea!
Yes! The fail-and-download approach would work, but I would personally see it as a fallback, as a second option, not the primary way, at least that was not intended. What I did locally to produce fresh images was basically this: cd recordings/
./record.sh
for i in actual-*.png ; do cp ${i} ${i/actual/expected}; done
for i in expected-*.png ; do zopflipng -y ${i} ${i}; done # https://github.com/google/zopfli
git add expected-*.png I can make that a second script
…and then command |
c64eb16
to
8189a28
Compare
8189a28
to
a187c2e
Compare
a187c2e
to
8677913
Compare
8677913
to
86b87f2
Compare
.. and point to it when the CI fails
86b87f2
to
84e89d9
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
This will prevent the next case of #102 right in the pull request introducing the regression.
This took quite a while to build, I ran into many interesting walls in the process and now finally have it working. Happy to answer questions about it.