Lucene.Net.TestFramework: Improved error handling and test reporting #1092
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…emoved debug code.
…Fixed error reporting, since NUnit swallows exceptions in these events. This will write them to stderr by default and allows enabling failing the tests if an exception occurs in LuceneTestCase.OneTimeSetUp() which will make them appear in the Test Output window in Visual Studio. The failures can be enabled by setting the tests:failontestfixtureonetimesetuperror system property to true.
…ment variable (defaulting to 'true') to set the system property of the same name.
…ic.Dictionary<TKey, TValue>, which will format itself. Marked fields private instead of internal.
… from SetUp() to TearDown() and only include them in the test message upon failure.
… messages from calls to Assert.Pass(), since this writes to the output and bloats the size of the test logs with no benefit.
… when maximumAllowedFailures is 0 and there are errors in the stderr file.
… when maximumAllowedFailures is 0 and there are errors in the stdout in the TestResults.trx file.
paulirwin
approved these changes
Jan 8, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Improved error handling and test reporting
Description
This fixes several concerns regarding output during testing:
LuceneTestCase.OneTimeStartUp()andLuceneTestCase.OneTimeTearDown()were not very visible because NUnit doesn't fail the test when they occur there. This changes it from re-throwing exceptions with an additional message to logging to the output. It goes to stderr in all cases except inOneTimeTearDown()where something is redirecting it to stdout. Either way, this changes the ADO pipeline to fail the build so we can avoid shipping the test framework with bugs in these methods.tests:failontestfixtureonetimesetuperrorthat can be used to cause the test framework to fail all of the tests when there is an error inOneTimeSetUp(). We may need to revisit this because there are asserts thrown here that users are supposed to see when dependency injection for codecs is not setup correctly.Assert.Pass()method because this bloats the size of the test logs with no benefit.