consider adding tests & validation

[alecristia]

Unit Tests for Individual Functions
Purpose: Unit tests verify the correctness of individual functions or methods in isolation from the rest of the system.
Sample Data: You often need sample input data that your functions will process. This data doesn't have to be extensive but should be representative of the types of inputs the functions will handle in production. For testing functions that manipulate data without needing to load files, you can typically create the necessary data directly within the test code (e.g., strings, numbers, dictionaries, or simple objects).
Integration Tests for Workflows
Purpose: Integration tests check the interactions between different parts of your application to ensure they work together as expected.
Sample Data: Depending on the complexity of the workflows, you might need more realistic or extensive sample data. For workflows that involve processing files (like RTTM or YAML files in your case), you'll likely need sample files. These should be placed in a known directory structure that your tests can access.
Approaches to Handling Sample Data
Mocking: For operations that interact with external systems or libraries (like file I/O, database access, or network requests), consider using mocking to simulate those interactions. Mocking libraries, such as unittest.mock in the Python standard library, can simulate file reads/writes or function calls and return pre-defined outputs without needing actual files or a network connection.
Fixture Files: For tests that require reading from or writing to files, prepare a set of fixture files that contain sample data for your tests. Keep these files in a test directory and make sure your test code knows how to locate and use them.
Temporary Data: For tests that generate output, consider using Python's tempfile module to create temporary files or directories. This approach ensures that your tests do not pollute the file system with test artifacts.
Next Steps
When we continue with writing tests:

Decide on the scope of both unit and integration tests based on the critical functionalities of your script.
Prepare sample input data or files needed for testing.
Explore Python's unittest framework (or alternatives like pytest) for structuring your tests.
Consider how to incorporate mocking for tests that involve file I/O, network requests, or interactions with external libraries.
Starting with a few key tests for critical functionalities can provide a solid foundation, and you can incrementally build upon this as your project grows. If you need to generate or use specific sample data for testing, it's usually resolved directly within the code or through the use of external fixture files designed specifically for testing purposes.