DBC Replacement - File Parsing

[BlakeFreer]

Goals

• Create a file representation of the database structure defined by DBC Replacement - Data Structure #355.

• Convert all the racecar/firmware DBCs to this file format.

• Parse the files in Python.

• Write good Python. Data parsing is a really interesting project since there are a lot of ways to use first-class functions and decorators to improve the behaviour without cluttering the code.

  • At any point, if you feel that you are repeating code, conditions, loops, error handling, etc, find a way to abstract out that behaviour. Our parsing code should focus more on the data structure than parsing.

Non-goals

• Write the bare file parser. Instead, choose a file format that has existing Python support like json, yaml, toml, etc.

Requirements

• The data should be validated as it is parsed. For example, we should ensure that all Signal bits fit inside of the message's data length without overlap.

• Any invalid / missing / unused value should cause an error during parsing.

Resources

We currently use cantools to parse our DBCs. Look for inspiration in their database code. https://github.com/cantools/cantools/tree/master/src/cantools/database/can

@AndrewI26 wrote this parser for the cangen yaml configuration. https://github.com/macformula/racecar/blob/main/scripts/cangen/cangen/config.py
Notice how it first reads the yaml to a dictionary, then parses the dictionary for keys. This is good design since we could easily switch from yaml to json because both can be read to a dictionary.

[BlakeFreer]

mkdocs is the framework which we use for our documentation website. It supports plugins which can have a structured configuration. For example, see the PrivacyConfig object in the "privacy" plugin from mkdocs-material

https://github.com/squidfunk/mkdocs-material/blob/master/src/plugins/privacy/config.py

You can look at the mkdocs imports and check out that repo to see how it implements the Config and Type classes.

The important part is each plugin only needs to describe the structure of its configuration scheme and a centralized function can handle all of the parsing