Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tag "productions" to allow using VSG as a VHDL parser #1309

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

jfcliche
Copy link

@jfcliche jfcliche commented Nov 3, 2024

Description

I was looking for a pure-python VHDL parser to serve as a back-end for a Sphinx domain that would allow extracting documentation straight from the VHDL source code, like we do with Python docstrings.
After looking around and testing various tools, VSG turned out the only VHDL processor that was fast and robust enough to process my code base. The only thing that was missing was information on the hierarchy of the VHDL elements (entities, ports, etc.), also known as "productions". VSG knows about this hierarchy when it "classfies" the tokens, but it does not save the information. (See Discussion #1131)

This pull requests adds a minimally-intrusive mechanism to tag the productions. A @utils.tagged_production decorator, when applied to a classify function, adds the name of the current production in the enter_prod list of the token at the beginning of the classify code, and also adds the name of that production to the leave_prod list of the token when the classify function exits. It is then trivial to reconstruct the hierarchy of the whole code using these tags.

This pull requests adds a tagging function and decoration in vhdlFile/utils.py, the two abovementioned lists in the item class (parser.py), and a few classify functions have been decorated. This is currently sufficient for me to extract the entity interface information, although so much more could be done. The code has been modified to make sure that convert_to is always used when we replace a token by another, otherwise we lose our production tagging information.

This pull request also simplifies the post_token_assignments() function in vhdlFile.py by removing some code repetitions. Although I did that work a few months ago, I am submitting it just anow and I just realised that another PR #1301 was submitted a few days ago to the same effect. That PR seems to implement changes that I also had in mind but did not dare do yet. I'd be happy to update my PR to harmonize it with PR #1301 if that helps.

The code passes the tox tests.

This is my first public PR ever. Let me know if other information is required.
JF

jfcliche [twiddlebug] added 2 commits December 17, 2024 23:19
@jfcliche
Copy link
Author

Here is a updated PR, which was merged with the latest master (5845f5) . I had to re-introduce the systematic use of replace_token() to reassign tokens in order to preserve the token attributes such as production tags. Without that the production tags are lost in the post-token assignment processing and my VHDL parser not work anymore.

tox generates lots of Pluggy errors, and I got a segfault at some point, but all tests eventually passed in the end.

JF

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

1 participant