-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tag "productions" to allow using VSG as a VHDL parser #1309
base: master
Are you sure you want to change the base?
Conversation
…ng point of the productions
…ing them, now including production enter/leave lists. Refactor code to make code changes more managable
…oken attributes such as production tags are preserved during parsing.
Here is a updated PR, which was merged with the latest master (5845f5) . I had to re-introduce the systematic use of replace_token() to reassign tokens in order to preserve the token attributes such as production tags. Without that the production tags are lost in the post-token assignment processing and my VHDL parser not work anymore. tox generates lots of Pluggy errors, and I got a segfault at some point, but all tests eventually passed in the end. JF |
Description
I was looking for a pure-python VHDL parser to serve as a back-end for a Sphinx domain that would allow extracting documentation straight from the VHDL source code, like we do with Python docstrings.
After looking around and testing various tools, VSG turned out the only VHDL processor that was fast and robust enough to process my code base. The only thing that was missing was information on the hierarchy of the VHDL elements (entities, ports, etc.), also known as "productions". VSG knows about this hierarchy when it "classfies" the tokens, but it does not save the information. (See Discussion #1131)
This pull requests adds a minimally-intrusive mechanism to tag the productions. A
@utils.tagged_production
decorator, when applied to aclassify
function, adds the name of the current production in theenter_prod
list of the token at the beginning of the classify code, and also adds the name of that production to theleave_prod
list of the token when the classify function exits. It is then trivial to reconstruct the hierarchy of the whole code using these tags.This pull requests adds a tagging function and decoration in
vhdlFile/utils.py
, the two abovementioned lists in theitem
class (parser.py
), and a few classify functions have been decorated. This is currently sufficient for me to extract the entity interface information, although so much more could be done. The code has been modified to make sure thatconvert_to
is always used when we replace a token by another, otherwise we lose our production tagging information.This pull request also simplifies the
post_token_assignments()
function invhdlFile.py
by removing some code repetitions. Although I did that work a few months ago, I am submitting it just anow and I just realised that another PR #1301 was submitted a few days ago to the same effect. That PR seems to implement changes that I also had in mind but did not dare do yet. I'd be happy to update my PR to harmonize it with PR #1301 if that helps.The code passes the tox tests.
This is my first public PR ever. Let me know if other information is required.
JF