Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Go] Fix for #4342--EmitToken(token) in Go target needs to be exported #4343

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from

Conversation

kaby76
Copy link
Contributor

@kaby76 kaby76 commented Jul 2, 2023

This is a fix for #4342.

This change is required for porting https://github.com/antlr/grammars-v4/tree/master/python/python3 to the Go target.

This change adds EmitToken(token) to the type interface for lexers. It's required because it's available--and used!!--in all the other targets. In addition, the call to EmitToken(token) must be "virtualized", which is accomplished using b.Virt.EmitToken(token). The Virt field is set prior to calling the parse in the driver code.

This change is a fix to what is just a terrible, terrible hack. A v-table call should have never been implemented this way!

var lexer = parser.NewPython3Lexer(str)
lexer.BaseLexer.Virt = lexer
var tokens = antlr.NewCommonTokenStream(lexer, 0)
var parser = parser.NewPython3Parser(tokens)
var tree = parser.File_input()

…. Required for porting https://github.com/antlr/grammars-v4/tree/master/python/python3 to the Go target. This change just mirrors the APIs for the other targets, like Java, CSharp, and JavaScript/TypeScript.

Signed-off-by: Ken Domino <kenneth.domino@domemtech.com>
@kaby76 kaby76 changed the title Fix for #4342--EmitToken(token) must be exported and virtualized [Go] Fix for #4342--EmitToken(token) must be exported and virtualized Jul 2, 2023
@kaby76 kaby76 marked this pull request as ready for review July 3, 2023 01:36
teverett pushed a commit to antlr/grammars-v4 that referenced this pull request Jul 8, 2023
* Fixes for #3539

Typescript cannot work because the declaration of the constructor for CommonToken() is wrong.

* Fix python/python3/TypeScript

* Remove grammars that only add to confusion.

* Update Dart port, but this cannot work because Antlr 4.13.0 Dart runtime missing types.

* Add Dart port for python3 grammar.

This base class for Dart works, but requires changes to Antlr. antlr/antlr4#4321

* Updates, but incomplete, for Cpp.

* Additional changes.

* Updates to get Cpp target to link.

* Changes for working Cpp target.

* Add in Cpp to workflow.

* Updates.

* Adjust to for rebuild.

* Remove bothersome tabs from source code.

* Rename tests into "small" and "large" tests to reflect that "slow parsers work on the small test suite".

* Fix names of test directories in desc.xml.

* Update for Go target. Remove python3-cpp.

* Getting Go target compiling--does not work yet.

* Fixes for Go target of python3 grammar.

This port works but only if the Go runtime is fixed, and "Virt" is assigned in the driver. See antlr/antlr4#4343 antlr/antlr4#4342

* Fix desc.xml
rvrooman-codelogic pushed a commit to CodeLogicIncEngineering/grammars-v4-public-fork that referenced this pull request Jul 27, 2023
* Fixes for antlr#3539

Typescript cannot work because the declaration of the constructor for CommonToken() is wrong.

* Fix python/python3/TypeScript

* Remove grammars that only add to confusion.

* Update Dart port, but this cannot work because Antlr 4.13.0 Dart runtime missing types.

* Add Dart port for python3 grammar.

This base class for Dart works, but requires changes to Antlr. antlr/antlr4#4321

* Updates, but incomplete, for Cpp.

* Additional changes.

* Updates to get Cpp target to link.

* Changes for working Cpp target.

* Add in Cpp to workflow.

* Updates.

* Adjust to for rebuild.

* Remove bothersome tabs from source code.

* Rename tests into "small" and "large" tests to reflect that "slow parsers work on the small test suite".

* Fix names of test directories in desc.xml.

* Update for Go target. Remove python3-cpp.

* Getting Go target compiling--does not work yet.

* Fixes for Go target of python3 grammar.

This port works but only if the Go runtime is fixed, and "Virt" is assigned in the driver. See antlr/antlr4#4343 antlr/antlr4#4342

* Fix desc.xml
@kaby76 kaby76 changed the title [Go] Fix for #4342--EmitToken(token) must be exported and virtualized [Go] Fix for #4342--EmitToken(token) in Go target needs to be exported Aug 27, 2023
@parrt
Copy link
Member

parrt commented Sep 4, 2023

@jimidle ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants