Incorporating PoS information explicitly for the edit_tree_lemmatizer

[oroszgy]

Hey,
we are using the new experimental edit tree lemmatizer for Hungarian in huspacy, and we are looking for a way to explicitly incorporate PoS information for the model, if it is possible at all. As far as I understood multitask learning is one way to help the model yielding PoS aware predictions. Is there any other way to condition the lemmatizer's predictions on PoS tags? For instance, is it possible to learn embeddings for part-of-speech tags in MultiHashEmbed? If yes, could you please guide me how to modify this config?

[oroszgy]

@kadarakos - this is the problem I mentioned

[adrianeboyd]

You need to use "annotating components" and include the feature in the attrs. Docs: https://spacy.io/usage/training#annotating-components

Here's a demo project that includes DEP as a tagger feature:

https://github.com/explosion/projects/tree/v3/pipelines/tagger_parser_predicted_annotations

[oroszgy]

Wow, that’s awesome! Thanks! :)

…

On 2022. Jan 21. 11:56 +0100, Adriane Boyd ***@***.***>, wrote: You need to use "annotating components" and include the feature in the attrs. Docs: https://spacy.io/usage/training#annotating-components Here's a demo project that includes DEP as a tagger feature: https://github.com/explosion/projects/tree/v3/pipelines/tagger_parser_predicted_annotations — Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.Message ID: ***@***.***>

[adrianeboyd]

As a note, I wouldn't be surprised if this didn't help, since the tagger's tok2vec already includes all the same underlying features, so adding a predicted (not gold) tag isn't adding much new info.

[honnibal]

Yeah the POS tagger doesn't even use any structured prediction tricks -- it's just one softmax over all the token vectors. So I wouldn't expect it to help. The parser features have a better chance. You could try putting the lemmatizer at the end of the pipeline and use the DEP feature perhaps.

[KennethEnevoldsen]

Hi @oroszgy, just wanted to let you know that the OdyCy pipeline (see paper here) performs a multistep approach for lemmatisation that uses a heuristic fallback from the lookup to the learned lemmatizer. From my understanding, it is good if you have a lexicon with high precision but low recall.

[oroszgy]

Thanks for the update. We ended up doing sg. very similar for HuSpaCy.

[KennethEnevoldsen]

Oh, that looks interesting. Thanks for sharing it will def. have a look.