Training pipelines (tok, lem, morph, parse) together or separately?

[LilitKharatyan]

I am training (almost) a full pipeline of models: tokenizer, trainable_lemmatizer, tagger, morphologizer, parser and I see that accuracy differs based on which pipelines are trained together, e.g. I receive the best results when morphologizer and tagger are trained together, while when I add for example parser to this pipeline of training, the results/ accuracy downgrade.
Is it a common behavior, and are there best practices on this issue, or am I doing smth wrong? Thanks

[adrianeboyd]

You probably aren't doing anything wrong. Multi-task learning may not always be beneficial and it's something you typically just have to try out for your task. In general closely related tasks are more likely to benefit (tagger+morphologizer vs. tagger+ner), but it can always depend on the exact data/config.