Doesn't support parameters that appear in more than one parameter group

[fizzAI]

If you try to use APOLLO to train, for example, Llama 3 1b, which has tied embeddings (IE the embed_tokens and lm_head layers use the same underlying parameters), it fails to load the optimizer with
ValueError: some parameters appear in more than one parameter group
For now you can just manually exclude the embedding layers from the optimizer, but it would be nice if it worked properly ootb :3

[zhuhanqing]

Hi @fizzAI , thank you so much for your suggestion!
We will work on your request and get back to u soon.