Repeating words after training

[PavlosMelissinos]

I've been trying to train the network but I've been getting captions that repeat the same word again and again, like so:

Predicted: a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a
Target: a skier is jumping over a kicker at a snow covered slope <END> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <>

Predicted: four four four four four four four four four four four four four four four four four four four four four four four four four four four four four four four four
Target: a giraffe is standing and one is sitting on a green meadow with <END> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <>

Sometimes more words appear; the caption is still gibberish though.

Predicted: a are and and and and in <END> <END> <END> <END> <END> <END> <END> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <>
Target: tourists are sitting on the deck of a ship and are <UNK> the <END> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <> <>

I've made slight modifications to your project in order to use rgb images as input instead of the generated cnn features. This is the result. Do you see something wrong in the actual definition or is the problem elsewhere? Any ideas?

Thanks in advance