Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Refactor beam search to use gluon hybrid blocks (#425)
This refactors beam search to group fixed-size operations in beam search into cached ops through Gluon HybridBlocks. My testing showed ~3% speed improvement. Not much, but consistent. Another change that is included here is to not use columns from `sequences` to pass into the decoder step module, but use `best_word_indices` from the previous iteration. NDArray indexing seems expensive and ideally we should aim for avoiding all indexing ops in an iteration.
- Loading branch information