Topics
In a typical RNN based implementation, say for language translation task, the
encoder encodes input at each timestep, and the encoded output at final timestep, becomes the input to the decoder. The decoder’s hidden state is initialized with this encoded output and bos (beginning of sentence) token is input at first time step. At each timestep, it takes previous generation as input along with the hidden state and keeps on generating until eos (end of sentence) token is generated.