A Recurrent Neural Network (RNN) often uses ordered sequences as inputs.
Real-world sequences have different lengths, especially in Natural Language
Processing (NLP) because all words don’t have the same number of characters and
all sentences don’t have the same number of words. In
PyTorch, the inputs of a neural network are often
managed by a
DataLoader
.
A DataLoader
groups the input in batches. This is better for training a neural
network because it’s faster and more efficient than sending the inputs one by
one to the neural network. The issue with this approach is that it assumes every
input has the same shape. As stated before, sequences don’t have a consistent
shape, so how one can train a RNN in PyTorch with variable-length sequences and
still benefit from the DataLoader
class?