Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models - Tech Sentiments