architecture. Instantiating a configuration with the defaults will yield a similar configuration to that of RoBERTa has almost similar architecture as compare to BERT, but in order to improve the results on BERT architecture, the authors made some simple design changes in its architecture and tra