work PROST: Physical Reasoning of Objects through Space and Time A probing dataset highlighting why the lack of real-world experience hinders language understanding. On Losses for Modern Language Models A thorough comparison of auxiliary pre-training tasks for BERT. fun
