Model Internal Sleuthing: Finding Lexical Identity and Inflectional Features in Modern Language Models

arXiv:2506.02132v5 Announce Type: replace-cross Abstract: Large transformer-based language models dominate modern NLP, yet our understanding of how they encode linguistic information relies primarily on studies of early models like BERT and GPT-2. We systematically probe 25 models from BERT Base to