We have achieved significant progress in understanding how programmers comprehend code, using neurophysiological and behavioral methods such as cognitive modeling, neuroimaging, and eye tracking.
We developed the first comprehensive computational model simulating how programmers retrieve variable values during code comprehension. This model accurately predicts which parts of the code are harder to understand or error-prone, and a web-based demo is in progress to make these insights accessible for real-world use.
We conducted a series of EEG and fMRI studies that have revealed how expertise and cognitive strategies influence programming. For instance, skilled programmers process code with less cognitive effort. Interestingly, commonly used experience measures do not predict programmer efficacy well, but self-estimation and indicators of learning eagerness are fairly accurate.
In a series of eye-tracking studies, we explored how visual cues such as comments and type annotations affect code comprehension. While programmers perceive comments as helpful, their actual impact on understanding is context-dependent. Similarly, type annotations often do not enhance comprehension as much as expected.
Most notably, we have discovered parallels between processing code and natural language, finding that programmers’ brains respond to confusing code elements in ways similar to unexpected words in sentences, suggesting shared cognitive mechanisms.
Using large language models (LLMs), we demonstrated their potential to mirror human judgments about "fuzzy" aspects of code, like smells and clarity, where traditional tools struggle. This work also highlights how LLMs improve with contextual information that benefits humans.
Additionally, we refined existing techniques for neuroimaging and eye-tracking studies. Notably, we identified better baselines for program comprehension tasks and called for standardization in eye-tracking data analysis to ensure robust and comparable results.
These insights pave the way for better programming tools, education, and metrics, fostering a deeper understanding of how we think and work with code.
