Periodic Reporting for period 1 - TOPS (Machine-Assisted Teaching for Open-Ended Problem Solving: Foundations and Applications)
Reporting period: 2022-04-01 to 2024-09-30
Computational thinking and problem-solving skills are essential for everyone in the 21st century, both for students to excel in STEM+Computing fields and for adults to thrive in the digital economy. Consequently, educators are putting increasing emphasis on pedagogical tasks in open-ended learning domains such as programming, conceptual puzzles, and virtual reality environments. When learning to solve open-ended tasks by themselves, novice learners often struggle because such tasks are typically underspecified, conceptual, and require multi-step reasoning. These struggling learners can benefit from individualized assistance, for instance, by receiving personalized curriculum across tasks or feedback within a task. Unfortunately, human tutoring resources are scarce, and receiving individualized human assistance is rather a privilege. Technology empowered by artificial intelligence has the potential to tackle this scarcity challenge by providing scalable and automated machine assistance. However, the state-of-the-art technology is limited: it is designed for well-defined procedural learning but not for open-ended conceptual problem-solving.
The TOPS project is devoted to address this fundamental societal challenge of providing cost-effective and inclusive education that fosters computational thinking and problem-solving skills. The project aims to develop novel techniques for machine-assisted teaching in open-ended learning domains. Most prominently, these techniques can synthesize new tasks of desirable characteristics and recommend the next task to the learner for efficient learning. Moreover, these techniques can provide explicable grade points when the learner is solving a task and actionable hints if the learner reaches an impasse. The project also involves designing new computational learning models that can be used to simulate and compare the usefulness of different forms of assistance. Finally, the project aims to demonstrate the performance of developed techniques in various open-ended learning domains, including visual programming and Python programming.
The TOPS project is devoted to address this fundamental societal challenge of providing cost-effective and inclusive education that fosters computational thinking and problem-solving skills. The project aims to develop novel techniques for machine-assisted teaching in open-ended learning domains. Most prominently, these techniques can synthesize new tasks of desirable characteristics and recommend the next task to the learner for efficient learning. Moreover, these techniques can provide explicable grade points when the learner is solving a task and actionable hints if the learner reaches an impasse. The project also involves designing new computational learning models that can be used to simulate and compare the usefulness of different forms of assistance. Finally, the project aims to demonstrate the performance of developed techniques in various open-ended learning domains, including visual programming and Python programming.
Our work on curriculum across tasks has introduced novel techniques that can synthesize new tasks and recommend the next suitable task to a learner. Regarding task synthesis, we have developed techniques for visual programming domains popularly used to introduce computing concepts in schools. These techniques allow teachers/learners to specify the desired task's concepts or difficulty level and generate high-quality tasks that match the specifications. Regarding task recommendation, we have developed general-purpose curriculum design techniques that broadly apply to any open-ended learning domains. These techniques can recommend new suitable tasks to a learner based on the learner's current knowledge, the desired target knowledge, and the tasks' difficulty.
Our work on feedback within a task has introduced novel techniques that can provide explicable grade points and actionable hints. Regarding explicable grades, we have developed a new reward design technique that can assign interpretable and informative rewards to guide the learning process in open-ended domains. Regarding actionable hints, we have introduced a series of techniques to automatically generate high-quality hints for Python programming. In particular, these techniques leverage recent advances in generative AI and large language models. Moreover, we have introduced a new form of quizzes for visual programming domains that can be useful in providing more engaging feedback to a learner.
We have also designed new computational models to capture the learner's knowledge and the execution behavior (i.e. how the learner solves a task using current knowledge). In turn, these models enable simulating and comparing the usefulness of different interventions when designing curriculum across tasks or providing feedback within a task.
We have demonstrated the performance of our techniques in a wide range of open-ended learning domains, including visual programming and Python programming. We have performed experimental evaluation of techniques using appropriate methodology involving simulation-based, observational, or interventional experiments.
Our work on feedback within a task has introduced novel techniques that can provide explicable grade points and actionable hints. Regarding explicable grades, we have developed a new reward design technique that can assign interpretable and informative rewards to guide the learning process in open-ended domains. Regarding actionable hints, we have introduced a series of techniques to automatically generate high-quality hints for Python programming. In particular, these techniques leverage recent advances in generative AI and large language models. Moreover, we have introduced a new form of quizzes for visual programming domains that can be useful in providing more engaging feedback to a learner.
We have also designed new computational models to capture the learner's knowledge and the execution behavior (i.e. how the learner solves a task using current knowledge). In turn, these models enable simulating and comparing the usefulness of different interventions when designing curriculum across tasks or providing feedback within a task.
We have demonstrated the performance of our techniques in a wide range of open-ended learning domains, including visual programming and Python programming. We have performed experimental evaluation of techniques using appropriate methodology involving simulation-based, observational, or interventional experiments.
The TOPS project addresses the fundamental societal challenge of providing cost-effective and inclusive education that fosters computational thinking and problem-solving skills. As a follow-up to research publications, we have worked with collaborators on the pilot deployment of our techniques that provide personalized feedback to learners during introductory programming. In the coming years, we foresee more opportunities for further interdisciplinary developments and the societal impact of this work. Beyond scientific and societal impact, the project will also help break new grounds and foster collaborative opportunities across different scientific fields, including cognitive science, computer science, learning sciences, neuroscience, and psychology.