Our visual system is equipped with a powerful plasticity mechanism, perceptual learning, which serves to improve perception of consistent inputs. However, the signals the visual system receives are extremely variable. How variability affects perceptual learning is unclear. In VarPL, we ask how the visual system tackles the challenge of variability for learning: variability could impair perceptual learning, or, like in language and motor learning, result in the ability to generalize from trained to new materials. To create effective training programs, e.g. for clinical applications, it is crucial to know how to reap the benefits of variability, or, conversely, to overcome the challenges variability poses. Yet, the neural mechanisms by which the visual system copes with variability are unknown, hampering this endeavor. To close this gap, we pursue a new theory, derived from the architecture of cortex: we hypothesize that perceptual learning is not limited to early visual areas, as traditionally envisaged. Instead, learning flexibly occurs at a ‘sweet spot’ along the visual hierarchy whose functional properties match the variability in the given environment. To test this theory, we investigate the role of stimulus variability during training. We hypothesize that variability during multi-day training pushes plasticity to higher brain areas in the visual processing hierarchy, whose tuning properties support generalization. While previous studies in visual learning have shied away from variability, we actively explore its role, and unravel previously unknown flexibility of the visual system that provides a path forward to application of perceptual learning in applied settings, e.g. for vision restoration.