Sound is inherently a temporal and sequential signal. Experience with sound therefore may help bootstrap – i.e., provide a kind of “scaffolding” for – the development of general cognitive abilities related to representing temporal or sequential patterns. Accordingly, the absence of sound early in development may result in disturbances to these sequencing skills. In support of this hypothesis we present two types of findings. First, normal-hearing adults do best on sequencing tasks when the sense of hearing, rather than vision, can be used. Second, recent findings suggest that deaf children have disturbances on exactly these same kinds of tasks that involve learning and manipulation of serial order information. We suggest that sound provides an “auditory scaffolding” for time and serial order behavior, possibly mediated through neural connections between the temporal and frontal lobes of the brain. Under conditions of auditory deprivation, auditory scaffolding is absent, resulting in neural reorganization and a disturbance to cognitive sequencing abilities.