This is because a dispersion takes place in prism and light ray is refracted that is they are bent when it passes through one medium at an angle and changes its speed. The different wavelengths of light is passed at different speed this gives the color formation in prism. When the light passes through glass it has two interfaces one entering and other leaving. So the spectrum is formed by prism not by glass.