Water molecules exhibit a special property that solid water molecules are a little further away in the crystalline structure of ice than in the liquid form.
In water liquid, the hydrogen bonds that exist between neighboring molecules make them closer to each other. Hence water molecules are closer in liquid than in solid form. Thus the density is higher for water than for Ice.
As the temperature increases beyond 4 deg., the density decreases due to thermal expansion of the liquid and this overcomes the hydrogen bonds.
So on either side of 4 deg C, the water molecules are more loosely packed as compared to water at 4 deg C.