When a pendulum swings from side to side it will obviously collide with the particles in the air and they might vibrate a bit to produce some sound. But we dont hear it , and I think that this could be because of two reasons. The first is that maybe the vibrations of the air particles dont reach our ears. The second reason could be that the vibrations do reach our ears, but because we can only hear from 20 Hz to 20,000 Hz , we dont hear it. It is possible that other animals who can hear a wider range of frequencies , are able to hear the sound of the pendulum.