There's one quite easy thing you might do but you'll need a couple of things:
- a buzzer,
- a battery,
- two wires,
- a switch,
- a piece of cardboard (it's size depends on the place you wanna put your buzzing device),
- duct tape.
What you need to do first is check if the buzzer actually works - pin or tape the wires to your battery and then to the buzzer (you might need to switch the wires if you connected them the other way around. Remember: '+' goes to a '+' and '-' goes to a '-' ;)
When you're sure it works you can start fixing the stuff to the cardboard and connecting the switch too. Switch works on a very basic idea of closing and opening the circuit. If you don't connect the switch your buzzing-board will be buzzing till the battery goes dry and everybody in the vicinity goes mad from the sound :D
Think of your hiding place first though so you could fix the switch as conveniently as possible.
After that all you need to do is to place it in a good hiding spot, turn off and make your folks go in circles looking for the source of those extremely annoying sounds.
I hope it helped, cheers!