Gas exerts pressure on the walls of the container in which it is kept. When we blow a balloon, we fill the balloon with air. When air enters, it pushes the walls of the rubber balloon and stretches it. When we fill more air in the balloon, it applies more force on the walls. And pressure is average force acting on a unit area of the wall of balloon. 

That is the example of gas applying pressure in balloon. If you consider our atmospheric pressure, assume a long cylindrical column of air which extends from surface of earth to very high altitudes. The air inside this cylindrical volume will have some mass whose weight acts on earth surface or anything above it.. That is the atmospheric pressure.