No "permanent magnet" is completely permanent. Heat, sharp impacts, stray magnetic fields, and age all conspire to rob a magnet of its force. A magnet gets its field when microscopic magnetic areas, called domains, all line up in the same direction. When the domains cooperate, the magnet's field is the sum of all the microscopic fields in it. If the domains fall into disorder, the individual fields cancel out, leaving the magnet weak.
There is no existence of permanent magnets in the world. Magnets lose their power when in contact with other magnet, iron filling and so on. But the main reason it loses its power is because of earthing, the magnetic power becomes less when kept on earth directly.