A harbour town is subjected to land and sea breezes quite frequently.

During the day, land of harbour town heats up more quickly than the sea. So the air over the land is hotter and hence at lower pressure as compared to the sea. So, hot air moves from the land to sea. This carries away some heat.. This prevents excessive heating in the harbour town.

The situation is exactly the opposite in night. Land cools down quickly than the sea. So, hot air moves from sea to land. This prevents excessive cold in the harbour town. 

However, on a town in a desert located on the same altitude, such land and sea breeze have no chance.

Sand of desert heats up quite rapidly and the air is quite hot and at considerable low pressure.  However, due to absence of any high pressure regions nearby, the hot air continues to stay at the same place.

During night, sand cools down quickly, and loses all the heat. The absence of any low pressure regions nearby cause excessive cold.

Thus, climate of a harbour town is more temperate than that of a town in desert at the same altitude.