That is because during the hot water bath we have had, the air nearby in the bathroom is warm. So our body feels warm.
if we have taken bath with cold water, still we feel warmer in the bath room than is the case if we walk to outside the bath room. This is because due to our body heat and the hot breath from our nose, the bath room is a little warmer. When we come outside the air is cooler than that. so we feel cold.