It is true that in most of the species especially mammals, the males are stronger, taller and bigger in size than females. It had been so since ages. This is true with in a society.
Women have oestrogen hormones while men have testosteron hormones which make them different in terms of reproductive function. Perhaps they also cause indirectly certain traits. The endocrine system regulates the body functions relating to metabolism, respiration , reproduction and others. Usually the growth (in height) stops in girls three years earlier than in boys. That could be another reason.
It is possible that during the time of apes the women preferred men who are bigger. It is possible that male species fought for females and only stronger survived. Due to the (sexual or otherwise) preferences of women, the taller, stronger and bigger male species survived. Such males caused more reproductions. It is possible that evolution took place in the direction of preference selection of women.
Since that time, due to the hereditary nature (ref: genetics), men have been always bigger than their contemporary women in any one particular society.