You must start with 2 brownies.
At each toll bridge, you give the half of brownies you have, i.e. 1 brownie. However,since the owners give back one brownie and have received only 1 brownie, they ought to return it.
This way, you give 1 brownie to each owner, he returns it back to you and you finally gift 2 brownies to your friend.