If your tie tag was broken LONG TIME ago, then you may say say that. But since you are interested in saying that in the PRESENT time (tense), it is better to say:
My tie tag is broken, so I can't wear it now.
This means that the tag is still broken at this moment, so you cannot wear that.
You may say this - even better :
The tag of my tie is broken, so I cannot wear it now.