Nobody's answered this question as I begin to write. We're getting into philosophy here! So I'll try to stick to philosophy, even though I could get into religion quite easily.
Most people believe that there is some sort of line beyond which a person's behavior, including his/her choosing behavior, should not go. Where that line is they argue about. However, even the most individualistic-minded people, who believe that they SHOULD be able to do whatever suits them, cry, "NO!" when something somebody else chooses to do affects them in a bad way.
"It's my choice" implies some freedom but does not imply an unlimited amount of it. I think it was Sartre who said that if you don't have absolute freedom you have no freedom at all, but most people don't use that thinking in real life... unless they're teenagers.
Some folks worry too much about their choices, and some should worry more. Stopping and thinking about all the consequences before making a choice - that is, worrying just the right amount - is what we call maturity.
Freedom and responsibility are so very intertwined that saying "It's my choice" always has fine print attached. My choice of chocolate over vanilla, or of $100 boots over $25 boots (assuming I can pay for them) may not be anything for someone else to worry about. My choice to fly to Europe next week and the heck with everything else - or my choice to do drugs - or my choice to have an affair with somebody - has repercussions on my husband, my children, my grandchildren, my other relatives, my friends, my neighbors, the organizations I belong to, and (in some sense) people I don't even know.
And if I can fool them or badger them into thinking they're wrong to feel affected, or into believing they shouldn't really count in the matter, shame on them and shame on me. That's not a popular take on the subject, but it's where I stand.
On the other hand... if we didn't make, or think about making, choices that influenced other people adversely, novel and film writers would be out of work! :^)
So ask your husband questions. And ask him to ask you questions, too.
I hope you don't have to ask him about something awful.