Things like that are definitely identified and dealt with in our home. There's no reason a 3 and 1/2 year old should feel it's okay to talk to anyone like that. If you allow it, you will most likely continue to see it grow through the years. It needs to be taken care of ASAP using whatever discipline technique your family uses. It's always important to explain the reason for discipline and give lots of love after. We have ours apologize too, but he has to know what he's apologizing for. He's hearing it from somewhere, or he's seeing an attitude like that from somewhere whether it's home, school, tv, neighbors, so you may even want to ask who he heard talk like that. Like us, you may find that there's something you or your husband has said that he caught on to even if it's something uncommon. We found that happened in our home just recently with something small, but it still opened our eyes.
Regardless of whether other kids his age do it or not, it's something that should be taken care of at this early stage. Good luck with it. You can do it, but stick to your guns whatever the disciplinary action is. Let him know ahead of time that if he talks like that again he will receive whatever specific consequence you choose and don't veer.