I understand why you would want to do something for yourself. Kickboxing is very rewarding and great exercise and it's empowering to you. That's very important.
It seems you've been placed in a position to "prove" yourself somehow, either by your own standards or those of your ex-husband. You comply with the court and do the right thing and it doesn't seem like it's enough to satisfy your ex-husband because it might seem your ex really doesn't want you to be happy or have a life of your own.
Your ex has placed high expectations on your reactions for a reason. Most of the time, those reasons have absolutely nothing to do with you but are a struggle for him to come to grips with so he takes it out on you. It's a guilt cycle for him (IMHO) and he's lashing out at you by making you feel guilty about doing something good for yourself (kickboxing) and using your son's feelings of being "excited" to see you as a shield to hide something he's not feeling right about. Make sense?
It's hard to suggest your next move on this. I can only try to explain what I would do. Go with your instincts. Spending all the time you can with your son is top priority and you might want to think about scheduling your Wednesday's off until you can get your ex to see reason. You might also want to look at the kickboxing class schedule again and see if there's another day you can rearrange to take in place of the Wednesday class. Swap a day for the Wednesday class.
I would also try to re-negotiate a win-win for the custody with the court system. Outsmart and outwit a situation to your advantage, all the while maintaining your integrity. Be nice about things but be firm. Try to see what's happening with the ex that might be causing all this tension. Ask open-ended questions (those that start with who, what when, where, how and why). Try to reason with him and explain that you need to work this out to both of your advantages as well as your son's. Ask him to stop putting your son in the middle (as you perceive him to be doing).