K.M.
E. - it's probably just a phase that your boy is going through. I would just remove the bath mat and not press the issue. There's something that makes your son uneasy about it and rather than "forcing" it on him or trying to acclimate him to it - just get rid of it until he's a few months older. In the whole grand scheme of things, he's not going to be fearful of bath mats when he's 5 and it's really not too much to just remove it until he's a little older.
You know your boy best though and I'm sure you'll make the best decision for him and your family.