M.C.
If he is happy and healthy otherwise, why not just let him go solo with you on trips? Not everyone wants friends tagging along 24/7 with no escape. If there is an issue with the phone, limit his time. You are the parent and you provide his phone. Give him set times he can use it. There are so many other things he can do to fill his time. Just to name a few: reading, drawing, writing, car games, trivia games, etc. while in the car or on the plane. At a destination? Even more to do! Shopping, conversing over a meal, taking a walk, visiting local sites, going to an activity like mini put, taking a swim in the hotel pool, etc. I can't imagine forcing a friend on him when he says he doesn't want one.