I would invite them to stay the night..we have done that before. But that may not be in the hand of cards you are willing to play. May just add more stress.
Sooo, this is what I would do. Give them step by step instructions. Reiterate the need to be on time because of parking. Tell them they can meet at your house before the game but if they are not there by __?__ a.m. then your car is pulling out of the driveway and they will have to refer to directions.
Once game starts then you are engaged in the game. NO LEAVING to find them. You have got to get your husband on board with this part. It is not yours or his responsibility to rescue them when they are careless to your instructions. Your responsibility is to your husband, the kids and the game. Tel them up front that if they can't find you then to meet back at your house for brunch or lunch. If they miss it then have the kids tell them the highlights. Put a smile on your face and say you are sorry THEY did not make it to the house in time to follow the family, or that THEY got lost and that THEY will have to try again sometime.
Do it all kindly and with a smile on your face. Their being potentially pissy should not effect you, your husband or the kids. You cover your end..and try to have them the best informed and then the day of what happens is in their hands.
I wish our family would show more interest in coming to our kids activities,sports,recitals and such. But..then it opens up another can of worms. They do what they can..and we are glad when they come. But I have learned to not let others' issues get my blood boiling..or interrupt our already planned day.
Good luck and best wishes!!