S.B.
Sadly the only way here is to be honest. I would just tell them you're not comfortable (not necessarily because of their age) but because of their driving record. It's better to be honest and not let your son go and have him be safe...than to try to protect your inlaws feelings and have something happen.
I know it's hard. I'm in a similar situation with my inlaws who live only an hour away and are livid that we don't let our daughter stay over night with them. First, she's only 4...but my FIL is an awful driver as well and they just don't pay attention to our daughter when they do have her. I know they get mad at me in particular and their feelings are hurt...but I'd rather my child be safe. We used to side-step the subject all the time but then they just became more demanding and we had to come clean and just admit we don't feel comfortable. I'm sure it hurt their feelings...but they got over it and the wonderful thing is they've stopped asking.