M.P.
I suggest, since he's able to sleep, to wait until morning to call the doctor. An ER visit is definitely not needed. Keep in mind that there are all sorts of germs in the ER to which none of you want exposure. The ER is for life threatening conditions. If his fever goes up to 105 or more than a trip is warranted.
He may or may not have pneumonia. I've had several upper respiratory illnes with those symptoms and didn't have pneumonia. They were all viral infections so I went home to drink plenty of fluids and to rest in bed. The one time I had pneumonia I did not have difficulty breathing or pain. I had had a low grade fever for weeks and was tired Which prompted a visit to the doctor. I was given aantibiotics and told to rest. It's rare to be hospitalized for pneumonia.
Know that a fever below 105 is a good thing. The fever is fighting the infection. We give acetaminophen or ibuprofen to make the child more comfortable not to cure the illness. I do not wake a child to give meds. Rest/sleep does more to cure than either one of those.
If his fever continues to go up and acetaminophen or ibuprofen doesn't bring it down and it continues to rise then it's time to call the ER advice nurse.
Since you are so anxious I suggest you sleep near him so he'll wake you if he needs you. You need sleep so you can deal with a sick child tomorrow and/or work.