I feel so sorry for you because I'm a grandmother now and went thru everything you wrote, plus some, with my now 36 yr. old daughter. The key word I saw you write, was "he's been difficult pretty much since he was born". This is what happened with my daughter. In my heart I knew that her reactions weren't normal but she was usually fairly "normal" when others came to visit. They couldn't believe the problems I was having with temper tantrums that were VERY frequent, lasting for several hours, spanking that only made her more angry, refusing to pick up her toys (and then falling asleep in the middle of the room after sitting there for 2 hrs. when I said she couldn't come out until her toys were put away), running away from time outs, sitting under our chairs and kicking us while we were still at the table eating, etc., etc.
We took her to a child pyschologist when she was 4 because she didn't outgrow the terrible two's, terrible three's, or terrible four's - it just got worse. In between these tantrums and strong willed stand off's she was the cutest, sweetest, most loving child you could ever want. A number of easy tests were done, we found her IQ to be 136 (very high), but she had mild learning disabilities in short & long term recall.
Things gradually got worse each year of school because she refused to attend special education classes (skipped going, even in elementary school), refused to organize her notebooks by color, and most importantly, she refused to admit that she had a problem.
Once she reached 16 yrs. old, by law we could no longer force her to receive help, attend counseling, etc.
To this day (36 yrs. old) she doesn't think she has a problem. I won't go into all of the adult issues she has.
We have a son who is 4 yrs. older than her and who never gave us any problems whatsoever. He put himself thru college (she is a H.S.dropout) and has a successful career. Both kids had the SAME biological parents and same upbringing.
I've read everything I can get my hands on over the years about ADD/ADHD and Bi-polar. I'm convinced this was/is at the root of her problems. She learned to self medicate with methamphetamine.
You have to follow your motherly instincts. YOU know what is normal and not normal. Please get your child both pyschological and medical help and don't let up until you get a diagnosis that you feel is right. I don't know about bi-polar (mood swings) but ADD/ADHD symptoms are usually present by the time a child is 5 yrs. of age. If the child/adult develops ADHD symptoms AFTER the age of 5 (but didn't have them earlier than 5) then they probably do not have ADHD.
I'm sorry this isn't very encouraging but with early diagnosis and the new medications, these things can be controlled and they can live a successful, happy, productive life. Don't wait!