Just wanted to send a virtual hug [J.]. I know how you feel. My son is almost 4 and has always been very temperamental. He had intense colic for his first 4 mos. and I've read that colicky babies are often a little more difficult for the long haul. I was a colic baby, myself, and I remained far more difficult, willful and such, than my younger brother. Some of it may be the stressful relationship, but I've also learned that babies are just different: some are easier, some are more difficult. Temperament may really be partly genetic.
But, since we can't change our child's genes, there are some environmental factors to consider. There's lots to tell, so I'll itemize! :)
SLEEP AND TIREDNESS:
Like some of the other responses, I've discovered that his sleep habits or tiredness definitely can impact his temperment. One mystery involved for me, was learning that he doesn't always fit a schedule. On some days, he needs more or less sleep, or needs to nap earlier or later in the day. I wasn't always making the connection between his attitude and tiredness if it seemed too early for his usual nap. And if I waited too long, he'd then be overtired, making going to sleep all the more difficult for him, insta-tantrum.
BLOOD SUGAR:
One wise woman once told me: "Feed him peanut butter when he does that." :) What she meant was, when their blood sugar dips, they can act like little tyrant nutcases, and a spoon of peanut butter, with its fat and protein content can work like magic. It's definitely worked for much of my son's fit-pitching. I've discovered: he just needs a snack! Again, somedays he needs more or less food, and at different times, so depending on a strict meal and snack schedule can be a recipe for disaster.
[NEGATIVE] ATTENTION/COMFORT:
The third pattern I've noticed involves his need for attention or comforting. Somedays, just like for adults, he seems to just wake up on the wrong side of the bed, a little grumpy and prone to irritation. On those days, it pays to give a little extra comfort and attention, _before_ the tantrums set in, or at least after one has blown over. I try not to time the extra attention with any ill-behavior so he doesn't make a connection that he's rewarded for it.
I tend to do as you mentioned: send him to his room to calm down. I've tried ignoring him completely but he's one stubborn fellow and can go on and on with it, building up both of our frustrations throughout the day. So, sending to his room to calm down is better for us, and sometimes he, too, will go in there and screech for 10-15min, though that seems to be reducing with time. Now, after he's calmed down, I try to offer some quite together play, like reading stories, or playing a board game together. I'll offer a snack or meal, and eat with him, watch some tv together and cuddle. This sort of thing has worked wonders for redirecting a bad day, calming his temperament, and reducing tantrums.
BOREDOM:
Thankfully, he doesn't do it much in public, which is another indicator to me that it could be a boredom thing... he has lots to keep his attention in a bright, busy supermarket... and he has face to face close proximity interaction with me from sitting in the cart and conversing together. At home, if I get busy doing other things for long periods, I think he sometimes just gets plain bored and hungry for my attention, which is totally natural for a small child. I think it's a mechanism built in to them to keep us on our toes! Therein lies the key to the mystery of "negative attention"! haha
I'm a full time student, so it's easy for me to get my head stuck in homework or housework and leave him puttering about the house on his own. And the past 6 months or so, he's reached a point of maturity where he really does enjoy a lot of solitary play, building blocks, colouring, pretend games with his toys. So, I don't feel I must give him constant attention, but setting aside short periods of regular one on one together play time throughout the day has made a big difference for the better.
CABIN FEVER AND OUTDOOR PLAY:
I can also see a big difference in getting him outside for some play and running around in fresh, open air. Just like adults, kids can get "cabin fever." Going to a playground, or even just to the back yard can help diffuse some of that pent up toddler energy and help calm them down for the day.
TV:
What he's watching on TV affects him, too... some of his favorite shows, though perfectly educational, can be pretty very lively and wind him up. If he's having a day prone to tantrum or bad attitude, I'll either turn off the tv altogether or will limit his tv time to calmer shows and movies, like Oswald or Caillou. Turning off the tv can cause a big fit, but within about 10 minutes, I'll hear him playing quietly and creatively in his room, happy as a lark.
SUGAR:
Oh, and finally: sugar. If he eats much sugar, he is usually no fun to be around, grumpy, demanding, irritable. Yuck. So, I try to avoid sugary snacks and instead feed his sweet tooth with fruit, smoothies, and fruit yogurts. Fruit on the Bottom is the least sugary, it seems, without resorting to stuff like aspartame and splenda, but you can also use plain yogurt mixed with fresh fruit mashed up with a smidgen of honey, fructose, or all fruit jam.
Good luck! It's hard and frustrating, I know... and it seems no single tactic works all the time perfectly, but these are the tools in my kit that have helped out a lot.
B.