You've got so many good replies, I didn't really need to reply as well, but I couldn't help myself, as your message really spoke to me. I think you've got enough on the poop, so I'll concentrate on how you are feeling.
I was 32 before I had my son, and because I developed health problems related to the pregnancy, I sometimes worry that I don't have enough energy and think maybe I shouldn't have waited so long to have kids!! So, I guess the moral of this story is that no matter what we do, we can't really win! There are so many different benefits to being young, and there are obviously a lot of challenges.
There unfortunately seems to be a bit of a stigma about really young mums, but I think its often out of fear or ignorance. Those people who judge us, no matter what they judge us for (not being good looking or successful or religious or whatever enough) are just reacting to the pressures of their own lives. Its not their fault that they don't understand people who are different, but that's just it - they don't get it, and they may never. I know its really hard, but know that they are not the law! They don't get to decide what you do with your life or how you feel about yourself, your son, your wonderful family and the world - you do!
The fact that you aren't super confident with your baby is probably what he's sometimes reacting to. I think most mums have been through similar times, wondering what's going on and so you are not alone in this, nor do these feelings necessarily mean that you are depressed! (although its not the end of the world if you do feel as though you might be truly depressed, as there is excellent help available now, and its nothing to be ashamed of)
Might I suggest that you try this simple imagery technique a few times to see what happens? Just before you bend down to get him, think about the most lovely moment you've spent with him recently, or the cutest photo of him, or just something that you adore about him and you'll probably feel a smile coming to your lips, and the warmth spreading through your heart. It might be good to practice this a few times when he isn't crying or when you aren't around your son. Once you've got the hang of tapping into your loving feelings, give it a try when he's crying sometime. Ask your Dad to let you try and comfort him. With this smile on your face and in your heart, the love that comes from you will encompass him and he will almost definitely respond!
You've probably already realized this yourself - children and babies are incredibly sensitive to our moods - I have a tendency to be stressed, and my son picks it up in an instant, and has trouble getting to sleep when I feel stressed. I've been working on using the same technique when I'm trying to put him to sleep etc., and its really helping.
Good luck, and know you've got all the support of all of these people on this site who know you will get through this, and have a wonderful relationship with your son. He's so lucky to have a good relationship with your dad as well!!!
Feel free to send me a message if you'd like any further info on all of this.
Namaste,
A.
P.S. You haven't mentioned your mom, do let me know if you'd like to email any more about this or anything.