Be patient and keep trying. Like the other moms have said, it takes a few days for your milk to come in. The colostrum you're giving him now is really beneficial.
When my son was born via emergency C-section, he had some minor breathing problems, but just enough to land him in the Special Care Nursery at the hospital. He was under an oxygen hood for a while. While there, the nurses gave him those asinine 1oz bottles of formula that leak everywhere when you turn them upside down, even though I TOLD them I wanted to nurse him. After my 4 days in the hospital I had to go home without my baby because he wasn't nursing as well as they'd have liked. (insert swearing here) I came back the next day and kept him with me the whole time so the nurses didn't have a chance to give him those bottles. He did go home with me that 5th day, nursed like a champ all day long too.
He is now 10, and STILL if things don't come super easy to/for him, he gives up before really trying.
Point of my story: Keep trying to nurse. You guys WILL get it, sooner rather than later.