My younger son totally refused a bottle even when my husband would try to give it and I wasn't around. Some babies just naturally prefer Mom and that's actually quite normal - they want the physical comfort of the skin to skin contact. So, you might try giving him the bottle that way OR have dad use a blanket that has your smell.
I did wait until he was 4 weeks old to ensure I had adequate milk supply - he was also a 9 lb baby, so that doesn't necessarily mean that Mom can't produce enough - frequent nursing (2-3 hrs during the day) AND no longer than 4-5 hrs at nite has been shown to be the key to success. My son gained 1 lb per week for the first two months and then slowed to 1/2 lb per week for the next several mos and then slowed down once he got mobile.
I will note that my older son had NO problems at all with the bottle, but he did have problems latching on nursing (he was in ICU for 3 days). So, my experience was absolutely that success with one approach does not necessarily lead to success with another approach.
Also, if you can't get him to take the bottle, you might find that you feel you're spending too much time nursing early on BUT in the long run, this saves a whole lot of time - no prep to go anywhere and babies at this age are extremely portable. Even when they're older a sling works great and in my experience NO one minds a breastfeeding infant that accompanies mom for the most part - certainly, the symphony or a black tie dinner might be the exception.
Also, in the early weeks the nursing actually provices Mom a chance to get the necessary rest. Personally, I really looked forward to the time to sit still in peace and quiet and read a book or just ponder the world. Once your baby is older that will be a very rare luxury.
Good luck!