I am both a divorced mom and a family law paralegal. Some of this will depend on your state of residence what the court will allow you to do. In most states, though, if the two parents agree on any kind of schedule, no matter how different, the court will accept the agreement and incorporate it into his Decree. A lot of courts do not like to have such extreme separations in time with either parent. Again, you'd have to check with an attorney in your state, but here, especially with how young your son is, the court would want him to be in your physical custody during each week and your ex having visitation every, or every-other weekend.
Now, that's the legal part of your question. Here's my mommy advice...
It doesn't matter what you and your ex agree on, as long as you can agree on ANYTHING AT ALL. you are already miles ahead of most divorced parents, who disagree on whether their kid should play soccer or basketball. If you and your ex agree on the schedule...there WILL BE A WAY to make it work. The most important thing is getting along as best you can for your child's sake. My ex and I argued for no reason for 2 years, until we finally put aside the hurt and anger. Now we get along pretty well, and even consider one another friends. We are both remarried. Since we have a somewhat friendly relationship, it make working out the to-and-from much easier with regards to babysitting and traveling to visitation. Good Luck.