Just do whatever you need to do to protect you and your daughter. If you don't want joint custody, fight for what you want, your daughter is certainly counting on you to do what is best for you and her. I also divorced when I had a young baby and that is exactly what I did. I fought for the things I thought important. And, if he is an absent father, he probably won't fight you on that issue anyway. He cannot leave you hight and dry without anything, regarless of wether the things in your home are "his" or things his family has given you. You have a marital home and at least half of what is in that house is yours. And although he is filing for divorce, and may be an absent father, is family is likely to want to continue their contact and care for their grandchild.
Find an outlet to discuss how you are feeling. It's important to protect your daughter but also allow yourself some venting, crying, whatever it is that you need. I certainly know how you are feeling, I was 27 when I went through that myself. Feel free to contact me should need someone to talk to. ____@____.com