Sit down with your husband and go through the finances with a fine-tooth comb. Really and truly figure out the money angle thoroughly - including tax implications. You probably don't have any benefits with this job so you won't be losing them - what would you be gaining by becoming a full time employee? Would it pay retirement benefits? That is something to consider.
Then talk about the schedule with picking up kids and doctor appointments and the like. If you are going to lose your job for not working more hours, what's your employer going to do if you have to take time off for sick kids, etc? You can't fit everything into a Friday.
Then talk about the "picking up of the slack" stuff. Don't let your husband pass it off. Really sit down with him and make a list of what has to get done and how much of it each of you is going to have to contribute. Don't let it just "hang" with you doing it all plus the extra job load.
What you haven't said here is how much he does in the house now. Does he help? Has he? Would he actually be willing to, or would you be stuck doing it alone? What do you really think in your heart?
I will point out that your five year old will be in school and you won't be paying full time daycare for long. After school care, yes. That's not as expensive as full time daycare.
Will you lose your skills if you stop working? That's another consideration. You may very well need to keep those skills up.
Another thought is to look for another part time job in a similar setting. You could always start looking now before you lose your current one. I don't know if you can collect unemployment for a part time job loss, but you should look into that. If your employer pays unemployment on you, that is a benefit you have the right to use, so don't quit your job. Let them lay you off instead.
As far as guilt goes? I think it was in a Monty Python movie that someone said that guilt is like a bag of bricks - you just need to put it down and walk away. If you can do that, you can think more clear-headed about what to do here. You need to be considering the upside for your children of working - not just the downside. They will need to work one day themselves and it is good for them to see that their mom had a job. You are an example for them. However, working for the sole sake of contributing isn't quite enough if in fact you just aren't clearing much money. You have a lot of responsibility on your shoulders with two kids. So that responsibility is contributing.
Honestly, I've read some doozies here on both sides of the coin in regards to working and contributing. From one extreme of someone who makes SO little compared to her husband, but feels like she "HAS" to pay him for fixing her car when it breaks because it's her car, to the other extreme of someone who simply refuses to work because all she wants is to stay home with her child while her husband works two jobs and STILL can't make ends meet. There is a happy medium there and there should be. I will tell you from my 30 plus years of marriage, 15 years of full time work, 15 years of being a SAHM and traveling around the globe in my marriage, that if you allow your husband to think that you don't "contribute" just because you don't work is a MISTAKE. You need to see your own selfworth and make darn sure he sees it too.
Many couples end up supporting the career of one spouse so that it can take off, mostly when a company asks for the spouse to consider relocating. Even without that, sometimes the working spouse is more willing to look for a better job because of only having one salary. If it weren't for the supporting spouse, that big "break" into a higher salary and job might not happen. So never assume that if you don't work, that you don't contribute to his success. That doesn't mean that he gets to come home and put his feet up and not lift a finger with hearth and home. He needs to be invested in family life. It just means that you are worth plenty, whichever route you choose.
I know this has been a long answer. I hope it helps you.