Hi U.,
My name is S. and I am a language teacher. I teach foreign students who are learning English K-8. I hear about your situation frequently. It would seem to me that 4 hours per day is not enough to expose them to German, hence their complaints. (I assume that the rest of their day, they are with Enlgish speakers.)
You could try having them watch German language cartoons on TV and not necessarily when you are with them, and have children's German books in the house that their caretaker can try to read to them. Play games with them in German. Explain directions in English and don't be afraid to use English here and there. It will only help them in the beginning. You can make German bingo with pictures! Play Simon Says in German. Draw expressive faces on paper and tell them which faces are happy, sad, surprised, etc. and then make a games with it. When you do daily activities, i.e, like eating or brushing teeth, tell them in simple and clear language what they are doing day after day. Use gestures and point to pictures as much as you can when you speak German. You have to make it enticing and fun for them if they find it a burden to try to understand you.
Code switching, which is moving back and forth between, for example, German and English, is effective for language learning. So if you know they know the words for "Please" and "milk", in German for example, When you say, "Please drink your milk", replace those familiar words with the German and keep the rest English. If I could speak German I would write it out for you! As they know more words, you can "scaffold" and replace more and more words into German. Don't rush it!
What works quite well also, as you probably know, is to take them to Germany for a few weeks; a sure way to get them speaking!
You are up against quite a struggle, though, if you are not there more than 4 hours a day and kindergarten is not too far away. The trick for true bilingualism is to try to have them know how to say the same things in German as they can say in English. This is difficult when they are learning English vocabulary at warp speed once they get into school. Again, keep German books in house that are similar to what they are exposed to at school and read to them, but don't make it a chore.
I am not sure where you are living, but I live in Massachusetts. Here, and in many states, when a child registers for public school they must fill out a home language survey. If you write that you speak German to your children, they will be on the radar for language testing, even if they do not need it. The testing is done to determine if any entering child needs ESL services. That's just so you know.
There are many good books out there on raising bilingual children and also sources on the web. The benefits to having your children bilingual are countless. There are many brain studies on children who are bilingual that are very interesting.
I hope that helps a little!
Good luck!
S. Sheasley