You know, I had a similar situation just last week! I have a friend, who I used to watch her kids (who are the same age as my kids) for her, and she has offered to watch mine, but I had never agreed. My issue was mostly that my son (16 months) can be quite a hand full and my daughter (3 years old) has a lot of separation anxiety, which stems from when I had to have my son by emergency c-section and didn't come home from a prenatal doctors appointment. Anyway I had an appointment that I couldn't take the kids with me, or it was recommended that they not be there. So I took her up on her offer to watch my kids. Honestly I was so worried they wouldn't do well, and would drive her crazy. When I got back to her house to pick up the kids, they were running around playing and did NOT want to leave.
As worried as I was, everything was completely fine.
I wouldn't worry about your kids feeling stressed. They may cry, for about 30 seconds, when you leave but they will be fine.