There are probably some "more advanced" hand held games on the shelves these days, but we got our son the Leap Frog Leapster when he was about the same age as your son. It's educational and fun for them; they don't even know they are learning!! My son is 5 now and still uses it. There were some games my sister gave me that I was concerned about "frustrating" him as you mentioned. So I held off on those for a little while, and he eventually wanted to try them, figured some of them out on his own, and didn't want my help anymore!! I hope this helps!!
P.S. Be sure to invest in rechargeable batteries, as long as the device you choose permits the usage of them. GOOD LUCK!!