Hi there, I also have KP. Lucky us :) You don't have to go to a dermatologist because there is nothing they can do - there is no "cure". But you can get OTC products that have made a big difference for me.
I use a product called Glytone Exfoliating Body Wash with a shower poof on my arms and thighs each day. I get it from a website called www.dermstore.com. It's $22, but you only use a tiny amount each day. Also, the website gives you a 10% discount for your one "favorite" product and they have free shipping. In addition to the Glytone, I use Eucerin Plus Intensive Repair Body Creme (from any drugstore) every morning after my shower. When I use it at night, too, I notice even more of an improvement. The condition will never completely go away, but I have far fewer bumps - it's just a little red and splotchy now. I can deal with that. The Dermstore website sells other lotions specifically for KP, but they're also really expensive so since the Eucerin seems to be working, I'm sticking with that.
Also, the Eucerin has alpha-hydroxy so it says to also use a sunscreen, but I have to admit the UV rays of the sun help to improve the condition so I don't use sunscreen on my arms or legs unless I'm going to the beach.
Having KP sucks, but after a lifetime of feeling ugly, I finally feel like something has helped. Good luck!