Ok A.... first of all, I doubt your husband notices ... and if he does, I doubt he cares that one is droopy. I've come to find that our husbands are so excited to see us naked that they don't care about our imperfections. .
As women, we are our own worst critics and I think the underlying problem is that you are too self conscious of the issue. As someone else said, the majority of women are lopsided, as am I!
Bottom line... if you think it will make you feel better, do it! But do your research and find an EXCELLENT surgeon because a lift is a lot of work and you want to make sure the scarring is minimal. The surgeon you choose should have pictures of his work from beginning to several years later to show the healing process and fading of the scar.
Good luck hun! And stop being so hard on yourself!