So I found this great product which works almost like magic at my local Wal-Mart!
It's called Krud Kutter and costs about $5.50 for a 32 oz. bottle. You just spray it on and wait about 3 minutes...then I used a putty knife to scrape off the residual glue with absolutely no damage to the wall.
You can also use it to remove gum from clothing, mildew stains, blood stains, cosmetics and more...well worth the money!!!