Ran into this bug in ASA 8.4 where access ASDM through vpn would not work, even through relevant management was correct and I was exempting my inside network from my vpn pool. The fix was adding “route lookup” to the end of the nat exemption.

nat (LAN,TWC) source static obj- obj- destination static obj-vpn obj-vpn route-lookup