Thanks for the replies, I think I may have figure it out.
The problem seems to go away when I don't load the FluidFrames/Portfolio addons. However, I like being able to move around and arrange all of the little buttons.
Does anybody know of another addon similar to FluidFrames/Portfolio that doesn't ruin the map? (or perhaps a way to fix them?)