If I understand correctly, when you miss clicking on something by a few pixels, you just want your click to be ignored, instead of clicking on whatever is in the game world behind the button you tried to click.
Other than the obvious "stop clicking your buttons" recommendation
the simplest solution would probably be for you to use kgPanels. Set up a panel
behind each action bar or other object you're clicking on. Make it transparent so you can't see it (or give it a visible background if you want) and enable the mouse on it, but don't set a click script on it. Then, when you miss the button, you'll click the panel instead, and it will basically be like you clicked on nothing, because the panel will catch your click and do nothing.