Buttons are always rectangular. The round button is just an illusion. To give your button the illusion of a round button you need your round border texture to overlay the icon and the cooldown aswell. To make it work make use of setpoint. The CooldownFrame is anchored to the icon mostly, so changing the setpoint of the icon will affect the cooldown frame aswell. Every icon has a button or frame it is achored to.
Example
Code:
icon:ClearAllPoints()
icon:SetPoint("TOPLEFT", frame, "TOPLEFT", 3, -3)
icon:SetPoint("BOTTOMRIGHT", frame, "BOTTOMRIGHT", -3, 3)
If your icontexture has round borders...my friend you are in trouble you shouldn't have done it this way. Overlaying the icon with a round border texture does work though.