Possible bug with %d in all format strings
So consider the following image here:
What I was trying to do was to show an integer in a string using fontstring:SetFormattedText() and %d. And for some reason it keeps returning the wrong value. The second value in my debug output is the value I put into the format string. And it just displays as something else.
The value I at first put into SetFormattedText was the calculation itself, and I wondered if that in some way was the issue. I then attempted to debug it by outputting the value, the result of the calculation (multiply by 100), and a string.format with the finished value put into a %d inside a string. And still 130 would turn into 129 when shown on screen.
I mean... really? Am I this dumb, am I missing something super obvious? :confused:
THEN... I tried replacing the %d with a %.0f, a floating point with no decimals. Still the same parameters put into the string.format function, I just changed the format string. And would you believe it... now it's right...?
Some more backstory; The value I calculated from moves from 0.5 to 1.5 in steps of 0.1. It was when going downwards the error occurred.
So I'm just assuming that numerical calculations with floating points for some weird reason aren't as accurate as they should be in WoW Lua, and that some rounding functions keep the values "right" while using them as values, but when put into a format string in a %d they just cut off the hidden decimals instead of rounding up? I tried using floor, I tried deducting the result of a modulo operation (which is to floor it), and nothing would fix it when using %d. Seems like once a variable has turned into a floating point, it's screwed.
Anyway, none of this should happen at all, should it? I just thought it was a really strange bug. And no matter how I try I can't see how the problem was my code. I mean, if a value is 130, says 130 when you print the value out, it should say 130 when put into a "%d" format string, right? Not 129. :(
For now I have replaced every single %d in my UI with %.0f. Might be a bit of an overkill, especially if it turns out I've missed something obvious here. But if not, and it really is a bug going on here, I'd rather go for "safe than sorry" until/if the bug can be sorted out. At least my overkill workaround gives me addons that work as I intended. :)
Since you didn't post the actual code, I can only imagine where things go wrong.
What I can tell you from my own experience is that when using frames/textures, certain number values like scale, size, position can behave a bit odd, caused by various things like the UIScale itself etc.
e.g. instead of anchoring the frame at an offset of 130, the offset actually is 129.9999999...
or setting an object's scale to 1.2 actually scales to 1.199999...
This was my code used in the example above:
My debug output was of both the self.scale and scale values, as well as a string.format output of the same text just to see if it deferred from SetFormattedText().
My self.scale value is only ever added or deducted the exact value of 0.1. But when going down from 1.5, the weirdness started happening. My self.scale variable printed out 1.5, 1.4, 1.3, 1.2 and so on, the scale variable printed out 150, 140, 130, 120 and so on, but the result appearing either in a string after using string.format on it, or directly to the screen in SetFormattedText turned into numbers like 129, 119 and so on.
I thought about how WoW stores positions and scales too, but that's just not the case here. I'm not printing out positions. I'm not retrieving any of these values from GetPoint or GetScale. It's all based on a fully controlled variable.
Appears to be the typical malfunction with floats but you can use math.ceil, your solution also works, to alleviate the problem. What is meant is that converting from/to floats may remove/add decimal values; at least this is what I myself have experienced.
as jeruku stated, it's a common precision bug.
It can be experienced with the ingame sliders aswell when using decimal values.
In my opinion, your best option to display the value is replacing %d with %.0f
You might want to use math.ceil to check the actual number, as jeruku suggested.
The overheat is minimal compared to the much better readability and usability.
see https://stackoverflow.com/questions/...581943#3581943 for reference
|All times are GMT -6. The time now is 04:27 PM.|
vBulletin © 2020, Jelsoft Enterprises Ltd
© 2004 - 2020 MMOUI