Quantcast Possible bug with %d in all format strings - WoWInterface
 
Thread Tools Display Modes
03-21-19, 05:19 AM   #1
Cogwerkz
A Deviate Faerie Dragon
 
Cogwerkz's Avatar
AddOn Author - Click to view addons
Join Date: Jul 2017
Posts: 12
Possible bug with %d in all format strings

So consider the following image here:



What I was trying to do was to show an integer in a string using fontstring:SetFormattedText() and %d. And for some reason it keeps returning the wrong value. The second value in my debug output is the value I put into the format string. And it just displays as something else.

The value I at first put into SetFormattedText was the calculation itself, and I wondered if that in some way was the issue. I then attempted to debug it by outputting the value, the result of the calculation (multiply by 100), and a string.format with the finished value put into a %d inside a string. And still 130 would turn into 129 when shown on screen.

I mean... really? Am I this dumb, am I missing something super obvious?

THEN... I tried replacing the %d with a %.0f, a floating point with no decimals. Still the same parameters put into the string.format function, I just changed the format string. And would you believe it... now it's right...?



Some more backstory; The value I calculated from moves from 0.5 to 1.5 in steps of 0.1. It was when going downwards the error occurred.

So I'm just assuming that numerical calculations with floating points for some weird reason aren't as accurate as they should be in WoW Lua, and that some rounding functions keep the values "right" while using them as values, but when put into a format string in a %d they just cut off the hidden decimals instead of rounding up? I tried using floor, I tried deducting the result of a modulo operation (which is to floor it), and nothing would fix it when using %d. Seems like once a variable has turned into a floating point, it's screwed.

Anyway, none of this should happen at all, should it? I just thought it was a really strange bug. And no matter how I try I can't see how the problem was my code. I mean, if a value is 130, says 130 when you print the value out, it should say 130 when put into a "%d" format string, right? Not 129.

For now I have replaced every single %d in my UI with %.0f. Might be a bit of an overkill, especially if it turns out I've missed something obvious here. But if not, and it really is a bug going on here, I'd rather go for "safe than sorry" until/if the bug can be sorted out. At least my overkill workaround gives me addons that work as I intended.

__________________
AzeriteUI
  Reply With Quote
03-21-19, 10:21 AM   #2
humfras
A Flamescale Wyrmkin
AddOn Author - Click to view addons
Join Date: Oct 2009
Posts: 131
Since you didn't post the actual code, I can only imagine where things go wrong.

What I can tell you from my own experience is that when using frames/textures, certain number values like scale, size, position can behave a bit odd, caused by various things like the UIScale itself etc.

e.g. instead of anchoring the frame at an offset of 130, the offset actually is 129.9999999...
or setting an object's scale to 1.2 actually scales to 1.199999...

Code:
/run local v=129.9999 print(v, string.format("%.0f",v), string.format("%d",v))
=>
129.9999 130 129
__________________
Author of VuhDo CursorCastBar OptiTaunt Poisoner RaidMobMarker

Last edited by humfras : 03-21-19 at 10:24 AM.
  Reply With Quote
03-24-19, 11:53 PM   #3
Cogwerkz
A Deviate Faerie Dragon
 
Cogwerkz's Avatar
AddOn Author - Click to view addons
Join Date: Jul 2017
Posts: 12
This was my code used in the example above:
Lua Code:
  1. local scale = self.scale*100
  2. self.scaleText:SetFormattedText("|cffaeaeaeScale|r: |cffffd200%d|r%% |cff666666(change with mousewheel)|r", scale)

My debug output was of both the self.scale and scale values, as well as a string.format output of the same text just to see if it deferred from SetFormattedText().

My self.scale value is only ever added or deducted the exact value of 0.1. But when going down from 1.5, the weirdness started happening. My self.scale variable printed out 1.5, 1.4, 1.3, 1.2 and so on, the scale variable printed out 150, 140, 130, 120 and so on, but the result appearing either in a string after using string.format on it, or directly to the screen in SetFormattedText turned into numbers like 129, 119 and so on.

I thought about how WoW stores positions and scales too, but that's just not the case here. I'm not printing out positions. I'm not retrieving any of these values from GetPoint or GetScale. It's all based on a fully controlled variable.
__________________
AzeriteUI
  Reply With Quote
03-25-19, 08:01 PM   #4
jeruku
A Cobalt Mageweaver
 
jeruku's Avatar
AddOn Author - Click to view addons
Join Date: Oct 2010
Posts: 200
Appears to be the typical malfunction with floats but you can use math.ceil, your solution also works, to alleviate the problem. What is meant is that converting from/to floats may remove/add decimal values; at least this is what I myself have experienced.
__________________
"I have not failed, I simply found 10,000 ways that did not work." - Thomas Edison
  Reply With Quote
03-26-19, 03:37 AM   #5
humfras
A Flamescale Wyrmkin
AddOn Author - Click to view addons
Join Date: Oct 2009
Posts: 131
as jeruku stated, it's a common precision bug.
It can be experienced with the ingame sliders aswell when using decimal values.

In my opinion, your best option to display the value is replacing %d with %.0f
You might want to use math.ceil to check the actual number, as jeruku suggested.
The overheat is minimal compared to the much better readability and usability.

see https://stackoverflow.com/questions/...581943#3581943 for reference
__________________
Author of VuhDo CursorCastBar OptiTaunt Poisoner RaidMobMarker
  Reply With Quote

WoWInterface » PTR » PTR UI Bugs » Possible bug with %d in all format strings

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off