Bytecode questions

[jeruku]

1) Is there a complete table of Bytecode values for WoW, say like a simple index and value table?

2) Would there be a way to gut a word and make it a byte code algorithm?

3) And if possible, how would one go about implementing one, the other, or both?


I read through Carbonite's code and found it rather interesting. After about 10 minutes I thought maybe one might be able to read/write bytecode as words, reducing the size of bloated file databases. I suppose ultimately I'm asking if one could "compress" character data using the WoW API bytecode functions.

[Farmbuyer]

What, exactly, are you referring to as "bytecode"?

That doesn't mean what you think it means in a Lua context. Or at least, not in a forum about Lua addons. Perhaps we can clear up any confusion.

[jeruku]

Originally Posted by Farmbuyer What, exactly, are you referring to as "bytecode"? That doesn't mean what you think it means in a Lua context. Or at least, not in a forum about Lua addons. Perhaps we can clear up any confusion.

Well in the context given it involves the string byte code to be precise. As seen in the ever popular addon Carbonite where it's used to condense values and coordinates. My question is whether or not it's possible to condense a "word" string such as that "Sharptalon's Claw" could be saved as something smaller in a "byte" string.


Also, I figured out the answer to my first question. This, when used in an addon, will output the values in an indexed format directly into the addons saved variables file and print it to the current chatframe.

if not aSavedVariable then aSavedVariable = {} end-- some SavedVariable table

aSavedVariable [0] = string.char(0)
print(0 .. " = " .. strbyte(string.char(0))
--tinsert(aSavedVariable , string.char(0))

for x = 1, 255 do
  aSavedVariable [x] = string.char(x)
  print(x .. " = " .. strbyte(string.char(x))
  --tinser(aSavedVariable , string.char(x))
end

[Nimhfree]

You could take your entire "dictionary" of words (names of quests of NPCs in this case) and compress them using whatever compression technique you want. You would then uncompress them in the running Lua code. Whether you choose to keep them compressed in memory and uncompress them on the fly, or uncompress them in memory depends on what tradeoffs you want to do -- time vs space. Personally, I do not compress any of the quest names nor NPC names that I use in my addons, but I have not made a study as to how much space I can save if I did nor how much overhead the uncompression takes. Also, being able to search through the names may be more difficult were they kept compressed in memory.

[Phanx]

Also, keep in mind that Blizzard's addon policy prohibits any kind of code obfuscation. Addons like Carbonite get special exceptions in writing from Blizzard, but generally, you should not be using obfuscating techniques like string compression unless you actually need to. For example, you would want to compress long strings for sending them over the addon comm channel. However, it is unnecessary to compress anything in your addon's saved variables. The disk space savings will be insignifcant at best, and any fractions of picoseconds you shave off the file read time will overshadowed by the additional time it takes to decompress the data.

For your specific example, a better solution would be to store the item's locale-independent ID. The ID for the Sharptalon's Claw item is 16305. It's shorter, and will work for everyone no matter what language they are playing in. Then, wherever you currentply parse the name out of item links to check against your data, parse out the ID instead.

[Nimhfree]

Just to reiterate one thing Phanx mentioned: store the locale-independent ID for anything you are dealing with, spells, NPCs, etc. A side benefit of this as experienced in Wholly is you can save TomTom waypoints in English, for example, but when you start up your client in Spanish next, those Wholly/TomTom waypoints will now be in Spanish. This is because Wholly stores just the ID and leaves it to the client to fill in the blanks. Now, admittedly Grail is a lot larger than it needs be because it stores the localized names of all the quests and NPCs but that is because searching would be much more tedious if quest names were not stored (because they would have to be gotten from the runtime which takes a couple minutes at best), and targeting an NPC (or any other activity that uses the NPC name (like presenting it in a tooltip)) would not work since a client is not guaranteed to know the localized value of something unless it has encountered it already. But in any case, when Grail records the quests completed, or the spells you have experienced, e.g., it uses Blizzard's IDs for them which is a practice I would think addons developers should use.

[Phanx]

Using the IDs for quests and NPCs is not really reliable, since those IDs aren't available in every context where you encounter the quest name or NPC name. This is probably because Blizzard doesn't actually use those IDs in their own UI at all.

[jeruku]

By no means do I want to obfuscate, hide, obscure, or "muddy" any code. Which I don't think Carbonite does, I mean they did but they don't anymore. Just that in the case of a large amount of names flying around, like in Wholly/Carbonite, it's hard to store that much information without bloating.

The data it displays would be no different, in fact commands could be used to de/compress and output requested data. Displayed bright as day would be the logic behind the code, leaving no one wondering how to de/compress the data. In this case the only confusion possible would be the lacking of Lua programming. I mean, isn't distributing a compressed file breaking the policy then anyway?


I do know the value of a good ID number versus a table, I in fact want to make an itemtype filter without using a localized database.

Indeed, the only code I can find that would fetch "some" quest data using a QuestID and QuestLevel is this:
Which I'm not sure if you can use these links in FontStrings but if one could, perhaps manipulation of the ItemRefTooltip frame could be done to display information alongside notes instead. Compressed or not, the notes would then take up less space. The compression just being an option at that point.

print("\124cffffff00\124Hquest:2:23\124h[Sharptalon's Claw]\124h\124r")

[SDPhantom]

Originally Posted by jeruku I mean, isn't distributing a compressed file breaking the policy then anyway?

The policy is centered around being able to freely view any code and/or data an addon uses. Putting an addon into a .ZIP file does not violate this because every computer on the internet can open and extract a .ZIP file without any trouble for the end user. Opening a .LUA file is just as easy if not easier. They are just plain text and can be read by any text editor. Requiring to dig through your code to figure out your compression algorithm is a different story.

[jeruku]

Originally Posted by SDPhantom The policy is centered around being able to freely view any code and/or data an addon uses. Putting an addon into a .ZIP file does not violate this because every computer on the internet can open and extract a .ZIP file without any trouble for the end user. Opening a .LUA file is just as easy if not easier. They are just plain text and can be read by any text editor. Requiring to dig through your code to figure out your compression algorithm is a different story.

Meh, I got bored of trying so I moved onto greener pastures.

Though I've given up on the idea due to excessive math I'll still defend the idea. Is not a tool used to de/compress files, Windows(explorer.exe) or otherwise? Then why would this be any different so long as the community that chooses to view it's code can read it. I'm not talking about doing it like in Carbonite where there is minor descriptions of the "Unpack"ers. I'm talking about having an entire .TXT file dedicated to explaining it if needed(or simple HTML if permitted). If one were storing their own data, they'd keep an uncompressed version of the compressed data in a .TXT file.

Full disclosure of what, how, and why.
For example, I got sick of looking up small notes so I made an addon fully documented using my accumulated knowledge and some helpful sites. It ended up around 450 lines, of which only 70 or so are code, and all it does is load with a slash command.

[Vlad]

jeruku, just do what you want to do, if you step on someones toes you'll be told so relax. :P

I doubt anyone will care to stop you if you compress data to save space, go a head and enjoy coding stuff!

[tinyu]

I think if your going to turn your code to bytecode then you should include instructions on how to turn it back in a readme file or something.

[semlar]

In order to access the compressed data in-game it needs to be decompressed, so while you may be saving disk space and memory usage you're sacrificing cpu cycles and FPS, which tend to be more valuable resources.

The trouble is that memory usage is what's visible in-game and players will often mistakenly think that it's an indicator of how efficient an addon is, so I guess if you want users to think your addon is lightweight you can compress it which ironically winds up costing them more resources.

I don't know what the solution to this is, but I admit that I will personally take measures to reduce the size of a database if it starts becoming too large. I had to import a 3mb table once and the game wound up expanding it to use 40mb of ram. By changing the structure of the table I was able to get it under 10mb but it did add some overhead to the lookups.

[jeruku]

ace-serializer-3-0

libcompress


Just to re-iterate, these are exactly what I was talking about. I felt as though this may have been leading into an argument so I looked for more information. It had only struck me after my last post that I should look for a compression library of some sort, so first thing this morning I looked for one.

[Dridzt]

Yes but these are doing it specifically for the purpose of transmitting through the addon comm channels.

What Semlar said still stands, they choose to trade cpu time at the send and receive ends in order to limit the comm bandwidth.

Compression is still rather pointless when your addon is not sending or receiving.

[jeruku]

Indeed, for I am not blind and possess the capacity to read as well. I've made appropriate notes and have considered using the libraries as they are intended when and if the time comes.

My thought WAS(past tense), to store a large number of words for each quest much like LightHeaded alongside something like Carbonite blended with Grail.

Just a passing thought of impulse and whimsy. Passed by but woefully not forgotten, as some make abundantly clear. Now I believe this discussion is quite finished so we may all move toward greener pastures and be better for it.