ParserLib

[rophy]

ParserLib is an embedded library which does all the combat message parsing and converting them into meaningful information.


To use ParserLib, your addon register events to ParserLib like this:

local parser = ParserLib:GetInstance("1.1')

parser:RegisterEvent("MyAddon", "CHAT_MSG_COMBAT_SELF_HIT", "MyAddonEventHandler")

very similar to what you did with frame:RegisterEvent(), when the registered event is fired, ParserLib will call your registered handler, but instead of arg1 arg2, it passes a table as the second parameter:

function MyAddonEventHandler(event, info)
  if info.type == "hit" then ChatFrame1:AddMessage(  info.source .. " hit " .. info.victim .. " for " .. info.damount .. "damage!" ) end
end

Of course UnregisterEvent() and UnregisterAllEvents() are available too.

That's it. You check the 'type' field to know what information are available in the table, and do whatever you want. Detailed description is in the documnetation.txt

All ParserLib do is convert the arg1 into a table, for now you'll still need to know which event to register.

Then what's the benefits for using ParserLib? These are what comes out of my mind:

• It already did the job to make itself work on all languages. So you don't have to worry about your parser failing in another localization. This is why I wanted to develop a library at the first place. It's so so so trouble to make sure your parser works in all languages. Many addons, even claiming themselves working in another localization, often fails to do so for some patterns.
  
• It only loads things when necessary, so the overhead is minimal. If you load the parser and register nothing, it loads almost nothing into memory other than the raw code. The event table is loaded into memory only at the first time that event occured, pattern table at the first time that pattern is required to parse.
  
• When multiple addons share the same parsing library, messages are only parsed once for all of them. Currently all addons do their own message parsing, which I feel is a waste of resource.

ParserLib is still in alpha stage, I think it's still not very ready to be released.


But I'd like the publics to help on testing, so I created two simple addons using the library:
Simple Combat Log and FuBar - DPS,

if you're interested on the project please go and download any of them to help testing.

[chuckg]

Any chance you could strip the ParserLib from your combatlog/dps monitor and post it as a separate entity? I'm a bit too lazy to pull the library out of your other mods!

[Nyrine]

OH, I like this !

Currently I do a bunch of combat log parsing myself, in my ProcWatchFu addon.

Using your library could make life a lot easier for me ! Thank you so much =)

Two questions though:

info.type is always the english ? "hit"/"miss" etc?
info.skill is the localized skill ? I would guess so, but you never know =)

[rophy]

Originally Posted by chuckg Any chance you could strip the ParserLib from your combatlog/dps monitor and post it as a separate entity? I'm a bit too lazy to pull the library out of your other mods!

The library is working well so far, but I'll need to finalize the variables in the table ( making sure that the variable name won't change after public release ), so I'm not releasing ParserLib itself yet. Please go download Simple Combat Log, the source code is short and simple, but it uses all info.type, so it's a very good 'hello world' example client addon.

Originally Posted by Nyrine OH, I like this ! Currently I do a bunch of combat log parsing myself, in my ProcWatchFu addon. Using your library could make life a lot easier for me ! Thank you so much =) Two questions though: info.type is always the english ? "hit"/"miss" etc? info.skill is the localized skill ? I would guess so, but you never know =)

type is always english,
skill is the string parsed from combat log, so yes it's localized, but it might also be a constant ParserLib_MELEE or ParserLib_DAMAGESHIELD, for the case of melee hits and damage shield reflects.

You can check documentation.txt in ParserLib folder for a complete description of variables for each 'type'. The txt file is still simple though, I'll write a more complete documentation soon.

[Grayhoof]

I'll be watching this closely. Once you think its more in a "release" state, I'll consider looking into switching over SCT/SCTD to use it. I got SCT working using the global strings, but I'll be the first to admit its probably not ideal and I don't maintain that part of it often.

[JoshBorke]

yes, i shall also consider switching to this mod for my combat parsing once it is more 'release' :-)

(not that anyone knows what my mod does ;-))

[Andalia]

I currently use Miles Combat Parser for my (private) mods.
It also works quite nice and im sure that it would have become a great addon if the autor didn't suddenly disappear.
I hope you release your lib soon so i can switch to it because i'm too lazy to do the parsing myself and using a beta mod isn't nice either.
Maybe you can get some things out of mcp that you have not included atm or so because im sure that the autor of it has found a pretty nice and fast way for parsing and the only thing it doesn't display afaik are glancing blows.

[rophy]

The documentation is finshed and I'd like those who conerns to take a look at here. It's also available in the ParserLib in Simple Combat Log.

ParserLib should be working fine now, but to know that it is "completed", I'll have to make sure ALL possible patterns are known and parsed correctly by ParserLib, which is hard to confirm, since no one really know exactly what possible patterns might be fired from each event. I don't have all the character classes, I didn't play through all the possible WoW encounters, so it can't be completed with me alone.

So if anyone want to help me to 'complete' it, then please go and download Simple Combat Log, I admit it's ugly, but it works very well as a debugging client addon, I'm just lazy to polish it up for now.

Another part is I have to make sure the variable naming be fixed, you may take a look for the current variable description at the documentation, I think the variable names should be pretty solid, but if anyone got any opinion about the var naming please let me know.

Originally Posted by Andalia I currently use Miles Combat Parser for my (private) mods. It also works quite nice and im sure that it would have become a great addon if the autor didn't suddenly disappear. I hope you release your lib soon so i can switch to it because i'm too lazy to do the parsing myself and using a beta mod isn't nice either. Maybe you can get some things out of mcp that you have not included atm or so because im sure that the autor of it has found a pretty nice and fast way for parsing and the only thing it doesn't display afaik are glancing blows.

Yeah, I did some simple test and MCP's parsing speed was better than mine by like 60%~80%, almost double as fast. I'll see what can be learned from it to improve my speed. ParserLib got its limitation for speed though:

1. It has to parse through the patterns in the correct sequence so that wrong result won't be parsed for all localizations, which MCP seems to ignored the problem to speed up performance.
2. MCP did some non-regexp keyword searching in the combat message to speed up, such as the "miss" in "%s misses you", this is not allowed if you want to be localization-independent.
3. ParserLib parses much more patterns than MCP.

Of course I'm sure something can be learned from MCP for parsing speed, that's not related to API though, my current focus is to finalize the API so that you can actually start using it without worring about the changes in API, variable names etc.

I already tried my best to optimize the memory by dynamically load on demand for everything, so after the API is finalized I'll start working on parsing speed.

But even in ParserLib's current form, its parsing speed should be about the same as other addons, if not faster. Just that MCP's goal was to optimize in speed so yeah, it beats ParserLib for that part, and it should. ParserLib's goal is a localization-independent parser, not a ultra-fast parser which only works in english.

[rophy]

Ok. Last week I have been trying to catch up the parsing speed of MCP.

Using the 10155 Ragnarnos fight event data set from MCP,
the 'core parsing function' of MCP takes 0.5 ~ 0.6s. ParserLib now takes 0.6~0.85s.

To further increase ParserLib's speed I've added the cocept of a "Parsing Optimizer", which can further reduces ParserLib parsing time to 0.6 ~ 0.7, which is almost the same as fast as MCP.

The optimizer actually is just a set of pattern - keyword , such as the keyword 'gain' in AURAADDEDSELFHELPFUL (You gain %s.), so the optimizer looks like this:

ParserLibOptomizer = {
AURAREMOVEDOTHER = "fade",
AURAREMOVEDSELF = "fade",
COMBATHITCRITOTHEROTHER = "crit",
......
}

The optimizer is completely optional, so when you don't have an optimizer on your language, ParsreLib still works fine, just that it'll be slower by may be 20%. But at the same it doesnt have to load that table into memory so memory usage is lower, basically that's a simple time-space trade off.

---

Finally I've found that the biggest bottleneck for speed is CompostLib, the above timing does not take CompostLib into account. When CompostLib is used (recycling the tables), the parsing speed increases from 0.6s to 1s, and the compost:Recycle() is responsible for about 0.3~0.4s of the time.

That's a big trade off betwen time and space. But I think I'll just stick with using CompostLib, because on real time, the speed will not make a actual noticeable difference; but when garbage collection occurs, you might notice a slight delay on your WoW client.

---

At last I have added Deformat(text, pattern) into the library, so when you want to parse for a custom pattern which is not supported by ParserLib, you can just call Deformat like this:

spell, target = parsereformat("Shield fades from Rophy.", AURAREMOVEDOTHER)

ParserLib's Deformat doesn't support trailers though, I think all patterns with possible trailers are already supported by ParserLib so don't bother adding that part.


I'm gonna do some final clean up on the codes, and then I'll submit ParserLib itself to WoWInterface. If you have any question or suggestion please leave a message here.

[ckknight]

Recommendation: take a look at BabbleLib, specifically BabbleLib-Core, which includes a _proper_ deformat.

Also, BabbleLib-Spell could be useful here as well.

[rophy]

Originally Posted by ckknight Recommendation: take a look at BabbleLib, specifically BabbleLib-Core, which includes a _proper_ deformat. Also, BabbleLib-Spell could be useful here as well.

Yeah, I checked it again and my deformat simply wasn't working, I only took a while to write that part and didn't do much testing, since that wasn't the core part of ParserLib, thanks.

For BabbleLib-Spell I don't think that's related to the message parsing itself, but yes it could be very useful to those who need it. I hope BabbleLib and ParserLib work well together.

[Hammer]

There's an error when trying to use dark pact(31 point warlock affliction talent)

<ParserLib-1.1-13> Your Dark Pact drains 150 Mana from Disrida. You gain 150 Mana. -> not found in event CHAT_MSG_SPELL_SELF_BUFF

Disrida is my pet's name.

[rophy]

wtf.... I should have added that pattern many versions ago, probably missed it when I rewrite the internal data structure, fixed.

Originally Posted by Hammer There's an error when trying to use dark pact(31 point warlock affliction talent) <ParserLib-1.1-13> Your Dark Pact drains 150 Mana from Disrida. You gain 150 Mana. -> not found in event CHAT_MSG_SPELL_SELF_BUFF Disrida is my pet's name.

The internal alpha of PetExtend (www.mpstark.wowinterface.com) now uses ParserLib to moniter pet DPS. The things that make this library important to me are the slight accuracy gain (now matches KombatStats, also by rophy), the fact that it's completly localized, and the ease of use. Thank you rophy for making this!

Just wanted to tell ya,
Mpstark

P.S. - Finish up the new version of KombatStats please! =P

[rophy]

Originally Posted by Mpstark The internal alpha of PetExtend (www.mpstark.wowinterface.com) now uses ParserLib to moniter pet DPS. The things that make this library important to me are the slight accuracy gain (now matches KombatStats, also by rophy), the fact that it's completly localized, and the ease of use. Thank you rophy for making this! Just wanted to tell ya, Mpstark P.S. - Finish up the new version of KombatStats please! =P

ahh, thanks, KombatStats and Simple Combat Log don't feel lonely now.