Extract "wow-update-12759.mpq"... meh!

Vlad · 08-14-10, 08:50 AM

Trying to extract the DBC and LUA/XML files but getting weird patch-like content in the files. How does the new patch system work?

Saiket · 08-14-10, 09:27 PM

I've been trying to figure this out with no luck too. Apparently Tekkub found a way to extract build 12694, which uses the same patch format. I couldn't find any mention of what tools he used though. The actual patch files look like bsdiff binary diffs with a proprietary blizzard header prepended to them. I had trouble applying any of the patches though, even with the extra header trimmed out.
____
Edit: On second look at what Tekkub committed, he didn't extract patch 12694 properly. He has the changes between builds 12644 (pre-streaming client) and 12635 (base streaming client); None of the changes from wow-update-12694.mpq are included though.

Vlad · 08-15-10, 10:22 AM

Yeah, I noticed that too. I've been opening update exes before (they are mpq too) and this file structure seems familiar, it's only hard to find out how we can now extract the up-to-date FrameXML folder when it appears that each new build will have it's own "patch" MPQ containing diff files and not the whole patched file, hmm...

Perhaps someone at Wowhead may help? They do extract DBC, BLP icons and similar and they are also subjected to the same difference patching system.

Bluspacecow · 08-15-10, 11:05 AM

I just used MPQ extractor.

You can find it on www.devklog.net

The last time I tried it was a few weeks ago and got a folder worth of files extracted.

Not sure if that's what you're looking for or if even if it extracts it right.

Wait.... I might of tried this on a previous build. I'll try to remember to repeat it when I get home.

Vlad · 08-15-10, 11:08 AM

Try it yourself Bluespacecow, use the MPQEditor, open "wow-update-12759.MPQ" and extract the "Interface/FrameXML" folder. Open up the files inside and be surprised.

Bluspacecow · 08-15-10, 12:21 PM

Originally Posted by Vladinator

Try it yourself Bluspacecow, use the MPQEditor, open "wow-update-12759.MPQ" and extract the "Interface/FrameXML" folder. Open up the files inside and be surprised.

Using OS X so can't use MPQEditor.

I note that at

http://www.zezula.net/en/mpq/download.html

Is a beta version for extracting wow-update-12694.MPQ

Is the problem extracting the files the problem ? Or the format they are in once extracted.

Try the Beta Version at that site above and extract with that if you haven't already. Zezula lists that MPQ (wow-update-12694.MPQ) as a partial MPQ file so wow-update-12759.mpq may be as well

Saiket · 08-15-10, 12:30 PM

Even with the beta version of that MPQEditor, it still extracts patch files. Its file properties dialog can tell you if a packed file is a patch, but that's it. I even tried using the multi-MPQ mode with build 12694 loaded over top of 12635. Still no luck though.

Vlad · 08-15-10, 01:02 PM

The problem is that like in this case, I want the latest FrameXML files. I naturally open "wow-update-12759.MPQ" as it's the update file downloaded with the launcher. Inside I see the various localizations and the "base" that is common. I open enUS/Interface/FrameXML/ and see several files. I open for example "ActionButton.lua" and see this (binary data):

Code:

PTCHÜB`@ÐAMD5_(¯¹{†œòv"°òõ¯Ãú*a¶¨VÉ*«Í¶0¦*û¬0XFRM³BSD0˜BˆBSDIFF40¨AAÐA®„e
€€	„ß#€
„š€…ÛX„À€€	„4%€
„ð€…3„E€€„á€€„¸€K„y*€…”`ƒ;€@ÿET_STABLEPET_STABLEET_STABLEPET_STABLEET_STABLE_UPDATPET_STABLE_SHOWHas to update everything for now, but this event should happŽen infrequently

Basically the updates contain diff patches and if the file is new then it's normal.

*Edit*

1. Extracted "locale-enUS.MPQ" and it's FrameXML
2. Extracted "wow-update-12694.MPQ" and it's FrameXML
3. Extracted "wow-update-12759.MPQ" and it's FrameXML

I downloaded bsdiff and bspatch (Binary diff/patch utility version 4.3) and when I try to patch a file from #1 by using a file from #2 I get "corrupt file", seems like they also added some custom data to their patching files, hmm...

Borlox · 08-16-10, 08:33 AM

I think the BSDIFF40 part in the PTCH files is invalid.
For example the bsdiff part of ActionButton.lua from wow-update-12759.mpq says the resulting file would be 146789209322029069 Bytes long (thats 130 Petabyte)

As far as I understand the format the data in bsdiff files should be compressed with bzip2, but there's clear text in the .lua diffs.

I didn't find any file format beginning with 'PTCH', so maybe that's a Blizzard invention.

Vlad · 08-16-10, 09:33 AM

Borlox, that's what I also found out.

There is some clear text indeed, the data that is to be inserted/replaced, e.g. is all there. Problem is figuring out where to put, replace, append, delete, e.g. I tought wowhead or mmochamp figured this out as they do read DBC files right, so they need to figure out how to patch their DBC with the updates so they can for example scan Spell.DBC for changes.

Asking Wowhead mods on IRC atm, let's see if there is a way.

Borlox · 08-16-10, 10:28 AM

Originally Posted by Vladinator

Asking Wowhead mods on IRC atm, let's see if there is a way.

I really hope they'll give you a clue...

yssaril · 08-16-10, 10:50 AM

at least on windows the addonkit from blizzard still works but you need 100% downloaded/patched and need to close the laucher/wow before you start extracting

Saiket · 08-16-10, 11:58 AM

I think the Addon Kit is skipping the wow-update MPQs too. I can't find any of those changes in its output. As an easy test case, check for <FrameXML/Util.lua>, which should be there since patch 12694.

Vlad · 08-16-10, 12:31 PM

Addon Kit only extracted from my retail, not beta. Also I think it also only extracts from the locale file and not including the patches.

Verissi · 08-16-10, 01:59 PM

Found some (if not all) of the Interface files that were patched in this build in the Data\Cache\enUS (or whatever locale you use) directory in the build's patch MPQ. From what I can tell, though, any totally new files aren't in there at all. The new spell flyout files aren't in there, for example, but are in another MPQ in text form rather than as a patch (since they're new).

Looks like it may take a bit more work than before to extract things that can be committed to a revision control repo or diffed before a new kit is released that can cope with the new scheme. Nothing a little scripting can't handle, but it's still a change to adapt to.

Vlad · 08-17-10, 02:01 AM

Adys from MMO-Champion and Ladik the author for MPQ Editor (not sure of the nick entirely) to integrate this feature into the editor itself most likely, patch the files as they are extracted from the updates -very neat ey?

Adys told me that this would not be available until several weeks later so we just hang in there, someone is working on fixing this issue we have.

Ladik · 08-18-10, 01:04 AM

I currently work on "Open MPQ in patch mode" feature of MPQ Editor. The desired functionalty is that you open a base MPQ, then add list of patch MPQs (currently, there are wow-update-12694.MPQ and wow-update-12759.MPQ), and it should seamlessly extract patched file already. I hope first experimental version might be available this week.

Saiket · 08-18-10, 03:09 AM

Originally Posted by Ladik

I currently work on "Open MPQ in patch mode" feature of MPQ Editor. The desired functionalty is that you open a base MPQ, then add list of patch MPQs (currently, there are wow-update-12694.MPQ and wow-update-12759.MPQ), and it should seamlessly extract patched file already. I hope first experimental version might be available this week.

Great utility, Ladik! I only just discovered MPQ Editor, and I have nothing but compliments for it; Light years better than WinMPQ. If I could request a feature, could you add a command-line option to load multiple MPQs in the given order? Once there are lots of patches, it would make loading them all up much simpler.

Something like this could work well:

Code:

MPQEditor.exe /multiple "enUS\locale-enUS.MPQ" "wow-update-12694.MPQ" "wow-update-12759.MPQ"

Ladik · 08-18-10, 03:15 AM

Originally Posted by Saiket

could you add a command-line option to load multiple MPQs in the given order?

Try this: MPQEditor huhu.mpq hehe.mpq hihi.mpq haha.mpq.
However, this opens the multiple MPQs in multi-mode. I guess it's time for big redesign of the entire command line syntax, because it has been there for like 6 years and too many things have changed :-)

Originally Posted by Saiket

Once there are lots of patches, it would make loading them all up much simpler.

Indeed. Yet another reason to redesign command line syntax.

Vlad · 08-18-10, 04:16 AM

Awesome, did you write it from scratch (the patching func) or did you borrow some ideas from the bsdiff sources? Just curious. I guess you wrote from scratch as you are probably super good at programming. :P

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode