Thread Tools Display Modes
Unread 08-14-10, 08:50 AM   #1
Vlad
A Molten Giant
 
Vlad's Avatar
AddOn Author - Click to view addons
Join Date: Dec 2005
Posts: 765
Extract "wow-update-12759.mpq"... meh!

Trying to extract the DBC and LUA/XML files but getting weird patch-like content in the files. How does the new patch system work?
Vlad is offline  
Unread 08-14-10, 09:27 PM   #2
Saiket
A Chromatic Dragonspawn
 
Saiket's Avatar
AddOn Author - Click to view addons
Join Date: Jul 2008
Posts: 152
I've been trying to figure this out with no luck too. Apparently Tekkub found a way to extract build 12694, which uses the same patch format. I couldn't find any mention of what tools he used though. The actual patch files look like bsdiff binary diffs with a proprietary blizzard header prepended to them. I had trouble applying any of the patches though, even with the extra header trimmed out.
____
Edit: On second look at what Tekkub committed, he didn't extract patch 12694 properly. He has the changes between builds 12644 (pre-streaming client) and 12635 (base streaming client); None of the changes from wow-update-12694.mpq are included though.

Last edited by Saiket : 08-14-10 at 09:36 PM.
Saiket is offline  
Unread 08-15-10, 10:22 AM   #3
Vlad
A Molten Giant
 
Vlad's Avatar
AddOn Author - Click to view addons
Join Date: Dec 2005
Posts: 765
Yeah, I noticed that too. I've been opening update exes before (they are mpq too) and this file structure seems familiar, it's only hard to find out how we can now extract the up-to-date FrameXML folder when it appears that each new build will have it's own "patch" MPQ containing diff files and not the whole patched file, hmm...

Perhaps someone at Wowhead may help? They do extract DBC, BLP icons and similar and they are also subjected to the same difference patching system.
Vlad is offline  
Unread 08-15-10, 11:05 AM   #4
Bluspacecow
Giver of walls of text :)
 
Bluspacecow's Avatar
AddOn Author - Click to view addons
Join Date: Dec 2006
Posts: 765
I just used MPQ extractor.

You can find it on www.devklog.net

The last time I tried it was a few weeks ago and got a folder worth of files extracted.

Not sure if that's what you're looking for or if even if it extracts it right.

Wait.... I might of tried this on a previous build. I'll try to remember to repeat it when I get home.
__________________
tuba_man on Apple test labs : "I imagine a brushed-aluminum room with a floor made of keyboards, each one plugged into a different test box somewhere. Someone is tasked with tossing a box full of cats (all wearing turtlenecks) into this room. If none of the systems catch fire within 30 minutes, testing is complete. Someone else must remove the cats. All have iPods." (http://community.livejournal.com/tec...t/2018070.html)
Bluspacecow is offline  
Unread 08-15-10, 11:08 AM   #5
Vlad
A Molten Giant
 
Vlad's Avatar
AddOn Author - Click to view addons
Join Date: Dec 2005
Posts: 765
Try it yourself Bluespacecow, use the MPQEditor, open "wow-update-12759.MPQ" and extract the "Interface/FrameXML" folder. Open up the files inside and be surprised.
Vlad is offline  
Unread 08-15-10, 12:21 PM   #6
Bluspacecow
Giver of walls of text :)
 
Bluspacecow's Avatar
AddOn Author - Click to view addons
Join Date: Dec 2006
Posts: 765
Originally Posted by Vladinator View Post
Try it yourself Bluspacecow, use the MPQEditor, open "wow-update-12759.MPQ" and extract the "Interface/FrameXML" folder. Open up the files inside and be surprised.
Using OS X so can't use MPQEditor.

I note that at

http://www.zezula.net/en/mpq/download.html

Is a beta version for extracting wow-update-12694.MPQ

Is the problem extracting the files the problem ? Or the format they are in once extracted.

Try the Beta Version at that site above and extract with that if you haven't already. Zezula lists that MPQ (wow-update-12694.MPQ) as a partial MPQ file so wow-update-12759.mpq may be as well
__________________
tuba_man on Apple test labs : "I imagine a brushed-aluminum room with a floor made of keyboards, each one plugged into a different test box somewhere. Someone is tasked with tossing a box full of cats (all wearing turtlenecks) into this room. If none of the systems catch fire within 30 minutes, testing is complete. Someone else must remove the cats. All have iPods." (http://community.livejournal.com/tec...t/2018070.html)
Bluspacecow is offline  
Unread 08-15-10, 12:30 PM   #7
Saiket
A Chromatic Dragonspawn
 
Saiket's Avatar
AddOn Author - Click to view addons
Join Date: Jul 2008
Posts: 152
Even with the beta version of that MPQEditor, it still extracts patch files. Its file properties dialog can tell you if a packed file is a patch, but that's it. I even tried using the multi-MPQ mode with build 12694 loaded over top of 12635. Still no luck though.
Saiket is offline  
Unread 08-15-10, 01:02 PM   #8
Vlad
A Molten Giant
 
Vlad's Avatar
AddOn Author - Click to view addons
Join Date: Dec 2005
Posts: 765
The problem is that like in this case, I want the latest FrameXML files. I naturally open "wow-update-12759.MPQ" as it's the update file downloaded with the launcher. Inside I see the various localizations and the "base" that is common. I open enUS/Interface/FrameXML/ and see several files. I open for example "ActionButton.lua" and see this (binary data):
Code:
PTCHB`@AMD5_({†œv"*aV*Ͷ0*0XFRMBSD0˜BˆBSDIFF40AAA„e
€€	„#€
„š€…X„€€	„4%€
„€…3„E€€„€€„€K„y*€…”`ƒ;€@ET_STABLEPET_STABLEET_STABLEPET_STABLEET_STABLE_UPDATPET_STABLE_SHOWHas to update everything for now, but this event should happŽen infrequently
Basically the updates contain diff patches and if the file is new then it's normal.

*Edit*

1. Extracted "locale-enUS.MPQ" and it's FrameXML
2. Extracted "wow-update-12694.MPQ" and it's FrameXML
3. Extracted "wow-update-12759.MPQ" and it's FrameXML

I downloaded bsdiff and bspatch (Binary diff/patch utility version 4.3) and when I try to patch a file from #1 by using a file from #2 I get "corrupt file", seems like they also added some custom data to their patching files, hmm...

Last edited by Vlad : 08-16-10 at 05:49 AM.
Vlad is offline  
Unread 08-16-10, 08:33 AM   #9
Borlox
A Deviate Faerie Dragon
 
Borlox's Avatar
AddOn Author - Click to view addons
Join Date: Jan 2007
Posts: 13
I think the BSDIFF40 part in the PTCH files is invalid.
For example the bsdiff part of ActionButton.lua from wow-update-12759.mpq says the resulting file would be 146789209322029069 Bytes long (thats 130 Petabyte)
As far as I understand the format the data in bsdiff files should be compressed with bzip2, but there's clear text in the .lua diffs.

I didn't find any file format beginning with 'PTCH', so maybe that's a Blizzard invention.
Borlox is offline  
Unread 08-16-10, 09:33 AM   #10
Vlad
A Molten Giant
 
Vlad's Avatar
AddOn Author - Click to view addons
Join Date: Dec 2005
Posts: 765
Borlox, that's what I also found out.

There is some clear text indeed, the data that is to be inserted/replaced, e.g. is all there. Problem is figuring out where to put, replace, append, delete, e.g. I tought wowhead or mmochamp figured this out as they do read DBC files right, so they need to figure out how to patch their DBC with the updates so they can for example scan Spell.DBC for changes.

Asking Wowhead mods on IRC atm, let's see if there is a way.
Vlad is offline  
Unread 08-16-10, 10:28 AM   #11
Borlox
A Deviate Faerie Dragon
 
Borlox's Avatar
AddOn Author - Click to view addons
Join Date: Jan 2007
Posts: 13
Originally Posted by Vladinator View Post
Asking Wowhead mods on IRC atm, let's see if there is a way.
I really hope they'll give you a clue...
Borlox is offline  
Unread 08-16-10, 10:50 AM   #12
yssaril
A Warpwood Thunder Caller
 
yssaril's Avatar
AddOn Author - Click to view addons
Join Date: Jan 2007
Posts: 96
at least on windows the addonkit from blizzard still works but you need 100% downloaded/patched and need to close the laucher/wow before you start extracting
yssaril is offline  
Unread 08-16-10, 11:58 AM   #13
Saiket
A Chromatic Dragonspawn
 
Saiket's Avatar
AddOn Author - Click to view addons
Join Date: Jul 2008
Posts: 152
I think the Addon Kit is skipping the wow-update MPQs too. I can't find any of those changes in its output. As an easy test case, check for <FrameXML/Util.lua>, which should be there since patch 12694.
Saiket is offline  
Unread 08-16-10, 12:31 PM   #14
Vlad
A Molten Giant
 
Vlad's Avatar
AddOn Author - Click to view addons
Join Date: Dec 2005
Posts: 765
Addon Kit only extracted from my retail, not beta. Also I think it also only extracts from the locale file and not including the patches.
Vlad is offline  
Unread 08-16-10, 01:59 PM   #15
Verissi
Premium Member
 
Verissi's Avatar
AddOn Author - Click to view addons
Join Date: Oct 2006
Posts: 99
Found some (if not all) of the Interface files that were patched in this build in the Data\Cache\enUS (or whatever locale you use) directory in the build's patch MPQ. From what I can tell, though, any totally new files aren't in there at all. The new spell flyout files aren't in there, for example, but are in another MPQ in text form rather than as a patch (since they're new).

Looks like it may take a bit more work than before to extract things that can be committed to a revision control repo or diffed before a new kit is released that can cope with the new scheme. Nothing a little scripting can't handle, but it's still a change to adapt to.
__________________
"I can calculate the motions of the heavenly bodies, but not the madness of people." - Sir Isaac Newton
"Half of twice as intimidating as Saurfang is still one whole Saurfang worth of intimidation." - Anticlaus, Gorefiend server
Verissi is online now  
Unread 08-17-10, 02:01 AM   #16
Vlad
A Molten Giant
 
Vlad's Avatar
AddOn Author - Click to view addons
Join Date: Dec 2005
Posts: 765
Adys from MMO-Champion and Ladik the author for MPQ Editor (not sure of the nick entirely) to integrate this feature into the editor itself most likely, patch the files as they are extracted from the updates -very neat ey?

Adys told me that this would not be available until several weeks later so we just hang in there, someone is working on fixing this issue we have.
Vlad is offline  
Unread 08-18-10, 01:04 AM   #17
Ladik
A Fallenroot Satyr
 
Ladik's Avatar
Join Date: Aug 2010
Posts: 23
I currently work on "Open MPQ in patch mode" feature of MPQ Editor. The desired functionalty is that you open a base MPQ, then add list of patch MPQs (currently, there are wow-update-12694.MPQ and wow-update-12759.MPQ), and it should seamlessly extract patched file already. I hope first experimental version might be available this week.

Last edited by Ladik : 08-18-10 at 01:11 AM.
Ladik is offline  
Unread 08-18-10, 03:09 AM   #18
Saiket
A Chromatic Dragonspawn
 
Saiket's Avatar
AddOn Author - Click to view addons
Join Date: Jul 2008
Posts: 152
Originally Posted by Ladik View Post
I currently work on "Open MPQ in patch mode" feature of MPQ Editor. The desired functionalty is that you open a base MPQ, then add list of patch MPQs (currently, there are wow-update-12694.MPQ and wow-update-12759.MPQ), and it should seamlessly extract patched file already. I hope first experimental version might be available this week.
Great utility, Ladik! I only just discovered MPQ Editor, and I have nothing but compliments for it; Light years better than WinMPQ. If I could request a feature, could you add a command-line option to load multiple MPQs in the given order? Once there are lots of patches, it would make loading them all up much simpler.

Something like this could work well:
Code:
MPQEditor.exe /multiple "enUS\locale-enUS.MPQ" "wow-update-12694.MPQ" "wow-update-12759.MPQ"
Saiket is offline  
Unread 08-18-10, 03:15 AM   #19
Ladik
A Fallenroot Satyr
 
Ladik's Avatar
Join Date: Aug 2010
Posts: 23
Originally Posted by Saiket View Post
could you add a command-line option to load multiple MPQs in the given order?
Try this: MPQEditor huhu.mpq hehe.mpq hihi.mpq haha.mpq.
However, this opens the multiple MPQs in multi-mode. I guess it's time for big redesign of the entire command line syntax, because it has been there for like 6 years and too many things have changed :-)

Originally Posted by Saiket View Post
Once there are lots of patches, it would make loading them all up much simpler.
Indeed. Yet another reason to redesign command line syntax.
Ladik is offline  
Unread 08-18-10, 04:16 AM   #20
Vlad
A Molten Giant
 
Vlad's Avatar
AddOn Author - Click to view addons
Join Date: Dec 2005
Posts: 765
Awesome, did you write it from scratch (the patching func) or did you borrow some ideas from the bsdiff sources? Just curious. I guess you wrote from scratch as you are probably super good at programming. :P
Vlad is offline  
 

Go BackWoWInterface » AddOns, Compilations, Macros » Cataclysm Beta » Extract "wow-update-12759.mpq"... meh!

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off