ResourceLib, PE Format, and WiX

Some time ago I reported a bug and provided a pull request to resourcelib, a managed library to read and write Win32 resources in executables or DLL’s. And unawarely, the next morning I was a maintainer of that library.

This blog post is about an issue we’ve received: someone tried to patched the Win32 resources of a setup.exe , an executable installer created with WiX. However, after changing the resources with resourdelib, the setup didn’t work anymore.

I’ve spent some time investigating this issue using dumpbin and reading the PE format specification. Because I don’t know how good Google is at indexing Github issues, I’ll also post my analysis here in my blog for reference. The original thread is here.

TL;DR: to me it looks like WiX is doing it wrong.

According to the output of dumpbin there are 7 sections in the executable the issuer provided:

In a hex viewer you can see that after the last section, the file continues for another 104205 bytes, starting with 0x4D 0x53 0x43 0x46 (MSCF, the magic number starting a cab file).

I patched the StringFileInfo resource using resourcelib, which changed the content of the .rsrc section only. Afterwards the file ended at 0x000715FF, i.e. the following 104205 bytes were missing.

By the way, the .wixburn section contains following bytes:

RAW DATA #4
  0046C000: 00 43 F1 00 02 00 00 00 04 12 28 81 2C 64 40 48  .C±.......(.,d@H
  0046C010: B2 B1 34 64 EC 08 65 64 00 16 07 00 00 00 00 00  ▓▒4d∞.ed........
  0046C020: 00 00 00 00 00 00 00 00 01 00 00 00 02 00 00 00  ................
  0046C030: 92 8E 01 00 7B 08 00 00                          ....{...

which means

Intermediate result

• the .wixburn section points to 104,205 bytes (102,034 + 2,171), starting at 0x00071600.
• the last PE section ends at 0x000715ff.
• after using the official Win32 API to edit resources, the file ends at 0x000715ff, and the following 104,205 byte are gone.

So after editing the resources, the exact payload WiX is referring to is eliminated.

Therefore my conclusion is that WiX

• adds a (small) section .wixburn pointing beyond the last section, and
• appends the payload (read: the cabinet file) at that location-

As far as I understand the specification, this procedure is not compliant with the PE format. That might be the reason why EndUpdateResource cuts the file after the last section when writing the changed resources.