Halo 2: Patch Format Types

Four years ago I made a blog post about a distant Halo 2 memory - discussing the thought process behind launching a modded mappack. I find myself during this global pandemic having more time at home, thus I booted up my old chipped Xbox and wanted to start writing this stuff down before it fades into the past of the Internet.

One thing that always escaped my mind as a young kid patching Halo 2 map files, is how did patch formats even work? So I began a deep dive investigation into .ppf , .sppf and .serenity patches from the Halo 2 world to figure out how they are made and what makes them different.

PPF

First up we have .ppf which was actually started a long time ago. I can't pinpoint the exact release date, but a few discussions with folks place it between 1994 and 1996. So a common question is how does a program that predated Halo 2 by 10 years work for patching the map files?

Lets take a step back and understand how patching works. Lets pretend you (the reader) and I (the writer) both have the same large file. If I wanted to correct something in that file, I have two options --> send you the modified file or tell you what to change in yours.

We are in the late 1990s or early 2000s here - it wasn't exactly easy to share large files (40-100mb) and not to mention probably illegal due to it being some licensed game content.

This enters the world of patches. An automated way of preforming actions on a clean (unmodified) file to produce the same exact dirty (modified) file.

So lets say I produce the following file.

ABCDEFG
Connor
Test

If I change it to

ABCDEGG
Connor
Test

I can tell my friend - "Go to the letter F in line 1 and switch to G" and that will work. However, in the real world not everything is pretty human readable text so that may not always work. So if I take those two files and ask PPF Studio to generate a patch. I get a tiny little 70 byte patch file.

As you can see, most of this file is just the "magic" to denote it as a PPF3.0 patch along with a little description. We can see that offset 5 (last line in image above) has 0x147 associated with it - which we have to think about. The first part (0x1) represents probably 1 byte to follow, followed by the byte - 0x47 (decimal 71 or ASCII 71 or G). So if we broke it down to English the character in position 6 will be replaced with 1 character which will be G.

Now above I simply select the "clean" file and then the 70 byte patch file produced. It probably works the same way as a human, but automated. It sees that offset 5 should be 0x147 (1 length of value 0x47) so it seeks to that location and sets the value, overwriting what is there. The clean file given should now match the modified file above.

Now lets talk about how this works in theory with "shifting". If you begin adding/removing bytes this will cause every character to "shift" the position it is in. This may result in a very large patch format which will take even longer to apply. As more than likely every byte after the shifted characters will have also moved a location.

Not to mention with a name PPF which stands for Playstation Patch Format proves this format wasn't even designed for Halo 2. Though I was curious who produced this patch type and it wasn't easy to dig that up. The best I got was the team behind PPF-O-Matic which was Icarus, H2O and Zweifeld of Paradox.

Paradox is an Amiga and console cracking group. They were originally formed in late 1989 by members of the Danish group Trilogy
Demozoo.org - Group #1853

These guys started with Amiga & Playstation mods, which were just patches on top of the games. It seems the Paradox team's first release was in 1995 on the Amiga so this group had been around for quite awhile. From research on those 3 members, 2/3 of them are from Germany so I can only guess that the 3rd is as well.

So now we enter the realm of Super PPF and Serenity, which were enhancements in the realm of .map patching.

SPPF

The story here is interesting - a member known as "low8" which had never before been seen in the scene posted this program around September of 2006. It was promoted as producing smaller patch files than PPF and including both the creation of patches and applying of patches inside the same tool.

Remember, the very generic .ppf patch required both PPF Studio (to make) and PPF O Matic (to apply), but could be applied to really any type of file.

SuperPPF did indeed produce smaller patches by introducing 4 blocks (chunks of data) to prevent large shifts. Basically if you added data into a specific tag those changes would only shift until that block ended, unlike .ppf which would make patches as large as the original map file. This meant that patches were generally smaller and the ease of use sky-rocked the program into the hands of all Halo 2 modders. If a mod was posted with only a .ppf patch normally the first few response were pleading with the poster to add support for .sppf or .serenity. Since smaller files were easier to transfer and apply.

So why did low8 produce Super PPF? The design was obviously a big joke rocking comic sans font and a huge mess of rainbow colors. The name hinted at mocking other members of the HaloMods scene. Was this another known member playing a joke?

Whoever low8 was took some small effort into building the most basic but effective patch tool of that time-frame. If we examine a disassembly of the application, we see all assembly metadata was empty.

[assembly: AssemblyCopyright("")]
[assembly: AssemblyTitle("")]
[assembly: AssemblyKeyFile("")]
[assembly: AssemblyDelaySign(false)]
[assembly: AssemblyTrademark("")]
[assembly: AssemblyKeyName("")]
[assembly: AssemblyProduct("")]
[assembly: AssemblyCompany("")]
[assembly: AssemblyConfiguration("")]
[assembly: AssemblyDescription("")]
[assembly: AssemblyVersion("1.0.2220.21515")]

The code for this application had zero obfuscation and looked like most first time developers with variables just being named "num1, num2, num3, num4". Though my guess is this was the intention. There was no error handling and there were no updates provided. This was SuperPPF - released once and quite popular despite all the strangeness that surrounded it. It was only after investigating Serenity did the dots begin to connect and allow us to discover who low8 was.

Serenity

This program was built for Halo 2 in mind, it was not a generic patch tool for anything. Since it was built specifically for Halo 2 it had some really interesting features.

• Very small patch size
• Built in author/main-menu support
• A parse-able patch file for systems to read

What sounded like a good idea in theory just wasn't ready for the time span it was built in. The first release of Serenity I see was version 1 on June 15, 2005 and it's release had mixed reactions. Some appreciated the small patch size and features it had -- it patched quickly and provided a more professional patch for modders.

However, building a patch format specific to Halo 2 required the map file to not be malformed completely. Some modders were just hex-editing a .map file at this point while any of the 15 other map editors may slightly invalidate some component of the map. This would mean that Serenity would simply not work, it expected perfection to produce perfection.

This would slowly change as versions were released:

• v1 - June 15, 2005 (Released alongside Insolence v1)
• v2 - June 29, 2005
• v2.02 - January 21, 2006
• v3 - March 20, 2006
• v3.1 - April 14, 2006
• v3.20 - June 25, 2006
• (Release of SuperPPF) - September 2006
• v3.30 - April 15, 2007

This proved to be problematic for the modders around during the releases. If you got a serenity patch you had to apply it for the version it was produced from. This became moot once v3 dropped as it could handle older Serenity patches.

Ignoring that though, Serenity was powerful and had this aura of professionalism around it. HaloMods started allowing file uploads for it because the file size was normally very small. Additionally some sites (I think Snaver) built a web solution that could parse the .serenity patches and extract the meta information from it. This produced a cool table of mods that was dynamically produced simply from the patch file.

However, getting to perfection took 7 releases - which in 2005/2006 was not easy for the community. While auto updating programs are just the norm in 2020 - that was not the case back then. This meant that folks needed to track down the tool, download a newer version and maybe update a dot net framework.

Iron_Forge was probably upset after building an amazing tool that folks continued to use .ppf for map patches. Computers weren't that fast in 2005, so Iron_Forge decided to troll the community and release SuperPPF under a false alias of "low8".

The idea was simple. Make something so ugly and crazy but slightly better than .ppf  to prove a point. I'm not sure if he intended the outcome. SuperPPF boomed as it just worked with maps thrown at it while the various Serenity versions suffered bugs and issues with malformed maps. Iron_Forge would regularly remind folks in the release threads of Serenity that issues were with the maps, not the tool.

After about 6 months after SuperPPF's release, Serenity v3.3 dropped and that would be the last release ever to land. It included an interesting snippet in the release notes.

New in Serenity v3.30 is NFG Mode...This will basically use the same 4 points as an sppf patch for those of you who are poor excuses for modders, don't care about the size of mod you're forcing people to get, and in general horrible human beings...This will allow you to take full advantage of the features Serenity provides, though it will mean for larger patches than the other 2 Serenity modes, which will usually prevent from halomods hosting, and thus additions to the serenity repository...It also means SPPF can be phased out the same way PPF has been, and we can all sleep easier at night...
Iron_Forge - Serenity v3.3 Release Notes

That comment just seemed strange especially with NFG mode which probably meant - "No Fucking Good" or "No Fucks Given". It seemed the joke had ran out and Iron_Forge just wanted people to stop using .ppf and now .sppf. So now Serenity in this specific mode would replicate the behavior of SuperPPF, but also include all the benefits Serenity had outside of the patch size.

Additionally Serenity v3.3 had introduced "Non-compliant" mode. The default mode was to break the map into 35 blocks + a block for every single tag.

A tag basically represented every resource of a map (biped, sound, item, etc) and was the building block the Blam engine leveraged for .map files.

This was great as any change to a tag would be isolated to that block, but break if a map was not built in the best of ways. So non-compliant mode just used a single block for all meta information which generally lead to a succesful creation of a patch (though less optimized).

All in all, Serenity was just a professional piece of software. Dialog boxes had filters to only look for .serenity files and likewise the clean file had a filter for .map files. It would read the Serenity patch to automatically find the clean map (if they were named the same as on the disc) as well as producing a new .map file named after the mod. This would prevent dirtying up clean maps which was a very helpful enhancement.

With that, I'll end this deep dive into the Halo 2 era of .map patching programs. If I messed anything up or you can correct something I said - feel free to leave a comment or contact me. This is only the beginning - more Halo scene history blogs to come.