Ramblings of a Tampa engineer

Apktool v2.9.0 has been released! This release is quite large with a good deal of changes and a ton of commentary to follow. If you aren't interested and just want the changes and download - head on the bottom or the doc site.

From the highest level - we will dig into the follow:

  • Android 14 Support
  • AAPT2 as default
  • Resources Modes
  • SnakeYAML
  • Surviving Decoding
  • Reworking Attributes/Arrays
  • Sponsors

Android 14 Support

Android 14 was unlike the previous releases for AOSP for a variety of reasons. This release introduced a new qualifier for gender, but what was odd about this qualifier is that it claimed the "inputPad0" that was found at byte 20 of the flags that was previously reserved/unused.

This meant that instead of increasing the size of the qualifier header - a reserved value was claimed. This becomes problematic for folks using older versions of Apktool now. As Apktool previous to 2.9.0 will skip that value - this will lead to duplicate resources and more than likely crash. This was important for another reason, because we did not see this patch applied to the original aapt build tool, only aapt2 was updated.

Additionally, a new optimization technique was created for aapt2 known as "--enable-compact-entries". This is a bleeding edge optimization only for applications built with a minimum of API 34. What this does under the hood is quite clever and deserving of its own blog.

In short we saw the introduction of COMPACT entries, which were commonly used when only strings were in play. This allowed some tricks to be used to store the index/type in the size/flags field, because you can assume the size of a string only entry set and re-purpose the size field for other information.

Finally, we saw the introduction of OFFSET16 as a flag, which allowed entries to be stored in a smaller data size than normal (int). Of course this would only work if the entry set can fit into the unsigned short size pool.

AAPT2 as Default

This was going to happen at some point eventually, but with Android 14 adding more and more features only to aapt2 - it became clear that Apktool defaulting to the original aapt was no longer the right call. Especially since the new qualifier (gender) is only supported in aapt2.

This exposed a flaw in the naming pattern of these flags and forced me to introduce --use-aapt1, and keep --use-aapt2 as a defunct parameter. Of course tool authors who wrap Apktool are going to struggle with this one, but I accept I named this parameter wrong in the beginning. It should have been something like --aapt-version {x}.

Introducing Resource Modes

While resolving issues I noticed a pattern to the tune of basically:

  • intentional malformed/unused resources
  • huge amount of gaps in resource table
  • confusion around dummy resources

So a new parameter called Resource Modes was added with 3 possibilities of remove, dummy or keep. The documentation explains the reasoning and history best with this, but this is a big step towards disassembling an application as the end user wants. We will now default to removing any unresolved resource instead of previously producing a dummy (and often invalid) resource.

Dropping SnakeYAML

We relied on SnakeYAML for over a decade to help us parse the YAML files, but they've had some struggles running on Android itself and some recent CVEs due to untrusted parsing. Apktool relies on an untrusted apktool.yml file, so sv99 set out to introduce a very small custom class for handling YAML files.

It went through months of testing during the construction of Apktool 2.9.0 so I'm confident all regressions have been found. With SnakeYAML gone - I can now safely assume I won't get anymore "SnakeYAML vulnerability" emails anymore, because as that hit 10+ occurrences I was starting to get annoyed.

đź’ˇ
Even when we used SnakeYAML - we had a custom class safe constructor so were not vulnerable to any untrusted code deserialization attacks.

Surviving Decoding

Much like I talked about during the release of 2.8.0 - I set out for this release to tackle every disassembly failure during Apktool resource parser that we had open bug reports for. We got tons of fixes for the following:

  • supporting misleading chunk header sizes
  • supporting entry offsets that resolve to nothing
  • supporting AXML namespaces with padded extra data
  • supporting AXML with duplicate end namespaces
  • supporting empty resource tables
  • supporting duplicate namespaces
  • supporting string blocks with more entries than strings
  • supporting unknown files that exceed file length limit
  • supporting smaller packed resource entries
  • supporting spanned strings that have spans outside length
  • supporting pre-1.0 Android resources

In short this required large chunks of rewriting our AXML/ARSC parser, but the journey left me with a far better understanding of the specification than previously. You should notice far more success with disassembly than the past.

Reworking Arrays/Attributes

Apktool was started before AOSP sources were available and had a good, but not bulletproof resolution for determining the types of attributes and arrays. This was noticed here and there over the years and some patches were thrown on top of an unstable house. Once fresh eyes looked at some logic - it was far easier to determine types of arrays/attributes than previously.

This release via #3326 ripped out all that logic and rewrote the detection of type and led to a more stable solution. This may lead to some changes in the disassembly, but for a proper resolution than the past.

Sponsors

I launched GitHub Sponsors to help provide another alternative for folks showing appreciation. I was quite happy to obtain a few donations in under of month from launching that platform.

  • Emerge Tools came online to sponsor the tool.
  • Sourcetoad (self employer) additionally joined to sponsor (as well as a few other projects).

This release had 327 commits by 10 people and 1 robot

  • Connor Tumbleson (iBotPeaches) - 282 commits
  • Dependabot (BOT) - 9 commits
  • Fmstrat - 8 commits
  • Slava Volkov - 8 commits
  • sv99 - 7 commits
  • ArjunaKumarMohanta - 3 commits
  • Ben Curtis - 3 commits
  • Igor Eisberg - 2 commits
  • Dougy Lee - 2 commits
  • Phone-Guru - 1 commit
  • maksz42 - 1 commit

Changes since 2.8.1

  • [#3145] Add manual YAML implementation to drop SnakeYAML dependency. (Thanks sv99)
  • [#3202] Add Android 14 framework support.
  • [#2836] Add Resource Modes to allow for more control over resource decoding.
  • [#3351] Add Docker image. (Thanks Fmstrat)
  • [#3366] Add support for compactly packed resources.
  • [#3367] Add support for 16bit offset entries in resource table.
  • [#3369] Add support for grammatical gender.
  • [#3199] Fix regression with detection of sparse resource table.
  • [#3298] Fix poisoning of sparse detection via sparse framework.
  • [#3205] Fix over-reading the Res Config Flags if size is larger than known.
  • [#2806] Fix improperly decoding attribute and arrays.
  • [#3346] Fix corruption of resource table when aapt2 is used.
  • [#2989] Fix decoding application with larger chunk header size than used.
  • [#1874] Fix decoding application with entries offset that resolved to NO_ENTRY (-1).
  • [#2587, #2863] Fix decoding application with larger axml namespace than known.
  • [#2070] Fix decoding application with misleading number of end namespace chunks.
  • [#2701] Fix decoding application with empty resource table.
  • [#2664] Fix decoding application with duplicate namespaces.
  • [#2972, #2420] Fix decoding application with missing namespace on system resources.
  • [#3236] Fix decoding application with string block with more string entries than strings.
  • [#3238] Fix decoding application with unknown files that exceed file name length limits.
  • [#2824] Fix decoding application with duplicate smaller res entries.
  • [#2829] Fix decoding application with styled string with span outside length of string.
  • [#3311] Fix decoding application from Android Milestone (pre 1.0) builds.
  • [#3198] Fix "disableZip64ExtraFieldValidation" patch by adding to helper scripts.
  • [#3204] Fix changing values of platformBuildVersionCode and platformBuildVersionName in apktool.yml.
  • [#3282] Fix ci build system to properly archive test builds post Kotlin migration.
  • [#2973] Fix Windows helper scripts to properly handle /c. (Thanks maksz42)
  • [#3368] Change aapt2 to be default build tool.
  • [#3222] Change build system to ignore missing javadoc blocks. (Thanks sv99)
  • [#2129] Change -c/--copy-original to no longer be deprecated.
  • [#2683, #2104] Change frequency of dummy resources to be less often after proper disassembly of resources.
  • [#3249] Change build system configuration to Kotlin DSL.
  • [#3353] Change build system to leverage Gradle version catalog.
  • [#3258] Change system to no longer generate APKTOOL_DUMMY_* resources.
  • [#3310] Change verbose mode to no longer print out method names.
  • [#3171] Refactor out specific manifest file decoder for empty resource table. (Thanks sv99)
  • [#3211] Refactor out duo logic with decoders during resource parser. (Thanks sv99)
  • [#3217] Refactor for specific method operations for loading full resource table. (Thanks sv99)
  • [#3242] Refactor ApkDecoder for leveraging ApkInfo for file and unknown files. (Thanks sv99)
  • [#3243] Refactor ResAttrDecoder into AXmlResourceParser. (Thanks sv99)
  • [#3228] Refactor Gradle validation job to only run if gradle/wrapper/gradle-wrapper.jar is modified. (Thanks ArjunaKumarMohanta)
  • [#3275] Upgrade Gradle to 8.3.0. (Thanks ArjunaKumarMohanta)
  • [#3327] Upgrade @actions/checkout to v4.
  • [#3240] Upgrade apache.commons/lang3 to 3.13.0.
  • [#3358] Upgrade apaches.common-io to 2.14.0.
  • [#3213, #3241, #3285, #3316, #3359] Upgrade gradle/gradle-build-action to 2.9.0.
  • [#3214] Upgrade gradle/wrapper-validation-action to 1.1.0.
  • [#3235] Upgrade xmlunit:xmlunit:1.6 to org.xmlunit:xmlunit-legacy:2.9.1.
  • [#3259] Upgrade com.github.johnrengelman.shadow to 8.1.1.

Notes

  • This release has moved to aapt2 being the default. If you'd like to return to the previous behavior, please use --use-aapt1 during build stage.

Download

You’ve successfully subscribed to Connor Tumbleson
Welcome back! You’ve successfully signed in.
Great! You’ve successfully signed up.
Success! Your email is updated.
Your link has expired
Success! Check your email for magic link to sign-in.