Taking a look at AAPT2
For those familiar with the Android ecosystem, applications built by developers and even some part of the Android system are built via AAPT. This stands for Android Asset Packaging Tool and is quite a nifty piece of software. It takes all the resources, including some preassembled files and zips them up into a zip file known in Android as an APK.
Apktool uses AAPT internally for the build process instead of manual rebuilding files like resources.arsc from scratch. AAPT can be leveraged with nearly 30 parameters, which are the relics of a piece of software patched over a decade. If you were to run aapt now, you would get the following commands to execute.
$ aapt
Android Asset Packaging Tool
Usage:
aapt l[ist] [-v] [-a] file.{zip,jar,apk}
aapt d[ump] [--values] WHAT file.{apk} [asset [asset ...]]
aapt p[ackage] [-d][-f][-m][-u][-v][-x][-z][-M AndroidManifest.xml] \
aapt r[emove] [-v] file.{zip,jar,apk} file1 [file2 ...]
aapt a[dd] [-v] file.{zip,jar,apk} file1 [file2 ...]
aapt v[ersion]
If you wanted to build an application without the benefit of an IDE, you would probably have to use AAPT manually with something like this.
aapt p -F output.apk -I path/to/current/android.jar -S res -M AndroidManifest.xml
This basically says
- p = package mode
- -F = Output file (apk)
- -I = Framework package include, to reference AOSP resources
- -S = Directories where resources can be found
- -M = Full path to AndroidManifest file
So let's make an example application with an AndroidManifest.xml file and one strings.xml resource file and try and build it with verbosity enabled.
aapt p -F output.apk -I ~/.local/share/apktool/framework/1.apk -S aapt1/res/ -M aapt1/AndroidManifest.xml -v
Configurations:
(default)
Files:
values/strings.xml
Src: () aapt1/res/values/strings.xml
AndroidManifest.xml
Src: () aapt1/AndroidManifest.xml
Resource Dirs:
Type values
values/strings.xml
Src: () aapt1/res/values/strings.xml
Including resources from package: /home/ibotpeaches/.local/share/apktool/framework/1.apk
applyFileOverlay for drawable
applyFileOverlay for layout
applyFileOverlay for anim
applyFileOverlay for animator
applyFileOverlay for interpolator
applyFileOverlay for transition
applyFileOverlay for xml
applyFileOverlay for raw
applyFileOverlay for color
applyFileOverlay for menu
applyFileOverlay for font
applyFileOverlay for mipmap
Creating 'output.apk'
Writing all files...
'AndroidManifest.xml' (compressed 52%)
'resources.arsc' (not compressed)
Generated 2 files
Included 0 files from jar/zip files.
Checking for deleted files
Done!
That seems easy and sure enough, we have our application.
➜ ~ unzip -l output.apk
Archive: output.apk
Length Date Time Name
--------- ---------- ----- ----
584 1980-12-31 19:00 AndroidManifest.xml
596 1980-12-31 19:00 resources.arsc
--------- -------
1180 2 files
➜ ~
Now imagine you had an application with 26 languages, 500 images, 200 layouts and 10,000 strings. The process of iterating every resources and building a finalized binary became slow. Slow enough that even a developer (Chainfire) got tired of waiting 30 seconds for an Android build of his application and submitted a fix to resolve that.
This would only delay the inevitable, though that fix worked quite well for awhile. As applications evolved a single dependency could bring in 1,000 strings. Applications were now building with multiple .dex files and upwards of tens of thousands of resources so the process became slow again.
Android was aware of this problem, but fixing a low level build piece of code was very difficult to do. So they started building "AAPT2" from scratch. Crawling GitHub we can see the first commit was on Nov 14, 2014.
From 6f6ceb7e1456698b1f33e04536bfb3227f9fcfcb Mon Sep 17 00:00:00 2001
From: Adam Lesinski <adamlesinski@google.com>
Date: Fri, 14 Nov 2014 14:48:12 -0800
Subject: [PATCH] AAPT2
First checking of AAPT2. The individual phases of AAPT2 work, but there
are some missing pieces.
For early testing we are missing:
- Need to properly mark file references and include them in package
- Need to package into zip
Final AAPT for apps we are missing:
- Need to crush PNGs
- Need to parse 9-patches
- Need to validate all of AndroidManifest.xml
- Need to write align method to align resource tables for splits.
Final AAPT for apps + system we are missing:
- Need to handle overlays
- Need to store comments for R file
- Need to handle --shared-lib (dynamic references too).
New AAPT features coming:
- Need to import compiled libraries
- Name mangling
- R file generation for library code
Change-Id: I95f8a63581b81a1f424ae6fb2c373c883b72c18d
It wasn't till Android Studio 3.0 (October 25, 2017) that AAPT2 would be set as default and released to the world of Android developers. Nearly 3 years after its creation -- was the world ready for a new and improved build tool?
Not really. The top search for "AAPT2" was asking how to disable it. Developers seemed to ignore the optional "opt-in" testing of AAPT2 and only when defaulted to the newer version notice the changes it brought, opting instead to revert back to the previous AAPT1 binary instead of addressing the issues it brought.
Was there anything wrong with AAPT2? I mean there are some bugs, but it's also a major rewrite of a core piece of software that was a decade old. There was bound to be issues and most of the reported issues aren't bugs, but instead new restrictions.
AAPT1 was too lenient, allowing you to do things that would create issues as Android updated. AAPT2 decided to stop those issues at build time instead of runtime. So what makes AAPT2 so different?
Executing it gives a very slim minimal output.
➜ ~ ./aapt2
no command specified
usage: aapt2 [compile|link|dump|diff|optimize|version] ...
We notice there is no longer a package command, but instead a compile & link command. This mirrors a more common build system where classes are compiled and then linked together in the final binary. This allows changes to one file to quickly be rebuilt and then linked into the final binary for a speedy build process.
So let's take this for a test drive with our sample application.
➜ ~ ./aapt2 compile --dir aapt1/res -o resources.zip -v
➜ ~ unzip -l resources.zip
Archive: resources.zip
Length Date Time Name
--------- ---------- ----- ----
199 1980-12-31 19:00 values_strings.arsc.flat
--------- -------
199
So we can see, although no output, AAPT2 crawled the resource directory and extracted our strings.xml file in the /res/values folder into values_strings.arsc.flat which we can guess is an arsc representation of our resource. I have no idea what arsc stands for, but I know it has something to do with Android resources.
So now lets attempt to build our application.
➜ ~ ./aapt2 link -o output2.apk --manifest aapt1/AndroidManifest.xml -I ~/.local/share/apktool/framework/1.apk resources.zip -v
~/.local/share/apktool/framework/1.apk: note: loading include path.
note: linking package 'com.connortumbleson.aapt1' using package ID 7f.
note: merging archive resources.zip.
note: merging resource table resources.zip@values_strings.arsc.flat.
note: enabling pre-O feature split ID rewriting.
AndroidManifest.xml: note: writing to archive (keep_raw_values=false).
note: writing AndroidManifest.xml to archive.
note: writing resources.arsc to archive.
Once again, another application compiled.
➜ ~ unzip -l output2.apk
Archive: output2.apk
Length Date Time Name
--------- ---------- ----- ----
392 1980-12-31 19:00 AndroidManifest.xml
548 1980-12-31 19:00 resources.arsc
--------- -------
940 2 files
➜ ~
The parameters for AAPT2 are a bit easier to understand as they are self descriptive and match much more to regular standards. The reason for this post is that Apktool is currently on the journey of bringing AAPT2 into Apktool and I've spent many hours with both tools understanding both their limitations and features.
I imagine AAPT1 will become legacy in a few years to slowly drift into the state of abandonment. While I'm angry at some issues AAPT2 is causing, I'm excited for the features and benefits it brings. I'll be following AAPT2 development closely to watch it evolve further.
If you are curious about the changes to AAPT2, I recommend following the changelog. If you are curious about the status of AAPT2 in Apktool, follow this bug report.
Featured image by Alex Knight / Unsplash