Apktool: Fixing Bug 713

This is my first technical blog exploring the path of fixing one of Apktool's more complicated bugs

Around December of 2014, Lollipop builds of LG started being leaked to the public. I got a bug report, shortly after, saying that decoding had stopped working in Apktool. Already having previously patched Lollipop support I was quite confused in what had broken.

So lets step in and see what the problem is.

$apktool if lge-res.apk -t bug713
I: Framework installed to: /home/ibotpeaches/apktool/framework/0-bug713.apk
$apktool if framework-res.apk -t bug713
I: Framework installed to: /home/ibotpeaches/apktool/framework/1-bug713.apk

Then lets do the decode.

$apktool d UnifiedEULA.apk -t bug713
I: Using Apktool 2.0.0-RC3 on UnifiedEULA.apk
I: Loading resource table
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: /home/ibotpeaches/apktool/framework/1-bug713.apk
W: Could not decode attr value, using undecoded value instead: ns=android, name=label, value=0x7f080000
W: Could not decode attr value, using undecoded value instead: ns=android, name=icon, value=0x7f02000c
W: Could not decode attr value, using undecoded value instead: ns=android, name=label, value=0x7f08001e
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
Exception in thread "main" brut.androlib.err.UndefinedResObject: resource spec: 0x7f080000

Wait what? Why did lge-res.apk install as 0-bug713.apk? The leading integer represents the packageId, and I haven't ever seen a valid packageId of 0 before. Generally Android uses 1 for framework-res.apk and 2-9 for other framework files. So we have spotted our first error in that lge-res.apk is installing incorrectly.

Secondly, UnifiedEULA.apk is failing to decode with a few resources. If we examine the first resource 0x7f080000. We can pull that resourceId apart to understand it.

• 0x7f = packageId
• 0x08 = typeId
• 0x0000 = entryId

So a packageId of 0x7f represents a local package. This means the resource table we are looking for is present in this APK (UnifiedEULA). So why didn't it find it? Lets double check using aapt (Android Asset Packaging Tool) to make sure this resource is in fact here.

$aapt d resources UnifiedEULA.apk | grep '0x7f080000'
  spec resource 0x7f080000 com.lge.eula:string/app_name: flags=0x00000004
    resource 0x7f080000 com.lge.eula:string/app_name: t=0x03 d=0x00000060 (s=0x0008 r=0x00)

So yes, the resource we are looking for is infact in this APK. This points to a fault in the decoding process of Apktool.

So why is lge-res installing as pkgId 0?

Lets go back to our trustworthy aapt and see what it has to say.

$aapt d resources lge-res.apk | more
  Package Groups (1)
   Package Group 0 id=0x02 packageCount=1 name=com.lge
    Package 0 id=0x00 name=com.lge

Uh what. How come aapt is reporting the packageId as 0x02 but Apktool is identifying it as 0x00?

Lets take a peek into the source of Apktool and see if a mistake is occurring.

Well thats not good. Reading the int directly from the stream reports as 0 as well. So lets take a look at a hex dump of the resources.arsc file.

As you can see the int (4 bytes prior to highlighted packageName) is 0. So this means we have a change in AOSP that has changed the basic storage of packageIds.

I started to look at the release documents of Lollipop but found nothing that had anything to do with packageIds. So off to the repository I go. I generally look in platform_frameworks_base and the subfolder tools/aapt. This gave me about ~100 commits to look through between Kitkat and now.

I started skimming the commit messages and spotted "Shared library resource support". This sounded like something related and sure enough the commit message confirmed.

Shared libraries can now export resources for applications to use.
    
Exporting resources works the same way the framework exports
resources, by defining the public symbols in res/values/public.xml.
    
Building a shared library requires aapt to be invoked with the
--shared-lib option. Shared libraries will be assigned a package
ID of 0x00 at build-time. At runtime, all loaded shared libraries
will be assigned a new package ID.
    
Currently, shared libraries should not import other shared libraries,
as those dependencies will not be loaded at runtime.
    
At runtime, reflection is used to update the package ID of resource
symbols in the shared library's R class file. The package name of
the R class file is assumed to be the same as the shared library's
package name declared in its manifest. This will be customizable in
a future commit.
    
See /tests/SharedLibrary/ for examples of a shared library and its
client.
    
Bug:12724178
Change-Id: I60c0cb8ab87849f8f8a1a13431562fe8603020a7

So incase you missed it. Check out the 3rd paragraph.

Building a shared library requires aapt to be invoked with the
--shared-lib option. Shared libraries will be assigned a package
ID of 0x00 at build-time. At runtime, all loaded shared libraries will be assigned a new package ID.

So we have found our culprit. This APK is using this new method of shared library resources. Thus, its packageId is set at runtime instead of buildtime.

The problem still remains though that our APKs can't decode unless they know the packageId of the resources. If you remember from above though, aapt had correctly determined the packageId of 0x02. How did it do that?

We take a look at the libs/androidfw/ResourceTypes.cpp file.

ResTable::ResTable()
-    : mError(NO_INIT)
+    : mError(NO_INIT), mNextPackageId(2)

As you can see from the diff report. The packageId is literally just hardcoded as 2.

So I patched Apktool quite abruptly with.

if (id == 0) {
    // This means we are dealing with a Library Package, we should just temporarily
    // set the packageId to the next available id . This will be set at runtime regardless, but
    // for Apktool's use we need a non-zero packageId.
    // AOSP indicates 0x02 is next, as 0x01 is system and 0x7F is private.
    id = 2;
}

Would it be that simple? Lets try again.

$apktool if lge-res.apk -t bug713
I: Framework installed to: /home/ibotpeaches/apktool/framework/2-bug713.apk

Good good. We have a correct packageId now. Lets try the decode again. I'll save you the stacktrace and just tell you. It failed, with the same error as before.

So why aren't resources being decoded? In Apktool we have two sets that hold all the resources decoded.

private final Set<ResPackage> mMainPackages = new LinkedHashSet<ResPackage>();
private final Set<ResPackage> mFramePackages = new LinkedHashSet<ResPackage>();

with each ResPackage containing a Map of resources indexed by the resourceId.

private final Map<ResID, ResResSpec> mResSpecs = new LinkedHashMap<ResID, ResResSpec>();

Each ResResSpec contains

public ResResSpec(ResID id, String name, ResPackage pkg, ResType type) {
    this.mId = id;
    this.mName = name;
    this.mPackage = pkg;
    this.mType = type;
}

So I set a breakpoint during this failure and was astounded. The mMainPackages count for mResSpecs was at 0. We had 0 ResResSpecs. So decoding was failing badly. Badly enough that not even a single resource was decoded.

So I went as far back into the Apktool procedure where mResSpecs was populated. Back in the process of Apktool, during decoding of the ResPackage, we read the ResTypes which in turn reads the configs (readConfigs), which in turn reads the resources (readEntry) of that configuration.

while (mHeader.type == Header.TYPE_TYPE) {
    readType();
}

So why was this check (shown above) failing? Header.TYPE_TYPE was 0x0202, and mHeader.type was 0x0203 and Apktool has no support for whatever that was corresponding to.

It appears we have something new, so back to the source (include/androidfw/ResourceTypes.h) we go.

-    RES_TABLE_TYPE_SPEC_TYPE    = 0x0202
+    RES_TABLE_TYPE_SPEC_TYPE    = 0x0202,
+    RES_TABLE_LIBRARY_TYPE      = 0x0203

We have found the new type and its called LIBRARY_TYPE. This type is before TYPE_SPEC_TYPE in the current APK. So Apktool is failing because it has no support for these. So referencing the linked commit above I found these additional structs.

/**
 * A package-id to package name mapping for any shared libraries used
 * in this resource table. The package-id's encoded in this resource
 * table may be different than the id's assigned at runtime. We must
 * be able to translate the package-id's based on the package name.
 */
struct ResTable_lib_header
{
    struct ResChunk_header header;

    // The number of shared libraries linked in this resource table.
    uint32_t count;
};

/**
 * A shared library package-id to package name entry.
 */
struct ResTable_lib_entry
{
    // The package-id this shared library was assigned at build time.
    // We use a uint32 to keep the structure aligned on a uint32 boundary.
    uint32_t packageId;

    // The package name of the shared library. \0 terminated.
    char16_t packageName[128];
};

So in short. If we encounter a chunk with a value of 0x0203 or TYPE_LIBRARY in the header, we then need to iterate over count and read the packageId & packageName for each count. Adding a quick patch into the brut.apktool/apktool-lib/src/main/java/brut/androlib/res/decoder/ARSCDecoder.java file.

We add the following function.

private void readLibraryType() throws AndrolibException, IOException {
    checkChunkType(Header.TYPE_LIBRARY);
    int libraryCount = mIn.readInt();

    int packageId;
    String packageName;

    for (int i = 0; i < libraryCount; i++) {
        packageId = mIn.readInt();
        packageName = mIn.readNullEndedString(128, true);
        LOGGER.info(String.format("Decoding Shared Library (%s), pkgId: %d", packageName, packageId));
    }
}

With the check added in readPackage() we are left with this new functionality.

     mPkg = new ResPackage(mResTable, id, name);
     nextChunk();
         
+    while (mHeader.type == Header.TYPE_LIBRARY) {
+        readLibraryType();
+    }

     while (mHeader.type == Header.TYPE_TYPE) {
         readType();
     }

This full commit may be found here. So we now are going to check for TYPE_LIBRARY and TYPE_TYPE respectively. Time to test again with our previously broken UnifiedEULA.apk.

$apktool d UnifiedEULA.apk -t bug713
I: Using Apktool 2.0.0-RC3 on UnifiedEULA.apk
I: Loading resource table...
I: Decoding Shared Library (com.lge), pkgId: 2
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: /home/ibotpeaches/apktool/framework/1-bug713.apk
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
I: Baksmaling classes.dex...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...
$

Now lets check if the shared resource is correctly referenced during rebuild.

$ apktool b UnifiedEULA
I: Using Apktool 2.0.0-RC3
I: Checking whether sources has changed...
I: Smaling smali folder into classes.dex...
I: Checking whether resources has changed...
I: Building resources...
I: Building apk file...
$

It built! So we correctly found the undocumented feature of "shared library resources" and added preliminary support for it.

That was the journey of fixing bug 713 of Apktool. While writing about it was quite easy, there was lots of delays in the original patch while I looked for commits, source, etc.

Thanks for the read and hope you liked my first technical writeup. Comments/critiques good or bad are appreciated!.