Upcoming Apktool 2.8.0

A few months ago I got pinged on the oddest message. I was being called out for not fixing Apktool obfuscation issues since I was allegedly being paid by a company to not fix the issue.

Knowing that was a complete lie - the real reason the bug tracker had expanded without crucial fixes was a combination of a few things.

1. My day/hobby life is no longer Android and APKs.
2. These newer obfuscation techniques have poked holes in the aged code base.
3. I had no drive/passion to work on it.

So one day I get a notification of a pull request from a user known as sv99. This is an alert letting me know that Google has decided to fork smali (A dependency of Apktool) and bring it up to date so I should switch to that version.

The maintainer of smali (Ben Grover) must be feeling the same burnout I had, because that repository had gone over a year without activity.

So with the child dependencies of Apktool getting a facelift I started noticing more and more pull requests coming in. sv99 was on a roll doing large refactors to take this Java 1.5 code base and bring it up to date with modern design patterns. I was nervous to merge these because my memory of the code base was weak and I was unsure of the impact of some of these changes.

So unfortunately days became weeks and I slowly went through these requests, but when I was done - no more came in. I went peeking on this author's GitHub to investigate and saw a comment directed towards me that said.

Original repository very slow receive PRs.

That was a little wake up I needed to give some love to Apktool again. If someone can spend unpaid time to help the project - the least I can do is review the request timely and/or share commit rights.

So off I went on a journey for some tasks.

Cleaning the bug tracker

The downside of any open source project in my opinion is the size of the backlog. If it isn't organized there is no chance of finding tasks to complete. What I'm finding as an annoyance is that all the features and bugs are mixed together in some pool.

So every day I have to look at over 100 issues that could be a mix of bugs and features.

I found a sweet feature of GitHub that will allow me to migrate all issues I tagged as "Feature" into a new discussion thread. I put up a warning that I was going to do this here and when I release Apktool 2.8.0 this migration will go live.

This paired with reviewing all tickets to merge/close issue will knock down my issue count to under 60 when this is done.

Revamping the docs

When you look at the current Apktool doc site - it just has aged a bit. It was created when we migrated away from the shutdown Google Code and was a basic template.

Most of this documentation just a more cleanly documented port of the old Google Code site. So I felt I needed to wake this up to entice more users to contribute documentation and notes.

I leveraged Docusaurus which I learned from React Native and started building it out.

Working at the moment to find a good medium between a modern design while at the same time not robbing the visitor from the crucial information they want to see immediately when visiting the site.

The code base is much more elegant to adapt the docs in comparison to the past so once I'm done porting old information in - the ability to introduce new docs and links to community tools will be much easier.

Fixing the attribute resolution

I wanted to dive into the code and partially fix a long standing bug with decoding the wrong attribute while parsing AXML files. As I started investigating this - it made no sense. We decode a value from a string pool and it was wrong.

I debated parsing the value from the resource map and it worked in that situation. So I was confused - it seemed obfuscation tools had found a trick that Android prefers the resource map over the string pool. So they fill the string pool with bogus/invalid values as they know exactly how Android parses the file.

// Android prefers the resource map value over what the String block has.
// This can be seen quite often in obfuscated apps where values such as:
// <item android:state_enabled="true" app:state_collapsed="false" app:state_collapsible="true">
// Are improperly decoded when trusting the String block.
// Leveraging the resource map allows us to get the proper value.
// <item android:state_enabled="true" app:d2="false" app:d3="true">
if (resourceMapValue != null) {
    return resourceMapValue;
}

if (stringBlockValue != null) {
    return stringBlockValue;
}

// In this case we have a bogus resource. If it was not found in either.
return "APKTOOL_MISSING_" + Integer.toHexString(resourceId);

So I adapted our parser and it fixed the issue, but I had to blow out some previous attempts at fixing the issue. It seemed it was the first time in awhile that I sat down and traced a problem to the source in order to apply the long-term intended patch.

Refactoring the chunk parser

As I was attempting to the fix attribute parser - I remembered how old Apktool code base had gotten. Maybe folks might not know, but Apktool started before AOSP was open source so it was built on pure research instead of viewing the specifications of the file formats.

So at times when I was comparing chunks to the spec. Apktool read an integer and shifted it into place and the spec was 2 shorts. Odd things like this make it incredibly difficult at times to line up the parser to the specification so I set off to rework that.

With this refactor I had a few goals:

• Support reading chunks in any order
• Unify the parser between AXML and ARSC

 private ResPackage[] readResourceTable() throws IOException, AndrolibException {
        Set<ResPackage> pkgs = new LinkedHashSet<>();
        ResTypeSpec typeSpec;

        chunkLoop:
        for (;;) {
            nextChunk();

            switch (mHeader.type) {
                case ARSCHeader.RES_NULL_TYPE:
                    readUnknownChunk();
                    break;
                case ARSCHeader.RES_STRING_POOL_TYPE:
                    readStringPoolChunk();
                    break;
                case ARSCHeader.RES_TABLE_TYPE:
                    readTableChunk();
                    break;

                // Chunk types in RES_TABLE_TYPE
                case ARSCHeader.XML_TYPE_PACKAGE:
                    mTypeIdOffset = 0;
                    pkgs.add(readTablePackage());
                    break;
                case ARSCHeader.XML_TYPE_TYPE:
                    readTableType();
                    break;
                case ARSCHeader.XML_TYPE_SPEC_TYPE:
                    typeSpec = readTableSpecType();
                    addTypeSpec(typeSpec);
                    break;
                case ARSCHeader.XML_TYPE_LIBRARY:
                    readLibraryType();
                    break;
                case ARSCHeader.XML_TYPE_OVERLAY:
                    readOverlaySpec();
                    break;
                case ARSCHeader.XML_TYPE_OVERLAY_POLICY:
                    readOverlayPolicySpec();
                    break;
                case ARSCHeader.XML_TYPE_STAGED_ALIAS:
                    readStagedAliasSpec();
                    break;
                default:
                    if (mHeader.type != ARSCHeader.RES_NONE_TYPE) {
                        LOGGER.severe(String.format("Unknown chunk type: %04x", mHeader.type));
                    }
                    break chunkLoop;
            }
        }

        if (mPkg.getResSpecCount() > 0) {
            addMissingResSpecs();
        }

        return pkgs.toArray(new ResPackage[0]);
    }

So I had a nice reworked loop to just parse chunks as we encountered them. Once again it felt right - the code base inherited over a decade ago was rewritten into a pattern that I understood and aligned to the AOSP specifications.

Revamping chat mediums

Finally, I just couldn't link towards Gitter anymore - I just never went to it. No one ever stayed around for responses anyway, much like IRC.

So now I'm solidified on Stack Overflow or GitHub Discussions.

So I'm finishing up the new docs site and ensuring no major regressions and excited to launch Apktool 2.8.0 and bring in a new modern era of patches and documentation.