Reproducing Android app deployments (or playing Angry Birds on NixOS)

Some time ago, I did a couple of fun experiments with my Android phone and the Android SDK. Moreover, I have developed a function that can be used to automate Android builds with Nix.

Not so long ago, somebody asked me if it would be possible to run arbitrary Android apps in NixOS. I realised that this was exactly the goal of my fun experiments. Therefore, I think it would be interesting to report about it.

Obtaining Android apps from a device

Besides development versions of apps that can be built with the Android SDK and deployed to a device or emulator through a USB connection, the major source of acquiring Android apps is the Google Playstore.

Although most devices (such as my phone and tablet) bundle the Google Playstore app as part of their software distributions, the system images that come with the Android SDK do not seem to have the Google Playstore app included.

Despite the fact that emulator system images do not have the Google Playstore app installed, we can still get most of the apps we want deployed to an emulator instance. What I typically do is installing an app on my phone with the Google Playstore, then downloading it from my phone and installing it in an emulator instance.

If I attach my phone to the computer and enable USB debugging on my device, I can run the following command to open a shell session:

$ adb -d shell

While navigating through the filesystem, I discovered that my phone stores apps in two locations. The system apps are stored in /system/app. All other apps reside in /data/app. One of the annoying things about the latter folder is that root access to my phone is restricted and I'm not allowed to read the contents of it:

$ cd /data/app
$ ls
opendir failed, Permission denied

Later I discovered that Android distributions use a tool called pm to deploy Android packages. Running the following command-line instruction gives me an overview of all the installed packages and the locations where they reside on the filesystem:

$ pm list packages -f
package:/system/app/GoogleSearchWidget.apk=android.googleSearch.googleSearchWidget
package:/data/app/com.example.my.first.app-1.apk=com.example.my.first.app
package:/system/app/KeyChain.apk=com.android.keychain
package:/data/app/com.appcelerator.kitchensink-1.apk=com.appcelerator.kitchensink
package:/system/app/Shell.apk=com.android.shell
package:/data/app/com.capcom.smurfsandroid-1.apk=com.capcom.smurfsandroid
package:/data/app/com.rovio.angrybirds-2.apk=com.rovio.angrybirds
package:/data/app/com.rovio.BadPiggies-1.apk=com.rovio.BadPiggies
package:/data/app/com.android.chrome-2.apk=com.android.chrome
...

As can be seen, the package manager shows me the location of all installed apps, including those that reside in the folder that I could not inspect. Moreover, downloading the actual APKs files through the Android debugger does not seem to be restricted either. For example, I can run the following Android debugger instruction to obtain Angry Birds that I have installed on my phone:

$ adb -d pull /data/app/com.rovio.angrybirds-2.apk
5688 KB/s (45874906 bytes in 7.875s)

Running arbitrary APKs in the emulator

In my earlier blog posts on automating Android builds with Nix, I have described how I implemented a Nix function (called androidenv.emulateApp { }) that generates scripts spawning emulator instances in which a development app is automatically deployed and started.

I have adapted this function to make it more convenient to deploy existing APKs and to make it more suitable for running apps for other purposes than development:

• The original script stores the state files of the emulator instance in a temp folder, which gets discarded afterwards. For test automation this is quite useful in most cases. However, we don't want to lose our savegames while playing games. Therefore, I added a parameter called avdHomeDir allowing someone to store the state files in a non-volatile location on the filesystem, such as the user's home directory. If this parameter is not provided, the script remains to use a temp directory.
• Since we want to keep the state of the emulator instance around, there is also no need to create it every time we launch the emulator. I have adapted the script in such a way that it only creates the AVD if it does not exists. Running the following instruction seems to be sufficient to check whether the AVD exists:
  
  

  $ android list avd | grep "Name: device"
  

  
  
• The same thing applies to the app that gets deployed to the emulator instance. It's only supposed to be deployed if it is not installed yet. Running the following command-line instruction did the trick for me:
  
  

  $ adb -s emulator-5554 pm list packages | \
      grep package:com.rovio.angrybirds
  package:com.rovio.angrybirds
  

  
  It shows me the name of the package if it is installed already.
  

Automatically starting apps in the emulator

As described in my earlier blog post, the script that launches the emulator can also automatically start the app. To do this, we need the Java package identifier of the app and the name of the start activity. While developing apps, these properties can be found in the manifest file that is part of the development repository. However, it's a bit trickier to obtain these attributes if you only have a binary APK.

I have discovered that the aapt tool (that comes with the Android SDK) is quite useful to find what I need. While running the following command-line instruction with the Angry Birds APK, I discovered the following:

$ aapt l -a com.rovio.angrybirds-2.apk
...
A: package="com.rovio.angrybirds" (Raw: "com.rovio.angrybirds")

E: application (line=47)
      A: android:label(0x01010001)="Angry Birds" (Raw: "Angry Birds")
      A: android:icon(0x01010002)=@0x7f020001
      A: android:debuggable(0x0101000f)=(type 0x12)0x0
      A: android:hardwareAccelerated(0x010102d3)=(type 0x12)0x0
      E: activity (line=48)
        A: android:theme(0x01010000)=@0x1030007
        A: android:name(0x01010003)="com.rovio.fusion.App" (Raw: "com.rovio.fusion.App")
        A: android:launchMode(0x0101001d)=(type 0x10)0x2
        A: android:screenOrientation(0x0101001e)=(type 0x10)0x0
        A: android:configChanges(0x0101001f)=(type 0x11)0x4a0
        E: intent-filter (line=49)
          E: action (line=50)
            A: android:name(0x01010003)="android.intent.action.MAIN" (Raw: "android.intent.action.MAIN")
          E: category (line=51)
            A: android:name(0x01010003)="android.intent.category.LAUNCHER" (Raw: "android.intent.category.LAUNCHER")

Somewhere at the end of the output, the package name is shown (com.rovio.angrybirds) and the app's activities. The activity that supports android.intent.action.MAIN intent is actually the one we are looking for. According to the information that we have collected, the start activity that we have to call is named com.rovio.fusion.App.

Writing a Nix expression

Now that we have retrieved the Angry Birds APK and discovered the attributes to automatically start it, we can automate the process that sets up an emulator instance. I wrote the following Nix expression to do this:

with import <nixpkgs> {};

androidenv.emulateApp {
  name = "angrybirds";
  app = ./com.rovio.angrybirds-2.apk;
  platformVersion = "18";
  useGoogleAPIs = false;
  enableGPU = true;
  abiVersion = "x86";
  
  package = "com.rovio.angrybirds";
  activity = "com.rovio.fusion.App";

  avdHomeDir = "$HOME/.angrybirds";
}

The above Nix expression sets the following parameters:

• The name parameter is simply used to make the Nix store path better readable.
• The app parameter points to the Angry Birds APK that I just downloaded from my phone. It gets automatically installed in the spawned emulator instance.
• platformVersion refers to the API-level of the system image that the emulator runs. API-level 18 corresponds to Android version 4.3
• If we need Google specific functionality (such as Google Maps) we need a Google API-enabled system image. Angry Birds does not seem to require it.
• To allow games to run smoothly, it's better to enable hardware GPU emulator/acceleration through the enableGPU parameter
• The abiVersion sets the CPU architecture of the emulator. Most apps are actually developed for armeabi-v7a and this is usually the safest or the only option that works (unless the app is not using any native code or supports other desired architectures). Angry Birds also supports x86 which can be much faster emulated.
  
• The package and activity parameters are used to automatically start the app
• We use the avdHomeDir parameter to persistently store the state of the emulator in the .angrybirds folder of my home directory, so that the progress is retained.

I can build the earlier Nix expression with the following command:

$ nix-build angrybirds.nix

And then play Angry Birds, by running:

./result/bin/run-test-emulator

The above script starts the emulator, installs Angry Birds, and starts it. This is the result (to rotate the screen I used the 7 and 9 keys on the numpad):

Isn't it awesome? ;)

Transferring state

I also discovered how to transfer the state of apps (such as settings and savegames) from a device to an emulator instance and vice-versa. For some games, you can obtain these through Android's backup functionality. The following instruction makes a backup from my phone of the state of a particular app:

$ adb -d backup com.rovio.angrybirds -f state
Now unlock your device and confirm the backup operation.

When running the above instruction, you'll be asked for confirmation for making the backup and some details to optionally encrypt it.

With the following instruction, I can restore the captured state in the emulator:

$ adb -s emulator-5554 restore com.rovio.angrybirds -f state
Now unlock your device and confirm the backup operation.

While running the latter operation, you'll also be asked for confirmation.

Conclusion

In this blog post, I have described how we can automatically deploy existing Android APKs in an emulator instance using the Nix package manager. I have used it to play Angry Birds and a couple of other Android games in NixOS.

There are few caveats that you have to keep in mind:

• I have observed that quite a few apps, especially games, have native dependencies. Most of these games only seem to work on ARM-based systems. Although x86 images are much faster to emulate, you will not benefit from the speed boost they may give you if this CPU architecture is not supported.
• Some apps use Google API specific functionality. Unfortunately, the Android SDK does not provide non-ARM based system images that support them. In a previous blog post, I have developed a Nix expression that can be used to create x86 Google API enabled system images from the ARM-based images, although it may be a bit tricky to set them up.
• Some apps may install additional files besides the APK when they are installed through the Google Playstore. For me running adb logcat and inspecting the error messages in the logs helped me out a few times.

Availability

The androidenv.emulateApp { } function is part of Nixpkgs.

It's also important to point out that the Nixpkgs repository does NOT contain any prepackaged Android games or apps. You have to obtain and deploy these apps yourself!