Selenium: more Android sweets

Welcome back to our series of articles about efficient Selenium infrastructure. Just to be sure you have seen all of them here is the complete list:

• Selenium: a new hope (part I)
• Selenium: a new hope (part II)
• Selenium: done in 60 seconds
• Selenium on Windows: revisited
• Selenium: easy as a pie
• Selenium: an Apple story
• Selenium: Growing Muscles
• Selenium: Clear as a Bell
• Selenium: Back to the Moon

All these articles were mainly dealing with desktop browsers which are still very important in everyday life. However we already live in a mobile-first world with more than a half of Internet traffic coming from mobile devices. Today I would like to talk in detail about efficient Android infrastructure.

Avoid Real Devices

Being a newcomer to mobile testing domain it seems to be natural to use real devices for testing mobile applications and mobile web. You start by buying a phone or a tablet PC and manually do some quick checks in your application. Then you certainly want to cover more screen resolutions and all Android versions supported by your product. So you buy a dozen of friends to your first device. The idyll usually ends when all these devices devices occupy all your table and you run out of budget for mobile devices.

When working in a rich company things can go a bit further. You buy so many devices that it makes no sense to run the same test manually in every device. Some kind of automation is needed. You take a mini-computer, from the third attempt buy a USB hub with sufficient charging power, connect all your devices and … continue to suffer. Being able to run tests automatically you still have to do a lot of manual device maintenance. Even the best devices discharge, lose WiFi connection and logout from Google account. From time to time you have to manually confirm software updates on every device. You also need to buy a stand where devices will live. And the most annoying — even if you are doing everything correctly devices can still become unavailable just during your release test run. I have even heard about really big software companies with a dedicated engineer doing exactly one thing — maintaining hundreds of devices in alive state.

Unless you have a weighty reason to use only real devices we do not recommend to do this. Real devices are a nightmare and do not scale well.

How Android Automation Works

Before we dive into the details let’s quickly repeat how Android automation is usually working. It should not be surprising that most of Android automation components are developed by Google and delivered within Android SDK.

Android Debug Bridge

The first tool well known among Android developers is Android Debug Bridge (or just adb). It is a command line tool allowing to do the most common operations with real devices and emulators such as: installing, starting and stopping applications, copying files back and forth, forwarding network ports, executing shell commands, debugging running applications and many more.

Internally ADB consists of an agent-daemon (installed to every Android device or emulator) called adbd. This daemon is waiting for commands coming from ADB server installed on developer machine. An ADB server is listening for user commands and sends them to respective device or emulator. Usually commands are being sent with command-line client called just adb. By default adbd is turned off on real devices and you have to launch it by allowing "Debugging via USB" in settings.

Android Emulator

Android automation would be slightly more complicated without Android emulator. This is a desktop application showing you just the same screen as real device and allowing to test mobile applications without real devices. When launching an emulator you can choose from a list of predefined configurations corresponding to real device models.

When needed you can create your own configuration with custom Android version, screen size, device pixel ratio, SD card size and so on. Google delivers Android emulators for both ARM and x86 platforms. Internally an emulator represents a standalone virtual machine using KVM and Qemu technologies.

Android Instrumentation Framework

Android automation is also based on Android Instrumentation Framework which is a low-level Java API allowing to subscribe to any events occurring in tested application, e.g. starting application, typing text in the fields, clicking on a button et cetera. You can also send just the same events to any part of a running application. Thus you have full control on what’s happening with the application.

UIAutomator

The next component is called UIAutomator. It is also a set of Java APIs (internally using Android Instrumentation Framework) used for cross-application testing as well as testing the interaction between user and system applications. Every application is seen by UIAutomator as a black box, you don’t have any access to application internals. This allows to “see” an application in the same way your real user sees it. You can use UIAutomator directly and implement test suites as Java classes. However the most obvious disadvantage of such approach for experienced Selenium users is that UIAutomator is not Selenium compatible. You have to learn everything from scratch.

Chromedriver

The last piece of Android puzzle is called Chromedriver. It is a standalone binary using JSON-based Chrome DevTools protocol to send commands to browser. Just the same binary can automate both Chrome Desktop or Chrome Mobile. You also need it when automating so-called hybrid mobile applications — using a built-in Chrome-based browser component to show web-pages instead of native Android components. Chromedriver is distributed as a Selenium-compatible web-server and you can work with it directly from Selenium tests. However one binary can only work with one browser instance and usually we need a third-party tool to launch multiple Chromedriver instances on different ports to serve parallel Selenium tests.

Appium

We now know all important Android automation components and need an open-source Selenium-compatible web-server using one of these components depending on what the user requests. Such server exists and is called Appium. Implemented in node.js, Appium introduces so-called Mobile JSONWire Protocol — a superset of the WebDriver Protocol adding mobile-specific actions such as tapping on the screen, rotating device, unlocking it and many more. To work with these new actions Appium developers also provide a set of client libraries for different programming languages based on original Selenium code.

An efficient Android node

There are a lot of different ways to shuffle Android automation components. Let’s do a mental experiment and try to start emulator-based Android automation from scratch going step by step.

• 1. Usually you start playing with new technologies on a clean virtual machine in your preferred cloud platform. Having a running virtual machine you download Android SDK from Google web site, unpack it, create an Android Virtual Device (aka avd) and start an emulator. It works but is EXTREMELY slow. This is because an ARM emulator launched by default is not fully using CPU capabilities of the host machine and thus is very very slow.
• 2. You spend hours digging the documentation and finally understand that only an x86 emulator can work fast and only when launched with -enable-kvm flag. You recreate an AVD, add a new flag to command and... it fails to start. Why is that? If you remember Android emulator is a standalone virtual machine and KVM stands for kernel-based virtual machine. To work properly such machine requires a set of additional instructions to be supported in the host-machine CPU. And this is what is usually missing in standard virtual machines in the cloud. A fast Android emulator can only be launched on a hardware server or a particular type of virtual machines supporting nested virtualization.
• 3. On the next day you take a hardware server and successfully launch an x86-emulator. It works slightly faster but a lot of applications including Chrome Mobile crash on startup. Why is that? Because starting from Android 3 applications require 3D acceleration support. In order to work properly you have to configure such support by enabling respective GPU drivers bundled with Android SDK.
• 4. Launching Android emulator was not easy, right? But the rest should be simple. Having an Android emulator instance and installed ADB we start Appium server and successfully launch our first test. Awesome!

• 5. Having a working prototype let’s now try to scale it. This should be as easy as launching more Android emulators. You launch e.g. 10 Android emulators and rerun the same test using 10 threads. But what’s happening? — Your test are randomly freezing! This time ADB is culpable. If you remember adb was not initially created as a debug tool and thus is not so efficient when working with multiple emulators in parallel.
• 6. How could we overcome such obstacle? — Very easy, we need to somehow use one adb instance per Android emulator. To achieve this on one host we should launch adb and respective emulator in isolated environment. One of the best lightweight isolation solutions right now is Docker. So we need a Docker container with ADB and Android emulator running inside. What else should be in this container? First of all Docker containers are usually started in headless environment, i.e. on a server without real display. So we need to install a headless X-server instance, such as Xvfb. Then we certainly need to have an Appium instance inside container. Although Appium should be able to work with multiple sessions in parallel, it's a node.js application which is a single-threaded stuff. Finally if we are going to test mobile web with Chrome Mobile or have a hybrid application - we have to add a correct version of the Chromedriver binary.

• 7. That’s it? Not yet. If you try to build and start such Android image you will instantly understand that it starts very slowly. Depending on the hardware it can take up to 30–40 seconds to complete. A virtual machine starts slowly and Android emulator is not an exception. Can we do better? Certainly! In December 2017 Google released a cool Android emulator feature called Android Quick Boot which is very similar to hibernation feature in desktop computers. You do a cold boot of an emulator first and wait 30–40 seconds to have a started emulator. Then you stop an emulator (usually by sending SIGTERM) and it automatically saves a memory snapshot to file. When you launch an emulator next time this snapshot file is read and used to quickly restore emulator state. With a snapshot it takes approximately 5–10 seconds to start an emulator.

You know see that Android automation is not so simple. Even knowing all possible difficulties you may encounter it is not so easy to build correct Docker images with Android emulator inside. Fortunately we have good news for you — we have done all this work and open-sourced the results. Just take a look at what we have:

1. Complete build files for Android images
2. An automation script allowing to build your custom image
3. A collection of prebuilt images with Android 4.4 and above

To quickly test something with Android images above do the following:

• Start a container like the following:

$ docker run -d --name android-test -e ENABLE_VNC=true \ --privileged -p 4444:4444 -p 5900:5900 selenoid/android:6.0

• Set your Selenium URL to http://localhost:4444/wd/hub (or the host where you launched container).
• Open VNC client and use localhost:5900 to access Android emulator screen and selenoid as a password.

How to build custom images

All these ready to use images were created to test mobile applications and thus contain neither Chromedriver nor Chrome Mobile itself. By default an URL to tested mobile application APK should be passed from the test using Appium app capability. If you wish to test mobile web or to bundle your APK to images do the following:

• Clone image build files:

$ git clone git@github.com:aerokube/selenoid-images.git

• Place application or Chrome Mobile APK to selenium/android directory:

$ cp /path/to/chrome-mobile.apk selenoid-images/selenium/android

• Run automate_android.sh script and answer the questions:

$ cd selenium
$ ./automate_android.sh

Selenoid

Ready to use Android images were certainly created mostly for automated testing. What we need is a Selenium protocol implementation launching everything inside Docker containers. As you could probably guess such implementation exists and is called Selenoid. By installing Selenoid here is what you get out of the box:

1. Fully automated installation
2. Slightly faster test execution speed
3. An ability to launch the same test in multiple Android versions in parallel on the same host
4. When debugging running tests — an ability to see Android emulator screen for every running session
5. Per-session logs saved to separate files and also accessible via HTTP protocol (in case of Android emulator you will see Appium logs)
6. Per-session video files similarly to log files accessible via HTTP
7. An ability to change screen resolution as well as Android device skin by changing test capabilities

Here is a demonstration video showing most of these features:

Scaling it

The last question I would like to discuss is how to create and scale a fault-tolerant Selenium cluster with Android emulators. In our previous articles we have already mentioned an extremely efficient Selenium load balancer called Ggr. The main idea of Ggr is having a list of upstream Selenium hosts to randomly distribute Selenium session requests over these hosts. A cluster with Android emulator is not exclusion and what you need to do is:

1. Distribute a set of hardware servers of virtual machines with nested KVM to two or more datacenters
2. Create an htpasswd file with at least one user for Ggr
3. Create a Ggr XML configuration file for the user from previous step enumerating Android hosts
4. Start Ggr binary or container pointing to the files from previous steps. This can be done either on one of Android hardware server or on a small virtual machine in the cloud. KVM is not required.

The last 3 steps take approximately 5 minutes and Ggr is a reliable software known to work months without any issue. Anyway you obtain only a single Ggr instance in one datacenter which can also become unresponsive in case of power outage or network loss in this datacenter. To have a truly fault-tolerant cluster we need to do a bit more:

• Install the second instance of Ggr with just the same configuration files to another datacenter
• Remember that Selenium protocol is HTTP-based and install two instances of Nginx or another reverse proxy HTTP server load balancing requests across two Ggr instances. Configuration file can look like the following:

upstream ggr {
    server ggr-1.example.com:5555 weight=10  max_fails=3 fail_timeout=10s; # Here we run Ggr on non-standard port 5555 to be able to run it on the same host as Nginx
    server ggr-2.example.com:5555 weight=10  max_fails=3 fail_timeout=10s;
}

server {
  listen 4444 default_server;
  listen [::]:4444 default_server;

  location / {
    proxy_pass http://ggr;
  }
}

• Add a reliable network load balancer (usually distributed to multiple datacenters too) — how to do this depends on your cloud provider (possible keywords can be Google Cloud Load Balancing, AWS Elastic Load Balancing, Azure Load Balancer and so on). After setting up a load balancer you obtain a permanent IP address.
• Assign some human-readable domain name (e.g. selenium.example.com) to this IP address by adding A and AAAA-records to DNS. You can now run the tests against http://selenium.example.com:4444/wd/hub and everything will work like a charm.

We provide an example repository demonstrating possible configuration files allowing to set up Nginx to work with Ggr:

https://github.com/aerokube/ggr-nginx-config

I hope you now have a lot more understanding of how Android automation works inside and how to set it up without any stress. If you have any questions — don’t hesitate to ask in our Telegram support channel. See you soon!