Selenium: Moon Environment Provisioning

Image of a lunar farm used under a Creative Commons license from the Space Sciences Institute. Available at http://ssi.org/space-art/ssi-sample-slides/.

Hi there! Hope you are doing well and already have a reliable and cost-efficient browser automation infrastructure. If not — take a look at the previous articles in our blog. During the months since our previous article, we released a great Moon feature that gives a lot more power to Selenium file upload operation. Today I would like to explain why we implemented it and what are possible applications.

Traditional File Upload

Although file upload is not a standard W3C WebDriver protocol feature, all existing Selenium implementations are supporting file upload to a browser running remotely. An example file upload code looks like this:

WebElement input = driver.findElement(By.cssSelector("input[type='file']"));
driver.setFileDetector(new LocalFileDetector());
input.sendKeys("/path/to/file/on/machine/which/runs/tests");

Behind the scenes this code does the following:

1. File is packed to a Zip archive.
2. Resulting archive bytes are encoded using Base64 algorithm.
3. Encoded bytes are sent to Selenium API as JSON.
4. Selenium API deserializes JSON and unpacks this archive to a temporary directory.
5. File input field value is set to file path in temporary directory.

Exactly the same approach is used when you want to upload a browser extension or the entire browser configuration directory called browser profile. The only difference is that in that case serialized arhive is inserted to browser capabilities which are sent to session startup API (POST /session). An example code to upload a Chrome extension is as follows:

ChromeOptions options = new ChromeOptions();
options.addExtensions(new File("/path/to/extension.crx")); // This file is encoded to Base64 and added as a capability
WebDriver driver = new RemoteWebDriver(new URL("http://moon.aerokube.local/wd/hub"), options);

This approach has several important disadvantages:

1. It requires too much memory. We already understood that in existing approach all uploaded file bytes are transferred to Selenium API as JSON. To read this JSON Selenium API needs to fully load it into memory. Depending on your test scenario you may need to upload dozens (or even hundreds) of megabytes in every browser session. So every such upload will require at least the same amount of memory from Selenium API. Depending on implementation this amount can double or even triple because you may want to store full JSON, then decoded archive, then some temporary data while unzipping this archive and so on. So when you run the same test scenario in parallel from your CI server, you can easily have dozens of browser sessions running in parallel and thus memory consumption can easily add up to several hundred or even thousands of megabytes.
2. It does not scale well. This is a logical consequence of increased memory consumption. The majority of Selenium API calls transfer only a few kilobytes of data. Normally, to handle such requests 100–200 megavbytes of memory for Selenium API is more than sufficient. But just to handle file uploads you often may need to increase this to 1–2 gigabytes. Also all the traffic containing uploaded file is passing through the same network load balancer, so more files you upload — more network bandwidth you require.
3. Limited functionality. In existing implementations file upload operations are limited by these 3 cases described before: file input field upload, browser extension upload and browser profile upload. Only the first one is supported in all browsers. Browser extensions can only be uploaded to Chromium-based browsers and Firefox. Remote browser profile upload is known to work in Firefox only. Meanwhile it’s hard to predict what test automation engineer may need from the browser. Browser nowadays is not only a tool to read text pages with images included. It’s also a first-class user of more advanced features like cryptography to provide secure data transfer and validation, multimedia to play sounds and video, interacting with devices to have video and audio conferences or print web pages and so on.

Introducing Context

It’s clear that instead of existing approach we need a generic and efficient way of transferring arbitrary files to remote browser. Instead of uploading desired files in request body, we can do better:

1. All files that you wish to upload are packed to a single archive.
2. This archive is then made available for download using an HTTP URL. For example you can store it as CI server artifact or simply upload to an S3 bucket.
3. When creating a new browser session — you simply provide this download URL.
4. Archive with your files is automatically downloaded and unpacked to a predictable directory.
5. Your browser can use these files any time they are needed. Some files like extensions can be loaded during browser startup. Other files can be opened later during test scenario execution.

Any such files needed by concrete browser are called browser context. Recent Moon versions now support a new browser startup parameter called context. In case of Selenium this is just a Moon capability that you can pass like this:

{
  "browserName":"chrome",
  "moon:options":{"context":"https://example.com/browser-data.tar.gz"}
}

This capability as you can see contains an HTTP URL for the archive containing the files you wish to upload. Although Zip archives are popular in all operating systems, we decided to use Gzip (*.tar.gz) format more popular in Unix-based operating systems like Linux or MacOS. Compared to standard Zip this format supports stream unpacking and thus unpacking such archive consumes less memory. In case of Playwright there is no capabilities concept, so the same context HTTP URL can be passed as Playwright URL parameter:

var browser = await chromium.connect({ timeout: 0, wsEndpoint: 'ws://moon.aerokube.local/playwright/chrome/playwright-1.23.3?headless=false&context=<uri-encoded-url-goes-here>' })

Let’s now take a look at possible use cases of this feature.

Context Use Cases

Prerequisites

Although this task can seem straightforward for experienced developers (and it is!), let’s first of all create an archive with the data you wish to upload. For example, let’s assume that you need to upload the following files:

---- some-file.txt
---- some-directory
     |
     ---- another-file.xpi
     ---- my-video.mp4

To create a *.tar.gz archive containing these files, you need to execute one command:

$ tar cvzf browser-data.tar.gz some-file.txt some-directory

Under Windows there is no such command, so you can use 7zip to create a tar archive and then compress it with gzip.

Now make sure resulting browser-data.tar.gz file is accessible using an HTTP URL, for example https://example.com/browser-data.tar.gz. Having this URL we can execute example code.

Example 1. “Uploading” files to browser.

We are now going to get the same result as traditional file upload feature provides, but without actually uploading your files to Selenium API. When using context capability your archive will be automatically downloaded to the browser pod before new browser session is started. Files are always unpacked to user home directory. By default user name is just user (but you can change it in Moon configuration), so home directory path for this user is /home/user. When context archive is unpacked you will have directory structure like this:

/home/user
        |
        ---- some-file.txt
        ---- some-directory
             |
             ---- another-file.xpi
             ---- my-video.mp4

For example we want to test uploading some-file.txt file to some form. To do this with context you only need to find respective file upload field in your code and set its value to some-file.txt location, i.e. /home/user/some-file.txt. Resulting Selenium Java code may look like:

ChromeOptions options = new ChromeOptions();
options.setCapability("moon:options", Map.of("context", "https://example.com/browser-data.tar.gz"));
WebDriver driver = new RemoteWebDriver(new URL("http://moon.aerokube.local/wd/hub"), options);
// Open test page here
WebElement input = driver.findElement(By.cssSelector("input[type='file']"));
input.sendKeys("/home/user/some-file.txt");

In this example moon.aerokube.local is Moon default domain when Moon is running in Minikube. If your Moon instance is running on another domain - use it instead.

Probably you could have heard about our new Selenium client called Lightning. If not — take a look at this article. This client supports Moon specific capabilities out of the box, so the same code in Lightning will look like this:

Capabilities capabilities = Capabilities.create().chrome()
        .extension(MoonCapabilities.class).context("https://example.com/browser-data.tar.gz");
WebDriver driver = WebDriver.create("http://moon.aerokube.local/wd/hub", capabilities);
// Open test page here
WebElement input = driver.elements().findFirst(By.cssSelector("input[type='file']"));
input.sendKeys("/home/user/some-file.txt");

The same code in Python will be:

from selenium.webdriver.chrome.options import Options
options = Options()
options.set_capability("moon:options": {"context": "https://example.com/browser-data.tar.gz"})

driver = webdriver.Remote(
    command_executor='http://moon.aerokube.local/wd/hub',
    options=options
)
# Open test page here
input = driver.find_element(By.CSS_SELECTOR, "input[type='file']")
input.send_keys("/home/user/some-file.txt");

Example 2. Using browser extensions.

We now understand how context feature works. Let's apply this knowledge and configure browser to use an extension. For example extension files for Chromium-based browsers end with *.crx and files for Firefox have *.xpi extension. Internally both extension files are just Zip archives with Javascript files inside. Every browser allows to load unpacked extensions from local file system. To do this in Google Chrome you need to add command line flags pointing to unpacked extension directory:

google-chrome --disable-extensions-except=/path/to/unpacked/extension --load-extension=/path/to/unpacked/extension

To use this in Selenium we need to repack extension contents to *.tar.gz. If extension file is called extension.crx, correct commands would be:

$ unzip extension.crx -d extension # Unpack extension contents to a directory called extension
$ tar cvzf extension.tar.gz extension # Pack the directory from previous step to *.tar.gz archive

Now when you provide an URL to extension.tar.gz as context capability, this archive will be unpacked to /home/user and extension contents will reside in /home/user/extension directory. We just need to pass Google Chrome command line flags to load extension from this directory using capabilities. An example Java code will look like:

ChromeOptions options = new ChromeOptions();
options.setCapability("moon:options", Map.of("context", "https://example.com/extension.tar.gz"));
options.addArguments("disable-extensions-except=/home/user/extension", "load-extension=/home/user/extension");
WebDriver driver = new RemoteWebDriver(new URL("http://moon.aerokube.local/wd/hub"), options);

The same in Lightning client:

Capabilities capabilities = Capabilities.create().chrome()
        .args("disable-extensions-except=/home/user/extension", "load-extension=/home/user/extension")
        .extension(MoonCapabilities.class).context("https://example.com/extension.tar.gz");
WebDriver driver = WebDriver.create("http://moon.aerokube.local/wd/hub", capabilities);

The same in Python:

from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("disable-extensions-except=/home/user/extension")
options.add_argument("load-extension=/home/user/extension")
options.set_capability("moon:options": {"context": "https://example.com/extension.tar.gz"})
driver = webdriver.Remote(
    command_executor='http://moon.aerokube.local/wd/hub',
    options=options
)

In the video below we are using context to upload Chrome extension with 2048 game and emulate playing this game:

Example 3. Overriding full browser profile.

Exactly the same approach can be used to completely replace browser profile directory. To do this — you pack this directory to profile.tar.gz archive:

$ tar cvzf profile.tar.gz profile # Here we assume that profile directory is called just profile

To use this profile in the browser you need to pass a command line flag. In Chrome this will look like --user-data-dir=/home/user/profile, for Firefox flag will be --profile=/home/user/profile. Respective Java code will look like this:

FirefoxOptions options = new FirefoxOptions();
options.setCapability("moon:options", Map.of("context", "https://example.com/profile.tar.gz"));
options.addArguments("--profile=/home/user/profile");
WebDriver driver = new RemoteWebDriver(new URL("http://moon.aerokube.local/wd/hub"), options);

The same in Lightning client:

Capabilities capabilities = Capabilities.create().firefox()
        .args("--profile=/home/user/profile")
        .extension(MoonCapabilities.class).context("https://example.com/profile.tar.gz");
WebDriver driver = WebDriver.create("http://moon.aerokube.local/wd/hub", capabilities);

The same in Python:

from selenium.webdriver.firefox.options import Options
options = Options()
options.add_argument("--profile=/home/user/profile")
options.set_capability("moon:options": {"context": "https://example.com/profile.tar.gz"})
driver = webdriver.Remote(
    command_executor='http://moon.aerokube.local/wd/hub',
    options=options
)

Example 4. Web-camera and microphone emulation.

You can now see that the same uniform context approach is universal and allows to efficiently replace file upload, browser extension upload and complete browser profile upload. But let's do something completely new. For example some web applications are now interacting with web camera and microphone. In order to test that this functionality works, you may need to send predictable video and audio stream to emulate how real user interacts with tested application. Fortunately, in Chrome you can mock web camera and microphone just by passing some more command line flags:

$ google-chrome --disable-gpu --use-fake-ui-for-media-stream --use-fake-device-for-media-stream --use-file-for-fake-video-capture=/path/to/webcam-video.y4m --use-file-for-fake-audio-capture=/path/to/mic-sound.wav

As you can see microphone sound should be in old-school *.wav format, whereas video file should be in *.y4m format. So let's first of all prepare such file from any *.mp4 video. This can be achieved with one ffmpeg command:

$ mkdir fake-media
$ ffmpeg -i my-video.mp4 -vf hflip -pix_fmt yuv420p -s 1280x720 fake-media/webcam-video.y4m # We convert my-video.mp4 to webcam-video.y4m
$ cp mic-sound.wav fake-media/mic-sound.wav # Copy microphone sound if needed
$ tar cvzf fake-media.tar.gz fake-media # Create context archive

Having context archive, create Selenium session with new command-line flags like we did in previous example. Selenium Java code will be:

ChromeOptions options = new ChromeOptions();
options.setCapability("moon:options", Map.of("context", "https://example.com/fake-media.tar.gz"));
options.addArguments("disable-gpu", "use-fake-ui-for-media-stream", "use-fake-device-for-media-stream", "use-file-for-fake-video-capture=/home/user/fake-media/webcam-video.y4m", "use-file-for-fake-audio-capture=/home/user/fake-media/mic-sound.wav");
WebDriver driver = new RemoteWebDriver(new URL("http://moon.aerokube.local/wd/hub"), options);

The same in Lightning client:

Capabilities capabilities = Capabilities.create().chrome()
        .args("disable-gpu", "use-fake-ui-for-media-stream", "use-fake-device-for-media-stream", "use-file-for-fake-video-capture=/home/user/fake-media/webcam-video.y4m", "use-file-for-fake-audio-capture=/home/user/fake-media/mic-sound.wav")
        .extension(MoonCapabilities.class).context("https://example.com/fake-media.tar.gz");
WebDriver driver = WebDriver.create("http://moon.aerokube.local/wd/hub", capabilities);

The same in Python:

from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("disable-gpu")
options.add_argument("use-fake-ui-for-media-stream")
options.add_argument("use-fake-device-for-media-stream")
options.add_argument("use-file-for-fake-video-capture=/home/user/fake-media/webcam-video.y4m")
options.add_argument("use-file-for-fake-audio-capture=/home/user/fake-media/mic-sound.wav")
options.set_capability("moon:options": {"context": "https://example.com/fake-media.tar.gz"})
driver = webdriver.Remote(
    command_executor='http://moon.aerokube.local/wd/hub',
    options=options
)

This how a resulting test execution will look like:

Conclusion

In this article we introduced a new efficient Moon file upload feature called context. We hope that is helps your team to build a truly efficient browser automation infrastructure. Don’t hesitate to send your interesting use cases of this feature to support(at)aerokube.com and we would be glad to describe them in Moon documentation. If you don’t plan to deploy your own infrastructure but want to use all powerful Moon features — take a look at Moon Cloud.