Appium: Automated App Testing

Published in

Building Niche

10 min readNov 13, 2018

source: https://www.testdevlab.com/blog/2018/01/how-we-unite-android-and-ios-test-automation-with-appium/

Automated front-end testing at Niche has really evolved over the past 2 years. It started with Nightmare.js, and quickly transitioned over to Nightwatch.js completing E2E tests. And then we added visual diffs. This led to an exponential increase in the rate of publishes. Once the alpha version of our app came along, it was obvious that we needed the same rigorous testing standards in place, and they needed to be cross-platform.

My Background

This quarter I’ve been celebrating. I just finished my first year in tech and at Niche. When I was in college, I started a cleaning company and was always very grateful to have that job and great relationships with customers. But I knew that time was ticking and I needed to challenge my mind, as it had been a few years since graduation, and my back was sore.

So I transitioned and built on the knowledge that I had from school. Two courses, statistics and database management at CMU, were turning points for me. I was an econ major for my undergrad, but these courses in particular got me excited about data and the stories you can parse out of it. After a few years, I was ready to revisit that. I began reviewing SQL principles and teaching myself to code on Codecademy. I found a job listing for QA Analyst at Niche and kept it open in a browser tab for weeks. It was my ‘apply when I feel ready’ job. QAs here do a lot of database validation and the hiring process requires a SQL portion. For better or worse, I had just gotten through the July/August moving season, where we did nothing but clean the apartments of Carnegie Mellon and Pitt students changing leases, and I had had enough. This had been my 7th summer cleaning up the biohazards left behind, and I was done. I decided to just do it, just apply, see what happened. And that’s when my great fortune began.

Luckily, I found myself interviewing for a team that is especially scrappy, even compared to Niche overall, where we have many people who took a non-traditional route into tech. I got the job and started on September 25, 2017. It would take until March when I finally stopped cleaning in the evenings and trusted that the new job would stick.

The Task

So now, after a year of being here and learning a ton,I was given the opportunity to join Team Alpha, Niche’s mobile app team, and be the lead QA spearheading app testing automation. I’ve been consumed by this, given full range to just explore and research and tackle it.

source: http://www.provasolutions.com/native-and-hybrid-app-testing

Choosing Appium

The main requirement of our app testing tool was the ability to handle a hybrid app. That is, can it test an app that has both native and WebViews? This was essential. The Niche mobile app is a combination of native code (powered by React-Native) and WebViews. WebViews allow us to display web content while staying within the app. So instead of needing to replicate a feature of the Niche website in native code, we can use a WebView to display the Niche website.

However, WebViews do not have all the features of mobile browsers, and we can’t be certain that the website accessed via WebView will behave in the same way as it does when accessed in a browser. We also cannot test them in the same way we test entirely native screens in the mobile app. Ultimately, after considering a few different options, I settled on Appium. Appium is coupled with WebdriverIO Testrunner, which we already use for visual diffs, as well as having the option to incorporate Selenium, which we use for Nightwatch. It complemented the rest of our tooling.

Setting up Appium for app testing isn’t very complicated, but there are a lot of moving parts. I wasn’t able to find one definitive guide to configure everything exactly the way I wanted, which was using Appium server, WDIO Testrunner, and Mocha framework with Chai assertions, specifically for app testing, rather than just mobile browser testing. Also, there wasn’t a whole lot of explanation surrounding the guides that I did cobble together to find my way.

In order to run tests on both Android and iOS devices, you will need Android Studio and Xcode, respectively. Unfortunately, Xcode is only available for those using a Mac. These 2 pieces of software are where device simulators are found. Within each, you can choose several different Android and iOS simulators, representing different models available for purchase. You can also choose various operating system versions.

This is also assuming that you already have Node.js and Java running on your system. If you don’t, you’ll want to get those installed.

Android Studio:

1) To begin, install Android Studio

Everything you need to test on an Android device is included with Android Studio (you used to have to download SDK Manager separately).
Once the installation completes, open Android SDK Manager (upper right corner of UI), where you will find the path for ANDROID_HOME, you will need this shortly.
SDK Manager can be used to download different Android operating systems to test on — a good start is 8.0 Oreo, more can be added later.

Android SDK Manager with a variety of SDK platforms available

Next you’ll use the AVD Manager to create an Android emulator (next to SDK Manager) — choose “Create Virtual Device” to begin. You’ll need to have an Android emulator running to use Appium with it.

Android Virtual Device (AVD) Manager with 2 emulators installed

Appium:

2) Download and install Appium Desktop — this has everything we need.

3) Globally install Appium Doctor via npm — npm i -g appium-doctor

By running appium-doctor command, you will be prompted to resolve gaps in your configuration for Appium
This will include setting system and environment variables: JAVA_HOME, %JAVA_HOME%/bin, and ANDROID_HOME
Once Appium Doctor returns a clean bill of health, you’re ready to configure a session.

4) Opening Appium, you’ll see a simple UI. Keep the default Host and Port settings. Click “Start Server”

5) In the server window, open the menu and choose “New Session Window” this is where you’ll add Desired Capabilities

Choose Automatic Server
Add in Desired Capabilities one at a time. Basics include deviceNamewith a value of Pixel_2, or Orientation= Portrait
You can find many possible configurations here, but they will change from project to project

The Appium UI will convert your list to json, e.g.,

{
  “appActivity”:   “MainActivity”,
  “appPackage”: “com.example.testapp”,
  “automationMName”: “UiAutomator2”,
  “platformName”: “Android”,
  “deviceName”: “Pixel_2”,
  “app”: “/Path/to/your/appfile.apk”
}

You can save your list of Desired Capabilities, which makes jumping between projects less cumbersome.
Once you have your Desired Capabilities filled out, click “Start Session”

At this point, you can see see the device within the Appium UI, but still need to handle getting WDIO Testrunner set up. The way this configuration will proceed is to use WDIO Testrunner and not Standalone Mode. This is because we want to have this testing environment scale with our needs, and using tests written in `describe` blocks, broken down into multiple suites, testing various functional zones of our app is the way to achieve this.

Webdriver

6) Within the same directory you installed Appium Doctor and were running that command, you will install WebdriverIO, and eventually get the Testrunner configured

Run npm install webdriverio --save-dev, once that has been installed, you can use the ./node_modules/.bin/wdio --help command, which tells you the Testrunner has been installed
You can then run the same command, but with a different flag, to configure your wdio.conf.js and package files, which is essentially where everything is specified for your app configuration: ./node_modules/.bin/wdio --config
Setting these configs are what will make or break your ease in using the Testrunner. For our purposes, I chose a local environment, Mocha, Spec reporter, the Appium service (though we will actually comment this out), and then filled in the repo-specific info.

7) Set up Babel (this is all configured to use Mocha)

Within your working directory, run npm install --save-dev babel-register babel-preset-es2015
Return to your wdio.conf.js file and add the following so that Mocha will register Babel:

mochaOpts: {
ui: ‘bdd’,
compilers: [‘js:babel-register’],
timeout: 30000
}

Run npm install --save-dev babel-plugin-transform-runtime babel-runtime
Create a .babelrc file at the directory level within this working directory

{
“presets”: [“es2015”],
“plugins”: [
[“transform-runtime”, {
“polyfill”: false
  }]
 ]
}

8) Remember choosing Spec Reporterwhile setting up your wdio.conf.js file? Time to install it:

Run npm install wdio-spec-reporter --save-dev and make sure it’s been added to your package.json:

{
“devDependencies”: {
“wdio-spec-reporter”: “~0.0.1”
  }
}

9) Choose an assertion library (I went with Chai)

Run npm install chai

10) At this point, your package.json should look pretty much like this:

{
“name”: “appium”,
“version”: “1.0.0”,
“description”: “niche-mobile-app testing”,
“main”: “index.js”,
“scripts”: { // I modified these scripts to fit my needs
  “android-tests”: “wdio wdio.conf.js”,
  “appium”: “appium”,
  “help”: “wdio --help”
  },
“repository”: {
“type”: “git”,
“url”: “appium”
  },
“keywords”: [
“appium”
  ],
“author”: “your name”,
“license”: “ISC”,“devDependencies”: {
“babel-plugin-syntax-async-functions”: “^6.8.0”,
“babel-plugin-transform-regenerator”: “^6.9.0”,
“babel-plugin-transform-runtime”: “^6.23.0”,
“babel-preset-es2015”: “^6.24.1”,
“babel-register”: “^6.26.0”,
“babel-runtime”: “^6.26.0”,
“chai”: “*”, //recommended via Chai's website
“mocha”: “*”,
“wdio-mocha-framework”: “^0.6.3”,
“wdio-spec-reporter”: “0.0.5”,
“webdriverio”: “^4.14.0”
  }
}

11) Time to modify your wdio.conf.js file to include everything you need.

Your wdio.conf.js file ought to look something like this:

const path = require(‘path’)
const host = ‘127.0.0.1’; // default appium host
const port = 4723; // default appium port
const APP_PATH = ‘/Path/to/your/appfile.apk’;
const waitforTimeout = 30 * 60000;
const commandTimeout = 30 * 60000;
var webdriverio = require(‘webdriverio’);
var expect = require(‘chai’).expect; //if you’re using Chai
var Launcher = require(‘webdriverio’).Launcher;exports.config = {
host: host,
port: port,
specs: [path.resolve(__dirname, ‘specs/path/to/your/testfile.js’)],
maxInstances: 1,
//
//If you have trouble getting all important capabilities together, //check out the
//capabilities: [
{
  appiumVersion: ‘1.9.1’, // Appium module version
  browserName: ‘Chrome’, // browser name is empty for native apps     
  platformName: ‘Android’,
  //app: ‘APP_PATH’, // Path to the native app
  //appPackage: ‘com.example.testapp’, // Package name of the app
  //appActivity: ‘MainActivity’, // App activity of the app
  platformVersion: ‘8.0’, // Android platform version
  deviceName: ‘emulator-5554’,
  waitforTimeout: waitforTimeout,
  commandTimeout: commandTimeout,
  newCommandTimeout: 30 * 60000,
  }
],
framework: ‘mocha’,
reporters: [‘spec’],
mochaOpts: {
ui: ‘bdd’,
compilers: [‘js:babel-register’],
timeout: 30000
},
// ===================
// Test Configurations
sync: true,
logLevel: ‘verbose’,
coloredLogs: true,
deprecationWarnings: true,
bail: 0,
screenshotPath: ‘./errorShots/’,
baseUrl: ‘http://localhost',
waitforTimeout: 10000,
connectionRetryTimeout: 90000,
connectionRetryCount: 3,
//
//=====
//Hooks
//=====
//WebdriverIO provides several hooks you can use to interfere with //the test process in order to enhance
//it and to build services around it. You can either apply a single //function or an array of
//methods to it. If one of them returns with a promise, WebdriverIO //will wait until that promise got
//resolved to continue.
onPrepare: function (capabilities) {
  console.log(‘<<< APP TESTS STARTED >>>’);
  require(‘babel-register’);
  },
onComplete: function () {
  console.log(‘<<< TESTING FINISHED >>>’);
  }
}

If you designated the Appium Service when you were using the ./node_modules/.bin/wdio --config command, you will also have the following:

//Test runner services
//Services take over a specific job you don’t want to take care of. //They enhance your test setup with almost no effort. Unlike //plugins, they don’t add new commands. Instead, they hook //themselves up into the test process.services: [‘appium’],
appium: {
args: {
  platformName: “Android”,
  platformVersion: “8.1.0”,
  deviceName: “Android Emulator”,
  app: “path/to/your/app.apk”,
  automationName: “UiAutomator2”
  }
}

You can either comment this out or delete it all together. You won’t need it, because we are using the Appium Desktop UI.

Appium running the following tests on Niche.com in Chrome browser

And that’s pretty much it. Assuming you’ve got your path set correctly to your test file, you should be able to run either ./node_modules/.bin/wdio wdio.conf.jsor whatever you have it set to in scripts within your package.json. If you don’t have tests or just want to run something to see if it’s all talking, try these simple browser tests:

var assert = require(‘assert’);describe(‘webdriver.io page’, () => {it(‘should have the right title’, () => {
  browser.url('https://www.niche.com/')
  const title = browser.getTitle()
  assert.equal(title, ‘Niche: Explore Schools, Companies, and Neighborhoods’)
})it(‘should go to scholarship’, () => {
  browser.url('https://www.niche.com/colleges/scholarships/')
  const title = browser.getTitle()
  assert.equal(title, ‘Find College Scholarships — Niche’)
})it(‘should go to college rankings’, () => {
  browser.url('https://www.niche.com/colleges/rankings/')
  const title = browser.getTitle()
  assert.equal(title, ‘2019 College Rankings — Niche’)
})it(‘should go to district search’, () => {
  browser.url('https://www.niche.com/k12/search/best-school-      districts/m/pittsburgh-metro-area/')
  const title = browser.getTitle()
  assert.equal(title, ‘2019 Best School Districts in the Pittsburgh    Area — Niche’)
})it(‘should go to district search’, () => {
  browser.url('https://www.niche.com/k12/d/mt-lebanon-school-district-pa/') 
  const title = browser.getTitle() 
  assert.equal(title, ‘Mt. Lebanon School District — Pennsylvania —   Niche’)
  })
})sorry for that odd spacing above in the code blocks

One thing to note: If you’re on a Mac, you will want to get Xcode (this takes a while), but you will need it for the command line tools, which can be installed by running xcode-select — tools after you’ve installed Xcode. Appium Doctor will also tell you to install Carthage, if you haven’t already. Android Studio is going to be a must if you’re working with .apk files.

That’s pretty much it. There are a million resources out there. My main references were:

In the not-too-distant-future I will be adding the ability to use an array of devices and emulators to simultaneously do E2E testing, and will be sure to add those steps.

— — — — — — — — — — — — — — — — — — — — — — — — — — — — — — —

Edit to add: a follow up can be found here, where I outline how to use Appium with physical iOS devices, rather than being stuck on simulators.