Test automation on devices.
WHAT MAKES IT HARDER
Device variation Slower execution Platform differences Emulator and device differences Application installation per run
WHAT TOOLS PROVIDE
Driving applications on both platforms, frequently with a shared interface.
WHAT TO AUTOMATE
Critical journeys Regression of stable functionality Checks across device configurations
WHAT NOT TO AUTOMATE
Anything requiring visual or tactile judgement Rapidly changing screens
WHAT TO RUN ON EMULATORS
Functional tests, for speed and breadth.
WHAT TO RUN ON DEVICES
Anything involving performance, sensors, or real network.
WHAT DEVICE FARMS PROVIDE
Breadth without owning devices.
WHAT THEY COST
Per-minute charges, and slower runs.
WHAT TO BE CAREFUL WITH
Tests depending on device state Permissions dialogs differing by version Application data persisting between runs
WHAT TO DO ABOUT THAT LAST POINT
Reset state between tests.
WHAT TO SELECT ELEMENTS BY
Accessibility identifiers, set deliberately.
WHY THAT HELPS TWICE
It makes tests stable and improves accessibility.
WHAT TO TEST MANUALLY REGARDLESS
The release build, on a physical device.
