Gaia UI Tests Introduction

  • Revision slug: Mozilla/Firefox_OS/Platform/Testing/gaia-ui-tests
  • Revision title: Gaia UI Tests Guidelines
  • Revision id: 390173
  • Created:
  • Creator: ypwalter
  • Is current revision? No
  • Comment

Revision Content

This page has not yet finished. Your feed back to gaia-ui-automation@mozilla.org would be greatly appreciated.

Introduction

    Gaiatest is a Python package based on Marionette, which is designed specifically for writing tests against Gaia. It's like a mobile phone version of Selenium Webdriver in python version. Basically, it's using python to communicate with device/emulator, and it would execute javascript inside device/emulator to act the device/emulator.

    The main repository is currently in github: Mozilla/gaia-ui-tests. Due to Taiwan's local test needs, we also have another repository synchronized with main repository: Mozilla-TWQA/gaia-ui-tests.

Go back to "Table of Contents"

 

Installation

Basic Information

    Gaia UI tests will utilize marionette client to communicate with marionette server on device/emulator. We will elaborate about the different ways to install different components for gaia-ui-tests later. There are two different ways to install gaia-ui-tests depending on your needs.

Route 1 - for test executor
    If you only want to run the tests without developing anything, just the command below in command line. It should

easy_install gaiatest

Route 2 - for developers (suggested option)
    If you anticipate in modifying gaiatest, you can do this instead:

git clone git://github.com/mozilla/gaia-ui-tests.git

    If you do not want to install gaiatest and dependencies into system's site-packages, follow the Virtual Environment Setup section's steps below. Or, you can just setup gaiatest by the following command:

cd gaia-ui-tests
python setup.py develop

    Please do notice that there are different branches in gaia-ui-tests. Choose the branch best describe your current B2G status. For example, if you wish to do the test on desktop version B2G, you might want to switch branch to desktop branch.

    If you have no further plan on modifying marionette client or marionette server code, you can skip the rest of this section and virtual environment setup section to "Running Tests" section.

    However, if you wish to take control over marionette client or marionette server. You will need to install most-up-to date marionette client  or matching marionette client version since gaia-ui-tests is using marionette client to communicate  between marionette server inside device/emulator.

    You can either download marionette from marionette client official distribution website or marionette client would come with your B2G code from github after excuted "config.sh". After that, you can find marionette clientside code at {B2G HOME DIRECTORY}/gecko/testing/marionette/client) You can find marionette serverside code at {B2G HOME DIRECTORY}/gecko/testing/marionette/. Please notice that you must config you B2G as an engineer build, or you will fail to find these codes.

Virtual Environment Setup

    The virtualenv can let you install gaiatest and all package dependencies into the virtual environment without affecting your system's site-packages. You could find the tutorial on virtualenv official website, or you can reference the tutorial down below:

    First of all, get the virtualenv installed in Ubuntu:

sudo apt-get install python-virtualenv

Creating and Activating
    Then create your virtual environment for Python and activate it:

virtualenv <ENV_NAME>
source <ENV_NAME>/bin/activate
    After activating the virtual environment, the prompt will looks like this:
(<ENV_NAME>)user@host:~$  

    You can creating more than one virtual environment for your purpose.

Setup the gaiatest
    Usually, you just only need to follow the steps to install dependencies for gaiatest into virtual environment.

cd gaia-ui-tests
python setup.py develop
Deactivating
    When you want to leave current virtual environment, input the command to deactivate it:
deactivate

Go back to "Table of Contents"

 

Running Tests

    Basically, we can run gaia-ui-tests on a device with marionette server, emulators, and desktop B2G. However, please notice again that you should choose the right branch of gaia-ui-tests describing your B2G the best as we talked on previous section.

Testing on Devices

    First of all, connect your device through USB to computer, and then forward the package from the phone to computer port:

adb forward tcp:2828 tcp:2828

    You might get weird error, such as "no permission", from the command above. In that case, you could try to do the commands below:

adb kill-server
sudo adb devices
adb forward tcp:2828 tcp:2828

    Now, it's all set before you start to run your tests!

Testing on Emulator

Automatically launched emulator

 

Mannually launched emulator

 

Testing on Desktop B2G

    After you downloaded the Gaia UI test from GitHub, you also can use desktop build to run the tests. The Firefox OS desktop client, also called the B2G desktop client,  lets you run Gaia and Web apps in a Gecko-based environment somewhat  similar to an actual device. It doesn't emulate device hardware, so it's  not adequate for testing device APIs, and isn't a replacement for  testing on actual hardware. However, it does have a several APIs enabled  that aren't available on Firefox such as the Contacts and Settings  APIs. It can therefore be useful during the development of your  application, or while working on the Gaia user interface itself.

    You can download the latest build of the desktop client from this location, but make sure you download the appropriate file for your operating system.

  • Mac: b2g-[VERSION].multi.mac64.dmg
  • Linux (32bit): b2g-[VERSION].multi.linux-i686.tar.bz2
  • Linux (64bit): b2g-[VERSION].multi.linux-x86_64.tar.bz2

Note 1: Unfortunately, due to Bug 832469 the nightly desktop builds do not currently work on Windows, so you will need either Mac or Linux to continue :

Note 2: If you do not have the operating systems installed on your machine, a virtual machine is fine as well.

    Once downloaded, you will need to extract the contents to a local folder. For the purposes of the rest of this guide, I’ll refer to this location as $B2G_HOME.

Add the following line to your gaia/profile/user.js file,

user_pref('marionette.force-local', true)

which on :

  • Mac is located in $B2G_HOME/B2G.app/Contents/MacOS
  • Linux is located in $B2G_HOME/b2g

    Because we’re running against the desktop client we must filter out all tests that are unsuitable. To run the tests, use the following command:

gaiatest --address=localhost:2828 --type=b2g-antenna-bluetooth-carrier-camera-sdcard-wifi-xfail gaiatest/tests/manifest.ini

 

Go back to "Table of Contents"

 

Test Types

 

Go back to "Table of Contents"

 

Test Variables

 

Go back to "Table of Contents"

 

Test Data Prerequisites

 

Go back to "Table of Contents"

 

How to Contribute

Get Github Account

 

Fork It and Clone It

 

Branch Out

 

Solve Issues

 

Send Pull Requests

 

Go back to "Table of Contents"

 

Writing Tests

Tests Guidelines

 

Test Examples

 

Functions Reference

 

Formatting

 

Debugging

 

CSS and XPath Locators

 

Go back to "Table of Contents"

Revision Source

<div class="warning">
  <p><em><strong>This page has not yet finished. Your feed back to gaia-ui-automation@mozilla.org would be greatly appreciated.</strong></em></p>
</div>
<h2 id="Introduction">Introduction</h2>
<p>&nbsp;&nbsp;&nbsp; Gaiatest is a Python package based on <a href="https://developer.mozilla.org/en-US/docs/Marionette" title="https://developer.mozilla.org/en-US/docs/Marionette">Marionette</a>, which is designed specifically for writing tests against <a href="https://github.com/mozilla-b2g/gaia" title="https://github.com/mozilla-b2g/gaia">Gaia</a>. It's like a mobile phone version of <a href="http://docs.seleniumhq.org/projects/webdriver/" title="http://docs.seleniumhq.org/projects/webdriver/">Selenium Webdriver</a> in python version. Basically, it's using python to communicate with device/emulator, and it would execute javascript inside device/emulator to act the device/emulator.</p>
<p>&nbsp;&nbsp;&nbsp; The main repository is currently in github:<a href="https://github.com/mozilla/gaia-ui-tests" title="https://github.com/mozilla/gaia-ui-tests"> Mozilla/gaia-ui-tests</a>. Due to Taiwan's local test needs, we also have another repository synchronized with main repository: <a href="https://github.com/Mozilla-TWQA/gaia-ui-tests" title="/Mozilla-TWQA/gaia-ui-tests">Mozilla-TWQA/gaia-ui-tests</a>.</p>
<p style="text-align: center;"><strong>Go back to <a href="#Introduction" title="#Introduction">"Table of Contents"</a></strong></p>
<p style="text-align: center;">&nbsp;</p>
<h2 id="Installation">Installation</h2>
<h3 id="Basic_information">Basic Information</h3>
<p>&nbsp;&nbsp;&nbsp; Gaia UI tests will utilize marionette client to communicate with marionette server on device/emulator. We will elaborate about the different ways to install different components for gaia-ui-tests later. There are two different ways to install gaia-ui-tests depending on your needs.</p>
<p><strong>Route 1 - for test executor</strong><br />
  &nbsp;&nbsp;&nbsp; If you only want to run the tests without developing anything, just the command below in command line. It should</p>
<pre>
<code>easy_install gaiatest
</code></pre>
<p><strong>Route 2 - for developers (suggested option)</strong><br />
  &nbsp;&nbsp;&nbsp; If you anticipate in modifying gaiatest, you can do this instead:</p>
<pre>
<code>git clone git://github.com/mozilla/gaia-ui-tests.git
</code></pre>
<p>&nbsp;&nbsp;&nbsp; If you do not want to install gaiatest and dependencies into system's site-packages, follow the <a href="#Virtual_Environment_Setup" title="#Virtual_Environment_Setup"><strong>Virtual Environment Setup</strong></a> section's steps below. Or, you can just setup gaiatest by the following command:</p>
<pre style="font-size: 14px;">
<code style="font-size: 14px;">cd gaia-ui-tests
python setup.py develop</code></pre>
<p>&nbsp;&nbsp;&nbsp; Please do notice that there are different branches in gaia-ui-tests. Choose the branch best describe your current B2G status. For example, if you wish to do the test on desktop version B2G, you might want to switch branch to desktop branch.</p>
<div class="note">
  <p><strong>&nbsp;&nbsp;&nbsp; If you have no further plan on modifying marionette client or marionette server code, you can skip the rest of this section and virtual environment setup section to <a href="#Running_Tests" title="#Running_Tests">"Running Tests" section</a>.</strong></p>
</div>
<p>&nbsp;&nbsp;&nbsp; However, if you wish to take control over marionette client or marionette server. You will need to install most-up-to date <a href="https://pypi.python.org/pypi/marionette_client/" title="https://pypi.python.org/pypi/marionette_client/">marionette client</a>&nbsp; or matching marionette client version since gaia-ui-tests is using marionette client to communicate&nbsp; between marionette server inside device/emulator.<br />
  <br />
  &nbsp;&nbsp;&nbsp; You can either download marionette from <a href="https://pypi.python.org/pypi/marionette_client/" title="https://pypi.python.org/pypi/marionette_client/">marionette client official distribution website</a> or marionette client would come with your<a href="https://github.com/mozilla-b2g/B2G" title="https://github.com/mozilla-b2g/B2G"> B2G code from github</a> after excuted "config.sh". After that, you can find marionette clientside code at {B2G HOME DIRECTORY}/gecko/testing/marionette/client) You can find marionette serverside code at {B2G HOME DIRECTORY}/gecko/testing/marionette/. Please notice that you must config you B2G as an engineer build, or you will fail to find these codes.</p>
<h3>Virtual Environment Setup</h3>
<p>&nbsp;&nbsp;&nbsp; The virtualenv can let you install gaiatest and all package dependencies into the virtual environment without affecting your system's site-packages.&nbsp;You could find the tutorial on <a href="https://pypi.python.org/pypi/virtualenv" title="https://pypi.python.org/pypi/virtualenv">virtualenv official website</a>, or you can reference the tutorial down below:</p>
<p>&nbsp;&nbsp;&nbsp; First of all, get the virtualenv installed in Ubuntu:</p>
<pre style="font-size: 14px;">
sudo apt-get install python-virtualenv</pre>
<p><strong>Creating and Activating</strong><br />
  &nbsp;&nbsp;&nbsp; Then create your virtual environment for Python&nbsp;and activate it:</p>
<pre style="font-size: 14px;">
virtualenv &lt;ENV_NAME&gt;
source &lt;ENV_NAME&gt;/bin/activate</pre>
<div>
  &nbsp; &nbsp; After activating the virtual environment, the prompt will looks like this:</div>
<div>
  <pre style="font-size: 14px;">
(&lt;ENV_NAME&gt;)user@host:~$ &nbsp;</pre>
  <p>&nbsp; &nbsp; You can creating more than one virtual environment for your&nbsp;purpose.<br />
    <br />
    <strong>Setup the gaiatest</strong><br />
    <span style="line-height: 1.572;">&nbsp;&nbsp;&nbsp; Usually, you just only need to follow the steps to install&nbsp;</span><span style="line-height: 1.572;">dependencies for gaiatest into virtual environment.</span></p>
  <pre style="font-size: 14px;">
cd gaia-ui-tests
python setup.py develop</pre>
  <strong>Deactivating</strong><br />
  <div>
    &nbsp; &nbsp; When you want to leave current virtual environment, input&nbsp;the command to deactivate it:</div>
  <div>
    <pre style="font-size: 14px;">
deactivate</pre>
  </div>
</div>
<p style="text-align: center;"><strong>Go back to <a href="#Introduction" title="#Introduction">"Table of Contents"</a></strong></p>
<p style="text-align: center;">&nbsp;</p>
<h2 id="Running_Tests">Running Tests</h2>
<p>&nbsp;&nbsp;&nbsp; Basically, we can run gaia-ui-tests on a device with marionette server, emulators, and desktop B2G. However, please notice again that you should <u><strong>choose the right branch</strong></u> of gaia-ui-tests describing your B2G the best as we talked on previous section.</p>
<h3 id="Testing_on_a_device">Testing on Devices</h3>
<p>&nbsp;&nbsp;&nbsp; First of all, connect your device through USB to computer, and then forward the package from the phone to computer port:</p>
<pre>
adb forward tcp:2828 tcp:2828
</pre>
<p>&nbsp;&nbsp;&nbsp; You might get weird error, such as "no permission", from the command above. In that case, you could try to do the commands below:</p>
<pre>
adb kill-server
sudo adb devices
adb forward tcp:2828 tcp:2828
</pre>
<p>&nbsp;&nbsp;&nbsp; Now, it's all set before you start to run your tests!</p>
<h3 id="Testing_on_emulator">Testing on Emulator</h3>
<p><strong>Automatically launched emulator</strong></p>
<p>&nbsp;</p>
<p><strong>Mannually launched emulator</strong></p>
<p>&nbsp;</p>
<h3 id="Testing_on_Desktop_B2G">Testing on Desktop B2G</h3>
<p>&nbsp;&nbsp;&nbsp; After you downloaded the Gaia UI test from GitHub, you also can use <span class="author-g-bbktpz122z7l6tqfmcrf">desktop build to run the tests.</span><span class="author-g-bbktpz122z7l6tqfmcrf"> The Firefox OS desktop client, also called the </span><span class="author-g-bbktpz122z7l6tqfmcrf i"><i>B2G desktop client</i></span><span class="author-g-bbktpz122z7l6tqfmcrf">,&nbsp; lets you run Gaia and Web apps in a Gecko-based environment somewhat&nbsp; similar to an actual device. It doesn't emulate device hardware, so it's&nbsp; not adequate for testing device APIs, and isn't a replacement for&nbsp; testing on actual hardware. However, it does have a several APIs enabled&nbsp; that aren't available on Firefox such as the Contacts and Settings&nbsp; APIs. It can therefore be useful during the development of your&nbsp; application, or while working on the Gaia user interface itself.</span></p>
<p>&nbsp;&nbsp;&nbsp; You can download the latest build of the desktop client from <a href="http://ftp.mozilla.org/pub/mozilla.org/b2g/nightly/latest-mozilla-b2g18/">this location</a>, but make sure you download the appropriate file for your operating system.</p>
<ul>
  <li><strong>Mac</strong>: b2g-[VERSION].multi.mac64.dmg</li>
  <li><strong>Linux (32bit)</strong>: b2g-[VERSION].multi.linux-i686.tar.bz2</li>
  <li><strong>Linux (64bit)</strong>: b2g-[VERSION].multi.linux-x86_64.tar.bz2</li>
</ul>
<div class="note">
  <p>Note 1: Unfortunately, due to <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=832469">Bug 832469</a> the nightly desktop builds do not currently work on Windows, so you will need either Mac or Linux to continue :</p>
  <p>Note 2: If you do not have the operating systems installed on your machine, a virtual machine is fine as well.</p>
</div>
<p>&nbsp;&nbsp;&nbsp; Once downloaded, you will need to extract the contents to a local folder. For the purposes of the rest of this guide, I’ll refer to this location as<code> <strong>$B2G_HOME</strong></code>.</p>
<p>Add the following line to your gaia/profile/user.js file,</p>
<pre>
<code>user_pref('marionette.force-local', true)</code></pre>
<p>which on :</p>
<ul>
  <li><strong>Mac</strong> is located in $B2G_HOME/B2G.app/Contents/MacOS</li>
  <li><strong>Linux</strong> is located in $B2G_HOME/b2g</li>
</ul>
<p>&nbsp;&nbsp;&nbsp; Because we’re running against the desktop client we must filter out all tests that are unsuitable. To run the tests, use the following command:</p>
<pre>
<code>gaiatest --address=localhost:2828 --type=b2g-antenna-bluetooth-carrier-camera-sdcard-wifi-xfail gaiatest/tests/manifest.ini</code></pre>
<p>&nbsp;</p>
<p style="text-align: center;"><strong>Go back to <a href="#Introduction" title="#Introduction">"Table of Contents"</a></strong></p>
<p style="text-align: center;">&nbsp;</p>
<h2 id="Test_Types">Test Types</h2>
<p>&nbsp;</p>
<p style="text-align: center;"><strong>Go back to <a href="#Introduction" title="#Introduction">"Table of Contents"</a></strong></p>
<p style="text-align: center;">&nbsp;</p>
<h2 id="Test_Variables">Test Variables</h2>
<p>&nbsp;</p>
<p style="text-align: center;"><strong>Go back to <a href="#Introduction" title="#Introduction">"Table of Contents"</a></strong></p>
<p style="text-align: center;">&nbsp;</p>
<h2 id="Test_Data_Prerequisites">Test Data Prerequisites</h2>
<p>&nbsp;</p>
<p style="text-align: center;"><strong>Go back to <a href="#Introduction" title="#Introduction">"Table of Contents"</a></strong></p>
<p style="text-align: center;">&nbsp;</p>
<h2 id="How_to_Contribute">How to Contribute</h2>
<h3 id="Get_your_github_account">Get Github Account</h3>
<p>&nbsp;</p>
<h3 id="Fork_it_and_clone_it">Fork It and Clone It</h3>
<p>&nbsp;</p>
<h3 id="Branch_out">Branch Out</h3>
<p>&nbsp;</p>
<h3>Solve Issues</h3>
<p>&nbsp;</p>
<h3>Send Pull Requests</h3>
<p>&nbsp;</p>
<p style="text-align: center;"><strong>Go back to <a href="#Introduction" title="#Introduction">"Table of Contents"</a></strong></p>
<p>&nbsp;</p>
<h2 id="Writing_Tests">Writing Tests</h2>
<h3 id="Test_guidelines_and_examples">Tests Guidelines</h3>
<p>&nbsp;</p>
<h3>Test Examples</h3>
<p>&nbsp;</p>
<h3 id="Functions_reference">Functions Reference</h3>
<p>&nbsp;</p>
<h3 id="Formatting">Formatting</h3>
<p>&nbsp;</p>
<h3 id="Debugging">Debugging</h3>
<p>&nbsp;</p>
<h3 id="CSS_Locators_and_XPath_Locators_Resources">CSS and XPath Locators</h3>
<p>&nbsp;</p>
<p style="text-align: center;"><strong>Go back to <a href="#Introduction" title="#Introduction">"Table of Contents"</a></strong></p>
Revert to this revision