The Tamarin acceptance tests can be used to verify your installation and or local changes that you have made to the source. To run acceptance tests on the android shell see the "Testing the Android Shell" heading below. Running the Tamarin test suite requires the following steps:
- Set the
AVMenvironment variable to the path of the AVMPlus executable (
avmshell). Note: the name of the
avmshellexecutable varies among the build systems; use the name of the AVM shell produced from the build process you used, above.
- Set the
BUILTINABCenvironment variable to the path of the
- You may build the .abc files build by downloading the latest asc.jar. The latest asc.jar can be downloaded from from ftp://ftp.mozilla.org/pub/js/tamarin...latest/asc.jar. Also the source to asc.jar may be downloaded and rebuilt from http://opensource.adobe.com/wiki/dis...exsdk/Flex+SDK.
- Set the
ASCenvironment variable, as described above.
- Set the
SHELLABCenvironment variable to the path of the
$ cd tamarin-redux/test/acceptance $ export ASC=/Users/build/hg/tamarin-redux/utils/asc.jar $ export BUILTINABC=/Users/build/hg/tamarin-redux/core/builtin.abc $ export SHELLABC=/Users/build/hg/tamarin-redux/shell/shell_toplevel.abc $ export AVM=/Users/build/hg/builds/5260-6d1899261bac/mac/avmshell_64 $ python runtests.py Tamarin tests started: 2010-09-28 10:37:06.410676 current configuration: x64-mac-tvm-release avm version: 5260:6d1899261bac Executing 2532 tests against vm: /Users/build/hg/builds/5260-6d1899261bac/mac/avmshell_64 2532 running abcasm/abs_helper.as skipping... reason: 2531 running abcasm/adhoc.abs skipping... reason: inconsistencies in different debug output, need to implement regex matching of diffs 2527 running abcasm/branchToCommon.abs 2530 running abcasm/arithmetic.abs 2529 running abcasm/bkpt.abs 2528 running abcasm/bkptline.abs 2526 running abcasm/bug_476556.abs 2524 running abcasm/bug_491056.abs ... test run PASSED! Tests complete at 2010-09-28 10:39:26.751797 Start Date: 2010-09-28 10:38:07.221457 End Date : 2010-09-28 10:39:26.751797 Test Time : 0:01:19.530340 passes : 59291 failures : 0 expected failures : 75 tests skipped : 76
-f flag tells
runtests.py to force recompilation of all the scripts. For help on
runtests.py options, see below or use
python runtests.py -h.
Notes on using acceptance runtests.py
In order to speed up the execution time of the acceptance testsuite, the runtests script automatically detects the number of cores on the machine and sets an equivalent number of threads. This can be overrided with the threads flag:
python ./runtests.py --threads=3
Threading on cygwin / windows
Due to an issue with cygwin python, threading does not work, and threads will always be set to 1. In order to use threading on windows you must install windows python and invoke runtests using that python executable in cygwin. The locations of AVM, ASC, BUILTINABC and SHELLABC must also be defined using windows paths (replace backslashes [\] with forward slashes [/]):
c:/Python26/python.exe ./runtests.py --threads=3
Other flags of interest
(For a list of all flags, run runtests.py with -h/--help)
--nohtml : will suppress the creation of an html output file
--rebuildtests : will only rebuild test .abc's, will not attempt to run them. Can also be using in conjunction with --threads
-f --forcerebuild force rebuild all test files
-q --quiet : will print a period (.) for each test run regardless of results, then print the test run summary when finished
--ascargs= : additional arguments to pass to asc.jar
--vmargs= : additional arguments to pass to avmshell
--random : run tests in random order
--timeout : max time for testrun
--verify : run a verify pass instead of running abcs
--verifyonly : run a -Dverifyonly pass: only checks test exitcode (only works with debugger vms)
Testing the Android Shell
The instructions above apply to running tests on an android device as well, but with a few differences. You should set environment variables as described above except for AVM, which should be:
$ export AVM=$TAMARIN_BUILD_TOP/platform/android/android_shell.sh
where $TAMARIN_BUILD_TOP is your main Tamarin repo folder. Note that AVM needs to point to android_shell.sh which is a proxy for running the tests on the phone. The /android-public/android-vars.sh script has commented-out examples of the environment settings for running tests on the android avm shell. To recap the other environment settings:
- Set the BUILTINABC environment variable to the path of the /core/builtin.abc.
- You may build the .abc files build by downloading the latest asc.jar. The latest asc.jar can be downloaded from from ftp://ftp.mozilla.org/pub/js/tamarin...latest/asc.jar. Also the source to asc.jar may be downloaded and rebuilt from http://opensource.adobe.com/wiki/display/flexsdk/Flex+SDK.
- Set the ASC environment variable, as described above.
- Set the SHELLABC environment variable to the path of the /shell/shell_toplevel.abc.
adb (Android Debug Bridge)
'adb' stands for Android debug Bridge. If you've correctly edited and run the android-vars.sh script mentioned on the Tamarin Build Documentation page you should already be pathed to the adb executable in the public sdk/ndk, but if not it exists at /android-public/android-sdk-mac_86/platform-tools.
adb supports ssh-like commands to an android device. adb automatically connects to a phone if connected by usb. adb shell "ls /" is a good test to see if connected. The command:
$ adb devices
lists the device id of each phone. If you have more than 1 phone connected to usb, use "adb -s <id>" for each phone. If this fails, do you have a dev OS installed? Under Settings->Applications->Development, is USB Debugging enabled? </id>
<id>Testsuite Setup </id>
<id>Example commands: </id>
<id>Root the phone </id>
$ adb root</id>
<id>Copy avmshell to /data/app. The shell can be named variously in the build but on the phone the executable must be 'avmshell' </id>
$ adb push avmshell /data/app/avmshell</id><id>
$ adb shell chmod 777 /data/app/avmshell</id>
<id>Copy android_runner.sh, if it doesn't already exist on the phone in /data/app </id>
$ adb push tamarin-redux/platform/android/android_runner.sh /data/app/android_runner.sh</id><id>
$ adb shell chmod 777 /data/app/android_runner.sh</id>
<id>Test it out with a simple .abc or no args for usage (should return EXITCODE=0) </id>
<id>$ tamarin-redux/platform/android/android_shell.sh hello.abc </id><id>hello </id><id>EXITCODE=0 </id>
<id>Running Acceptance Tests </id>
<id>--androidthreads sets the number of threads to the number of phones found via usb. Running with 2 or more phones uses threads to run tests in parallel and reduces running times dramatically. You must, in any case, at least use --androidthreads or --threads=1 (or will fail due to multiple tests running). </id>
<id>$ export AVM=<full-path-to-tamarin-redux>/platform/android/android_shell.sh </full-path-to-tamarin-redux></id><id><full-path-to-tamarin-redux>$ cd <full-path-to-tamarin-redux>/test/acceptance </full-path-to-tamarin-redux></full-path-to-tamarin-redux></id><id><full-path-to-tamarin-redux><full-path-to-tamarin-redux>$ ./runtests.py --androidthreads </full-path-to-tamarin-redux></full-path-to-tamarin-redux></id>
<id><full-path-to-tamarin-redux><full-path-to-tamarin-redux>Note on using an emulator: the emulator does seem to support hardware floating point but gives a BusError when running avmshell -Darm_vfp. It's unreliable, losing its adb connection frequently. An emulator is ok for testing a few abc files but for any consistency a real device is better. </full-path-to-tamarin-redux></full-path-to-tamarin-redux></id>