MDN’s new design is in Beta! A sneak peek: https://blog.mozilla.org/opendesign/mdns-new-design-beta/

自動化測試介紹

正在翻譯中。

每天在好幾個瀏覽器與設備上,運行手動測試數次,既乏味又浪費時間。要有效率的處理這種事,就要開始熟悉自動化工具。我們會在這篇文章看看有哪些可用的工具、如何使用它們、以及如何使用如 Sauce Labs 與 Browser Stack 的商業化瀏覽器測試程式之基本講述。

先決條件: 熟悉 HTMLCSSJavaScript 核心語言的基本;跨瀏覽器測試的重要原則
目標: 提供理解自動化測試的需求、它如何讓你生活變得簡單、還有如何透過一些商業產品令事情更簡易。

自動化讓事情變簡單

在這個模組中,我們詳細介紹了不同的方式,來測試你的網站和應用程序,並解釋了跨瀏覽器測試工作應該在哪些瀏覽器上進行測試、輔助功能要點……等等。聽來要做很多事呢,不是嗎?

我們同意手動測試前述的一切,真的很累。幸好有很多工具,可以讓我們不用這麼累。有兩個主要方法,可以自動執行我們討論過的測試:

  1. 使用如 GruntGulpGulpnpm scripts 之類的任務執行器(task runner)來跑測試,並在組建過程中清理你的程式碼。這個方法很適合如清理並最小化程式碼、增加 CSS 前輟或最大化跨瀏覽器的 transpiling nascent JavaScript 功能...之類的任務。
  2. 使用如 Selenium 之類的瀏覽器自動化系統,在安裝好的瀏覽器跑指定測試並傳回結果,並在瀏覽器出問題的時候警告你。諸如 Sauce LabsBrowser Stack 之類的商業跨瀏覽器測試程式都是基於 Selenium,但能讓你用簡單的介面,遠端訪問他們設好的東西,如此一來,就能省下自己架設測試系統的心力。

我們會在下一篇文章,專注於如何設定基於 Selenium 的個人測試系統。這篇文章則會專注於如何設定任務執行器,並簡單與系統化地,使用前述的商業系統。

注意:the above two categories are not mutually exclusive. It is possible to set up a task runner to access a service like Sauce Labs via an API, run cross browser tests, and return results. We will look at this below as well.

使用任務執行器以自動化測試工具

如同前述,你可以透過任務執行器運行在組建功能中,所有想要自動化的常見任務,例如在每次存檔,或其他時候,清理並壓縮程式碼。在這個章節,我們將專注於如何用對初學者友善的選項,透過 Node 與 Gulp 執行自動化任務。

設定 Node 與 npm

今日,此類工具大都基於 Node.js。所以,你需要從 nodejs.org 安裝它:

  1. 從上面的網站下載安裝程式。
  2. 如同安裝其他程式般地安裝它。注意 Node 還會安裝 Node Package Manager(npm),它能讓你輕易安裝 軟體包、分享你自己寫的軟體包、還有在你的專案運行腳本。
  3. 安裝完成後,請輸入以下指令以測試 node 是否已安裝到電腦裡面,它會回傳 Node 與 npm 的版本:
    node -v
    npm -v
  4. 如果已經安裝過 Node/npm,你應該更新它們到最新版本。要更新 Node 的最可行方法,是從上述網站下載並安裝更新的軟體包。要更新 npm,請在文字介面輸入以下指令:
    npm install npm@latest -g

注意:如果因為權限問題而失敗,Fixing npm permissions 應該對你有所幫助。

要在專案裡面使用 node/npm 軟體包,你需要把專案所在目錄設為 npm 專案。它很簡單。

例如說,先來做個 test 目錄,以便不操心自己搞壞什麼。

  1. 選個合適的地方建立目錄。可以在檔案管理員的 UI 完成,或是輸入以下指令:
    mkdir node-test
  2. 要把這目錄變成 npm 專案,就要到 test 把此目錄初始化。請輸入:
    cd node-test
    npm init
  3. 第二個指令(npm init)會問你幾個問題,以便取得專案所需的資訊。你可以把一切都以預設帶過。
  4. 問完所有問題後,它會問你是否對設定滿意。輸入 yes 並按下 Enter 鍵,npm 就會在目錄產生一個稱為 package.json 的檔案。

這個檔案基本上就是個專案的設定檔。你可以之後再來設定,但目前它大概長成這個樣子:

{
  "name": "node-test",
  "version": "1.0.0",
  "description": "Test for npm projects",
  "main": "index.js",
  "scripts": {
    "test": "test"
  },
  "author": "Chris Mills",
  "license": "MIT"
}

有了這個檔案,你已經可以開始了。

設定 Gulp 自動化

來看看怎麼用 Gulp 設定一些測試工具的自動化。

  1. 要開始的話,得先建立一個 test npm 專案。使用的程式會在下面的章節提到。
  2. 接著,你需要有些簡單的 HTML、CSS、JavaScript 來測試系統:你可以複製我們的 index.htmlmain.jsstyle.css 到專案裡面,一個稱為 src 的目錄。現在你可以隨意嘗試測試內容,不過請注意這些工具不會直接在 JS/CSS 裡面運作:你需要外部的檔案。
  3. 首先,你要下這個指令,以全域(意思是說,它能在所有專案使用)的形式安裝 gulp:
    npm install --global gulp-cli
  4. 接著在 npm 專案輸入以下指令,以便專案將 gulp 認定為安裝所須:
    npm install --save-dev gulp
  5. 在專案裡面建立一個叫 gulpfile.js 的檔案。這個檔案能運行所有我們需要做的任務。在檔案裡面加這個指令:
    var gulp = require('gulp');
    
    gulp.task('default', function() {
      console.log('Gulp running');
    });
    這檔案需要我們之前安裝過的 gulp 模組,接著會跑些只顯示訊息的基本任務:它至少讓我們知道 Gulp 可以動。每個 gulp task 的基本格式都一樣——gulp 會執行 task() 方法,並給出兩個參數——任務的名稱、還有指示如何完成任務的回傳函式。
  6. 現在你可以跑 gulp task 了——輸入這個指令吧:
    gulp
    

讓 Gulp 幹些實事

要讓 Gulp 真的能幹些事情,就得先想想我們想要它幹什麼。我們的專案想要做這些合理的基本功能:

  • 用 html-tidy、css-lint、js-hint 來 lint、報告、修理 HTML/CSS/JS 的常見錯誤(請參見 gulp-htmltidygulp-csslintgulp-jshint)。
  • 用 Autoprefixer 掃描我們的 CSS 並在需要時增加 vendor prefixes(請參見 gulp-autoprefixer
  • 用 babel 把新式的 JavaScript 語法轉換為能在老舊瀏覽器運作的語法(請參見 gulp-babel

請詳見上面我們使用的 gulp 軟體包連結,以獲取完整指引。

要用套件的話,要先透過 npm 安裝之,之後在 gulpfile.js 上面引用需要的套件,再接著到下面加入想測試的東西,最後把你的任務命名為 default

在往下一步開始進發以前,把 default task 改成:

gulp.task('default', [ ]);

在陣列裡面寫下所有想在命令列輸入 gulp 後,希望 Gulp 運作的命令。

html-tidy

  1. Install using the following line:
    npm install --save-dev gulp-htmltidy
    

    Note: --save-dev adds the package as a dependency to your project. If you look in your project's package.json file, you'll see an entry for it has been added to the devDependencies property.

  2. Add the following dependencies to gulpfile.js:
    var htmltidy = require('gulp-htmltidy');
  3. Add the following test to the bottom of gulpfile.js:
    gulp.task('html', function() {
      return gulp.src('src/index.html')
            .pipe(htmltidy())
            .pipe(gulp.dest('build'));
    });
  4. Add 'html' as an item inside the array in the default task.

Here we are grabbing our development index.html file — gulp.src() allows us to grab a source file to do something with.

We next use the pipe() function to pass that source to another command to do something else with. We can chain as many of these together as we want. We first run htmltidy() on the source, which goes thorugh and fixes errors in our file. The second pipe() function writes the output HTML file to the build directory.

In the input version of the file, you may have noticed that we put an empty <p> element; htmltidy has removed this by the time the output file has been created.

Autoprefixer and css-lint

  1. Install using the following lines:
    npm install --save-dev gulp-autoprefixer
    npm install --save-dev gulp-csslint
  2. Add the following dependencies to gulpfile.js:
    var autoprefixer = require('gulp-autoprefixer');
    var csslint = require('gulp-csslint');
  3. Add the following test to the bottom of gulpfile.js:
    gulp.task('css', function() {
        return gulp.src('src/style.css')
            .pipe(csslint())
            .pipe(csslint.formatter('compact'))
            .pipe(autoprefixer({
                browsers: ['last 5 versions'],
                cascade: false
            }))
            .pipe(gulp.dest('build'));
    });
  4. Add 'css' as an item inside the array in the default task.

Here we grab our style.css file, run csslint on it (which outputs a list of any errors in your CSS to the terminal), then runs it through autoprefixer to add any prefixes needed to make nascent CSS features run in older browsers. At the end of the pipe chain, we output our modified prefixed CSS to the build directory. Note that this only works if csslint doesn't find any errors — try removing a curly brace from your CSS file and re-running gulp to see what output you get!

js-hint and babel

  1. Install using the following lines:
    npm install --save-dev gulp-babel babel-preset-es2015
    npm install jshint gulp-jshint --save-dev
    
  2. Add the following dependencies to gulpfile.js:
    var babel = require('gulp-babel');
    var jshint = require('gulp-jshint');
    
  3. Add the following test to the bottom of gulpfile.js:
    gulp.task('js', function() {
        return gulp.src('src/main.js')
            .pipe(jshint())
            .pipe(jshint.reporter('default'))
            .pipe(babel({
                presets: ['es2015']
            }))
            .pipe(gulp.dest('build'));
    });
  4. Add 'js' as an item inside the array in the default task.

Here we grab our main.js file, run jshint on it and output the results to the terminal using jshint.reporter; we then pass the file to babel, which converts it to old style syntax and outputs the result into the build directory. Our original code included a fat arrow function, which babel has modified into an old style function.

Further ideas

Once all this is all set up, you can run the gulp command inside your project directory, and you should get an output like this:

You can then try out the files output by your automated tasks by looking at them inside the build directory, and loading build/index.html in your web browser.

If you get errors, check that you've added all the dependencies and the tests as shown above; also try commenting out the HTML/CSS/JavaScript code sections and then rerunning gulp to see if you can isolate what the problem is.

Gulp comes with a watch() function that you can use to watch your files and run tests whenever you save a file. For example, try adding the following to the bottom of your gulpfile.js:

gulp.task('watch', function(){
  gulp.watch('src/*.html', ['html']);
  gulp.watch('src/*.css', ['css']);
  gulp.watch('src/*.js', ['js']);
});

Now try entering the gulp watch command into your terminal. Gulp will now watch your directory, and run the appropriate tasks whenever you save a change to an HTML, CSS, or JavaScript file.

Note: The * character is a wildcard character — here we're saying "run these tasks when any files of these types are saved. You could also use wildcards in your main tasks, for example gulp.src('src/*.css') would grab all your CSS files and then run piped tasks on them.

Note: One problem wth our watch command above is that our CSSLint/Autoprefixer combination throws full-blown errors when a CSS error is encountered, which stops the watch working. You'll have to restart the watch once a CSS error is encountered, or find another way to do this.

There's a lot more you can do with Gulp. The Gulp plugin directory has literally thousands of plugins to search through.

Other task runners

There are a number of other task runners available. We certainly aren't trying to say that Gulp is the best solution out there, but it works for us and it is fairly accessible to beginners. You could also try using other solutions:

  • Grunt works in a very similar way to Gulp, except that it relies on tasks specified in a config file, rather than written using JavaScript. See Getting started with Grunt for more details.
  • You can also run tasks directly using npm scripts located inside your package.json file, without needing to install any kind of extra task runner system. This works on the premise that things like Gulp plugins are basically wrappers around command line tools. So if you can work out how to run the tools using the command line, you can then run them using npm scripts. It is a bit more tricky to work with, but can be rewarding for those who are strong with their command line skills. Why npm scripts? provides a good introduction with a good deal of further information.

Using Sauce Labs to speed up browser testing

Test your code on Firefox for free on more than 800 browser/OS combos with our partner Sauce Labs.

There are other commercial browser testing systems available, but in this article we'll focus on Sauce Labs. We're not saying that this is necessarily the best tool available, but it is a good one that is smple for beginners to get up and running with.

The basic premise with such applications is that the company that runs each one has a huge server farm that can run many different tests. When you use the service, you provide a URL of a page you want to test, along with information such as what browsers you want it tested in. The app then configures a new VM with the OS and browser you specified, and returns the test results in the form of screenshots, videos, logfiles, text, etc.

You can then step up a gear, using an API to access functionality programmatically, which means that such apps can be combined with task runners, your own local Selenium environments, etc., to create automated tests.

Getting started with Sauce Labs

Let's get started with a Sauce Labs Trial.

  1. Create a Sauce Labs trial account.
  2. Sign in. This should happen automatically after you verify your e-mail address.

The basics: Manual tests

The Sauce Labs dashboard has a lot of options available on it. For now, make sure you are on the Manual Tests tab.

  1. Click Start a new manual session.
  2. In the next screen, type in the URL of a page you want to test (use http://mdn.github.io/learning-area/javascript/building-blocks/events/show-video-box-fixed.html, for example), then choose a browser/OS combination you want to test by using the different buttons and lists. There is a lot of choice, as you'll see!
  3. When you click Start session, a loading screen will then appear, which spins up a virtual machine running the combination you chose.
  4. When loading has finished, you can then start to remotely test the web site running in the chosen browser.
  5. From here you can see the layout as it would look in the browser you are testing, move the mouse around and try clicking buttons, etc. The top menu allows you to:
    • Stop the session
    • Give someone else a URL so they can observe the test remotely.
    • Copy text/notes to a remote clipboard.
    • Take a screenshot.
    • Test in full screen mode.

Once you stop the session, you'll return to the Manual Tests tab, where you'll see an entry for each of the previous manual sessions you started. Clicking on one of these entries shows more data for the session. In here you can for example download any screenshots you took, watch a video of the session, and view data logs for the session.

Note: This is already very useful, and way more convenient than having to set all these emulators and virtual machines by yourself.

Advanced: The Sauce Labs API

Sauce Labs has a restful API that allows you to programmatically retreive details of your account and existing tests, and annotate tests with further details, such as their pass/fail state, which aren't recordable by manual testing alone. For example, you might want to run one of your own Selenium tests remotely using a Sauce Labs, to test a certain browser/OS combination, and then pass the test results back to Sauce Labs.

It has a number of clients available to allow you to make calls to the API using your favourite environment, be it PHP, Java, Node.js, etc.

Let's have a brief look at how we'd access the API using Node.js and node-saucelabs.

  1. First, set up a new npm project to test this out, as detailed in Setting up Node and npm. Use a different directory name than before, like sauce-test for example.
  2. Install the Node Sauce Labs wrapper using the following command:
    npm install saucelabs
  3. Create a new file inside your project root called call-sauce.js. give it the following contents:
    var SauceLabs = require('saucelabs');
    
    var myAccount = new SauceLabs({
      username: "your-sauce-username",
      password: "your-sauce-api-key"
    });
    
    myAccount.getAccountDetails(function (err, res) {
      console.log(res);
      myAccount.getServiceStatus(function (err, res) {
        // Status of the Sauce Labs services
        console.log(res);
        myAccount.getJobs(function (err, jobs) {
          // Get a list of all your jobs
          for (var k in jobs) {
            if ( jobs.hasOwnProperty( k )) {
              myAccount.showJob(jobs[k].id, function (err, res) {
                var str = res.id + ": Status: " + res.status;
                if (res.error) {
                  str += "\033[31m Error: " + res.error + " \033[0m";
                }
                console.log(str);
              });
            }
          }
        });
      });
    });
  4. You'll need to fill in your Sauce Labs username and API key in the indicated places. These can be retrieved from your User Settings page. Fill these in now.
  5. Make sure everything is saved, and run your file like so:
    node call-sauce

進階:自動化測試

我們會在下一章覆蓋實際運行的 Sauce Lab 自動化測試。

總結

這是一切都還蠻簡單的,但我想你能看到自動化工具,在測試方面提供了很大的幫助。

下篇文章我們來關注怎麼用 Selenium 設定你自己的區域自動化系統,並與 Sauce Labs 做結合。

文件標籤與貢獻者

 此頁面的貢獻者: iigmir
 最近更新: iigmir,