Thread Sanitizer

  • Revision slug: Thread_Sanitizer
  • Revision title: Thread Sanitizer
  • Revision id: 325339
  • Created:
  • Creator: cdiehl
  • Is current revision? No
  • Comment Moved From User:cdiehl to Thread_Sanitizer

Revision Content

Introduction

Thread Sanitizer is a data race detector developed and maintained by Google. This document is in development but shall already very briefly explain the usage of the Thread Sanitizer with Mozilla Firefox while new content will be added in the near future.

Thread Sanitizer supports two major modes of operation, hybrid and pure happens-before (default).

hybrid - reports more false positives but is faster, more predictable and finds more true races.

pure happens-before - will not report false positives unless the code uses lock-less synchronisation methods but may miss races.

 

Setup

TSan can be used with PIN or Valgrind, we describe here the usage of Valgrind+TSan.

As of today Valgrind+TSan is supported on the following platforms:

Ubuntu 12.04 LTS 32-bit (Valgrind)
Ubuntu 12.04 LTS 64-bit (Valgrind)
MacOS 10.6 32-bit (Valgrind)
MacOS 10.7 32-bit (Valigrind)

However as of writing this I got some aborts with the 64-bit versions on Linux during the launch of Firefox, therefore this document describes the 32-bit version on Ubuntu 12.04.

Download

You can either build TSan from the source or run the self-contained shell script. To make things easy we choose the latter.

git clone http://git.chromium.org/chromium/deps/valgrind/binaries.git tsan

Add TSan to your PATH environment variable in your .bashrc to run it in a convenient way.

export PATH=$PATH:$HOME/tsan/third_party/valgrind/linux_x86/bin

Run

valgrind-tsan.sh --hybrid=no --announce-threads --ignore=tsan.ignore --gen-suppressions $OBJ_DIR/dist/bin/firefox -P valgrind -no-remote 2>&1 | tee tsan.log

 

Examining Reports with VIM

TSan reports are formatted with VIM folds thus provides us with the oppurtunity to examine them in VIM in an easy way. In order to do so add and adjust your VIM configurations to the settings below.

TSan+VIM

$HOME/.vim/scripts.vim

if did_filetype()
  finish
endif
let lnum = 1
while lnum < 100
  if (getline(lnum) =~ 'ThreadSanitizerValgrind')
    setfiletype tsan
  endif
  set foldlevel=1
  let lnum = lnum + 1
endwhile

$HOME/.vim/syntax/tsan.vim

sy match TS_Head              /Possible data race during.*:/
sy match TS_Concurrent        /Concurrent .* happened at or after these points:/
sy match TS_MemoryDescr       /Address .* is .* bytes inside data symbol.*/
sy match TS_MemoryDescr       /Location .* bytes inside a block starting at .* of size .* allocated.*/
sy match TS_Locks             /Locks involved in this report.*/
sy match TS_Fold              /\{\{\{/
sy match TS_Fold              /\}\}\}/
sy match TS_FirstFunc         /#0  0x[0-9A-F]\+: .*/
hi TS_Head          ctermfg=Red
hi TS_Concurrent    ctermfg=Magenta
hi TS_MemoryDescr   ctermfg=Cyan
hi TS_Locks         ctermfg=Green
hi TS_Fold          cterm=bold
hi TS_FirstFunc     cterm=bold
" vim: fdl=1

$HOME/.vimrc

set path=,,.,../include
function! Gfw()
  let b = bufnr('')
  normal mz
  let b = bufnr('')
  wincmd w
  exe "b " . b
  normal `zgF
endfun
nnoremap ;f :call Gfw()<cr>

 

Once you have completed this step you can open a TSan log file in VIM, split the screen in half with the report on the left and the source on the right.

:vsplit
;f
Ctrl+ww

Inside VIM, point your cursor to the beginning of the path of the source file and hit ;f doing so will open the source on the left side of the split VIM screen. Press Ctrl+ww to switch back to the left side.

 

Ignores

TSan offers the possibility of using 'ignores' to specify locations which shall not be instrumented. This can save time in the excution process. Wildcards are supported as well. The structure of such a ignore file with its supported keywords is shown below.

# shared libraries
obj:
# source files
src:
# functions
fun:
# functions will be ignored together with other functions they call
fun_r:
fun_hist:

A sample ignore file can be found here: http://people.mozilla.com/~cdiehl/tsan/tsan.ignore


Suppressions

TSan also supports the known 'suppressions' functionality of Valgrind. Multiple stack traces for one suppression rule are supported.

{
  suppression_name
  tool_name:warning_name
  {
    fun:mangled_function_name_wildcard
    obj:object_name_wildcard
    ...
    fun:demangled_function_name_wildcard
  }
  {
    ...
    fun:another_mangled_function_name_wildcard
  }
}

A sample suppression file can be found here: http://people.mozilla.com/~cdiehl/tsan/tsan.sup

 

Verifying Races

TSan has a built-in race verifier, once the race verifier confirmed a race it is a 100% proof of a race. If the race verifier did not detect a race it proves nothing.

valgrind-tsan.sh --race-verifier=tsan.log $OBJ_DIR/dist/bin/firefox -P valgrind -no-remote 2>&1 | tee tsan.verified

 

References

Revision Source

<h2 id="Introduction"><a name="introduction"><strong>Introduction</strong></a></h2>
<p>Thread Sanitizer is a data race detector developed and maintained by Google. This document is in development but shall already very briefly explain the usage of the Thread Sanitizer with Mozilla Firefox while new content will be added in the near future.</p>
<p>Thread Sanitizer supports two major modes of operation, hybrid and pure happens-before (default).</p>
<p><em>hybrid</em> - reports more false positives but is faster, more predictable and finds more true races.</p>
<p><em>pure happens-before</em> - will not report false positives unless the code uses lock-less synchronisation methods but may miss races.</p>
<p>&nbsp;</p>
<h2 id="Setup"><strong>Setup</strong></h2>
<p>TSan can be used with PIN or Valgrind, we describe here the usage of Valgrind+TSan.</p>
<p>As of today Valgrind+TSan is supported on the following platforms:</p>
<p>Ubuntu 12.04 LTS 32-bit (Valgrind)<br />
  Ubuntu 12.04 LTS 64-bit (Valgrind)<br />
  MacOS 10.6 32-bit (Valgrind)<br />
  MacOS 10.7 32-bit (Valigrind)</p>
<p>However as of writing this I got some aborts with the 64-bit versions on Linux during the launch of Firefox, therefore this document describes the 32-bit version on Ubuntu 12.04.</p>
<p><strong>Download</strong></p>
<p>You can either build TSan from the source or run the self-contained shell script. To make things easy we choose the latter.</p>
<pre>
git clone http://git.chromium.org/chromium/deps/valgrind/binaries.git tsan</pre>
<p>Add TSan to your PATH environment variable in your .bashrc to run it in a convenient way.</p>
<pre>
export PATH=$PATH:$HOME/tsan/third_party/valgrind/linux_x86/bin</pre>
<p><strong>Run</strong></p>
<pre>
valgrind-tsan.sh --hybrid=no --announce-threads --ignore=tsan.ignore --gen-suppressions $OBJ_DIR/dist/bin/firefox -P valgrind -no-remote 2&gt;&amp;1 | tee tsan.log</pre>
<p>&nbsp;</p>
<h2 id="Examining_Reports_with_VIM"><strong>Examining Reports with VIM</strong></h2>
<p>TSan reports are formatted with VIM folds thus provides us with the oppurtunity to examine them in VIM in an easy way. In order to do so add and adjust your VIM configurations to the settings below.</p>
<p><a href="http://people.mozilla.com/~cdiehl/tsan/tsan%2Bvim.png" target="_self"><img alt="TSan+VIM" src="https://people.mozilla.com/~cdiehl/tsan/tsan%2Bvim.png" /></a></p>
<p><strong>$HOME/.vim/scripts.vim</strong></p>
<pre class="brush: bash">
if did_filetype()
&nbsp; finish
endif
let lnum = 1
while lnum &lt; 100
&nbsp; if (getline(lnum) =~ 'ThreadSanitizerValgrind')
&nbsp;&nbsp;&nbsp; setfiletype tsan
&nbsp; endif
&nbsp; set foldlevel=1
&nbsp; let lnum = lnum + 1
endwhile</pre>
<p><strong>$HOME/.vim/syntax/tsan.vim</strong></p>
<pre class="brush: bash">
sy match TS_Head              /Possible data race during.*:/
sy match TS_Concurrent        /Concurrent .* happened at or after these points:/
sy match TS_MemoryDescr       /Address .* is .* bytes inside data symbol.*/
sy match TS_MemoryDescr       /Location .* bytes inside a block starting at .* of size .* allocated.*/
sy match TS_Locks             /Locks involved in this report.*/
sy match TS_Fold              /\{\{\{/
sy match TS_Fold              /\}\}\}/
sy match TS_FirstFunc         /#0  0x[0-9A-F]\+: .*/
hi TS_Head          ctermfg=Red
hi TS_Concurrent    ctermfg=Magenta
hi TS_MemoryDescr   ctermfg=Cyan
hi TS_Locks         ctermfg=Green
hi TS_Fold          cterm=bold
hi TS_FirstFunc     cterm=bold
" vim: fdl=1</pre>
<p><strong>$HOME/.vimrc</strong></p>
<pre class="brush: bash">
set path=,,.,../include
function! Gfw()
&nbsp; let b = bufnr('')
&nbsp; normal mz
&nbsp; let b = bufnr('')
&nbsp; wincmd w
&nbsp; exe "b " . b
&nbsp; normal `zgF
endfun
nnoremap ;f :call Gfw()&lt;cr&gt;</pre>
<p>&nbsp;</p>
<p>Once you have completed this step you can open a TSan log file in VIM, split the screen in half with the report on the left and the source on the right.</p>
<pre class="brush: bash">
:vsplit
;f
Ctrl+ww</pre>
<p>Inside VIM, point your cursor to the beginning of the path of the source file and hit ;f doing so will open the source on the left side of the split VIM screen. Press Ctrl+ww to switch back to the left side.</p>
<p>&nbsp;</p>
<h2 id="Ignores"><strong>Ignores</strong></h2>
<p>TSan offers the possibility of using 'ignores' to specify locations which shall not be instrumented. This can save time in the excution process. Wildcards are supported as well. The structure of such a ignore file with its supported keywords is shown below.</p>
<pre>
# shared libraries
obj:
# source files
src:
# functions
fun:
# functions will be ignored together with other functions they call
fun_r:
fun_hist:
</pre>
<p>A sample ignore file can be found here: <a href="http://people.mozilla.com/~cdiehl/tsan/tsan.ignore" title="http://people.mozilla.com/~cdiehl/tsan/tsan.ignore">http://people.mozilla.com/~cdiehl/tsan/tsan.ignore</a></p>
<h2 id="
__Suppressions"><br />
  <strong>Suppressions</strong></h2>
<p>TSan also supports the known 'suppressions' functionality of Valgrind. Multiple stack traces for one suppression rule are supported.</p>
<pre>
{
&nbsp; suppression_name
&nbsp; tool_name:warning_name
&nbsp; {
&nbsp;&nbsp;&nbsp; fun:mangled_function_name_wildcard
&nbsp;&nbsp;&nbsp; obj:object_name_wildcard
&nbsp;&nbsp;&nbsp; ...
&nbsp;&nbsp;&nbsp; fun:demangled_function_name_wildcard
&nbsp; }
&nbsp; {
&nbsp;&nbsp;&nbsp; ...
&nbsp;&nbsp;&nbsp; fun:another_mangled_function_name_wildcard
&nbsp; }
}</pre>
<p>A sample suppression file can be found here: <a href="http://people.mozilla.com/~cdiehl/tsan/tsan.sup" title="http://people.mozilla.com/~cdiehl/tsan/tsan.sup">http://people.mozilla.com/~cdiehl/tsan/tsan.sup</a></p>
<p>&nbsp;</p>
<h2 id="Verifying_Races"><strong>Verifying Races</strong></h2>
<p>TSan has a built-in race verifier, once the race verifier confirmed a race it is a 100% proof of a race. If the race verifier did not detect a race it proves nothing.</p>
<pre>
valgrind-tsan.sh --race-verifier=tsan.log $OBJ_DIR/dist/bin/firefox -P valgrind -no-remote 2&gt;&amp;1 | tee tsan.verified</pre>
<p>&nbsp;</p>
<h2 id="References"><strong>References</strong></h2>
<ul>
  <li><a href="http://code.google.com/p/thread-sanitizer/wiki/PopularDataRaces" title="http://code.google.com/p/thread-sanitizer/wiki/PopularDataRaces">http://code.google.com/p/thread-sanitizer/wiki/PopularDataRace</a></li>
</ul>
Revert to this revision