Thread Sanitizer Redirect 1

What is Thread Sanitizer?

Thread Sanitizer (TSan) is a fast data race detector for C/C++ programs. It uses a compile-time instrumentation to check all non-race-free memory access at runtime. Unlike other tools, it understands compiler-builtin atomics and synchronization and therefore provides very accurate results with no real false positives. More information on how TSan works can be found on the Thread Sanitizer wiki.

Public Builds

Note: No public builds are available at this time yet.

Manual Build

Build prerequisites

Note: This section assumes you're using Linux to build. The build process has also been confirmed to work on Mac OS X, but there hasn't been much further testing on this platform. If you are testing Mac OS X and encounter problems, please let us know.

LLVM/Clang

The TSan instrumentation is implemented as an LLVM pass and integrated into Clang. Therefore, we must first get and build LLVM and Clang.

Note: At the time of writing, TSan has only been integrated into LLVM/Clang SVN trunk versions and might not be included yet in regular releases. Therefore, the manual describes how to check out LLVM/Clang and build from source.

The following commands do a fresh SVN checkout/build of LLVM, Clang and the compiler runtime libraries with a revision confirmed to work:

# Latest revision confirmed to build with Linux and MacOSX (10.7 and 10.8).
REV=192787
svn co -r $REV http://llvm.org/svn/llvm-project/llvm/trunk llvm
cd llvm
export LLVM_HOME=`pwd` # We will refer to this variable later during the build.
(cd tools && svn co -r $REV http://llvm.org/svn/llvm-project/cfe/trunk clang)
(cd projects && svn co -r $REV http://llvm.org/svn/llvm-project/compiler-rt/trunk compiler-rt)

Now we can start our build, which can take a while depending on your machine specs. In order to setup the build with the recommended cmake method, you need to have at least cmake 2.8.8 installed. Adjust the -j parameter to your needs depending on your CPUs:

mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE:STRING=Release $LLVM_HOME
make -j16

Building with cmake is highly recommended. In case you cannot build with cmake for some reason, it is still possible to build without it. Instead, use these steps then for building:

# Use these steps only if the cmake build steps cannot be used
mkdir build
(cd build && ../configure --enable-optimized && make -j 10)

Please note that when building without cmake, the path to your resulting binaries will be different. For example, with cmake, the clang binary will be at $LLVM_HOME/build/bin/clang while with the other build method, you will get it at $LLVM_HOME/build/Release+Asserts/bin/clang.

Building Firefox

Getting the source

If you don't have a source code repository clone yet, you need to get yourself a clone of mozilla-central.

Adjusting the build configuration

Create the build configuration file .mozconfig with the following content in your mozilla-central directory:

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/objdir-ff-tsan
mk_add_options MOZ_MAKE_FLAGS=-j12

# Enable LLVM specific code and build workarounds
ac_add_options --enable-llvm-hacks

# Ensure you set this to your LLVM_HOME path
export LLVM_HOME="/path/to/your/llvm"

# Set CC/CXX based on LLVM_HOME
export CC="$LLVM_HOME/build/bin/clang"
export CXX="$LLVM_HOME/build/bin/clang++"

# Add TSan to our compiler flags
export CFLAGS="-fsanitize=thread -fPIC -pie"
export CXXFLAGS="-fsanitize=thread -fPIC -pie"

# Additionally, we need the TSan flag during linking. Normally, our C/CXXFLAGS would
# be used during linking as well but there is at least one place in our build where
# our CFLAGS are not added during linking.
# Note: The use of this flag causes Clang to automatically link the TSan runtime :)
export LDFLAGS="-fsanitize=thread -fPIC -pie"

# These three are required by TSan
ac_add_options --disable-jemalloc
ac_add_options --disable-crashreporter
ac_add_options --disable-elf-hack

# Keep symbols to symbolize TSan traces
export MOZ_DEBUG_SYMBOLS=1
ac_add_options --enable-debug-symbols
ac_add_options --disable-install-strip

# Settings for an opt build
ac_add_options --enable-optimize="-O2 -gline-tables-only"
ac_add_options --disable-debug

Starting the build process

Now you start the build process using the regular make -f client.mk command.

Starting Firefox

After the build has completed, you can start Firefox from the objdir as usual.

Building only the JavaScript shell

If you want to build only the JavaScript shell instead of doing a full Firefox build, the build script below will probably help you to do so. Before using it, you must of course adjust the path name for LLVM_ROOT to match your setup. Once you have adjusted everything, execute this script in the js/src/ subdirectory and pass a directory name as the first parameter. The build will then be created in a new subdirectory with that name.

#! /bin/sh

if [ -z $1 ] ; then
    echo "usage: $0 <dirname>"
elif [ -d $1 ] ; then
    echo "directory $1 already exists"
else
    autoconf2.13
    mkdir $1
    cd $1
    LLVM_ROOT="/path/to/llvm"
    CC="$LLVM_ROOT/build/bin/clang" \
    CXX="$LLVM_ROOT/build/bin/clang++" \
    CFLAGS="-fsanitize=thread -fPIC -pie" \
    CXXFLAGS="-fsanitize=thread -fPIC -pie" \
    LDFLAGS=""-fsanitize=thread -fPIC -pie" \
            ../configure --disable-debug --enable-optimize="-O2 -gline-tables-only" --enable-llvm-hacks
    make -j 8
fi

Using LLVM Symbolizer for faster/better traces

By default, TSan traces are symbolized because otherwise, the runtime suppression list wouldn't work. The default way of symbolizing the traces is much slower than using llvm-symbolizer for that purpose:

LLVM ships with a symbolizer binary that TSan will readily use to immediately output symbolized traces much faster. Furthermore, additional debug information is included in traces when llvm-symbolizer is used. To use it, either make sure the llvm-symbolizer binary is in your PATH or set the environment variable TSAN_OPTIONS="external_symbolizer_path=/path/to/llvm-symbolizer" before running the process. If you've built TSan as described above, then you will want to set the variable to $LLVM_HOME/build/bin/llvm-symbolizer.

Ignoring known races

In some cases, races can be confirmed to be benign and fixing them would cause possible performance issues. TSan offers two possibilities to ignore such known races.

Compile-time blacklisting

Functions with racy memory access can be flagged, such that the compiler will not instrument them. In our codebase, this is done using the MOZ_TSAN_BLACKLIST macro on the function definition. It will expand to the appropriate attributes if the compiler supports them. Compile-time blacklisting is the preferred way of blacklisting, because it does not have a performance impact. Still, blacklisting should be done very carefully so we won't miss important bugs.

Runtime suppressions

To prevent races from showing up at runtime, TSan also provides a runtime suppression list. This list is a good tool to temporarily hide some traces, especially for debugging purposes. However, unlike compile-time blacklisting, runtime suppressions do have a performance impact, especially if the race in question keeps showing up a lot. For more information on how to use the runtime suppression list, see the TSan wiki.

Document Tags and Contributors

Contributors to this page: cdiehl
Last updated by: cdiehl,