What is Thread Sanitizer?
Thread Sanitizer (TSan) is a fast data race detector for C/C++ programs. It uses a compile-time instrumentation to check all non-race-free memory access at runtime. Unlike other tools, it understands compiler-builtin atomics and synchronization and therefore provides very accurate results with no real false positives. More information on how TSan works can be found on the Thread Sanitizer wiki.
Note: No public builds are available at this time yet.
The TSan instrumentation is implemented as an LLVM pass and integrated into Clang. Therefore, we must first get and build LLVM and Clang.
The following commands do a fresh SVN checkout/build of LLVM, Clang and the compiler runtime libraries with a revision confirmed to work:
# Latest revision confirmed to build with Linux and MacOSX (10.7 and 10.8). REV=192787 svn co -r $REV http://llvm.org/svn/llvm-project/llvm/trunk llvm cd llvm export LLVM_HOME=`pwd` # We will refer to this variable later during the build. (cd tools && svn co -r $REV http://llvm.org/svn/llvm-project/cfe/trunk clang) (cd projects && svn co -r $REV http://llvm.org/svn/llvm-project/compiler-rt/trunk compiler-rt)
Now we can start our build, which can take a while depending on your machine specs. In order to setup the build with the recommended
cmake method, you need to have at least cmake 2.8.8 installed. Adjust the
-j parameter to your needs depending on your CPUs:
mkdir build cd build cmake -DCMAKE_BUILD_TYPE:STRING=Release $LLVM_HOME make -j16
Building with cmake is highly recommended. In case you cannot build with cmake for some reason, it is still possible to build without it. Instead, use these steps then for building:
# Use these steps only if the cmake build steps cannot be used mkdir build (cd build && ../configure --enable-optimized && make -j 10)
Please note that when building without
cmake, the path to your resulting binaries will be different. For example, with cmake, the
clang binary will be at $LLVM_HOME/build/bin/clang while with the other build method, you will get it at $LLVM_HOME/build/Release+Asserts/bin/clang.
Getting the source
If you don't have a source code repository clone yet, you need to get yourself a clone of mozilla-central.
Adjusting the build configuration
Create the build configuration file
.mozconfig with the following content in your mozilla-central directory:
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/objdir-ff-tsan mk_add_options MOZ_MAKE_FLAGS=-j12 # Enable LLVM specific code and build workarounds ac_add_options --enable-llvm-hacks # Ensure you set this to your LLVM_HOME path export LLVM_HOME="/path/to/your/llvm" # Set CC/CXX based on LLVM_HOME export CC="$LLVM_HOME/build/bin/clang" export CXX="$LLVM_HOME/build/bin/clang++" # Add TSan to our compiler flags export CFLAGS="-fsanitize=thread -fPIC -pie" export CXXFLAGS="-fsanitize=thread -fPIC -pie" # Additionally, we need the TSan flag during linking. Normally, our C/CXXFLAGS would # be used during linking as well but there is at least one place in our build where # our CFLAGS are not added during linking. # Note: The use of this flag causes Clang to automatically link the TSan runtime :) export LDFLAGS="-fsanitize=thread -fPIC -pie" # These three are required by TSan ac_add_options --disable-jemalloc ac_add_options --disable-crashreporter ac_add_options --disable-elf-hack # Keep symbols to symbolize TSan traces export MOZ_DEBUG_SYMBOLS=1 ac_add_options --enable-debug-symbols ac_add_options --disable-install-strip # Settings for an opt build ac_add_options --enable-optimize="-O2 -gline-tables-only" ac_add_options --disable-debug
Starting the build process
Now you start the build process using the regular
make -f client.mk command.
After the build has completed, you can start Firefox from the
objdir as usual.
LLVM_ROOT to match your setup. Once you have adjusted everything, execute this script in the
js/src/ subdirectory and pass a directory name as the first parameter. The build will then be created in a new subdirectory with that name.
#! /bin/sh if [ -z $1 ] ; then echo "usage: $0 <dirname>" elif [ -d $1 ] ; then echo "directory $1 already exists" else autoconf2.13 mkdir $1 cd $1 LLVM_ROOT="/path/to/llvm" CC="$LLVM_ROOT/build/bin/clang" \ CXX="$LLVM_ROOT/build/bin/clang++" \ CFLAGS="-fsanitize=thread -fPIC -pie" \ CXXFLAGS="-fsanitize=thread -fPIC -pie" \ LDFLAGS=""-fsanitize=thread -fPIC -pie" \ ../configure --disable-debug --enable-optimize="-O2 -gline-tables-only" --enable-llvm-hacks make -j 8 fi
Using LLVM Symbolizer for faster/better traces
By default, TSan traces are symbolized because otherwise, the runtime suppression list wouldn't work. The default way of symbolizing the traces is much slower than using
llvm-symbolizer for that purpose:
LLVM ships with a symbolizer binary that TSan will readily use to immediately output symbolized traces much faster. Furthermore, additional debug information is included in traces when
llvm-symbolizer is used. To use it, either make sure the
llvm-symbolizer binary is in your PATH or set the environment variable
TSAN_OPTIONS="external_symbolizer_path=/path/to/llvm-symbolizer" before running the process. If you've built TSan as described above, then you will want to set the variable to
Ignoring known races
In some cases, races can be confirmed to be benign and fixing them would cause possible performance issues. TSan offers two possibilities to ignore such known races.
Functions with racy memory access can be flagged, such that the compiler will not instrument them. In our codebase, this is done using the MOZ_TSAN_BLACKLIST macro on the function definition. It will expand to the appropriate attributes if the compiler supports them. Compile-time blacklisting is the preferred way of blacklisting, because it does not have a performance impact. Still, blacklisting should be done very carefully so we won't miss important bugs.
To prevent races from showing up at runtime, TSan also provides a runtime suppression list. This list is a good tool to temporarily hide some traces, especially for debugging purposes. However, unlike compile-time blacklisting, runtime suppressions do have a performance impact, especially if the race in question keeps showing up a lot. For more information on how to use the runtime suppression list, see the TSan wiki.