mozilla

Revision 378299 of Building with Profile-Guided Optimization

  • Revision slug: Building_with_Profile-Guided_Optimization
  • Revision title: Building with Profile-Guided Optimization
  • Revision id: 378299
  • Created:
  • Creator: Ehsan
  • Is current revision? No
  • Comment

Revision Content

Introduction

The Mozilla build system contains support for building with Profile-Guided Optimization (PGO) with GCC 4 or newer and Microsoft Visual C++ 2005 (Professional Edition) or newer. A PGO build consists of two passes: a first pass to build instrumented binaries, then a second pass to re-build optimized binaries using profile information gleaned from running the instrumented binaries. The Mozilla build system will run both passes for you, you simply need to provide a script to run the application through some profiling scenarios in between the build passes. {{ Note("The script does not need to be automated! If you are not running your builds in an automated environment, you can simply have it launch the browser, then interact with the browser to generate the profiling data and close it when you\'re done.") }}

Getting Started

In order to build Firefox with Profile Guided Optimizations, you should first make sure that you can do a simple build of Firefox.  You need to set up the build prerequisites and environment.  Once you have done that, you need to change your mozconfig according to the Build Configuration section below, and then run make -f client.mk profiledbuild (see the Building section below.)  If you're not sure what profile generation script to use, then you probably want to use what we use for nightly builds (see the end of the Build Configuration section.)

Build Configuration

You will need to setup a mozconfig as normal with the build options you want. You will also need to specify one additional option in your mozconfig for the PGO build process, the script to run between build passes. You set this with mk_add_options PROFILE_GEN_SCRIPT like so:

 mk_add_options PROFILE_GEN_SCRIPT=/home/user/run-firefox.sh

This command will be run directly from make, so ensure that the file is executable. On Windows you will need to specify the interpreter name in front of the script, like so:

 mk_add_options PROFILE_GEN_SCRIPT="sh /home/user/run-firefox.sh"

The script does not have to be a shell script, it can be Perl or Python or anything else that you can run from the command line, just make sure to specify the right command for running it.

You can use the {{source("browser/config/mozconfigs/win32/nightly#3", "same script the nightly builds use")}} with the following:

mk_add_options PROFILE_GEN_SCRIPT='$(MAKE) -C $(MOZ_OBJDIR) pgo-profile-run'

The profile generation script

When your script is run it is passed in an environment variable, OBJDIR, containing the path to the object directory. A simple shell script for Linux that just launches the browser would look like this:

 #!/bin/sh
 export NO_EM_RESTART=1
 mkdir $OBJDIR/_profileprofile
 $OBJDIR/dist/bin/firefox -no-remote -profile $OBJDIR/_profileprofile

It simply creates an empty profile directory, then runs Firefox with that profile directory. You would have to manually run your profiling scenarios and close the browser.

Building

PGO builds require you to start your build from a different target in client.mk. Instead of the usual make -f client.mk build, you simply do make -f client.mk profiledbuild. {{ Note("PGO builds will take much longer than normal builds! With GCC, you will wind up doing two complete clobber builds. With MSVC, you will only do one full build, and then another pass of re-linking, but re-linking libxul can take over a half hour on a fast machine, and use over one gigabyte of RAM.") }}{{ WarningStart}}Linking a PGO build is known to exhaust the 32 bit address space on Win32.  To work around this either use a 64 bit compiler or use the /3GB boottime switch to give the linker more address space.  See Bug 543034 for details.{{WarningEnd}}

Revision Source

<h2 id="Introduction" name="Introduction">Introduction</h2>
<p>The Mozilla build system contains support for building with Profile-Guided Optimization (PGO) with GCC 4 or newer and Microsoft Visual C++ 2005 (Professional Edition) or newer. A PGO build consists of two passes: a first pass to build instrumented binaries, then a second pass to re-build optimized binaries using profile information gleaned from running the instrumented binaries. The Mozilla build system will run both passes for you, you simply need to provide a script to run the application through some profiling scenarios in between the build passes. {{ Note("The script does not need to be automated! If you are not running your builds in an automated environment, you can simply have it launch the browser, then interact with the browser to generate the profiling data and close it when you\'re done.") }}</p>
<h2>Getting Started</h2>
<p>In order to build Firefox with Profile Guided Optimizations, you should first make sure that you can do a <a href="/en-US/docs/Simple_Firefox_build" title="/en-US/docs/Simple_Firefox_build">simple build of Firefox</a>.&nbsp; You need to set up the build prerequisites and environment.&nbsp; Once you have done that, you need to change your <a href="/en-US/docs/Configuring_Build_Options" title="/en-US/docs/Configuring_Build_Options">mozconfig</a> according to the <a href="#Build_Configuration" title="#Build_Configuration">Build Configuration</a> section below, and then run <code>make -f client.mk profiledbuild</code> (see the <a href="#Building" title="#Building">Building</a> section below.)&nbsp; If you're not sure what profile generation script to use, then you probably want to use what we use for nightly builds (see the end of the <a href="#Build_Configuration" title="#Build_Configuration">Build Configuration</a> section.)</p>
<h2 id="Build_Configuration" name="Build_Configuration"><a name="Build_Configuration"></a>Build Configuration</h2>
<p>You will need to setup a mozconfig as normal with the build options you want. You will also need to specify one additional option in your mozconfig for the PGO build process, the script to run between build passes. You set this with <code>mk_add_options PROFILE_GEN_SCRIPT</code> like so:</p>
<pre class="eval">
 mk_add_options PROFILE_GEN_SCRIPT=/home/user/run-firefox.sh
</pre>
<p>This command will be run directly from make, so ensure that the file is executable. On Windows you will need to specify the interpreter name in front of the script, like so:</p>
<pre class="eval">
 mk_add_options PROFILE_GEN_SCRIPT="sh /home/user/run-firefox.sh"
</pre>
<p>The script does not have to be a shell script, it can be Perl or Python or anything else that you can run from the command line, just make sure to specify the right command for running it.</p>
<p>You can use the {{source("browser/config/mozconfigs/win32/nightly#3", "same script the nightly builds use")}} with the following:</p>
<pre class="eval">
<span id="the-code">mk_add_options PROFILE_GEN_SCRIPT='$(MAKE) -C $(MOZ_OBJDIR) pgo-profile-run'</span>
</pre>
<h2 id="The_profile_generation_script" name="The_profile_generation_script">The profile generation script</h2>
<p>When your script is run it is passed in an environment variable, <code>OBJDIR</code>, containing the path to the object directory. A simple shell script for Linux that just launches the browser would look like this:</p>
<pre class="eval">
 #!/bin/sh
 export NO_EM_RESTART=1
 mkdir $OBJDIR/_profileprofile
 $OBJDIR/dist/bin/firefox -no-remote -profile $OBJDIR/_profileprofile
</pre>
<p>It simply creates an empty profile directory, then runs Firefox with that profile directory. You would have to manually run your profiling scenarios and close the browser.</p>
<h2 id="Building" name="Building"><a name="Building"></a>Building</h2>
<p>PGO builds require you to start your build from a different target in client.mk. Instead of the usual <code>make -f client.mk build</code>, you simply do <code>make -f client.mk profiledbuild</code>. {{ Note("PGO builds will take much longer than normal builds! With GCC, you will wind up doing two complete clobber builds. With MSVC, you will only do one full build, and then another pass of re-linking, but re-linking libxul can take over a half hour on a fast machine, and use over one gigabyte of RAM.") }}{{ WarningStart}}Linking a PGO&nbsp;build is known to exhaust the 32 bit address space on Win32.&nbsp; To work around this either use a 64 bit compiler or use the /3GB&nbsp;boottime switch to give the linker more address space.&nbsp; See <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=543034" title="https://bugzilla.mozilla.org/show_bug.cgi?id=543034">Bug 543034</a> for details.{{WarningEnd}}</p>
Revert to this revision