Debug builds (--enable-debug) and non-optimized builds (--disable-optimize) are much slower. Any performance metrics gathered by such builds are largely unrelated to what would be found in a release browser.
Many Firefox builds have a diagnostic tool that causes crashes to happen sooner and produce much more actionable information, but also slow down regular usage substantially. In particular, "GC poisoning" is used in all debug builds, and in optimized Nightly builds (but not opt Developer Edition or Beta builds). The poisoning can be disabled by setting the environment variable
before starting the browser.
Especially on Linux, accelerated graphics can sometimes lead to severe performance problems even if things look ok visually. Normally you would want to leave acceleration enabled while profiling, but on Linux you may wish to disable accelerated graphics (Preferences -> Advanced -> General -> Use hardware acceleration when available).
Currently the Gecko Profiler has limitations in the UI for inverted call stack top function analysis which is very useful for finding heavy functions that call into a whole bunch of code. Currently such functions may be easy to miss looking at a profile, so feel free to also use your favorite native profiler. It also lacks features such as instruction level profiling which can be helpful in low level profiling, or finding the hot loop inside a large function, etc. Some example tools include Instruments on OSX (part of XCode), RotateRight Zoom on Linux (uses perf underneath), and Intel VTune on Windows or Linux.