Adding a new Telemetry probe

  • Revision slug: Performance/Adding_a_new_Telemetry_probe
  • Revision title: Adding a new Telemetry probe
  • Revision id: 333069
  • Created:
  • Creator: Vladan
  • Is current revision? No
  • Comment Fixed spelling etc.

Revision Content

If a user has opted into submitting performance data to Mozilla, the Telemetry system will collect various measures of Firefox performance, hardware, usage and customizations and submit it to Mozilla. The Telemetry data collected by a single client can be examined from the integrated about:telemetry browser page, while the aggregated reports across entire user populations are publicly available at https://metrics.mozilla.com.

The following sections explain how to add a new measurement to Telemetry.

Telemetry Histograms

Telemetry histograms are the preferred way to track numeric measurements such as timings. Telemetry also tracks more complex data types such as slow SQL statement strings, browser hang stacks and system configurations, but adding such measurements to Telemetry is slightly more involved and is not covered in this document.

The histogram below is taken from Firefox's about:telemetry page. It shows a histogram used for tracking plugin shutdown times and the data collected over a single Firefox session. The timing data is grouped into buckets where the height of the blue bars representing the number of items in each bucket. The tallest bar, for example, indicates that there were 63 plugin shutdowns lasting between 129ms and 204ms.

Sample Telemetry histogram "PLUGIN_SHUTDOWN_MS" taken from Firefox's about:telemetry page

Choosing a Histogram Type

The first step to adding a new histogram is to choose the histogram type that best represents the data being measured. The sample histogram used above is an "exponential" histogram.

The following types are available:

  • flag: This histogram type allows you to record a single value. This type is useful if you need to track whether a feature was ever used during a Firefox session. You only need to add a single line of code which sets the flag when the feature is used because the histogram is initialized with a default value of false (flag not set).
  • boolean: These histograms only record boolean values. Multiple boolean entries can be recorded in the same histogram during a single browsing session, e.g. if a histogram is measuring user choices in a dialog box with options "Yes" or "No", a new boolean value is added every time the dialog is displayed.
  • enumerated: This histogram type is intended for storing "enum" values. An enumerated histogram consists of a fixed number of "buckets", each of which is associated with a consecutive integer value (the bucket's "label"). Each bucket corresponds to an enum value and counts the number of times its particular enum value was recorded. You might use this type of histogram if, for example, you wanted to track the relative popularity of SSL handshake types. Whenever you started an SSL handshake, you would record one of a limited number of enum values which uniquely identifies the handshake type.
  • linear: Linear histograms are similar to enumerated histograms, except each bucket is associated with a range of values instead of a single enum value. The range of values covered by each bucket increases linearly from the previous bucket, e.g. one bucket might count the number of occurrences of values between 0 to 9, the next bucket would cover values 10-19, the next 20-29, etc. This bucket type is useful if there aren't orders of magnitude differences between the minimum and maximum values stored in the histogram, e.g. if the values you are storing are percentages 0-100%.
  • exponential: Exponential histograms are similar to linear histograms but the range of values covered by each bucket increases exponentially. As an example of its use, consider the timings of an I/O operation whose duration might normally fall in the range of 0ms-50ms but extreme cases might have durations in seconds or minutes. For such measurements, you would want finer-grained bucketing in the normal range but coarser-grained bucketing for the extremely large values. An exponential histogram fits this requirement since it has "narrow" buckets near the minimum value and significantly "wider" buckets near the maximum value.

Declaring a Histogram

Histograms should be declared in the toolkit/components/telemetry/Histograms.json file. These declarations are checked for correctness at compile time and used to generate C++ code. It is also possible to create histograms at runtime dynamically, but this is primarily done by add-ons when they create their own histograms in Telemetry.

The following is a sample histogram declaration from Histograms.json for a histogram named MEMORY_RESIDENT which tracks the amount of resident memory used by a process:

"MEMORY_RESIDENT": {
  "kind": "exponential",
  "low": "32 * 1024",
  "high": "1024 * 1024",
  "n_buckets": 50,
  "description": "Resident memory size (KB)"
},

Note that histogram declarations in Histograms.json are converted to C++ code so the right-hand sides of fields can be simple C++ expressions or constant names as in the "low" field above.

There are 7 possible fields in a histogram declaration:

  • kind: Required. One of the histogram types described in the previous section. Different histogram types require different fields to be present in the declaration.
  • low: Optional, the default value is 0. This field represents the minimum value expected in the histogram. Note that all histograms automatically get a bucket with label "0" for counting values below the "low" value.
  • high: Required for linear and exponential histograms. The maximum value to be stored in a linear or exponential histogram. Any recorded values greater than this maximum will be counted in the last bucket.
  • n_buckets: Required for linear and exponential histograms. The number of buckets in a linear or exponential histogram.
  • n_values: Required for enumerated histograms. Similar to n_buckets, it represent the number of elements in the enum.
  • description: Required. A description of the data tracked by the histogram.
  • cpp_guard: Optional. This field specifies an #ifdef condition around this histogram's declaration. This is typically used for platform-specific histograms, e.g. "cpp_guard": "ANDROID"

Adding a JavaScript Probe

A Telemetry probe is the name of the code that measures and stores values in a histogram. Probes in privileged JavaScript code can make use of the nsITelemetry interface to get references to existing Histogram objects. Once you have a reference to a Histogram object, you can record a new value by calling add on the histogram.

let histogram = Services.telemetry.getHistogramById("PLACES_AUTOCOMPLETE_1ST_RESULT_TIME_MS");
histogram.add(measuredDuration);

Add-on code can dynamically add new histograms to Telemetry:

const ADDON_NAME = "Telemetry test addon";
const ADDON_HISTOGRAM_NAME = "addon-histogram";

let Telemetry = Services.telemetry;
Telemetry.registerAddonHistogram(ADDON_NAME, ADDON_HISTOGRAM_NAME, 1, 5, 6, Telemetry.HISTOGRAM_LINEAR);

let histogram = Telemetry.getAddonHistogram(ADDON_NAME, ADDON_HISTOGRAM_NAME);
histogram.add(1);

Adding a C++ Probe

Probes in native code can also use the nsITelemetry interface, but the helper functions declared in Telemetry.h are more convenient:

/**
 * Adds sample to a histogram defined in Histograms.json
 *
 * @param id - histogram id
 * @param sample - value to record.
 */
void Accumulate(ID id, uint32_t sample);

/**
 * Adds time delta in milliseconds to a histogram defined in Histograms.json
 *
 * @param id - histogram id
 * @param start - start time
 * @param end - end time
 */
void AccumulateTimeDelta(ID id, TimeStamp start, TimeStamp end = TimeStamp::Now());

The histogram names declared in Histograms.json are automatically transformed into constants in the mozilla::Telemetry namespace:

mozilla::Telemetry::Accumulate(mozilla::Telemetry::STARTUP_CRASH_DETECTED, true);

The Telemetry.h header also declares the helper AutoTimer and AutoCounter classes. Objects of these types automatically record a histogram value when they go out of scope:

nsresult
nsPluginHost::StopPluginInstance(nsNPAPIPluginInstance* aInstance)
{
  Telemetry::AutoTimer<Telemetry::PLUGIN_SHUTDOWN_MS> timer;
  ...
  return NS_OK;
}

Miscellaneous

  • getHistogramById will throw an NS_ERROR_ILLEGAL_VALUE JavaScript exception if it is called with an invalid histogram ID
  • Flag histograms will ignore any changes after the flag is set, so once the flag is set, it cannot be unset
  • Histograms which track timings in milliseconds or microseconds should suffix their names with "_MS" and "_US" respectively. Flag-type histograms should have the suffix "_FLAG" in their name.
  • If a histogram does not specify a "low" value, it will always have a "0" bucket (for negative or zero values) and a "1" bucket (for values between 1 and the next bucket)
  • The histograms on the about:telemetry page only show the non-empty buckets in a histogram except for the bucket to the left of the first non-empty bucket and the bucket to the right of the last non-empty bucket
  • Changing histogram declarations after the histogram has been released is tricky. You may need to create a new histogram with the new parameters.
    • For enum histograms, it's prudent to set "n_buckets" to a slightly larger value than needed since new elements may be added to the enum in the future.

Revision Source

<p>If a user has opted into submitting performance data to Mozilla, the Telemetry system will collect various measures of Firefox performance, hardware, usage and customizations and submit it to Mozilla. The Telemetry data collected by a single client can be examined from the integrated <em>about:telemetry</em> browser page, while the aggregated reports across entire user populations are publicly available at <a href="https://metrics.mozilla.com" title="https://metrics.mozilla.com">https://metrics.mozilla.com</a>.</p>
<p>The following sections explain how to add a new measurement to Telemetry.</p>
<h2 id="Telemetry_Histograms">Telemetry Histograms</h2>
<p>Telemetry histograms are the preferred way to track numeric measurements such as timings. Telemetry also tracks more complex data types such as slow SQL statement strings, browser hang stacks and system configurations, but adding such measurements to Telemetry is slightly more involved and is not covered in this document.</p>
<p>The histogram below is taken from Firefox's <em>about:telemetry</em> page. It shows a histogram used for tracking plugin shutdown times and the data collected over a single Firefox session. The timing data is grouped into buckets where the height of the blue bars representing the number of items in each bucket. The tallest bar, for example, indicates that there were 63 plugin shutdowns lasting between 129ms and 204ms.</p>
<p><img alt="Sample Telemetry histogram &quot;PLUGIN_SHUTDOWN_MS&quot; taken from Firefox's about:telemetry page" src="/files/4437/sampleHistogram.png" style="width: 328px; height: 376px;" /></p>
<h2 id="Choosing_a_Histogram_Type">Choosing a Histogram Type</h2>
<p>The first step to adding a new histogram is to choose the histogram type that best represents the data being measured. The sample histogram used above is an "exponential" histogram.</p>
<p>The following types are available:</p>
<ul>
  <li><strong>flag:</strong> This histogram type allows you to record a single value. This type is useful if you need to track whether a feature was ever used during a Firefox session. You only need to add a single line of code which sets the flag when the feature is used because the histogram is initialized with a default value of false (flag not set).</li>
  <li><strong>boolean</strong>: These histograms only record boolean values. Multiple boolean entries can be recorded in the same histogram during a single browsing session, e.g. if a histogram is measuring user choices in a dialog box with options "Yes" or "No", a new boolean value is added every time the dialog is displayed.</li>
  <li><strong>enumerated</strong>: This histogram type is intended for storing "enum" values. An enumerated histogram consists of a fixed number of "buckets", each of which is associated with a consecutive integer value (the bucket's "label"). Each bucket corresponds to an enum value and counts the number of times its particular enum value was recorded. You might use this type of histogram if, for example, you wanted to track the relative popularity of SSL handshake types. Whenever you started an SSL handshake, you would record one of a limited number of enum values which uniquely identifies the handshake type.</li>
  <li><strong>linear:</strong> Linear histograms are similar to enumerated histograms, except each bucket is associated with a range of values instead of a single enum value. The range of values covered by each bucket increases linearly from the previous bucket, e.g. one bucket might count the number of occurrences of values between 0 to 9, the next bucket would cover values 10-19, the next 20-29, etc. This bucket type is useful if there aren't orders of magnitude differences between the minimum and maximum values stored in the histogram, e.g. if the values you are storing are percentages 0-100%.</li>
  <li><strong>exponential:</strong> Exponential histograms are similar to linear histograms but the range of values covered by each bucket increases exponentially. As an example of its use, consider the timings of an I/O operation whose duration might normally fall in the range of 0ms-50ms but extreme cases might have durations in seconds or minutes. For such measurements, you would want finer-grained bucketing in the normal range but coarser-grained bucketing for the extremely large values. An exponential histogram fits this requirement since it has "narrow" buckets near the minimum value and significantly "wider" buckets near the maximum value.</li>
</ul>
<h2 id="Declaring_a_Histogram">Declaring a Histogram</h2>
<p>Histograms should be declared in the <a href="http://dxr.mozilla.org/mozilla-central/toolkit/components/telemetry/Histograms.json.html" title="http://dxr.mozilla.org/mozilla-central/toolkit/components/telemetry/Histograms.json.html">toolkit/components/telemetry/Histograms.json</a> file. These declarations are checked for correctness at <a href="http://dxr.mozilla.org/mozilla-central/toolkit/components/telemetry/gen-histogram-data.py.html" title="http://dxr.mozilla.org/mozilla-central/toolkit/components/telemetry/gen-histogram-data.py.html">compile time</a> and used to generate C++ code. It is also possible to create histograms at runtime dynamically, but this is primarily done by add-ons when they create their own histograms in Telemetry.</p>
<p>The following is a sample histogram declaration from <em>Histograms.json</em> for a histogram named <code>MEMORY_RESIDENT</code> which tracks the amount of resident memory used by a process:</p>
<pre class="brush: js">
"MEMORY_RESIDENT": {
&nbsp; "kind": "exponential",
&nbsp; "low": "32 * 1024",
&nbsp; "high": "1024 * 1024",
&nbsp; "n_buckets": 50,
&nbsp; "description": "Resident memory size (KB)"
},</pre>
<p>Note that histogram declarations in <em>Histograms.json</em> are converted to C++ code so the right-hand sides of fields can be simple C++ expressions or constant names as in the "low" field above.</p>
<p>There are 7 possible fields in a histogram declaration:</p>
<ul>
  <li><strong>kind</strong>: Required. One of the histogram types described in the previous section. Different histogram types require different fields to be present in the declaration.</li>
  <li><strong>low</strong>: Optional, the default value is 0. This field represents the minimum value expected in the histogram. Note that all histograms automatically get a bucket with label "0" for counting values below the "low" value.</li>
  <li><strong>high</strong>: Required for linear and exponential histograms. The maximum value to be stored in a linear or exponential histogram. Any recorded values greater than this maximum will be counted in the last bucket.</li>
  <li><strong>n_buckets</strong>: Required for linear and exponential histograms. The number of buckets in a linear or exponential histogram.</li>
  <li><strong>n_values</strong>: Required for enumerated histograms. Similar to n_buckets, it represent the number of elements in the enum.</li>
  <li><strong>description</strong>: Required. A description of the data tracked by the histogram.</li>
  <li><strong>cpp_guard</strong>: Optional. This field specifies an #ifdef condition around this histogram's declaration. This is typically used for platform-specific histograms, e.g. "cpp_guard": "ANDROID"</li>
</ul>
<h2 id="Adding_a_JavaScript_Probe">Adding a JavaScript Probe</h2>
<p>A Telemetry probe is the name of the code that measures and stores values in a histogram. Probes in privileged JavaScript code can make use of the <a href="http://dxr.mozilla.org/mozilla-central/toolkit/components/telemetry/nsITelemetry.idl" title="http://dxr.mozilla.org/mozilla-central/toolkit/components/telemetry/nsITelemetry.idl">nsITelemetry</a> interface to get references to existing Histogram objects. Once you have a reference to a Histogram object, you can record a new value by calling <code>add</code> on the histogram.</p>
<pre class="brush: js">
let histogram = Services.telemetry.getHistogramById("PLACES_AUTOCOMPLETE_1ST_RESULT_TIME_MS");
histogram.add(measuredDuration);</pre>
<p>Add-on code can dynamically add new histograms to Telemetry:</p>
<pre class="brush: js">
const ADDON_NAME = "Telemetry test addon";
const ADDON_HISTOGRAM_NAME = "addon-histogram";

let Telemetry = Services.telemetry;
Telemetry.registerAddonHistogram(ADDON_NAME, ADDON_HISTOGRAM_NAME, 1, 5, 6, Telemetry.HISTOGRAM_LINEAR);

let histogram = Telemetry.getAddonHistogram(ADDON_NAME, ADDON_HISTOGRAM_NAME);
histogram.add(1);</pre>
<h2 id="Adding_a_C.2B.2B_Probe">Adding a C++ Probe</h2>
<p>Probes in native code can also use the <a href="http://dxr.mozilla.org/mozilla-central/toolkit/components/telemetry/nsITelemetry.idl" title="http://dxr.mozilla.org/mozilla-central/toolkit/components/telemetry/nsITelemetry.idl">nsITelemetry</a> interface, but the helper functions declared in <a href="http://dxr.mozilla.org/mozilla-central/toolkit/components/telemetry/Telemetry.h#l43" title="http://dxr.mozilla.org/mozilla-central/toolkit/components/telemetry/Telemetry.h">Telemetry.h</a> are more convenient:</p>
<pre class="brush: cpp">
/**
 * Adds sample to a histogram defined in Histograms.json
 *
 * @param id - histogram id
 * @param sample - value to record.
 */
void Accumulate(ID id, uint32_t sample);

/**
 * Adds time delta in milliseconds to a histogram defined in Histograms.json
 *
 * @param id - histogram id
 * @param start - start time
 * @param end - end time
 */
void AccumulateTimeDelta(ID id, TimeStamp start, TimeStamp end = TimeStamp::Now());</pre>
<p>The histogram names declared in <em>Histograms.json</em> are automatically transformed into constants in the <code>mozilla::Telemetry</code> namespace:</p>
<pre class="brush: cpp">
mozilla::Telemetry::Accumulate(mozilla::Telemetry::STARTUP_CRASH_DETECTED, true);</pre>
<p>The <em>Telemetry.h</em> header also declares the helper <code>AutoTimer</code> and <code>AutoCounter</code> classes. Objects of these types automatically record a histogram value when they go out of scope:</p>
<pre class="brush: cpp">
nsresult
nsPluginHost::StopPluginInstance(nsNPAPIPluginInstance* aInstance)
{
  Telemetry::AutoTimer&lt;Telemetry::PLUGIN_SHUTDOWN_MS&gt; timer;
  ...
  return NS_OK;
}
</pre>
<h2 id="Miscellaneous">Miscellaneous</h2>
<ul>
  <li><code>getHistogramById</code> will throw an NS_ERROR_ILLEGAL_VALUE JavaScript exception if it is called with an invalid histogram ID</li>
  <li>Flag histograms will ignore any changes after the flag is set, so once the flag is set, it cannot be unset</li>
  <li>Histograms which track timings in milliseconds or microseconds should suffix their names with "_MS" and "_US" respectively. Flag-type histograms should have the suffix "_FLAG" in their name.</li>
  <li>If a histogram does not specify a "low" value, it will always have a "0" bucket (for negative or zero values) and a "1" bucket (for values between 1 and the next bucket)</li>
  <li>The histograms on the <em>about:telemetry</em> page only show the non-empty buckets in a histogram except for the bucket to the left of the first non-empty bucket and the bucket to the right of the last non-empty bucket</li>
  <li>Changing histogram declarations after the histogram has been released is tricky. You may need to create a new histogram with the new parameters.
    <ul>
      <li>For enum histograms, it's prudent to set "n_buckets" to a slightly larger value than needed since new elements may be added to the enum in the future.</li>
    </ul>
  </li>
</ul>
Revert to this revision