No Proxy For configuration

  • Revision slug: No_Proxy_For_configuration
  • Revision title: No Proxy For configuration
  • Revision id: 179971
  • Created:
  • Creator: Kohei
  • Is current revision? No
  • Comment /* Category */

Revision Content

Abstract

This document provides a comprehensive discussion of the manual proxy feature "No Proxy for:", including configuration issues, testing and bugs.

Audience

Mozilla users, mozilla developers, mozilla testers.

Overview

As browsers rapidly grew in popularity in the mid-90's, many network administrators added proxy servers. Initially, proxy servers were used for access control and performance. Many networks had limited access to the public network via proxy servers. Because the public network was small in scope and connections were slow, a caching proxy could often improve the overall performance.

This feature was originally designed as a "blacklist" of sites or domains that was within the intranet, and should not be accessed via the proxy server. Due to various limitations, this feature should be used for only the most simple blacklist scenarios. In almost all cases, PAC gives better control and flexibility.

User interface

"No proxy for" is an optional field, part of "Manual proxy configuration". The field is relatively short, but can hold reasonably long (2K+ characters) entries. New profiles contain the values "localhost, 127.0.0.1", by default.

Entry points:

  • Preferences | Advanced |  Proxies
  • control-click menu for off line-online icon (network plug)

Configuration

The no proxy list is composed of either domain elements or IPv4 address elements. The elements are separated by either a space (" ") or a comma (",").

Note for former-IE users: "*" is supported only at the beginning of domain filters (*.mozilla.org).

To block a... put this into "No Proxy for" ...for example use... Limitations
domain, including sub-domains domain suffix, starting with a dot ".mozilla.org, mozilla.org" You must use both entries to fully block a single domain.
hostname (without domain) hostname-only (see problems below) "localhost" Also blocks any possible domains that start with the entry ("www.otherdomain.localhost")
hostname (with domain) domain name "www.mozilla.org" Also blocks any hostnames or domains that end in the same string (other-www.mozilla.org)
IP address IP address "1.2.3.4" Does not block hostnames that resolve to the IP address ("127.0.0.1" does not block "localhost")
Network network w/ CIDR block "10.0.0.0/8" Does not block hostnames that resolve to the IP address range (10.0.0.0/8 is not "no proxy for intranet hostnames")
optional - port-specific (optional) ":" + port number "<FILTER>:81" Only black-lists port. Only applies to one port (no support for ranges and/or multiple ports). Port-only filters ":80" or "*:80" are not supported.
Formats that are not accepted Example
Domain filters with interior wildcards www.*.com
IP address string prefixes 127. or 10.0.*
IP addresses with wildcards in quads 10.*.*.*

Preferences

Name network.proxy.no_proxy_on
Default value localhost, 127.0.0.1

By default "localhost" and "127.0.0.1" are excluded, since most people assume these should connect to the local system.

Note: When IPv6 support is added, additional addresses will need to be added and tested.

Communicator used "network.proxy.none"

Limitations

  • No IPv6 support - The backend stores IPv4 addresses as IPv6, but IPv6 formats are not supported.
  • Scalability - Not usable for local domains with numerous hostnames. Not usable for large number of filters (ad blocking).
  • Mozilla implements this feature with significant limitations, users may find that writing a PAC file is more suitable for their needs.

Technical discussion

The most important aspect of "no proxy for" is that it uses a right-sided string compare. This algorithm works in most cases, but is also the root cause of most broken functionality. In recent versions, more logic was added for IP addresses, but the legacy problems remain for domain names.

The main problem is rooted in the DNS syntax, which allows a hostname to exist at the node level of a domain name. In other words, for the domain "mozilla.org", the valid entries would syntactically fall into three categories:

  1. Subdomains (and the contents of those subdomains) of mozilla.org ("server.office.mozilla.org")
  2. hostnames in mozilla.org ("www.mozilla.org")
  3. node-level entries of mozilla.org ("mozilla.org").

You cannot specify a prefix that will do a suffixed-string compare that will match all three cases precisely.

  • ".mozilla.org" will not match "mozilla.org"
  • "mozilla.org" will match all three cases, but also match any domains that end in the string "mozilla.org", such as "www.amozilla.org".

Testing

Contributors can test this feature, even without a proxy server, using a "negative proxy server test". Proxy connections that fail return an error "The proxy server you have configured  cannot be found", so configure your browser to use a non-existent HTTP proxy (hostname: "imaginary", port "80"). Test each destination in an http: URL. All proxied URLs will return errors, all non-proxied connections will be attempted normally (direct connection).

Hostnames filter test destination result
basic filtering unit tests (local host) localhost localhost direct
localhost. localhost proxy
127.0.0.1 local host direct
confirm the filter uses only suffix matches (hostname unit tests) hostname hostname direct
name hostname direct
host hostname proxy
domains with numbers 3com.com .3com.com direct
FQDNs
hostname.domain.com hostname.domain.com domain.com proxy
hostname.domain.com hostname.domain.com direct
hostname.domain.com host.hostname.domain.com direct
.domain.com .domain.com domain.com
hostname.domain.com
host.hostname.domain.com
direct
direct
direct
*.domain.com *.domain.com same results as ".domain.com"
*domain.com same results as "domain.com"
IP address
host IP address 127.0.0.1 127.0.0.1 direct
127.0.0.0 127.0.0.1 proxy
network range 127.0.0.0/8 127.0.0.1 direct
127/8 127.0.0.1 proxy
127.*.*.* 127.0.0.1 proxy
127. 127.0.0.1 proxy
Ports 1 127.0.0.1 proxy
no port .mozilla.org www.mozilla.org:80
www.mozilla.org:81
direct
direct
port number .mozilla.org:80 www.mozilla.org:80
www.mozilla.org
direct
direct
:81 www.mozilla.org:81 proxy

Developer notes

The no_proxy for logic is written in C++. PAC is written in JS, so there are potential problems with feature consistency and porting. David Baron has pointed out that the original PAC code in the "classic" tree is written in C++. The PAC in C++ has not been tested in mozilla, so porting PAC in C++ forward would not be a panacea.

The relevant code lives in {{ Source("netwerk/base/src/nsProtocolProxyService.cpp", "nsProtocolProxyService.cpp") }}.

  • {{ LXRSearch("ident", "i", "nsProtocolProxyService", "nsProtocolProxyService") }}::{{ LXRSearch("ident", "i", "PrefsChanged", "PrefsChanged") }} - loads preferences
  • {{ LXRSearch("ident", "i", "nsProtocolProxyService", "nsProtocolProxyService") }}::{{ LXRSearch("ident", "i", "LoadFilters", "LoadFilters") }} - parses filters
  • {{ LXRSearch("ident", "i", "nsProtocolProxyService", "nsProtocolProxyService") }}::{{ LXRSearch("ident", "i", "ExamineForProxy", "ExamineForProxy") }} - decides to check filters if configuration is "manual"
  • {{ LXRSearch("ident", "i", "nsProtocolProxyService", "nsProtocolProxyService") }}::{{ LXRSearch("ident", "i", "CanUseProxy", "CanUseProxy") }} - performs URL vs. filter comparison

Notable bugs

  • {{ Bug("172083") }} - {{ mediawiki.external('meta') }} Proxy: "no proxy for" items
  • {{ Bug("80917") }} - Proxy: "No Proxy" w/ form based UI
  • {{ Bug("91587") }} - Proxy: "no proxy for" default domain filtering fails w/ non-FQDN (e.g., http://web/)
  • {{ Bug("277378") }} - No proxy for: support IPv6 address literals
  • {{ Bug("136789") }} - Proxy: no proxy IP entries do not block DNS resolved IPs
  • {{ Bug("314712") }} - No proxy for: "hostname.domain.com" should block only "hostname.domain.com"
  • {{ Bug("72444") }} - Proxy: "bypass proxy server for local addresses" (IE pref)
  • {{ Bug("260883") }} - "No proxy for" does not use FQDN wildcards "*" like IE

Bugzilla sources

{{ Bug("17158") }} comment 21:

the correct separator are spaces or commas. So use this-

dogwood.state.mo.us .intra.state.mo.us dor.intranet
or
dogwood.state.mo.us, .intra.state.mo.us, dor.intranet

Note that you don't need to (read shouldn't) put a * for all hosts with that
domain ending. Corollary- a no_proxies_on entry of netscape.com will be
applicable to all hosts ending at netscape.com including foonetscape.com but an
entry of .netscape.com will specifically be applicable to all hosts in the
domain of netscape.com.

Original Document Information

  • Author(s): Benjamin Chuang
  • Last Updated Date: November 2, 2005
  • Copyright Information: Portions of this content are © 1998–2007 by individual mozilla.org contributors; content available under a Creative Commons license | Details.

Revision Source

<h3 id="Abstract" name="Abstract"> Abstract </h3>
<p>This document provides a comprehensive discussion of the manual proxy feature "No Proxy for:", including configuration issues, testing and bugs.
</p>
<h3 id="Audience" name="Audience"> Audience </h3>
<p>Mozilla users, mozilla developers, mozilla testers.
</p>
<h3 id="Overview" name="Overview"> Overview </h3>
<p>As browsers rapidly grew in popularity in the mid-90's, many network administrators added proxy servers. Initially, proxy servers were used for access control and performance. Many networks had limited access to the public network via proxy servers. Because the public network was small in scope and connections were slow, a caching proxy could often improve the overall performance.
</p><p>This feature was originally designed as a "blacklist" of sites or domains that was within the intranet, and should not be accessed via the proxy server. Due to various limitations, this feature should be used for only the most simple blacklist scenarios. In almost all cases, PAC gives better control and flexibility.
</p>
<h3 id="User_interface" name="User_interface"> User interface </h3>
<p>"No proxy for" is an optional field, part of "Manual proxy configuration". The field is relatively short, but can hold reasonably long (2K+ characters) entries. New profiles contain the values "localhost, 127.0.0.1", by default.
</p><p>Entry points:
</p>
<ul><li> Preferences | Advanced |  Proxies
</li><li> control-click menu for off line-online icon (network plug)
</li></ul>
<h3 id="Configuration" name="Configuration"> Configuration </h3>
<p>The no proxy list is composed of either domain elements or IPv4 address elements. The elements are separated by either a space (" ") or a comma (",").
</p>
<div class="note">
<p>Note for former-IE users: "*" is supported only at the beginning of domain filters (*.mozilla.org).
</p>
</div>
<table class="fullwidth-table"> <tbody><tr> <th>To block a...</th> <th>put this into "No Proxy for"</th> <th>...for example use...</th> <th>Limitations</th> </tr> <tr> <td>domain, including sub-domains</td> <td>domain suffix, starting with a dot</td> <td>".mozilla.org, mozilla.org"</td> <td>You must use both entries to fully block a single domain.</td> </tr> <tr> <td>hostname (without domain)</td> <td>hostname-only (see problems below)</td> <td>"localhost"</td> <td>Also blocks any possible domains that start with the entry ("www.otherdomain.localhost")</td> </tr> <tr> <td>hostname (with domain)</td> <td>domain name</td> <td>"www.mozilla.org"</td> <td>Also blocks any hostnames or domains that end in the same string (other-www.mozilla.org)</td> </tr> <tr> <td>IP address</td> <td>IP address</td> <td>"1.2.3.4"</td> <td>Does not block hostnames that resolve to the IP address ("127.0.0.1" does not block "localhost")</td> </tr> <tr> <td>Network</td> <td>network w/ CIDR block</td> <td>"10.0.0.0/8"</td> <td>Does not block hostnames that resolve to the IP address range (10.0.0.0/8 is not "no proxy for intranet hostnames")</td> </tr> <tr> <td>optional - port-specific</td> <td>(optional) ":" + port number</td> <td>"&lt;FILTER&gt;:81"</td> <td>Only black-lists port. Only applies to one port (no support for ranges and/or multiple ports). Port-only filters ":80" or "*:80" are not supported.</td> </tr>
</tbody></table>
<table class="fullwidth-table"> <tbody><tr> <th>Formats that are not accepted</th> <th>Example</th> </tr> <tr> <td>Domain filters with interior wildcards</td> <td>www.*.com</td> </tr> <tr> <td>IP address string prefixes</td> <td>127. or 10.0.*</td> </tr> <tr> <td>IP addresses with wildcards in quads</td> <td>10.*.*.*</td> </tr>
</tbody></table>
<h3 id="Preferences" name="Preferences"> Preferences </h3>
<table class="fullwidth-table"> <tbody><tr> <td>Name</td> <td><code>network.proxy.no_proxy_on</code></td> </tr> <tr> <td>Default value</td> <td><code>localhost, 127.0.0.1</code></td> </tr>
</tbody></table>
<p>By default "localhost" and "127.0.0.1" are excluded, since most people assume these should connect to the local system.
</p>
<div class="note">
<p>Note: When IPv6 support is added, additional addresses will need to be added and tested.
</p>
</div>
<p>Communicator used "<code>network.proxy.none</code>"
</p>
<h3 id="Limitations" name="Limitations"> Limitations </h3>
<ul><li> No IPv6 support - The backend stores IPv4 addresses as IPv6, but IPv6 formats are not supported.
</li><li> Scalability - Not usable for local domains with numerous hostnames. Not usable for large number of filters (ad blocking).
</li><li> Mozilla implements this feature with significant limitations, users may find that writing a PAC file is more suitable for their needs.
</li></ul>
<h3 id="Technical_discussion" name="Technical_discussion"> Technical discussion </h3>
<p>The most important aspect of "no proxy for" is that it uses a right-sided string compare. This algorithm works in most cases, but is also the root cause of most broken functionality. In recent versions, more logic was added for IP addresses, but the legacy problems remain for domain names.
</p><p>The main problem is rooted in the DNS syntax, which allows a hostname to exist at the node level of a domain name. In other words, for the domain "mozilla.org", the valid entries would syntactically fall into three categories:
</p>
<ol><li> Subdomains (and the contents of those subdomains) of mozilla.org ("server.office.mozilla.org")
</li><li> hostnames in mozilla.org ("www.mozilla.org")
</li><li> node-level entries of mozilla.org ("mozilla.org").
</li></ol>
<p>You cannot specify a prefix that will do a suffixed-string compare that will match all three cases precisely. </p>
<ul><li> ".mozilla.org" will not match "mozilla.org"
</li><li> "mozilla.org" will match all three cases, but also match any domains that end in the string "mozilla.org", such as "www.amozilla.org".
</li></ul>
<h3 id="Testing" name="Testing"> Testing </h3>
<p>Contributors can test this feature, even without a proxy server, using a "negative proxy server test". Proxy connections that fail return an error "The proxy server you have configured  cannot be found", so configure your browser to use a non-existent HTTP proxy (hostname: "imaginary", port "80"). Test each destination in an http: URL. All proxied URLs will return errors, all non-proxied connections will be attempted normally (direct connection).
</p>
<table class="fullwidth-table"> <tbody><tr> <th>Hostnames</th> <th>filter</th> <th>test destination</th> <th>result</th> </tr> <tr> <td>basic filtering unit tests (local host)</td> <td>localhost</td> <td>localhost</td> <td>direct</td> </tr> <tr> <td> </td><td>localhost.</td> <td>localhost</td> <td>proxy</td> </tr> <tr> <td> </td><td>127.0.0.1</td> <td>local host</td> <td>direct</td> </tr> <tr> <td>confirm the filter uses only suffix matches (hostname unit tests)</td> <td>hostname</td> <td>hostname</td> <td>direct</td> </tr> <tr> <td> </td><td>name</td> <td>hostname</td> <td>direct</td> </tr> <tr> <td> </td><td>host</td> <td>hostname</td> <td>proxy</td> </tr> <tr> <td>domains with numbers</td> <td>3com.com</td> <td>.3com.com</td> <td>direct</td> </tr> <tr> <th>FQDNs</th> <td> </td><td> </td><td> </td></tr> <tr> <td>hostname.domain.com</td> <td>hostname.domain.com</td> <td>domain.com</td> <td>proxy</td> </tr> <tr> <td> </td><td>hostname.domain.com</td> <td>hostname.domain.com</td> <td>direct</td> </tr> <tr> <td> </td><td>hostname.domain.com</td> <td>host.hostname.domain.com</td> <td>direct</td> </tr> <tr> <td>.domain.com</td> <td>.domain.com</td> <td>domain.com<br>hostname.domain.com<br>host.hostname.domain.com</td> <td>direct<br>direct<br>direct</td> </tr> <tr> <td>*.domain.com</td> <td>*.domain.com</td> <td> </td><td>same results as ".domain.com"</td> </tr> <tr> <td> </td><td>*domain.com</td> <td> </td><td>same results as "domain.com"</td> </tr> <tr> <th>IP address</th> <td> </td><td> </td><td> </td></tr> <tr> <td>host IP address</td> <td>127.0.0.1</td> <td>127.0.0.1</td> <td>direct</td> </tr> <tr> <td> </td><td>127.0.0.0</td> <td>127.0.0.1</td> <td>proxy</td> </tr> <tr> <td>network range</td> <td>127.0.0.0/8</td> <td>127.0.0.1</td> <td>direct</td> </tr> <tr> <td> </td><td>127/8</td> <td>127.0.0.1</td> <td>proxy</td> </tr> <tr> <td> </td><td>127.*.*.*</td> <td>127.0.0.1</td> <td>proxy</td> </tr> <tr> <td> </td><td>127.</td> <td>127.0.0.1</td> <td>proxy</td> </tr> <tr> <th>Ports</th> <td>1</td> <td>127.0.0.1</td> <td>proxy</td> </tr> <tr> <td>no port</td> <td>.mozilla.org</td> <td>www.mozilla.org:80<br>www.mozilla.org:81</td> <td>direct<br>direct</td> </tr> <tr> <td>port number</td> <td>.mozilla.org:80</td> <td>www.mozilla.org:80<br>www.mozilla.org</td> <td>direct<br>direct</td> </tr> <tr> <td> </td><td>:81</td> <td>www.mozilla.org:81</td> <td>proxy</td> </tr>
</tbody></table>
<h3 id="Developer_notes" name="Developer_notes"> Developer notes </h3>
<p>The no_proxy for logic is written in C++. PAC is written in JS, so there are potential problems with feature consistency and porting. David Baron has pointed out that the original PAC code in the "classic" tree is written in C++. The PAC in C++ has not been tested in mozilla, so porting PAC in C++ forward would not be a panacea.
</p><p>The relevant code lives in {{ Source("netwerk/base/src/nsProtocolProxyService.cpp", "nsProtocolProxyService.cpp") }}.
</p>
<ul><li> {{ LXRSearch("ident", "i", "nsProtocolProxyService", "nsProtocolProxyService") }}::{{ LXRSearch("ident", "i", "PrefsChanged", "PrefsChanged") }} - loads preferences
</li><li> {{ LXRSearch("ident", "i", "nsProtocolProxyService", "nsProtocolProxyService") }}::{{ LXRSearch("ident", "i", "LoadFilters", "LoadFilters") }} - parses filters
</li><li> {{ LXRSearch("ident", "i", "nsProtocolProxyService", "nsProtocolProxyService") }}::{{ LXRSearch("ident", "i", "ExamineForProxy", "ExamineForProxy") }} - decides to check filters if configuration is "manual"
</li><li> {{ LXRSearch("ident", "i", "nsProtocolProxyService", "nsProtocolProxyService") }}::{{ LXRSearch("ident", "i", "CanUseProxy", "CanUseProxy") }} - performs URL vs. filter comparison
</li></ul>
<h3 id="Notable_bugs" name="Notable_bugs"> Notable bugs </h3>
<ul><li> {{ Bug("172083") }} - {{ mediawiki.external('meta') }} Proxy: "no proxy for" items
</li><li> {{ Bug("80917") }} - Proxy: "No Proxy" w/ form based UI
</li><li> {{ Bug("91587") }} - Proxy: "no proxy for" default domain filtering fails w/ non-FQDN (e.g., <a class=" external" href="http://web/" rel="freelink">http://web/</a>)
</li><li> {{ Bug("277378") }} - No proxy for: support IPv6 address literals
</li><li> {{ Bug("136789") }} - Proxy: no proxy IP entries do not block DNS resolved IPs
</li><li> {{ Bug("314712") }} - No proxy for: "hostname.domain.com" should block only "hostname.domain.com"
</li><li> {{ Bug("72444") }} - Proxy: "bypass proxy server for local addresses" (IE pref)
</li><li> {{ Bug("260883") }} - "No proxy for" does not use FQDN wildcards "*" like IE
</li></ul>
<h3 id="Bugzilla_sources" name="Bugzilla_sources"> Bugzilla sources </h3>
<p>{{ Bug("17158") }} comment 21:
</p>
<blockquote>
the correct separator are spaces or commas. So use this-<br><br>dogwood.state.mo.us .intra.state.mo.us dor.intranet<br>or<br>dogwood.state.mo.us, .intra.state.mo.us, dor.intranet<br><br>Note that you don't need to (read shouldn't) put a * for all hosts with that <br>domain ending. Corollary- a no_proxies_on entry of netscape.com will be <br>applicable to all hosts ending at netscape.com including foonetscape.com but an <br>entry of .netscape.com will specifically be applicable to all hosts in the <br>domain of netscape.com.<br>
</blockquote>
<div class="originaldocinfo">
<h2 id="Original_Document_Information" name="Original_Document_Information"> Original Document Information </h2>
<ul><li> Author(s): Benjamin Chuang
</li><li> Last Updated Date: November 2, 2005
</li><li> Copyright Information: Portions of this content are © 1998–2007 by individual mozilla.org contributors; content available under a Creative Commons license | <a class="external" href="http://www.mozilla.org/foundation/licensing/website-content.html">Details</a>.
</li></ul>
</div>
Revert to this revision