mozilla
Your Search Results

    Writing WebSocket servers

    This article is in need of an editorial review.

    Overview

    A WebSocket server is a TCP application listening on any port of a server that follows a specific protocol, simple as that. The task of creating a custom server tends to scare people; however, it can be easy to implement a simple WebSocket server on your platform of choice.

    A WebSocket server can be written in any server-side programming language that is capable of Berkeley sockets, such as C(++) or Python or even PHP and server-side JavaScript. This is not a tutorial in any specific language, but serves as a guide to facilitate writing your own server.

    You will need to already know how HTTP works and have medium programming experience. Depending on language support, knowledge of TCP sockets may be required. The scope of this guide is to present the minimum knowledge you need to write a WebSocket server.

    Read the latest official WebSockets specification, RFC 6455. Sections 1 and 4-7 are especially interesting to server implementors. Section 10 discusses security and you should definitely peruse it before exposing your server.

    A WebSocket server is explained on a very low level here. WebSocket servers are often separate and specialized servers (for load-balancing or other practical reasons), so you will often use a reverse proxy (such as a regular HTTP server) to detect WebSocket handshakes, pre-process them, and send those clients to a real WebSocket server. This means that you don't have to bloat your server code with cookie and authentication handlers (for example).

    Step 1: The WebSocket Handshake

    First of all, the server must listen for incoming socket connections using a standard TCP socket. Depending on your platform, this may be handled for you already. For this example, let's assume that your server is listening on example.com, port 8000, and your socket server responds to GET requests on /chat.

    Warning: The server may listen on any port it chooses, but if it chooses any port other than 80 or 443, it may have problems with firewalls and/or proxies. Connections on port 443 tend to succeed more often but of course, that requires a secure connection (TLS/SSL). Also, note that most browsers (notably Firefox 8+) do not allow connections to insecure WebSocket servers from secure pages.

    The handshake is the "Web" in WebSockets. It's the bridge from HTTP to WS. In the handshake, details of the connection are negotiated, and either party can back out before completion if the terms are unfavorable. The server must be careful to understand everything the client asks for, otherwise security issues will be introduced.

    Client Handshake Request

    Even though you're building a server, a client still has to start the WebSocket handshake process. So you must know how to interpret the client's request. The client will send a pretty standard HTTP request that looks like this (the HTTP version must be 1.1 or greater, and the method must be GET):

    GET /chat HTTP/1.1
    Host: example.com:8000
    Upgrade: websocket
    Connection: Upgrade
    Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
    Sec-WebSocket-Version: 13
    
    

    The client can solicit extensions and/or subprotocols here; see Miscellaneous for details. Also, common headers like User-Agent, RefererCookie, or authentication headers might be there as well. Do whatever you want with those, they don't directly pertain to the WebSocket. It's also safe to ignore them. In many common setups, a reverse proxy has already dealt with them.

    If any header is not understood or has an incorrect value , the server should send a "400 Bad Request" and immediately close the socket. As usual, it may also give the reason why the handshake failed in the HTTP response body, but the message may never be displayed (browsers at least do not display it). If the server doesn't understand that version of WebSockets, it should send a Sec-WebSocket-Version header back that contains the version(s) it does understand. (This guide explains v13, the newest). Now, let's move on to the most curious header, Sec-WebSocket-Key.

    Tip: All browsers will send an Origin header. You can use this header for security (checking for same origin, whitelisting/ blacklisting, etc.) and send a 403 Forbidden if you don't like what you see. However, be warned that non-browser agents can just send a faked Origin. Most applications will reject requests without this header.

    Tip: The request-uri (/chat here) has no defined meaning in the spec. So many people cleverly use it to let one server handle multiple WebSocket applications. For example, example.com/chat could invoke a multiuser chat app, while /game on the same server might invoke a multiplayer game.

    Note: Regular HTTP status codes can only be used before the handshake. After the handshake succeeds, you have to use a different set of codes (defined in section 7.4 of the spec).

    Server Handshake Response

    When it gets this request, the server should send a pretty odd-looking (but still HTTP) response that looks like this (remember each header ends with \r\n and put an extra \r\n after the last one):

    HTTP/1.1 101 Switching Protocols
    Upgrade: websocket
    Connection: Upgrade
    Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=
    
    

    Additionally, the server can decide on extension/subprotocol requests here; see Miscellaneous for details. The Sec-WebSocket-Accept part is interesting. The server must derive it from the Sec-WebSocket-Key that the client sent. To get it, concatenate the client's Sec-WebSocket-Key and "258EAFA5-E914-47DA-95CA-C5AB0DC85B11" together (it's a "magic string"), take the SHA-1 hash of the result, and return the base64 encoding of the hash.

    FYI: This seemingly overcomplicated process exists so that it's obvious to the client whether or not the server supports WebSockets. This is important because security issues might arise if the server accepts a WebSockets connection but interprets the data as a HTTP request.

    So if the Key was "dGhlIHNhbXBsZSBub25jZQ==", the Accept will be "s3pPLMBiTxaQ9kYGzzhZRbK+xOo=". Once the server sends these headers, the handshake is complete and you can start swapping data!

    The server can send other headers like Set-Cookie, or ask for authentication or redirects via other status codes, before sending the reply handshake.

    Keeping track of clients

    This doesn't directly relate to the WebSocket protocol, but it's worth mentioning here: your server will have to keep track of clients' sockets so you don't keep handshaking again with clients who have already completed the handshake. The same client IP address can try to connect multiple times (but the server can deny them if they attempt to many connections in order to save itself from Denial-of-Service attacks).

    Step 2: Exchanging Data Frames

    Either the client or the server can choose to send a message at any time — that's the magic of WebSockets. However, extracting information from these so-called "frames" of data is a not-so-magical experience. Although all frames follow the same specific format, data going from the client to the server is masked using XOR encryption (with a 32-bit key). Section 5 of the specification describes this in detail.

    Format

    Each data frame (from the client to the server or vice-versa) follows this same format:

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-------+-+-------------+-------------------------------+
    |F|R|R|R| opcode|M| Payload len |    Extended payload length    |
    |I|S|S|S|  (4)  |A|     (7)     |             (16/64)           |
    |N|V|V|V|       |S|             |   (if payload len==126/127)   |
    | |1|2|3|       |K|             |                               |
    +-+-+-+-+-------+-+-------------+ - - - - - - - - - - - - - - - +
    |     Extended payload length continued, if payload len == 127  |
    + - - - - - - - - - - - - - - - +-------------------------------+
    |                               |Masking-key, if MASK set to 1  |
    +-------------------------------+-------------------------------+
    | Masking-key (continued)       |          Payload Data         |
    +-------------------------------- - - - - - - - - - - - - - - - +
    :                     Payload Data continued ...                :
    + - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +
    |                     Payload Data continued ...                |
    +---------------------------------------------------------------+

    RSV1-3 can be ignored, they are for extensions.

    The MASK bit simply tells whether the message is encoded. Messages from the client must be masked, so your server should expect this to be 1. (In fact, section 5.1 of the spec says that your server must disconnect from a client if that client sends an unmasked message.) When sending a frame back to the client, do not mask it and do not set the mask bit. We'll explain masking later. Note: You have to mask messages even when using a secure socket.

    The opcode field defines how to interpret the payload data: 0x0 for continuation, 0x1 for text (which is always encoded in UTF-8), 0x2 for binary, and other so-called "control codes" that will be discussed later. In this version of WebSockets, 0x3 to 0x7 and 0xB to 0xF have no meaning.

    The FIN bit tells whether this is the last message in a series. If it's 0, then the server will keep listening for more parts of the message; otherwise, the server should consider the message delivered. More on this later.

    Decoding Payload Length

    To read the payload data, you must know when to stop reading. That's why the payload length is important to know. Unfortunately, this is somewhat complicated. To read it, follow these steps:

    1. Read bits 9-15 (inclusive) and interpret that as an unsigned integer. If it's 125 or less, then that's the length; you're done. If it's 126, go to step 2. If it's 127, go to step 3.
    2. Read the next 16 bits and interpret those as an unsigned integer. You're done.
    3. Read the next 64 bits and interpret those as an unsigned integer. You're done.

    Reading and Unmasking the Data

    If the MASK bit was set (and it should be, for client-to-server messages), read the next 4 octets (32 bits); this is the masking key. Once the payload length and masking key is decoded, you can go ahead and read that number of bytes from the socket. Let's call the data ENCODED, and the key MASK. To get DECODED, loop through the octets (bytes a.k.a. characters for text data) of ENCODED and XOR the octet with the (i modulo 4)th octet of MASK. In pseudo-code (that happens to be valid JavaScript):

    var DECODED = "";
    for (var i = 0; i < ENCODED.length; i++) {
        DECODED[i] = ENCODED[i] ^ MASK[i % 4];
    }

    Now you can figure out what DECODED means depending on your application.

    Message Fragmentation

    The FIN and opcode fields work together to send a message split up into separate frames.  This is called message fragmentation. Fragmentation is only available on opcodes 0x0 to 0x2.

    Recall that the opcode tells what a frame is meant to do. If it's 0x1, the payload is text. If it's 0x2, the payload is binary data. However, if it's 0x0, the frame is a continuation frame.  This means the server should concatenate the frame's payload to the last frame it recieved from that client. Here is a rough sketch, in which a server reacts to a client sending text messages.  The first message is sent in a single frame, while the second message is sent across three frames.  FIN and opcode details are shown only for the client:

    Client: FIN=1, opcode=0x1, msg="hello"
    Server: (process complete message immediately) Hi.
    Client: FIN=0, opcode=0x1, msg="and a"
    Server: (listening, new message containing text started)
    Client: FIN=0, opcode=0x0, msg="happy new"
    Server: (listening, payload concatenated to previous message)
    Client: FIN=1, opcode=0x0, msg="year!"
    Server: (process complete message) Happy new year to you too!

    Notice the first frame contains an entire message (has FIN=1 and opcode!=0x0), so the server can process or respond as it sees fit.  The second frame sent by the client has a text payload (opcode=0x1), but the entire message has not arrived yet (FIN=0).  All remaining parts of that message are sent with continuation frames (opcode=0x0), and the final frame of the message is marked by FIN=1.  Section 5.4 of the spec describes message fragmentation.

    Pings and Pongs: The Heartbeat of WebSockets

    At any point after the handshake, either the client or the server can choose to send a Ping to the other party. When the Ping is recieved, the recipient must send back a Pong as soon as possible. You can use this to make sure that the client is still connected, for example.

    A ping or pong is just a regular frame, but it's a control frame. Pings have an opcode of 0x9, and Pongs have an opcode of 0xA. When you get a Ping, send back a Pong with the exact same Payload Data as the Ping (for pings and pongs, the max payload length is 125). You might also get a Pong without ever sending a ping; ignore this if it happens.

    If you have gotten more than one Ping before you get the chance to send a Pong, you only send one Pong.

    Step 4: Closing the connection

    TODO. Please expand this section if you are equipped to do so.

    Miscellaneous

    WebSocket codes, extensions, subprotocols, etc. are registered at the IANA WebSocket Protocol Registry.

    WebSocket extensions and subprotocols are negotiated via headers during the handshake. Sometimes extensions and subprotocols seem too similar to be different things, but there is a clear distinction. Extensions control the WebSocket frame and modify the payload, while subprotocols structure the WebSocket payload and never modify anything. Extensions are optional and generalized (like compression); subprotocols are mandatory and localized (like ones for chat and for MMORPG games).

    Extensions

    This section needs expansion. Please edit if you are equipped to do so.

    Think of an extension as compressing a file before e-mailing it to someone. Whatever you do, you're sending the same data in different forms. The recipient will eventually be able to get the same data as your local copy, but it is sent differently. That's what an extension does. WebSockets defines a protocol and a simple way to send data, but an extension such as compression could allow sending the same data but in a shorter format.

    Extensions are explained in sections 5.8, 9, 11.3.2, and 11.4 of the spec.

    TODO

    Subprotocols

    Think of a subprotocol as a custom XML schema or doctype declaration. You're still using XML and its syntax, but you're additionally restricted by a structure you agreed on. WebSocket subprotocols are just like that. They do not introduce anything fancy, they just establish structure. Like a doctype or schema, both parties must agree on the subprotocol; unlike a doctype or schema, the subprotocol is implemented on the server and cannot be externally refered to by the client.

    Subprotocols are explained in sections 1.9, 4.2, 11.3.4, and 11.5 of the spec.

    A client has to ask for a specific subprotocol. To do so, it will send something like this as part of the original handshake:

    GET /chat HTTP/1.1
    ...
    Sec-WebSocket-Protocol: soap, wamp
    
    

    or, equivalently:

    ...
    Sec-WebSocket-Protocol: soap
    Sec-WebSocket-Protocol: wamp
    
    

    Now the server must pick one of the protocols that the client suggested and it supports. If there are more than one, send the first one the client sent. Imagine our server can use both soap and wamp. Then, in the response handshake, it'll send:

    Sec-WebSocket-Protocol: soap
    
    

    The server can't send more than one Sec-Websocket-Protocol header.
    If the server doesn't want to use any subprotocol, it shouldn't send any Sec-WebSocket-Protocol header. Sending a blank header is incorrect.
    The client may close the connection if it doesn't get the subprotocol it wants.

    If you want your server to obey certain subprotocols, then naturally you'll need extra code on the server. Let's imagine we're using a subprotocol json. In this subprotocol, all data is passed as JSON. If the client solicits this protocol and the server wants to use it, the server will need to have a JSON parser. Practically speaking, this will be part of a library, but the server will need to pass the data around.

    Tip: To avoid name conflict, it's recommended to make your subprotocol name part of a domain string. If you are building a custom chat app that uses a proprietary format exclusive to Example Inc., then you might use this: Sec-WebSocket-Protocol: chat.example.com. For different versions, a widely-used method is to add a / followed by the version number, like chat.example.com/2.0. Note that this isn't required, it's just an optional convention, and you can use any string you wish.

    Document Tags and Contributors

    Contributors to this page: kdbanman, oddalot, Arshabh, dron, laobubu, mrbrc, bitrayne, anistark, rvighne, styfle
    Last updated by: oddalot,