We released nghttp2 v1.16.0. We summarizes the changes below.

libnghttp2

Previously, if libnghttp2 is built with DEBUGBUILD macro defined, it prints out debug messages into stderr. In this release, Anders Bakken added nghttp2_set_debug_vprintf_callback() function to set a callback which can customize how debug message is processed. The parameters passed to the callback are suitable for use with vfprintf(3) function.

libnghttp2_asio

We fixed the bug which causes crash if nghttp2::asio_http2::server::response::end() is called from outside nghttp2 callback (e.g., asynchronous timer callback).

nghttpx

We have added --backend-connect-timeout option to specify how long nghttpx waits until backend TCP connection is established.

The new option --ecdh-curves lets you specify the list of named curve for use in TLS.

We have added TLS signed_certificate_timestamp extension support. signed_certificate_timestamp extension is defined in RFC 6962. The new option --tls-sct-dir is used to specify the directory which contains *.sct files. These files are read in start up, and sent to client in TLS handshake. The format of *.sct files is the same as the one that nginx and Apache mod_ssl_ct use. For additional certificates specified by --subcert option, we extended the syntax of the option, and now it can take sct-dir parameter which takes the directory that should contain *.sct files for the certificate.

h2load

We have added --header-table-size and --encoder-header-table-size options to specify HPACK header table size for both direction.

We released nghttp2 v1.15.0. We summarizes the changes below.

libnghttp2

Previously, the maximum size of dynamic header table size used by HPACK encoder was limited to 4KiB regardless of SETTINGS_HEADER_TABLE_SIZE sent by peer. In this release, we added nghttp2_option_set_max_deflate_dynamic_table_size() to change the maximum value of encoder’s maximum dynamic header table size. With this option, nghttp2 based client/server can experiment the larger or smaller dynamic table size.

Previously, we could not return successfully from nghttp2_data_source_read_callback without reading anything or NGHTTP2_ERR_DEFERRED return value. The latter requires nghttp2_session_resume_data(), and is not a good workaround. In this release, application can now return NGHTTP2_ERR_CANCEL from nghttp2_data_source_read_callback without reading anything, and it signals the libnghttp2 to return to the application code immediately.

To offer the opportunity to implement https://tools.ietf.org/html/draft-benfield-http2-debug-state-01 to the nghttp2 based servers, we added API functions to export internal HTTP/2 state data from nghttp2_session object. In this release, we export the data marked as “required” in the draft. Here is the list of the added functions:

• nghttp2_session_get_hd_deflate_dynamic_table_size() which returns the dynamic table size of HPACK encoder

• nghttp2_session_get_hd_inflate_dynamic_table_size() which returns the dynamic table size of HPACK decoder

• nghttp2_session_get_local_settings() which returns local HTTP/2 SETTINGS in effect; this is the SETTINGS sent from the local endpoint to the remote one

• nghttp2_session_get_local_window_size() which returns the connection window size

• nghttp2_session_get_stream_local_window_size() which returns the stream window size for given stream

Third-Party

We have updated neverbleed, and it now supports ECDSA certificate.

src

Now applications under src directory compiles with OpenSSL 1.1.0.

nghttpx

To utilize the new feature to change HPACK encoder’s dynamic table size described above, we added new options to achieve this. The new options are:

• --frontend-http2-encoder-dynamic-table-size
• --frontend-http2-decoder-dynamic-table-size
• --backend-http2-encoder-dynamic-table-size
• --backend-http2-decoder-dynamic-table-size

These options default to 4KiB.

We have added tls_sni to mruby Nghttpx::Env class, which returns the server name sent in TLS SNI from client.

Previously, we have --frontend-http2-window-bits and its family options. They were not flexible because they only accept number of bits. Now they have been deprecated, and instead we have introduced --frontend-http2-window-size and its family options, which take the size in integer, rather than bits. The deprecated options still work, and are translated into the new options, but we encourage users to update configuration to use new options.

We have implemented TCP write buffer optimization presented by Kazuho’s slide. In short, this optimization limits the number of bytes to write to TCP socket based on the TCP CWND, and just write the bytes which can be sent in 1 RTT. This avoids excessive commitment of low prioritized data to the TCP socket, and implementation can quickly respond to the high prioritized data. This optimization is experimental, and enabled by --frontend-http2-optimize-write-buffer-size, and only works with HTTP/2 TLS connections. At the moment, only Linux is supported.

We also added HTTP/2 window size auto tuning optimization. It adjusts connection window size of frontend HTTP/2 connection based on RWIN. This is highly experimental, and may not work as expected. This feature is experimental, enabled by --frontend-http2-optimize-window-size, and only works with HTTP/2 TLS connections. At the moment, only Linux is supported. In the future release, we may drop the requirement of TLS for this optimization.

We added workaround for std::make_shared bug in Xcode7, 7.1, and 7.2 to prevent nghttpx from crashing.

We fixed the bug that bytes are doubly counted towards rate limit for TLS connections.

Previously, with default mode, server header field was rewritten to “nghttpx” and its version. Now --no-server-rewrite option disables this, and just forwards the server header field from the backend. We have added --server-name option to specify the server header field value. If both options are present, --no-server-rewrite takes precedence.

Previously, we ignored invalid header field coming from HTTP/2. Now they are treated as stream error.

nghttp and nghttpd

We have added --encoder-header-table-size option to specify the HPACK encoder’s maximum dynamic header table size.

Python

We have added ALPN support, and now requires at least Python 3.5.

We released nghttp2 v1.14.1.

In this release, we fixed the bug which causes GOAWAY race with new incoming stream on server side. The bug has been reported in GH-681. This is a regression introduced in 16c4611. We were happy with that commit since nghttp2 server passed all strict mode h2spec tests. However, it turned out that it could not handle some cases well, and one of them is GOAWAY race on server side. We reverted part of that commit to fix this issue. This bug only affects nghttp2 server side session. The client side nghttp2 session is not affected by this bug.

We released nghttp2 v1.14.0. We summarizes the changes below per category.

libnghttp2

Wenfeng Liu contributed several commits to mainly HPACK related code. Most notably, Wenfeng added nghttp2_hd_deflate_hd_vec() function, which can takes multiple output buffers to encode HTTP header fields, in a same spirit of writev(2). Wenfeng also cleaned up source code, and added optimizations.

We added nghttp2_on_invalid_header_callback to pass the invalid header fields to application. We say header field is invalid if it contains a character which is now allowed in header field. Previously, libnghttp2 silently ignored them. Now application can use this callback to catch these header fields, and it can reset a stream if it wishes.

HTTP/2 priority handling is complex thing, but we fixed a bug that libnghttp2 performs wrong tree operation to avoid dependency cycle. https://tools.ietf.org/html/rfc7540#section-5.3.3 explains how to transform dependency tree to avoid circular dependency. Previously, we wrongly always moved the dependent stream under the root stream. The correct destination is the parent stream of the stream to reprioritize. This is not a security bug.

We have deprecated NGHTTP2_INITIAL_MAX_CONCURRENT_STREAMS macro, which was defined as INT32_MAX. Actually, the SETTINGS value can contain 32 bit unsigned integer, it is not really an initial value. We think deprecation does not affect most of the application, since typically they requires much lower concurrent stream limit (say, 100).

We have tightened up stream state handling for server side session, and now nghttpx and nghttpd can pass all h2spec tests with strict mode enabled.

Documentation

We removed old documentation about HPACK differential coding. It was removed when we ditched reference set from HPACK specification.

We also mention about ALPN support in nghttpx HOW-TO.

nghttpx

We have got several bug report about the issue that the backend connection cannot be established. We added several WARN level log messages to debug this situation easier.

Previously, we silently changed pushed stream’s priority if it is CSS, Javascript, or html, by mutating server side priority tree, so that they can be sent along with associated resource (usually, parent html). There is a discussion in httpbis mailing list which argues that dependency tree is for client, and changing it in server side is not what client expects. Ideally, it is a browser’s job to prioritize pushed resource, but we need 1 RTT to get this PRIORITY frame from browser. We will work on better approach how to prioritize pushed stream in initial phase.

nghttpx can now log the backend host and port in access log file. See --accesslog-format option for more details.

We have fixed the bug that api and healthmon parameter do not work with --http2-proxy option.

nghttpx now reloads configuration file when it receives SIGHUP.

nghttp

nghttp now accepts multiple -p option to set weight for corresponding URI in command-line.

deflatehd

We fixed it so that it only emits dynamic header table size update when header table size is changed from default.

We released nghttp2 v1.13.0.

This release fixes several build issues, and add a new library feature. nghttpx session affinity feature has been improved.

We can now cancel non-DATA frame transmission from nghttp2_before_frame_send_callback by returning NGHTTP2_ERR_CANCEL from there. This could be useful for some cases where we know the transmission of a frame should be avoided. The upcoming use case is cache digest. After queuing PUSH_PROMISE frame, but before actually sending the frame, we may know client has the fresh copy of the resources by CACHE_DIGEST frame. In that case, it is desirable to cancel transmission of PUSH_PROMISE frame. nghttp2_on_frame_not_send_callback is invoked if frame transmission is canceled as usual.

We fixed number of warnings with Sphinx 1.4.0.

nghttpx’s session affinity using client IP is improved by using the similar hashing technique described in https://github.com/RJ/ketama

We released nghttp2 v1.12.0.

This release adds 2 new API functions to libnghttp2. It also adds HTTP/1.1 POST support to h2load. nghttpx gets new features, and performance improvements.

We’ve added new API function, nghttp2_option_set_max_send_header_block_length(). Previously, the maximum size of header block to send (calculated using the same algorithm using nghttp2_hd_deflate_bound()) is hard coded as 64KiB. This is probably sufficient in practice, but sometimes it is desirable to send more, for example, testing purposes. nghttp2_option_set_max_send_header_block_length() lets you set this maximum size by application for its demand.

The another new API function is nghttp2_session_set_local_window_size(). Previously, application can increase or even decrease local window size (which is the window size the remote endpoint must obey when sending DATA to local endpoint) by using nghttp2_submit_window_size(). But it was a bit hard to use since it takes the difference against the current window size. It also affects the current received byte count which is not acknowledged by WINDOW_UPDATE frame, which is not always desirable. The new function allows application to set the absolute value of local window size instead of relative value, and libnghttp2 makes it happen. It does not affect the current received byte count.

We have client and server implementations under examples directory written for tutorials. Previously, they only supported NPN TLS extension. Now they finally get ALPN TLS extension support.

h2load has supported HTTP POST for quite sometime, but it turned out that its support was limited to HTTP/2 and SPDY. Now we implemented HTTP POST for HTTP/1.1 as well. There is a known limitation. When HTTP POST is used with HTTP/1.1 connection, -m option is ignored, and it is treated as -m1, which means that HTTP pipelining is disabled.

We added alternative mode to nghttpx frontend, and introduced 2 modes in this release. This mode is set using --frontend option.

The first alternative mode is health monitor mode. It is enabled per listener address by specifying healthmon parameter to --frontend option. If it is enabled, all requests coming from that listener are replied with HTTP 200 status code with no response body, without contacting backend server at all. This mode is useful to do health check for nghttpx itself.

The second alternative mode is API mode. It is enabled per listener address by specifying api parameter to --frontend option. The requests coming from this listener are treated specially as API request. In this release, we have only one API endpoint, which is PUT /api/v1beta1/backendconfig. The client can post snippet of nghttpx configuration file containing backend option lines to this API endpoint. On reception, nghttpx will parse the request body, and replace the backend servers on the fly without restarting nghttpx process. The request body is the same format as the file given to --conf option.

nghttpx gets client IP based session affinity support. It is enabled per forwarding pattern basis (see <PATTERN> in --backend option) by adding affinity=ip parameter. If it is enabled, all connections from the same IP address will be forwarded to the same backend address. The session affinity may break if one of the backend gets unreachable, or backend settings are reloaded or replaced by API.

We identified longer TTFB and connect times (you know h2load has these metrics) when lots of concurrent connections are made to nghttpx. We fixed this issue in this release, and now nghttpx does not exhibit longer waiting times in this condition.

We released nghttp2 v1.11.1.

This release fixes the bug in nghttpx. The bug is that regular timeout on HTTP/1 backend makes that backend unavailable for load balancing.

We released nghttp2 v1.11.0.

This release fixes some bugs both in libnghttp2 and applications, and fixes minor API issue. We fixed compile error with OpenSSL 1.1.0 pre5. We also improved nghttpx functionality.

API bugs

libnghttp2 offers HPACK API. nghttp2_hd_inflate_hd() decodes incoming header block data to actual header field name/value pair. The shortcoming of its API is that its input parameter is not qualified as const. To fix this issue, we added new nghttp2_hd_inflate_hd2(). Other than the added const-ness, they share the same functionality.

Previously, nghttp2_submit_ping() ignored PING flag passed as flags parameter. This release fixes this bug.

kqueue issues

We use libev for our applications. According to the libev manual, kqueue is not selected as eligible backend for the BSD based system other than NetBSD. Because we usually use event for sockets, we explicitly enabled kqueue backend if it is available. For nghttpx, we have --no-kqueue option to disable it.

nghttp : Fixed zero length DATA frame issue

We fixed zero length DATA frame issue described in https://blogs.dropbox.com/tech/2016/05/enabling-http2-for-dropbox-web-services-experiences-and-observations/

This issue arises if nghttp client sends request body with content-length header field. If server only sends WINDOW_UPDATE with the same amount of content-length, tranfer stalls while all data have been sent. This comes from the design of libnghttp2; its nghttp2_data_source_read_callback is only called when there is at least 1 byte of flow control window. But it can completely be avoided. Since we have content-length, we can just tell libnghttp2 that we have read all data now. This effectively eliminates 0 length DATA farme

nghttpx: New attributes for mruby object, and other enhancements

In nghttpx, we added new env.server_addr, env.tls_used and env.server_port attributes. These are all documented in nghttpx manual page.

The timing when PID file is saved is pushed back, and it is now saved when setup of all listening sockets has completed. This is useful when we do graceful shutdown of nghttpx. Now we can just check PID file is changed, and then issue QUIT signal to the old process.

--backend option now has sni parameter to specify SNI field to send backend HTTPS server.

Previously, when we call fetch-ocsp-response script, we didn’t pass environ to the new program. Now we use execv to pass environ.

Previously, the proto and tls parameter in --backend option must be the same if they are the same routing pattern. Now this requirement is removed, and we can mix their combinations for the same pattern.

For HTTP/2 backend, we now consider the backend is alive only when we receive SETTINGS ACK from backend; just successfully connecting to the backend is not enough.

We released nghttp2 v1.10.0.

This release adds ALTSVC frame support in libnghttp2. nghttp gets new option to exercise expect/continue dance with server. nghttpx gets several new features, robust load balancing, and bug fixes.

ALTSVC frame support, and enchanced HTTP/2 extension support

ALTSVC frame is defined in RFC 7838. nghttp2’s ALTSVC frame support is very simple, and it does not parse Alt-Svc-Field-Value, and it just passes the received value to the application. This is because ALTSVC aware application most likely has a parser for this since ALTSVC can be sent via regular HTTP header field Alt-Svc.

By default, incoming ALTSVC frame is ignored. To receive ALTSVC frame, create nghttp2_option object, and use nghttp2_option_set_builtin_recv_extension_type(opt, NGHTTP2_ALTSVC), and pass the object to the nghttp2_session_client_new2() or nghttp2_session_client_new3(). Note that server always ignores ALTSVC regardless of the option settings.

To send ALTSVC frame, use nghttp2_submit_altsvc().

nghttp2 has introduced custom frame support in v1.8.0. So you can create your own ALTSVC handling if you wish.

Previously, we didn’t pass unknown incoming SETTINGS value to the application. Now those values are also passed to the application via nghttp2_on_frame_recv_callback.

nghttp : expect/continue exercises

Jacob Champion has implemented expect/continue dance with server in nghttp client. With new --expect-continue option, nghttp briefly waits for 100-continue from server before sending request body. HTTP/2 has flow control, and it is regarded as better feature for this, but this new option is useful to debug the server’s expect/continue feature since it is still supported by HTTP/2.

nghttpx: better load balancing, and bug fixes

Brian Suh fixed the bug that nghttpx returned 502 on FreeBSD (and possibly Mac OS X).

nghttpx now has the feature to detect the backend is online or offline. nghttpx’s --backend option now gets additional parameter rise=<N>, and fall=<N>. The fall=<N> specifies that if the backend cannot be connected in <N> times in a row, it is considered as offline, and excluded from load balancing group. The rise=<N> specifies that if nghttpx can connect to the offline backend in <N> times in a row, the backend is considered as online, and it is eligible for load balancing again. By default, these feature are disabled, and backend never be excluded from load balancing group.

Previously, when nghttpx could not connect to the one of backend, it did not use another backend, and just returns 5xx status code. Now it tries another backends in case of connect failure.

We fixed the bug that server push from mruby script did not work.

The server push using Link header field is extended, and now nghttpx can push resources using Link header field in non-final(1xx) response header fields from backend server.

It is not a secret that we used nginx as backend Web server for nghttp2.org. We use nghttpx as frontend “Edge” reverse proxy, and all requests except for the ones to /httpbin were routed to nginx via HTTP/2 protocol.

Today, we have replaced backend nginx with h2o. h2o is a relatively new brazing fast Web server. Actually, we don’t require performance in our low traffic Web site. The reason we chose h2o is its better HTTP/2 implementation. Since we use HTTP/2 as backend protocol between proxy and backend server, we’d like to use better backend HTTP/2 server. h2o supports both HTTP/1.1 as well as HTTP/2 without TLS by default without extra configuration. Currently, HTTP/2 priority in the frontend proxy connection is not propagated to the backend connection, so we couldn’t utilize the great scheduling mechanism built in h2o. When we implement it, we definitely want clever HTTP/2 scheduler like h2o.

We still use nghttpx as frontend reverse proxy.

Previously, with nginx, we added link header field in nginx configuration so that nghttpx could push resource based on it. Now we initiate push using nghttpx’s mruby scripting feature. The following script initiates the push for the resource “/stylesheet/screen.css” when request path is either “/” or “/index.html”:

1
2
3
4
5
6
7
8
9
10
11

class App
  def on_req(env)
    req = env.req
    path = req.path
    if path == '/' || path == '/index.html'
      req.push '/stylesheets/screen.css'
    end
  end
end

App.new

And pass the file path containing the above script to nghttpx using --mruby-file option.