nghttp2.org

HTTP/2 C library and tools

Nghttp2 v1.21.0

We have released nghttp2 v1.21.0.

libnghttp2

The bug that nghttp2_session_want_write may return 0 if there is pending frames after GOAWAY frame is submitted has been fixed.

build

_U_ macro has been eliminated in favor of old school (void)VAR for better compiler compatibility.

libnghttp2_asio

The asio client now sends PING frame when it gets idle for 30 seconds.

src

Mozilla’s “Modern compatibility” ciphers are used by default.

nghttpx

The bug that -v option does not print out version number has been fixed.

The workaround of getaddrinfo failure with AI_ADDRCONFIG has been applied.

nghttpx now escapes certain characters in access log.

nghttpx now enables backend pattern matching with --http2-proxy option as well.

TLSv1.3 Endpoint Is Now Online

In order to contribute to the development of the TLSv1.3 protocol, we have enabled TLSv1.3 support in nghttpx, and made it publicly available at https://nghttp2.org:13443. It is currently draft-19.

nghttpx uses OpenSSL as an underlying TLS backend. OpenSSL community has done a great job, and been developing TLSv1.3 support (they are still WIP), and it is available in OpenSSL master branch. So just building nghttpx with this bleeding edge version of OpenSSL could enable TLSv1.3 support. But we have taken one step further, and implemented 0-RTT early data support using new SSL_read_early_data API.

For those of you to build OpenSSL from their git repository, in order to enable TLSv1.3 support, pass enable-tls1_3 option to Configure script.

In order to enable 0-RTT support in nghttpx, check out nghttpx: Enable TLSv1.3 0-RTT early data support. Please note that this branch may be rebased time to time.

To send 0-RTT early data with openssl s_client, first save a session, like so:

1
$ openssl s_client -connect nghttp2.org:13443 -sess_out session.dat

The session is saved in session.dat file. We use this file to resume the session. 0-RTT early data should be stored in file. We have the following file for this example:

1
2
3
4
5
$ cat http.txt
GET / HTTP/1.1
Host: nghttp2.org:13443

$ # Don't forget to the trailing empty line to terminate HTTP request!

Then run the following command to resume session, and send 0-RTT early data:

1
$ openssl s_client -connect nghttp2.org:13443 -sess_out session.dat -sess_in session.dat -early_data http.txt

If 0-RTT early data is sent, and accepted by the server, you will see the HTTP response header fields and body sent from the server.

Nghttp2 v1.20.0

We have released nghttp2 v1.20.0.

libnghttp2

Alexis La Goutte fixed the issue found by PVS Studio.

New API, nghttp2_option_set_no_closed_streams, has been added. By default, libnghttp2 retains closed streams as suggested by RFC 7540, Section 5.3.4. If this option is used, libnghttp2 discards closed streams from memory in order to save memory usage.

build

SPDY has been dropped by Chromium and Firefox, and the maintenance of SPDY related code becomes burden for us. To start deprecation process, now the detection for spdylay library, which does SPDY protocol handling, is disabled by default. For those of you who want to enable SPDY support, --with-spdylay option must be given to configure script.

Since systemd support is integrated into nghttpx (see nghttpx section below), --with-systemd option is added to configure script.

fuzz

nghttp2 project has been accepted by OSS-Fuzz project. The fuzz directory of nghttp2 project contains the fuzz target source code. It also contains test corpus files which were generated by capturing communications during h2spec tests, and accessed by nghttp client.

libnghttp2_asio

Amir Pakdel added an ability to shut down server gracefully.

clemahieu fixed a crash in client code.

clemahieu fixed the infinite loop bug in acceptor handler.

clemahieu fixed the iterator invalidation bug in server.

src

If nghttp2 is built with OpenSSL master branch or BoringSSL, the applications, nghttp, nghttpd, nghttpx, and h2load, enable TLSv1.3 by default. Note that TLSv1.3 is not finalized yet, and TLSv1.3 support in OpenSSL is still WIP.

nghttpx

The server version number is now stripped from Server header field.

Previously, nghttpx will use only one single thread inside the worker process if --workers=1 (this is default). If --workers=N, N > 1, we use additional threads for accepting connections, or API request processing, etc. Now we use the same processing model for N > 1 even if N == 1. To restore the original single thread execution mode, --single-worker option is added.

We fixed the bug that API and mruby request did not participate graceful shutdown.

--frontend-max-requests option has been added to limit the number of requests per connection. For HTTP/1.1, this limits the nubmer of keep alive requests per single connection.

This release added configuration revision, which is considered opaque string, and changes after reloading configuration with SIGHUP. This revision is returned as a response to configrevision API endpoint. This allows external application to know whether nghttpx has finished reloading new configuration or not. Note that this revision does not change on backendconfig API calls.

redirect-if-not-tls parameter has been added to --backend option. nghttpx now responds to the request with 308 status code to redirect the request to https URI if frontend connection is not TLS encrypted, and redirect-if-no-tls parameter is used in --backend option. The port number in Location header field is 443 by default (thus omitted), but it can be configurable using --redirect-https-port option.

--tls-proto-list option has been deprecated, and instead, these 2 new options have been added: --tls-min-proto-version and --tls-max-proto-version to specify minimum and maximum TLS protocol version respectively. Versions between the two are enabled. The deprecated --tls-proto-list has empty default value, and acts like enabling only specific protocol versions in the range for now.

Previously, after sending SIGUSR2 to the original master process, and the new master process gets ready, user has to send SIGQUIT to the original master process to shut it down gracefully. With this release, the new master process automatically sends SIGQUIT to the original master process when it is ready to serve requests, eliminating for user to send SIGQUIT manually.

Tomasz Torcz added systemd support to nghttpx. Type=notify can be used in unit file, and it will send new master process PID to systemd around fork.

This release fixes the bug that nghttpx crashes on SIGHUP with multi thread configuration.

Nghttpx::Response#send_info method has been added to mruby scripting. When used, it sends 1xx non-final (informational) response.

nghttpx has supported multiple certificates using --subcert option. Previously, SNI hostname is used to select certificate. With this commit, signature algorithm presented by client is also taken into consideration. nghttpx now accepts certificates which share the same hostname (CN, SAN), but have different signature algorithm (e.g., ECDSA+SHA256, RSA+SHA256).

Now POST method is recommend for backendconfig API request.

Bernard Spil disabled PSK feature when nghttp2 is built with LibreSSL which has removed PSK.

nghttp

Christoph Wolters added support for link rel=“preload” for --get-assets.

h2load

There was a bug in the code to calculate statistics. This bug was revealed when some connections were closed due to an error. It has been fixed in this release.

Nghttp2 v1.19.0

We have released nghttp2 v1.19.0.

libnghttp2

We fixed memory leak bug which only occurs in server side session. Client side sessions are not affected. This bug was detected by LLVM libFuzzer with HTTP/2 corpus that h2o project uses. Due to the bad code path which nullifies next pointers of linked list in a certain condition, nghttp2_stream object is not going to be freed. We highly encourage to upgrade the existing installation to this latest version.

Alexis La Goutte sent series of patches to fix several issues found by PVS studio.

doc

makovich sent a patch to state that building nghttp2 with jemalloc does not work on Alpine Linux due to its inability of replacing malloc.

nghttp

Benedikt Christoph Wolters sent a patch to take into account authority specified in -H option to find links in HTML page (--get-assets option). Now authority and scheme are considered to find these links.

nghttpx

accesslog-write-early option has been added. If it is used, access log is written when response header block is sent, rather than after request transaction finishes.

client-ciphers option has been added. Previously, ciphers option sets cipher list for both frontend and backend TLS connections. Now ciphers option only sets cipher list for frontend connections. The new client-ciphers option sets cipher list for backend connection.

Similarly, we added client-no-http2-cipher-black-list option to disable HTTP/2 cipher black list enforcement on backend connection. The exiting no-http2-cipher-black-list option disables HTTP/2 cipher black list on frontend connection.

We fix the bug that no-http2-cipher-black-list (which is now client-no-http2-cipher-black-list) does not work on backend HTTP/2 connections.

We added PSK cipher suite support to nghttpx. Read this article to know how to use PSK cipher suites.

Now accept-proxy-protocol option was deprecated. To accept PROXY protocol, use proxyproto keyword in frontend option.

Nghttp2 v1.18.1

We have released nghttp2 v1.18.1.

This release fixes several bugs in nghttpx proxy server. Since v1.18.0 release, dynamic DNS feature has been added to nghttpx. This release fixes these DNS related bugs. User reported that nghttpx exited with assertion error in libev code when DNS was enabled. After investigating it, it turned out that this bug had existed well before DNS was added, but enabling DNS helped to trigger the bug.

Nghttp2 v1.18.0

We have released nghttp2 v1.18.0. The changes are summarized below.

libnghttp2

Since the previous release, “Content-Length: 0” in 204 status response has been treated as error, as per RFC 7230. But it turned out that some widely used services send them. To workaround this issue, it is now allowed, but ignored. That is application never get “Content-Length” header field in 204 status response.

build

Because of dynamic DNS support for nghttpx, c-ares library is now required to build bundled applications.

examples

tiny-nghttpd has been removed. Nowadays nghttpd does the better job in this area.

nghttpx

nghttpx gets backend dynamic DNS support. Previously, backend host name is resolved at start up or configuration reloading, and nghttpx keeps using those addresses through out its entire session. Now with “dns” parameter in backend option, nghttpx resolves host name dynamically. For performance reasons, nghttpx caches the resolved addresses for configured period of time (see dns-cache-timeout option). By default, this feature is not used. To use this feature, add “dns” parameter to backend option:

1
backend=example.com,80;;dns

Previously, backend API request can only contain numeric addresses, but with “dns” parameter, it can contain non-numeric host name as well.

We reworked error log format. Now they are documented in nghttpx(1) manual page.

frontend-keep-alive-timeout option has been added to specify the period during which HTTP/1 keep alive connection stays open.

The bug that fetch-ocsp-response script cannot run with OpenSSL 1.1.0 has been fixed.

Nghttp2 v1.17.0

We have released nghttp2 v1.17.0. The changes are summarized below.

libnghttp2

In this release, libnghttp2 by default disallows content-length header field in 1xx, 204, or 200 to a CONNECT request as described in RFC 7230.

libnghttp2_asio

Previously, server-side on_close callback was not called when connection was closed while streams were still alive. Now on_close callback is called for active streams on connection close.

build

Remo E provided a patch to include MSVC version resource in cmake Windows build.

nghttpx

We fixed the bug that sometimes made nghttpx crash if --backend-http-proxy-uri was used.

We fixed the bug that one HTTP header fields from HTTP/1.1 backend were split into multiple fields in some situations.

We fixed the bug that zero-length POST was not forwarded to HTTP/1.1 backend, causing dead lock.

We removed optional reason phrase from SPDY response header fields. This is OK since reason phrase is optional.

To align the changes made in libnghttp2 that disallows content-length in 1xx, 204, or 200 to a CONNECT request, we did the same thing to HTTP/1.1 backend. We also disallow transfer-encoding in those status codes as well.

dalf provided a patch to fix compile failure with BoringSSL.

nghttpd, nghttpx, and libnghttp2_asio

We fixed the bug that mandatory SP after status code wass missing in HTTP/1.1 status line.

Nghttp2 v1.16.1

We released nghttp2 v1.16.1.

We fixed the bug that nghttp2 HPACK decoder may decode wrong integer because of undefined behaviour.

We fixed the bug in nghttpx that may make nghttpx crash if final response after non-final response from origin server is forwarded to HTTP/1.1 client.

Nghttp2 v1.16.0

We released nghttp2 v1.16.0. We summarizes the changes below.

libnghttp2

Previously, if libnghttp2 is built with DEBUGBUILD macro defined, it prints out debug messages into stderr. In this release, Anders Bakken added nghttp2_set_debug_vprintf_callback() function to set a callback which can customize how debug message is processed. The parameters passed to the callback are suitable for use with vfprintf(3) function.

libnghttp2_asio

We fixed the bug which causes crash if nghttp2::asio_http2::server::response::end() is called from outside nghttp2 callback (e.g., asynchronous timer callback).

nghttpx

We have added --backend-connect-timeout option to specify how long nghttpx waits until backend TCP connection is established.

The new option --ecdh-curves lets you specify the list of named curve for use in TLS.

We have added TLS signed_certificate_timestamp extension support. signed_certificate_timestamp extension is defined in RFC 6962. The new option --tls-sct-dir is used to specify the directory which contains *.sct files. These files are read in start up, and sent to client in TLS handshake. The format of *.sct files is the same as the one that nginx and Apache mod_ssl_ct use. For additional certificates specified by --subcert option, we extended the syntax of the option, and now it can take sct-dir parameter which takes the directory that should contain *.sct files for the certificate.

h2load

We have added --header-table-size and --encoder-header-table-size options to specify HPACK header table size for both direction.

Nghttp2 v1.15.0

We released nghttp2 v1.15.0. We summarizes the changes below.

libnghttp2

Previously, the maximum size of dynamic header table size used by HPACK encoder was limited to 4KiB regardless of SETTINGS_HEADER_TABLE_SIZE sent by peer. In this release, we added nghttp2_option_set_max_deflate_dynamic_table_size() to change the maximum value of encoder’s maximum dynamic header table size. With this option, nghttp2 based client/server can experiment the larger or smaller dynamic table size.

Previously, we could not return successfully from nghttp2_data_source_read_callback without reading anything or NGHTTP2_ERR_DEFERRED return value. The latter requires nghttp2_session_resume_data(), and is not a good workaround. In this release, application can now return NGHTTP2_ERR_CANCEL from nghttp2_data_source_read_callback without reading anything, and it signals the libnghttp2 to return to the application code immediately.

To offer the opportunity to implement https://tools.ietf.org/html/draft-benfield-http2-debug-state-01 to the nghttp2 based servers, we added API functions to export internal HTTP/2 state data from nghttp2_session object. In this release, we export the data marked as “required” in the draft. Here is the list of the added functions:

  • nghttp2_session_get_hd_deflate_dynamic_table_size() which returns the dynamic table size of HPACK encoder

  • nghttp2_session_get_hd_inflate_dynamic_table_size() which returns the dynamic table size of HPACK decoder

  • nghttp2_session_get_local_settings() which returns local HTTP/2 SETTINGS in effect; this is the SETTINGS sent from the local endpoint to the remote one

  • nghttp2_session_get_local_window_size() which returns the connection window size

  • nghttp2_session_get_stream_local_window_size() which returns the stream window size for given stream

Third-Party

We have updated neverbleed, and it now supports ECDSA certificate.

src

Now applications under src directory compiles with OpenSSL 1.1.0.

nghttpx

To utilize the new feature to change HPACK encoder’s dynamic table size described above, we added new options to achieve this. The new options are:

  • --frontend-http2-encoder-dynamic-table-size
  • --frontend-http2-decoder-dynamic-table-size
  • --backend-http2-encoder-dynamic-table-size
  • --backend-http2-decoder-dynamic-table-size

These options default to 4KiB.

We have added tls_sni to mruby Nghttpx::Env class, which returns the server name sent in TLS SNI from client.

Previously, we have --frontend-http2-window-bits and its family options. They were not flexible because they only accept number of bits. Now they have been deprecated, and instead we have introduced --frontend-http2-window-size and its family options, which take the size in integer, rather than bits. The deprecated options still work, and are translated into the new options, but we encourage users to update configuration to use new options.

We have implemented TCP write buffer optimization presented by Kazuho’s slide. In short, this optimization limits the number of bytes to write to TCP socket based on the TCP CWND, and just write the bytes which can be sent in 1 RTT. This avoids excessive commitment of low prioritized data to the TCP socket, and implementation can quickly respond to the high prioritized data. This optimization is experimental, and enabled by --frontend-http2-optimize-write-buffer-size, and only works with HTTP/2 TLS connections. At the moment, only Linux is supported.

We also added HTTP/2 window size auto tuning optimization. It adjusts connection window size of frontend HTTP/2 connection based on RWIN. This is highly experimental, and may not work as expected. This feature is experimental, enabled by --frontend-http2-optimize-window-size, and only works with HTTP/2 TLS connections. At the moment, only Linux is supported. In the future release, we may drop the requirement of TLS for this optimization.

We added workaround for std::make_shared bug in Xcode7, 7.1, and 7.2 to prevent nghttpx from crashing.

We fixed the bug that bytes are doubly counted towards rate limit for TLS connections.

Previously, with default mode, server header field was rewritten to “nghttpx” and its version. Now --no-server-rewrite option disables this, and just forwards the server header field from the backend. We have added --server-name option to specify the server header field value. If both options are present, --no-server-rewrite takes precedence.

Previously, we ignored invalid header field coming from HTTP/2. Now they are treated as stream error.

nghttp and nghttpd

We have added --encoder-header-table-size option to specify the HPACK encoder’s maximum dynamic header table size.

Python

We have added ALPN support, and now requires at least Python 3.5.