We released nghttp2 v1.7.0.

This release contains the more strict error handling in libnghttp2 code to provide more diagnostics to applications. We made many improvements to h2load, and nghttpx in this release.

For libnghttp2, we now validate important header fields, :authority, host, and :scheme, strictly by checking the characters used in them. These header fields are usually used by server as routing purpose, and they may appear in different header fields when forwarded. It is better to basic error handling here so that we can reduce possible attack vectors.

Previously, nghttp2_submit_headers and nghttp2_submit_request functions did not return error when self dependency was made. Now it returns error code when such argument is passed.

Previously, when linking OpenSSL library statically, build was failed because -ldl was not passed to linker, so we had to add it using LIBS. Now it is automatically added to linker options.

In libnghttp2_asio library, nghttp2::asio_http2::server::http2::get_io_services() has been renamed as nghttp2::asio_http2::server::http2::io_services() to follow nameing convention. Previously, calling nghttp2::asio_http2::server::stop() did not stop acceptor. Now it stops acceptor too.

h2load now support UNIX domain socket to connect to server. Previously, -m option of h2load was auto, which defaulted to the number of URIs passed in command-line. Now it is removed, and the default value is 1. This is because -m option specifies the number of pipelined requests in HTTP/1, and disabling HTTP pipelining more reflects the practical usage of HTTP/1. The statistics calculation of h2load is now sampling based, instead of taking into account of all requets/clients to reduce memory consumption.

nghttpd now gets --no-content-length option to omit content-length in response.

nghttpx now interleaves pushed streams with the associated stream if pushed streams are JavaScript or CSS resources, which is determined by content-type response header field. The initial value of request/response buffer size (--backend-request-buffer and --backend-response-buffer options) is now increased to 128K. We have fixed the bug that --listener-disable-timeout option is not used. Now nghttpx does not emit :authority if request does not contain authority information. This happens when :authority header field is missing in HTTP/2 request, and origin or asterisk form is used in HTTP/1.1 request. nghttpx now supports RFC 7239 Forwarded header field. By default, Forwarded header field is not added. When it is instructed to be added, nghttpx emits obfuscated strings for by and for parameters. Vernon Tang fixed the bug which causes crash at start up when running on IPv6 only. Now nghttpx does not apply --max-header-fields and --header-field-buffer to response header fields, since they are meant to be applied to request header fields only. ayanamist fixed the bug that nghttpx wrongly lower-cased header field value supplied via --add-request-header-field and add-response-header-field options.

We released nghttp2 v1.6.0.

This release fixes heap-use-after-free bug in idle stream handling code. We strongly recommend to upgrade the older installation to this latest version as soon as possible. Other than that we have minor polish up in libnghttp2 code base, and some new features to asio library, and h2load.

We tightened up the error condition when we received frame which is not allowed in that state. We now make it connection error if we are sure that peer really violates the protocol.

Previously, the default remote SETTINGS_MAX_CONCURRENT_STREAMS value was unlimited as per RFC 7540. But this was problematic, and this could lead to many REFUSED_STREAM, or ENHANCE_YOUR_CALM error. To avoid this situation, and make it more friendly to peer, we assume that the default remote SETTINGS_MAX_CONCURRENT_STREAMS is 100. When we get this SETTINGS value from peer, we will update it. The application can change this initial value using nghttp2_option_set_peer_max_concurrent_streams() API.

Previously, stream object for pushed stream was not created when nghttp2_submit_push_promise() returned. It was created after nghttp2_before_frame_send_callback was called. So application had to wait for its invocation when it wanted to use stream object. Now stream object is created right after successful nghttp2_submit_push_promise() call.

We added new API, nghttp2_session_create_idle_stream(). This function creates idle stream without telling it the remote endpoint.

Previously, if we wanted to build libnghttp2 only, we had to give --disable-app, --disable-examples, --disable-hpack-tools, and --disable-python-bindings options to configure script. We added --lib-only configure option as a short hand for these options.

The previous package build failed without OpenSSL 1.0.2. This was fixed in this release by the patch contributed by Sunpoet Po-Chuan Hsieh.

Andreas Pohl added several new server APIs. nghttp2::asio_http2::server::http2::get_io_services() returns all underlying io_service objects. nghttp2::asio_http2::server::request::remote_endpoint() returns the remote endpoint address.

We added configurable connect, and read timeout for asio client API. See nghttp2::asio_http2::client::session::connect_timeout(), and nghttp2::asio_http2::client::session::read_timeout() API. This involved backend incompatible change, and the application must be recompiled to use new asio library (this applies to asio library only, and libnghttp2 is fully backward compatible to 1.0.0).

Similarly, we added configurable TLS handshake, and read timeout for asio server API. See nghttp2::asio_http2::server::http2::tls_handshake_timeout(), and nghttp2::asio_http2::server::http2::read_timeout() API.

h2load now shows min, max, mean and sd (standard deviation) of request per second value calculated per connection. This is useful to see how performance differs between connections.

We released nghttp2 v1.5.0.

This release introduces new APIs, and fixes several bugs, and adds new features to bundled applications.

First, let’s talk about new APIs.

The new API nghttp2_session_change_stream_priority() changes the priority of given stream without sending PRIORITY frame. This is very useful for server to change priority of streams, especially for pushed streams. By default, pushed streams depend on the associated stream, but if we are pushing resources included in critical rendering path, it would be better to at least interleave them with the associated stream. In this case, using this new API is very handy.

The new API nghttp2_session_check_server_session() is very simple function, and returns nonzero if given HTTP/2 session is configured as server side.

The new API nghttp2_session_check_request_allowed() is for client to check that it can send request at this time. There are several reasons that request can no longer be issued (e.g., all available stream IDs have been spent; or GOAWAY has been received).

The new API nghttp2_session_upgrade2() deprecates existing nghttp2_session_upgrade(). nghttp2_session_upgrade() is called in both client and server when connection was upgraded to HTTP/2 from HTTP/1.1 via HTTP Upgrade mechanism. Originally reported in curl issue #521, we discovered that it has a design flaw. That is it does not tell the library the request method upgraded request used. By default, nghttp2 library checks HTTP messaging rules. One of them is that the length of response body matches the length included in content-length. In particular, when method is HEAD, in client side library checks that response body is really 0, no matter what the content-length header field contains. Suppose the first request with HTTP Upgrade has HEAD method, and it is upgraded. Since nghttp2_session_upgrade() cannot tell the library what method is, library attempts to check the response body length matches content-length header field. But this will fail since server returns 0 length body, and non-zero length header field most likely exists. This results in sending RST_STREAM from client side. The new nghttp2_session_upgrade2() has additional parameter to tell library the method is HEAD or not, and it solves this issue. All programs using nghttp2_session_upgrade() are advised to migrate to nghttp2_session_upgrade2() as soon as possible. To workaround the above problem in nghttp2_session_upgrade(), it now does not check response body length against content-length for the upgraded request, but this is just a gas stop measure.

The new flags NGHTTP2_NV_FLAG_NO_COPY_NAME and NGHTTP2_NV_FLAG_NO_COPY_VALUE have been introduced in this release. They are used to specify flags field of nghttp2_nv. Previously, when application passed header fields to functions like nghttp2_submit_request() or nghttp2_submit_response(), they are all copied. This is very convenient and quite safe, since caller does not have to worry about the memory leak or dangling pointers. But obviously this memory copy could add overhead especially if header fileds are long or many. The new flags can tell the library that it does not have to copy header name or value, thereby eliminates the overhead of copying. It is a responsibility of applications to retain the header fields until the library finished using them. For server application, usually response headers are retained until streams are closed, due to logging variables, so there is a good chance for them to optimize a bit and squeeze the more speed.

OK, that is all for new stuff for library API. Let’s see the bug fix in library. The documentation of nghttp2_session_find_stream() said that it returns root stream (stream ID is 0) if 0 is passed as stream ID, but previously it returned always NULL. Now it is corrected, and it returns root stream.

Kamil Dudka fixed borken linage with --disable-static in unit tests.

We added ALPN support for asio library.

For h2load, we made some bug fixes, and added new features. We fixed the bug which caused crash when dealing with “connection: close” from HTTP/1.1 server. Kit Chan fixed the bug that h2load goes into infinite loop when timing script file starts with 0.0 in first line. Previously, user cannot override user-agent header field with -H option, but now they can do that. We now show “space savings” to measure header compression efficiency. This is only useful for HTTP/2 or SPDY. We fixed bug that application protocol is not shown with OpenSSL < 1.0.2. We introduced --h1 option which is a short hand of --npn-list=http/1.1 and --no-tls-proto=http/1.1, and effectively forces http/1.1 usage for both http and https URI.

For nghttpx, we finally support server push from HTTP/2 backend. PUSH_PROMISE frame from backend is forwarded to frontend session, and delivered to client. We fixed the bug that causes connection failure with backend proxy URI. We now use the host name given in --backend-tls-sni-field to verify certificate host name. We now log :authority header field value as $http_host if available. This is useful since HTTP/2 request might not contain host header field.

For nghttpd, we fixed crash with CONNECT request. Previously, nghttpd does not set content-type header fields. In this release, nghttpd reads /etc/mime.types and set content-type header field depending on the extension of the requested resource.

For nghttp, request method is now record in har correctly. The HTTP method in HTTP Upgrade is now configurable using -H with :method.

Finally, we upgraded clang-format-3.6. We also upgraded mruby to 1.2.0.

I gave a talk at HTTP/2 + ATS meetup in Yahoo! Japan Tokyo office on November, 6, 2015. My talk was titled “Stream scheduling utilizing HTTP/2 priority”, which described detailed mechanism of stream scheduling algorithm implemented in nghttp2. As written in last slide, the original idea came from h2o project. I made some modifications to suit my taste.

We released nghttp2 v1.4.0.

This release includes number of fixes for libnghttp2. We briefly explain notable bug fixes here. Previously, libnghttp2 ignored CONTINUATION frames if preceding HEADERS frame contained padding. The appearance of CONTINUATION is rare these days, but padding is used in some services already, and we may see CONTINUATION somewhere too. The second and third bugs are SETTINGS and HPACK dynamic table size related bugs. The second bug is that previously libnghttp2 did not shrink to minimum size of requested dynamic table size contained in SETTINGS frame sent from local endpoint if it contains several SETTINGS_HEADER_TABLE_SIZE. Now it is corrected, and libnghttp2 shrinks to the minimum size. The third bug is that due to the ambiguous text in RFC 7540 and 7541, we interpreted that if receiver received SETTINGS containing SETTINGS_HEADER_TABLE_SIZE, it always has to send dynamic table size update in the next compressed header block. But it turns out that it is not the intention of the specification author. The intended behaviour is the receiver is required to send dynamic table size update only when it really changed maximum dynamic table size. Depending on the SETTINGS_HEADER_TABLE_SIZE and the current maximum dynamic table size, the table size may not change. The original bug report is GH-396.

To protect from suspicious peer, if libnghttp2 detected peer is flooding local endpoint with excessive SETTINGS and PING, nghttp2_session_mem_recv() and nghttp2_session_recv() now return fatal error code NGHTTP2_ERR_FLOODED.

Application now can return NGHTTP2_ERR_PAUSE from nghttp2_send_data_callback to indicate that application wants to make nghttp2_session_mem_send() or nghttp2_session_send() return immediately without processing next outgoing frames.

libnghttp2 now hides all internal private symbols using -fvisibility=hidden, if compiler supports it. This should not affect current applications since they should use public APIs only. It will increase performance, but the downside is that now static library is required to run unit tests. To offer the same functionality of python HPACK binding, deflatehd and inflatehd applications which previously used internal private functions, we added the following functions to public API: nghttp2_hd_deflate_get_num_table_entries(), nghttp2_hd_deflate_get_table_entry(), nghttp2_hd_deflate_get_dynamic_table_size(), nghttp2_hd_deflate_get_max_dynamic_table_size() and their nghttp2_hd_inflater counterparts. These are used to get header table entry using 1-based index, and current and maximum dynamic table size.

configuration summary now prints out more detailed information, including CFLAGS and LIBS for each detected third party library.

Applications under src directory can be now compiled with BoringSSL. Building programs under examples, including libevent-client and libevent-server, with BoringSSL is a bit tricky, since you have to build libevent with BoringSSL first. We haven’t verified that works.

nghttpx has new options, and some bug fixes. The bug that PUT method is replaced with POST in HTTP/1 frontend was fixed. Lucas Pardue added TLS dynamic record size behaviour command line options. --tls-dyn-rec-warmup-threshold option sets the number of bytes before switching to full TLS record size. --tls-dyn-rec-idle-timeout option specifies the idle time duration. When it is passed, TLS record size is back to 1300 bytes. Peeyush Aggarwal added --fastopen option to enable RFC 7413 (TCP Fast Open) for listening connections. We also added neverbleed support to nghttpx. It is disabled by default, and is enabled by --with-neverbleed configure option. mruby scripting API has changed along with the option name (now unified --mruby-file option), see the manual page for details. We changed default timeouts for read sockets to 1 minutes. Previous timeout is a bit too long.

h2load also gets new features. Lucas Pardue added option --rate-period for user-definable rate period. Previously, it was fixed to 1 second, but now user can specify any value, from subseconds period to more than 1 second. Previously, if HTTP/1 is used, h2load did not reconnect to the server if server closed connection gracefully with “Connection: close”. Apache web server does this actively after processing several number of requests. If that happened, h2load marked all remaining requests assigned to that connection failure. Now it is corrected, and h2load connects to the server again, and continues to load testing. We also enabled SSL/TLS session resumption in h2load client.

We simplified h2load rate mode in this release. In this change, we removed -C option. Instead, -c option is used to specify the number of connections to be made, and it is now required argument if more than 1 client are required (this is usually the case). The number of requests made per connection is calculated simply by -n / -c.

In h2load, we changed the handling of -n option when --timing-script-file is used. If -n is used with --timing-script, it specifies the number of requests each client will make rather than the total number of requests h2load will perform across clients. This handling applies to rate mode as well.

nghttp now allows multiple -c option occurrences, and takes min and last value, and sends both of them in SETTINGS to simulate multiple dynamic table size changes.

We released nghttp2 v1.3.4.

This release fixes improper signal handling in nghttpx.

Janusz Dziemidowicz improved nghttpx init script so that it now checks whether new process actually runs on upgrade.

We released nghttp2 v1.3.3.

In this release, we fixed the bug that handled padding in DATA frame wrongly. We added additional compiler warning flags in the hope that it will make library code more rigid. We have made an improvement in HPACK dynamic table search, and it is now faster than the previous release using the real data.

Lucas Pardue added HTTP/1.1 support to h2load. Now it can be used for HTTP/1.1 tests as well as HTTP/2. This makes comparison between HTTP/1.1 and HTTP/2 a lot easier.

nghttpx now consists of 2 processes. One process is called master process, and reads configuration, and setup listening ports. It spawns a process called worker process, which handles all networking operations from remote clients. The latter drops privileges if –user is specified. The master process does not drops privileges so that when new process using new binary is spawned from master process as hotswap, it can still read private keys which requires privileged user permission.

nghttpx now changes permission of UNIX domain socket file to the user given in --user.

We fixed the bug in nghttpx that PROXY protocol did not work if frontend is configured to use TLS.

We are excited about the announcement of neverbleed, privilege separation engine for OpenSSL / LibreSSL, from Kazuho Oku. We have local repository to integrate this work into nghttpx. Hopefully, we can include it in the next release.

We released nghttp2 v1.3.2.

In this release, we fixed assertion failure in nghttpx on premature connection termination during last phase of TLS handshake. In library code, when request HEADERS frame cannot be sent due to the too large header fields, we previously did not open stream, so on nghttp2_on_frame_not_send_callback, we could not retrieve per-stream user data using nghttp2_session_get_stream_user_data(). Now it first opens stream and then checks the size of headers, so nghttp2_session_get_stream_user_data() can work.

For nghttpx, we added x-http2-push header field for pushed resource. This practice is also done in h2o web server.

We released nghttp2 v1.3.1.

In this release, we fixed several bugs in library and added improvements to h2load and nghttpx.

We fixed rough edges of stream priority handling in nghttp2 library code. It was minor, but worth updating.

We shipped init script for nghttpx, but it missed --daemon option. So unless it is enabled in its configuration file, start/stop script hangs. This bug was fixed by Tomasz Buchert.

Lucas Pardue improved validation of timing script input in h2load.

Previously, in h2load, we measured TTFB (time to first byte) in first socket read, rather than first byte of response body. Now it is corrected.

We added PROXY protocol version 1 support to nghttpx. If --accept-proxy-protocol option is used, nghttpx expects PROXY protocol line on frontend connection. The obtained client address is used in X-Forwarded-For header field.

We extended push functionality in nghttpx, and it can now push resources specified in Link header fields which come from HTTP/2 backend. Also we now allow absolute URI in Link header fields.

We added mruby support in nghttpx. It is disabled by default. Use --with-mruby configure option to enable it. The nghttp2 archive includes mruby source code, so user doesn’t have to download it themselves. Please note that mruby package in Debian/Ubuntu does not work with nghttpx, since it does not enable C++ ABI mode. nghttpx runs given mruby scripts in 2 phases: request phase, and response phase. The request phase is when set of request header fields are received from client. The response phase is when set of response header fields are received from backend server. We have documented all mruby APIs. User can override request/response header fields, change request path, and even return custom response without forwarding request to backend or discarding response from backend. Checkout examples.

We released nghttp2 v1.3.0.

In this release, we added upper limit for the number of the streams pushed from server and not yet opened for client side session. Specifically, these streams are in “reserved (remote)” state as described in RFC 7540, Section 5.1.. The thing is RFC 7540 does not say anything about the upper limit of these streams. It says the number of maximum concurrent streams, but it only applies to the opened streams, not applied to streams in “reserved (remote)” state. From this release, nghttp2 library limits the number of these streams less than or equal to 200 by default. Applications can change this limit by using nghttp2_option_set_max_reserved_remote_streams() function. Those streams which exceed this limit will be automatically closed with RST_STREAM frame.

We added public APIs to expose dependency tree information. The nghttp2 library handles HTTP/2 dependency priority just fine, but there is a need to access to the dependency tree from applications. One of the use case is to schedule stream handling in threads. In the most of the cases, the number of available threads are smaller than incoming streams. Applications can get the dependency information from nghttp2 library, and schedule them in the order by the priority. nghttp2_session_find_stream() function will return stream object with the given stream ID. Applications can traverse dependency tree from this object. nghttp2_session_get_root_stream() will return the stream object which is the root of whole dependency tree. See nghttp2_stream_* functions to access the properties of stream object.

We rewrite priority tree handling. Now it is simpler and robust. We now use priority queue per stream, which contains the stream which has ready to send a frame, or one of its descendants have a frame to send. We maintain invariant that if a stream is queued, then its ancestors are also queued (except for root). When we re-schedule stream after transmission, we re-schedule all ancestors, so that streams on the other path can get a chance to send. The basic idea is the same with h2o’s scheduler, but there are differences in the details.

We found that openssl ocsp command exits withs status code 0 even if ocsp query or validation was failed. We added extra code to fetch-ocsp-response script to handle these situations.

Now nghttpx and nghttpd terminate HTTP/2 connection with GOAWAY of error code INADEQUATE_SECURITY if one of black listed cipher suites (see RFC 7540, Appendix A) was negotiated.

Tomasz Buchert improved handling of /dev/stdout and /dev/stderr for logging in nghttpx.

We added HEAD method support to nghttpd.

Lucas Pardue added Timing-script and base URI support to h2load. See #330 for more details.

Nora Shoemaker added timeout options to h2load. See #331 for more details.