We just released nghttp2 v0.7.1. The library documentation is still based on h2-14. nghttp, nghttpd, nghttpx and h2load additionally advertise h2-16 ALPN token to interop with the implementations which require h2-16.

This release fixes GOAWAY handling to conform to the HTTP/2 specification. TLS read/write error in nghttp, nghttpd, nghttpx and h2load has been fixed. We enabled TLS session ticket with automatic key rotation in nghttpx. We also add an option to specify key parameter from file (but without automatic rotation) to nghttpx to share same keys with multiple instances. For documentation wise, man pages are now now generated using Sphinx. We don’t use help2man anymore.

We just released nghttp2 v0.7.0. The library documentation is still based on h2-14. nghttp, nghttpd, nghttpx and h2load additionally advertise h2-16 ALPN token to interop with the implementations which require h2-16.

We added some new APIs. The custom memory allocator nghttp2_mem was introduced in this release. We have also added several initialization fucntions to use this allocator. The new nghttp2_session_set_next_stream_id() and nghttp2_session_get_next_stream_id() functions have been added. nghttp2_session_set_next_stream_id() sets next outgoing stream ID, while nghttp2_session_get_next_stream_id() returns the next outgoing stream ID. These fucntions are useful when streams are used as anchors in dependency tree and leave them idle.

Now nghttp, nghttpd, nghttpx and h2load applications use libev instead of libevent. Libevent is still required to build source files under examples directory.

nghttp has supported the feature to reading files from stdin to upload to a server for quite some time, but it only supported files and did not work with non-file (e.g., echo -n "foo" | nghttp -d- http://nghttp2.org/). Kazuho Oku contributed the patch to fix this issue.

nghttp2 v0.6.7 has been released. We still advertises h2-14 support in ALPN, but we have merged h2-16 features in this release. They are binary compatible, so we use h2-14, which is the interop draft version.

For library code, there are several changes. Due to the change made into h2-16, PRIORITY frame is now allowed in any state state. Also we refined GOAWAY handling by utilizing last-stream-id extensively. nghttp2_session_want_read() and nghttp2_session_want_write() reflect this change so as to make graceful shutdown easier.

nghttpx got some enhancement. It now has the feature to configure access logging format. In addition to the usual combined log fields, we have added additional custom fields. Some of the new custom fields were contributed by Lucas Pardue. The cipher suite requirement in nghttpx was now removed and it only requires TLSv1.2 for HTTP/2 connection due to the changes in h2-16. nghttpx also disabled spdy/3 and spdy/2 by default, since they are now being deprecated world wide.

nghttp got some interesting options. The most notable option is -r. If it is given with file name, nghttp saves HTTP transactions in well-known HAR format. It is expected to be used with -a option.

h2load finally got ALPN support.

For python bindings, client implementation was contributed by Kyle Schomp.

nghttp command-line client has some useful options to test your HTTP/2 server implementation. This blog post introduces such command-line options.

We recommend to enable -nv options for nghttp to see what is happening on HTTP/2 framing layer. It is very good source of information for debugging.

Flow control

To test whether server enfoces flow control window size advertised by client, use -W and/or -w option. -W option specifies connection level flow control window size. It specifies the number of bits rather than the size directly, so if you specify -W14, the window size becomes 16383 (= 2 ** 14 - 1). Similarly, -w option specifies stream level flow control window size. Since HTTP/2 does not have dedicated way to reduce connection level window size, we achieve this by not sending WINDOW_UPDATE frame to server to keep the available window size less than or equal to the specified size. Therefore, to test connection window size less than default value which is 65535, nghttp should request resource more than 65535 bytes, otherwise -W has no effect. If nghttp found any errors related to flow control, it emits FLOW_CONTROL_ERROR and exists nonzero.

Header compression and multiplexing

HTTP/2 uses header compression technique called HPACK. We know that, by iterop experience, single request/response is not enough to test HPACK. The problem is more likely happening after several request/response transactions. This is because HPACK uses stateful compression mechanism and it requires both server and client share same compression context. If they are out of sync, problem appears. So testing HPACK must be done with multiple requests and responses. This means that it also exercises multiplexing.

For this purpose, -a option is handy. If -a is used, nghttp parses downloaded HTML file and finds statically linked resources, such as CSS, Javascript and images. Then it requests them which are multiplexed in the same HTTP/2 session (same connection).

If nghttp client found an error while decoding header block, it emits COMPRESSION_ERROR and exists nonzero.

Uploading

To upload the content of a file, use -d option. By default, nghttp adds content-length header field when uploading data. Specifying --no-content-length option will omit content-length header field from request header.

Padding

-b option adds given number of bytes to HEADERS or DATA frame as padding. This will verify that whether server can correctly parse frames by taking into account the existence of padding bytes.

CONTINUATION frame

Using --continuation will send very large headers to server using CONTINUATION frame. The total header block size (HEADERS + CONTINUATION) is around 18K bytes.

Priority

Use -a option and see that how server reacts to priority hints given from client. If -a is used, nghttp parses downloaded HTML file and finds statically linked resources, such as CSS, Javascript and images. Then it requests them which are multiplexed in the same HTTP/2 session (same connection). By default, nghttp prioritizes resources as follows:

1. HTML (highest)
2. CSS (high)
3. Javascripts (JS) (middle)
4. images (low)

Then it creates dependency tree and issues requests with this dependency hints. To verify how server reacts to this hints, -r option is very useful. It saves the HTTP transactions to given file as HAR format. Then you can use HAR file viewer to see the load times. To compare the result without prority, issue the same request with --no-dep option, which entirely omits priority hints.

I blogged about how HTTP/2 dependency based prioritization works in several month ago. The prioritization works fine even at that time, but there is one problem. It is a bit hard to track down and check that dependency is actually working from nghttp client logging. On the other hand, we also want a tool to measure how well server reacts to the priority hints client offers.

As a first step to tackle these problem, I added a feature to nghttp client program to output HTTP/2 transaction in well-known HAR format. We can use existing HAR file viewer to visualize the transactions and easily see how well server uses/obeys priority.

Let’s see how dependency information affects the resource transfer. Currently, nghttp prioritizes resources in 4 categories:

1. HTML (highest)
2. CSS (high)
3. Javascripts (JS) (middle)
4. images (low)

nghttp requests single HTML file and parses it while download HTML and issues additional HTTP/2 requests to server to download assets (e.g., CSS, JS and images) linked from the HTML. It does not support dynamic content yet.

I used excellent HAR file viewer in this experiment. The web content used in this experiment are taken from http2rulez, and I use nghttpx proxy server to process prioritization and content transfer scheduling.

Here is the result without priority information:

CSS and JS are marked as shared orange and red respectively. As you can see, everything are interleaved and due to several image transfer, some of CSS and JS load times are quite slow. bootstrap.css was loaded in 417.3ms and jquery-ui.min.js was 468.9ms. Most images were loaded before them.

Let’s see how dependency based priority makes difference here. Here is the result with priority information:

The above picture clearly shows CSS and JS were loaded much faster than the version without priority. bootstrap.css, the 129.4KB largest CSS resource, was loaded in 62.6ms while jquery-ui.min.js was in 159ms. All CSS and JS are prioritized over images and this is exactly what the client asked to the server.

We released new nghttp2 v0.6.6. The supported HTTP/2 protocol draft version is still h2-14.

This release fixes heap-use-after free bug in priority handling.

We also fixed several bugs in priority handling code as well. The notable one is that priority is ignored when a stream that is designated as dependency parent is absent. Now it is corrected and the default priority is assigned for that case.

nghttp commnad-line client now accepts multiple -v options. Specifying -v options more than once increases verbosity.

nghttpx’s log level now accepts NOTICE level, which is now default. This feature was contributed by Lucas Pardue. The help message of -L option in nghttpx wrongly stated log level WARNING, but it is now corrected to WARN. nghttpx also implements dynamic TLS record size.

h2load now prints SSL/TLS cipher name and parameters when SSL/TLS connection is used.

We released new nghttp2 v0.6.5. While we merged new draft-15 features, it is compatible with draft-14 framing layer. So this release still announces h2-14 as ALPN identifier.

This release fixes heap-use-after free bug in nghttpx. The bug was found by scan-build tool.

h2load, the benchmark tool for HTTP/2 and SPDY, now has --input-file and --header options. By default, h2load takes URIs in command-line. Instead, user can store URIs in a file and give it with --input-file option to h2load. Then h2load reads URIs from that file. This is convenient if lots of URIs are needed. --header option is add additional request headers or replace pre-defined headers (e.g., :scheme). These features were contributed by Kenny (kang-yen) Peng.

nghttpx now has --strip-incoming-x-forwarded-for option. If given, it strips incoming X-Forwarded-For request header. This feature was contributed by Lucas Pardue.

nghttp, nghttpd and nghttpx now disable SSLv3 completely.

nghttpx now emits alt-svc draft-04 compliant Alt-Svc header field.

We released bug fix release of nghttp2 v0.6.4. It still implements h2-14 and HPACK-09.

This release fixes the application (nghttpd, nghttpx and h2load) crash with libc++ when threading is enabled. This crash was caused because of heap-use-after-free of std::mutex object. Interestingly, libstdc++ did not exhibit this issue and clang address sanitizer also did not report anything. Hopefully, this release fixes the reported crash in Mac OS X (since it is likely to use libc++), and Mac OS X users finally get multi-threaded nghttp2 applications. So try building nghttp2 without –disable-threads and run one of applications and see they do not crash.

We happily announce yet another release of nghttp2 v0.6.3. It still implements h2-14 and HPACK-09.

This release addresses portability/compatibility issues reported by package maintainer and users. One of the problem was caused by use of std::future. Some platform lacks std::future, which causes compile error on application under src directoy. In this release, we check that whether std::future is available or not and if not, define NOTHREADS and exclude multi threading code entirely from applications.

Surprisingly, nghttp2 did not check first SETTINGS (a part of conneciton preface) strictly. From this release, nghttp2 library code checks whether SETTINGS is received as a first frame and if other frame type is received, treats it as PROTOCOL_ERROR.

The library code itself was optimized a bit more and improved its efficiency.

We happily announce immediate availability of nghttp2 v0.6.2. It still implements h2-14 and HPACK-09. This release fixes memory leaks. We added nghttp2_option_set_recv_client_preface() function. Previously, nghttp2 library only handles HTTP/2 frames and does not recognize first 24 bytes of client connection preface. This design choice is done due to the fact that server may want to detect the application protocol based on first few bytes on clear text communication. But for simple servers which only speak HTTP/2, it is easier for developers if nghttp2 library takes care of client connection preface as well. If this option is enabled, nghttp2 library checks first 24 bytes client connection preface. If it is not a valid one, nghttp2_session_recv() and nghttp2_session_mem_recv() will return new error code NGHTTP2_ERR_BAD_PREFACE, which is fatal error. By default, the option is turned off for backward compatibility.

The other thing worth mentioning is that now nghttp2_stream_resume_deferred_data() does not fail if data is deferred by flow control.

Previously processing incoming connection level WINDOW_UPDATE frame is quite expensive, but it is fixed and now 2x faster than v0.6.1.

We introduced the experimental libnghttp2_asio C++ library. It is built on top of libnghttp2 and intended to provide higher level APIs to build HTTP/2 client/server easily. Currently, only server APIs are provided. It depends on Boost::Asio and Boost::Thread libraries. For example, the rather useless simple HTTP/2 server which returns “hello, world” for any HTTP request is as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

#include <nghttp2/asio_http2.h>

using namespace nghttp2::asio_http2;
using namespace nghttp2::asio_http2::server;

int main(int argc, char **argv)
{
  http2 server;

  server.listen
    ("127.0.0.1", 3000,
     [](std::shared_ptr<request> req, std::shared_ptr<response> res)
     {
       res->write_head(200);
       res->end("hello, world");
     });
}

For more details, consult library documentation.

We added examples/tiny-nghttpd server, which is stripped faster version of nghttpd. Its purpose is measure performance bottleneck in libnghttp2 code, avoding any overhead in external I/O library. Currently it requires epoll, so it can be built only on linux.