We released nghttp2 v0.7.10.

This release adds new APIs. We added nghttp2_session_consume_connection and nghttp2_session_consume_stream functions. The existing nghttp2_session_consume function affects both connection and stream level flow control window. The new nghttp2_session_consume_connection only affects connection level flow control window and nghttp2_session_consume_stream only affects stream level.

We also added nghttp2_send_data_callback to avoid data copy when sending request/response body. We saw meaningful performance improvement in nghttpd and tiny-nghttpd server using this new feature.

To make building libnghttp2 under Windows easier, we added NGHTTP2_EXTERN macro to export public functions in Windows build. lib/Makefile.msvc is also updated by Remo E.

For documentation, now .rst file is generated per API function, which we hope makes the documentation more readable and easy to navigate.

We fixed incorrect header field ordering bug in Python binding. We also added async request/response body generation.

nghttpx proxy server gets OCSP stapling support. We use external Perl script developed under h2o project, to update OCSP response. --tls-ctx-per-worker option was removed since it did not show any performance improvement and just complicated code base. To make debugging easier, now stderr is redirected to errorlog file.

We released nghttp2 v0.7.9.

This release fixes corrupted header fields in PUSH_PROMISE issued by nghttpx. The bug that h2load crashes if the number of thread given in -t is greater than the number of client given in -c has also been fixed. For examples, the bug that system header location has precedence over in-package header files has been fixed.

nghttp and nghttpd get new option --hexdump, which hexdumps incoming traffic, similar format as hexdump -C. It could be useful for further debugging.

h2load gets -d option to upload data to server. With this option, h2load now can load test gRPC server (probably unary RPC only?). For example, suppose we are going to load test gRPC greeter_server contained in grpc-common/cpp/helloworld. We have to add following 2 header fields to mimic gRPC request: “te: trailers” and “content-type: application/grpc”. We captured gRPC request body from greeter_client and wrote it in file called grpc-upload. For this particular request body, you can create this file using following python script:

1
2

d='\x00\x00\x00\x00\x07\x0a\x05\x77\x6f\x72\x6c\x64'
with open('grpc-upload', 'w') as f: f.write(d)

The final command-line code is something like this:

1
2
3
4
5

$ h2load http://localhost:50051/helloworld.Greeter/SayHello \
      -d grpc-upload \
      -H 'te: trailers'
      -H 'content-type: application/grpc'
      -n1000000 -c100 -m100

Google announced gRPC back in February and everyone was excited about it. It is a RPC framework using new ProtocolBuffer3 based on HTTP/2 as transport. Because it is RPC framework, user should look into its API and ProtocolBuffer basics. But as HTTP/2 implementor, we are interested in how HTTP/2 is used under the hood. So we decided to proxy the communication between gRPC client and server with nghttpx proxy server and see what’s going on in transport level.

We used C++ client and server pulled from grpc-common. Build everything as instructed in README. We made one modification to the greeter_client.cc to change remote port number to 3000 to connect to nghttpx proxy. We used nghttpx --http2-bridge mode. The configuration was like this:

1
2
3
4
5
6

greeter_client --HTTP/2-- nghttpx --HTTP/2-- greeter_server
gRPC                 --http2-bridge          gRPC
                     -f127.0.0.1,3000
                     -b127.0.0.1,50051
                     --frontend-no-tls
                     --backend-no-tls

Then we observed the traffic between greeter_client and greeter_server.

greeter_client sent the following request header fields:

1
2
3
4
5
6

:method: POST
:scheme: http
:path: /helloworld.Greeter/SayHello
:authority: localhost:3000
te: trailers
content-type: application/grpc

This is the usual HTTP/2 request, but one thing to note is te: trailers. According to RFC 7230, section 4.3.:

The “TE” header field in a request indicates what transfer codings, besides chunked, the client is willing to accept in response, and whether or not the client is willing to accept trailer fields in a chunked transfer coding.

HTTP/2 disallows transfer-encoding stuff, but it only allows “trailers” in “TE” header field:

The presence of the keyword “trailers” indicates that the client is willing to accept trailer fields in a chunked transfer coding, as defined in Section 4.1.2, on behalf of itself and any downstream clients.

So in HTTP/1.1, trailer fields are only allowed in chunked transfer encoding. In HTTP/2, chunked transfer encoding is deprecated because it has more elegant framing and thanks to it, trailer fields are always allowed.

gRPC does not work without “TE” header field.

The request body was like this (in hexdump):

1
2

00000000  00 00 00 00 07 0a 05 77  6f 72 6c 64              |.......world|
0000000c

This would be data serialized by ProtocolBuffer. We don’t go down the detail in ProtocolBuffer here.

nghttpx forwarded this requests to greeter_server with additional header fields, like “Via” header field.

Upon receiving request, greeter_server processed the request and responded with the following response header fields:

1

:status: 200

Well, that’t the most simple HTTP/2 response, followed by response body (again, in hexdump):

1
2
3

00000000  00 00 00 00 0d 0a 0b 48  65 6c 6c 6f 20 77 6f 72  |.......Hello wor|
00000010  6c 64                                             |ld|
00000012

Then server sent trailer fields:

1
2

grpc-status: 0
grpc-message:

nghttpx forwarded this reponse to the greeter_client and it shows the result to the console:

1

Greeter received: Hello world

So in tranport wise, gRPC uses POST request and response uses trailer fields. This means that we can proxies the gRPC traffic using HTTP/1.1 path as well, since we can forward it in chunked transfer encoding. This just comes from our curiosity and we have no idea this is useful in practice. But this shows the great effort we put in HTTP/2 to interoperate with existing HTTP/1.1 infrastructure.

To prove this, we added another nghttpx to set up HTTP/1.1 path:

1
2
3
4
5
6

greeter_client --HTTP/2-- nghttpx[1] --HTTP/1-- nghttpx[2] --HTTP/2-- greeter_server
gRPC                  -f127.0.0.1,3000        --http2-bridge          gRPC
                      -b127.0.0.1,3001       -f127.0.0.1,3001
                      --frontend-no-tls       -b127.0.0.1,50051
                                              --frontend-no-tls
                                              --backend-no-tls

And this works! nghttpx[1] forwarded gRPC request in HTTP/1.1 chunked transfer-encoding:

1
2
3
4
5
6
7

POST /helloworld.Greeter/SayHello HTTP/1.1
Host: 127.0.0.1:3001
Te: trailers
Content-Type: application/grpc
Transfer-Encoding: chunked
X-Forwarded-Proto: http
Via: 2 nghttpx

In response chain, nghttpx[2] forwarded response header to nghttpx[1]:

1
2
3
4

HTTP/1.1 200 OK
Transfer-Encoding: chunked
Server: nghttpx nghttp2/0.7.9-DEV
Via: 2 nghttpx

… with trailer fields appended after the last chunk.

We released nghttp2 v0.7.8.

This release fixes the h2load crash bug if multiple threads are used (-t option). The patch was contributed from Stefan Eissing.

For “http” or “https” URIs, nghttp2 library now validate “:path” pseudo header field strictly. Now it requires to start with “/”. The only exception is OPTIONS request and in that case “:path” pseudo header field can include “*” to represent system-wide OPTIONS request.

Now all header field name and value presented by callback (e.g., nghttp2_on_frame_recv_callback, nghttp2_on_header_callback, etc) are guaranteed to be NULL-terminated. This is useful because most C library string functions require NULL-terminated string.

We released nghttp2 v0.7.7.

This is basically the same as v0.7.6. We missed updating man pages in v0.7.6 release, so v0.7.7.

We released nghttp2 v0.7.6.

This release made several enhancements to the library. First we made sending trailer part (trailer header field) easier. Previously, nghttp2_submit_request() and nghttp2_submit_response() sets END_STREAM flag set for the last DATA frame and we have no way to send trailer part. To fix this issue, we added NGHTTP2_DATA_FLAG_NO_END_STREAM flag. If application sets this flag along side with NGHTTP2_DATA_FLAG_EOF in nghttp2_data_source_read_callback, the library does not set END_STREAM flag. Then the application can use new nghttp2_submit_trailer() function to send trailer part, which is a HEADERS frame with END_STREAM flag set. The all bundled applications and libnghttp2_asio utilize this feature to send trailer part.

Second enhancement is that now library refuses PUSH_PROMISE frame if unacked local SETTINGS includes ENABLE_PUSH == 0 (disabling server push). Previously the library refuses PUSH_PROMISE only after SETTINGS are acked.

nghttpx has several enhancements in this release. It now accepts multiple backend server addresses for HTTP/2 backend. Previously the number of HTTP/2 backend server is limited to 1 per worker. In this release, we added --backend-http2-connections-per-worker option to specify the number of HTTP/2 backend connection per worker. As the library gets enhanced support for trailer part, nghttpx now supports trailer part for both HTTP/2 and HTTP/1. That means nghttpx can proxy gRPC traffic, which requires trailer support. we also fixed the bug that server push is broken after HTTP upgrade. The another bug, which crashes nghttpx when upgrading HTTP/2 failed, was also fixed.

Stefan Eissing wrote a patch to nghttpx to replace C++11 thread_local keyword with traditional pthread functions. Since Mac OS X’s xcode toolchain does not support thread_local, it may be possible to use nghttpx in multi-threaded mode on Mac OS X finally.

For nghttpd and nghttp, we added --trailer option to send trailer part to exercise trailer part functionality.

For nghttp, Kazuho Oku kindly sent us a patch not to send pseudo-headers in HTTP Upgrade request (which itself is HTTP/1.1 message).

nghttp now treats request as success only if it sees the END_STREAM from peer. Previously it treated a request as success if it is closed (with/without error), before connection close.

We fixed nghttpd bug that “date” header field value is not updated.

For python binding, we fixed bug that push response header fields are not passed to callback in client library.

We rewrite libnghttp2_asio library and its API is not compatible to the earlier versions. Also we added client API. See libnghttp2_asio documentation about the example use of these APIs.

We released nghttp2 v0.7.5. This release still uses h2-14 identifier in library code, but h2-14 through to final draft version is binary compatible and nghttp2 supports all features in the latest draft.

In this release, we implemented validation against HTTP Messaging semantics described in HTTP/2 draft, section 8.. We do not perform all the checks described in that section, but this time we implemented basic and fundamental checks which makes application code much cleaner. This validation pretty much restricts nghttp2 library to HTTP/2 use only. For those applications to use nghttp2 as non-http use case, use nghttp2_option_set_no_http_messaging() to disable this feature. For more detailed information about this subject, see the manual.

Previously, in verbose output of nghttp, nghttpd and nghttpx, we show “noind=1” if header field is encoded in HPACK Never Indexed representation. Now it is shown as “sensitive” to make its meaning more clearer. Other implementations use the same wording.

nghttpd got new option -a, --address=<ADDR> that allows nghttpd to bind to a non-default address. This feature was contributed by Brian Card.

In the previous release, we updated nghttp --stat option. Now we make its terminology and processing model to match those in Resource Timing TR.

Let’s talk about nghttpx proxy server. It now supports UNIX domain socket for both frontend and backend. We omitted minor version in Via header field and access logging if HTTP version is HTTP/2. We fixed the bug that nghttpx crashes if it receives more bytes than Content-Length from HTTP/1 backend.

We fixed asio-sv2 so that it compiles under OS X.

And finally as we know HTTP/2 specification was approved by IETF, we now announce h2 ALPN ID in nghttp, nghttpd, nghttpx and h2load!

For future release plan, we continue to develop nghttp2 in 0.7 series until HTTP/2 RFC is published. After RFC publication we’ll finally release version 1.0.0.

We released nghttp2 v0.7.4. This release still uses h2-14 identifier in library code.

In this release, we fixed few corner case bugs in the library code.

We rewrite nghttp’s --stat option output completely. It now shows the timings of request and completion of each resources:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

$ nghttp -nas https://nghttp2.org
***** Statistics *****

Request timing:
  complete: relative time from protocol handshake to stream close
   request: relative   time  from   protocol   handshake  to   request
            transmission.  If '*' is shown, this was pushed by server.
   process: time for request and response
      code: HTTP status code
      size: number  of   bytes  received  as  response   body  without
            inflation.
       URI: request URI

sorted by 'complete'

complete  request    process  code size request path
 +12.76ms     +118us  12.65ms  200   9K /
 +20.75ms * +10.51ms  10.23ms  200   8K /stylesheets/screen.css
 +32.89ms   +12.86ms  20.03ms  200   3K /javascripts/octopress.js
 +33.12ms   +12.86ms  20.26ms  200   3K /javascripts/modernizr-2.0.js
+124.05ms   +12.87ms 111.18ms  200 171K /images/posts/with-pri-blog.png
+150.66ms   +12.86ms 137.80ms  200 174K /images/posts/without-pri-blog.png

The list is sorted by complete time, that is time point its stream was closed. Notice that request field of /stylesheets/screen.css has *, which means that it was pushed by server.

nghttpd has a bug that multiple -p option won’t work, but it is not fixed.

As we wrote in nghttp2.org enabled HTTP/2 server push, nghttpx gets HTTP/2 server push support by inspecting Link header field from backend server. nghttpx now rewrites Host and :authority header fields when forwarding to backend server by default. --no-host-rewrite option to disable it.

h2load now shows time for request min, max, mean and standard deviation:

1
2
3
4
5
6

finished in 7.10s, 14092 req/s, 55.67MB/s
requests: 100000 total, 100000 started, 100000 done, 100000 succeeded, 0 failed, 0 errored
status codes: 100000 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 414200800 bytes total, 2723100 bytes headers, 409600000 bytes data
                     min         max         mean         sd        +/- sd
time for request:   283.86ms       1.46s    659.70ms    150.87ms    84.68%

To make command-line operation easier, we added bash_completion files for nghttp, nghttpd, nghttpx and h2load. They reside in doc/bash_completion directory.

We implemented HTTP/2 server push in nghttpx and we enabled server push in nghttp2.org site.

When you access https://nghttp2.org via HTTP/2 protocol, CSS file /stylesheets/screen.css is pushed to client. If you inspect the response header closely, you will find Link header field like this:

1

link: </stylesheets/screen.css>; rel=preload; as=stylesheet

We operate nghttp2.org site using nghttpx proxy server in front and nginx on its back. The Link header field above is generated by nginx. When nghttpx proxy sees this in response header, it initiates server push for resource denoted by path enclosed by ‘<’ and ‘>’. We specifically check link relation preload to decide the resource should be pushed or not. See the manual page to enable this feature in nghttpx.

We also push different resources, CSS and font files, under https://nghttp2.org/documentation/

We released nghttp2 v0.7.3. v0.7.2 was replaced with v0.7.3 shortly after its release because it contains apparent bugs. The v0.7.3 release still uses h2-14 identifier in library code.

Since v0.7.3 is quick bug fix release, we summarize changes made in both v0.7.3 and v0.7.2. We added nghttp2_submit_shutdown_notice() API function. It is intended to be used by server and tells client that server has started graceful shutdown by sending GOAWAY with (1 << 31) - 1 as last-stream-ID. We also added nghttp2_session_get_last_proc_stream_id(), which returns the largest stream ID the library called nghttp2_on_frame_recv_callback() for and can be passed as last-stream-id paramter to nghttp2_submit_goaway() if application has no more specific knowledge about last-stream-ID.

nghttpx has several options added. They are mostly optimzation purpose. We tightened up HTTP header field validation in nghttpx, particularly for Content-Length header field.

To upgrade ease of use, options which take SIZE as metavar can use units (K, M and G, powers of 1024) along with leading digits. For example, now we can specify --read-rate=4K, which is identical to --read-rate=4096.

Since nghttpx has many features, we started to add integration testing framework using Go programming language and its go test feature. It resides in integration-tests directory.