Google announced gRPC back in February and everyone was excited about it. It is a RPC framework using new ProtocolBuffer3 based on HTTP/2 as transport. Because it is RPC framework, user should look into its API and ProtocolBuffer basics. But as HTTP/2 implementor, we are interested in how HTTP/2 is used under the hood. So we decided to proxy the communication between gRPC client and server with nghttpx proxy server and see what’s going on in transport level.

We used C++ client and server pulled from grpc-common. Build everything as instructed in README. We made one modification to the greeter_client.cc to change remote port number to 3000 to connect to nghttpx proxy. We used nghttpx --http2-bridge mode. The configuration was like this:

1
2
3
4
5
6

greeter_client --HTTP/2-- nghttpx --HTTP/2-- greeter_server
gRPC                 --http2-bridge          gRPC
                     -f127.0.0.1,3000
                     -b127.0.0.1,50051
                     --frontend-no-tls
                     --backend-no-tls

Then we observed the traffic between greeter_client and greeter_server.

greeter_client sent the following request header fields:

1
2
3
4
5
6

:method: POST
:scheme: http
:path: /helloworld.Greeter/SayHello
:authority: localhost:3000
te: trailers
content-type: application/grpc

This is the usual HTTP/2 request, but one thing to note is te: trailers. According to RFC 7230, section 4.3.:

The “TE” header field in a request indicates what transfer codings, besides chunked, the client is willing to accept in response, and whether or not the client is willing to accept trailer fields in a chunked transfer coding.

HTTP/2 disallows transfer-encoding stuff, but it only allows “trailers” in “TE” header field:

The presence of the keyword “trailers” indicates that the client is willing to accept trailer fields in a chunked transfer coding, as defined in Section 4.1.2, on behalf of itself and any downstream clients.

So in HTTP/1.1, trailer fields are only allowed in chunked transfer encoding. In HTTP/2, chunked transfer encoding is deprecated because it has more elegant framing and thanks to it, trailer fields are always allowed.

gRPC does not work without “TE” header field.

The request body was like this (in hexdump):

1
2

00000000  00 00 00 00 07 0a 05 77  6f 72 6c 64              |.......world|
0000000c

This would be data serialized by ProtocolBuffer. We don’t go down the detail in ProtocolBuffer here.

nghttpx forwarded this requests to greeter_server with additional header fields, like “Via” header field.

Upon receiving request, greeter_server processed the request and responded with the following response header fields:

1

:status: 200

Well, that’t the most simple HTTP/2 response, followed by response body (again, in hexdump):

1
2
3

00000000  00 00 00 00 0d 0a 0b 48  65 6c 6c 6f 20 77 6f 72  |.......Hello wor|
00000010  6c 64                                             |ld|
00000012

Then server sent trailer fields:

1
2

grpc-status: 0
grpc-message:

nghttpx forwarded this reponse to the greeter_client and it shows the result to the console:

1

Greeter received: Hello world

So in tranport wise, gRPC uses POST request and response uses trailer fields. This means that we can proxies the gRPC traffic using HTTP/1.1 path as well, since we can forward it in chunked transfer encoding. This just comes from our curiosity and we have no idea this is useful in practice. But this shows the great effort we put in HTTP/2 to interoperate with existing HTTP/1.1 infrastructure.

To prove this, we added another nghttpx to set up HTTP/1.1 path:

1
2
3
4
5
6

greeter_client --HTTP/2-- nghttpx[1] --HTTP/1-- nghttpx[2] --HTTP/2-- greeter_server
gRPC                  -f127.0.0.1,3000        --http2-bridge          gRPC
                      -b127.0.0.1,3001       -f127.0.0.1,3001
                      --frontend-no-tls       -b127.0.0.1,50051
                                              --frontend-no-tls
                                              --backend-no-tls

And this works! nghttpx[1] forwarded gRPC request in HTTP/1.1 chunked transfer-encoding:

1
2
3
4
5
6
7

POST /helloworld.Greeter/SayHello HTTP/1.1
Host: 127.0.0.1:3001
Te: trailers
Content-Type: application/grpc
Transfer-Encoding: chunked
X-Forwarded-Proto: http
Via: 2 nghttpx

In response chain, nghttpx[2] forwarded response header to nghttpx[1]:

1
2
3
4

HTTP/1.1 200 OK
Transfer-Encoding: chunked
Server: nghttpx nghttp2/0.7.9-DEV
Via: 2 nghttpx

… with trailer fields appended after the last chunk.