The ngtcp2 programmers’ guide for early adopters

This document is written for early adopters of ngtcp2 library. It describes a brief introduction of programming ngtcp2.

Prerequisites

Reading RFC 9000 and RFC 9001 helps you a lot to write QUIC application. They describes how TLS is integrated into QUIC and why the existing TLS stack cannot be used with QUIC.

QUIC requires the special interface from TLS stack, which is probably not available from most of the existing TLS stacks. As far as I know, the TLS stacks maintained by the active participants of QUIC working group only get this interface at the time of this writing. In order to build QUIC application you have to choose one of them. Here is the list of TLS stacks which are supposed to provide such interface and for which we provide crypto helper libraries:

Creating ngtcp2_conn object

ngtcp2_conn is the primary object to present a single QUIC connection. Use ngtcp2_conn_client_new() for client application, and ngtcp2_conn_server_new() for server.

They require ngtcp2_callbacks, ngtcp2_settings, and ngtcp2_transport_params objects.

The ngtcp2_callbacks contains the callback functions which ngtcp2_conn calls when a specific event happens, say, receiving stream data or stream is closed, etc. Some of the callback functions are optional. For client application, the following callback functions must be set:

For server application, the following callback functions must be set:

ngtcp2_crypto_* functions are a part of ngtcp2 crypto API which provides easy integration with the supported TLS backend. It vastly simplifies TLS integration and is strongly recommended.

ngtcp2_settings contains the settings for QUIC connection. All fields must be set. Application should call ngtcp2_settings_default() to set the default values. It would be very useful to enable debug logging by setting logging function to ngtcp2_settings.log_printf field. ngtcp2 library relies on the timestamp fed from application. The initial timestamp must be passed to ngtcp2_settings.initial_ts field in nanosecond resolution. ngtcp2 cares about the difference from that initial value. It could be any timestamp which increases monotonically, and actual value does not matter.

ngtcp2_transport_params contains QUIC transport parameters which is sent to a remote endpoint during handshake. All fields must be set. Application should call ngtcp2_transport_params_default() to set the default values.

Client application has to supply Connection IDs to ngtcp2_conn_client_new(). The dcid parameter is the destination connection ID (DCID), and which should be random byte string and at least 8 bytes long. The scid is the source connection ID (SCID) which identifies the client itself. The version parameter is the QUIC version to use. It should be NGTCP2_PROTO_VER_V1.

Similarly, server application has to supply these parameters to ngtcp2_conn_server_new(). But the dcid must be the same value which is received from client (which is client SCID). The scid is chosen by server. Don’t use DCID in client packet as server SCID. The version parameter is the QUIC version to use. It should be NGTCP2_PROTO_VER_V1.

A path is very important to QUIC connection. It is the pair of endpoints, local and remote. The path passed to ngtcp2_conn_client_new() and ngtcp2_conn_server_new() is a network path that handshake is performed. The path must not change during handshake. After handshake is confirmed, client can migrate to new path. An application must provide actual path to the API function to tell the library where a packet comes from. The “write” API function takes path parameter and fills it to which the packet should be sent.

TLS integration

Use of ngtcp2 crypto API is strongly recommended because it vastly simplifies the TLS integration.

The most of the TLS work is done by the callback functions passed to ngtcp2_callbacks object. There are some operations left to application in order to make TLS integration work. We have a set of helper functions to make it easier for applications to configure TLS stack object to work with QUIC and ngtcp2. They are specific to each supported TLS stack:

They make the minimal QUIC specific changes to TLS stack object. See the ngtcp2 crypto API header files for each supported TLS stack. In order to make these functions work, we require that a pointer to ngtcp2_crypto_conn_ref must be set as a user data in TLS stack object, and its ngtcp2_crypto_conn_ref.get_conn must point to a function which returns ngtcp2_conn of the underlying QUIC connection.

If you do not use the above helper functions, you need to generate and install keys to ngtcp2_conn, and pass handshake messages to ngtcp2_conn as well. When TLS stack generates new secrets, they have to be installed to ngtcp2_conn by calling ngtcp2_crypto_derive_and_install_rx_key() and ngtcp2_crypto_derive_and_install_tx_key(). When TLS stack generates new crypto data to send, they must be passed to ngtcp2_conn by calling ngtcp2_conn_submit_crypto_data().

When QUIC handshake is completed, ngtcp2_callbacks.handshake_completed callback function is called. The local and remote endpoint independently declare handshake completion. The endpoint has to confirm that the other endpoint also finished handshake. When the handshake is confirmed, client side ngtcp2_conn will call ngtcp2_callbacks.handshake_confirmed callback function. Server confirms handshake when it declares handshake completion, therefore, separate handshake confirmation callback is not called.

Read and write packets

ngtcp2_conn_read_pkt() processes the incoming QUIC packets. In order to write QUIC packets, call ngtcp2_conn_writev_stream() or ngtcp2_conn_write_pkt(). The destlen parameter must be at least the value returned from ngtcp2_conn_get_max_udp_payload_size().

In order to send stream data, the application has to first open a stream. Use ngtcp2_conn_open_bidi_stream() to open bidirectional stream. For unidirectional stream, call ngtcp2_conn_open_uni_stream(). Call ngtcp2_conn_writev_stream() to send stream data.

An application should pace sending packets. ngtcp2_conn_get_send_quantum() returns the number of bytes that can be sent without packet spacing. After one or more calls of ngtcp2_conn_writev_stream() (it can be called multiple times to fill the buffer sized up to ngtcp2_conn_get_send_quantum() bytes), call ngtcp2_conn_update_pkt_tx_time() to set the timer when the next packet should be sent. The timer is integrated into ngtcp2_conn_get_expiry().

Packet handling on server side

Any incoming UDP datagram should be first processed by ngtcp2_pkt_decode_version_cid(). It can handle Connection ID more than 20 bytes which is the maximum length defined in QUIC v1. If the function returns NGTCP2_ERR_VERSION_NEGOTIATION, server should send Version Negotiation packet. Use ngtcp2_pkt_write_version_negotiation() for this purpose. If ngtcp2_pkt_decode_version_cid() succeeds, then check whether the UDP datagram belongs to any existing connection by looking up connection tables by Destination Connection ID. If it belongs to an existing connection, pass the UDP datagram to ngtcp2_conn_read_pkt(). If it does not belong to any existing connection, it should be passed to ngtcp2_accept(). If it returns NGTCP2_ERR_RETRY, the server should send Retry packet (use ngtcp2_crypto_write_retry() to create Retry packet). If it returns an other negative error code, just drop the packet to the floor and take no action, or send Stateless Reset packet (use ngtcp2_pkt_write_stateless_reset() to create Stateless Reset packet). Otherwise, the UDP datagram is acceptable as a new connection. Create ngtcp2_conn object and pass the UDP datagram to ngtcp2_conn_read_pkt().

Dealing with early data

Client application has to load resumed TLS session. It also has to set the remembered transport parameters using ngtcp2_conn_set_early_remote_transport_params() function.

Other than that, there is no difference between early data and 1RTT data in terms of API usage.

If early data is rejected by a server, client must call ngtcp2_conn_early_data_rejected(). All connection states altered during early data transmission are undone. The library does not retransmit early data to server as 1RTT data. If an application wishes to resend data, it has to reopen streams and writes data again. See ngtcp2_conn_early_data_rejected().

Stream data ownership

Stream data passed to ngtcp2_conn must be held by application until ngtcp2_callbacks.acked_stream_data_offset callbacks is invoked, telling that the those data are acknowledged by the remote endpoint and no longer used by the library.

Timers

The library does not ask an operating system for any timestamp. Instead, an application has to supply timestamp to the library. The type of timestamp in ngtcp2 library is ngtcp2_tstamp which is nanosecond resolution. The library only cares the difference of timestamp, so it does not have to be a system clock. A monotonic clock should work better. It should be same clock passed to ngtcp2_settings.initial_ts. The duration in ngtcp2 library is ngtcp2_duration which is also nanosecond resolution.

ngtcp2_conn_get_expiry() tells an application when timer fires. When timer fires, call ngtcp2_conn_handle_expiry() and ngtcp2_conn_write_pkt() (or ngtcp2_conn_writev_stream()).

After calling these functions, new expiry will be set. The application should call ngtcp2_conn_get_expiry() to restart timer. If ngtcp2_conn_get_expiry() returned NGTCP2_ERR_IDLE_CLOSE, it means that an idle timer has expired for this particular connection. In this case, drop the connection without calling ngtcp2_conn_write_connection_close().

Connection migration

In QUIC, client application can migrate to a new local address. ngtcp2_conn_initiate_immediate_migration() migrates to a new local address without checking reachability. On the other hand, ngtcp2_conn_initiate_migration() migrates to a new local address after a new path is validated (thus reachability is established).

Closing connection abruptly

In order to close QUIC connection abruptly, call ngtcp2_conn_write_connection_close() and get a terminal packet. After the call, the connection enters the closing state.

The closing and draining state

After the successful call of ngtcp2_conn_write_connection_close(), the connection enters the closing state. When ngtcp2_conn_read_pkt() returns NGTCP2_ERR_DRAINING, the connection has entered the draining state. In these states, ngtcp2_conn_writev_stream() and ngtcp2_conn_read_pkt() return an error (either NGTCP2_ERR_CLOSING or NGTCP2_ERR_DRAINING depending on the state). ngtcp2_conn_write_connection_close() returns 0 in these states. If an application needs to send a packet containing CONNECTION_CLOSE frame in the closing state, resend the packet produced by the first call of ngtcp2_conn_write_connection_close(). Therefore, after a connection has entered one of these states, the application can discard ngtcp2_conn object. The closing and draining state should persist at least 3 times the current PTO.

Error handling in general

In general, when error is returned from the ngtcp2 library function, call ngtcp2_conn_write_connection_close() to get terminal packet. If the successful call of the function creates non-empty packet, the QUIC connection enters the closing state.

If NGTCP2_ERR_DROP_CONN is returned from ngtcp2_conn_read_pkt(), a connection should be dropped without calling ngtcp2_conn_write_connection_close(). Similarly, if NGTCP2_ERR_IDLE_CLOSE is returned from ngtcp2_conn_handle_expiry(), a connection should be dropped without calling ngtcp2_conn_write_connection_close(). If NGTCP2_ERR_DRAINING is returned from ngtcp2_conn_read_pkt(), a connection has entered the draining state, and no further packet transmission is allowed.

The following error codes must be considered as transitional, and application should keep connection alive:

Version negotiation

Version negotiation is configured with the following ngtcp2_settings fields:

client_chosen_version passed to ngtcp2_conn_client_new() also influence the version negotiation process.

By default, client sends client_chosen_version passed to ngtcp2_conn_client_new() in other_versions field of version_information QUIC transport parameter. That means there is no chance for server to select the other compatible version. Meanwhile, ngtcp2 supports QUIC v2 draft version (NGTCP2_PROTO_VER_V2_DRAFT). Including both NGTCP2_PROTO_VER_V1 and NGTCP2_PROTO_VER_V2_DRAFT in ngtcp2_settings.other_versions field allows server to choose NGTCP2_PROTO_VER_V2_DRAFT which is compatible to NGTCP2_PROTO_VER_V1.

By default, server sends NGTCP2_PROTO_VER_V1 in other_versions field of version_information QUIC transport parameter. Because there is no particular preferred versions specified, server will accept any supported version. In order to set the version preference, specify ngtcp2_settings.preferred_versions field. If it is specified, server sends them in other_versions field of version_information QUIC transport parameter unless ngtcp2_settings.other_versionslen is not zero. Specifying ngtcp2_settings.other_versions overrides the above mentioned default behavior. Even if there is no overlap between ngtcp2_settings.preferred_versions and other_versions field plus client_chosen_version from client, as long as client_chosen_version is supported by server, server accepts client_chosen_version.

If client receives Version Negotiation packet from server, ngtcp2_conn_read_pkt() returns NGTCP2_ERR_RECV_VERSION_NEGOTIATION. ngtcp2_callbacks.recv_version_negotiation is also invoked if set. It will provide the versions contained in the packet. Client then either gives up the connection attempt, or selects the version from Version Negotiation packet, and starts new connection attempt with that version. In the latter case, the initial version that used in the first connection attempt must be set to ngtcp2_settings.original_version. The client version preference that is used when selecting a version from Version Negotiation packet must be set to ngtcp2_settings.preferred_versions. ngtcp2_settings.other_versions must include the selected version. The selected version becomes client_chosen_version in the second connection attempt, and must be passed to ngtcp2_conn_client_new().

Server never know whether client reacted upon Version Negotiation packet or not, and there is no particular setup for server to make this incompatible version negotiation work.