Programmers' Guide¶
Architecture¶
The most notable point in nghttp2 library architecture is it does not perform any I/O. nghttp2 only performs HTTP/2 protocol stuff based on input byte strings. It will calls callback functions set by applications while processing input. The output of nghttp2 is just byte string. An application is responsible to send these output to the remote peer. The callback functions may be called while producing output.
Not doing I/O makes embedding nghttp2 library in the existing code base very easy. Usually, the existing applications have its own I/O event loops. It is very hard to use nghttp2 in that situation if nghttp2 does its own I/O. It also makes light weight language wrapper for nghttp2 easy with the same reason. The down side is that an application author has to write more code to write complete application using nghttp2. This is especially true for simple "toy" application. For the real applications, however, this is not the case. This is because you probably want to support HTTP/1 which nghttp2 does not provide, and to do that, you will need to write your own HTTP/1 stack or use existing third-party library, and bind them together with nghttp2 and I/O event loop. In this point, not performing I/O in nghttp2 has more point than doing it.
The primary object that an application uses is nghttp2_session
object, which is opaque struct and its details are hidden in order to
ensure the upgrading its internal architecture without breaking the
backward compatibility. An application can set callbacks to
nghttp2_session
object through the dedicated object and
functions, and it also interacts with it via many API function calls.
An application can create as many nghttp2_session
object as it
wants. But single nghttp2_session
object must be used by a
single thread at the same time. This is not so hard to enforce since
most event-based architecture applicatons use is single thread per
core, and handling one connection I/O is done by single thread.
To feed input to nghttp2_session
object, one can use
nghttp2_session_recv()
or nghttp2_session_mem_recv()
functions.
They behave similarly, and the difference is that
nghttp2_session_recv()
will use nghttp2_read_callback
to get
input. On the other hand, nghttp2_session_mem_recv()
will take
input as its parameter. If in doubt, use nghttp2_session_mem_recv()
since it is simpler, and could be faster since it avoids calling
callback function.
To get output from nghttp2_session
object, one can use
nghttp2_session_send()
or nghttp2_session_mem_send()
. The
difference between them is that the former uses
nghttp2_send_callback
to pass output to an application. On
the other hand, the latter returns the output to the caller. If in
doubt, use nghttp2_session_mem_send()
since it is simpler. But
nghttp2_session_send()
might be easier to use if the output buffer
an application has is fixed sized.
In general, an application should call nghttp2_session_mem_send()
when it gets input from underlying connection. Since there is great
chance to get something pushed into transmission queue while the call
of nghttp2_session_mem_send()
, it is recommended to call
nghttp2_session_mem_recv()
after nghttp2_session_mem_send()
.
There is a question when we are safe to close HTTP/2 session without
waiting for the closure of underlying connection. We offer 2 API
calls for this: nghttp2_session_want_read()
and
nghttp2_session_want_write()
. If they both return 0, application
can destroy nghttp2_session
, and then close the underlying
connection. But make sure that the buffered output has been
transmitted to the peer before closing the connection when
nghttp2_session_mem_send()
is used, since
nghttp2_session_want_write()
does not take into account the
transmission of the buffered data outside of nghttp2_session
.
Includes¶
To use the public APIs, include nghttp2/nghttp2.h
:
#include <nghttp2/nghttp2.h>
The header files are also available online: nghttp2.h and nghttp2ver.h.
Remarks¶
Do not call nghttp2_session_send()
, nghttp2_session_mem_send()
,
nghttp2_session_recv()
or nghttp2_session_mem_recv()
from the
nghttp2 callback functions directly or indirectly. It will lead to the
crash. You can submit requests or frames in the callbacks then call
these functions outside the callbacks.
nghttp2_session_send()
and nghttp2_session_mem_send()
send first
24 bytes of client magic string (MAGIC)
(NGHTTP2_CLIENT_MAGIC
) on client configuration. The
applications are responsible to send SETTINGS frame as part of
connection preface using nghttp2_submit_settings()
. Similarly,
nghttp2_session_recv()
and nghttp2_session_mem_recv()
consume
MAGIC on server configuration unless
nghttp2_option_set_no_recv_client_magic()
is used with nonzero
option value.
HTTP Messaging¶
By default, nghttp2 library checks HTTP messaging rules described in
HTTP/2 specification, section 8.
Everything described in that section is not validated however. We
briefly describe what the library does in this area. In the following
description, without loss of generality we omit CONTINUATION frame
since they must follow HEADERS frame and are processed atomically. In
other words, they are just one big HEADERS frame. To disable these
validations, use nghttp2_option_set_no_http_messaging()
.
For HTTP request, including those carried by PUSH_PROMISE, HTTP message starts with one HEADERS frame containing request headers. It is followed by zero or more DATA frames containing request body, which is followed by zero or one HEADERS containing trailer headers. The request headers must include ":scheme", ":method" and ":path" pseudo header fields unless ":method" is not "CONNECT". ":authority" is optional, but nghttp2 requires either ":authority" or "Host" header field must be present. If ":method" is "CONNECT", the request headers must include ":method" and ":authority" and must omit ":scheme" and ":path".
For HTTP response, HTTP message starts with zero or more HEADERS frames containing non-final response (status code 1xx). They are followed by one HEADERS frame containing final response headers (non-1xx). It is followed by zero or more DATA frames containing response body, which is followed by zero or one HEADERS containing trailer headers. The non-final and final response headers must contain ":status" pseudo header field containing 3 digits only.
All request and response headers must include exactly one valid value for each pseudo header field. Additionally nghttp2 requires all request headers must not include more than one "Host" header field.
HTTP/2 prohibits connection-specific header fields. The following header fields must not appear: "Connection", "Keep-Alive", "Proxy-Connection", "Transfer-Encoding" and "Upgrade". Additionally, "TE" header field must not include any value other than "trailers".
Each header field name and value must obey the field-name and field-value production rules described in RFC 7230, section 3.2.. Additionally, all field name must be lower cased. While the pseudo header fields must satisfy these rules, we just ignore illegal regular headers (this means that these header fields are not passed to application callback). This is because these illegal header fields are floating around in existing internet and resetting stream just because of this may break many web sites. This is especially true if we forward to or translate from HTTP/1 traffic.
For "http" or "https" URIs, ":path" pseudo header fields must start with "/". The only exception is OPTIONS request, in that case, "*" is allowed in ":path" pseudo header field to represent system-wide OPTIONS request.
With the above validations, nghttp2 library guarantees that header
field name passed to nghttp2_on_header_callback()
is not empty.
Also required pseudo headers are all present and not empty.
nghttp2 enforces "Content-Length" validation as well. All request or response headers must not contain more than one "Content-Length" header field. If "Content-Length" header field is present, it must be parsed as 64 bit signed integer. The sum of data length in the following DATA frames must match with the number in "Content-Length" header field if it is present (this does not include padding bytes).
Any deviation results in stream error of type PROTOCOL_ERROR. If error is found in PUSH_PROMISE frame, stream error is raised against promised stream.
Implement HTTP/2 non-critical extensions¶
As of nghttp2 v1.8.0, we have added HTTP/2 non-critical extension framework, which lets application send and receive HTTP/2 non-critical extension frames.
To send extension frame, use nghttp2_submit_extension()
, and
implement nghttp2_pack_extension_callback
. The callback
implements how to encode data into wire format. The callback must be
set to nghttp2_session_callbacks
using
nghttp2_session_callbacks_set_pack_extension_callback()
.
For example, we will illustrate how to send ALTSVC frame.
typedef struct {
const char *origin;
const char *field;
} alt_svc;
ssize_t pack_extension_callback(nghttp2_session *session, uint8_t *buf,
size_t len, const nghttp2_frame *frame,
void *user_data) {
const alt_svc *altsvc = (const alt_svc *)frame->ext.payload;
size_t originlen = strlen(altsvc->origin);
size_t fieldlen = strlen(altsvc->field);
uint8_t *p;
if (len < 2 + originlen + fieldlen || originlen > 0xffff) {
return NGHTTP2_ERR_CANCEL;
}
p = buf;
*p++ = originlen >> 8;
*p++ = originlen & 0xff;
memcpy(p, altsvc->origin, originlen);
p += originlen;
memcpy(p, altsvc->field, fieldlen);
p += fieldlen;
return p - buf;
}
This implements nghttp2_pack_extension_callback
. We have to
set this callback to nghttp2_session_callbacks
:
nghttp2_session_callbacks_set_pack_extension_callback(
callbacks, pack_extension_callback);
To send ALTSVC frame, call nghttp2_submit_extension()
:
static const alt_svc altsvc = {"example.com", "h2=\":8000\""};
nghttp2_submit_extension(session, 0xa, NGHTTP2_FLAG_NONE, 0,
(void *)&altsvc);
Notice that ALTSVC is use frame type 0xa
.
To receive extension frames, implement 2 callbacks:
nghttp2_unpack_extension_callback
and
nghttp2_on_extension_chunk_recv_callback
.
nghttp2_unpack_extension_callback
implements the way how to
decode wire format. nghttp2_on_extension_chunk_recv_callback
implements how to buffer the incoming extension payload. These
callbacks must be set using
nghttp2_session_callbacks_set_unpack_extension_callback()
and
nghttp2_session_callbacks_set_on_extension_chunk_recv_callback()
respectively. The application also must tell the library which
extension frame type it is willing to receive using
nghttp2_option_set_user_recv_extension_type()
. Note that the
application has to create nghttp2_option
object for that
purpose, and initialize session with it.
We use ALTSVC again to illustrate how to receive extension frames. We
use different alt_svc
struct than the previous one.
First implement 2 callbacks. We store incoming ALTSVC payload to
global variable altsvc_buffer
. Don't do this in production code
since this is not thread safe:
typedef struct {
const uint8_t *origin;
size_t originlen;
const uint8_t *field;
size_t fieldlen;
} alt_svc;
/* buffers incoming ALTSVC payload */
uint8_t altsvc_buffer[4096];
/* The length of byte written to altsvc_buffer */
size_t altsvc_bufferlen = 0;
int on_extension_chunk_recv_callback(nghttp2_session *session,
const nghttp2_frame_hd *hd,
const uint8_t *data, size_t len,
void *user_data) {
if (sizeof(altsvc_buffer) < altsvc_bufferlen + len) {
altsvc_bufferlen = 0;
return NGHTTP2_ERR_CANCEL;
}
memcpy(altsvc_buffer + altsvc_bufferlen, data, len);
altsvc_bufferlen += len;
return 0;
}
int unpack_extension_callback(nghttp2_session *session, void **payload,
const nghttp2_frame_hd *hd, void *user_data) {
uint8_t *origin, *field;
size_t originlen, fieldlen;
uint8_t *p, *end;
alt_svc *altsvc;
if (altsvc_bufferlen < 2) {
altsvc_bufferlen = 0;
return NGHTTP2_ERR_CANCEL;
}
p = altsvc_buffer;
end = altsvc_buffer + altsvc_bufferlen;
originlen = ((*p) << 8) + *(p + 1);
p += 2;
if (p + originlen > end) {
altsvc_bufferlen = 0;
return NGHTTP2_ERR_CANCEL;
}
origin = p;
field = p + originlen;
fieldlen = end - field;
altsvc = (alt_svc *)malloc(sizeof(alt_svc));
altsvc->origin = origin;
altsvc->originlen = originlen;
altsvc->field = field;
altsvc->fieldlen = fieldlen;
*payload = altsvc;
altsvc_bufferlen = 0;
return 0;
}
Set these callbacks to nghttp2_session_callbacks
:
nghttp2_session_callbacks_set_on_extension_chunk_recv_callback(
callbacks, on_extension_chunk_recv_callback);
nghttp2_session_callbacks_set_unpack_extension_callback(
callbacks, unpack_extension_callback);
In unpack_extension_callback
above, we set unpacked alt_svc
object to *payload
. nghttp2 library then, calls
nghttp2_on_frame_recv_callback
, and *payload
will be
available as frame->ext.payload
:
int on_frame_recv_callback(nghttp2_session *session,
const nghttp2_frame *frame, void *user_data) {
switch (frame->hd.type) {
...
case 0xa: {
alt_svc *altsvc = (alt_svc *)frame->ext.payload;
fprintf(stderr, "ALTSVC frame received\n");
fprintf(stderr, " origin: %.*s\n", (int)altsvc->originlen, altsvc->origin);
fprintf(stderr, " field : %.*s\n", (int)altsvc->fieldlen, altsvc->field);
free(altsvc);
break;
}
}
return 0;
}
Finally, application should set the extension frame types it is willing to receive:
nghttp2_option_set_user_recv_extension_type(option, 0xa);
The nghttp2_option
must be set to nghttp2_session
on
its creation:
nghttp2_session_client_new2(&session, callbacks, user_data, option);