QUIC tunnels for multi-link management

A QUIC-based multi-link management including satellite links

In the recent years, web has evolved significantly passing from simple HTML content-based services to complex multimedia streaming (both live and on-demand), interactive and cloud applications which are generating dynamic and greedy traffic, leading to the need to improve network performance in terms of higher capacity and lower latency. At the same time, new standard protocols were proposed, introducing flexibility and new features not available before by the use of TCP-based protocols. We are talking in particular about the QUIC protocol suite which is the foundation of HTTP/3 and will represent the future of web-communication.

Thanks to some of its disruptive characteristics, such as the 0-RTT connection establishment using UDP, QUIC can also be adapted for the effective management of multiple and heterogeneous links. This approach is particularly relevant in the framework of the recently introduced ATSSS mechanism in future 5G networks, as well as for Performance Enhancing Proxy (PEP) architectures in mixed terrestrial/satellite communications.

In the following, after a short introduction to the QUIC protocol, we propose the design and implementation of a QUIC-based tunnelling solution to enable an enhanced communication approach leveraging multiple links (either at the access or at the backhaul of a network), each associated with a different network technology (e.g., satellite, WiFi or terrestrial) or administrative domain.

What’s QUIC?

Let start to address what QUIC is not:

1. It is not just a transport protocol: it includes congestion, flow control and error recovery functions, authentication/security mechanisms, mobility support with a priority-based object transmission closely embedded with HTTP protocols and applications;
2. It is not a new TCP version: QUIC offers all these functions (and many more) based solely on the connection-less transport protocol UDP.

QUIC is therefore a wide-spanning protocol stack, which shall be implemented by both web browsers and web servers, opening the way to the recently defined HTTP/3 (which at the moment of writing is in the latest standardization stages). More specifically, HTTP/3 specification [1] explicitly mandates the use of QUIC transport [2] together with other new related specifications.

QUIC is based on UDP but it keeps the concepts of a “connection-oriented” protocol while filling some gaps when using TCP in previous HTTP/2 and HTTP/1.1. The overall new protocol stack is reported in figure, with to the left the classic HTTP/2 stack and to the right the new HTTP/3 stack.

Figure 1: HTTP over TCP (left, HTTP/2), QUIC (right, HTTP/3)

QUIC key features can be summarized in the following points:

1. Being based on UDP, it still requires to implement some TCP features such as a congestion control (actually a revisited version of the TCP Cubic or BBR [3]), flow control, and error management (retransmissions, timeouts) which are in practice placed at the application layer (cross-layering approach)
2. Mandatory use of 3 in its datagram-based version for UDP (DTLS)
3. The communication is enhanced with the introduction of the STREAM concept: the transmission is organized into STREAMs such that in case of a packet loss, only the target STREAM is interested in retransmissions while the rest of the information can be processed (avoiding the head-of-line blocking issue with TCP). This means that during the download of a web-page, in case problem on the STREAM carrying an image, the rest of the page can still be presented and not blocked by the TCP retransmission in-order mechanisms.
4. QUIC is intimately bound with HTTP/s, with explicit mapping of HTTP objects to STREAMs
5. Introduction to Forward Error Correction (FEC, not shown in figure) transmission for lossy links, with the aim to reduce retransmissions (please note that this aspect is not available in TCP)
6. Speedup the establishment of the connection by introducing the 0-RTT feature inherited by TLS1.3: most of the time, the first message will be used to establish the connection and to transfer application data at the same time.
7. The connection is not identified by the IP/port tuples at the socket level. Instead, a Connection ID (CID) is introduced and delivered within the QUIC conversation to distinguish different end-to-end relations. Within the same connection, several STREAMs can be defined, delivering QUIC packets (QPACK).
8. Since the CID is handled at the application level (and not the UDP socket underneath), it can be exploited to introduce new features like the Connection Migration with the possibility to handover the QUIC connection between different networks without reset of the connection in case of a network change (e.g., NAT rebinding) due to mobility or multi-homed configurations

For deeper details on QUIC we wish to redirect interested readers to RFC specifications as well as the descriptions available on several technical and white papers on the Internet, such as https://cloudflare-quic.com/

Most of the devices allows the possibility to enable the use of QUIC as part of HTTP/3 both on desktop web browsers (e.g. Safari, Google Chrome, Firefox) and as option also in Smart-Phones, as shown in Figure 2 WebKit advanced configuration page.

Figure 2: Experimental Features on an iOS device allow to enable HTTP/3 capabilities

Concerning the server side, we are witnessing a constant increase in its adoption by big players (such as Google, Facebook, etc.).

QUIC multiverse

QUIC was released by Google in the earlier stages, but since then it was forked by the IETF working group for its further development; therefore, two versions of the QUIC protocol stack are currently available, showing some differences and in practice incompatible:

1. Google QUIC (GQUIC)
2. IETF QUIC (IQUIC)

The IQUIC aims to solve some implementation issues of GQUIC, by opening its specification to the broader Internet community (IETF/ RFC) and so becoming the reference version for HTTP/3. IQUIC also carries a release number in its headers, which is representing the compatibility to such specification.

Without going deeply in the study of the QUIC protocol variants, we can highlight some substantial differences between the two implementations. At first we define the ‘flow’ as the flow of packets between two peers; we assume the client (C) as the initiator of the communication and the server (S) as the end-point. Therefore, two directions are defined: C => S and S =>C. In the following, some differences are reported:

Table 1 Differences between QUIC versions

	GQUIC	IQUIC
Connection	It considers a bidirectional flow between client and server. The connection is identified by a single unique Connection ID on both directions. The ID has a fixed 8 bits length	It considers a unidirectional flow between client and server. The communication between the two peers is composed by two distinct connections identified by two unique Connection IDs, one for each direction (C => S and S =>C). The ID has a variable size
Encryption	It uses its own cryptographic suite called ‘QUIC Crypto’ QUIC-c [4]	It is based on TLS 1.3 [5]
Connection Migration	A peer is allowed to accept a flow of messages identified by the connection ID (protected by means QUIC-c) even if IP/UDP parameters have been changed	Before to accept incoming packets with different IP/UDP parameters, a Path Validation procedure must be completed. In this way: i) the reachability of the link on path is ensured; ii) the peer is verified (‘authenticated’ on the new path); iii) the connection ID should change to preserve privacy
Header	The packet number is in clear text in the header as well as the connection ID	The packet number is encrypted with the payload.

Joining terrestrial and satellite links

Modern network applications are characterized by a higher and higher need of resources, with significant increase in average size of web contents which are becoming more and more demanding, including interactive contents and multimedia. Many contents requested at the same time through the same link, may result in some cases in underperformance due to congestion. Instead, the joint use of different links can be considered to optimize users’ experience, by introducing multi-homed or multi-links configurations, either as backhaul solutions or at the access.

With backhaul we refer to transport links which are used within the core network of the Internet Service Provider (ISP) to connect its users to the Data Network (i.e., Internet). In this case, use of redundant or parallel links is in charge to the network operator, making their existence transparent to the end-users. On the contrary, we can see a more and more common availability of different accesses to the Internet directly in the end-users devices, as in particular it is true for Smart-Phones, which have for instance both wireline access when at home (via Wi-Fi) and radio access to the 4G/5G network. Until now, with few exceptions, such links are not enabled simultaneously.

While considering the use of links with similar characteristics may improve on resilience, the use of very different links (a “hybrid network”) can bring additional benefits. The reference is in particular to the combination of terrestrial and satellite technologies. So far, the satellite was not seriously considered for a backhaul solution or complementary access for many reasons:

• high costs;
• high complexity of the terminals (big antennas for high data rates);
• usually confined only to rural areas as the last resort solution.

Nowadays, large investments in Very High Throughput Satellites systems (VHTS – i.e., KONNECT VHTS in GEO orbit) as well as to LEO mega constellations (i.e., StarLink and OneWeb) delivered a significant reduction of satellite communications costs, leading big companies of ICT to recognize the potential of the satellite segment and to start to invest money on their turn.

In this context, the satellite segment can be exploited mainly in two ways, in both rural and urban environments:

• as an alternative link to face network congestions on the primary link or for “channel bonding” to improve channel capacity
• as a backup link to recover and/or restore connectivity in case of link outage (e.g., natural disasters) and improve resilience

Definitively, a hybrid network is defined as the combination of a terrestrial and a satellite link, as represented in Figure 3, either for the access or for the backhaul.

Figure 3: Typical network architecture considering a hybrid backhaul network

Satellite in the access network

A significant case in which the use of very different access links can bring several service improvements, is represented by the Access Traffic Steering, Switching & Splitting (ATSSS) recently defined for 5G networks since release 16, leveraging non-3GPP access. Even though 3GPP only defined terrestrial alternative access mechanisms, namely WiFi (WLAN) and cable (wireline), possible extensions to the use of satellite are possible. In ATSSS, uplink and downlink traffic, independently, can make use of the main link (i.e., 5G New Radio), or the secondary non-3GPP link (i.e., WiFi), or both, on the basis of the network status, the applications requirements and eventually the service operator policies. In this way, if considering as alternative link a satellite one, it would be possible to improve service resilience, availability, mobility, and coverage; at the same time, thanks to the broadcasting nature of the satellite, it would be possible to send for instance live IPTV over satellite in downlink alleviating the terrestrial core network load in case of big events (i.e., sports, concerts, etc.).

In ATSSS the management of the multiple links is delegated to the end-users equipment (UEs) and the 5G User Plane Function (UPF), via dedicated standardized Multi-Access protocols such as MP-TCP or ATSSS-LL.

Hybrid Satellite-Terrestrial Backhaul

In satellite communications (SatCom), Performance Enhancement Proxies (PEPs) are adopted to speedup communication between client and server [6] and are placed at the edges of the SatCom link. PEPs are the perfect place where to enable a multi-link management, enabling a hybrid network backhaul. Considering the satellite as either supplementary or complementary technology together with traditional terrestrial networks, it allows achieving better performance and new services. In such a situation, the PEP agent is in charge of manage data flows that could dynamically be moved across different network domains during their lifetime.

The legacy PEP actually is a hardware-based agent, running customized network software and protocols based on a static configuration and architecture. Recently, in the pursue of softwarization which characterizes modern networks (including 5G), a virtual-PEP (vPEP) agent was proposed and presented in [7] and [8] along with a preliminary functional validation.

The vPEP represents a chain of Virtual Network Functions (VNF – IP layer and above) deployed to allow an optimized integration of a satellite component and enabling the seamless traffic steering and switching required. A vPEP-enabled architecture can transparently make use of the satellite link or terrestrial link, transparently to the end-users, following specific optimization logics for routing, steering and bonding using dedicated VNFs chains and a specific tunnelling protocol.

Reference architecture

We can refer to the typical scenario previously described, considering Internet connectivity from end user terminals through 4G/5G or wireline fixed access networks. The satellite can represent an additional access (via ATSSS) or backhauling link (via vPEP), supporting the terrestrial access network to the core networks with the aim to offload the ISP terrestrial network while offering satellite-based services (such as multicast). In this way, common TCP/IP applications can alternatively cross a terrestrial link, a satellite link or simultaneously both on the basis of the QoS targets and service requirements.

Since the satellite link could be untrusted and, very likely, independently managed by a Satellite Network Operator (SNO), it is necessary to define robust multi-access protocols suitable to the exploitation of multiple links. This means that such protocols shall work with NAT, different subnets and potentially the presence of Layer7 firewalls, leveraging in practice just plain IP connectivity.

Please note that in case of ATSSS, a layer 2 Multi-Access protocol is defined, namely ATSSS-LL, but its implementation is currently left open and vendor-dependent and, consequently, leaving a technology gap to be filled.

In both access and backhauling cases, we can therefore consider a similar architecture, in which dedicated agents are introduced to enable multi-link management. In Figure 4 we report one of the two cases, a vPEP-enabled hybrid backhauling; a similar configuration is also applicable to ATSSS, in which the multi-link management is a component of the UE and the UPF of the 5G network.

Figure 4: Reference scenario where a set of web-clients generates web-traffic, requesting data to the servers located in the Data Network. Web-traffic is handled by vPEP agents deployed at the edges of the backhaul network provided with both Terrestrial and Satellite links.

In this context, it is necessary to implement a Multi Link/Multi Access protocol suitable to enable the communication on both links (and in principle also to more than 2 links), and the idea is to leverage the QUIC protocol.

Why not TCP?

The main objective of using multiple links is not only to select the most suitable link on a per-connection or per-packet basis, but also to dynamically move data flows across different networks domains. A TCP connection moved towards a new network managed within a different administrative domain can be compromised. In fact, in case NAT, firewalls (with or without deep packet inspection) and different subnets, the transport-layer socket (based on the tuple [IP/port]) may change, resulting in a RESET from the server and requiring the establishment of a new TCP connection (“ERROR NETWORK CHANGED” appears on most of the web browsers). New connection implies a new handshake phase on the new path, which takes at least 1 RTT over the new link and may introduce resource waste and idle times, making a direct migration of the connections not straightforward.

To overcome these limitations, some TCP based solutions allow the management of a multipath or multihoming communication giving support to network change (e.g., MP-TCP, SCTP). In fact, specifically for 5G ATSSS the MP-TCP support is considered mandatory.

Nevertheless, MP-TCP suffer of some lacks if considering the reference scenario described above. In fact, it can be configured with static and predefined path-manager during the setup phase, in particular: i) ‘default’ will use as default link the one with lowest RTT and then, when the congestion window is full, it will start using the next higher RTT link; ii) ‘redundant’ allows the transmission of packets on all available links ensuring the communication via other links in case of a link outage (working as ‘backup’). Then, target links to be used by MP-TCP require a priori configuration. Even if recently a smarter MP-TCP connection management was defined [9], we decided to focus on a flexible alternative solution on the basis of the newly defined QUIC protocol. Also MP-QUIC can be considered, even it still a draft IETF specification, and lately included in Rel-18 of 5G.

As already discussed, QUIC natively supports the Connection Migration feature allowing the communication to be moved on a different link, while guaranteeing a 0-RTT connection establishment. The migration concluded when a change of the network parameters (i.e., Source-IP) has been detected by one peer that continue to transmit data on the link where packets arrived. To note that in case of i) GQUIC the communication will continue directly on the new path while in case of ii) IQUIC the Path Validation procedure is required to guarantee the reachability on the new path requiring an additional 1RTT. Definitively, a subset of the GQUIC features can be exploited to address all the traffic steering, switching and splitting operations in a very efficient way by defining a set of so called “QUIC-based tunnels”.

QUIC tunnels

The concept

At first, let’s recall the concept of a ‘tunnel’. Technically it is a method for transporting user data across a network using protocols that are typically not supported by that network or just to add an additional layer of security and privacy: tunnelling works by encapsulating user packets inside other packets (IP-over-IP) and usually is used in Virtual Private Networks (VPNs). Logically it represents a 1:1 association between two peers. Therefore, a QUIC tunnel can be defined as an association between a client and a server leveraging on QUIC at transport layer, irrespective from the actual network domain crossed. QUIC tunnels are then used to efficiently manage hybrid satellite-terrestrial backhaul links of our reference architecture.

The main idea is to terminate all incoming TCP connections on the first vPEP, then to encapsulate the payload (e.g. “GET” request) in QUIC packets and finally to forward them to a corresponding vPEP endpoint, which shall restore the original TCP connection to the final web-server.

A set of QUIC tunnels can be therefore created when crossing the hybrid backhaul network. In this way the vPEPs who are in charge of enabling such tunnels assume two main roles:

• Proxy server to eventually speedup connection over satellite links
• QUIC proxy element to handle dynamically the change of the path among different backhaul networks

Please note that thanks to multiplexing features of QUIC, a single QUIC tunnel can carry an aggregate of traffic related to different clients. Based on the state of the network or on the policy of the ISP, it is possible to move dynamically this aggregation from one network to another one. Several configurations can be deployed: a predefined number of tunnels can be established related to priority levels, types of applications, logical data paths within the network (network slices) etc. according to the flexibility required.

In the laboratory

The proposed QUIC-tunnel architecture, implemented into a Linux-based testbed, is represented in Figure 5. Three Area Networks are identified:

• Access Network (Client-side)
• Backhaul Network (Proxy-side)
• Data Network (Server-side)

On the Client-side, the Access Network offers to different web-clients connectivity towards remote web-servers located in the Server-side. The Proxy-side includes the Backhaul Network wrapped by a pair of vPEPs to provide an efficient transparent service using QUIC-based tunnels.

Actually, the two vPEP agents are represented in the figure by the Redirector + Proxy-Client and Proxy-Server as a client and a server respectively. The vPEP-client is in charge to aggregate web-traffic into categories to be associated with separate QUIC tunnels (Redirector). In addition, the vPEP-client is authorized to move tunnels among different backhaul links (terrestrial and satellite), following pre-determined control rules (Proxy-Client). The vPEP-server is in charge to terminate QUIC tunnels and to restore legacy TCP connection to the web-server (Proxy-Server).

Once the traffic is managed via a set of tunnels, in case of need, the default path (i.e., terrestrial) can be offloaded by selectively moving one or more tunnels to an alternative path (i.e., satellite). While the Redirector is in charge of create the association “traffic : tunnel” in the Client-side, the Proxy-Client element is the entity in charge of perform the migration of the tunnel in the Proxy-side.

Figure 5: All the traffic related to Flat users (yellow tube) is redirected in the low-priority tunnel (#2) while the one related to Business users (green tube) is redirected to the high priority tunnel (#1).

Conclusion

This work proposes a practical solution for the management of multiple links in a hybrid backhaul network including a high delay satellite link. A subset of features of the transport protocol QUIC, chosen as the baseline for the HTTP/3, is selected to implement a new network element, the vPEP, in a 5G perspective. In fact, the vPEP is defined as a set of functions aimed for the optimization of the communication between Web clients and server considering a heterogeneous backhaul network setup. Two are the focal point of this work:

1. Easily migration of a connection between different network domains thanks to the design features of the QUIC (a UDP-based transport protocol)
2. Aggregate approach to take decision and to perform operations not on the single connections but on an aggregate, i.e., a set of TCP connections united by common characteristic (coming from same Area Network, belonging to the same user, requesting similar contents, etc …)

References

[1] Hypertext Transfer Protocol Version 3 (HTTP/3), Internet-Draft [ONLINE]
[2] QUIC: A UDP-Based Multiplexed and Secure Transport, Internet-Draft [ONLINE]
[3] QUIC Loss Detection and Congestion Control, Internet-Draft [ONLINE]
[4] QUIC Crypto [ONLINE]
[5] “The Transport Layer Security (TLS) Protocol Version 1.3”, Eric Rescorla, RFC 8446
[6] “Performance Enhancing Proxies Intended to Mitigate Link-Related Degradations”, Jim Griner and John Border and Markku Kojo and Zach D. Shelby and Gabriel Montenegro, RFC 3135
[7] Abdelsalam, A., Bujari, A., Luglio, M., Munaretto, D., Palazzi, C.E., Quadrini, M., Romano, S.P., Roseti, C., Zampognaro, F. “Implementation of Virtualised Network Functions (VNFs) for Broadband Satellite Networks” (2019) 2019 European Conference on Networks and Communications, EuCNC 2019, art. no. 8801954, pp. 182-186. DOI: 10.1109/EuCNC.2019.8801954
[8] A. Bujari, M. Luglio, C. E. Palazzi, M. Quadrini, C. Roseti and F. Zampognaro, “A Virtual PEP for Web Optimization over a Satellite-Terrestrial Backhaul,” in IEEE Communications Magazine, vol. 58, no. 10, pp. 42-48, October 2020, DOI: 10.1109/MCOM.001.2000322.
[9] Intel, Multipath TCP Daemon, https://github.com/intel/mptcpd

 

© RomARS – Mattia Quadrini