Studies of High-Speed TCP/IP Data Transfer Techniques for Heterogeneous Networks with High-Speed Wireless Communications

(1)

Studies of High-Speed TCP/IP Data Transfer Techniques for

Heterogeneous Networks with High-Speed Wireless

Communications

高速無線通信を含む多様なネットワークのための高速 TCP/IP ^{データ転送技術の研究}

February 2018

Yohei HASEGAWA

長谷川洋平

(2)

(3)

Studies of High-Speed TCP/IP Data Transfer Techniques for

Heterogeneous Networks with High-Speed Wireless

Communications

高速無線通信を含む多様なネットワークのための高速 TCP/IP ^{データ転送技術の研究}

February 2018

Waseda University

Graduate School of Fundamental Science and Engineering Department of Computer Science and Communications Engineering

Research on Image Information

Yohei HASEGAWA

長谷川洋平

(4)

(5)

Abstract

Since the Internet and Internet-based networks have a huge variety of kind of networks and links including high-speed wireless links, there are many issues related to transport protocol to utilize network bandwidth eﬃciently. Our works cover high-speed data transfer techniques including transport layer techniques and overlay network layer. Techniques in overlay network layer also include a topology detection technique to clarify the structure of networks. Our work shows that the design and implementation of these techniques drastically improve data transfer throughput over high-speed wireless communications and wide-area networks. We have evaluated the performance of the proposed techniques through analysis, simulations, and experiments.

We start by describing the issues related data transfer over heterogeneous networks, especially networks including high-speed wireless communications such as millimeter wave bands and optical wavebands. We demonstrate the throughput model of existing transport protocols are not robust enough when it was used in the wireless communications with higher frequency bands. We also point out issues on congestion control in overlay networks, multipath TCP data transfer, and network topology detections.

We then present new techniques and their platform “TCP-Booster” to improve data transfer throughput over heterogeneous networks, including high-speed and long-distance wireless communications. The proposed techniques are a high-speed TCP which have suﬃcient robustness against packet losses in high-speed wireless communications, an inter- node congestion control method which enables low latency and high-speed data transfer on overlay networks, an application-level multi-path data transfer scheduling technique which realizes wireless bandwidth resource aggregation, and topology detection technique by which overlay-network nodes can detect topology of the data-link layer (Ethernet) network by their own without assistance by the other node.

We show performance evaluation of the proposed techniques. We show some simulation tests to conﬁrm characteristic of the proposed techniques, and we also show experimental

(6)

results on emulated networks, using our testbed implementation. The emulation tests include free-space optical communication to a geosynchronous-earth-orbit satellite.

We also describe two field tests of TCP-Booster with the proposed techniques. We confirmed data transfer throughput improvement by TCP-Booster at a test on 40Gbps free-space optical communication system and a global cloud storage service over the In- ternet via a dedicated trans-pacific submarine cable.

We conclude the thesis by discussing the broader implication of our work on future heterogeneous networks with large-capacity wireless communications.

(7)

List of Figures

1.1 Summary of the issues. . . 6

1.2 Case for TCP-Booster with four proposed techniques. . . 7

1.3 List of the proposed techniques and the ﬁeld tests. . . 11

2.1 Target area of existing protocols and TCP-FSO. . . 19

3.1 Internal organization of the TCP-Booster. . . 22

3.2 Example of parameter settings using Tcptables/ParamManager. . . 25

4.1 Example of data transfer using FSOC system. . . 28

4.2 Layering block of end-to-end data transfer with FSOC. . . 29

4.3 TCP-FSO functions. . . 31

4.4 Queueing points and traﬃc marge points in data transmission. . . 32

4.5 Transaction by ﬁne grained retransmission timer. . . 35

4.6 Example transaction when signal loss point was near the sender. . . 36

4.7 Delay-based ACWND control. . . 37

4.8 Measurement of ACK ﬂight size using timestamps. . . 38

4.9 ACK retransmission option (Vital-ACK). . . 39

4.10 Test settings. . . 40

4.11 Throughput in limited sending queue: (a)TCP-FSO, (b)Cubic. . . 42

4.12 Throughput of TCP-FSO and Cubic on random loss. . . 43

4.13 Throughput of ideal, TCP-FSO model, and measured. . . 44

4.14 Throughput of lower packet loss rate. . . 45

4.15 TCP-FSO throughput in 10Gbps link with random loss. . . 45

4.16 Cubic throughput in 10Gbps link with random loss. . . 46

4.17 Recovery time of RTO case. . . 47

4.18 Throughput with and without ACK congestion control. . . 48

(11)

5.1 Coverage of Local-model and e2e-model. . . 52

5.2 metrics used by Model-controller. . . 53

5.3 Local-model and e2e-model control at v_i． . . . 53

5.4 example of throughput control by INCC. . . 54

5.5 ODFα controls by OD-Loss． . . . 56

5.6 Simulated network． . . . 57

5.7 Throughput by the conventional method. . . 58

5.8 Queued data size by the conventional method. . . 58

5.9 Throughput by the proposed method. . . 59

5.10 Queued data size by the proposed method． . . . 59

5.11 QEI against link speed diﬀerenciation． . . . 60

5.12 QEI against link delay diﬀerenciation． . . . 60

5.13 A network of in-lab tests for OD． . . . 61

5.14 ODF settings and throughput． . . . 62

5.15 Throughput against propagation delay． . . . 63

5.16 Overview of the test in mobile network． . . . 64

5.17 File size and download time． . . . 64

6.1 Example of multipath TCP communication. . . 69

6.2 ATLB data distribution. . . 70

6.3 Overlay network with multipath TCP. . . 71

6.4 Network topology in the simulation． . . . 73

6.5 Robustness for link bandwidth diﬀerentiation． . . . 74

6.6 Robustness for link delay diﬀerentiation． . . . 75

6.7 Robustness for link packet loss rate diﬀerentiation． . . . 75

6.8 Network topology for multipath data transfer test． . . . 76

6.9 Throughput of multipath data transfer on Wireless LAN． . . . 77

7.1 Example of determining connecting point of n_x+1. . . 82

7.2 Paths between hosts in network with one switch: (a) connecting point t is a link; (b) t is a switch. . . 84

7.3 Networks with switches: (a) connecting point t is a link; (b) t is a switch. (c) network with unknown host n_x+1 connected to switchs_l . . . 85

(12)

7.4 Example for proof of theorem 1: (a) segment edge is connected to a host;

(b) both segment edges are connected to the network. . . 88

7.5 Example testing conﬁgurations: (a) two paths share a switch; (b) two paths do not share a switch. . . 90

7.6 Example of topology detection: (a) original network, (b) unknown host n₅ connected through lⁿ_s¹ 1, (c)n₅ connected towards lⁿ_s² 1, and (d)n₅ connected towardsl_sⁿ⁴ 1. . . 92

7.7 Hop and degree of switches. . . 93

7.8 Number of SSDs for tree network with height of (a) eight and (b) four. . . 94

7.9 Number of SSDs (a) when degree was constant and (b) when degree was exponentially distributed. . . 96

7.10 Network testbed. . . 97

7.11 Detection time vs. timeout setting. . . 97

8.1 Field test system set up. . . 102

8.2 Scene of ﬁeld test at Taikicho. . . 102

8.3 Throughput history of TCP-FSO. . . 103

8.4 Sampled packet loss rate. . . 103

8.5 Overview of test for cloud-strage service． . . . 105

8.6 File transfer throughputs (a) download, (b) upload． . . . 107

8.7 Estimated service throughput to North America． . . . 108

(13)

List of Tables

1.1 Summary of the backgrounds, issues, and proposals. . . 8

3.1 Buﬀering modes of TCPProxy. . . 23

3.2 Transfer mode of TCPProxy． . . . 23

4.1 System parameters. . . 40

5.1 Contents of local-model and e2e-model. . . 53

5.2 Machines and tools. . . 61

5.3 ODF settings and throughput． . . . 62

5.4 OD method and packet loss rate． . . . 62

7.1 List of elements in network with unknown host nx+1 connected to link l_s^s_p1^c1. 84 7.2 List of elements in network with unknown host n_x+1 connected to switch s_l. 87 7.3 Eﬀect of background TCP traﬃc on accuracy. . . 99

8.1 Parameters of the ﬁeld test system . . . 103

8.2 Machines and tools． . . . 105

(14)

(15)

Preface

The works in this dissertation are parts of my results for 20 years at Waseda University and NEC Corporation. As a researcher of network protocols, my research history has a long list of networking research ﬁelds including mobile networks, terrestrial networks, submarine networks, satellite networks, and so on. I do not think the research list is completed, but I am delighted that I had some conclusions as results at Waseda University.

My days in the graduate school of Waseda University have been precious to me;

learning new topics and refreshing my knowledge. It is always exciting for me to be with people who have a passion for accomplishing their works.

I would like to thank my advisor, Professor Jiro Katto, for his mentoring and support throughout my days in the graduate school. I also like to thank my co-advisors, Prof.

Nozomu Togawa and Prof. Hidenori Nakazato, for their invaluable comments to my thesis.

I would like to thank all colleagues in NEC, in particular, Masahiro Jibiki, Tutomu Murase, Hideyuki Shimonishi, Kazuya Suzuki, Yasuhiro Mizukoshi, Koji Hino, Takashi Egawa, Yoshiaki Kiriha, Yasuhiro Yamasaki, Ichiro Yamaguchi, Takayuki Hama, and Makoto Fujinami. Their cooperations were invaluable to me.

My wife, Chikako and three children, Kenji, Shinji, and Saori made great sacriﬁces in providing me a time to complete this thesis. This thesis would not have been possible without the great support and patience of my family. I would like to express my sincere gratitude to them.

Yohei Hasegawa

Yokohama, 19th January, 2018

(16)

(17)

Chapter 1 Introduction

1.1 Background and Motivation

The internet has been growing in the last 20 years, and mobile internet is growing rapidly.

Global mobile data traffic reached 7.2 exabytes (7,200,000 terabytes ) per month at the end of 2016, taking a significant part of IP traffic of 96 exabytes per month [1, 2].

Networks based on the Internet technologies have been vastly increasing, and the scope of application of them has also been expanding. In particular, progress in wireless communication technologies such as millimeter wireless communications (mmWave) [3,4]

and free-space optical communications (FSOC) [5, 6] is remarkable.

Due to the broad expansion of the scope of application, TCP/IP [7,8] communications are supposed to meet various needs. Hence it is diﬃcult to deal with them using one TCP algorithm and parameter setting. The Internet consists of a variety of links with a wide range of bit rate, delay, and packet loss rate. For example, 300 bps LPWA [9] wireless communications, several Giga bps wireless link of mmWave communications, 400G bps optical links [10] whose bit error rate is below 10⁻³⁰.

Among types of the link of the Internet, high-speed wireless communications based on higher frequency band including mmWave and FSOC are the extraordinary cases where legacy TCPs cannot get suﬃcient performance on them. It is known that bit error rate of wireless communication increases in proportion to the square of its frequency [11,12,13].

Hence, the throughput of TCP follows in inversely proportional to the square root of packet loss rate[14]. In other words, TCP throughput will degrade despite the increase of wireless link capacity.

Moreover, considering a variety of the internet, it is quite diﬃcult for a single TCP algorithm to utilize the capacity of various kind of network, avoiding congestion of traﬃc

(18)

CHAPTER 1. INTRODUCTION

from a large type of end-hosts. Many techniques have been studied to relax the problem.

For example, performance enhancement proxies [15] are used for satellite communications where propagation delay is too long for general TCP to utilize their capacity.

To improve data transfer throughput, techniques which split a TCP connection in a network have been studied, especially for satellite communications and mobile communications [16, 17, 18, 19, 20,21]. They use tuned TCP for the link where TCP throughput should be improved. These techniques are called as Split-TCP, Application level relay, and Overlay networks.

Split-TCPs has issues on optimizing end-to-end throughput and round-trip delay.

There are some studies related to end-to-end throughput and queuing delay, but they are not suﬃcient in many cases as we describe later.

Another issue of TCP/IP data transfer is to utilize the whole available resource in a network path from sender to receiver. TCP/IP uses a single route between end-hosts, but, recently, network environments with multiple paths are becoming common[22], especially in wireless network and the Internet.

In the optimization of TCP data transfer, lack of data-link layer information is also a critical factor, but it is available only for administrators of the local network. No interface is usually available for end users. It is very convenient for rate control algorithms to know a lower layer network topology, especially Ethernet network topology.

Thus, we pointed four main issues to improve end-to-end data transfer throughput, as 1) TCP throughput in high-speed wireless communications, 2) Optimization in throughput and delay of Split-TCP data transfer, 3) Concurrent multi-path data, 4) Ethernet layer topology detection. We describe the detailed background of each issue.

1.1.1 TCP for High-speed wireless

Carrier wave frequencies of wireless communications are shifting to higher bands to obtain higher bit rate. Free-space optical communication (FSOC) [5, 6, 23] and millimeter wave (mmWave) [3] are considered as key technologies of next-generation wireless communications, such as mobile communications and satellite communications. 28GHz or higher mmWave bands will be used for next-generation mobile systems and wireless transport system. 60GHz mmWave band has been used for IEEE802.11ad [4].

FSOC oﬀers very high-speed data transfer rates using optical wavebands around 200 THz. Using similar wavebands, it can be as fast as optic ﬁber communications. With

(19)

narrow optical beams, FSOC also can be used for long-distance communication such as several kilometers in the air and several thousand kilometers to a satellite [24,25]. FSOC is considered signiﬁcant for observation satellites, which acquire large amounts of data using high-resolution image sensors. Observation satellites often require data transfer speed of several gigabits per second or higher to complete data transmission within the limited time on their orbit, when they can access ground stations.

For example, an observation satellite with high-resolution image sensor could need 8 Gbps data transfer throughput to a ground station via an FSOC feeder link, when it transmits 300 GigaByte of image data in 5 minutes (300GB / 5 min = 1GB/s = 8Gbps).

High-resolution image sensors such as 4K camcorders often have about 20 Mbps data rate, and the data becomes 300 GB a day. Low earth orbit (LEO) satellite have limited time to access a ground station because its orbital period is not same as the Earth’s rotation period. It could be as short as 5 minutes a day, depending on available access angle to the ground station.

However, communications using extreme high-frequency band often suﬀers signal errors, when they lost sight of the line between a sender and a receiver. FSOC suf- fers from signal attenuation by air turbulence, clouds or precipitation, and atmospheric scintillations[26]. For long distance communications, FSOC systems also have diﬃculties in optical beam tracking due to the narrowness of their optical beams. Especially in long distance FSOC, slight vibration can cause optical beam tracking failure, which results in a signal loss of several hundred milliseconds.

Many high-speed TCPs have been proposed including those for radio frequency satellite communications [27, 28, 29, 30]. We think TCP has many advantages including compatibility with many applications and CPU eﬃciency compared to UDP based protocols which have rate control in Application-layer. However, to our knowledge, none have yet been able to achieve suﬃcient data transfer throughput in the environment with frequent signal error and large delay such as satellite FSOC. It is expected that TCPs should be able to transfer data to geosynchronous orbit (GEO) satellites at rates of 10 Gbps or higher, even when 10% packet loss and 125 ms delay occur in FSOC. These are a hundred times higher bit rate and thousand times worse packet loss rate, compared to typical GEO satellite communications.

There are at least four issues regarding TCPs that need to be addressed to achieve high- speed data transfer on long-distance high-speed wireless communications: 1) congestion

(20)

control on high-speed networks with large delay, 2) retransmission on high packet loss rate networks, 3) recovery from long-time signal loss, and 4) asymmetric bandwidth. We discuss these issues in this dissertation.

1.1.2 Optimizing Split-TCP data transfer

As described above, Split-TCPs, overlay networks, and application layer networks are ones of techniques to achieve high-speed data transfer over a heterogeneous network in which a single high-speed TCP cannot cover. Split-TCP have been studied for more than 20 years, and have been deployed mainly for mobile communications and satellite communications.

Performance of Split-TCP have been studied, including throughput improvement and queuing delay [31, 16, 18, 32]. In an overlay network in which high-speed TCP is introduced in a section, since the data transfer rate in a section of each network is signiﬁcantly improved, a large queuing delay could occur in the bottleneck section in the network.

In particular, the conventional feedback control for suppressing the throughput has a problem that the throughput is excessively suppressed in a wide area network having a delay.

Regarding the optimization for end-to-end performance, it is often a bottleneck section that the last-mile data transfer throughput to an end-host. End-hosts often use very small receiving buﬀer for memory eﬃciency, but it results in low throughput.

1.1.3 Concurrent multi-path data transfer

Recently, network environments with multiple paths are becoming common[22,33]. Espe- cially, multipath communications are becoming popular to improve the stability of wireless communications by aggregating multiple diﬀerent channels. In addition to a redundant operation of multipath communication for higher reliability, a load-balancing operation has been used for its high performance, especially in a multipath wireless environment.

Load-balancing over multipath communications, however, has two potential problems.

First one is a diﬃculty in implementation. Users need to add new communication scheme into their operating systems to use multiple paths, and users also need to modify their application to use the new communication scheme. We believe that simple technique which can be easily introduced to several environments is desired.

The second problem is diﬃculty in utilizing multiple paths for single data transfer.

(21)

The ordinary single TCP cannot provide suﬃcient throughput if packets are delivered via multiple network paths [34, 35]. This is because TCP is designed for connections that traverse a single path between a pair of hosts. Out-of-data delivery via multiple paths degrades TCP throughput.

1.1.4 Ethernet topology discovery

In the optimization of TCP data transfer, lack of data-link layer information is also a critical factor, but these are available only for administrators of the local network. No interface is usually opened for end users. It is very convenient for rate control algorithms to know a lower layer network topology, especially Ethernet network topology.

Ordinary data-link topology detection techniques require the network routers and end- hosts to have speciﬁc functionalities. For example, some topology detection techniques need to gather forwarding table information from network routers or switches[36, 37].

Other techniques require a router to respond[39, 40, 41]. Even an end-host-based topology detection technique [38] needs other hosts to respond to packets only for topology detection. However, Ethernet switches in LANs usually do not support network management functionalities for topology detection. Also, hosts in LAN, such as network printers, have no particular functionality for topology detection.

1.1.5 Summary

In Figure 1.1, we summarize the issues we discussed above. The current Internet has intensive complexity since the massive growth in more than 20 years. Wireless communications are the vital part of the Internet, and long distance and high-speed wireless communications have been studied as a part of the future Internet. Wireless wavebands are shifting to higher bands, but frequent bit errors degrade data transfer throughput of TCP. Split-TCP overlay networks have been studied, but they have issues with end- to-end congestion control to reduce queuing delay. Multipath data transfer is one of an important topic to utilize network resources, especially wireless resources. However, we need to relax data-sorting bottleneck of multipath data transfer. Topology detection is essential to utilize network resource eﬃciently, but, it is diﬃcult for users who do not have the privilege to access management function of network switches.

(22)

The Internet

Unknown datalink layer topology

3G: 700MHz 5G: 28GHz FSOC: 200THz

Data-sorting bottleneck Congestion in Overlay network Mobile internet

Wireless access

Split-TCP GW

1995 2000 2005 2010 2017

Frequent bit errors Large delay Long-distance

Higher-frequency

Increased complexity

Wireless backbone

Multipath Overlay network

Split-TCP GW

Partial admin.

privilege of NW

Insufficient TCP throughput

Figure 1.1: Summary of the issues.

(23)

1.2 Contributions

Based on the above discussions, we propose techniques that improve end-to-end data transfer performance by decomposing networks with complex characteristics into some simple sections and performing appropriate TCP control for each section[42,43,44]．We named the apparatus which we implemented the proposed techniques “TCP-Booster”.

TCP-Booster has new and improved features including the following four proposed techniques.

1. High-speed TCP for super high frequency wireless communications[42, 45].

2. Congestion control among overlay network path of Split-TCP[46].

3. Concurrent multi-path data transfer technique[47, 48].

4. Ethernet layer topology detection technique for overlay network node[49].

In Figure 1.2, we illustrate an overview of case for TCP-Booster with the four proposed techniques we discuss in each chapter. We also summarize the backgrounds, issues, and proposed techniques in Table 1.1. Here, we brieﬂy describe the overview of TCP-Booster and each proposed part in TCP-Booster.

The TCP-Booster

In Chapter 3, we propose TCP/IP performance optimization system, which we call TCP-Booster. TCP-Booster is a platform of the proposed techniques which we discuss in Chapter 4 to 7. TCP-Booster decomposes complexity of heterogeneous networks splitting networks into some parts, and build high-performance overlay networks which have

TCP- Booster TCP- Booster

(Chapter3) Data Center

User (with small buffer)

Mobile user (with multiple ch.) OverDrive

Inter-node congestion control

TCP- Booster

1. TCP-FSO (Chapter4)

4. Ethernet topology detection (Chapter7)

FSOC Field test (Chapter8)

2. Congestion control for overlay networks (Chapter5)

Cloud-storage Service from Japan to the USA Field test (Chapter8)

Cloud-storage service user 3. Multipath-TCP

(Chapter6)

Figure 1.2: Case for TCP-Booster with four proposed techniques.

(24)

Table 1.1: Summary of the backgrounds, issues, and proposals.

Increased complexity

Frequent bit errors and large delay The growth of the Internet

The demand for high-speed and low delay data transfer

Background Issue Proposal

TCP-Booster (Chapter 3) :

A Split-TCP overlay network system which decomposes complexity of the Internet.

The demand for high-speed wireless on high frequency wavebands

TCP-FSO (Chapter 4) :

A TCP with a significant robustness against packet loss and delay.

Congestion in

overlay network CongestioncontrolforSplit-TCP(Chapter5) : Aninter-nodecongestioncontrolandanoption to optimize TCP to a host with small buffer.

The demand for utilizing

multiple network resources Bottleneck in data-

sorting from paths Multipath data transfer (Chapter 6) : An arrival-time matching load-balancing to utilize an aggregated network resource.

The demand for utilizing

datalink layer resources Unknown datalink Layer topology (No admin. privilege)

Ethernet topology detection (Chapter 7) : An Ethernet topology detection method which can be completed by a single host.

signiﬁcant robustness against delay and packet-loss in physical networks and links, such as submarine cables, satellite link, mmWave, and free-space optical communications.

TCP for high-speed wireless communications

In Chapter 4, we propose a transmission control protocol (TCP) for long distance high- speed wireless communications, including free-space optical communications (FSOC). The extremely high frequency of wireless communications enables high bit rate, but also frequent signal errors, including burst errors. It can be a quite severe problem for conventional high-speed TCPs. To achieve 10 Gbps or higher data transfer throughput on FSOC, the proposed TCP (designated “TCP-FSO”) has improved and new features including multi-layer congestion control, retransmission control with packet loss point estimation, delay-based ACK congestion control, and ACK retransmission control. We evaluated data transfer throughput of TCP-FSO and the other TCPs, by throughput model analysis and experiment on a real implementation. Obtained results show that TCP-FSO achieves far higher data transfer throughput than other high-speed TCPs. For example, it achieved a thousand times higher throughput than the other high-speed TCPs in a real FSOC environment.

Congestion control for overlay networks

(25)

In Chapter 5, we propose inter-node congestion control method (INCC) which is controlled based on performance models of an end-to-end path and local links. We also propose a new TCP option, we call OverDrive option (OD), to improve throughput to a host with small receiving buﬀer. We validated that INCC can reduce maximum 99%

of the buﬀer in TCP-Booster, comparing to legacy Split-TCP data transfer. We also validated OD improved throughput by 500 times to a receiver with 1MB TCP buﬀer in a network with 300ms RTT.

Multipath TCP

In Chapter 6, we propose a data distribution method for multipath TCP communication that improves end-to-end performance and reduces the data-sorting cost. We call this method Arrival-Time matching Load-Balancing (ATLB). The method calculates the data arrival time for each path, considering the time that data segments spend in the TCP queue at a sender and the time needed for data segments to pass through the network.

ATLB enables in-order data delivery to a receiver, and the data sort cost at a receiver is reducible. Thus, ATLB enables high throughput communication. Moreover, we also propose overlay network architecture to deploy multipath TCP communication scheme with ATLB. Our overlay network approach is based on transparent TCP proxy which connections from a user are split and then transferred on multipath TCP with ATLB.

Ethernet topology discovery

In Chapter 7, we propose Ethernet topology detection techniques that can be run using only the generally available functionalities of hosts and switches. We have developed three techniques for Ethernet topology detection.

• A technique for topology detection, based on shared switch detection (SSD), that tests whether two paths share a switch.

• A technique for performing SSD from a single host in a switched Ethernet network.

• A technique to reduce the number of SSDs needed for topology detection.

These techniques enable a single host to detect the topology of an Ethernet network consisting of switches with no management functionality. They use only basic Ethernet forwarding functionality and general request-replies (ICMP ECHO REPLY, TCP SYN- ACK, and so on) from hosts in the network.

(26)

Performance validation on a ﬁeld test

In Chapter 8, we describe two filed test using TCP-Booster. The first one is a throughput test on FSOC system on moving vehicle, where packet loss rate is significantly high around 10%. The second is a demonstration test with global cloud-storage service from Tokyo to North America.

1.3 Thesis Organization

The rest of the thesis is organized as follows:

Chapter 2 discusses the related work on TCP, overlay network, concurrent multipath data transfer technique, and network topology detection technique. In Chapter 3, we describe an overview of TCP-Booster which is a platform to deploy our proposed technique into network systems. In Chapter 4, we present a TCP for long-distance high- speed wireless communications. In Chapter 5, we present a congestion control techniques for overlay networks, which consists of inter-node congestion control and new buﬀer control option of TCP. In Chapter 6, we describe a concurrent multi-path data transfer technique for our overlay network. In Chapter 7, we present Ethernet topology detection techniques.

In Chapter 8, we describe two ﬁeld tests on FSOC testbed system and global Internet.

In Chapter 9, we summarize the contribution of this thesis, we present future works, and finally, we offer concluding remarks. We illustrate a list of the proposed techniques and field tests in Figure 1.3.

(27)

TCP for long-distance high-speed wireless communications (Chapter 4)

Multipath data transfer (Chapter 6) Ethernet topology detection (Chapter 7)

Congestion control for Split-TCP overlay network (Chapter 5) TCP-Booster : TCP/IP performance optimization system (Chapter 3)

Test on 40Gbps FSOC on a vehicle

Test on cloud-storage service from Japan to the USA Field test and deployment of TCP-Booster (Chapter 8) The proposed techniques

Performance and functional validations

Figure 1.3: List of the proposed techniques and the ﬁeld tests.

(28)

(29)

Chapter 2 Related Work

2.1 Overview

In this chapter, we describe the related works of transport protocols, overlay networks, multipath data transfer techniques, and topology discovery techniques. The works in Section 2.2 are related to TCP-FSO we propose in Chapter 4. We discuss speciﬁc parts of transport protocols in Section 2.2.1 to 2.2.3. In Section 2.3, we discuss overlay networks which are related to techniques we propose in Chapter 5. The works in Section 2.4 are related to Multipath data transfer technique we describe in Chapter 6. The works in Section 2.5 are related to topology detection technique we propose in Chapter 6. In Section 2.6, we summarize novelty of the proposed techniques and TCP-Booster.

2.2 Transport Protocols

2.2.1 High-speed TCPs

Many TCPs have been proposed for high-speed data transfer on the Internet. Cubic [27] is widely used as a default TCP on current Linux OS. Cubic is designed to have high performance in large delay networks, solving fairness problem between ﬂows with diﬀerent RTTs. Hybla[28] is designed for heterogeneous networks including large delay networks and radio links. It increases its congestion window quickly on a large delay network.

TCP Vegas [50] has a delay-based congestion control method. Vegas achieves low delay data transfer, but it does not get fair throughput when it competes with loss-based TCP ﬂows. FAST TCP [51] is another delay-based congestion control algorithm which uses proportional control instead of a linear increase of Vegas.

Westwood[29] is a TCP for wired and wireless links. It estimates wireless signal error

(30)

CHAPTER 2. RELATED WORK

packet loss, taking into account delay when packets are lost, but it takes as long as round trip time (RTT). TCP-Real[52] is also designed for wired and wireless networks, and has receiver-oriented congestion control mechanism.

Balakrishnan et al. proposed TCP-INT which have a congestion control method which considers total size of TCP connections in a host[53]．Langley et al. proposed QUIC which have ﬂow control for parallel data transfer, considering total size of receiving buﬀer [54]．

SCPS-TP (Space Communications Protocol Specification, TransportProtocol)[30] has a set of options and modifications for environments with long delays and high bit error rates. It uses NewReno[55] and Vegas [50] for congestion control and has an optional mode that does not use congestion control. It also has retransmission control options that do not use the exponential backoff during retransmissions. SCPS-TP supposed to have higher data transfer throughput than the former high-speed TCPs described above.

However, it is also supposed that even SCPS-TP cannot get several Giga bps data transfer throughput due to large delay and frequent packet loss, such as 250ms RTT and 1% packet loss rate.

TCP-based protocols have significant advantages in inter-operability, both for applications and to other hosts, although modified TCPs can be used only in local networks when they are not compatible with specifications of IETF RFCs. Another advantage of TCP is that they have many optional functions, such as selective acknowledgment and window scaling. Operating systems are highly sophisticated as means of enabling high-speed TCP data transfer. Assistance for TCP processing is provided by software and hardware such as Intel CPUs with DPDK[56] and TCP offload processing on network cards [57].

ACK congestion control of TCP has been proposed[58]. It reduces ACK traﬃc based on loss of ACK. However, in the wireless network where bit errors occur, it will make ACK traﬃc too small due to frequent ACK loss.

2.2.2 Multi-layer Congestion Controls

Multi-layer congestion control schemes have been proposed to improve data transfer throughput in wireless or wide area networks. Kliazovich proposed cross-layer congestion controls[59] for wireless ad-hoc networks. In this scheme, TCP’s sending rate is controlled using available bandwidth information from network layer and data-link layer.

However, this scheme needs as long as RTT to update the information.

Balakrishnan proposed explicit loss notiﬁcation to improve web performance over wire-

(31)

less communications[60]. Qazi proposed congestion control scheme which estimates of congestion by combining the explicit congestion notiﬁcation (ECN) marks of multiple packets [61].

Szilagyi et al. proposed “throughput guidance” which uses explicit notiﬁcations of available throughput from mobile base-station to core network and a server[62]．

2.2.3 UDP based Protocols

Saratoga [63] is a UDP-based reliable transport protocol for satellite communications. It is robust against delay and bit errors. For example, it can transfer data at several mega bps even for10⁻⁵ bit errors and 600 ms delay. It is designed to be used on small hardware, so that it can be onboard satellites. UDT[64] is another UDP-based protocol with reliability control and congestion control, which is designed for high-performance data transfer in large delay networks. XTP[65] is a transport layer protocol for high-speed networks. XTP provides rate control in which the maximum bandwidth can bespeciﬁed as well as what size burst data can be accepted.

UDP based data transport protocols[63, 64, 65] for satellite communications have a robust performance against packet loss and delay, but they have diﬃculty in performance tuning for high-speed data transfers. They have a rate control mechanism in the application layer, but it consumes many CPU resources for fast and accurate rate control.

2.3 Overlay Networks

Split-TCP data transfer, which is an application level data forwarding have been studied for long time mainly for communications with large delay, such as mobile networks, satellite communications, and long distance communications[31, 16, 18,32]．

Liu et al. analyzed TCP data transfer throughput when a TCP connection is split into multiple connections in the network[19]．Baccelli et al. analyzed queuing delay of Split-TCP data transfer, which is caused by diﬀerence among TCP connections[66]．

Katabi et al. proposed a congestion control technique XCP which uses ECN (Explicit Congestion Notiﬁcation) feedback from receiving node citekatabi2001.

Wydrowski et al. proposed NaxNet in which nodes exchange available data transfer rate and a sender node decides transmission throughput [68].

Dukkipati et al. proposed RCP which uses explicit throughput notiﬁcation between

(32)

switches in network and end-hosts[69]．

In addition of the techniques above, some ﬂow control methods [70, 71] and active queue management techniques [72, 73, 74] are available in router and switch. However, most feedback techniques could suppress too much over the link with a large delay.

2.4 Multipath Data Transfer Techniques

To improve data communication performance of TCP on multiple network paths, Lee and Chen proposed modiﬁcations of TCP[75, 76]. These are mainly aimed at improving TCP performance in multipath forwarding networks where there are out-of-order packet deliv- eries. These approaches cannot always utilize each path, though, because their congestion control mechanisms remain almost the same as those of ordinary single TCP to enable friendliness and interoperability with regular TCP.

Zhang and Rojvi proposed communication schemes with multiple TCP connections [77, 78] They are parallel data transfer techniques in the application layer and multipath TCP communication methods in the TCP layer. The application-layer techniques are mainly based on partial and parallel requests; for example, the range request of http1.1, XFTP[79], PA[80], and gridftp[81]. The multipath TCP techniques [77,78,82,83,84] are based on the distribution of data and parallel data transfer with multiple TCP connections between two end-points.

SCTP [85] also supports multihoming. However, SCTP uses only one primary path and switches to a secondary path when a failure occurred in the primary path. Several extensions of SCTP have been proposed to enhance multipath data transfer. BA-SCTP [86] is a bandwidth aggregation protocol based on SCTP. BA-SCTP has load-balancing based on weighted round robin.

Conventional multipath data transfer methods still encounter a bottleneck in the data- sorting process at a receiver host. For example, multipath TCP methods with congestion window based data distribution are assumed that they do not work in RWIN bottleneck case or in environments where no packet loss occur such as LAN with L2 ﬂow control.

Thus, in some situations, it is possible that conventional multipath TCP methods send data segments to each connection almost equally despite each path having a diﬀerent delay or bandwidth. As a result, out-of-data arrival occurs at a receiver host. A conventional multipath TCP receiver needs a huge receiving buﬀer to sort the data segments from

(33)

each path; otherwise, an exhausted receiving buﬀer will result in a small TCP receiving window and degraded throughput. Also, in high-speed communication, the data-sorting bottleneck becomes more severe because a larger TCP window in each path increases end-to-end delay. Thus, high-performance multipath TCP communication requires an eﬃcient data distribution method.

2.5 Topology Discovery Techniques

We review related work on topology detection by describing the three main approaches.

In the ﬁrst approach, the IP forwarding table MIB[87] (management information base), which contains information gathered from routers and switches via SNMP[88], is used.

This approach can be adapted to various kinds of networks including Ethernet ones. In- ference techniques[36][37] have been proposed for use when a complete forwarding table with information from all switches and routers is unavailable. However, this approach still needs to access MIB information for most of the routers and switches in the network through their management functionalities. This approach is thus not suitable when that access privilege is not granted to the one wanting to obtain topology information.

Moreover, this approach cannot be used to detect the topology of networks consisting of switches that have no (SNMP) management functionality.

In the second approach[39][40, 41], the network topology is determined by grouping sets of addresses obtained using a probing tool such as traceroute on ICMP or Ethernet OAM [89,90]. Rocketfuel [40] is a set of techniques that implement this approach/router- level mapping tool where one of the underlying ideas is to focus on one speciﬁc ISP network at the time and to map it as completely as possible. It uses techniques that reduce the number of probe packets needed to infer the network topology.

This approach can be used in several situations because it can be used to detect the network topology without having network administrator privileges. The recently standardized Ethernet OAM (IEEE802.1ag[91]) can be used for Ethernet level probing using, for example, the ping and traceroute tools. However, Ethernet OAM cannot detect the topology of a network containing switches without Ethernet OAM functionality.

In the third approach, the network topology is inferred by monitoring packet arrival patterns at receiving hosts [92, 93, 38]. The multicast-based topology detection technique[92] refers to the packet loss pattern. If multiple hosts did not receive a probe

(34)

packet, they are assumed to be in the same part of the multicast tree and to share the link in which the packet was dropped. However, this technique still requires that all the hosts have particular functionality for topology detection.

In short, previously reported topology detection techniques are often unable to detect the network topology because they require that the routers, switches, or hosts support particular functionalities. This is especially true for Ethernet networks because the LAN switches rarely have network management functionality. Network administrators need a topology detection technique that does not depend on the Ethernet switches and end hosts having management functionality.

(35)

2.6 Summary

As we discussed above, there are many eﬀorts regarding the data transfer for various kind of networks.

Many high-speed TCPs have been studied in the last twenty years, but packet loss rate supposed in future high-speed wireless communications is too much for current TCPs.

Considering the existing TCP and UDP based protocols, we set our target in TCP de- velopment as illustrated in Figure 2.1. In Figure 2.1, BDP stands for bandwidth-delay product.

Overlay networks on the Internet have been studied as a future network architecture.

Moreover, some overlay networks are introduced in the commercial use, for example, in mobile networks, satellite communications. However, they have many issues to be solved, especially in throughput and queuing delay.

Many multipath data transfer techniques have been proposed to get the aggregate bandwidth of multiple paths. However, there was no load-balancing technique to relax data sorting overhead between subﬂows of multipath TCP before our ATLB described in Chapter 6.

Topology discovery technique will play an important roll in future networks where many overlay networks exist on one physical network, and they have to share physical resources. However, topology discovery techniques need privileged access to network apparatuses. Some techniques detect topology using probe packets from hosts in the network, But these techniques still need cooperation among many hosts in the network.

Packet Loss Rate Bandwidth-Delay product [Mbit]

10

^-1

10

^-4

10

^-7

10

⁵

10

³

10

¹

TCP-FSO SCPS-TP

High-speed TCPs Cubic

UDP based protocols UDT

Saratoga

Figure 2.1: Target area of existing protocols and TCP-FSO.

(36)

(37)

Chapter 3 TCP-Booster: A Split-TCP Data Transfer System ¹

3.1 Overview

In this chapter, we illustrate a functional overview of TCP-Booster which is a data transfer system with proposed techniques. TCP-Booster provides faster data transfer for end- hosts, splitting a TCP connection between hosts. TCP-Booster uses TCP algorithm and parameters which are customized for the network sections.

With TCP-Boosters, end-users can use high-speed data transfer service without installing customized TCP. (It is always hard to deploy new technologies to all end-hosts.) TCP-Booster provides its service as TCP/IP/Ethernet virtual network. TCP-Booster can also oﬀer some application service such as web service in its overlay network.

To decompose complexity of end-to-end network which consists of a various type of networks and links, TCP-booster can set diﬀerent TCP/IP algorism and parameters to each network and each host, considering TCP ports and IP address. Also, shorter round trip time by Split-TCP enables eﬃcient feedback to congestion control of TCP.

Remarkable features of TCP-Booster is listed as follows and discussed later in this dissertation.

1. TCP for High-speed wireless communications

2. Split congestion control throughout the end-to-end path 3. Heterogeneous multi-path aggregation

4. Ethernet network topology detection

1This chapter is adapted from the work published in [46]

(38)

CHAPTER 3. TCP-BOOSTER

Using the proposed techniques, TCP-Booster improves end-to-end data transfer throughput more than 1000 times in some situations, for example, satellite FSOC scenario, comparing to the data transfer by legacy TCPs. In the satellite FSOC scenario, TCP-Booster provides customized TCP for satellite FSOC section and also provides proper TCP settings to shared network section such as the Internet.

Drastic improvement in data transfer throughput makes new applications possible; for example, observation satellites with ultra high-resolution image sensors, high-resolution remote desktop services, long-distance large ﬁle sharing. As we describe in Chapter 8, ten times faster data transfer in cloud storage service relaxes user frustrations. Gigabit service of Mobile FSOC will broaden the application of satellite/airplane information services.

We illustrate the functional structure of TCP-Booster in Figure 3.1, which consists of TCPProxy, TCP, IP/Ethernet, ParamManager, Tcptables, and ModelController. We describe each part in the following subsections.

Tcptables ModelController

TCPProxy

TCP Param

Manager

IP / Ethernet

TCP-FSO

Figure 3.1: Internal organization of the TCP-Booster.

(39)

3.2 TCPProxy

TCPProxy terminates a connection from upstream node/host and connects to downstream node/host. And then, TCPProxy transfer data using these two connections. TCPProxy has options for buﬀering mode and data transfer mode. TCPProxy also controls enabling or disabling of multipath data transfer and inter-node congestion control.

TCPProxy has four buffering modes, as shown in Table 3.1. The mode SS is used when TCPProxy needs large size of the buffer for disruptive networks such as low earth orbit satellite which can communicate to a ground station in very limited time in their orbit, for example. The mode WL and FG are useful for wireless links where loss of signals often occurs. The mode LED has the minimum size of the buffer for data transfer operations. LED is the most frequently used, because most of the Internet is relatively stable, comparing to high-speed wireless communications.

Table 3.1: Buﬀering modes of TCPProxy.

mode buﬀer size

SS Static size of buﬀer

WL α×T put

FG α×RT T ×T put

LED equal to processing size of TCPProxy

TCPProxy has three data transfer mode: 1. Transparent mode by which end-hosts do not see TCP-Boosters’ IP address. 2. Forwarding mode by which original source host does not see TCP-Booster, but original destination host see TCP-Booster. 3. Redirect mode in which a conﬁguration ﬁle designates destination address. In each mode, TCPProxy set the source and destination address in IP header as shown in Table 3.2.

Table 3.2: Transfer mode of TCPProxy．

mode src address dst address Transparent Original source Original destination

Forward TCPProxy Original destination Redirect TCPProxy designated destination

(40)

3.3 TCP

Several TCP algorithms are available in TCP-Booster, such as Cubic[27], Vegas[50]， Hybla[28], Westwood[29], (Reno[55], TCP-FSO[42] , and etc.

TCP-FSO, which is a TCP we have proposed, have a signiﬁcant advantage in high- speed wireless communications where the error rate is relatively high. We designed TCP- FSO for use in networks with secured bandwidth. TCP-Booster provides fair share between TCP-FSO and legacy TCPs using QoS service in IP layer. TCP-FSO will not share bandwidth with the legacy TCP when they compete with the other TCP traﬃc in a network without QoS service. We describe the detail of TCP-FSO in Chapter 4.

3.4 IP/Ethernet

TCP-Booster provides packet ﬁltering function at IP and Ethernet layer, with which a connection can be forwarded by application level, IP level, or Ethernet level. When a connection is forwarded at an application level, high-speed TCP can be used towards next a hop node or a ﬁnal destination host.

IP layer also provides QoS services such as packet scheduling and traﬃc policing [94].

Traffic-priority-control and stochastic-fairness-queuing are used to make various kind of TCP traffic share the bandwidth fairly. Token bucket filter is suited to specify a target rate for TCP-FSO traffic.

Software-defined networking (SDN) technologies are often used with TCP-Booster system to make network configuration flexible [95, 96]. Using SDN, failure recovery such as path protection and network restoration can be configured easily, and it is beneficial for TCP-Booster service to be safer.

3.5 ParamManager

ParamManager set parameters of TCP/IP when a new connection is established, and also during data transfer. ParamManager monitors establishment of TCP connection from other nodes and application process. When a new connection is established, ParamMan- ager chooses parameter of TCP/IP, depending on ﬁltering results of IP address and port number. ParamManager can set parameters of most of TCP/IP parameters, including buﬀer size, TCP options.

(41)

ParamManager can change parameters also during data transfer. For example, TCP- Proxy with Model-controller adjusts TCP buﬀer size after session establishment and during data transfer, considering throughput history. Note that some parameters cannot be changed during data transfer, because they must be ﬁxed when connection establishment;

for example, MSS, window scaling, TCP timestamp, and SACK.

3.6 Tcptables

Tcptables is an application to conﬁgure ParamManager from user command line input.

With Tcptables and ParamManager, an administrator of TCP-Booster can conﬁgure cus- tom settings of TCP for each end-host, application, TCP connection, and next-hop network/link.

Figure 3.2 is an example of TCP/IP parameters of TCP connections, using Tcptables and ParamManager. As shown in Fig 3.2, TCP-Booster oﬀers proper TCP/IP parameters for each connection to get reasonable throughput.

In the legacy system, each application process often set TCP/IP parameters, but it is diﬃcult for application developers to decide TCP/IP parameters, considering all type of networks where the application is used. TCP-Booster oﬀers TCP/IP parameter management for each connection, as a system function. TCP-Booster can set TCP/IP parameters of for each connection from each application in TCP-Booster.

TCPProxy

TCP3 :

tcp_rmem = 8192 32768 262144 tcp_wmem = 4096 16384 877824 tcp_congestion_control = fso

TCP2 :

tcp_rmem = 8192 32768 262144 tcp_wmem = 4096 16384 877824 tcp_congestion_control = fso

TCP1 :

tcp_rmem = 8192 32768 262144 tcp_wmem = 4096 16384 877824 tcp_congestion_control = fso tcp_acwnd = 1

IP.dst = 10.56.23.0/24 dport= 5001

dport= 80 sport = 21

Figure 3.2: Example of parameter settings using Tcptables/ParamManager.

(42)

3.7 ModelController

ModelController provides inter-node congestion control (INCC) between TCP-Boosters on the end-to-end data transfer path. ModelController calculates two type of performance model which TCP should follow. One model is an optimized model for data transfer to the next hop TCP-Booster (Local-model). The other model is a synthesized model to optimize end-to-end data transfer (e2e-model).

ModelController dynamically decides the model to use during data transfer, considering queued data size in TCP-Booster. Using ModelController, TCP-Booster aims low latency data transfer, keeping its throughput.

We describe the detail of the inter-node congestion control in Chapter 5.

3.8 Summary

In this chapter, we described TCP-Booster which is a platform for the proposed techniques.

TCP-Booster improves data transfer throughput using customized TCP and splitting TCP. We show TCP-FSO in TCP-Booster achieve more than 1000 times higher in case that frequent errors occur in a wireless network. We also expect a few times throughput improvement by splitting TCPs without ineﬃcient queueing delays by the eﬀect of Model- Controller’s internode congestion control. Multipath data transfer with ATLB will utilize aggregated network bandwidth between end-hosts. We show examples of performance evaluation result using TCP-Booster in Chapter 8.

Along with the four proposed techniques, TCP-Booster has some unique features such as Tcptables/ParamManager and buﬀering mode of TCPProxy. They might be a niche function for most of the users, but they make advantages in data transfer performance optimization. Parameter optimization is one of our future works.

On TCP-Booster, the proposed techniques can be deployed to the real world. Trans- parent mode of TCPProxy makes it possible that TCP-Booster serves for end-users without installing any software. End-users can beneﬁt advantages of TCP-Booster even without modiﬁcation of network setting in end-hosts.

(43)

Chapter 4 TCP-FSO: A TCP for Long-Distance High-Speed Wireless

Communications ¹

4.1 Overview

In this chapter, we describe a transmission control protocol, which we designate as “TCP- FSO”, for high-speed wireless communications including satellite FSOC. First, we deﬁne target environment, and we discuss issues for TCP-FSO. Then we illustrate the detail of TCP-FSO, and we show evaluation results.

Initially, we designed TCP-FSO for satellite Free-Space Optical Communications (FSOC), and we describe satellite FSOC system in this section. However, our technique also eﬀec- tive to another high-speed wireless communication systems such as mmWave transport systems.

TCP-FSO has customized congestion control, retransmission control, delay-based ACK congestion control and ACK retransmission control functions. These functions enable TCP-FSO to achieve throughput at least several times higher than that of the other high-speed TCPs in environments with considerable delay and frequent packet loss.

We evaluated the performance of TCP-FSO in environments where packet loss and delay are emulated as long-distance high-speed wireless communications such as satellite FSOC. We conﬁrmed that TCP-FSO had far higher data transfer throughput than the other TCPs when packet loss rate is high and delay is large.

1This chapter is adapted from the work published in [42, 45]

(44)

CHAPTER 4. TCP-FSO

4.2 Target environments and issues

The target FSOC system consists of an optical beam tracking module (FSO antenna) and an optical transponder that has an Ethernet interface to the user side.

Figure 4.1 shows an example of data transfer using the FSOC system. We assume that the FSOC is from a satellite or an airplane to a ground station. The FSOC downlink speed ranges from 100 Mbps to 100 Gbps. The uplink assumed is not only FSOC but also a radio frequency (RF) wireless link, because satellites often have tight resource limitations.

The uplink speed assumed ranges from 1 Mbps to 100 Gbps. Data transfer delay on FSOC will be from 1 to 125 ms, taking into account the near ﬁeld to GEO satellites. Data from a satellite or an airplane are transferred to a data center via a terrestrial network such as the Internet.

Packet loss rate in FSOC could be worse than in other wireless communications because FSOC’s optical beams can be disturbed by air turbulence, obscurations, and beam tracking errors. Air turbulence occurs due to heat diﬀerences between air cells on the FSOC optical beam route[26]. Obscurations are caused by clouds, rain, dust, birds, and so on. Beam tracking errors occur when an FSO antenna moves or vibrates because of wind or earthquakes, or moves by itself. Packet loss of 10% or more often occurs in FSOC.

Tracking errors sometimes cause FSOC to lose its optical beam signals, and the loss duration may range from several milliseconds to as much as a few seconds. Clouds may make packet loss duration last from several ten seconds to as much as several hours. We consider packet loss duration up to several ten seconds.

Satellite / Airplane Ground station Data center

FSOC downlink

FSOC/RF uplink The

Internet (100 Mbps ~ 100 Gbps)

(1 Mbps ~ 100 Gbps)

(45)

CHAPTER 4. TCP-FSO

TCPProxy

TCP TCP-FSO IP Host

TCP-Booster

Host Host

IP

Eth switch Eth switch Eth

Eth

TCPProxy TCP

IP TCP-Booster

IP

Eth Eth

Transponder FSO antenna

Transponder FSO antenna FSOC

Host Host Host

Satellite / airplane Ground station

Host Host Host Data center

Internet The TCP-FSO

Figure 4.2: Layering block of end-to-end data transfer with FSOC.

Figure 4.2 is a block diagram of an end-to-end network system. We assume split-TCP data transfer by a TCP-Booster in a satellite and a ground station. The proposed TCP- FSO is used between TCP-Boosters. We assume some hosts share FSOC links so that many TCPs send data to FSOC links concurrently. Some packets are transferred in the IP layer in TCP-Boosters. We assume that some hosts are connected to each Ethernet switch.

For achieving high-speed data transfer in FSOC, the following critical issues should be solved.

1) Congestion control on high-speed large-delay network

It is difficult for TCPs to avoid congestion in high-speed, large-delay networks because their control is based on a feedback loop with a large delay between a TCP sender and a receiver. Furthermore, since the other traffic may occupy network buffer resources quickly on high-speed networks, it is quite difficult for TCPs to share bandwidth quickly and efficiently with other TCPs.

2) Retransmission on high packet loss rate network

TCP data transfer often stalls temporarily when packet loss rate is so high. During the fast-retransmission phase, once TCP sent all data in congestion window, then TCP increase its congestion window just as large as acknowledged data size. However, the acknowledged data size is very small because TCP retransmits insuﬃcient size of data once.

A new technique is needed to prevent data transmission stall during fast-retransmissions.

(46)

CHAPTER 4. TCP-FSO

3)Recovery from long-time signal lost

In recovering from FSOC signal loss, a sender TCP cannot know the accurate sequence number to retransmit, because the last ACK packet may not represent the data that arrived at a receiver. In large delay networks, there is a signiﬁcant diﬀerence between the last ACK packet and the data that have arrived. Another problem is that a sender is only a trigger for recovery. It takes a considerable delay in the recovery to start.

4)Asymmetric bandwidth

TCPs use a lot of ACK packets during data transfer. For example, Linux TCP uses around 30Mbps ACK traﬃc for 10 Giga bps data transfer. If ACK throughput exceeds link capacity, its queuing delay aﬀects data transfer throughput. On the other hand, fewer ACKs also cause a delay before ACK transmission. It results in longer response time and lower throughput. So, ACK rate should be adjusted, considering queuing delay properly.

Studies of High-Speed TCP/IP Data Transfer Techniques for Heterogeneous Networks with High-Speed Wireless Communications

Studies of High-Speed TCP/IP Data Transfer Techniques for

Heterogeneous Networks with High-Speed Wireless

Communications

高速無線通信を含む多様なネットワークのための 高速 TCP/IP データ転送技術の研究

February 2018

Yohei HASEGAWA

長谷川 洋平

Studies of High-Speed TCP/IP Data Transfer Techniques for

Heterogeneous Networks with High-Speed Wireless

Communications

高速無線通信を含む多様なネットワークのための 高速 TCP/IP データ転送技術の研究

February 2018

Waseda University

Graduate School of Fundamental Science and Engineering Department of Computer Science and Communications Engineering

Research on Image Information

Yohei HASEGAWA

長谷川 洋平

Abstract

Contents

List of Figures

List of Tables

Preface

Chapter 1 Introduction

1.1 Background and Motivation

1.1.1 TCP for High-speed wireless

1.1.2 Optimizing Split-TCP data transfer

1.1.3 Concurrent multi-path data transfer

1.1.4 Ethernet topology discovery

1.1.5 Summary

1.2 Contributions

1.3 Thesis Organization

Chapter 2

Related Work

2.1 Overview

2.2 Transport Protocols

2.2.1 High-speed TCPs

2.2.2 Multi-layer Congestion Controls

2.2.3 UDP based Protocols

2.3 Overlay Networks

2.4 Multipath Data Transfer Techniques

2.5 Topology Discovery Techniques

2.6 Summary

10

10

10

10

10

10

TCP-FSO SCPS-TP

High-speed TCPs Cubic

UDP based protocols UDT

Saratoga

Chapter 3

TCP-Booster: A Split-TCP Data Transfer System 1

3.1 Overview

3.2 TCPProxy

3.3 TCP

3.4 IP/Ethernet

3.5 ParamManager

3.6 Tcptables

3.7 ModelController

3.8 Summary

Chapter 4

TCP-FSO: A TCP for Long-Distance High-Speed Wireless

Communications 1

4.1 Overview

4.2 Target environments and issues

高速無線通信を含む多様なネットワークのための高速 TCP/IP ^{データ転送技術の研究}

長谷川洋平

高速無線通信を含む多様なネットワークのための高速 TCP/IP ^{データ転送技術の研究}

長谷川洋平

TCP-Booster: A Split-TCP Data Transfer System ¹

Communications ¹