Tracking missions

6.5.2.1 Pedestrian tracking

There are two ways of tracking in this system (i.e., mobile tracking and image processing tracking). The tracking process is an estimation of the history of an object’s motions. It generates a path history of the moving object. The position of the assigned object must be known before starting tracking. Therefore, the object must be recognizable in previous frames. Author used UAV has the same flight controller in Figure 6.4 and uses the same protocols (i.e., MAVlink) to execute visual tracking mission. Generally, a visual tracking system depends on camera resolutions, distance, frame rates, and brightness. Therefore, UAV camera specs and environment can affect the tracking process performance. In the present study, the UAV camera has a resolution of 640x480 and 30fps. Image detection modes in the tracking process can be described as below:

• Kalman Filter: to reduce the noise that initiated by inaccurate and noisy data input inside the video feeds. Author used this filter with OpenCV and the main classifier.

• Haar-cascade: it uses as rapid classifier to detect and train object object. This study uses upper body and full body detection. It is an effective object detection method proposed by Paul Viola and Michael Jones [Aljehani, 2016a].

• Color Detection: to detect skin colors at different brightness levels, author changed the red-green-blue color space into luminance-blue-difference-red-difference (YCbCr) color space, which also detects specific colors.

• Particle Object: this detection mode is based on previously saved frames and special features of the object. However, this mode proved error-prone and unreliable in human detection.

• Human Upper Body: Haar-cascade upper part of the human body (size range: 4 meter) in 640x480 resolution video feed with 30 fps.

• Human Full Body: Haar-cascade whole length of the human body (size range: 6 meter) in 640x480 resolution video feed with 30 fps.

Full-body and upper-body were processed in the tracking ranges of six, five, and three meters, respectively. In all of these ranges, the tracking has been optimized by Kalman filter-ing and Haar-cascade algorithms in OpenCV to increase the reliability of the trackfilter-ing process and ensure UAV does not follow a random object in the sight of UAV camera [Barták, 2015].

As the tracking can be affected by the camera’s quality and the brightness level, the author implemented full-body and upper-body detection with skin-color detection in the present experiments [Aljehani, 2016a]. The tracking experiments have been verified through actual flight using the process of image tracking as shown in Figure 6.5. A number of reassignments points have occurred during the tracking process. The algorithms that author has used in tracking mission are shown in Figure 6.6.

UAV Images

Coordinate Target History Haar Cascade

Kalman Filter

PID Control

Figure 6.5: Autonomous tracking using kalman algorithms and Haar cascade.

Face to Upper body Full-body detection Multiple Filters (i.e., Non-Maxima Suppression Feature) with

Full-body detection Figure 6.6:Tracking System using OpenCV and ROS.

Figures 6.7-6.11 illustrate the autonomous tracking trajectories and checkpoints during the experiments of actual flight in an indoor environment. The author got the UAV tra-jectories from UAV sensors that send the value of pitch, yaw, and roll. These values can be displayed as a map of the flight trajectories. However, human trajectories (target) from smartphone application (participatory sensing) [Kusano, 2013]. Moreover, the author used the camera in the back (see Figure 6.8) to record the trajectory of the target and UAV. During upper-body tracking, the UAV lost the target three times due to the short distance between the current and the previous target-detection checkpoints. Two of these incidences occurred at the same point. During full-body tracking, the UAV remained at more than four meters from the target, and the image was more easily retained than the upper-body tracking. Con-sequently, the target was lost only once. The performance of full-body tracking improved in combination with color detection, even when the target made critical movements (see Figure 6.11). These three scenarios confirmed the ability of the UAV-based system to track pedes-trians and generate routes by image processing and low video resolutions in the absence of smart devices tracking system. Using many algorithms can enhance the tracking process.

However, human variation and monitoring are significant for accurate tracking results. The height of the UAV can be higher if the UAV has a camera gimbal. Figure 6.7 demonstrates the full-body tracking process. Pis the destination ofA(target),Brefers to UAV andBξA

is the distance between UAV and the target, it must be least than tracking threshold, and that depends on the UAV camera specifications. His the height of the UAV. In the present study, the tested UAV camera was fixed on the front of the UAV. Therefore, it must be in four meters height or less as long as the full-body is detectable. AξPis the target direction to a destination ofP. These parameters are varied from UAV model to another.

45^o 0^o

walking Standing

A ξ _P

Cr H

{P}

{B}

H {A}

{A}

{B}

Bξ A< Threshold

Figure 6.7:Image processing tracking system.

Figure 6.8:Image tracking in real flight experiment.

Figure 6.9:The result of full-body tracking by the UAV system after recording the performance through camera and GPS in real flight

experiment.

Figure 6.10: Upper-body tracking by the UAV system result after recording the performance through camera and GPS in real flight

experiment.

Figure 6.11: Upper-body with skin-color detection tracking by the UAV system after recording the performance through camera and

GPS in real flight experiment.

6.5.2.2 Mobile tracking

Mobile tracking is an essential tracking process in this work. After a disaster occurs, trians’ smart devices can provide GPS and accelerometer data to give a history of the pedes-trians taken routes. These routes can be across buildings or private properties. The only issue with this method is that there is no image to evaluate the taken paths. So, author can use mobile tracking data with scanning UAV and tracking UAV which tracks human with an absence of mobile to evaluate the area. It also can be used by Follow-me feature by fol-lowing mobile GPS data. Then, UAV can autonomously cover that route and provide its images. So, the pilot can have two important tracking data; i.e., the image of the route and pedestrians’ taken routes to evaluate and build the final safe map (see Figure 6.12). Author used an android device to be tracked and MAVlink message to start the Follow-me process.

Also, the pilot can configure Follow-me behaviors using the following parameters:

• NAV_FT_MIN_HT: Vehicle follow-me height relative to the home or arm flight modes.

The default and minimum height is range of eight meters.

• NAV_FT_FS: Flight position relative to the user when follow-me mode is operated.

• 0 = Follow from the front right of the user.

• 1 = Follow from behind or trail the user (Default).

• 2 = Follow from the front of the user.

• 3 = Follow from the front left of the user.

Device tracking

Aerial images

Figure 6.12:Mobile tracking and UAV surveying.