Abstract — In this paper, a method f design oriented to indoor and out navigation is illustrated. In order to provi the solution here presented, a brief discu drawbacks of state-of-the-art technolo Finally, an application of such a method navigation system for blindfolded people is Keywords — Autonomous Navigation, D cost. I. INTRODUCTION HERE are a lot of systems propose that face the issue of autonomous n for several purposes, such as guidance [1]–[6], UAVs (Unmanned Aerial V robotic applications [9]–[12], and also in the quality of life of visually impaired The methods proposed are based on diff such as GPS (Global Positioning (Infrared) [9], [13], RFID (Radio Freque [5], [10], [16], Ultrasonic [12], [16], an various technologies, including G Navigation System) [1]–[4], IR-Ultras Ultrasonic [6], [11], Ultrasonic-INS [8], However, these technologies present For example, the accuracy of GPS is operating indoors due to limited satell moreover, today’s GPS based navigation not function well at cities with ta Ultrasonic cannot provide high angular the wide beam angle (which can be ab even wider) [8]; RFID systems are ge and attempting to read several tags at a signal collision and ultimately to furthermore, RFID readers cannot be location [18]. Some limitations can b systems based on sensor fusion (e.g., IR- Ultrasonic, etc.), but that involves co expensive systems. Corresponding author: Gianluca Susi is wi Electronic Engineering, University of Rome “T Politecnico 1, 00133 Rome, Italy (e-mail: gianluca Alessandro Cristini is with the Department of E University of Rome “Tor Vergata”, Via del Polite Italy (e-mail: alessandro.cristini@students.uniroma Mario Salerno is with the Department of E University of Rome “Tor Vergata”, Via del Polite Italy (salerno@uniroma2.it). Emiliano Daddario is with the Department of E University of Rome “Tor Vergata”, Via del Polite Italy (emiliano182@yahoo.it). A low-cost i autono Gianluca Susi, Member, IE T for low-cost system tdoor autonomous ide a motivation for ussion of the typical ogies is reported. for the design of a s shown. Depth camera, Low- ed in the literature navigation, applied e of land vehicles Vehicles) [7], [8], n order to improve people [13]–[16]. ferent technologies, System) [7], IR ency Identification) nd combinations of GPS-INS (Inertial sonic [14], RFID- etc. t some drawbacks. not reliable when ite reception [10], n systems often do all buildings [5]; r resolution due to bout 30 degrees or enerally expensive time may result in data loss [17]; installed at every e overcome using -Ultrasonic, RFID- omplex and often th the Department of Tor Vergata”, Via del a.susi@uniroma2.it). Electronic Engineering, ecnico 1, 00133 Rome, a2.eu). Electronic Engineering, ecnico 1, 00133, Rome, Electronic Engineering, ecnico 1, 00133, Rome, A typical autonomous nav outlined as follows: 1. A sensing unit (SU, e.g., dep 2. A processing unit (PU); 3. A feedback/control unit (FC headphones, etc.). In addition, in order to realiz even in outdoor scenarios, a should be used. Fig. 1. Scheme of a typic portable system. In the last years, many works field of autonomous navigation are used as sensing units [19]–[ In this work, a low-cost mo terrestrial navigation, based presented. We will introduce technologies, and then we will p in the design phase, making problems arising from the techn Finally, a prototype aimed a navigation will be presented, an in terms of performance and co II. STRUCTURED LIGHT AND T DEPTH CAM As a newly developing categ hardware, the depth camera te epoch for 3D geometric inform of applications like geometry re and human motion tracking, ca fast analysis of 3D scenes. Unt of a depth camera was approxim SL (Structured Light) depth available on the large-scale products based on this techn indoor and outdoor terr omous navigation mode EEE, Alessandro Cristini, Mario Salerno, and Emi vigation system could be th camera, ultrasonic, etc.); CU, e.g., actuators, displays, ze a portable system, usable an on-board power supply cal autonomous navigation s have been proposed in the n, in which depth cameras 21]. odel for indoor and outdoor on depth data, will be today's available low-cost present a method to be used possible to avoid typical nologies taken into account. at human blind autonomous nd the results will be shown st. TIME-OF-FLIGHT CONSUMER MERAS gory of distance measuring echnologies opened a new mation acquisition. A plenty econstruction, mixed reality an be obtained thanks to the til a few years ago, the cost mately 10 k€; from 2010 the h sensing technology is market, and then many nology have been released restrial el iliano Daddario (e.g., Microsoft Kinect v.1, Asus Xtion, etc.). The subsequent cost reduction (approximately 0.1 k$) together with their sensing range and compatibility, have allowed the use of RGB-D (RGB-Depth) input devices in many everyday applications, such as gaming, biomedical field, art and many others. Also, there has been a remarkable widespread use of low-cost sensing technologies on interesting navigation tasks (e.g., indoor blind navigation, underground exploration, autonomous mapping of buildings, night vision, or remote exploration of dangerous areas [22] ). Limitations of these kinds of approach have been stressed in the literature [23], and some of these can lead to specific problems (e.g. disparity holes for transparent, shiny or matte and absorbing objects, sunlight interference) [24]–[25]. These drawbacks can be reduced by proper techniques and algorithms, which usually exploit the large amount of information that it is possible to obtain through the devices by data fusion (e.g., multiple IR camera configurations and RGB-Depth data fusion) [26], [27], [28]. In these years, consumer depth-cameras based on other technologies have appeared on the consumer market. In particular, ToF (Time-of-Flight) technology (e.g., Senz3d, Kinect v2.0) is becoming affordable (less than 0.2 k$). Furthermore they are able to show better performances in some scenarios: higher SNR and resolution, higher sensing range, no depth shadow due to the single viewpoint. In recent models, a better response to varying lighting conditions is provided thanks to the integration of ambient light rejection filters. ToF cameras are the new attractive candidates to be used in low-cost depth sensing based systems. III. THE MODEL In this section a low-cost model is introduced; despite the presence of the mentioned drawbacks, the aim of this work will be to preserve the reliability for navigation tasks. For this purpose, different design choices will be identified, in regard to the specific application. A. Hardware The proposed model is shown in Fig. 2. With the aim of schematizing the information/power flow, the system will be described as a particularized version of the general one as follows. It has been depicted as a set of seven blocks properly connected: sensing b. (block 1), processing b. (block 2), media encoding b. (block 3), actuator driver b. (block 4), media output b. (block 5), actuator b. (block 6), on-board power supply b. (block 7). Block 1 is the root of the information flow; it consists of a SL or ToF RGB-D input device, allowing depth mapping in absence of light. For outdoor navigation, because of the sunlight interference, a new-generation consumer ToF camera is suggested (as the Kinect v.2 that implements an integrated ambient light rejection filter). It is directly connected to block 2. Because of the high data throughput generated by this CV (Computer Vision)-based system, a wired connector is needed between them. One of the most common connector types is USB 2.0, or 3.0 (e.g., Kinect v.2 for Windows). Fig. 2. Hardware scheme of the navigation aid model; colors are related to that in the previous figure. Block 2 should consist of a small embedded Linux computer, for high performance computing and increased portability. This block numerically processes the input stream through the processing chain (explained in the next subsection), in order to compute the current internal status of the system, and then it controls block 3 and block 4 accordingly. These can either be featured inside the same Linux computer, or consist of external boards (e.g. block 3 can be a programmable audio DSP). Block 3 is responsible for converting the received system status into a meaningful sound/video output for the user (e.g., DAC, hardware acceleration). For maximum comfort, Bluetooth devices can be used as the media output (i.e., block 5). Block 4 is responsible for driving block 6. During the prototype phase, a GPIO (General Purpose Input/Output) interface between them is often used. Block 4 can be a different controller from block 2, for example a fast prototyping board (e.g., Arduino) connected to it via wired serial interface or wireless communication interface (Wi-Fi for easy setup, Bluetooth Low Energy for extended battery life). However, some popular embedded Linux computers include GPIO interfaces for physical computing (e.g., BeagleBone Black), then including block 2 and block 4. Block 6 is responsible of providing tactile and/or proprioceptive feedback to the end user, also acting as a haptic display used to convey visual information. In Fig. 2, the latter is represented by a stepper motor with a threaded shaft used as a linear actuator, but other solutions can be taken into account. In non human-oriented systems the FCU can consist of a control block connected to the subsequently device. Block 7 guarantees portability. Of course, some blocks (such as wireless headphones for block 5) may use their own battery, especially for wireless operation; other blocks (e.g., block 1) can be powered from block 2 via USB (e.g., Xtion). B. Software The choice of a consumer RGB-D input device as sensing unit implies a lot of advantages: above all, the chance to realize a low cost implementation. However, this kind of devices presents peculiarities that can lead to specific problems when applied in navigation tasks. In order to overcome these typical probl chain (implemented by block 2) is provid Fig. 3. The processing chain schema which the processing block elaborate autonomous navigation model proposed. The processing chain can be divided follows. The CS (Correction Step) is an elab implements algorithms aimed at recover data, as in the case of reflective, tra surfaces, depth discontinuity, or sun Note that, this block is crucial for the co avoidance of obstacles on a route. Man be used for limiting this problem. For ex disparity holes, a joint-bilateral filter applied to depth pixels, through the con data, depth information and a temporal as shown in [29]; to recover sunlight in cameras the method proposed in [30], b and depth information, can be applie using both RGB and Depth data are rep [31]; note that, the need of RGB d approaches not compatible with dark co This limitation could be overcome structured light with IR stereo technique [26]. The latter solution is very effectiv couple of devices, and then an appropria process to avoid interference. The dev also used with the aim of extendin improving the resolution and overco Another method used in order to avoi among cameras consists of giving a motion to the sensors [27], [28]. Wit making an appropriate choice among th mentioned, considerations about comp and, on the other hand, complexi introduced by the different approaches into account. The PS (Processing Step) operates reference areas from the depth map information from a set of pixels many used, in relation to the specific appl evaluation of the average or the maximu can be made for a low level feed encoding; also, tracking/recognition a introduced at this step. A review of such in [31]. ems, a processing ded. atizes the steps by s the data in the . d in four steps, as boration block that ring missing depth ansparent or matte nlight interference. orrect detection and ny approaches can xample, to recover can be iteratively nsideration of RGB l consistency map, nterference in ToF based on RGB data ed. Other methods ported in reference data makes these ondition scenarios. e by combining es, as illustrated in ve, but it requires a ate synchronization vice overlapping is ng the coverage, oming occlusions. id the interference small amount of th the purpose of he solutions above puting capabilities ity and latencies s have to be taken s the grouping of flow. To extract y methods can be lication: a simple um on sets of pixels dback information algorithms can be h methods is listed The ES (Encoding Step) con code, that can be used for audi people), or video (e.g., navig feedback. A proper choice of used is necessary. For exampl applications, the audio feedbac phenomena on natural inform environment. For instance, phenomena, for the case applications a tactile feedback i The AS (Adapter Step) pro the information to the user se realize the normalization/comp linear actuators, regulation of different audio-coded items or t adaptation phenomena (audio/ta IV. IMPLEMENTATIO With the aim of evaluating model, a low-cost prototype sy autonomous navigation has bee principles illustrated above, and The protoype is composed as 1); Raspberry Pi (block 2 and b 5). On the basis of the acquir generates proper acoustic feedb of both an accurate analysis an depth map. DSP software based engine, SimpleOpenNI and Op Libraries, has been used for described in [32], but addi implemented as AS, in order amplitudes related to near obje by an on board battery (block 7) We have arranged four wa proper obstacle sets placed in room, preserving the same dif [32]. The system synthesizes th to the obstacle detection for a g five blindfolded people (four tr made, and a first set of TT (Number of Collisions) have obtained show satisfactory resu adaptation of the individuals to the other hand, NoC results alm In a second phase, we have r with the tactile one, repeating information by means of a be actuators (block 6) controlle Similar performance in terms but NoC performance have be four consecutive trials performe different walking paths, The p in Table 1. The total system cos TABLE 1: AVERAGES OF TT CONFIGURATIONS, A (AUD Conf. Trial n.1 Trial n.2 TT(A) 167 s 150 s TT(T) 190 s 146 s NoC(A) 4.5 5 NoC(T) 4.75 3.25 nverts the information into a io and/or tactile (e.g., blind gation in dark conditions) the kind of feedback to be e, in some human-oriented ck could result in masking mation coming from the in order to avoid these of sightless navigation is preferred. vides the representation of enses. For example, it can pression on the excursion of f the amplitudes related to the compensation of human actile memory effect). ON AND RESULTS g the effectiveness of the ystem aimed at human blind en realized using the design d tested on a specific task. s follows: Kinect v.1 (block block 3); headphones (block red depth data, the system backs, synthesized by means nd the interpretation of the d on PureData for the sound pen Sound Control OSCp5 the processing chain, as ing a compression curve r to emphasize the sound ects. The system is supplied ). alking paths by means of different ways in a closed fficulty for each path as in he acoustic feedback related given path. Sets of trials on rials per people) have been (Travel Times) and NoC been obtained. The data ults in terms of TT, i.e., an o the proposed method. On most constant. replaced the audio feedback g the trials, providing the elt array composed of four d by Arduino (block 4). of TT have been obtained, en improved. In relation to ed by the individuals on the performance is summarized st is reported in Table 2. T AND NOC FOR THE TWO DIO) AND T (TACTILE). 2 Trial n.3 Trial n.4 65 s 68 s 101 s 80 s 3.75 4.25 2.75 2 TABLE 2: SYSTEM COSTS Device Approx. Cost ($) Kinect v.1 130 Headphones 20 Arduino 60 Raspberry Pi 75 Battery 20 Linear actuators + el. components 25 TOTAL 330 Future improvements will consist in making the system able to work at a higher depth range, and increasing robustness to strong-light conditions. The best solution could be data fusion obtained by new generation consumer ToF RGB-D cameras (e.g., Kinect v.2). The cost of these kind of devices remain affordable. Both the plenty of applications that can be realized and the better quality of the obtainable information, result in a very attractive approach. REFERENCES [1] M. A. Jaradat, M. F. Abdel-Hafez, K. Saadeddin, and M. A. Jarrah, “Intelligent Fault Detection and Fusion for INS/GPS Navigation System,” in Proc. of the 9th Int. Symposium on Mechatronics and its Applications (ISMA13), Amman, Jordan, April 9-11, 2013. [2] S. Zhao, Y. Chen, H. Zhang, and J. A. Farrell, “Differential GPS aided Inertial Navigation: a Contemplative Realtime Approach,” in 19th International Federation of Automatic Control (IFAC) World Congress, Cape Town, South Africa, August 24-29, 2014, pp. 8959-8964. [3] M. Ilyas, Y. Yang, Q. S. Qian, and R. Zhang, “Low-cost IMU/Odometer/GPS Integrated Navigation Aided with Two Antennae Heading Measurement for Land Vehicle Application,” in 2013 25th Chinese Control and Decision Conference (CCDC), 2013, pp. 4521-4526. [4] Y. Wu, “Versatile Land Navigation Using Inertial Sensors and Odometry: Self-calibration, In-motion Alignment and Positioning,” in Inertial Sensors and Systems - Symposium Gyro Technology (ISS-SGT), Karlsruhe, Sept.16 to 17, 2014. [5] W. Cheng, S. Wang, and X. Cheng, “Virtual Track: Applications and Challenges of the RFID System on Roads,” IEEE Network, Jen.-Feb. 2014, pp. 42-47. [6] M. Ragul and V. Venkatesh, “Autonomous Vehicle Transportation Using Wireless Technology,” International Journal of Engineering and Technology (IJET), vol. 5, no. 2, pp. 811-819, Apr-May 2013. [7] M. Garg, A. W. Ansari, S. Choudhury, R. Singh, and R. Garg, “Wireless Transmission of GPS Values for UAV’s navigation using 802.15.4b Lan Standard Protocol (Zigbee),” Global Journal of Computer Science and Technology Network, Web & Security, vol. 13, Issue 17, pp. 15-20, 2013. [8] L. E. Muñoz, P. Castillo, and P. Garcia, “Observer-control scheme for autonomous navigation: Flight tests validation in a quadrotor vehicle,” in 2013 Int. Conf. on Unmanned Aircraft Systems (ICUAS), May 28-31, 2013, Grand Hyatt Atlanta, Atlanta, GA, pp. 795-804. [9] Y. Do and J. Kim, “Infrared Range Sensor Array for 3D Sensing in Robotic Applications,” Int. J. of Advanced Robotic Systems, vol. 10, pp. 1-9, 2013. [10] B. Olszewski, S. Fenton, B. Tworek, J. Liang, and K. Yelamarthi, “RFID Positioning Robot: An Indoor Navigation System,” in 2013 Int. Conf. on Electro/Information Technology (EIT), 9-11 May 2013, pp. 1-6. [11] S. Arulselvi, “Robot Navigation System with RFID and Ultrasonic Sensors,” Middle-East J. of Scientific Research, vol. 20, no. 9, pp. 1133-1137, 2014. [12] J. Lim, S. J. Lee, G. Tewolde, and J. Kwon, “Ultrasonic-Sensor Deployment Strategies and Use of Smartphone Sensors for Mobile Robot Navigation in Indoor Environment,” in 2014 IEEE Int. Conf. on Electro/Information Technology, 5-7 June 2014, pp. 593-598. [13] A. S. Al-Fahoum, H. B. Al-Hmoud, and A. A. Al-Fraihat, “A Smart Infrared Microcontroller-Based Blind Guidance System,” Active and Passive Electronic Components, Hindawi Publishing Corporation, pp. 1-7, 2013. [14] F. Prattico, C. Cera, and F. Petroni, “A new hybrid infrared- ultrasonic electronic travel aids for blind people,” Sensors and Actuators A, vol. 201, pp. 363-370, 2013. [15] R. Fayad and A. Hage-Diab, “Design of an Ultrasonic Auditory- Vision Teaching Device,” in 2013 2nd Int. Conf. on Advances in Biomedical Engineering (ICABME), 11-13 Sept. 2013, pp. 152-154. [16] M. S. AlZuhair, A. B. Najjar, and E. Kanjo, “NFC based applications for visually impaired people - A Review,” in 2014 IEEE Int. Conf. on Multimedia and Expo Workshop (ICMEW), 14- 18 July 2014, pp. 1-6. [17] M. Kaur, M. Sandhu, N. Mohan, and S. Sandhu, “RFID Technology Principles, Advantages, Limitations & Its Applications,” Int. J. of Computer and Electrical Engineering, vol. 3, no. 1, pp. 151-157, 2011. [18] T. K. Dao, H. L. Hguyen, T. T. Pham, E. Castelli, V. T. Nguyen, and D. V. Nguyen, “User Localization in Complex Environments by Multimodal Combination of GPS, WiFi, RFID, and Pedometer,” The Scientific World Journal, Hindawi Publishing Corporation, pp. 1-7, 2014. [19] D. Maier, A. Hournung, and M. Bennewitz, “Real-Time Navigation in 3D Environments Based on Depth Camera Data,” in 2012 12th IEEE-RAS Int. Conf. on Humanoid Robots (HUMANOIDS), Osaka, Japan, Nov. 29 to Dec. 1, 2012, pp. 692-697. [20] J. Biswas and M. Veloso, “Depth Camera Based Indoor Mobile Robot Localization and Navigation,” in Proc. of IEEE International Conference on Robotics and Automation, May, 2012, pp. 1697- 1702. [21] D. Maier, C. Lutz, and M. Bennewitz, “Autonomous Biped Navigation Through Clutter,” in Proc. of 2nd workshop on robots in clutter: preparing robots to the real world, 2013. [22] A. Oliver, S. Kang, B. C. Wünsche, and B. MacDonald, “Using the Kinect as a Navigation Sensor for Mobile Robotics,” in Proc. of the 27th Conference on Image and Vision Computing (IVCNZ ’12), New Zealand, 2012, pp. 509-514. [23] M. R. Andersen, T. Jensen, P. Lisouski, A. K. Mortensen, M. K. Hansen, T. Gregersen, and P. Ahrendt, Kinect Depth Sensor Evaluation for Computer Vision Applications. Technical report ECE-TR-6. Department of Engineering – Electrical and Computer Engineering, Aarhus University, Denmark, 2012. [24] I. Lysenkov and V. Rabaud, “Pose Estimation of Rigid Transparent Objects in Transparent Clutter,” 2013 IEEE Int. Conf. on Robotics and Automation (ICRA), Karlsruhe, Germany, 6-10 may 2013, pp. 162-169. [25] N. Alt, P. Rives, and E.Steinbach, “Reconstruction of transparent objects in unstructured scenes with a depth camera,” in 2013 20th IEEE Int. Conf. on Image Processing (ICIP), Melbourne, Australia, 15-18 Sept. 2013, pp. 4131-4135. [26] F. Alhwarin, A. Ferrein, and I. Scholl, “IR Stereo Kinect: Improving Depth Images by Combining Structured Light with IR Stereo, ” in Proc. of the 13th Pacific Rim International Conference on Artificial Intelligence (PRICAI 2014), Gold Coast, QLD, Australia, Dec. 1-5, 2014, vol. 8862, 2014, pp 409-421. [27] A. Maimone and H. Fuchs, “Reducing interference between multiple structured light depth sensors using motion, ” in 2012 IEEE Virtual Reality Short Papers and Posters (VRW), Costa Mesa, CA, USA, 4-8 March 2012, pp. 51-54. [28] A. Butler, S. Izadi, O. Hilliges, D. Molyneaux, S. Hodges, and D. Kim, “Shake’n’Sense: Reducing Interference for Overlapping Structured Light Depth Cameras,” CHI '12 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, New York, NY, USA, 2012, pp. 1933-1936. [29] N. Silberman and R. Fergus, “Indoor scene segmentation using a structured light sensor,” in 2011 IEEE Int. Conf. on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain, 6-13 Nov. 2011, pp. 601–608. [30] M. Lindner, I. Schiller, A. Kolb, and R. Koch, “Time-of-flight sensor calibration for accurate range sensing, ” Computer Vision and Image Understanding, vol. 114, Issue 12, pp. 1318-1328, 2010. [31] J. Han, L. Shao, D. Xu, and J. Shotton, “Enhanced Computer Vision with Microsoft Kinect Sensor: A Review, ” IEEE Transactions on Cybernetics, vol. 43, Issue 5, pp. 2171-2172, 2013. [32] M. Salerno, M. Re, A. Cristini, G. Susi, M. Bertola, E. Daddario, and F. Capobianco, “AudiNect: An Aid for the Autonomous Navigation of Visually Impaired People, Based On Virtual Interface,” Int. J. of Human Computer Interaction (IJHCI), vol. 4, Issue 1, pp. 25-33, 2013.