«UNIVERSITY OF CALIFORNIA RIVERSIDE Mobile and Stationary Computer Vision based Traffic Surveillance Techniques for Advanced ITS Applications A ...»
UNIVERSITY OF CALIFORNIA
Mobile and Stationary Computer Vision based Traffic Surveillance
Techniques for Advanced ITS Applications
A Dissertation submitted in partial satisfaction
of the requirements for the degree of
Doctor of Philosophy
Professor Matthew Barth, Chairperson
Professor Gerardo Beni
Dr. ZuWhan Kim Copyright by Meng Cao The Disserlationof Meng Cao is approvedby CommitteeChairperson Universitvof California.Riverside Acknowledgments First of all, I would like to express my deep gratitude to my advisor Professor Matthew J. Barth, for the source of knowledge, guidance and support that he has been to me over the past years. His encouragement of my work and commitment to whatever project I would undertake was inspiring, and has made the time I have spent here at UC Riverside enjoyable. His sense of humor, the optimistic view toward life, as well as the talent of seeing the big picture, will benefit me in all my life. Without his help, I would not have been here.
I would also give my appreciation to Dr. ZuWhan Kim, my supervisor while I was staying at California PATH, for his deep insight, extensive knowledge and his ongoing effort of extending and improving his great software package, Cvzulib. He has been a tremendous mentor, collaborator, friend, and even a brother for me. I own him a great many heartfelt thanks.
I am also grateful to my committee member Professor Gerardo Beni, for the helpful discussion, patient teaching and the encouragement since the first year in my graduate school. I had the luck to find with Dr. Xiaoyun Lu and Dr. Steven Shladover two extraordinary personalities as my co-supervisors, in the time I was staying at California PATH. With their motivation, dedication, and love for their work and life, they certainly did not teach me only “engineering”. Some of the work I present here is based on team iv work with Dr. ZuWhan Kim, Dr. Xiaoyun Lu and Dr. Steve Shladover. Thanks for everything.
I am happy to have had some very excellent colleagues, the first of whom that I would like to thank being Anh Vu, for his effort and help in designing and constructing the vision sensor system used in this thesis. Thanks to Mike Todd, for the kind help whenever I needed. His big smile is always the sign of the TSR group. My thanks also go to Dr. Jie Du, Dr. Weihua Zhu, Dr. Kanok Boriboonsomin, George Scora, Alex Vu, Lili Huang, Anning Chen, Ashkan Sharafsaleh and Susan Dickey. I could not have afforded this Ph.D program without financial support, and I therefore would like to acknowledge the sponsors: the Department of Electrical Engineering of UC Riverside, CE-CERT at UC Riverside, UC Transportation Center and California PATH.
I have had a very wonderful time in the past 5 year, largely due to my great friends here. You know who you are-- Xiaotao Zou, who picked me up when I first landed at United States and been my roommate for three years. Zijun Fang, Gang Liu, Weihua Zhu, Bi Song, Peilin Fu, Grace Gao, Xiaoyu Wu, Shania Huang, Daniel Kuang, Yilei Xu from UC Riverside, as well as Allen Yao, Yiguang Xuan, Jing Xiong and Xiaomeng Zhong from UC Berkeley. I cherish you all.
I would also appreciate my childhood-mates: Yan Xu, Zishi Feng, Chenxin Zhang and college-mates: Chao Liu and Jiafeng Guo, for our friendships and all the experience
Lei, I am honored that you have stood steadfastly with me through all the ups and downs of life. May our love continue to grow.
Thanks, my dear Dad and Mom. So less words, and so much to add. I will never forget the words you put in my heart on that cold night when I was in middle school. I am strong when I am on your shoulder. I love you, for ever.
During the past decade, new sensing technologies, such as inductive loops, laser range scanners, radar detectors and computer vision sensors have been greatly enhanced and applied to the Intelligent Transportation System (ITS) area. Among all these sensor systems, computer vision-based approaches are one of the most popular and promising techniques used in ITS for traffic evaluation and management, driver assistance, as well as other safety related research. This is primarily due to the advantages of easy maintenance, high flexibility, and low cost for traffic scene monitoring and analysis.
Many stationary vision sensors have been already installed near the roadway, particularly at intersections. In addition, more and more vision sensors are now being installed on mobile vehicles, in order to have real time surrounding traffic information. This dissertation focuses on both mobile and stationary computer vision based traffic surveillance techniques, including the development of a new vision sensor, a survey and
of ITS areas with high quantitative requirements. These areas are outlined in further detail below.
Portable Loop Fault Detection For many years, it has been difficult to quantitatively measure real-time freeway traffic conditions. Numerous research projects have been carried out in traffic surveillance; for example, the Freeway Performance Measurement System (PeMS) operated by Caltrans and UC Berkeley collects, processes, aggregates, and examines traffic data primarily through loop detectors buried beneath the freeway. This type of stationary embedded loop sensor system provides a point measurement for the traffic flow, roadway occupancy and average speed. By aggregating these directly measured traffic data, they can be used to estimate and provide a larger picture of the traffic conditions in certain area. However, the results obtained from embedded loop sensors are not entirely reliable. The embedded loop data delivered to a Traffic Management Center (TMC) may contain errors at one or more sensors, and between the loop detector and the TMC database. As a result, loop fault detection is important. In this dissertation, a stationary-vision based technique has been developed as part of a Portable Loop Fault Detection Tool (PLFDT). This work is complementary to recent research focusing on aggregated faulty loop data at a macroscopic level (the macroscopic level generally
intersections)). The objectives of the PLFDT is to develop a real time, multi-lane, multi-vehicle tracking system for freeways using video cameras as the baseline measurement technique to compare the loop detection signal for direct fault detection for inductive loop system.
Localized Traffic Density Measurement The embedded loop sensor system provides a direct measurement to traffic flow, roadway occupancy and average speed (only for double-loop detector). This type of sensor network does not directly measure traffic density; instead it can only be estimated.
In this dissertation, we have developed systematic techniques to measure traffic conditions by utilizing both on- and off-board computer vision systems. A unique development technique is a combined computer vision and Global Positioning System (GPS) equipped mobile traffic surveillance system to measure localized traffic density. In addition, we correlate the localized density measurement with estimates from embedded loop sensor system using a space-time diagram. Experiments have shown the complementary nature of these sensing techniques. Further, most traffic surveillance computer vision algorithms and techniques are typically based on observing vehicles from stationary rectilinear cameras mounted near roadways. For many applications, some of the key tasks include extracting traffic information such as average traffic speed, flow,
localized traffic map around specific vehicles in the traffic stream. In this dissertation, we have developed a vision-equipped vehicle test bed for traffic surveillance purposes and have experimentally demonstrated the generation of localized traffic density from video processing and synthesizing. In contrast to the off-board surveillance systems (e.g.
embedded loop sensor networks and stationary vision monitoring system), this type of on-board surveillance system provides a temporal- and spatial- continuous measurement of the localized traffic density.
One of the key components developed is an Orthogonal Omni-directional Vision (OODV) System that has been developed to observe lane-level activity surrounding a vehicle, as well as the ability to observe the surrounding roadway geometry. This vision system uses a special catadioptric mirror providing a 360 degree orthogonal view of the environment. It is different from other catadioptric mirror-based Omni-directional vision systems in that it directly provides an orthogonal image without the need of warping a polar-coordinate based image to a perspective view. Based on this unique OODV, a roadway traffic surveillance system was designed and implemented. It consists of three
A GPS stamped roadway traffic data collection technique;
Post video processing that includes automatic vehicle detection/tracking with the
A traffic parameter (localized density) estimation process.
Combined with a GPS receiver that provides approximately 2 - 3 meters spatial resolution, this traffic surveillance system can be applied not only in several traffic applications which require localized traffic density/flow/average speed measurements, but also in some other applications that require detailed roadway geometry acquisition, and vehicle activity analysis. Based on experimentation, it has been shown that the designed mobile surveillance system reports a high detection rate under the dynamic freeway environment in the experiments, with assistance from a human interactive detection module.
In order to have a better understanding of dynamic traffic conditions, we have incorporated this localized traffic density measurements into a Dynamic Roadway Network Database (DRND), which has been developed to fuse the roadway traffic data and the probe vehicle data. We believe that with the increasing use of on-board vision sensors, more and more localized traffic information samples can be reported to this type of database. The combinational analysis of temporal-spatial variable density and the embedded loop sensor data will provide a better and more reliable method for traffic condition estimation and prediction.
Bicycle Safety Support System
process, computer vision techniques are being applied in safety studies as well. In the third part of this dissertation, stationary vision based observations have been made of the timing of bicyclists’ intersection crossing maneuvers, to support of efforts in improving traffic signal timing to accommodate the needs of bicyclists. Video recordings were made of bicyclists’ crossings and the video images were processed to extract the bicyclists’ trajectories. These were synchronized with video images of the traffic signals so that the timing of the bicyclists’ maneuvers could be determined relative to the signal phases.
The processed data have yielded cumulative distributions of the crossing speeds of bicyclists who did not have to stop at the intersection and the start-up times and final crossing speeds of the bicyclists who had to cross from a standing start. This study provides a foundation in recommendation of minimal green signal time in terms of safety purpose.
The key contributions of this dissertation are:
A time-space diagram based flow calculation method from localized density has been proposed and experimentally verified;
Loop Fault Detection Tool (PLFDT) to provide baseline data; and A stationary vision-based intersection monitoring system has been developed for the quantitative study of bicyclist crossings at signalized intersections.
List of Figures
List of Table
1.1 Computer Vision as A Powerful Tool in ITS
1.2 Problem Statement
1.3 Contributions of the Dissertation
1.4 Organization of the Dissertation
2. Background and Literature Review
2.1 Roadway Traffic Surveillance Systems
2.1.1 Three Levels of Traffic Data Representation
2.1.2 Embedded Loop Data Collection Infrastructure
2.1.3 PeMS System
2.2 Review of Vision Based Vehicle Detection Algorithms
2.2.1 Knowledge Based Methods
2.2.2 Motion Based Methods
2.2.3 Some Other Algorithms
2.3 Review of Vision Based Vehicle Tracking Algorithms
2.3.1 Region Based Tracking Algorithms
2.3.2 Contour Based Tracking Algorithms
2.3.3 Feature Based Tracking Algorithms
2.3.4 3D Model Based Tracking Algorithms
2.4 Review on Loop Fault Detection Systems
2.5 Review on the Studies of Bicyclist Intersection Crossing Maneuvers
3. OODV System and Video Processing Algorithm
3.1 The Design of OODV System
3.1.1 The Structure of the OODVs
3.1.2 Characterization of the OODV Projection Model
3.1.3 Some Discussion on the OODV Projection Model
3.1.4 Highlights of the OODV Projection Model
3.2 OODV-Based Vehicle Detection Algorithm
3.3 Rectilinear Camera-Based Vehicle Detection and Tracking Algorithm
4. Stationary Vision Based Portable Loop Fault Detection Tool (PLFDT) Development................. 45
4.1 Vision Sensor Provides Baseline Data
4.2 Development of Portable Loop Fault Detection Tool (PLFDT)
4.2.1 Overall System Structure of the PLFDT
4.2.2 Mobile Pole for Roadside Video Camera Mounting
xiv 4.2.3 Interface with Control Cabinet
4.3 Vision Based Multi-lane Vehicle Tracking Software Development
4.3.1 Synchronization of the two Computers with Wireless Communication
4.3.2 Real-Time Multi-lane Vehicle Tracking Algorithm
4.4 Comparison of Physical Loop and Virtual Loop
4.5 Experiment Data Analysis
5. Mobile Traffic Surveillance System Using a Unique OODV Sensor