«Jiangqin Sun B.S. Soochow University, 2007 M.S. Harbin Institute of Technology, 2009 A dissertation submitted in partial fulﬁllment of the ...»
Temporal Signature Modeling and Analysis
B.S. Soochow University, 2007
M.S. Harbin Institute of Technology, 2009
A dissertation submitted in partial fulﬁllment of the
requirements for the degree of Doctor of Philosophy
in Imaging Science
Chester F. Carlson Center for Imaging Science
College of Science
Rochester Institute of Technology
December 6, 2014
Signature of the Author
Coordinator, Ph.D. Degree Program Date
CHESTER F. CARLSON CENTER FOR IMAGING SCIENCE
COLLEGE OF SCIENCE
ROCHESTER INSTITUTE OF TECHNOLOGY
ROCHESTER, NEW YORK
CERTIFICATE OF APPROVAL
Although there is a large amount of image data captured, most of them are not available to the public. The other part of this thesis developed a method of generating spatial-spectral-temporal synthetic images by enhancing the capacity of a current tool called DIRISG (Digital Imaging and Remote Sensing Image Generation). Currently, DIRSIG can only model limited temporal signatures. In order to observe general temporal changes in a process within the scene, a process model, which links the observable signatures of interest temporally, should be developed and incorporated into DIRSIG. The sub process models could be categorized into two types. One is that the process model drives the property of each facet of the object changing over time, and the other one is to drive the geometry location of the object in the scene changing as a function of time. Two example process models are used to show how process models can be incorporated into DIRSIG.
The Ph.D. study at RIT was a great experience for me. During the last few years, I was oﬀered a lot of opportunities to learn and grow. I feel grateful to all the people I meet at RIT, who altogether makes RIT a warm place to stay.
First, I would like to thank my advisor, Dr. David Messinger, for his constant support, encouragement, and patience. He has been a wonderful advisor in balancing guidance and giving freedom to students, which allows us to work eﬀectively and creatively. This work would not be possible without his mentoring.
I also would like to thank my committee members: Dr. Tony Harkin, Dr. Carl Salvaggio and Dr. Joel Kastner for taking the time to provide their support and give me valuable feedback on my work.
I would like to thank Dr. Michael Gartley for his kindness to teach me DIRSIG.
Many thanks to Jason Faulring for helping me setting up the parking lot experiment.
I thank Rolando Raqueño, Andrew Scott, and Nina Raqueño for their discussion, feedback, and support in the parking lot modeling in DIRSIG.
I would like to all the CIS admin staﬀ, who makes things so well organized. In particular I’d like to thank Cindy Schultz, Sue Chan, Marilyn Lockwood, and Joyce French.
It would be very diﬃcult for me to complete the Ph.D. study without the support from friends, roommates, and fellow graduate students. Special thanks to Dr.
Weihua Sun, Lin Chen, Dr. Yujie Qiu, Wei Yao and Fan Jiang for their support and companion through the toughest ﬁrst year study in U.S..
Most importantly, I would like to thank my family for being always supportive to whatever decision I made. I would like to thank my Fiancé Philipp for his caring, encouragement, and making me a better person.
3.1 Scenarios of parking spot status change................ 62
3.2 Result Accuracy analysis of the parking status extraction...... 63
Introduction More than 150 Earth observation satellites are currently in orbit carrying sensors to monitor the earth and provide us a large number of valuable images. Many companies and government agencies are still working on constructing next generation satellites. For example, the ESA is developing ﬁve new missions called Sentinels to complement the capacities of the existing satellites in the next several years.
Besides, there are many airborne sensors available to take images of certain sites when required. During the 2012 summer, RIT (Rochester Institute of Technology) performed a large scale experiment in Avon area of Rochester which is named as SHARE2012. Several types of airborne sensors (including WASP, ALS-60, ProSpecTIR VS, MircroHSI and PI Sensor) ﬂew over the site with targets set up in a designed way. The ground truth was measured and recorded along with the weather information. The collected data are shared around the world. The large amount of temporal digital satellite and aerial images calls for the corresponding development in data processing techniques to combine and fuse the temporal data from diﬀerent sources in order to understand the data in a high level, such as extracting hidden events or activities. One part of the thesis will focus on building a generic framework to capture the temporal events from the data.
Although there is a large amount of image data captured and stored over time, most of them are not available to the public. The experiment event like SHARE2012
CHAPTER 1. INTRODUCTION
at RIT does not happen often, because such large scale experiments involve intense amount of eﬀorts and time. Besides, a lot of the data do not oﬀer the “ground truth”, which is required to test and validate the algorithm. The other part of the thesis will focus on generating temporal synthetic images by enhancing the capacity of a current tool called DIRISG (Digital Image and Remote Sensing Image Generation).
In this thesis, we will illustrate temporal signature modeling in DIRSIG ﬁrst to provide a method to generate the spatial-spectral-temporal synthetic remote sensing images. This could further aid the research in temporal signature analysis by oﬀering the test data and their “ground truth”.
1.1 Temporal Signature Modeling in DIRSIG The DIRS lab (Digital Imaging and Remote Sensing Laboratory) at Rochester Institute of Technology has spent the past 20+ years developing the DIRSIG tool. Over the past 20+ years, DIRSIG has been involved with a great development starting from simplistic thermal image rendering of a 2D scene. Nowadays, the DIRSIG is a complex synthetic image generation model of 3D scene, which also is designed to produce broad-band, multi-spectral and hyper-spectral imagery through the integration of a suite of ﬁrst principles based radiation propagation sub models. These sub models are responsible for tasks ranging from the BRDF (bi-directional reﬂectance distribution function) predictions of a surface to the dynamic scanning geometry of a line scanning imaging instrument. In addition to sub models that have been speciﬁcally created for the DIRSIG model, there are also components such as MODTRAN (MODerate resolution atmospheric TRANsmission) and FASCODE which are included in DIRSIG as workhorses for the multi- and hyper-spectral community.
All modeled components are combined using a spectral representation, and the integrated radiance images can be simultaneously produced for an arbitrary number of user deﬁned band passes. According to the surface temperature of the scene, the self-emitted radiances are calculated by a passive thermodynamic model which includes the time history of environmental and meteorological parameters. Then
CHAPTER 1. INTRODUCTION
the surface reﬂected radiances are determined to compute the sensor reaching radiances through the included MODTRAN model. DIRSIG can produce multi or hyper-spectral remote sensing images between the bandpass 0.2 to 20 m with high radiometric ﬁdelity[31, 92]. In addition, DIRSIG also produces per-pixel “truth”, so many algorithm developers take this advantage of DIRSIG to validate their algorithms and make improvements.
However, except for vehicle moving, the solar and historical weather related short term temporal changing, DIRSIG does not currently include long term temporal signatures of the scene easily. To perform trade studies, algorithm training or even hypothesis testing, the user needs to manually create the scene with each individual element changing frame by frame as a function of time, which is time consuming and labor intensive. Take the scene of MCV (Midland Cogeneration Venture) Power Plant in Michigan for example: Figure 1.1 shows the WASP images of the midland scene. By zooming in the center part of the scene, more details of the scene can be observed; there are various kinds of activities going on there, such as tanks with water ﬁlled in and released out, stacks releasing plumes, parking lots with cars arriving and leaving, a lake with surface temperature changing over the year and so on. In order to accurately describe the scene at a speciﬁc time, the user is required to manually set all characterizations for each scene element. For example, the user needs to ﬁgure out how much water is in the tanks and what is the temperature, which direction the plume would be blown and how strong the wind is, how many cars in the parking lot and how they are distributed, how the surface temperature of the lake looks like and so on. When time changes, the user needs to re-attribute all these properties to the scene element, which is a tedious process.
GIS (Geographic Information System) is a spatial temporal system designed to store, manage, process, and visualize time varying geographical data. Over time, GIS uses snapshot model, space time composite, spatio-temporal object model, event-based/state-based model, object-oriented model, version-diﬀerence model and so on to represent the spatial temporal geographical data in the spatio-temporal database. The dynamical property of GIS is shown in the database, and the representation of the relationship between the dynamic object elements in GIS is achieved through the management of so called relational database. Recently, the integration of process models and GIS is beginning to be realized and the proposed next generation GIS will use process models to govern the dynamics, adaption and evolution among the object elements in the system.
BIM (Building Information Modeling) is a intelligent model-based building design system, which incorporates the physical and functional characteristics into the
CHAPTER 1. INTRODUCTIONsystem. BIM which integrates the time information to the three spatial dimensions is often referred to as 4D BIM. 4D BIM uses a process model to direct the life cycle of a project by linking all the model elements in the construction schedule. Each model element describes a discrete, time-driven construction activity as stated in the schedule. 4D BIM can facilitate the decision makers to learn a intuitive understanding of the process by visualizing the whole result consisting of diﬀerent event phenomena from diﬀerent model elements on the time axe.
In the ﬁeld of computer graphics, a lot of work has been done on measuring and modeling the time-varying appearance of the natural phenomena[98, 44, 107].
Gu et al develops a model called space-time appearance factorization to factor space and time-varying eﬀects. Sun et al measured the time-varying BRDFs of a wide range of phenomena with a self-developed acquisition system at a time sample space within 36 seconds and shared the database online. Those modeled natural phenomena include drying of various types of paints, wetting and drying of rough surfaces(cement, plaster and fabrics)[107, 44], the accumulation of dusts on surfaces[52, 107], corrosion and rusting of metals[44, 74], the weathering stone and so on. These time-varying appearance of the diﬀerent materials and surfaces are also interesting to remote sensing communities.
The ﬁrst part of this thesis is intended to show the research of incorporating temporal signatures of the scene into DIRSIG, and these temporal signatures of the scene are driven by the process model. The enhanced DIRSIG could automatically create a scene with the property of each individual element driven by an external physical model as a function of time. The global process model could comprise many sub process models, each of which is designed to describe the temporal changing characterizations of the corresponding element in the scene. The changing characterizations of a scene element may include temperature, material property, geometry position and orientation, and so on. This research would enhance the ability of DIRSIG to simulate a complex scene with diverse activities and aid the algorithm development and test community to save signiﬁcant and labor. By adopting the method of incorporating process models into DIRSIG, ﬁnally we expect to be able to create a scene which could capture not only spatial-spectral information,