FREE ELECTRONIC LIBRARY - Dissertations, online materials

Pages:   || 2 |

«Abstract. Image stitching is used to combine several individual images having some overlap into a composite image. The quality of image stitching is ...»

-- [ Page 1 ] --

Seamless Image Stitching in the Gradient Domain

Anat Levin, Assaf Zomet, Shmuel Peleg, and Yair Weiss

School of Computer Science and Engineering

The Hebrew University of Jerusalem

91904, Jerusalem, Israel

alevin,peleg,yweiss @cs.huji.ac.il, zomet@cs.columbia.edu

Abstract. Image stitching is used to combine several individual images having

some overlap into a composite image. The quality of image stitching is measured

by the similarity of the stitched image to each of the input images, and by the visibility of the seam between the stitched images.

In order to define and get the best possible stitching, we introduce several formal cost functions for the evaluation of the quality of stitching. In these cost functions, the similarity to the input images and the visibility of the seam are defined in the gradient domain, minimizing the disturbing edges along the seam. A good image stitching will optimize these cost functions, overcoming both photometric inconsistencies and geometric misalignments between the stitched images.

This approach is demonstrated in the generation of panoramic images and in ob- ject blending. Comparisons with existing methods show the benefits of optimiz- ing the measures in the gradient domain.

1 Introduction Image stitching is a common practice in the generation of panoramic images and ap- plications such as object insertion, super resolution [1] and texture synthesis [2]. An example of image stitching is shown in Figure 1. Two images Á ½,Á¾ capture different portions of the same scene, with an overlap region viewed in both images. The images should be stitched to generate a mosaic image Á. A simple pasting of a left region from Á½ and a right region from Á ¾ produces visible artificial edges in the seam between the images, due to differences in camera gain, scene illumination or geometrical misalign- ments.

The aim of a stitching algorithm is to produce a visually plausible mosaic with two desirable properties: First, the mosaic should be as similar as possible to the input images, both geometrically and photometrically. Second, the seam between the stitched images should be invisible. While these requirements are widely acceptable for visual examination of a stitching result, their definition as quality criteria was either limited or implicit in previous approaches.

In this work we present several cost functions for these requirements, and define the mosaic image as their optimum. The stitching quality in the seam region is measured This research was supported (in part) by the EU under the Presence Initiative through contract IST-2001-39184 BENOGO.

Current Address: Computer Science Department, Columbia University, 500 West 120th Street, New York, NY 10027 Input image ½½

–  –  –

Fig. 1. Image stitching. On the left are the input images. is the overlap region. On top right is a simple pasting of the input images. On the bottom right is the result of the GIST1 algorithm.

in the gradient domain. The mosaic image should contain a minimal amount of seam artifacts, i.e. a seam should not introduce a new edge that does not appear in either Á ½ or Á¾. As image dissimilarity, the gradients of the mosaic image Á are compared with the gradients of Á½ Á¾. This reduces the effects caused by global inconsistencies between the stitched images. We call our framework GIST: Gradient-domain Image STitching.

We demonstrate this approach in panoramic mosaicing and object blending. Analytical and experimental comparisons of our approach to existing methods show the benefits in working in the gradient domain, and in directly minimizing gradient artifacts.

1.1 Related Work There are two main approaches to image stitching in the literature, assuming that the images have already been aligned. Optimal seam algorithms[3, 2, 4] search for a curve in the overlap region on which the differences between Á ½ Á¾ are minimal. Then each image is copied to the corresponding side of the seam. In case the difference between Á½ Á¾ on the curve is zero, no seam gradients are produced in the mosaic image Á.

However, the seam is visible when there is no such curve, for example when there is a global intensity difference between the images. This is illustrated on the first row of Figure 2. In addition, optimal seam methods are less appropriate when thin strips are taken from the input images, as in the case of manifold mosaicing [5].

The second approach minimizes seam artifacts by smoothing the transition between the images. In Feathering [6] or alpha blending, the mosaic image Á is a weighted combination of the input images Á ½ Á¾. The weighting coefficients (alpha mask) vary as Inp. image ½½ Inp. image ½¾ Feathering Pyr. blending Opt. Seam GIST Fig. 2. Comparing stitching methods with various sources for inconsistencies between the input images. The left side of Á½ is stitched to right side of Á¾. Optimal seam methods produce a seam artifact in case of photometric inconsistencies between the images (first row). Feathering and pyramid blending produce double edges in case of horizontal misalignments (second row). In case there is a vertical misalignments (third row), the stitching is less visible with Feathering and GIST.

a function of the distance from the seam. In pyramid blending[7], different frequency bands are combined with different alpha masks. Lower frequencies are mixed over a wide region, and fine details are mixed in a narrow region. This produces gradual transition in lower frequencies, while reducing edge duplications in textured regions. A related approach was suggested in [8], where a smooth function was added to the input images to force a consistency between the images in the seam curve. In case there are misalignments between the images[6], these methods leave artifacts in the mosaic such as double edges, as shown in Figure 2.

In our approach we compute the mosaic image Á by an optimization process that uses image gradients. Computation in the gradient domain was recently used in compression of dynamic range[9], image editing [10], image inpainting [11] and separation of images to layers [12–15]. The closest work to ours was done by Perez et. al. [10], who suggest to edit images by manipulating their gradients. One application is object insertion, where an object is cut from an image, and inserted to a new background image. The insertion is done by optimizing over the derivatives of the inserted object, with the boundary determined by the background image. In sections 4,5 we compare our approach to [10].

2 GIST: Image Stitching in the Gradient Domain We describe two approaches to image stitching in the gradient domain. Section 2.1 describes GIST1, where the mosaic image is inferred directly from the derivatives of the input images. Section 2.2 describes GIST2, a two-steps approach to image stitching.

Section 2.3 compares the two approaches to each other, and with other methods.

–  –  –

Ô ´Â½ ¾ Ï µ is

where Í is a uniform image, and the distance between ½ ¾ on :

Ô Ô ´Â½ ¾ ϵ Ï ´Õµ ½ ´Õµ   ¾ ´Õµ Ô (2) Õ¾ with ¡ Ô denoting the Ô -norm.

The dissimilarity Ô between the images is defined by the distance between their derivatives. A dissimilarity in the gradient domain is invariant to the mean intensity of the image. In addition it is less sensitive to smooth global differences between the input images, e.g. due to non-uniformness in the camera photometric response and due to scene shading variations. On the overlap region, the cost function Ô penalizes for derivatives which are inconsistent with any of the input images. In image locations where both Á ½ and Á¾ have low gradients, Ô penalizes for high gradient values in the mosaic image. This property is useful in eliminating false stitching edges.

The choice of norm (parameter Ô) has implications on both the optimization algorithm and the mosaic image. The minimization of Ô (Equation 1) for Ô ½ is convex, and hence efficient optimization algorithms can be used. Section 3 describes a minimization scheme for ¾ by existing algorithms, and a novel fast minimization scheme for ½. The mask image Ï was either a uniform mask (for ½ ) or the Feathering mask (for ¾ ), which is linear with the signed-distance from the seam. The influence of the choice of Ô on the result image is addressed in the following sections, with the introduction of alternative stitching algorithms in the gradient domain.

2.2 GIST2: Stitching Derivative Images

A simpler approach is to stitch the derivatives of the input images:

–  –  –

In stage (2) above, any stitching algorithm may be used. We have experimented with Feathering, pyramid blending [7], and optimal seam. For the optimal seam we used ´Ýµ that minimizes the sum of absolute the algorithm in [2], finding the curve Ü differences in the input images. Stage (3), the optimization under ½ ¾, is described in Section 3. Unlike the GIST1 algorithm described in the previous section, we found minor differences in the result images when minimizing Ô under ½ and ¾.

Optimal seam Optimal seam on the gradients

–  –  –

Fig. 3. Stitching in the gradient domain. The input images appear in Figure 1, with the overlap region marked by a black rectangle. With the image domain methods (top panels) the stitching is observable. Gradient-domain methods (bottom panels) overcome global inconsistencies.

2.3 Which Method to Use ?

In the previous sections we presented several stitching methods. Since stitching results are tested visually, selecting the most appropriate method may be subject to personal taste. However, a formal analysis of properties of these methods is provided below.

Based on those properties in conjunction with the experiments in Section 4, we recommend using GIST1 under ½.

Theorem 1. Let Á½ Á¾ be two input images for a stitching algorithm, and assume there ´Ýµ, such that for each Õ ¾ ´ ´Ýµ ݵ, Á ½ ´Õµ Á¾ ´Õµ.

Let Í be a is a curve Ü uniform image. Then the optimal seam solution Á, defined below, is a global minimum of Ô ´Á Á½ Á¾ Í µ defined in Eq.1, for any ¼ Ô ½.

–  –  –

The reader is referred to [16] for a proof. The theorem implies that GIST1 under ½ is as good as the optimal seam methods when a perfect seam exists. Hence the power of GIST1 under ½ to overcome geometric misalignments similarly to the optimal seam methods. The advantage of GIST1 over optimal seam methods is when there is no perfect seam, for example due to photometric inconsistencies between the input images.

This was validated in the experiments.

We also show an equivalence between GIST1 under ¾ and Feathering of derivatives (GIST2) under ¾ (Note that feathering derivatives is different from Feathering the images).

Theorem 2. Let Á½ Á¾ be two input images for a stitching algorithm, and let Ï be a Feathering mask.

Let, the overlap region of Á ½ Á¾, be the entire image (without loss of generality, as Ï ´Õµ ½ for Õ ¾ ½, and Ï ¼ for Õ ¾ ¾ ). Let Á ×Ø be the minimum

of ¾ ´Á Á½ Á¾ Ï µ defined in Eq. 1. Let be the following field:

´ÕµÖÁ½ ´Õµ · ´½   Ï ´ÕµµÖÁ¾ ´Õµ

Ï Then Á ×Ø is the image with the closest gradient field to under ¾.

The proof can be found in [16] as well. This provides insight into the difference between GIST1 under ½ and under ¾ : Under ¾, the algorithm tends to mix the derivatives and hence blur the texture in the overlap region. Under ½, the algorithm tends to behave similarly to the optimal seam methods, while reducing photometric inconsistencies.

–  –  –

For an even number of samples, the median is taken to be the average of the two middle samples.

In regions where a single image Á is used, the median is taken on the predictions of Á´Ü ݵ given its four neighbours and the derivatives of image Á. For example, when the derivatives of image Á are 0, the algorithm performs an iterated median filter of the neighbouring pixels. In the overlap region of Á½ Á¾, the median is taken over the predictions from both images.

At every iteration, the algorithm performs a coordinate descent and improves the cost function until convergence. As the cost function is bounded by zero, the algorithm always converges.

However, although the cost function is convex, the algorithm does not always converge to the global optimum1. To improve the algorithm convergence and speed, we combined it in a multiresolution scheme using multigrid [18]. In extensive experiments with the multi-resolution extension the algorithm always converged to the global optimum.

4 Experiments We have implemented various versions of GIST and applied them to panoramic mosaicing and object blending.

First, we compared GIST to existing image stitching techniques, which work on the image intensity domain: Feathering [6], Pyramid Blending [7], and ’optimal seam’ (Implemented as in [2]). The experiments (Figure 3) validated the advantage in working in the gradient for overcoming photometric inconsistencies. Second, we compared the results of GIST1 (Section 2.1), GIST2 (Section 2.2) and the method by Perez. et. al. [10]. Results of these comparisons are shown, for example, in Figures 4,5, and analyzed in the following sections.

4.1 Stitching Panoramic Views The natural application for image stitching is the construction of panoramic pictures from multiple input pictures. Geometrical misalignments between input images are caused by lens distortions, by the presence of moving objects, and by motion parallax. Photometric inconsistencies between input images may be caused by a varying gain, by lens vignetting, by illumination changes, etc.

Pages:   || 2 |

Similar works:

«Good Practice No. 12 December 2010 IMPROVING PATIENT HANDOVER 1. Purpose The purpose of this document is to summarise the evidence and share good practice on effective handover of teams because there is emerging evidence that patient handover is often a weak link in the delivery of care. In addition, there is no published method that forms the gold standard and there is large variation in practice.2. Introduction The implementation of the Working Time Directive has meant significant changes to...»

«• • St. Joseph's Province The Fathers and Brothers of the Province extend their sympathy Condolences and prayers to the Rev. J. M. Murphy, O.P., the Rev. R. E. Bond, O.P., and Brother Ferrer Quigley, O.P., on the death of their fathers; to the Revs. C. S. ]om, O.P., and Brother Timothy Myers, O.P., on the death of their mothers; to the Rev. J. H. Kenny, O.P., and the Rev. E. L. Martin, O.P., on the death of their brothers. On the evening of September 2~th, at the National Shrine of the...»

«16 7 Guignebert, G 82 The Jewish ~ o r l d the in time of Jesus in the time of By CHARLES GUIGNEBERT Late Professor of Christianity at the Sorbonne in the time of W i t h An I n t r o d u c t i o n b y Dr. C h a r l e s F r a n c i s P o t t e r New Hyde Park, New York UNIYERBITY B O O X S First Printing, December 1959 Second Printing, October 1961 Third Printing, October 1965 Copyright 1959. FIRSTAMERICAN EDITION by University Books, lnc. Library of Congress Catalog Card No. 59-14528...»

«The Impact of Financial Structure on the Cost of Solar Energy Michael Mendelsohn, Claire Kreycik, Lori Bird, Paul Schwabe, and Karlynn Cory NREL is a national laboratory of the U.S. Department of Energy, Office of Energy Efficiency & Renewable Energy, operated by the Alliance for Sustainable Energy, LLC. Technical Report NREL/TP-6A20-53086 March 2012 Contract No. DE-AC36-08GO28308 The Impact of Financial Structure on the Cost of Solar Energy Michael Mendelsohn, Claire Kreycik, Lori Bird, Paul...»

«MINIMALITY OF THE WELL-ROUNDED RETRACT ALEXANDRA PETTET & JUAN SOUTO Abstract. We prove that the well-rounded retract of SOn \ SLn R is a minimal SLn Z-invariant spine.1. Introduction In this note we are interested in a certain SLn Z-invariant deformation retract of the symmetric space Sn = SOn \ SLn R. To every element A ∈ SLn R one can associate the lattice AZn in Rn. The element A is well-rounded if the set of shortest non-zero vectors of the lattice AZn generate Rn as a real vector...»

«1 IBR (Infectious Bovine Rhinotracheitis) This is caused by a herpes virus – similar to the virus that causes chicken pox. There is no crossover of the virus but in the same way that previous sufferers of chicken pox can have shingles normally at a time of stress – IBR virus can be ‘carried’ by a cow and come back at stressful times, like calving, causing further spread of disease.SYMPTOMS The Symptoms are generally respiratory signs from mild → severe → death. Can occur at any age....»

«Bringing Calm to Chaos A critical incident review of the San Bernardino public safety response to the December 2, 2015, terrorist shooting incident at the Inland Regional Center Rick Braziel, Frank Straub, George Watson, and Rod Hoops Bringing Calm to Chaos A critical incident review of the San Bernardino public safety response to the December 2, 2015, terrorist shooting incident at the Inland Regional Center Rick Braziel, Frank Straub, George Watson, and Rod Hoops This project was supported by...»

«Gleanings g Alberta Council Trefoil Guilds Annual Newsletter Jan 2014 Edition Published by Girl Guides of Canada, Alberta Council 11055 107 Street, Edmonton AB Excerpts from a letter to the local paper by T5H 2Z6 Sheryl Paquette of 3rd Parkland, Silver Willow (780) 424-5510 Guild, on the occasion of: e-mail info@albertagirlguides.com Website: www.albertagirlguides.com 100th Anniversary of Girl Guides in Alberta Alberta Trefoil Guild Adviser: Gail Armstrong As a long time member of the...»

«NEW ZEALAND JOURNAL OF ARCHAEOLOGY                   This document is made available by The New Zealand   Archaeological Association under the Creative Commons   Attribution‐NonCommercial‐ShareAlike 3.0 Unported License.   To view a copy of this license, visit   http://creativecommons.org/licenses/by‐nc‐sa/3.0/.  NEW ZEALAND JOURNAL OF ARCHAEOLOGY                   This document is made available by The New Zealand  ...»

«ARENBERG DOCTORAL SCHOOL Faculty of Engineering Science Privacy-Preserving E-Commerce Protocols Alfredo Rial Dissertation presented in partial fulfillment of the requirements for the degree of Doctor in Engineering March 2013 Privacy-Preserving E-Commerce Protocols Alfredo RIAL Supervisory Committee: Dissertation presented in partial Prof. dr. ir. Yves Willems, chair fulfillment of the requirements for Prof. dr. ir. Bart Preneel, supervisor the degree of Doctor Prof. dr. ir. Vincent Rijmen in...»

«OXYCHEM C AUSTIC S ODA H ANDBOOK Occidental Chemical Corporation (OxyChem) is a leading North American manufacturer of polyvinyl chloride (PVC) resins, chlorine and caustic soda, key building blocks for a variety of indispensable products such as plastics, pharmaceuticals and water treatment chemicals. Other OxyChem products include caustic potash, chlorinated organics, sodium silicates, chlorinated isocyanurates and calcium chloride. For every product it makes, OxyChem’s market position is...»

«KATERINA PASTRA Cognitive Systems Research Institute (CSRI) • 7 Makedonomachou Prantouna Street, P.C. 11525, Athens, Greece E-mail: kpastra@csri.gr • Tel. +30 2110124543 (internal line 301) • Date of Birth: 01-02-1978 Nationality: Greek Job Experience Director 2011 – ongoing Cognitive Systems Research Institute (CSRI) Athens, Greece & Senior Researcher Institute for Language and Speech Processing (ILSP) ATHENA Research Center Athens, Greece Senior Researcher 2008 – 2011 Research...»

<<  HOME   |    CONTACTS
2016 www.dissertation.xlibx.info - Dissertations, online materials

Materials of this site are available for review, all rights belong to their respective owners.
If you do not agree with the fact that your material is placed on this site, please, email us, we will within 1-2 business days delete him.