Lihi Panoramas

Back to Lihi's main | Photomosaics wiki page

Flickr discussion groups

Filckr Lords of Photo-Stitching Panoramas created with Hugin

What each boundary cost does

On the left is the result of edge cost and on the right gradient cost. The edge cost looks for curve continuation thus it can ignore inconsistencies due to texture rather than structure. The building edge, door, etc. are better aligned in the edge-cost result but the gradient result is better since it hides the leaves.

Result with edge cost

Result with gradient cost

Disagreement between edge cost and gradient cost

A more illustrative comparison of ordering a pair of images. Top row shows the panoramas obtained with each order. Second row shows edges overlayed on top of each order. Order 2 is a lot more expensive due to the numerous edges on the stairs. In the third row I marked by red the pixels where the gradient across the image boundary is high. The gradients resulting from misaligning the plane are more significant than those resulting from misaligning the stairs.

Order 1

Order 2

Cost = 75

Cost = 600

Cost = 67

Cost = 56

Agreement between edge cost and gradient cost

A simple example to show why the order matters.

Order 1

Order 2

Cost = 45

Cost = 780

Cost = 69

Cost = 444

Importance of ordering images

A simple example using two images shows what is a "good" order and what is a "bad" order.

Good order

Bad order

Ordering images in a panorama

Overall appraoch

Given a set of images placed on the same canvas with overlaps between them, we wish to find the best order of the images. In "best" we mean what people would consider as the best composition. The approach we've adopted has two steps:

1. Given the positions of the images on the canvas we build a list of all possible orderings of the images:

Build a non-directional graph where each node represents an image and edges connect between overlapping images.

Find all the corresponding Directional Acyclic Graphs. The topological order of the nodes in the DAG is the corresponding order of the images. This is done with this matlab code package (relevant paper included): Acyclic.tgz

2. For each order of the images compute the overall cost and keep the best one. We plan to examine three different costs:

The sum of the gradients across all visible image boundaries.
The sum of squared differences between images in the overlap area near visible image boundaries.
Some function which penalizes for edges which start/end at visible image boundaries.

Possible costs

1. Sum of gradients:

$Cost1=\sum _{r,c,b}I_{x}^{2}(r,c,b)+I_{y}^{2}(r,c,b)$ where $r=1,\ldots ,rows$ , $c=1,\ldots ,cols$ , $b=1,\ldots ,color\ bands$ Computed over the visible image boundaries.

2. Sum of squared color differences

$Cost2=\sum _{r,c,b}(I1(r,c,b)-I2(r,c,b))^{2}$ where $r=1,\ldots ,rows$ , $c=1,\ldots ,cols$ , $b=1,\ldots ,color\ bands$ and $I1,I2$ are the top image and the one under it. The cost is computed over a region around visible image boundaries.

3. Quality of edge completion

$Cost3=\sum _{e}Line\ Fit\ Quality(e,nearest\ edge)$ where $e\in all\ edges\ intrsecting\ visible\ boundaries$ and $Line\ Fit\ Quality(e,nearest\ edge)=min(\sum _{i}dist(actual\ positions,fitted\ positions),MAX\ COST)$

Each edge is represented by 10 discrete positions. We fit a line to all 20 points together. The line fit quality equals the sum of squared distances between the fitted position and the actual positions. An upper bound $MAX\ COST$ is a parameter to set.

Experiments

Synthetic Experiment

I took an image, cut it into three overlapping pieces and corrupted two of them in the overlap area (the gray area is the corrupted part of the image):

Piece 1

Piece 2

Piece 3

The four possible orderings of the images were found correctly using the Acyclic code. Computing Cost1 for all orderings and selecting the minimal one resulted in hiding the corrputed regions:

Best result according to Cost1

Pumpkin Experiment

This is an experiment with a dataset of 5 images. Only four were matched and placed on the canvas. All of them are overlapping. The Acyclic code found 24 possible orderings. Results: