Geometric Constraints in Image Sequences and Neural Networks for Object Recognition

Sponsor: OSU Center for Mapping /NASA
Project period: 1996 - 1998
Principal Investigator: Dr. Ron Li
Project Team: Dr. F. Ma, Z. Tu, W. Wang and H.Z. Tseng

Description (1996~1997):

Spatially referenced mobile mapping image sequences contain rich information for applications such as transportation and utility management. Automatic object recognition and measurement of objects from the images for reducing human operations and enhancing efficiency is a challenge in mobile mapping data processing. This project uses Hopfield Neural Networks for utility object recognition and photogrammetric measurements. Specifically, street light poles are modeled in the 3-D object space and compared with the image sequences. The neurons of the net are formed by vector edge features in stereo images. The established Hopfield model first find all light poles in the images. In the second procedure, corresponding light poles in stereo images are identified. Finally the photogrammetric triangulation supplies 3-D positions of the poles in the object space. Such automation is particularly important for building special layers, for example, traffic signs, firehydrants, and road centerline in GIS.

 

Annual Report (.ps)

Description (1997~1998):

Trucks in image sequences taken by the Airborne Integrated Mapping System (AIMS) are recognized. The recognition is achieved by two steps. Step I detects lines and regions from original imagery, and step II recognizes trucks using a two-layer Hopfield neural network. Extraction of other image features such as roads from AIMS data has also been completed.

The edges are extracted through a new edge detector which combines a first-order derivative operator with a second-order derivative operator. A threshold is specified to obtain edge seeds whose lengths are large. Regions are segmented by gray scale and texture features. Region boundaries are then extracted by morphological transformations. With the help of region boundaries, the edge seeds are aggregated and linked. Using information from DEM slope grouping, the edges are divided into different areas. Within these areas, edges are linked by Hough transformation, curvefitting, and morphological processing. For example, roads are delineated through an anti-parallel model and are intensified through morphological transformations and an active contour model. Those ground objects above the ground, for example, buildings, cars and trees, are extracted through both shadow and DEM slope grouping results and classified by shape and structure.

With the edges and regions detected from the imagery, a two-layer Hopfield neural network is used to recognize trucks. Matching of edges between candidates and the 3-D truck model may have difficulties in shadows which vary from truck to truck. Region pattern matching, while giving us geometric constraints of the top and shadow of the truck, may not be able to distinguish detailed changes in shape. Thus, a two-layer Hopfield neural network that integrates both line pattern and region pattern recognition schemes provides a solution to truck recognition.

Annual Report (zip)

E-mail:

<li.282@osu.edu>