WO2009061170A2

WO2009061170A2 - Apparatus and method for multiple-view panoramic imaging

Info

Publication number: WO2009061170A2
Application number: PCT/MY2008/000131
Authority: WO
Inventors: Hock Woon Hon
Original assignee: Mimos Berhad
Priority date: 2007-11-09
Filing date: 2008-11-10
Publication date: 2009-05-14
Also published as: WO2009061170A3; MY147066A

Abstract

The present invention relates to apparatus and method for multiple-view panoramic imaging. According to one aspect of the present invention, the apparatus for multiple-view panoramic imaging includes a line-scan camera (107), a rotary stage (103), a capture plane (108), and a controller (111, 112) for each panoramic and multiple-view rotation. The apparatus further includes a frame grabber as signal capturing device (109) and master clock (110) for motion and image synchronization. The method for multiple-view panoramic imaging comprising capturing and generating multiple-view panoramic images using a single line-scan camera (107) wherein the camera (107) has 2 (two) distant centers (104, 105) of rotation enabling multiple-view rotation and panoramic rotation. Panoramic rotation is a rotation of the rotary stage (103) while the multiple-view rotation is a rotation by a capture plane (108).

Description

APPARATUS AND METHOD FOR MULTIPLE-VIEW PANORAMIC

IMAGING

The present invention relates to apparatus and method for multiple-view panoramic imaging using a single line-scan camera, more particularly for capturing and generating multiple-view panoramic images using dual rotational motions with two (2) distant centers of rotation.

BACKGROUND TO THE INVENTION

Panoramic imaging is widely used due its ability to provide a relatively wide field of view which is advantageous to, but not limited to, areas of machine vision, robotics and surveillance.

Generally, panoramic imaging systems include devices which are capable of generating images from a wide angle of a scene by utilizing multiple cameras for panoramic imaging. An example of panoramic imaging system is found in the U.S. Patent No. 7,092,014 which utilizes, multiple cameras in camera array form rotated through a capture cylinder to generate a plurality of longitudinal image arrays from a plurality of capture images. The longitudinal camera array allows the surrounding scene to be captured from multiple different planes that are substantially parallel to the ends of the capture cylinder to render the scene. However, the movement is restricted within the range of capture cylinder.

Another example of a panoramic imaging system is found in the U.S. Patent No. 5,403,140 which is a dynamic sweeping mechanism that drives a line- scan camera's field of view in a direction which is substantially perpendicular to the scan direction. The operation of the dynamic sweeping mechanism enables the line scan camera to make multiple scans across a target, providing the image processor the necessary image information to generate a two-dimensional image of the object. It also uses a single line-scan camera for panoramic imaging, however since it only involves a single rotation; it produces a single-view panoramic image data.

SUMMARY OF THE INVENTION

The present invention is directed to overcoming one or more of the disadvantages set forth above. Briefly summarized, the present invention relates to apparatus and method for multiple-view panoramic imaging.

According to one aspect of the present invention, an apparatus for multiple- view panoramic imaging including a line-scan camera, a rotary stage, a capture plane, and a controller for each panoramic and multiple-view rotation. The apparatus further includes a frame grabber as signal capturing device and master clock for motion and image synchronization. The acquired image data contains embedded multiple-view as well as three-dimensional information.

According to another aspect of the present invention, a method for multiple- view panoramic imaging comprising capturing and generating multiple-view panoramic images using a single line-scan camera having a dual rotational motion with two (2) distant centers of rotation for panoramic rotations and multiple-view rotations. The panoramic rotation is a global rotation of the overall apparatus in a single direction and continual manner while the multiple-view rotation is a local rotation involving rotations of the line-scan camera bidirectionally in a discrete manner. The combination of the dual- rotational motion forms the basis for the multiple view imaging.

According to still another aspect of the present invention, said apparatus for multiple-view panoramic imaging can be used to capture customized perspective view stereoscopic images in a single panoramic rotation. The disparity can be pre-calculated for a given targeted scene so that the optimum three-dimensional information can be presented to the observer without causing eyestrain. According to yet another aspect of the present invention, the image sampling /object-image space resolution on the motion axis (x axis) can be increased or decreased by controlling the rotating speed of the rotary stage. The vertical axis resolution can be controlled by using a commercially available line-scan camera with different resolution.

According to a final aspect of the present invention, said apparatus is designed in such a way that it has the flexibility to use sensor with different spectral response to capture images from different spectral range, for example, visible light sensor and thermal sensors.

These and other aspects, objects, features and advantages of the present invention will be more clearly understood and appreciated from a review of the following detailed description of the preferred embodiment and appended claims, and by reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The specific features, aspects, and advantages of the present invention will become better understood with regard to the following description, appended claims, and accompanying drawings where:

Fig. 1a illustrates a perspective view apparatus and setup of the multiple-view panoramic system according to the present invention;

Fig. 1b illustrates a top view of global and local rotations according to the present invention;

Fig. 2 illustrates signal control and synchronization between panoramic rotation, multiple-view rotation and image capture according to the present invention; and Fig. 3 is a flowchart illustrating rotation timing and image capture mechanism.

DETAILED DESCRIPTION OF THE INVENTION

In the following description of the preferred embodiments of the present invention, reference is made to the accompanying drawings which form a part hereof, and in which is shown by way of illustration specific embodiments in which the invention may be practiced. It is understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the present invention.

Referring to Fig. 1a, there is illustrated an embodiment of the present invention to capture and generate multiple-view panoramic image of interest. A single line-scan camera 107 is placed on top of a capture plane 108 which sits on top of a rotary stage 103. The capture plane 108 rotates bi-directionally by a rotary gear 113 which is controlled by controller 111 and resulting in a multiple-view rotation. The rotary stage 103, on the other hand, rotates in a single direction by a rotary gear 102 which is controlled by controller 112 and resulting in a panoramic rotation. The rotary stage 103 is mounted on a supporter 101 which is hooked to a standard tripod or placed on the floor.

A signal capturing device 109 analyzes and constructs the composite images captured by the line-scan camera 107. Said signal capturing device 109 used is a frame grabber.

A master clock 110 synchronizes rotations carried by controller 111 using rotary gear 102 and controller 112 using rotary gear 113 in rotating to result in synchronized rotations for multiple view imaging.

Fig. 1b shows the two (2) rotations that constitute multiple-view panoramic imaging. First rotation is a panoramic rotation which is a global rotation of the overall apparatus having a center of rotation 104 on the middle of the rotary stage 103 that rotates in a single direction and continual manner. The second rotation is a multiple-view rotation which is a local rotation involving rotations of the line-scan camera 107 bidirectionally in a discrete manner having a center of rotation 105 on the middle of the capture plane 108.

When the global rotation rotates, the overall apparatus rotates in a single direction in a continual manner and the capture plane 108 and the line-scan camera 107 rotates in the same direction.

When the localized rotation rotates, the line-scan camera 107 moves in a number of discrete positions in both direction, either positive or negative, one direction at a given time. The rotation is bidirectional to eliminate the need of resetting to home position once N_Max view is reached, where N_Max represents the total number of perspective views of the apparatus setup predefined by the user.

The localized rotation facilitates the pointing direction of the camera and it does not affect the global rotation. It only moves and points the camera 107 to a certain direction or perspective by setting the discrete angle position through the rotary gear 113.

Distance 106 between the two rotation centers 104, 105 determines the resultant parallax that is produced at the end of a single rotation. The further the distance is the larger the parallax is resulted in the perspective images, hence more spatial information is derived from the resultant perspective images. The formula to determine the parallax between two consecutive views is depicted in following equation:

where nand n-1 are two consecutive perspective views

Fig. 2 and Fig. 3 illustrate image capture mechanism and synchronization of the panoramic and multiple-view rotation by master clock 110. Initially, the rotary stage 103 is set to home position and a starting panoramic pulse 301 is fired to controller 111 to rotate rotary gear 113 to initiate a panoramic rotation. After the first panoramic pulse 301 is fired, a series of pulses 303 to 308 are fired, sequentially, to initiate multiple-view rotation before the second panoramic pulse 302 is fired to initiate the panoramic rotation. The period of time taken for the first panoramic pulse 301 and the second panoramic pulse 302 is linearly dependent upon the maximum number of view (N_Max) in the multiple view rotation which is predefined by the user.

NMa_x represents the total number of perspectives and this is determined by the user and is stored in a computer. No represents perspective view of the line- scan camera 107, once the image is captured, the line-scan camera points to the next location to capture image data from different perspective which is Ni.

At the point where the multiple-view rotation pulse 303 is sent to rotate the capture plane 108 to point the camera 107 to a certain preset angle, an image triggering pulse 309 is sent to the capture device in order to capture video signal from the line-scan camera 107. Time taken between the multiple view pulse and the image capture pulse is illustrated in the shaded area 311.

Duration between 303 and 304 is the time taken between two consecutive multiple view rotation pulses. The multiple view pulses are fired until the last position NM_BX is reached.

When the last position N_Ma% in the multiple view rotation is reached, that is when the entire perspective view image information has been captured, the second signal pulse 302 is fired to move the panoramic rotary stage 103 to a new position. The same process for the multiple-view rotation is repeated, but this time, the pointing position is from the NMax to (/v^ax x 2) in a reverse direction. This is to save the time traveled for the multiple-view rotary stage to the home position 103 and fully utilized the rotation in the bidirectional mode. In case where /V_Max view is not reached, the camera 107 moves to new multiple view position and the image capture process is repeated until the last multiple view position is reached.

If /V_Max for a certain panoramic view is reached but not the last panoramic position, i.e. after 360 degree, then the camera moves to a new panoramic position and the image capture process is repeated until the last panoramic position is reached.

Claims

1. An apparatus for multiple-view panoramic imaging using dual rotational motions, comprising: a rotary stage (103) mounted on a supporter (101); a capture plane (108) mounted on said rotary stage (103); a single line-scan camera (107) mounted on said capture plane (108); a controller (111) controlling rotary gear (113) that rotates said capture plane (108); and a controller (112) controlling rotary gear (102) that rotates said rotary stage (103).

2. An apparatus for multiple-view panoramic imaging using dual rotational motions according to claim 1 , wherein the apparatus further comprising signal capturing device (109) to analyze and construct composite images captured by the line-scan camera (107).

3. An apparatus for multiple-view panoramic imaging using dual rotational motions according to claim 1, wherein the apparatus further comprising a master clock (110) to synchronizes controller (111) and controller (112) in rotating the rotary gears (102, 113).

4. An apparatus for multiple-view panoramic imaging using dual rotational motions according to claim 2, wherein the signal capture device (109) is a frame grabber.

5. An apparatus for multiple-view panoramic imaging using dual rotational motions according to claim 1, wherein rotation of rotary stage (103) results in a panoramic rotation with a center of rotation (104).

6. An apparatus for multiple-view panoramic imaging using dual rotational motions according to claim 1, wherein rotation of capture plane (108) results in a multiple-view rotation with a center of rotation (105).

7. An apparatus for multiple-view panoramic imaging using dual rotational motions according to claim 1 , wherein the apparatus is capable of capturing visible light images if the camera (107) is a visible light camera.

8. An apparatus for multiple-view panoramic imaging using dual rotational motions according to claim 1, the apparatus is capable of capturing night vision image if the camera (107) is a thermal camera.

9. A method for multiple-view panoramic imaging using dual rotational motions, comprising the steps of:

(a) setting the rotary stage (103) to home position and firing a starting pulse (301) to the controller (111) to rotate the rotary gear (113) to initiate a panoramic rotation;

(b) firing pulses (303-308) to initiate a multiple-view rotation before the second panoramic rotation is fired (302), wherein the pulses (303-308) are fired to rotate the capture plane (108) to point the line-scan camera (107) to a certain preset angle;

(c) sending an image capture pulse (309) to initiate capture of lines of image data by the line-scan camera (107) that further rotates continuously until a total number of all the perspective view image data have been captured; and

(d) firing a second pulse (302) to initiate the second panoramic rotation and at the same time, rotating the capture plane (108) in reverse direction from its last position to another preset angle.

10. A method for multiple-view panoramic imaging using dual rotational motions according to claim 9, wherein the period of time between first panoramic rotation pulse (301) and second panoramic rotation pulse (302) is linearly dependent upon the maximum number of views in the multiple view rotation.

11. A method for multiple-view panoramic imaging using dual rotational motions according to claim 9, wherein captured images by the line-scan camera (10) are analyzed and constructed by a signal capturing device (109).

12. A method for multiple-view panoramic imaging using dual rotational motions according to claim 9, further comprises synchronization of panoramic and multiple-view rotation having two different centers of rotations (104, 105) by master clock (110) to produce multiple-view images.

13. A method for multiple-view panoramic imaging using dual rotational motions according to claim 9, wherein a resultant parallax is determined from the distance (106) between panoramic center of rotation (104) and multiple- view center of rotation (105).

14. A method for multiple-view panoramic imaging using dual rotational motions according to claim 9, wherein resolution object-image space resolution on the motion axis is adjusted by controlling the rotation speed of the rotary stage (103).

15. A method for multiple-view panoramic imaging using dual rotational motions according to claim 9, wherein the maximum number of view in the multiple view rotation is predefined by the user.

16. A method for multiple-view panoramic imaging using dual rotational motions according to claim 9, wherein if the maximum number of view is not reached, the camera (107) is moved to a new multiple view position and the image capture process is repeated until the last multiple view position is reached.

17. A method for multiple-view panoramic imaging using dual rotational motions according to claim 9, wherein if the maximum number of view has been reached but not the last panoramic position, the camera (107) is moved to a new panoramic position and the image capture process is repeated until the last panoramic position is reached.