US20080285878A1 - Image capturing apparatus and method for correcting captured image - Google Patents
Image capturing apparatus and method for correcting captured image Download PDFInfo
- Publication number
- US20080285878A1 US20080285878A1 US12/174,508 US17450808A US2008285878A1 US 20080285878 A1 US20080285878 A1 US 20080285878A1 US 17450808 A US17450808 A US 17450808A US 2008285878 A1 US2008285878 A1 US 2008285878A1
- Authority
- US
- United States
- Prior art keywords
- image
- captured image
- written material
- captured
- section
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/74—Projection arrangements for image reproduction, e.g. using eidophor
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/04—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa
- H04N1/19—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa using multi-element arrays
- H04N1/195—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa using multi-element arrays the array comprising a two-dimensional array or a combination of two-dimensional arrays
- H04N1/19594—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa using multi-element arrays the array comprising a two-dimensional array or a combination of two-dimensional arrays using a television camera or a still video camera
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/04—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa
- H04N1/19—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa using multi-element arrays
- H04N1/195—Scanning arrangements, i.e. arrangements for the displacement of active reading or reproducing elements relative to the original or reproducing medium, or vice versa using multi-element arrays the array comprising a two-dimensional array or a combination of two-dimensional arrays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
- H04N1/387—Composing, repositioning or otherwise geometrically modifying originals
Definitions
- the present invention relates to a captured image projection apparatus and a correction method for a captured image.
- Projectors are used in presentations, meetings, etc. to expandingly project documents (characters and drawings, etc.) stored in personal computers on a screen.
- Projectors generally have a configuration for converting image information of documents obtained from personal computers, etc. into projection light by using an image conversion element such as liquid crystal, micro mirror array, etc.
- an angle detection sensor for detecting the inclination angle of the camera unit with respect to the document mounting.
- the present invention was made in view of the above circumstance, and relates to a captured image projection apparatus and a captured image correction method which are capable of dealing with various corrections necessary for adjusting a captured image before projection, and of performing an accurate correction.
- an image capturing apparatus which includes: a document table adapted to have a written material placed thereon; an image capturing section which captures an image of the written material; an obtaining section which obtains image information from the captured image, the image information including four lines located closest to four circumferential sides, respectively, of the captured image; and a correction section which corrects distortion of an image region corresponding to the written material within the captured image based on the image information, the correction of the distortion comprising deforming the image region into a rectangular shape.
- FIG. 1 is a configuration diagram of a captured image projection apparatus which is common to each embodiment of the present invention
- FIG. 2 is a block diagram showing a schematic electric structure of the captured image projection apparatus of FIG. 1 ;
- FIG. 3 is a block diagram showing a schematic electric structure of a calligraphy/drawing camera according to first to tenth embodiments of the present invention
- FIG. 4 is a flowchart showing a basic operation of the captured image projection apparatus which is common to each embodiment
- FIG. 5 is a flowchart showing a procedure of a rectangular conversion process by a PC according to the first embodiment
- FIGS. 6A to 6C are explanatory diagrams for the rectangular conversion process by the PC
- FIGS. 7A to 7E are explanatory diagrams showing the content of shape conversion in the rectangular conversion process by the PC
- FIG. 8 is a flowchart showing a procedure of an aspect ratio conversion process by the PC
- FIGS. 9A to 9D are explanatory diagrams showing the content of the aspect ratio conversion process by the PC
- FIG. 10 is a flowchart showing an operation of the PC after the aspect ratio conversion process by the PC is completed
- FIGS. 11A to 11D are explanatory diagrams showing the content of a cut out process by the PC
- FIG. 12 is a flowchart showing the content of a direction detection process by the PC in the second embodiment
- FIGS. 13A to 13C are explanatory diagrams showing a reason why a deformation is produced in a captured image, and a state where a deformation is produced;
- FIG. 14 is a flowchart showing a procedure for obtaining an image capturing angle according to the third embodiment
- FIGS. 15A and 15B are explanatory diagrams showing a state of a calligraphy/drawing camera when obtaining an image capturing angle
- FIG. 16 is a flowchart showing a procedure for obtaining a image capturing angle according to the fourth embodiment
- FIGS. 17A and 17B are explanatory diagrams showing a state of the calligraphy/drawing camera when obtaining an image capturing angle
- FIG. 18 is a flowchart showing a procedure of a rotation correction process by the PC according to the fifth embodiment
- FIG. 19 is an explanatory diagram showing an order of scanning directions to a captured image
- FIGS. 20A and 20B are exemplary diagrams showing a relation among a document, a scanning direction, and a marginal distribution
- FIGS. 21A and 21B are exemplary diagrams showing a relation among a document, scanning direction, and a marginal distribution in a case where the marginal distribution is not normal;
- FIG. 22 is a flowchart showing a procedure of a rotation correction process by the PC according to the sixth embodiment.
- FIGS. 23A to 23C are explanatory diagrams of the rotation correction process by the PC according to the sixth embodiment.
- FIG. 24 is a flowchart showing a procedure of a rotation correction process by the PC according to the seventh embodiment
- FIGS. 25A to 25C are explanatory diagrams of the rotation correction process by the PC according to the seventh embodiment.
- FIG. 26 is a flowchart showing an operation of the PC according to the eighth embodiment.
- FIGS. 27A and 27B are explanatory diagrams showing the operation of the PC according to the eighth embodiment.
- FIG. 28 is a flowchart showing an operation of the PC according to the ninth embodiment.
- FIGS. 29A and 29B are explanatory diagrams showing the operation of the PC according to the ninth embodiment.
- FIG. 30 is a flowchart showing an operation of the PC according to the tenth embodiment.
- FIGS. 31A and 31B are explanatory diagrams showing the operation of the PC according to the tenth embodiment
- FIG. 32 is a block diagram showing a schematic electric structure of the calligraphy/drawing camera according to the eleventh embodiment of the present invention.
- FIG. 33 is a flowchart showing the content of an image capturing angle obtaining process performed by the PC according to the eleventh embodiment.
- FIG. 34 is an explanatory diagrams showing a state of the calligraphy/drawing camera when obtaining an image capturing angle.
- a captured image projection apparatus comprises a projector 1 , a lap-top personal computer (hereinafter referred to as PC) 2 , and a calligraphy/drawing camera 3 , as its configuration diagram being shown in FIG. 1 .
- the projector 1 and the PC 2 are connected via an RGB cable 4 .
- the PC 2 and the calligraphy/drawing camera 3 are connected via an USB cable 5 .
- the projector 1 is structured such that a projection unit 1 A in which an optical system such as a projection lens, etc. is stored is provided on the front surface of the projector body. An optical system such as a projection lens, etc. is stored in the projection unit 1 A.
- the projector 1 projects documents and images on a screen S (or a whiteboard, etc.) by irradiating a light from the projection unit 1 A.
- the calligraphy/drawing camera 3 comprise a camera mounting 3 A, a support pillar 3 B which is provided to stand on the camera mounting 3 A, and a camera 3 C set on the top end of the support pillar 3 B.
- the camera 3 C is provided so as to be able to turn upward and downward, and leftward and rightward.
- a written material A such as a newspaper, an article in a magazine, a presentation paper, a written document, etc. is mounted on the camera mounting 3 A as illustrated, and is image-captured (imaged, picked-up) by the camera 3 C.
- the projector 1 mainly comprises a CPU 11 , a ROM 12 , a RAM 13 , a display unit 14 , and a key input unit 15 , as its schematic electrical structure being shown in FIG. 2 . Further, the projector has an input terminal 16 for an RGB signal, and the RGB cable 4 is connected to the input terminal 16 .
- the display unit 14 is the projection section according to the present invention.
- the display unit 14 includes a light source such as krypton lamp, etc., an image conversion element such as liquid crystal, micro mirror array, etc., a drive circuit, and an optical system such as a projection lens, etc.
- the display unit 14 projects documents and captured imaged sent from the PC 2 on the screen S.
- the image conversion element is for converting a light from the light source into a projection light
- the drive circuit is for driving the image conversion element in accordance with an RGB signal input from the input terminal 16 .
- the key input unit 15 is constituted by a plurality of operation buttons for receiving operations on the projector 1 by a user.
- the CPU 11 operates in accordance with a program stored in the ROM 12 , and controls each of the aforementioned components by using the RAM 13 as a work memory.
- the input terminal 16 is provided in an unillustrated input image processing circuit which is connected to the CPU 11 .
- the PC 2 mainly comprises a CPU 21 , a RAM 22 , a storage device 23 , an input device 24 such as a plurality of keys provided on the body of the PC 2 and a mouse connected to the body, etc., and a display device 25 constituted by an LCD and a drive circuit therefore, etc. Further, the PC 2 has an RGB output terminal 26 to which the RGB cable 4 is connected, and a USB connection terminal 27 to which the USB cable 5 is connected.
- the storage device 23 is constituted by a hard disk or the like having a relatively large storage capacity.
- An image processing program and an image display program are stored in the storage device 23 .
- the image processing program is a program for performing later-described various image processes to be applied to a captured image.
- the image display program is a program for displaying an image based on image data on the display device 25 or for making the image be projected by the projector 1 .
- the RGB output terminal 26 is provided in an unillustrated image signal processing circuit which is connected to the CPU 21 .
- the USB connection terminal 27 is provided in an unillustrated USB input/output interface circuit which is connected to the CPU 21 .
- FIG. 3 is a block diagram showing the electric structure of the calligraphy/drawing camera 3 (mainly the camera 3 C in the present embodiment).
- the calligraphy/drawing camera 3 (mainly the camera 3 C in the present embodiment) comprises a CCD 31 and a signal processing unit 32 , as its electric structure being shown in FIG. 3 .
- the CCD 31 is an imaging element for imaging a written material A via an unillustrated lens.
- the signal processing unit 32 applies various signal processes to an analog imaging signal output from the CCD 31 and then outputs the signal as a digital image signal.
- the image signal output from the signal processing unit 32 is sent to an LCD 33 and displayed thereon as a photo-object image when the calligraphy/drawing camera 3 is in an image capture standby state.
- the image signal is compressed by a CPU 34 in accordance with a predetermined format such as JPEG and stored in an image memory 35 . Further, the image signal is output from a USB interface 36 to the PC 2 via the USB cable 5 .
- the compressed image data stored in the image memory 35 is read by the CPU 34 if necessity arises.
- the read image data is expanded and then reproduced and displayed on the LCD 33 as a still image, or output to the PC 2 .
- the image memory 35 is a non-volatile memory such as a flash memory, etc. which is built inside the camera 3 C or detachable from the camera 3 C.
- the calligraphy/drawing camera 3 comprises a ROM 37 , a RAM 38 , and a key input unit 39 .
- Various control programs necessary for the CPU 34 to perform compression and expansion of the image data or to control the entire device are stored in the ROM 37 .
- the RAM 38 is a work memory of the CPU 34 .
- the key input unit 39 is constituted by a shutter key and many other operation keys.
- the calligraphy/drawing camera 3 image-captures an arbitrary document or the like (written material A) and outputs the image data of the captured document to the PC 2 (step S 101 ).
- the PC 2 corrects the captured image by performing various image processes such as a deformation correction process, a rotation process, a cut out process, etc. on the input image data (step S 102 to S 104 ). Then, the PC 2 outputs the processed image data to the projector 1 in accordance with the display program (step S 105 ).
- the PC 2 performs part or all of the image processes in accordance with necessity.
- the projector 1 projects the captured image of the document or the like on the screen S based on the image data input from the PC 2 (step S 106 ).
- the rectangular conversion process is a process for correcting deformation in a captured image and converting the shape of the image into a rectangle.
- the rectangular conversion process will now be explained with reference to FIG. 5 showing a flowchart of the procedure of the process, and FIGS. 6A to 6C showing diagrams explaining the procedure.
- the PC 2 identifies edges in a captured image G 1 in accordance with changes in color and luminance (step S 11 , FIG. 6A ).
- the PC 2 finds straight lines M 1 to M 4 which are close to four sides L 1 to L 4 of the outer circumference of the captured image G 1 from the identified edges (step S 112 , FIG. 6B ).
- the PC 2 further finds intersections of the straight lines M 1 to M 4 , i.e., four vertexes (step S 113 ).
- the PC 2 regards the quadrangle formed by the four vertexes as a written material region GA in the captured image G 1 , as shown in FIGS. 7A to 7E .
- the PC 2 converts the shape of the written material region GA in a way that the four vertexes of the region GA are moved to four vertexes of the smallest rectangle B that circumscribes the written material region GA (step S 114 , FIG. 6C ). Due to this, an image G 2 which is a result of correcting the deformation of the shape with respect to a rectangle (hereinafter this deformation will be referred to as rectangular deformation) is obtained in the written material region GA. Needless to say, also in the case where the rectangular deformation of the written material region GA is a trapezoidal deformation, this deformation is corrected by the above-described process.
- the PC 2 performs an aspect ratio conversion process, as its procedure being shown in a flowchart of FIG. 8 .
- the PC 2 upon receiving a user's operation for selecting a sheet, the PC 2 performs setting for the sheet (step S 121 ). If a regular sheet is not selected by the user in this step, the aspect ratio conversion process is terminated.
- the PC 2 sets the aspect ratio to “1.41:1” (step S 124 ).
- the PC 2 sets the aspect ratio to “1:1.41” (step S 126 ).
- PC 2 transforms the written material region GA which has been through the rectangular deformation correction, by either of the set aspect ratios (step S 127 ).
- the PC 2 performs the transformation based on coordinate data of the four vertexes of the smallest rectangle B used in the conversion (step S 114 ) in the rectangular conversion process explained above.
- the PC 2 transforms the written material region GA by expanding the region GA in the vertical direction such that the aspect ratio of the region GA will be “1:1.41”, as shown in FIG. 9 .
- the PC 2 transforms the written material region GA by expanding the region GA in the horizontal direction such that the aspect ratio will be “1.41:1”. As a result, the PC 2 obtains an image G 3 in which the written material region GA has the aspect ratio of the regular sheet.
- the PC 2 cuts out the region corresponding to the sheet from the image G 3 which has been subjected to the above-described transformation based on the coordinate data of the four vertexes of the smallest rectangle B used in the conversion (step S 114 ) in the rectangular conversion process as shown in FIG. 11 (step S 131 ). Then, the PC 2 sends an image G 4 obtained after adjustment which consists of only the cut-out image, i.e., the written material region GA to the display software (step S 132 ), so that the image G 4 will be displayed on the display device 25 .
- the captured image projection apparatus having the same structure as that shown in FIG. 1 to FIG. 3 detects whether the written material A in the captured image is placed lengthwise or widthwise when adjusting the captured image of the written material A.
- the PC 2 has a function for obtaining the angle of the lens of calligraphy/drawing camera 3 at which the written material A fully falls within the image range of camera 3 at the time the written material A is image-captured (this angle will hereinafter be referred to as angle of view), and for obtaining the resolution of the image output from the calligraphy/drawing camera 3 .
- angle of view the angle of the lens of calligraphy/drawing camera 3 at which the written material A fully falls within the image range of camera 3 at the time the written material A is image-captured
- this angle of view this angle will hereinafter be referred to as angle of view
- a program for controlling the PC 2 to perform a direction detection process to be described below is stored.
- the direction detection process by the PC 2 will now be explained with reference to a flowchart shown in FIG. 12 .
- the PC 2 first obtains the angle of view of the calligraphy/drawing camera 3 (step S 141 ). Then, the PC 2 obtains the resolution of the image data (step S 142 ). Subsequently, the PC 2 obtains four vertexes of the quadrangle corresponding to the sheet (written material region GA) from the captured image (step S 143 ). This process is the same as the process in the rectangular conversion process (see FIG. 5 ) explained in the first embodiment.
- the PC 2 calculates the amount of deformation in the image based on the coordinates of the obtained four vertexes by using the formerly obtained angle of view and resolution (step S 144 ).
- the calculation method for this process will now be explained.
- deformation in an image is produced if the sheet (written material A) is image-captured obliquely in a state where the camera 3 C of the calligraphy/drawing camera 3 is positioned off the sheet when the sheet is seen from right above, as shown in FIG. 13A .
- the region corresponding to the sheet (written material region GA) appears as a trapezoid as shown in FIG. 13B .
- the calligraphy/drawing camera 3 seen from the side at this time is as shown in FIG. 13C .
- the height N of the sheet with respect to the lower side length M is derived from [Expression 2] indicated below.
- N HT * tan ⁇ ( ⁇ + ⁇ ) + tan ⁇ ( ⁇ - ⁇ ) tan ⁇ ( ⁇ ) * 2 [ Expression ⁇ ⁇ 2 ]
- the PC 2 adds the amount of deformation calculated in the manner described above to the length of height and length of width of the sheet region respectively (step S 145 ).
- the PC 2 determines that the written material A is placed lengthwise (step S 147 ).
- the PC 2 determines that the written material A is placed widthwise (step S 148 ).
- the present embodiment it can be automatically detected whether the written material A is placed lengthwise or widthwise. Accordingly, the above-described adjustment of the captured image can be performed without the user's designation of the orientation of the written material A.
- the present embodiment relates to a method of a captured image projection apparatus having the same configuration as that shown in FIG. 1 to FIG. 3 , for automatically obtaining an image capturing angle used for adjusting a captured image of the aforementioned written material A.
- the PC 2 has a function for obtaining the angle of view of the calligraphy/drawing camera 3 at the time the written material A is image-captured, and the resolution of the image output from the calligraphy/drawing camera 3 .
- a program for controlling the PC 2 to perform a later-described inclination measuring process is stored in the storage device 23 of the PC 2 .
- steps S 151 to S 153 are performed by user's operations and steps S 154 to S 156 are performed by the PC 2 (automatically).
- steps S 151 to S 153 are performed by user's operations and steps S 154 to S 156 are performed by the PC 2 (automatically).
- the user places a specific mark M at a point reached by drawing a vertical line VL from the camera 3 C of the calligraphy/drawing camera 3 to the camera mounting 3 A (step S 151 ), as shown in FIGS. 15A and 15B .
- FIG. 15A is a front view showing the state of the calligraphy/drawing camera 3 where the mark M is placed, and FIG.
- 15B is a left side view of the calligraphy/drawing camera 3 in the same state.
- the user sets the orientation of the camera 3 C in a manner that the mark M appears within the image range of the camera 3 C (step S 152 ), and image-captures the camera mounting 3 A in this state (step S 153 ).
- the user operates the PC 2 to start the inclination measuring process.
- the PC 2 performs edge identification on a captured image G 1 , and obtains the position of the mark M (step S 154 ). Note that the positional data to be obtained represents coordinate values pixel-wise.
- the PC 2 calculates the distance (number of pixels) between the coordinates of the center of the captured image G 1 and the mark M (step S 155 ).
- the PC 2 calculates the inclination of the camera 3 C, i.e., the image capturing angle with respect to the upper surface of the camera mounting 3 A, based on the angle of view of the calligraphy/drawing camera 3 (step S 156 ).
- the captured image projection apparatus can adjust the captured image by using the image capturing angle ( ⁇ ) obtained in the manner described above. That is, the captured image projection apparatus carries out trapezoidal correction in the vertical direction and in the horizontal direction on the captured image of the written material A placed on the camera mounting 3 A, by using the image capturing angle ( ⁇ ). Due to this, the same image as that which is obtained when the written material A is image-captured vertically can be obtained.
- the present embodiment relates to a method of the captured image projection apparatus having the same configuration as that shown in FIG. 1 to FIG. 3 , for automatically obtaining an image capturing angle used for adjusting a captured image of the written material A in a different manner from the third embodiment.
- the calligraphy/drawing camera 3 can allow the height of the camera 3 C to be arbitrarily adjusted with respect to the camera mounting 3 A, and the PC 2 can obtain information regarding the height.
- a program for controlling the PC 2 to perform a later-described inclination measuring process is stored in the storage device 23 of the PC 2 .
- steps S 161 to S 163 are performed by users operation, and steps S 164 to S 166 are performed by the PC 2 (automatically).
- the user prepares a sheet (reference sheet) T on which squares representing an X-Y coordinate system are printed, as shown in FIGS. 17A and 17B .
- the user places the sheet on the camera mounting 3 A of the calligraphy/drawing camera 3 , and adjusts the position of the sheet such that the point reached by drawing a vertical line VL from the camera 3 C is the origin O (0, 0) (step S 161 ).
- FIG. 161 the procedure of the captured image projection apparatus for obtaining the image capturing angle in the present embodiment will be explained with reference to a flowchart shown in FIG. 16 .
- steps S 161 to S 163 are performed by users operation, and steps S 164 to S 166 are performed by the PC 2 (automatically).
- FIG. 17A is a front view showing the state of the calligraphy/drawing camera 3 where the sheet is placed at the right position
- FIG. 17B is a left side view of the calligraphy/drawing camera 3 in the same state.
- the user arbitrarily sets the orientation of the camera 3 C (step S 162 ), and image-captures the sheet T in this state (step S 163 ).
- the user operates the PC 2 to start the inclination measuring process.
- the PC 2 first performs edge identification on a captured image G 1 and obtains the coordinates of the edges on the squares (step S 164 ).
- the PC 2 obtains the distance between the coordinates of the center of the captured image G 1 and the origin O of the square coordinates by the number of pixels (step S 165 ).
- the PC 2 calculates the image capturing angle based on the distance and the height of the camera 3 C (step S 166 ).
- the captured image can be adjusted by using the image capturing angle ( ⁇ ) obtained in the manner described above. That is, the captured image projection apparatus performs trapezoidal correction in the vertical direction and in the horizontal direction on the captured image of the written material A placed on the camera mounting 3 A, by using the image capturing angle ( ⁇ ). Due to this, the same image as that which is obtained when the written material A is image-captured vertically can be obtained.
- the present embodiment relates to a method of the captured image projection apparatus having the same configuration as that shown in FIG. 1 to FIG. 3 , for correcting the orientation of the captured image of the written material A, which is due to a difference in orientation (inclination in a rotary direction) between the written material A and the camera 3 C.
- a program for controlling the PC 2 to perform a later-described rotation correction process is stored in the storage device 23 of the PC 2 .
- the procedure of the rotation correction process by the PC 2 will be explained with reference to a flowchart shown in FIG. 18 .
- the PC 2 corrects the rectangular deformation in the captured image input from the calligraphy/drawing camera 3 , in accordance with, for example, the method explained in the first embodiment (step S 181 ).
- the PC 2 investigates the marginal distribution of the character strings in the corrected captured image in different scanning directions, in a predetermined order shown in FIG. 19 (step S 182 ).
- the PC 2 checks whether the obtained marginal distribution is a normal distribution or not each time the marginal distribution in one of the different scanning directions is obtained (step S 183 ). In a case where the document (character strings) in the written material A is as shown in FIG.
- a normal marginal distribution E 1 as shown in FIG. 20B can be obtained if the captured image is not rotated with respect to the scanning direction.
- a not normal marginal distribution E 2 as shown in FIG. 21A is obtained.
- the PC 2 decides the angle of the scanning direction at this time as a correction angle (step S 184 ). After this, the PC 2 corrects the orientation of the captured image by rotating the captured image by the correction angle (step S 185 ). Due to this, the captured image projection apparatus can adjust the captured image such that the document D becomes horizontal with no inclination as shown in FIG. 20A .
- documents are written in roughly two different manners, namely, a vertical writing manner and a horizontal writing manner (the manner shown in the drawings). For example, if the document is vertically written, the captured image can be adjusted in a way that the character strings become vertical.
- the present embodiment relates to a method of the captured image projection apparatus having the same configuration as that shown in FIG. 1 to FIG. 3 , for correcting the orientation of the captured image of the written material A, which is due to a difference in orientation (inclination in a rotary direction) between the written material A and the camera 3 C, in a different manner from the fifth embodiment.
- a program for controlling the PC 2 to perform a later-described rotation correction process is stored in the storage device 23 of the PC 2 .
- the procedure of the rotation correction process by the PC 2 will be explained with reference to a flowchart shown in FIG. 22 .
- the PC 2 first corrects the rectangular deformation in a captured image G 1 input from the calligraphy/drawing camera 3 in accordance with, for example, the method explained in the first embodiment (step S 191 , FIG. 23A ). Then, the PC 2 identifies any edges in the corrected captured image in accordance with changes in color and luminance (step S 192 ). Then, the PC 2 finds at least one line L which corresponds to the outer circumference from the identified edges (step S 193 , FIG. 23B ). Subsequently, the PC 2 obtains the inclination of the image based on the line L (step S 194 ).
- the inclination obtained in this step includes the smaller one of the angle of the line L to a vertical line and angle of the line L to a horizontal line, and the rotation direction of the angle.
- the PC 2 adjusts the captured image G 1 to be vertical or horizontal by rotating the captured image G 1 based on the obtained inclination value (angle and rotation direction) (step S 195 , FIG. 23C ).
- the captured image projection apparatus can automatically obtain an adjusted captured image G 2 , by image-capturing the written material A in such a way that one of the material's sides is included in the captured image.
- the present embodiment relates to a method of the captured image projection apparatus having the same configuration as that shown in FIG. 1 to FIG. 3 , for correcting the orientation of the captured image of the written material A, which is due to a difference in orientation (inclination in a rotary direction) between the written material A and the camera 3 C, in a different manner from the fifth embodiment and sixth embodiment.
- a program for controlling the PC 2 to perform a later-described rotation correction process is stored in the storage device 23 of the PC 2 .
- a predetermined character recognition program involving the use of an OCR (Optical Character Reader), etc. for extracting and recognizing characters from an image is stored in the storage device 23 .
- the procedure of the rotation correction process by the PC 2 will be explained with reference to a flowchart shown in FIG. 24 .
- the PC 2 extracts an image region Ga in which characters are present, from a captured image G 1 after being subjected to correction of rectangular deformation in accordance with, for example, the method explained in the first embodiment (step S 201 , FIG. 25A ). This step may be omitted by regarding the full captured image G 1 as a region in which characters are present. Then, the PC 2 performs character recognition by rotating the extracted region Ga stepwise four times by 90° each time (step S 202 ), and obtains the angle at which the characters can be properly recognized (step S 203 , FIG. 25B ).
- the extraction of the image region Ga and the recognition of the characters are performed by reading the above-described character recognition program from the storage device 23 .
- the PC 2 rotates the entire captured image G 1 by the obtained angle (step S 204 , FIG. 25C ). Therefore, according to the present embodiment, the captured image projection apparatus can obtain an adjusted captured image G 2 in which the orientation of the characters is proper.
- the captured image projection apparatus having the same configuration as that shown in FIG. 1 to FIG. 3 performs the process shown in a flowchart of FIG. 26 in order to adjust a captured image of the written material A.
- a program for this process is stored in the storage device 23 of the PC 2 .
- the PC 2 first identifies any edges in a captured image G 1 in accordance with changes in color and luminance (step S 211 ), and determines whether circumferential four sides can be found in the identified edges. If determining that four sides can be found (step S 212 : YES), the PC 2 obtains the four circumferential lines from the identified edges and four vertexes in accordance with the method explained in the first embodiment. After this, the PC 2 obtains an image G 2 consisting only of a written material region (sheet region) in the captured image G 1 in accordance with the process explained in the earlier embodiment (step S 213 , FIG. 27A ).
- the PC 2 performs the following process. First, as shown in FIG. 27B , the user directly designates the four vertexes of the sheet region in the captured image G 1 by operation of the mouse, etc., in order for the PC 2 to obtain the positions of the four vertexes (step S 214 ). After this, the PC 2 transforms the quadrangle (sheet region) having the four vertexes into the smallest rectangle that circumscribes the quadrangle (step S 215 ).
- the captured image projection apparatus can obtain an adjusted captured image G 2 by a simple process, even if the difference in color and luminance between the written material region and its surrounding region is small and the circumferential four sides can not be recognized from the identified edges.
- the ninth embodiment of the present invention relates to a process of the PC 2 in the captured image projection apparatus according to, for example, the above-described third to seventh embodiments, for cutting out a necessary region from a captured image of a written material A which can not fully fall inside the image region of the camera 3 C, after performing correction of rectangular deformation or inclination on this captured image in the manners described above.
- the PC 2 receives user's designation of a rectangular region corresponding to a necessary region in an image G 2 after being corrected (transformed), in accordance with an operation by the user (step S 221 ).
- the designation of the necessary region is done by user's designating two vertexes on the same diagonal line of the rectangular region which the user wants to project, by operation of the mouse.
- the PC 2 cuts out the designated region from the image G 2 (step S 222 ).
- the PC 2 sends an image GB representing the cut-out region to the display software (step S 223 , FIG. 29B ), so that the image GB is displayed on the display device 25 .
- the present embodiment also relates to a process of the PC 2 in the captured image projection apparatus according to, for example, the above-described third to seventh embodiments, for cutting out a necessary region from a captured image of a written material A which can not fully fall inside the image region of the camera 3 C, after performing correction of rectangular deformation or inclination on this captured image in the manners described above.
- the PC 2 receives user's designation of a polygonal region corresponding to a necessary region in an image G 2 after being corrected (transformed), in accordance with an operation by the user (step S 231 ).
- the designation of the necessary region is done by user's designating each vertex of the polygonal region which the user wants to project, by operation of the mouse.
- the PC 2 cuts out the smallest rectangle that circumscribes the designated polygonal region from the image G 2 (step S 232 ).
- the PC 2 deletes (or whitens) regions GC corresponding to portions which are not designated by the user in the cut-out region (step S 233 ), and sends an image GB representing the region cut out in step S 232 to the display software (step S 234 , FIG. 31B ), so that the image GB is displayed on the display device 25 .
- the captured image projection apparatus can deal with the case where the user's necessary region is not rectangle.
- the eleventh embodiment of the present invention relates to a method for automatically obtaining an image capturing angle used for adjusting the captured image of the written material A accurately and in real time.
- the captured image projection apparatus having the same configuration as that shown in FIG. 1 to FIG. 3 additionally comprises a first gravity sensor 40 secured to the camera 3 C and a second gravity sensor 41 secured to the camera mounting 3 A, as shown in FIG. 32 .
- a program for controlling the PC 2 to perform a later-described image capturing angle obtaining process is stored in the storage device 23 of the PC 2 .
- the content of the image capturing angle obtaining process by the PC 2 in the present embodiment will be explained with reference to a flowchart shown in FIG. 33 .
- the PC 2 obtains a vertical direction L 1 from the first gravity sensor 40 added to the camera 3 C (step S 251 ), and obtains the angle ⁇ of the camera 3 C with respect to the vertical direction L 1 (step S 252 ).
- the PC 2 obtains a vertical direction L 2 from the second gravity sensor 41 added to the camera mounting 3 A (step S 253 ), and obtains the angle ⁇ of the camera mounting 3 A with respect to the vertical direction L 2 (step S 254 ).
- the PC 2 obtains the true inclination angle ⁇ (image capturing angle) of the camera 3 C by subtracting the inclination angle ⁇ of the camera mounting 3 A from the inclination angle ⁇ of the camera 3 C (step S 255 ).
- the captured image projection apparatus can adjust the captured image by using the image capturing angle ( ⁇ ) obtained in the way described above. That is, trapezoidal correction in the vertical direction and in the horizontal direction is performed on the captured image of the written material A placed on the camera mounting 3 A by using the image capturing angle ( ⁇ ). Due to this, the same image as that obtained by image-capturing the written material A vertically can be obtained.
- the captured image projection apparatus can automatically obtain the image capturing angle ( ⁇ ) accurately in real time, without the preparatory steps as in the third and fourth embodiments, which is convenient for the user.
- the captured image is corrected based on image information obtained from the captured image, not only correction of trapezoidal deformation but also various corrections can be performed on the captured image. Due to this, it is possible to deal with various corrections necessary for adjustment of the captured image before projection.
- the inclination of the captured image can be corrected based on a proper image capturing angle directly obtained from the captured image. Due to this, it becomes possible to accurately perform correction on the captured image before projection.
- a proper image capturing angle can be obtained even in a case where the captured image is obtained in a state where the written material is not placed on the camera mounting and the camera mounting is inclined. Due to this, correction before projection, of the captured image obtained by the camera supported by the camera mounting can be accurately performed.
Abstract
A calligraphy/drawing camera image-captures a document and sends a resultant captured image to a personal computer (S101). The personal computer applies various image processes such as correction of various deformations including a trapezoidal deformation based on image information obtained from the captured image, a rotation process, a cut out process, etc. to the captured image (S102 to S104). The personal computer sends the processed image to a projector (S105). The projector projects the processed image on a screen (S106). Since the captured image is corrected based on the image information obtained from the captured image, not only correction of a trapezoidal deformation but also various other corrections can be done on the captured image.
Description
- The present application is a Divisional Application of U.S. application Ser. No. 10/858,708, filed Jun. 1, 2004, which is incorporated herein by reference.
- 1. Field of the Invention
- The present invention relates to a captured image projection apparatus and a correction method for a captured image.
- 2. Description of the Related Art
- Recently, projectors are used in presentations, meetings, etc. to expandingly project documents (characters and drawings, etc.) stored in personal computers on a screen. Projectors generally have a configuration for converting image information of documents obtained from personal computers, etc. into projection light by using an image conversion element such as liquid crystal, micro mirror array, etc.
- In a case where a projector is used in a meeting, etc., in addition to the documents in the personal computer, written materials (newspapers, articles in magazines, presentation papers, written documents, etc.) one has on hand may need to be projected. In this case, since the written materials need to be digitalized (need to be converted into image data), image capturing of the written materials is done by using a so-called calligraphy/drawing camera. As the calligraphy/drawing camera, for example, one that has a configuration of a digital camera having a CCD and a MOS type image capture element is used.
- In image-capturing the written materials using the calligraphy/drawing camera for digitalization, it is necessary to match the orientation of the paper and that of the calligraphy/drawing camera. For example, in image-capturing, if the camera is inclined, that is, if the center axis of the lens is not perpendicular to the written material, the resultant captured image is trapezoidally deformed and cannot be a rectangular image. A method of solving this problem is disclosed in, for example, Unexamined Japanese Patent Application KOKAI Publication No. 2002-354331 (
FIG. 6 ). According to this method, in an image capturing apparatus in which a camera unit is supported by a support pillar on a document mounting, an angle detection sensor for detecting the inclination angle of the camera unit with respect to the document mounting is provided. By using the inclination angle detected by the angle detection sensor, the trapezoidal deformation in the captured image is corrected. - However, according to this method for correcting the captured image based only on the inclination angle of the camera unit, adjustment of the captured image before projection is limited to only correction of the trapezoidal deformation. For example, in image-capturing a written material for projecting the material on a screen, it is difficult to match the size, the orientation, etc. of the photo-object (written material) with the image range of the camera accurately. If correction other than correction for trapezoidal deformation is required as in this case, the above-described method cannot adjust the captured image.
- Further, according to the above-described method, in a case where image-capturing of a large-sized written material is done without mounting the written material on the document mounting and by adjusting the orientation of the camera unit, if the written material is inclined, it is difficult to obtain the accurate inclination angle of the camera unit, i.e. the accurate image capturing angle with respect to the written material. Accordingly, in this case, there is a problem that trapezoidal deformation in the captured image cannot be properly corrected.
- The present invention was made in view of the above circumstance, and relates to a captured image projection apparatus and a captured image correction method which are capable of dealing with various corrections necessary for adjusting a captured image before projection, and of performing an accurate correction.
- According to one aspect of the invention, an image capturing apparatus is provided which includes: a document table adapted to have a written material placed thereon; an image capturing section which captures an image of the written material; an obtaining section which obtains image information from the captured image, the image information including four lines located closest to four circumferential sides, respectively, of the captured image; and a correction section which corrects distortion of an image region corresponding to the written material within the captured image based on the image information, the correction of the distortion comprising deforming the image region into a rectangular shape.
- These objects and other objects and advantages of the present invention will become more apparent upon reading of the following detailed description and the accompanying drawings in which:
-
FIG. 1 is a configuration diagram of a captured image projection apparatus which is common to each embodiment of the present invention; -
FIG. 2 is a block diagram showing a schematic electric structure of the captured image projection apparatus ofFIG. 1 ; -
FIG. 3 is a block diagram showing a schematic electric structure of a calligraphy/drawing camera according to first to tenth embodiments of the present invention; -
FIG. 4 is a flowchart showing a basic operation of the captured image projection apparatus which is common to each embodiment; -
FIG. 5 is a flowchart showing a procedure of a rectangular conversion process by a PC according to the first embodiment; -
FIGS. 6A to 6C are explanatory diagrams for the rectangular conversion process by the PC; -
FIGS. 7A to 7E are explanatory diagrams showing the content of shape conversion in the rectangular conversion process by the PC; -
FIG. 8 is a flowchart showing a procedure of an aspect ratio conversion process by the PC; -
FIGS. 9A to 9D are explanatory diagrams showing the content of the aspect ratio conversion process by the PC; -
FIG. 10 is a flowchart showing an operation of the PC after the aspect ratio conversion process by the PC is completed; -
FIGS. 11A to 11D are explanatory diagrams showing the content of a cut out process by the PC; -
FIG. 12 is a flowchart showing the content of a direction detection process by the PC in the second embodiment; -
FIGS. 13A to 13C are explanatory diagrams showing a reason why a deformation is produced in a captured image, and a state where a deformation is produced; -
FIG. 14 is a flowchart showing a procedure for obtaining an image capturing angle according to the third embodiment; -
FIGS. 15A and 15B are explanatory diagrams showing a state of a calligraphy/drawing camera when obtaining an image capturing angle; -
FIG. 16 is a flowchart showing a procedure for obtaining a image capturing angle according to the fourth embodiment; -
FIGS. 17A and 17B are explanatory diagrams showing a state of the calligraphy/drawing camera when obtaining an image capturing angle; -
FIG. 18 is a flowchart showing a procedure of a rotation correction process by the PC according to the fifth embodiment; -
FIG. 19 is an explanatory diagram showing an order of scanning directions to a captured image; -
FIGS. 20A and 20B are exemplary diagrams showing a relation among a document, a scanning direction, and a marginal distribution; -
FIGS. 21A and 21B are exemplary diagrams showing a relation among a document, scanning direction, and a marginal distribution in a case where the marginal distribution is not normal; -
FIG. 22 is a flowchart showing a procedure of a rotation correction process by the PC according to the sixth embodiment; -
FIGS. 23A to 23C are explanatory diagrams of the rotation correction process by the PC according to the sixth embodiment; -
FIG. 24 is a flowchart showing a procedure of a rotation correction process by the PC according to the seventh embodiment; -
FIGS. 25A to 25C are explanatory diagrams of the rotation correction process by the PC according to the seventh embodiment; -
FIG. 26 is a flowchart showing an operation of the PC according to the eighth embodiment; -
FIGS. 27A and 27B are explanatory diagrams showing the operation of the PC according to the eighth embodiment; -
FIG. 28 is a flowchart showing an operation of the PC according to the ninth embodiment; -
FIGS. 29A and 29B are explanatory diagrams showing the operation of the PC according to the ninth embodiment; -
FIG. 30 is a flowchart showing an operation of the PC according to the tenth embodiment; -
FIGS. 31A and 31B are explanatory diagrams showing the operation of the PC according to the tenth embodiment; -
FIG. 32 is a block diagram showing a schematic electric structure of the calligraphy/drawing camera according to the eleventh embodiment of the present invention; -
FIG. 33 is a flowchart showing the content of an image capturing angle obtaining process performed by the PC according to the eleventh embodiment; and -
FIG. 34 is an explanatory diagrams showing a state of the calligraphy/drawing camera when obtaining an image capturing angle. - One embodiment of the present invention will now be explained with reference to the drawings.
- A captured image projection apparatus comprises a
projector 1, a lap-top personal computer (hereinafter referred to as PC) 2, and a calligraphy/drawing camera 3, as its configuration diagram being shown inFIG. 1 . Theprojector 1 and thePC 2 are connected via anRGB cable 4. ThePC 2 and the calligraphy/drawing camera 3 are connected via anUSB cable 5. - The
projector 1 is structured such that aprojection unit 1A in which an optical system such as a projection lens, etc. is stored is provided on the front surface of the projector body. An optical system such as a projection lens, etc. is stored in theprojection unit 1A. Theprojector 1 projects documents and images on a screen S (or a whiteboard, etc.) by irradiating a light from theprojection unit 1A. - The calligraphy/
drawing camera 3 comprise a camera mounting 3A, asupport pillar 3B which is provided to stand on the camera mounting 3A, and acamera 3C set on the top end of thesupport pillar 3B. Thecamera 3C is provided so as to be able to turn upward and downward, and leftward and rightward. When the calligraphy/drawing camera 3 is used, a written material A such as a newspaper, an article in a magazine, a presentation paper, a written document, etc. is mounted on the camera mounting 3A as illustrated, and is image-captured (imaged, picked-up) by thecamera 3C. - The
projector 1 mainly comprises aCPU 11, aROM 12, aRAM 13, adisplay unit 14, and akey input unit 15, as its schematic electrical structure being shown inFIG. 2 . Further, the projector has aninput terminal 16 for an RGB signal, and theRGB cable 4 is connected to theinput terminal 16. - The
display unit 14 is the projection section according to the present invention. Thedisplay unit 14 includes a light source such as krypton lamp, etc., an image conversion element such as liquid crystal, micro mirror array, etc., a drive circuit, and an optical system such as a projection lens, etc. Thedisplay unit 14 projects documents and captured imaged sent from thePC 2 on the screen S. The image conversion element is for converting a light from the light source into a projection light, and the drive circuit is for driving the image conversion element in accordance with an RGB signal input from theinput terminal 16. Thekey input unit 15 is constituted by a plurality of operation buttons for receiving operations on theprojector 1 by a user. - The
CPU 11 operates in accordance with a program stored in theROM 12, and controls each of the aforementioned components by using theRAM 13 as a work memory. Theinput terminal 16 is provided in an unillustrated input image processing circuit which is connected to theCPU 11. - The
PC 2 mainly comprises aCPU 21, aRAM 22, astorage device 23, aninput device 24 such as a plurality of keys provided on the body of thePC 2 and a mouse connected to the body, etc., and adisplay device 25 constituted by an LCD and a drive circuit therefore, etc. Further, thePC 2 has anRGB output terminal 26 to which theRGB cable 4 is connected, and aUSB connection terminal 27 to which theUSB cable 5 is connected. - The
storage device 23 is constituted by a hard disk or the like having a relatively large storage capacity. An image processing program and an image display program are stored in thestorage device 23. The image processing program is a program for performing later-described various image processes to be applied to a captured image. The image display program is a program for displaying an image based on image data on thedisplay device 25 or for making the image be projected by theprojector 1. TheRGB output terminal 26 is provided in an unillustrated image signal processing circuit which is connected to theCPU 21. TheUSB connection terminal 27 is provided in an unillustrated USB input/output interface circuit which is connected to theCPU 21. -
FIG. 3 is a block diagram showing the electric structure of the calligraphy/drawing camera 3 (mainly thecamera 3C in the present embodiment). The calligraphy/drawing camera 3 (mainly thecamera 3C in the present embodiment) comprises aCCD 31 and asignal processing unit 32, as its electric structure being shown inFIG. 3 . TheCCD 31 is an imaging element for imaging a written material A via an unillustrated lens. Thesignal processing unit 32 applies various signal processes to an analog imaging signal output from theCCD 31 and then outputs the signal as a digital image signal. The image signal output from thesignal processing unit 32 is sent to anLCD 33 and displayed thereon as a photo-object image when the calligraphy/drawing camera 3 is in an image capture standby state. When the calligraphy/drawing camera 3 is in an image capture state, the image signal is compressed by aCPU 34 in accordance with a predetermined format such as JPEG and stored in animage memory 35. Further, the image signal is output from aUSB interface 36 to thePC 2 via theUSB cable 5. - The compressed image data stored in the
image memory 35 is read by theCPU 34 if necessity arises. The read image data is expanded and then reproduced and displayed on theLCD 33 as a still image, or output to thePC 2. Theimage memory 35 is a non-volatile memory such as a flash memory, etc. which is built inside thecamera 3C or detachable from thecamera 3C. Further, the calligraphy/drawing camera 3 comprises aROM 37, aRAM 38, and akey input unit 39. Various control programs necessary for theCPU 34 to perform compression and expansion of the image data or to control the entire device are stored in theROM 37. TheRAM 38 is a work memory of theCPU 34. Thekey input unit 39 is constituted by a shutter key and many other operation keys. - The outline of the operation of the captured image projection apparatus having the above configuration will now be explained with reference to a flowchart shown in
FIG. 4 . - In the captured image projection apparatus, the calligraphy/
drawing camera 3 image-captures an arbitrary document or the like (written material A) and outputs the image data of the captured document to the PC 2 (step S101). ThePC 2 corrects the captured image by performing various image processes such as a deformation correction process, a rotation process, a cut out process, etc. on the input image data (step S102 to S104). Then, thePC 2 outputs the processed image data to theprojector 1 in accordance with the display program (step S105). ThePC 2 performs part or all of the image processes in accordance with necessity. Theprojector 1 projects the captured image of the document or the like on the screen S based on the image data input from the PC 2 (step S106). - Next, an operation according to the present embodiment for adjusting the captured image of the written material A captured by the calligraphy/
drawing camera 3 for projecting the image on the screen S will be explained. - In the captured image projection apparatus, when a captured image is output from the calligraphy/
drawing camera 3 to thePC 2, thePC 2 performs a rectangular conversion process. The rectangular conversion process is a process for correcting deformation in a captured image and converting the shape of the image into a rectangle. The rectangular conversion process will now be explained with reference toFIG. 5 showing a flowchart of the procedure of the process, andFIGS. 6A to 6C showing diagrams explaining the procedure. - The
PC 2 identifies edges in a captured image G1 in accordance with changes in color and luminance (step S11,FIG. 6A ). ThePC 2 finds straight lines M1 to M4 which are close to four sides L1 to L4 of the outer circumference of the captured image G1 from the identified edges (step S112,FIG. 6B ). ThePC 2 further finds intersections of the straight lines M1 to M4, i.e., four vertexes (step S113). ThePC 2 regards the quadrangle formed by the four vertexes as a written material region GA in the captured image G1, as shown inFIGS. 7A to 7E . Then, thePC 2 converts the shape of the written material region GA in a way that the four vertexes of the region GA are moved to four vertexes of the smallest rectangle B that circumscribes the written material region GA (step S114,FIG. 6C ). Due to this, an image G2 which is a result of correcting the deformation of the shape with respect to a rectangle (hereinafter this deformation will be referred to as rectangular deformation) is obtained in the written material region GA. Needless to say, also in the case where the rectangular deformation of the written material region GA is a trapezoidal deformation, this deformation is corrected by the above-described process. - Subsequently, the
PC 2 performs an aspect ratio conversion process, as its procedure being shown in a flowchart ofFIG. 8 . In this process, upon receiving a user's operation for selecting a sheet, thePC 2 performs setting for the sheet (step S121). If a regular sheet is not selected by the user in this step, the aspect ratio conversion process is terminated. In a case where the regular sheet is selected and the sheet is placed lengthwise (step S122 and step S123: both YES), thePC 2 sets the aspect ratio to “1.41:1” (step S124). In contrast, if the sheet is placed widthwise (step S123: NO, and step S125: YES), thePC 2 sets the aspect ratio to “1:1.41” (step S126). - After this,
PC 2 transforms the written material region GA which has been through the rectangular deformation correction, by either of the set aspect ratios (step S127). In this process, thePC 2 performs the transformation based on coordinate data of the four vertexes of the smallest rectangle B used in the conversion (step S114) in the rectangular conversion process explained above. For example, in a case where widthwise placement of a regular sheet is designated, thePC 2 transforms the written material region GA by expanding the region GA in the vertical direction such that the aspect ratio of the region GA will be “1:1.41”, as shown inFIG. 9 . In a case where lengthwise placement of a regular sheet is designated, thePC 2 transforms the written material region GA by expanding the region GA in the horizontal direction such that the aspect ratio will be “1.41:1”. As a result, thePC 2 obtains an image G3 in which the written material region GA has the aspect ratio of the regular sheet. - After this, as shown in a flowchart of
FIG. 10 , thePC 2 cuts out the region corresponding to the sheet from the image G3 which has been subjected to the above-described transformation based on the coordinate data of the four vertexes of the smallest rectangle B used in the conversion (step S114) in the rectangular conversion process as shown inFIG. 11 (step S131). Then, thePC 2 sends an image G4 obtained after adjustment which consists of only the cut-out image, i.e., the written material region GA to the display software (step S132), so that the image G4 will be displayed on thedisplay device 25. - Due to this, when the written material A is image-captured by the calligraphy/
drawing camera 3 and projected on the screen S, a fine captured image having no rectangular deformation and no unnecessary frame image can be projected on the screen S. - Next, the second embodiment of the present invention will be explained. In the present embodiment, unlike the first embodiment, the captured image projection apparatus having the same structure as that shown in
FIG. 1 toFIG. 3 detects whether the written material A in the captured image is placed lengthwise or widthwise when adjusting the captured image of the written material A. - In the present embodiment, the
PC 2 has a function for obtaining the angle of the lens of calligraphy/drawing camera 3 at which the written material A fully falls within the image range ofcamera 3 at the time the written material A is image-captured (this angle will hereinafter be referred to as angle of view), and for obtaining the resolution of the image output from the calligraphy/drawing camera 3. In thestorage device 23 of thePC 2, a program for controlling thePC 2 to perform a direction detection process to be described below is stored. - The direction detection process by the
PC 2 will now be explained with reference to a flowchart shown inFIG. 12 . ThePC 2 first obtains the angle of view of the calligraphy/drawing camera 3 (step S141). Then, thePC 2 obtains the resolution of the image data (step S142). Subsequently, thePC 2 obtains four vertexes of the quadrangle corresponding to the sheet (written material region GA) from the captured image (step S143). This process is the same as the process in the rectangular conversion process (seeFIG. 5 ) explained in the first embodiment. - Next, the
PC 2 calculates the amount of deformation in the image based on the coordinates of the obtained four vertexes by using the formerly obtained angle of view and resolution (step S144). The calculation method for this process will now be explained. Generally, deformation in an image is produced if the sheet (written material A) is image-captured obliquely in a state where thecamera 3C of the calligraphy/drawing camera 3 is positioned off the sheet when the sheet is seen from right above, as shown inFIG. 13A . In the captured image, the region corresponding to the sheet (written material region GA) appears as a trapezoid as shown inFIG. 13B . The calligraphy/drawing camera 3 seen from the side at this time is as shown inFIG. 13C . - As illustrated in
FIGS. 13A to 13C , in a case where -
- the angle of view of the camera is represented by α,
- the resolution of the image is represented by H (pixel value),
- the length of the upper side of the sheet region is represented by L (pixel value),
- the length of the lower side of the sheet region is represented by M (pixel value), and
- the height of the sheet region is represented by HT (pixel value),
- the angle β within the angle of view corresponding to the sheet region is represented by β=α×HT/H, and
- the slope of the camera is represented by Δ.
- In this case, [Expression 1] indicated below is established.
-
- Δ is derived from [Expression 1].
- After Δ is derived, the height N of the sheet with respect to the lower side length M is derived from [Expression 2] indicated below.
-
- Then, the
PC 2 adds the amount of deformation calculated in the manner described above to the length of height and length of width of the sheet region respectively (step S145). After this addition, if the length of height is larger than the length of width (step S146: YES), thePC 2 determines that the written material A is placed lengthwise (step S147). On the contrary, if the length of height is not larger than the length of width (step S146: NO), thePC 2 determines that the written material A is placed widthwise (step S148). - As explained above, according to the present embodiment, it can be automatically detected whether the written material A is placed lengthwise or widthwise. Accordingly, the above-described adjustment of the captured image can be performed without the user's designation of the orientation of the written material A.
- Next, the third embodiment of the present invention will be explained. The present embodiment relates to a method of a captured image projection apparatus having the same configuration as that shown in
FIG. 1 toFIG. 3 , for automatically obtaining an image capturing angle used for adjusting a captured image of the aforementioned written material A. In the present embodiment, thePC 2 has a function for obtaining the angle of view of the calligraphy/drawing camera 3 at the time the written material A is image-captured, and the resolution of the image output from the calligraphy/drawing camera 3. Further, a program for controlling thePC 2 to perform a later-described inclination measuring process is stored in thestorage device 23 of thePC 2. - Next, the procedure of the captured image projection apparatus for obtaining the image capturing angle will be explained with reference to a flowchart shown in
FIG. 14 . Note that steps S151 to S153 are performed by user's operations and steps S154 to S156 are performed by the PC 2 (automatically). First, as a preparatory step, the user places a specific mark M at a point reached by drawing a vertical line VL from thecamera 3C of the calligraphy/drawing camera 3 to the camera mounting 3A (step S151), as shown inFIGS. 15A and 15B .FIG. 15A is a front view showing the state of the calligraphy/drawing camera 3 where the mark M is placed, andFIG. 15B is a left side view of the calligraphy/drawing camera 3 in the same state. Next, the user sets the orientation of thecamera 3C in a manner that the mark M appears within the image range of thecamera 3C (step S152), and image-captures the camera mounting 3A in this state (step S153). Subsequently, the user operates thePC 2 to start the inclination measuring process. ThePC 2 performs edge identification on a captured image G1, and obtains the position of the mark M (step S154). Note that the positional data to be obtained represents coordinate values pixel-wise. Then, thePC 2 calculates the distance (number of pixels) between the coordinates of the center of the captured image G1 and the mark M (step S155). After this, thePC 2 calculates the inclination of thecamera 3C, i.e., the image capturing angle with respect to the upper surface of the camera mounting 3A, based on the angle of view of the calligraphy/drawing camera 3 (step S156). - Angle calculation at this time is performed in the manner described below. In a case where, as shown in
FIG. 15B , -
- the angle of view of the camera is represented by α,
- the length of a circular arc AB is represented by L, and
- the length of a circular arc CD is represented by M,
- the inclination (image capturing angle Δ) of the
camera 3C to be calculated is represented by (Δ)=α×M/L. - Further, the length (L) of the circular arc AB and the length (M) of the circular arc CD correspond to the line segment EF and line segment GH of the captured image G1, respectively. Therefore, the inclination (Δ) of the
camera 3C is derived by Inclination (Δ)=α×line segment EF/line segment GH. - Accordingly, in the present embodiment, the captured image projection apparatus can adjust the captured image by using the image capturing angle (Δ) obtained in the manner described above. That is, the captured image projection apparatus carries out trapezoidal correction in the vertical direction and in the horizontal direction on the captured image of the written material A placed on the camera mounting 3A, by using the image capturing angle (Δ). Due to this, the same image as that which is obtained when the written material A is image-captured vertically can be obtained.
- Next, the fourth embodiment of the present invention will be explained. The present embodiment relates to a method of the captured image projection apparatus having the same configuration as that shown in
FIG. 1 toFIG. 3 , for automatically obtaining an image capturing angle used for adjusting a captured image of the written material A in a different manner from the third embodiment. In the captured image projection apparatus of the present embodiment, the calligraphy/drawing camera 3 can allow the height of thecamera 3C to be arbitrarily adjusted with respect to the camera mounting 3A, and thePC 2 can obtain information regarding the height. Further, a program for controlling thePC 2 to perform a later-described inclination measuring process is stored in thestorage device 23 of thePC 2. - The procedure of the captured image projection apparatus for obtaining the image capturing angle in the present embodiment will be explained with reference to a flowchart shown in
FIG. 16 . Note that steps S161 to S163 are performed by users operation, and steps S164 to S166 are performed by the PC 2 (automatically). First, as a preparatory step, the user prepares a sheet (reference sheet) T on which squares representing an X-Y coordinate system are printed, as shown inFIGS. 17A and 17B . The user places the sheet on the camera mounting 3A of the calligraphy/drawing camera 3, and adjusts the position of the sheet such that the point reached by drawing a vertical line VL from thecamera 3C is the origin O (0, 0) (step S161).FIG. 17A is a front view showing the state of the calligraphy/drawing camera 3 where the sheet is placed at the right position, andFIG. 17B is a left side view of the calligraphy/drawing camera 3 in the same state. The user arbitrarily sets the orientation of thecamera 3C (step S162), and image-captures the sheet T in this state (step S163). Then, the user operates thePC 2 to start the inclination measuring process. ThePC 2 first performs edge identification on a captured image G1 and obtains the coordinates of the edges on the squares (step S164). Then, thePC 2 obtains the distance between the coordinates of the center of the captured image G1 and the origin O of the square coordinates by the number of pixels (step S165). Then, thePC 2 calculates the image capturing angle based on the distance and the height of thecamera 3C (step S166). - The calculation of the angle is performed in the manner described below. In
FIG. 17B , in a case where -
- the height of the camera is represented by H, and
- the shift amount (distance between the coordinates of the center of the captured image G1 and the origin O of the square coordinates) is represented by W,
- the inclination (image capturing angle) A of the
camera 3C is derived by inclination Δ=arctan (W/H). - Therefore, according to the present embodiment, the captured image can be adjusted by using the image capturing angle (Δ) obtained in the manner described above. That is, the captured image projection apparatus performs trapezoidal correction in the vertical direction and in the horizontal direction on the captured image of the written material A placed on the camera mounting 3A, by using the image capturing angle (Δ). Due to this, the same image as that which is obtained when the written material A is image-captured vertically can be obtained.
- The fifth embodiment of the present invention will now be explained. The present embodiment relates to a method of the captured image projection apparatus having the same configuration as that shown in
FIG. 1 toFIG. 3 , for correcting the orientation of the captured image of the written material A, which is due to a difference in orientation (inclination in a rotary direction) between the written material A and thecamera 3C. A program for controlling thePC 2 to perform a later-described rotation correction process is stored in thestorage device 23 of thePC 2. - The procedure of the rotation correction process by the
PC 2 will be explained with reference to a flowchart shown inFIG. 18 . ThePC 2 corrects the rectangular deformation in the captured image input from the calligraphy/drawing camera 3, in accordance with, for example, the method explained in the first embodiment (step S181). Then, thePC 2 investigates the marginal distribution of the character strings in the corrected captured image in different scanning directions, in a predetermined order shown inFIG. 19 (step S182). Then, thePC 2 checks whether the obtained marginal distribution is a normal distribution or not each time the marginal distribution in one of the different scanning directions is obtained (step S183). In a case where the document (character strings) in the written material A is as shown inFIG. 20A , a normal marginal distribution E1 as shown inFIG. 20B can be obtained if the captured image is not rotated with respect to the scanning direction. In a case where the captured image is rotated with respect to the scanning direction, a not normal marginal distribution E2 as shown inFIG. 21A is obtained. - When a normal marginal distribution E is obtained in any scanning line (step S183: YES,
FIG. 21B ), thePC 2 decides the angle of the scanning direction at this time as a correction angle (step S184). After this, thePC 2 corrects the orientation of the captured image by rotating the captured image by the correction angle (step S185). Due to this, the captured image projection apparatus can adjust the captured image such that the document D becomes horizontal with no inclination as shown inFIG. 20A . Generally, documents are written in roughly two different manners, namely, a vertical writing manner and a horizontal writing manner (the manner shown in the drawings). For example, if the document is vertically written, the captured image can be adjusted in a way that the character strings become vertical. - Next, the sixth embodiment of the present invention will be explained. The present embodiment relates to a method of the captured image projection apparatus having the same configuration as that shown in
FIG. 1 toFIG. 3 , for correcting the orientation of the captured image of the written material A, which is due to a difference in orientation (inclination in a rotary direction) between the written material A and thecamera 3C, in a different manner from the fifth embodiment. A program for controlling thePC 2 to perform a later-described rotation correction process is stored in thestorage device 23 of thePC 2. - The procedure of the rotation correction process by the
PC 2 will be explained with reference to a flowchart shown inFIG. 22 . ThePC 2 first corrects the rectangular deformation in a captured image G1 input from the calligraphy/drawing camera 3 in accordance with, for example, the method explained in the first embodiment (step S191,FIG. 23A ). Then, thePC 2 identifies any edges in the corrected captured image in accordance with changes in color and luminance (step S192). Then, thePC 2 finds at least one line L which corresponds to the outer circumference from the identified edges (step S193,FIG. 23B ). Subsequently, thePC 2 obtains the inclination of the image based on the line L (step S194). The inclination obtained in this step includes the smaller one of the angle of the line L to a vertical line and angle of the line L to a horizontal line, and the rotation direction of the angle. ThePC 2 adjusts the captured image G1 to be vertical or horizontal by rotating the captured image G1 based on the obtained inclination value (angle and rotation direction) (step S195,FIG. 23C ). - Accordingly, even if the written material A is not a document, and moreover, is a picture or a photograph including no characters, the captured image projection apparatus can automatically obtain an adjusted captured image G2, by image-capturing the written material A in such a way that one of the material's sides is included in the captured image.
- Next, the seventh embodiment of the present invention will be explained. The present embodiment relates to a method of the captured image projection apparatus having the same configuration as that shown in
FIG. 1 toFIG. 3 , for correcting the orientation of the captured image of the written material A, which is due to a difference in orientation (inclination in a rotary direction) between the written material A and thecamera 3C, in a different manner from the fifth embodiment and sixth embodiment. A program for controlling thePC 2 to perform a later-described rotation correction process is stored in thestorage device 23 of thePC 2. Further, a predetermined character recognition program involving the use of an OCR (Optical Character Reader), etc. for extracting and recognizing characters from an image is stored in thestorage device 23. - The procedure of the rotation correction process by the
PC 2 will be explained with reference to a flowchart shown inFIG. 24 . ThePC 2 extracts an image region Ga in which characters are present, from a captured image G1 after being subjected to correction of rectangular deformation in accordance with, for example, the method explained in the first embodiment (step S201,FIG. 25A ). This step may be omitted by regarding the full captured image G1 as a region in which characters are present. Then, thePC 2 performs character recognition by rotating the extracted region Ga stepwise four times by 90° each time (step S202), and obtains the angle at which the characters can be properly recognized (step S203,FIG. 25B ). The extraction of the image region Ga and the recognition of the characters are performed by reading the above-described character recognition program from thestorage device 23. ThePC 2 rotates the entire captured image G1 by the obtained angle (step S204,FIG. 25C ). Therefore, according to the present embodiment, the captured image projection apparatus can obtain an adjusted captured image G2 in which the orientation of the characters is proper. - The eight embodiment of the present invention will now be explained. According to the present embodiment, the captured image projection apparatus having the same configuration as that shown in
FIG. 1 toFIG. 3 performs the process shown in a flowchart ofFIG. 26 in order to adjust a captured image of the written material A. A program for this process is stored in thestorage device 23 of thePC 2. - In the present embodiment, the
PC 2 first identifies any edges in a captured image G1 in accordance with changes in color and luminance (step S211), and determines whether circumferential four sides can be found in the identified edges. If determining that four sides can be found (step S212: YES), thePC 2 obtains the four circumferential lines from the identified edges and four vertexes in accordance with the method explained in the first embodiment. After this, thePC 2 obtains an image G2 consisting only of a written material region (sheet region) in the captured image G1 in accordance with the process explained in the earlier embodiment (step S213,FIG. 27A ). - In a case where four sides can not be found, i.e. circumferential four sides can not be recognized in the identified edges (step S212: NO), the
PC 2 performs the following process. First, as shown inFIG. 27B , the user directly designates the four vertexes of the sheet region in the captured image G1 by operation of the mouse, etc., in order for thePC 2 to obtain the positions of the four vertexes (step S214). After this, thePC 2 transforms the quadrangle (sheet region) having the four vertexes into the smallest rectangle that circumscribes the quadrangle (step S215). - Therefore, according to the present embodiment, the captured image projection apparatus can obtain an adjusted captured image G2 by a simple process, even if the difference in color and luminance between the written material region and its surrounding region is small and the circumferential four sides can not be recognized from the identified edges.
- The ninth embodiment of the present invention will be explained. The present embodiment relates to a process of the
PC 2 in the captured image projection apparatus according to, for example, the above-described third to seventh embodiments, for cutting out a necessary region from a captured image of a written material A which can not fully fall inside the image region of thecamera 3C, after performing correction of rectangular deformation or inclination on this captured image in the manners described above. - In the present embodiment, as shown in a flowchart of
FIG. 28 , thePC 2 receives user's designation of a rectangular region corresponding to a necessary region in an image G2 after being corrected (transformed), in accordance with an operation by the user (step S221). The designation of the necessary region is done by user's designating two vertexes on the same diagonal line of the rectangular region which the user wants to project, by operation of the mouse. Then, thePC 2 cuts out the designated region from the image G2 (step S222). Then, thePC 2 sends an image GB representing the cut-out region to the display software (step S223,FIG. 29B ), so that the image GB is displayed on thedisplay device 25. - Therefore, final adjustment can be performed on a captured image obtained by image-capturing a written material A which does not fully fall inside the image range.
- The tenth embodiment of the present invention will be explained. The present embodiment also relates to a process of the
PC 2 in the captured image projection apparatus according to, for example, the above-described third to seventh embodiments, for cutting out a necessary region from a captured image of a written material A which can not fully fall inside the image region of thecamera 3C, after performing correction of rectangular deformation or inclination on this captured image in the manners described above. - In the present embodiment, as shown in a flowchart of
FIG. 30 , thePC 2 receives user's designation of a polygonal region corresponding to a necessary region in an image G2 after being corrected (transformed), in accordance with an operation by the user (step S231). The designation of the necessary region is done by user's designating each vertex of the polygonal region which the user wants to project, by operation of the mouse. Then, thePC 2 cuts out the smallest rectangle that circumscribes the designated polygonal region from the image G2 (step S232). Then, thePC 2 deletes (or whitens) regions GC corresponding to portions which are not designated by the user in the cut-out region (step S233), and sends an image GB representing the region cut out in step S232 to the display software (step S234,FIG. 31B ), so that the image GB is displayed on thedisplay device 25. - Therefore, final adjustment can be performed on a captured image obtained by image-capturing a written material A which does not fully fall inside the image range. In addition, the captured image projection apparatus can deal with the case where the user's necessary region is not rectangle.
- The eleventh embodiment of the present invention will be explained. The present embodiment relates to a method for automatically obtaining an image capturing angle used for adjusting the captured image of the written material A accurately and in real time. In the present embodiment, the captured image projection apparatus having the same configuration as that shown in
FIG. 1 toFIG. 3 additionally comprises afirst gravity sensor 40 secured to thecamera 3C and asecond gravity sensor 41 secured to the camera mounting 3A, as shown inFIG. 32 . A program for controlling thePC 2 to perform a later-described image capturing angle obtaining process is stored in thestorage device 23 of thePC 2. - The content of the image capturing angle obtaining process by the
PC 2 in the present embodiment will be explained with reference to a flowchart shown inFIG. 33 . ThePC 2 obtains a vertical direction L1 from thefirst gravity sensor 40 added to thecamera 3C (step S251), and obtains the angle α of thecamera 3C with respect to the vertical direction L1 (step S252). Then, thePC 2 obtains a vertical direction L2 from thesecond gravity sensor 41 added to the camera mounting 3A (step S253), and obtains the angle β of the camera mounting 3A with respect to the vertical direction L2 (step S254). After this, thePC 2 obtains the true inclination angle Δ (image capturing angle) of thecamera 3C by subtracting the inclination angle β of the camera mounting 3A from the inclination angle α of thecamera 3C (step S255). - Therefore, according to the present embodiment, the captured image projection apparatus can adjust the captured image by using the image capturing angle (Δ) obtained in the way described above. That is, trapezoidal correction in the vertical direction and in the horizontal direction is performed on the captured image of the written material A placed on the camera mounting 3A by using the image capturing angle (Δ). Due to this, the same image as that obtained by image-capturing the written material A vertically can be obtained.
- Furthermore, the captured image projection apparatus can automatically obtain the image capturing angle (Δ) accurately in real time, without the preparatory steps as in the third and fourth embodiments, which is convenient for the user.
- In the embodiments described above, there has been explained a captured image projection apparatus in which the projection section of the present invention is realized by the
projector 1 and the correction section of the present invention is realized by thePC 2. In this explanation, various image processes relating to correction of a captured image are performed by thePC 2. However, the present invention is not limited to this, but the above-described various image processes may be performed by theprojector 1 and the calligraphy/drawing camera 3. In this case, thePC 2 may not be used. Further, theprojector 1 and the calligraphy/drawing camera 3 may be integral. - Each of the above-described embodiments may not be realized separately, but may be realized in combination.
- As explained above, according to the present invention, since the captured image is corrected based on image information obtained from the captured image, not only correction of trapezoidal deformation but also various corrections can be performed on the captured image. Due to this, it is possible to deal with various corrections necessary for adjustment of the captured image before projection.
- Further, according to the present invention, the inclination of the captured image can be corrected based on a proper image capturing angle directly obtained from the captured image. Due to this, it becomes possible to accurately perform correction on the captured image before projection.
- Still further, according to the present invention, a proper image capturing angle can be obtained even in a case where the captured image is obtained in a state where the written material is not placed on the camera mounting and the camera mounting is inclined. Due to this, correction before projection, of the captured image obtained by the camera supported by the camera mounting can be accurately performed.
- Various embodiments and changes may be made thereunto without departing from the broad spirit and scope of the invention. The above-described embodiments are intended to illustrate the present invention, not to limit the scope of the present invention. The scope of the present invention is shown by the attached claims rather than the embodiments. Various modifications made within the meaning of an equivalent of the claims of the invention and within the claims are to be regarded to be in the scope of the present invention.
- This application is based on Japanese Patent Application No. 2003-157384 filed on Jun. 2, 2003 and including specification, claims, drawings and summary. The disclosure of the above Japanese Patent Application is incorporated herein by reference in its entirety.
Claims (12)
1. An image capturing apparatus comprising:
a document table adapted to have a written material placed thereon;
an image capturing section which captures an image of the written material;
an obtaining section which obtains image information from the captured image, the image information including four lines located closest to four circumferential sides, respectively, of the captured image; and
a correction section which corrects distortion of an image region corresponding to the written material within the captured image based on the image information, the correction of the distortion comprising deforming the image region into a rectangular shape.
2. The image capturing apparatus according to claim 1 , wherein the image information obtained by the obtaining section includes intersections of the four lines.
3. The image capturing apparatus according to claim 1 , wherein the correction of the distortion by the correction section comprises altering an aspect ratio of the image region based on the image information.
4. The image capturing apparatus according to claim 1 , further comprising a projection section which projects an image corrected by the correction section.
5. The image capturing apparatus according to claim 1 , wherein the correction section comprises a determination section which determines an orientation of the written material of which the image is captured, based on the image information.
6. The image capturing apparatus according to claim 5 , wherein the image information includes character recognition information; and
wherein the determination section determines the orientation of the written material by rotating at least the image region corresponding to the written material by an angle a plurality of times, and by identifying an angle at which it is possible to recognize characters in the image region corresponding to the written material.
7. The image capturing apparatus according to claim 1 , wherein the correction section corrects an inclination of the captured image.
8. The image capturing apparatus according to claim 7 , wherein the image information includes marginal distribution information of the image region corresponding to the written material;
wherein the correction section comprises an inclination angle obtaining section which obtains an inclination angle of the captured image based on a plurality of marginal distributions obtained along different directions in the image region corresponding to the written material; and
wherein the correction section corrects the inclination of the captured image based on the inclination angle obtained by the inclination angle obtaining section.
9. The image capturing apparatus according to claim 1 , further comprising a cutting-out section which cuts out the image region corresponding to the written material after the captured image is corrected by said correction section.
10. The image capturing apparatus according to claim 1 , further comprising:
a designation reception section which receives, from a user, designation of a region to be cut out when from the captured image after the captured image is corrected by the correction section; and
a cutting-out section which cuts out the designated region from the captured image after being the captured image is corrected.
11. A method for correcting a captured image, for a captured image projection apparatus which projects a captured image, which is obtained by capturing an image of a written material, on a screen, the method comprising:
selecting four lines located closest to four circumferential sides, respectively, of the captured image; and
correcting the captured image based on the selected plurality of lines.
12. A computer-readable storage medium having a computer-readable program stored thereon that is executable a computer that is included in a captured image projection apparatus which projects a captured image, which is obtained by capturing an image of a written material, on a screen, wherein the program is executable by the computer to control the computer to function as:
a selection section which selects four lines located closest to four circumferential sides, respectively, of the captured image; and
a correction section which corrects the captured image based on the selected plurality of lines.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/174,508 US20080285878A1 (en) | 2003-06-02 | 2008-07-16 | Image capturing apparatus and method for correcting captured image |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2003-157384 | 2003-06-02 | ||
JP2003157384A JP4508553B2 (en) | 2003-06-02 | 2003-06-02 | Captured image projection device and captured image correction method |
US10/858,708 US7412113B2 (en) | 2003-06-02 | 2004-06-01 | Captured image projection apparatus and captured image correction method |
US12/174,508 US20080285878A1 (en) | 2003-06-02 | 2008-07-16 | Image capturing apparatus and method for correcting captured image |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/858,708 Division US7412113B2 (en) | 2003-06-02 | 2004-06-01 | Captured image projection apparatus and captured image correction method |
Publications (1)
Publication Number | Publication Date |
---|---|
US20080285878A1 true US20080285878A1 (en) | 2008-11-20 |
Family
ID=33487397
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/858,708 Active 2026-01-27 US7412113B2 (en) | 2003-06-02 | 2004-06-01 | Captured image projection apparatus and captured image correction method |
US12/174,508 Abandoned US20080285878A1 (en) | 2003-06-02 | 2008-07-16 | Image capturing apparatus and method for correcting captured image |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US10/858,708 Active 2026-01-27 US7412113B2 (en) | 2003-06-02 | 2004-06-01 | Captured image projection apparatus and captured image correction method |
Country Status (7)
Country | Link |
---|---|
US (2) | US7412113B2 (en) |
EP (1) | EP1629665A1 (en) |
JP (1) | JP4508553B2 (en) |
KR (1) | KR100792106B1 (en) |
CN (1) | CN100414958C (en) |
TW (1) | TWI248761B (en) |
WO (1) | WO2004107735A1 (en) |
Cited By (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060033884A1 (en) * | 2002-12-20 | 2006-02-16 | Casio Computer Co., Ltd. | Projection device projection system, and image obtainment method |
US20110210945A1 (en) * | 2010-02-25 | 2011-09-01 | Nikon Corporation | Projector |
US20120293692A1 (en) * | 2011-05-20 | 2012-11-22 | Kenji Namie | Apparatus, method, and system of image processing, and recording medium storing image processing control program |
US9317893B2 (en) | 2013-03-26 | 2016-04-19 | Sharp Laboratories Of America, Inc. | Methods and systems for correcting a document image |
US20160314563A1 (en) * | 2013-12-25 | 2016-10-27 | Grg Banking Equipment Co., Ltd. | Method for correcting fragmentary or deformed quadrangular image |
CN107113412A (en) * | 2014-07-31 | 2017-08-29 | 三星电子株式会社 | Display device, display calibration equipment, display correction system and display bearing calibration |
Families Citing this family (52)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2005006228A (en) * | 2003-06-13 | 2005-01-06 | Casio Comput Co Ltd | Projector |
JP4155890B2 (en) * | 2003-07-15 | 2008-09-24 | カシオ計算機株式会社 | Projector, projector tilt angle acquisition method, and projection image correction method |
JP3969363B2 (en) * | 2003-07-30 | 2007-09-05 | カシオ計算機株式会社 | Projector and projection image correction method for projector |
JP4631261B2 (en) * | 2003-09-29 | 2011-02-16 | カシオ計算機株式会社 | Captured image projection apparatus, image processing method thereof, and program |
JP4561397B2 (en) * | 2005-02-23 | 2010-10-13 | カシオ計算機株式会社 | Image acquisition method and camera device |
JP4561399B2 (en) * | 2005-02-24 | 2010-10-13 | カシオ計算機株式会社 | Image recording method and camera apparatus |
JP2006254271A (en) * | 2005-03-14 | 2006-09-21 | Times Corporation | Magnifier reading aid |
JP4535381B2 (en) * | 2005-06-22 | 2010-09-01 | 株式会社エルモ社 | Document presentation device |
US8625854B2 (en) * | 2005-09-09 | 2014-01-07 | Industrial Research Limited | 3D scene scanner and a position and orientation system |
JP2007078821A (en) * | 2005-09-12 | 2007-03-29 | Casio Comput Co Ltd | Projector, projecting method and program |
JP4588000B2 (en) * | 2006-09-04 | 2010-11-24 | 三星デジタルイメージング株式会社 | Imaging apparatus and imaging method |
JP4854539B2 (en) * | 2007-02-21 | 2012-01-18 | キヤノン株式会社 | Image processing apparatus, control method thereof, and program |
US20080226281A1 (en) * | 2007-03-13 | 2008-09-18 | Real D | Business system for three-dimensional snapshots |
KR100768776B1 (en) * | 2007-04-20 | 2007-10-22 | 유니챌(주) | System for inputting word-image into information device and method for inputting word-image |
US8007111B2 (en) * | 2007-10-22 | 2011-08-30 | Seiko Epson Corporation | Projector having image pickup unit, distance measuring unit, image signal processing unit, and projection unit and projection display method |
JP5217537B2 (en) * | 2008-03-18 | 2013-06-19 | セイコーエプソン株式会社 | Projector, electronic device, and projector control method |
CN101271357B (en) * | 2008-05-12 | 2011-02-16 | 北京中星微电子有限公司 | Content recording method and device of writing board |
JP4583478B2 (en) | 2008-06-11 | 2010-11-17 | ルネサスエレクトロニクス株式会社 | Method for overlaying display of design image and photographed image, display device, and display program |
JP5413002B2 (en) * | 2008-09-08 | 2014-02-12 | ソニー株式会社 | Imaging apparatus and method, and program |
JP5240093B2 (en) * | 2009-06-25 | 2013-07-17 | 株式会社リコー | ID card shooting system, ID card shooting method and program |
TWI439788B (en) * | 2010-01-04 | 2014-06-01 | Ind Tech Res Inst | System and method for projection correction |
US8212945B2 (en) | 2010-04-01 | 2012-07-03 | Seiko Epson Corporation | Method and apparatus for calibrating a projector for image warping |
WO2011152166A1 (en) * | 2010-05-31 | 2011-12-08 | 株式会社Pfu | Overhead scanner apparatus, image processing method, and program |
JP4932930B2 (en) * | 2010-07-02 | 2012-05-16 | ルネサスエレクトロニクス株式会社 | Method for overlaying display of design image and photographed image, display device, and display program |
JP5839785B2 (en) * | 2010-07-06 | 2016-01-06 | キヤノン株式会社 | Projection system, projection apparatus, and imaging apparatus |
CN102156972B (en) * | 2011-04-19 | 2012-07-25 | 清华大学 | Image tilting correcting method and system |
CN102831578B (en) | 2011-06-15 | 2014-12-17 | 富士通株式会社 | Image processing method and image processing device |
CN103593643B (en) * | 2012-08-16 | 2019-02-12 | 百度在线网络技术(北京)有限公司 | A kind of method and system of image recognition |
CN102855482A (en) * | 2012-08-16 | 2013-01-02 | 东莞宇龙通信科技有限公司 | Method and device for processing picture |
JP6154109B2 (en) * | 2012-09-03 | 2017-06-28 | プラス株式会社 | Whiteboard and image correction method |
CN103220482B (en) * | 2013-03-12 | 2016-07-27 | 广东威创视讯科技股份有限公司 | A kind of full degree of freedom adjusts the method for projected picture position and size |
CN103679901B (en) * | 2013-12-30 | 2017-01-04 | 威海北洋电气集团股份有限公司 | Improve method and visitor's registration integral machine of OCR certificate recognition efficiency |
DE102015000377A1 (en) * | 2014-02-07 | 2015-08-13 | Adobe Systems, Inc. | Providing a drawing aid using feature detection and semantic tagging |
CN103957370B (en) * | 2014-03-28 | 2017-06-06 | 苏州佳世达光电有限公司 | Optical projection system and its trapezoidal distortion correction method with trapezoid correction function |
CN105022983A (en) * | 2014-04-30 | 2015-11-04 | 聚晶半导体股份有限公司 | Image recognition system applying multi-lens assistance and method thereof |
US20160110599A1 (en) * | 2014-10-20 | 2016-04-21 | Lexmark International Technology, SA | Document Classification with Prominent Objects |
US10311318B2 (en) * | 2015-07-17 | 2019-06-04 | Rakuten, Inc. | Reference line setting device, reference line setting method and reference line setting program |
CN105141866B (en) * | 2015-09-06 | 2018-10-16 | 青岛海信电器股份有限公司 | A kind of projected picture correcting method and device |
WO2018139156A1 (en) * | 2017-01-26 | 2018-08-02 | ソニー・オリンパスメディカルソリューションズ株式会社 | Medical observation device and control method |
US20210168279A1 (en) * | 2017-04-06 | 2021-06-03 | Huawei Technologies Co., Ltd. | Document image correction method and apparatus |
CN107396076A (en) * | 2017-08-21 | 2017-11-24 | 深圳大学 | A kind of portable translucent correction system, equipment and its control method |
WO2019100407A1 (en) * | 2017-11-27 | 2019-05-31 | 华为技术有限公司 | Positioning of terminal screen based on transformation relation of coordinates of marking graphic points in pattern |
US10477034B2 (en) * | 2018-04-09 | 2019-11-12 | Xerox Corporation | Digital image-paper registration error correction through image shear |
CN109271981A (en) * | 2018-09-05 | 2019-01-25 | 广东小天才科技有限公司 | A kind of image processing method, device and terminal device |
CN109005394B (en) * | 2018-09-19 | 2019-11-29 | 青岛海信激光显示股份有限公司 | A kind of bearing calibration of projected image and projector |
CN110276427A (en) * | 2019-07-05 | 2019-09-24 | 华东师范大学 | A kind of calligraphy teaching broadcast relay system and relaying method based on two dimensional code positioning |
CN110460769B (en) * | 2019-07-05 | 2021-08-17 | 浙江大华技术股份有限公司 | Image correction method, image correction device, computer equipment and storage medium |
CN112435172A (en) * | 2019-08-26 | 2021-03-02 | 北京三好互动教育科技有限公司 | Image correction method and device, storage medium and electronic equipment |
CN112437250A (en) * | 2019-08-26 | 2021-03-02 | 北京三好互动教育科技有限公司 | Teaching equipment that slope was shot |
CN113206987B (en) * | 2021-05-06 | 2023-01-24 | 北京有竹居网络技术有限公司 | Method, device and terminal for projection display and non-transitory storage medium |
CN113340266A (en) * | 2021-06-02 | 2021-09-03 | 江苏豪杰测绘科技有限公司 | Indoor space surveying and mapping system and method |
CN114222188A (en) * | 2021-12-28 | 2022-03-22 | 深圳小湃科技有限公司 | Full-screen display method, device and equipment based on rotary screen and storage medium |
Citations (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4964066A (en) * | 1986-09-05 | 1990-10-16 | Sharp Kabushiki Kaisha | Device for editing and displaying data recorded on specialized sheets or records |
US4963986A (en) * | 1987-08-12 | 1990-10-16 | Canon Kabushiki Kaisha | Image input apparatus for resetting the amount of exposure from a correction value for a predetermined standard value |
US5077811A (en) * | 1990-10-10 | 1991-12-31 | Fuji Xerox Co., Ltd. | Character and picture image data processing system |
US5355420A (en) * | 1989-12-08 | 1994-10-11 | Xerox Corporation | Method and apparatus for identification of document skew |
US5719969A (en) * | 1992-06-30 | 1998-02-17 | Canon Kabushiki Kaisha | Data input apparatus and data processing apparatus |
US5940420A (en) * | 1996-10-08 | 1999-08-17 | Trimedyne, Inc. | Split-flow laser cooling cavity |
US5940128A (en) * | 1992-06-10 | 1999-08-17 | Canon Kabushiki Kaisha | Object imaging apparatus with object slant correction facility |
US20010020977A1 (en) * | 2000-01-20 | 2001-09-13 | Ricoh Company, Limited | Digital camera, a method of shooting and transferring text |
US6304298B1 (en) * | 1995-09-08 | 2001-10-16 | Orad Hi Tec Systems Limited | Method and apparatus for determining the position of a TV camera for use in a virtual studio |
US20010046309A1 (en) * | 2000-03-30 | 2001-11-29 | Toshio Kamei | Method and system for tracking a fast moving object |
US6389155B2 (en) * | 1997-06-20 | 2002-05-14 | Sharp Kabushiki Kaisha | Image processing apparatus |
US6430320B1 (en) * | 1998-04-09 | 2002-08-06 | Hewlett-Packard Company | Image processing system with automatic image cropping and skew correction |
US20030025788A1 (en) * | 2001-08-06 | 2003-02-06 | Mitsubishi Electric Research Laboratories, Inc. | Hand-held 3D vision system |
US20030117425A1 (en) * | 2001-11-06 | 2003-06-26 | O'leary Peter | Electronic simulation of interaction with printed matter |
US20040012679A1 (en) * | 2002-07-17 | 2004-01-22 | Jian Fan | Systems and methods for processing a digital captured image |
US20040101211A1 (en) * | 2002-11-26 | 2004-05-27 | Eastman Kodak Company | Document skew accommodation |
US20040165232A1 (en) * | 1998-10-28 | 2004-08-26 | Fujitsu Limited | Image reader |
US7001024B2 (en) * | 2001-02-14 | 2006-02-21 | Ricoh Company, Ltd. | Image input apparatus using projected light |
US20060088226A1 (en) * | 2000-04-10 | 2006-04-27 | Fuji Xerox Co., Ltd. | Image reader |
US7065261B1 (en) * | 1999-03-23 | 2006-06-20 | Minolta Co., Ltd. | Image processing device and image processing method for correction of image distortion |
US7171056B2 (en) * | 2003-02-22 | 2007-01-30 | Microsoft Corp. | System and method for converting whiteboard content into an electronic document |
Family Cites Families (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3497660B2 (en) * | 1996-04-26 | 2004-02-16 | 株式会社東芝 | LCD projector |
GB9711022D0 (en) | 1997-05-28 | 1997-07-23 | Rank Xerox Ltd | Text/image selection from document images |
KR100256973B1 (en) * | 1997-08-01 | 2000-05-15 | 구자홍 | Program downloading device of flash memory |
EP1100257B1 (en) | 1998-06-23 | 2011-12-07 | Sharp Kabushiki Kaisha | Image processor, image processing method, and medium on which image processing program is recorded |
JP2002354331A (en) | 2001-05-25 | 2002-12-06 | Canon Inc | Imaging equipment and electronic component |
US6873732B2 (en) | 2001-07-09 | 2005-03-29 | Xerox Corporation | Method and apparatus for resolving perspective distortion in a document image and for calculating line sums in images |
-
2003
- 2003-06-02 JP JP2003157384A patent/JP4508553B2/en not_active Expired - Fee Related
-
2004
- 2004-06-01 US US10/858,708 patent/US7412113B2/en active Active
- 2004-06-02 TW TW093115728A patent/TWI248761B/en not_active IP Right Cessation
- 2004-06-02 CN CNB2004800153603A patent/CN100414958C/en not_active Expired - Fee Related
- 2004-06-02 EP EP04745695A patent/EP1629665A1/en not_active Ceased
- 2004-06-02 KR KR1020057022966A patent/KR100792106B1/en active IP Right Grant
- 2004-06-02 WO PCT/JP2004/007999 patent/WO2004107735A1/en active Application Filing
-
2008
- 2008-07-16 US US12/174,508 patent/US20080285878A1/en not_active Abandoned
Patent Citations (21)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4964066A (en) * | 1986-09-05 | 1990-10-16 | Sharp Kabushiki Kaisha | Device for editing and displaying data recorded on specialized sheets or records |
US4963986A (en) * | 1987-08-12 | 1990-10-16 | Canon Kabushiki Kaisha | Image input apparatus for resetting the amount of exposure from a correction value for a predetermined standard value |
US5355420A (en) * | 1989-12-08 | 1994-10-11 | Xerox Corporation | Method and apparatus for identification of document skew |
US5077811A (en) * | 1990-10-10 | 1991-12-31 | Fuji Xerox Co., Ltd. | Character and picture image data processing system |
US5940128A (en) * | 1992-06-10 | 1999-08-17 | Canon Kabushiki Kaisha | Object imaging apparatus with object slant correction facility |
US5719969A (en) * | 1992-06-30 | 1998-02-17 | Canon Kabushiki Kaisha | Data input apparatus and data processing apparatus |
US6304298B1 (en) * | 1995-09-08 | 2001-10-16 | Orad Hi Tec Systems Limited | Method and apparatus for determining the position of a TV camera for use in a virtual studio |
US5940420A (en) * | 1996-10-08 | 1999-08-17 | Trimedyne, Inc. | Split-flow laser cooling cavity |
US6389155B2 (en) * | 1997-06-20 | 2002-05-14 | Sharp Kabushiki Kaisha | Image processing apparatus |
US6430320B1 (en) * | 1998-04-09 | 2002-08-06 | Hewlett-Packard Company | Image processing system with automatic image cropping and skew correction |
US20040165232A1 (en) * | 1998-10-28 | 2004-08-26 | Fujitsu Limited | Image reader |
US7065261B1 (en) * | 1999-03-23 | 2006-06-20 | Minolta Co., Ltd. | Image processing device and image processing method for correction of image distortion |
US20010020977A1 (en) * | 2000-01-20 | 2001-09-13 | Ricoh Company, Limited | Digital camera, a method of shooting and transferring text |
US20010046309A1 (en) * | 2000-03-30 | 2001-11-29 | Toshio Kamei | Method and system for tracking a fast moving object |
US20060088226A1 (en) * | 2000-04-10 | 2006-04-27 | Fuji Xerox Co., Ltd. | Image reader |
US7001024B2 (en) * | 2001-02-14 | 2006-02-21 | Ricoh Company, Ltd. | Image input apparatus using projected light |
US20030025788A1 (en) * | 2001-08-06 | 2003-02-06 | Mitsubishi Electric Research Laboratories, Inc. | Hand-held 3D vision system |
US20030117425A1 (en) * | 2001-11-06 | 2003-06-26 | O'leary Peter | Electronic simulation of interaction with printed matter |
US20040012679A1 (en) * | 2002-07-17 | 2004-01-22 | Jian Fan | Systems and methods for processing a digital captured image |
US20040101211A1 (en) * | 2002-11-26 | 2004-05-27 | Eastman Kodak Company | Document skew accommodation |
US7171056B2 (en) * | 2003-02-22 | 2007-01-30 | Microsoft Corp. | System and method for converting whiteboard content into an electronic document |
Non-Patent Citations (1)
Title |
---|
http://cgm.cs.mcgill.ca/~orm/maer.html, "The minimum area enclosing rectangle for a convex polygon", December 17th, 1998 * |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20060033884A1 (en) * | 2002-12-20 | 2006-02-16 | Casio Computer Co., Ltd. | Projection device projection system, and image obtainment method |
US7524066B2 (en) * | 2002-12-20 | 2009-04-28 | Casio Computer Co., Ltd. | Projection device projection system, and image obtainment method |
US20110210945A1 (en) * | 2010-02-25 | 2011-09-01 | Nikon Corporation | Projector |
US20120293692A1 (en) * | 2011-05-20 | 2012-11-22 | Kenji Namie | Apparatus, method, and system of image processing, and recording medium storing image processing control program |
US8605174B2 (en) * | 2011-05-20 | 2013-12-10 | Ricoh Company, Ltd. | Apparatus, method, and system of image processing, and recording medium storing image processing control program |
US9317893B2 (en) | 2013-03-26 | 2016-04-19 | Sharp Laboratories Of America, Inc. | Methods and systems for correcting a document image |
US20160314563A1 (en) * | 2013-12-25 | 2016-10-27 | Grg Banking Equipment Co., Ltd. | Method for correcting fragmentary or deformed quadrangular image |
US9773299B2 (en) * | 2013-12-25 | 2017-09-26 | Grg Banking Equipment Co., Ltd. | Method for correcting fragmentary or deformed quadrangular image |
CN107113412A (en) * | 2014-07-31 | 2017-08-29 | 三星电子株式会社 | Display device, display calibration equipment, display correction system and display bearing calibration |
Also Published As
Publication number | Publication date |
---|---|
TW200503531A (en) | 2005-01-16 |
KR100792106B1 (en) | 2008-01-04 |
WO2004107735A1 (en) | 2004-12-09 |
JP4508553B2 (en) | 2010-07-21 |
CN100414958C (en) | 2008-08-27 |
CN1799252A (en) | 2006-07-05 |
US7412113B2 (en) | 2008-08-12 |
TWI248761B (en) | 2006-02-01 |
KR20060015327A (en) | 2006-02-16 |
EP1629665A1 (en) | 2006-03-01 |
JP2004363736A (en) | 2004-12-24 |
US20050163396A1 (en) | 2005-07-28 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7412113B2 (en) | Captured image projection apparatus and captured image correction method | |
JP4363151B2 (en) | Imaging apparatus, image processing method thereof, and program | |
JP4341629B2 (en) | Imaging apparatus, image processing method, and program | |
US8009929B2 (en) | Image-capturing apparatus, method, and program which correct distortion of a captured image based on an angle formed between a direction of an image-capturing apparatus and a gravitational direction | |
WO2013108727A1 (en) | Image-capturing device, captured image processing system, program, and recording medium | |
WO2003005702A1 (en) | Tilt correction of electronic images | |
US7068855B2 (en) | System and method for manipulating a skewed digital image | |
JPH08181828A (en) | Picture input device | |
US7633654B2 (en) | Camera, computer and projector for use in sensed image projection system for projecting the image of a projection target onto a screen, and image processing method | |
JP2016167311A (en) | Image processing apparatus and image processing method | |
US8582920B2 (en) | Presentation device | |
JP2006211046A (en) | Photographing apparatus, image processing method for photographing apparatus, and program | |
US20100149557A1 (en) | Image processing apparatus and image processing method | |
JP4363154B2 (en) | Imaging apparatus, image processing method thereof, and program | |
JP3582988B2 (en) | Non-contact image reader | |
JP2005303941A (en) | Correction reference designation device and correction reference designation method | |
JP4363153B2 (en) | Imaging apparatus, image processing method thereof, and program | |
JP4929572B2 (en) | Image projection apparatus, image projection method, and image position correction program | |
JP2007148612A (en) | Photographing device, image processing method and image processing program | |
JP2004318896A (en) | Non-contact image reading apparatus | |
JP2021140352A (en) | Card cut-out device and program | |
JPH02170280A (en) | Document inclination corrector | |
JP2005260691A (en) | Photographing apparatus, image acquisition method and program therefor | |
JPH11238123A (en) | Image correction processing method, image correction processor, digital camera and information recording medium | |
JPH04362859A (en) | Picture input device |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |