WO2004100066A2 - Structure-preserving clone brush - Google Patents

Structure-preserving clone brush Download PDF

Info

Publication number
WO2004100066A2
WO2004100066A2 PCT/US2004/013123 US2004013123W WO2004100066A2 WO 2004100066 A2 WO2004100066 A2 WO 2004100066A2 US 2004013123 W US2004013123 W US 2004013123W WO 2004100066 A2 WO2004100066 A2 WO 2004100066A2
Authority
WO
WIPO (PCT)
Prior art keywords
region
image
destination
color
relative
Prior art date
Application number
PCT/US2004/013123
Other languages
French (fr)
Other versions
WO2004100066A3 (en
Inventor
Byong Mok Oh
Fredo Durand
Original Assignee
Mok3, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=33313569&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=WO2004100066(A2) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Application filed by Mok3, Inc. filed Critical Mok3, Inc.
Publication of WO2004100066A2 publication Critical patent/WO2004100066A2/en
Publication of WO2004100066A3 publication Critical patent/WO2004100066A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/60—Editing figures and text; Combining figures or text

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention provides tools and techniques for clone brushing pixels in an image while accounting for inconsistencies in apparent depth and orientation within the image. The techniques do not require any depth information to be present in the image, and the data structure of the image is preserved. The techniques allow for color compensation between source and destination regions. A snapping technique is also provided to facilitate increased accuracy in selecting source and destination positions.

Description

The corresponding source region is determined in a source obtaining step 1214. The corresponding source region is then copied to the destination region in a copying step 1216. [0069] FIG. 13 shows an exemplary application of a structure-preserving clone brush, in accordance with an embodiment of the invention. The original image 1300 has been extended into a new image 1302 by copying portions of the image 1300 while maintaining consistency with perspective. The world plane has been defined in this example by the sets of parallel lines p' 1304 and p" 1306.
[0070] In some embodiments, the functionality of the systems and methods described above can be implemented as software on a general purpose computer. In such an embodiment, the program can be written in any one of a number of high-level languages, such as FORTRAN, PASCAL, C, C++, LISP, JAVA, or BASIC. Further, the program can be written in a script, macro, or functionality embedded in commercially available software, such as VISUAL BASIC. The program may also be implemented as a plug-in for commercially or otherwise available image editing software, such as ADOBE PHOTOSHOP. Additionally, the software could be implemented in an assembly language directed to a microprocessor resident on a computer. For example, the software could be implemented in Intel 80x86 assembly language if it were configured to run on an IBM PC or PC clone. The software can be embedded on an article of manufacture including, but not limited to, a "computer-readable medium" such as a floppy disk, a hard disk, an optical disk, a magnetic tape, a PROM, an EPROM, or CD-ROM. [0071] While the invention has been particularly shown and described with reference to specific embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. The scope of the invention is thus indicated by the appended claims and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced.

Claims

19 -CLAIMS
1. A clone-brushing method of painting in an image, the method comprising: a) specifying a first world plane in the image; b) providing a source position and a destination position in the image; c) identifying a destination region in the image relative to the destination position; d) determining a source region in the image relative to the first world plane and corresponding to the destination region; e) transforming image information of the source region relative to the first world plane to image information of the destination region; and f) copying the transformed image information to the destination region.
2. The method of claim 1, wherein the source region in the image is determined via a homography defined by the first world plane.
3. The method of claim 1, wherein step a) comprises specifying two sets of parallel lines.
4. The method of claim 1, wherein step e) further comprises a bilinear interpolation of image information in the source region relative to the first world plane.
5. The method of claim 1 further comprising: providing a first color sample region for the source region; providing a second sample color region for the destination region; and computing a color ratio between the first color sample region and the second color sample region, wherein step e) further comprises applying the color ratio to the image information of the source region.
6. The method of claim 5, wherein the color ratio is computed using Gaussian weighted averages of the first and second sample color regions.
7. The method of claim 5, wherein the first color sample region is provided with respect to the first world plane.
- 20 - 8. The method of claim 1 , further comprising specifying a second world plane and a relative scale factor in the image, wherein: step d) comprises determining a source region in the image relative to the first world plane and corresponding to the destination region relative to the second world plane and the relative scale factor; and step e) comprises transforming the image information of the source region relative to the first world plane to image information of the destination region relative to the second world plane and the relative scale factor.
9. The method of claim 8, wherein specifying the second world plane comprises specifying two sets of parallel lines.
10. The method of claim 8, wherein specifying the relative scale factor comprises specifying a line segment of unit length relative the first world plane and specifying a line segment of unit length relative to the second world plane.
11. A clone-brushing method of painting in an image, the method comprising: a) providing a first color sample region; b) providing a second color sample region; c) computing a color ratio between the first color sample region and the second color sample region; d) providing a source position in the image; e) providing a destination position in the image; f) identifying a destination region in the image relative to the destination position; g) determining a source region in the image corresponding to the destination region; h) applying the color ratio to image information of the source region and transforming the image information of the source region to image information of the destination region; and i) copying the transformed image information to the destination region.
12. The method of claim 11 , wherein the color ratio is computed using Gaussian weighted averages of the first and second sample color regions.
13. A clone-brushing method of painting in an image, the method comprising: - 21 - a) providing a source position in the image; b) providing an initial destination position in the image; c) determining a snapped destination position; d) identifying a destination region in the image relative to the snapped destination position; e) determining a source region in the image corresponding to the destination region; f) transforming image information of the source region to image information of the destination region; and g) copying the transformed image information to the destination region.
14. The method of claim 13, wherein step c) comprises searching a collection of candidate destination positions.
15. The method of claim 14, wherein step c) further comprises applying a quality metric to the source position, applying the quality metric to the candidate destination positions, and determining a snapped destination position from the collection of candidate destination positions whose quality is similar to the quality of the source position.
16. The method of claim 15, wherein the quality metric is a Gaussian- weighted color average for a region surrounding the position.
17. The method of claim 15, wherein the quality metric compensates for regional color variation by applying a color ratio.
18.- A system for clone-brushing in an image, the system comprising: a computer comprising a processor, memory, and a display, the memory containing instractions that, when executed by the processor, cause the computer to: receive an input image; interact with a user to specify a first world plane in the image; interact with a user to provide a source position and a destination position in the image; interact with a user to identifying a destination region in the image relative to the destination position; - 22 - determine a source region in the image relative to the first world plane and corresponding to the destination region; transform image information of the source region relative to the first world plane to image information of the destination region; and copy the transformed image information to the destination region.
19. The system of claim 18, wherein the instructions, when executed by the processor, further cause the computer to interact with the user to specify a world plane by drawing two sets of parallel lines.
20. The system of claim 18, wherein the instractions, when executed by the processor, further cause the computer to interact with the user to: provide a first color sample region for the source region; provide a second sample color region for the destination region; and compute a color ratio between the first color sample region and the second color sample region, wherein step the color ratio is applied to the image information of the source region.
PCT/US2004/013123 2003-04-30 2004-04-28 Structure-preserving clone brush WO2004100066A2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US46662803P 2003-04-30 2003-04-30
US60/466,628 2003-04-30
US10/601,842 2003-06-23
US10/601,842 US7327374B2 (en) 2003-04-30 2003-06-23 Structure-preserving clone brush

Publications (2)

Publication Number Publication Date
WO2004100066A2 true WO2004100066A2 (en) 2004-11-18
WO2004100066A3 WO2004100066A3 (en) 2009-04-09

Family

ID=33313569

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2004/013123 WO2004100066A2 (en) 2003-04-30 2004-04-28 Structure-preserving clone brush

Country Status (2)

Country Link
US (5) US7327374B2 (en)
WO (1) WO2004100066A2 (en)

Families Citing this family (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040196282A1 (en) * 2003-02-14 2004-10-07 Oh Byong Mok Modeling and editing image panoramas
US7327374B2 (en) * 2003-04-30 2008-02-05 Byong Mok Oh Structure-preserving clone brush
US7425958B2 (en) * 2004-10-25 2008-09-16 Adobe Systems Incorporated Perspective editing tools for 2-D images
US7525555B2 (en) * 2004-10-26 2009-04-28 Adobe Systems Incorporated Facilitating image-editing operations across multiple perspective planes
JP2008520052A (en) 2004-11-12 2008-06-12 モク３， インコーポレイテッド Method for transition between scenes
US9769354B2 (en) 2005-03-24 2017-09-19 Kofax, Inc. Systems and methods of processing scanned data
US8645870B2 (en) 2005-03-31 2014-02-04 Adobe Systems Incorporated Preview cursor for image editing
US7636089B2 (en) * 2005-09-09 2009-12-22 Microsoft Corporation Photo mantel view and animation
WO2008060933A2 (en) 2006-11-13 2008-05-22 Everyscape, Inc. Method for scripting inter-scene transitions
US9053530B2 (en) 2006-11-27 2015-06-09 Google Inc. Method for sliced inpainting
DE102008048383A1 (en) * 2008-09-22 2010-03-25 Baumer Inspection Gmbh Automatic patching of wood panels with aesthetic reconstruction
US9576272B2 (en) 2009-02-10 2017-02-21 Kofax, Inc. Systems, methods and computer program products for determining document validity
US9767354B2 (en) 2009-02-10 2017-09-19 Kofax, Inc. Global geographic information retrieval, validation, and normalization
US9535599B2 (en) * 2009-08-18 2017-01-03 Adobe Systems Incorporated Methods and apparatus for image editing using multitouch gestures
KR101209791B1 (en) * 2009-12-04 2012-12-10 현대자동차주식회사 Metal separator of fuel cell and method for treating surface of the same
US8989515B2 (en) 2012-01-12 2015-03-24 Kofax, Inc. Systems and methods for mobile image capture and processing
US10146795B2 (en) 2012-01-12 2018-12-04 Kofax, Inc. Systems and methods for mobile image capture and processing
US9355312B2 (en) 2013-03-13 2016-05-31 Kofax, Inc. Systems and methods for classifying objects in digital images captured using mobile devices
US9208536B2 (en) 2013-09-27 2015-12-08 Kofax, Inc. Systems and methods for three dimensional geometric reconstruction of captured image data
US9350969B2 (en) 2013-04-19 2016-05-24 Adobe Systems Incorporated Target region filling involving source regions, depth information, or occlusions
US9380286B2 (en) 2013-04-19 2016-06-28 Adobe Systems Incorporated Stereoscopic target region filling
US20140316841A1 (en) 2013-04-23 2014-10-23 Kofax, Inc. Location-based workflows and services
DE202014011407U1 (en) 2013-05-03 2020-04-20 Kofax, Inc. Systems for recognizing and classifying objects in videos captured by mobile devices
US20150097827A1 (en) * 2013-10-09 2015-04-09 Adobe Systems Incorporated Target Region Fill Utilizing Transformations
JP2016538783A (en) 2013-11-15 2016-12-08 コファックス， インコーポレイテッド System and method for generating a composite image of a long document using mobile video data
CN105447895A (en) * 2014-09-22 2016-03-30 酷派软件技术（深圳）有限公司 Hierarchical picture pasting method, device and terminal equipment
US9760788B2 (en) 2014-10-30 2017-09-12 Kofax, Inc. Mobile document detection and orientation based on reference object characteristics
US10127634B2 (en) 2015-04-08 2018-11-13 Google Llc Image editing and repair
US10467465B2 (en) 2015-07-20 2019-11-05 Kofax, Inc. Range and/or polarity-based thresholding for improved data extraction
US10242285B2 (en) 2015-07-20 2019-03-26 Kofax, Inc. Iterative recognition-guided thresholding and data extraction
US9779296B1 (en) 2016-04-01 2017-10-03 Kofax, Inc. Content-based detection and three dimensional geometric reconstruction of objects in image and video data
US10803350B2 (en) 2017-11-30 2020-10-13 Kofax, Inc. Object detection and image cropping using a multi-detector approach
CN109462748B (en) * 2018-12-21 2020-10-09 福州大学 Stereo video color correction algorithm based on homography matrix
US11294556B1 (en) * 2021-01-05 2022-04-05 Adobe Inc. Editing digital images using multi-panel graphical user interfaces
US11573689B1 (en) * 2021-10-21 2023-02-07 Adobe Inc. Trace layer for replicating a source region of a digital image

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4898537A (en) * 1988-12-12 1990-02-06 Pryor Paul L Locating system for maps and other diagrams and displays
US5054008A (en) * 1989-10-16 1991-10-01 Dwight Darling World time device
US5475507A (en) * 1992-10-14 1995-12-12 Fujitsu Limited Color image processing method and apparatus for same, which automatically detects a contour of an object in an image
US5798761A (en) * 1996-01-26 1998-08-25 Silicon Graphics, Inc. Robust mapping of 2D cursor motion onto 3D lines and planes
US5818415A (en) * 1995-06-30 1998-10-06 Nec Corporation Graphic form inputting apparatus
US5835241A (en) * 1996-05-30 1998-11-10 Xerox Corporation Method for determining the profile of a bound document with structured light
US6073056A (en) * 1997-04-08 2000-06-06 Larry J. Winget Method and system for building a data model of a physical part in a data format useful for and reproduction of the part
US20030086627A1 (en) * 2001-10-10 2003-05-08 Berriss William P. Method and apparatus for searching for and retrieving colour images

Family Cites Families (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5131058A (en) * 1990-08-24 1992-07-14 Eastman Kodak Company Method for obtaining output-adjusted color separations
US5347620A (en) * 1991-09-05 1994-09-13 Zimmer Mark A System and method for digital rendering of images and printed articulation
US5469536A (en) * 1992-02-25 1995-11-21 Imageware Software, Inc. Image editing system including masking capability
US6147688A (en) * 1993-06-28 2000-11-14 Athena Design Systems, Inc. Method and apparatus for defining and selectively repeating unit image cells
US5544291A (en) * 1993-11-10 1996-08-06 Adobe Systems, Inc. Resolution-independent method for displaying a three dimensional model in two-dimensional display space
US5511153A (en) * 1994-01-18 1996-04-23 Massachusetts Institute Of Technology Method and apparatus for three-dimensional, textured models from plural video images
WO1996013006A1 (en) * 1994-10-20 1996-05-02 Mark Alan Zimmer Digital mark-making method
US5649173A (en) * 1995-03-06 1997-07-15 Seiko Epson Corporation Hardware architecture for image generation and manipulation
US5710833A (en) * 1995-04-20 1998-01-20 Massachusetts Institute Of Technology Detection, recognition and coding of complex objects using probabilistic eigenspace analysis
US5719599A (en) * 1995-06-07 1998-02-17 Seiko Epson Corporation Method and apparatus for efficient digital modeling and texture mapping
GB9518530D0 (en) * 1995-09-11 1995-11-08 Informatix Inc Image processing
US5706416A (en) * 1995-11-13 1998-01-06 Massachusetts Institute Of Technology Method and apparatus for relating and combining multiple images of the same scene or object(s)
US5828793A (en) * 1996-05-06 1998-10-27 Massachusetts Institute Of Technology Method and apparatus for producing digital images having extended dynamic ranges
US5946425A (en) * 1996-06-03 1999-08-31 Massachusetts Institute Of Technology Method and apparatus for automatic alingment of volumetric images containing common subject matter
US5808623A (en) * 1996-10-07 1998-09-15 Adobe Systems Incorporated System and method for perspective transform in computer using multi-pass algorithm
US5990900A (en) * 1997-12-24 1999-11-23 Be There Now, Inc. Two-dimensional to three-dimensional image converting system
US6134346A (en) * 1998-01-16 2000-10-17 Ultimatte Corp Method for removing from an image the background surrounding a selected object
US6147689A (en) * 1998-04-07 2000-11-14 Adobe Systems, Incorporated Displaying 2D patches with foldover
US6333749B1 (en) * 1998-04-17 2001-12-25 Adobe Systems, Inc. Method and apparatus for image assisted modeling of three-dimensional scenes
US6421049B1 (en) * 1998-05-11 2002-07-16 Adobe Systems, Inc. Parameter selection for approximate solutions to photogrammetric problems in interactive applications
US6198852B1 (en) * 1998-06-01 2001-03-06 Yeda Research And Development Co., Ltd. View synthesis from plural images using a trifocal tensor data structure in a multi-view parallax geometry
WO1999063488A1 (en) * 1998-06-02 1999-12-09 Sony Corporation Image processing device and image processing method
US6084592A (en) * 1998-06-18 2000-07-04 Microsoft Corporation Interactive construction of 3D models from panoramic images
US6268846B1 (en) * 1998-06-22 2001-07-31 Adobe Systems Incorporated 3D graphics based on images and morphing
US6134345A (en) * 1998-08-28 2000-10-17 Ultimatte Corporation Comprehensive method for removing from an image the background surrounding a selected subject
US6434269B1 (en) * 1999-04-26 2002-08-13 Adobe Systems Incorporated Smart erasure brush
US6456297B1 (en) * 2000-05-10 2002-09-24 Adobe Systems Incorporated Multipole brushing
US6628279B1 (en) * 2000-11-22 2003-09-30 @Last Software, Inc. System and method for three-dimensional modeling
US7003150B2 (en) * 2001-11-05 2006-02-21 Koninklijke Philips Electronics N.V. Homography transfer from point matches
US7199793B2 (en) 2002-05-21 2007-04-03 Mok3, Inc. Image-based modeling and photo editing
US20040196282A1 (en) 2003-02-14 2004-10-07 Oh Byong Mok Modeling and editing image panoramas
US7327374B2 (en) 2003-04-30 2008-02-05 Byong Mok Oh Structure-preserving clone brush
JP2008520052A (en) 2004-11-12 2008-06-12 モク３， インコーポレイテッド Method for transition between scenes
WO2008060933A2 (en) 2006-11-13 2008-05-22 Everyscape, Inc. Method for scripting inter-scene transitions

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4898537A (en) * 1988-12-12 1990-02-06 Pryor Paul L Locating system for maps and other diagrams and displays
US5054008A (en) * 1989-10-16 1991-10-01 Dwight Darling World time device
US5475507A (en) * 1992-10-14 1995-12-12 Fujitsu Limited Color image processing method and apparatus for same, which automatically detects a contour of an object in an image
US5818415A (en) * 1995-06-30 1998-10-06 Nec Corporation Graphic form inputting apparatus
US5798761A (en) * 1996-01-26 1998-08-25 Silicon Graphics, Inc. Robust mapping of 2D cursor motion onto 3D lines and planes
US5835241A (en) * 1996-05-30 1998-11-10 Xerox Corporation Method for determining the profile of a bound document with structured light
US6073056A (en) * 1997-04-08 2000-06-06 Larry J. Winget Method and system for building a data model of a physical part in a data format useful for and reproduction of the part
US20030086627A1 (en) * 2001-10-10 2003-05-08 Berriss William P. Method and apparatus for searching for and retrieving colour images

Also Published As

Publication number Publication date
US20040217975A1 (en) 2004-11-04
US7327374B2 (en) 2008-02-05
US8379049B2 (en) 2013-02-19
US8174538B2 (en) 2012-05-08
US20080088641A1 (en) 2008-04-17
US20120236019A1 (en) 2012-09-20
US20100073403A1 (en) 2010-03-25
US20130145298A1 (en) 2013-06-06
WO2004100066A3 (en) 2009-04-09
US7593022B2 (en) 2009-09-22

Similar Documents

Publication Publication Date Title
WO2004100066A2 (en) Structure-preserving clone brush
US8694890B2 (en) Use of color in a site analysis report
Grooteman An adaptive directional importance sampling method for structural reliability
US6628295B2 (en) Modifying a stylistic property of a vector-based path
US10565792B2 (en) Approximating mesh deformations for character rigs
JP4949407B2 (en) Optimization-based visual context management
US11475617B2 (en) Path-constrained drawing with visual properties based on drawing tool
JP2000513844A (en) Sparse array image correlation
Ko et al. A ternary 4-point approximating subdivision scheme
US20160019609A1 (en) Cost computation device, cost computation method, and computer-readable recording medium
CN110889725A (en) Online advertisement CTR estimation method, device, equipment and storage medium
Oosterlee et al. Accurate American option pricing by grid stretching and high order finite differences
CN110782504A (en) Curve determination method, device, computer readable storage medium and equipment
CN111125564B (en) Thermodynamic diagram generation method, thermodynamic diagram generation device, thermodynamic diagram generation computer device and thermodynamic diagram generation storage medium
CN115147474B (en) Method and device for generating point cloud annotation model, electronic equipment and storage medium
CN112132859A (en) Sticker generation method, apparatus, medium, and electronic device
CN110826121A (en) Method and system for automatically positioning house type corridor and entrance
CN104423964A (en) Method and system used for determining visualization credibility
US7412689B1 (en) Method and apparatus for creating a hierarchical model to facilitate identifying components in a programming language file
CN111583376A (en) Method and device for eliminating black edge in illumination map, storage medium and electronic equipment
US8095886B2 (en) Context based navigation
WO2023115312A1 (en) Handwriting generation method and apparatus, computer device, and readable medium
CN116012427A (en) Electronic painting pigment acquisition method, device, equipment and storage medium
CN110619600B (en) Neural network model training method and device, storage medium and electronic equipment
Kimmel An optimal time algorithm for shape from shading

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase in:

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 69(1) EPC. EPO FORM 1205A DATED 13-01-06

122 Ep: pct application non-entry in european phase