WO2003077198A1

WO2003077198A1 - System and method for registering spectacles image

Info

Publication number: WO2003077198A1
Application number: PCT/JP2003/002892
Authority: WO
Inventors: Takehiko Yoshida
Original assignee: Vision Optic Co., Ltd.; Tada, Hisanobu
Priority date: 2002-03-13
Filing date: 2003-03-12
Publication date: 2003-09-18
Also published as: TW200410037A; JP3598296B2; AU2003213324A1; TWI230835B; JP2003271946A

Abstract

A spectacles image registering system which can pick up spectacles image of a predetermined quality irrespective of time, place or operator and rapidly obtain spectacles image to be synthesized with a portrait image at a low cost. This system comprises an image pickup device (10) to pick up frontal image of spectacles, an image pickup device (100) to pick image of the spectacles from above, and a computer (200) which generates image data to be synthesized with the portrait image by image processing of the picked-up frontal image and registers it together with specifications of the spectacles. The image processing comprises spectacles outer periphery detection for detecting the outer periphery of the spectacles from a background part toward a spectacles part, and lens outer periphery detection for detecting the outer periphery of the lens from a lens part to a lens peripheral part by estimating the lens part from the detected spectacles outer periphery, and generates a frame mask image and a lens mask image to be synthesized with the portrait image. Registered data is transmitted to a spectacles sales system (300) via a network.

Description

Description Glasses image registration system and glasses image registration method

The present invention relates to an eyeglass image registration system and an eyeglass image registration method for generating and registering image data that can be combined with a human image on a computer screen by photographing eyeglasses, and more particularly to a sales service for eyeglasses using a computer. The present invention relates to a glasses image registration system and a glasses image registration method for generating and registering an image data to be used in a glasses sales system that performs the image processing. Background art

In the past, eyeglasses were sold by a customer going to an eyeglass store, actually trying on the eyeglasses prepared at the eyeglass store and selecting the desired glasses.

In this case, the power of the lens can be dealt with by measuring the eyesight or selecting the optimum power from the corrective lens, but the color of the lens and whether the frame is suitable for actual fitting It was necessary to check and all kinds of glasses had to be prepared. In particular, today, eyeglass frames are made of metal frames made of metal such as titanium, cell frames made of plastic, nylon rolls with a nylon thread hung on a metal or plastic rim, and holes made in the lens. As eyeglasses with new designs are produced one after another in a wide variety of products, such as screwed two-points, it is difficult to keep all types of glasses in stores.

For this reason, as disclosed in Japanese Patent Application Laid-Open No. 2000-35092, Using a terminal installed at the store, the customer's face image is captured, combined with the pre-stored eyeglass image, and displayed on the computer screen. An eyeglass sales system has been developed that allows designers to select their eyeglasses.

In addition, due to the development of network technology such as the Internet, a virtual store has been set up on the network, and customers have access to the virtual store to sell eyeglasses without manual intervention. A system has also been developed, but in this case as well, it receives face image data from the consumer's computer, generates an image synthesized with the glasses image stored in advance, and sends it to the consumer's computer for screen display. By doing so, a glasses sales system that allows consumers to select the glasses of the desired design is conceivable.

In the above-mentioned eyeglass sales system, the image data of the customer's face may be used by photographing it with, for example, a digital camera and taking it into the consumer's computer.

In such a glasses sales system, in order to combine a glasses image with a person image, it is necessary that the image data of the glasses be registered in a form that can be combined in advance.

Such image data can be generated by using CAD data when CAD is used for designing glasses, but there are many cases where CAD is not used for designing glasses. In order to do this, it is necessary to raise the drawing again, or to photograph the manufactured glasses with a camera, and generate synthesizable image data from them.

However, recreating a drawing would be a reworking process, and it would not be possible to quickly process eyeglasses prototyped at a manufacturing site, such as accepting an order immediately at a store or a virtual store on the Internet. Have difficulty In addition, a method of generating image data that can be synthesized by shooting glasses with a camera is to shoot the glasses with predetermined quality, select an appropriate image from the captured glasses images, and import it to a computer. It is necessary to convert the frame and lens parts of the glasses into the specified data. Conventionally, since photographing and computer processing require specialized knowledge and skill, a professional photographer shoots glasses in a studio and views image data taken by a computer graphics expert. —I was processing it in the evening.

This requires enormous costs for shooting and computer processing, and also requires transport of eyeglasses for shooting and computer processing, which is a major obstacle to reducing the price of glasses and promptly selling them. Therefore, a main object of the present invention is to provide a low-cost and quickly obtainable image data that can be combined with a human image at any time, where, and by any person. It is to provide. Disclosure of the invention

The glasses image registration system according to claim 1, wherein the image capturing unit captures an image of the glasses, and a combining image generating unit configured to generate a combining image for combining a human image from the image captured by the capturing unit. And registration means for registering the composition image generated by the composition image generation means together with the specifications of the glasses.

An image of the glasses is taken by the photographing means, an image for composition which can be combined with the human image is produced by the image generating means for composition, and specifications of the glasses are produced by the registration means Because it is registered with the user, image data that can be combined with a human image can be obtained quickly and at low cost, no matter when, where, and by whom.

The eyeglass image registration system according to claim 2 is the eyeglass image registration system according to claim 1, wherein the photographing means includes a photographing box having an openable / closable door and accommodating glasses for photographing. It is equipped with lighting means for illuminating the housed glasses and imaging means for taking images of the glasses housed in the shooting box and obtaining digital images. The glasses for shooting are housed in the shooting box, and the door is closed. This is for taking a glasses image.

As a result, it is possible to easily take a spectacle image of a predetermined quality at any time, anywhere and by anyone without being affected by external light.

The eyeglass image registration system according to claim 3 is the eyeglass image registration system according to claim 1 or 2, wherein the photographing unit includes a unit that photographs a front image of the glasses.

As a result, it is possible to acquire a glasses image for frontal composition with a human image. A glasses image registration system according to a fourth aspect is the glasses image registration system according to any one of the first to third aspects, wherein the photographing unit includes a unit that photographs a bird's-eye view image of the Megane.

As a result, an external appearance image of the glasses used in the glasses sales system can be obtained.

The eyeglass image registration system according to claim 5 is the eyeglass image registration system according to any one of claims 1 to 4, wherein the composition image generation unit specifies a frame portion of the captured glasses. It has a mask generating means for generating a mask including a frame mask image and a lens mask image designating a lens portion of the taken glasses.

With the generated frame mask image, the frame part when combining the eyeglasses with the human image can be replaced with the captured eyeglass image, and the generated lens mask The lens portion used to combine the glasses with the person image using the mask image can be translucently combined with the glasses lens, making it easy to combine the person image and the glasses in the glasses sales system.

The eyeglass image registration system according to claim 6 is the magne image registration system according to claim 5, wherein the mask generation unit is configured to detect an outer periphery of the eyeglasses of the captured eyeglass image and to perform an image capturing operation. A lens outer circumference detecting means for detecting the outer circumference of the lens of the eyeglasses image, wherein a frame mask image is generated from the outer circumference of the glasses detected by the outer circumference detecting means of the glasses and the outer circumference of the lens detected by the outer circumference detecting means, It has a function of generating a lens mask image from the outer periphery of the lens detected by the outer periphery detecting means.

The outer circumference of the glasses detected by the outer circumference detecting means represents the outer shape of the glasses, and the outer circumference of the lens detected by the outer circumference detecting means represents the range of the lens in the glasses. The range surrounded by the outer periphery of the lens can be regarded as the frame portion of the glasses, and the range surrounded by the detected outer periphery of the lens can be regarded as the lens portion of the glasses.

Through such image processing, a frame mask image and a lens mask image can be automatically generated, and anyone can register image data for synthesis that can be easily synthesized with a human image.

Also, not only full-rim glasses such as metal frames and cell frames, but also glasses that lack a part or the front of the rim, such as nyrolls or two-points, are automatically synthesized without the need for special information. Image data can be generated.

An eyeglass image registration system according to claim 7, wherein the eyeglass periphery detecting means is a captured eyeglass. The present invention further includes a glasses boundary detecting means for detecting a boundary of the glasses from the background part of the image toward the glasses part, wherein the lens outer circumference detecting means includes a lens part estimating means for estimating the lens part from the outer circumference of the glasses detected by the glasses outer circumference detecting means. Lens boundary detecting means for detecting a boundary of the lens from the lens part estimated by the lens part estimating means toward the peripheral part of the lens.

By detecting the boundary of the glasses from the background portion of the glasses image toward the glasses portion, the outer periphery of the glasses can be easily detected. In addition, the lens portion of the glasses image is estimated from the detected outer periphery of the glasses, and the boundary is detected from the vicinity of the lens portion to the periphery of the lens, so that the label attached to the lens and the reflection in the lens are detected. And other factors, and the outer periphery of the lens can be easily detected. The eyeglass image registration system according to claim 8 is the magne image registration system according to claim 7, wherein the eyeglass perimeter detection means uses a boundary of the eyeglasses detected by the eyeglass boundary detection means as a starting point, and determines a neighborhood of the eyeglass periphery. Including a glasses boundary tracking means for sequentially detecting and tracking the boundaries of the glasses, and the lens outer circumference detection means sequentially detects and tracks the boundaries of the nearby lenses starting from the lens boundaries detected by the lens boundary detection means. It includes lens boundary tracking means.

The eyeglass boundary tracking means sequentially detects and tracks the boundaries of nearby glasses starting from the detected eyeglass boundary, so that the outer circumference of the eyeglasses can be detected efficiently, and rims and nose pads with complex shapes can be used. Even if it has a complicated outer shape, it is possible to correctly detect the outer periphery of the glasses. Also, since the lens boundary tracking means sequentially detects and tracks the boundaries of nearby lenses starting from the detected boundary of the lens, the outer periphery of the lens can be detected efficiently.

The eyeglass image registration system according to claim 9 is the eyeglass image registration system according to any one of claims 6 to 8, wherein: The output means includes a spectacle outer circumference interpolation means for interpolating the detected defective portion on the outer periphery of the spectacles, and the lens outer peripheral detecting means includes a lens outer peripheral interpolation means for interpolating the detected defective portion on the outer periphery of the lens.

As a result, even if a missing portion is generated in the outer circumference data because the boundary is not clear in the eyeglass outer circumference detecting means or the lens outer circumference detecting means, the outer circumference data of the defective portion is estimated from the detected outer circumference data. It is possible to interpolate.

The eyeglass image registration system according to claim 10, wherein the eyeglass periphery detection means is an incorrect projection on the periphery of the detected glasses. The lens outer periphery detecting means includes a lens outer periphery illegal protrusion removing means for removing the detected irregular protrusion on the outer periphery of the lens.

As a result, even when an incorrect projection is generated in the outer circumference data due to a shadow, reflection, or the like in the eyeglass outer circumference detecting means or the lens outer circumference detecting means, the outer circumference data of the projection is estimated from the detected outer circumference data, and the illegal data is detected. It is possible to remove the unnecessary projections.

An eyeglass image registration system according to claim 11 is the eyeglass image registration system according to any one of claims 6 to 10, wherein the mask generation unit acquires eyeglass model information. The eyeglass outer circumference detecting means and the lens or lens outer circumference detecting means have a function of detecting the outer circumference of the glasses and the outer circumference of Z or the lens by utilizing the model information of the eyeglasses obtained by the eyeglass model information obtaining means. Things.

This makes it possible to automatically generate a mask by using the acquired model information of the eyeglasses even when the type of eyeglasses or the type of eyeglasses whose detection of the outer circumference of the lens is difficult is included. Further, when the operator determines that the initially generated mask is inappropriate, the mask may be re-created by pressing the model selection button of the glasses on the screen.

As a result, even when it is difficult to handle all types of glasses with a single processing, it is possible for the operator to easily generate a mask by pressing the model selection button.

The eyeglasses image registration system according to claim 12 is the eyeglasses image registration system according to any one of claims 5 to 11, wherein the mask generation unit includes the generated frame mask image and the generated frame mask image. The left and right comparison correction means for comparing and correcting the left and right images with respect to the lens mask image.

This makes it possible to correct even if there is a shadow or reflection on the left or right side of the frame or lens of the glasses and an incorrect part is generated in a part of the frame mask image or the lens mask image.

The eyeglass image registration system according to claim 13 is the eyeglass image registration system according to any one of claims 5 to 12, wherein the mask generation unit uses point data specified by a pointing device. It includes a mask manual editing means for editing a mask.

As a result, even when it is difficult to automatically generate a frame mask image or a lens mask image, it is possible to manually specify the frame portion or the lens portion of the glasses image captured by the pointing device, thereby manually setting the frame mask image or the lens portion. It becomes possible to edit the mask image and correct the automatically generated image data.

The eyeglass image registration system according to claim 14 is the eyeglass image registration system according to any one of claims 1 to 13, wherein the image-for-synthesis generation unit includes a pre-registered human image and a photographed image. The glasses images It includes a composite display means for displaying and composing.

This makes it possible to perform a test to combine the captured glasses image with a human image on the spot, so that if the shooting is not appropriate, it can be retaken on the spot and provide high-quality image data for synthesis. .

The eyeglass image registration system according to claim 15 is the eyeglass image registration system according to claim 14, wherein the combining display unit specifies a color and a transmittance of a lens of the glasses image to be combined with the human image. It has a function.

This allows you to specify the color and transmittance of the lens when combining the captured glasses image with the person image on the spot, so that you can change the registered contents of the product or change It is possible to redo and provide higher quality image data for synthesis.

The eyeglass image registration system according to claim 16 is the eyeglass image registration system according to claim 14 or claim 15, wherein the combining display unit adjusts a position of the glasses image to be combined with the human image. It has a function to perform.

Accordingly, when the captured glasses image is combined with the human image on the spot, the relative position between the human image and the glasses can be changed, so that it is possible to set an optimal position for the combination.

The eyeglass image registration system according to claim 17 is the eyeglass image registration system according to any one of claims 1 to 16, wherein the registration unit includes a label provided on a lens unit of the captured glasses. It is provided with a label reading means for reading the label.

As a result, it is possible to automatically read the label imprinted on the captured glasses image and register the product, preventing the occurrence of artificial registration errors and providing image data for composition more easily. it can. In addition, by detecting the outer circumference of the glasses and the outer circumference of the lens based on the read label information, false detection due to noise such as shadows and reflections is suppressed, and higher reliability is achieved. Mask generation can be performed.

A glasses image registration system according to claim 18 is the glasses image registration system according to any one of claims 1 to 17, wherein the communication transmits data registered by the registration unit to the glasses sales system. It has the means.

As a result, the registered data can be provided to the eyeglass sales system online, which makes it possible to sell eyeglasses more quickly and at lower cost.

The eyeglasses image registration method according to claim 19, further comprising: a step of photographing an image of the glasses, a step of generating an image for combination to be combined with a human image from the photographed image, and a step of generating the image for combination. It is provided with a step to be registered along with the specifications of the glasses.

It includes a step of taking a glasses image, generating a combining image that can be combined with a human image from the captured image, and registering the generated combining image together with the specifications of the glasses, so that when, where, and by whom Even if it is performed, image data that can be combined with a human image can be quickly obtained at low cost.

The eyeglass image registration method according to claim 20 is the method according to claim 19, wherein the step of generating the image for combination includes a frame mask image that specifies a frame portion of the captured glasses. And generating a mask including a lens mask image designating a captured lens portion of the lens.

The frame part when combining the glasses with the person image by the generated frame mask image can be replaced with the captured glasses image, and the lens part when combining the glasses with the person image using the generated lens mask image is the lens of the glasses. And a translucent composition, and the composition of the person image and the glasses in the glasses sales system can be easily performed. The eyeglass image registration method according to claim 21 is the eyeglass image registration method according to claim 20, wherein the step of generating the mask includes the step of detecting the outer periphery of the eyeglasses of the captured eyeglass image. A step of detecting the outer circumference of the lens of the captured eyeglass image, a step of generating a frame mask image from the outer circumference of the detected eyeglasses and the outer circumference of the detected lens, and a step of generating a lens mask image from the outer circumference of the detected lens And steps.

The perimeter of the glasses detected by the step of detecting the perimeter of the glasses represents the outer shape of the glasses, and the perimeter of the lenses detected by the step of detecting the perimeter of the lenses represents the range of the lenses in the glasses. The range surrounded by the outer circumference of the glasses and the detected outer circumference of the lens can be regarded as the frame part of the glasses, and the range surrounded by the detected outer circumference of the lens can be considered as the lens part of the glasses.

Also, not only full-rim glasses such as metal frames and cell frames, but also glasses that lack a part or the front of the rim, such as Nyrol Pit Point, etc., automatically without special information. Image data for synthesis can be generated.

A method for registering eyeglasses images according to claim 22 is the method for registering eyeglasses according to claim 21, wherein the step of detecting the outer periphery of the glasses is directed from a background portion of the captured glasses image toward the glasses portion. The step of detecting the outer periphery of the lens includes the step of estimating the lens unit from the outer periphery of the glasses detected by the step of detecting the outer periphery of the glasses, and the step of detecting the lens unit from the estimated lens unit. The border of the lens towards the periphery Detecting.

By detecting the boundary of the glasses from the background portion of the glasses image toward the glasses portion, the outer periphery of the glasses can be easily detected. In addition, the lens portion of the glasses image is estimated from the detected outer periphery of the glasses, and the boundary is detected from the vicinity of the lens portion to the periphery of the lens, so that the label attached to the lens and the reflection in the lens are detected. And other factors, and the outer periphery of the lens can be easily detected. The eyeglass image registration method according to claim 23 is the method according to claim 22, wherein the step of detecting the outer periphery of the glasses includes the step of detecting the boundary of the glasses. The method includes a step of sequentially detecting and tracking the boundary of the neighboring glasses as a starting point, and the step of detecting the outer periphery of the lens includes the step of detecting the boundary of the lens as a starting point. It includes steps for sequentially detecting and tracking boundaries.

In the step of detecting the boundary of the glasses, the detected boundary is used as a starting point to sequentially detect and track the boundaries of nearby glasses, so that the outer circumference of the glasses can be detected efficiently, and a rim or nose pad having a complicated shape can be detected. It is possible to correctly detect the outer periphery of the glasses even if the outer shape of the glasses is intricate due to the shape of the glasses. In the step of detecting the boundary of the lens, the boundary of the nearby lens is sequentially detected and tracked starting from the detected boundary, so that the outer periphery of the lens can be detected efficiently.

The eyeglass image registration method according to claim 24 is the eyeglass image registration method according to any one of claims 21 to 23, wherein the step of detecting the outer circumference of the eyeglasses includes the step of detecting the outer circumference of the detected eyeglasses. The step of detecting the outer periphery of the lens includes the step of interpolating the defective part of the detected outer periphery of the lens.

This allows you to detect the outer circumference of the glasses and detect the outer circumference of the lens Even in the case where a missing portion is generated in the outer peripheral data because the boundary is not clear or the like, the outer peripheral data of the defective portion can be estimated and interpolated from the detected outer peripheral data.

The eyeglass image registration method according to claim 25 is the eyeglass image registration method according to any one of claims 21 to 24, wherein the step of detecting the outer circumference of the eyeglasses includes the outer circumference of the detected eyeglasses. The step of detecting the outer periphery of the lens includes the step of removing the illegal projection of the lens.

In this way, even if an incorrect projection is generated in the outer circumference data due to shadows or reflections in the step of detecting the outer circumference of the glasses or the step of detecting the outer circumference of the lens, the detected outer circumference data is used to determine the outer circumference of the projection. By estimating the data, it is possible to remove the illegal protrusion.

The eyeglass image registration method according to claim 26 is the eyeglass image registration method according to any one of claims 21 to 25, wherein the step of generating a mask acquires model information of the glasses. Including step means, the step of detecting the outer circumference of the glasses and the step of detecting the outer circumference of the lens detect the outer circumference of the glasses and the outer circumference of Z or the lens using the acquired model information of the glasses. It includes steps.

This makes it possible to automatically generate a mask by using the acquired model information of the glasses, even when the model includes a model in which it is difficult to detect the outer circumference of the glasses or the outer circumference of the lens.

When the operator determines that the initially generated mask is inappropriate, the mask may be recreated by pressing the glasses model selection button on the screen.

This makes it possible for the operator to select a model even if it is difficult to handle all types of glasses with a single process. A mask can be easily generated by pressing a button.

The eyeglass image registration method according to claim 27 is the eyeglass image registration method according to any one of claims 20 to 26, wherein the step of generating a mask includes generating the frame mask image and / or This step includes a step of comparing the left and right images of the generated lens mask image and correcting them.

The eyeglass image registration method according to claim 28 is the eyeglass image registration method according to any one of claims 20 to 27, wherein the step of generating a mask is specified by a pointing device. It includes steps to edit the mask from the last day.

This makes it possible to manually edit the frame mask image and lens mask image by specifying the frame and lens portions of the glasses taken by the pointing device, even if it is difficult to automatically generate the frame mask image and lens mask image. Or correct the automatically generated image data.

The eyeglass image registration method according to claim 29 is the eyeglass image registration method according to any one of claims 19 to 28, wherein the step of generating the image for synthesis is a person registered in advance. It includes the step of combining and displaying the image and the captured glasses image.

As a result, it is possible to perform a test of combining the captured glasses image with a human image on the spot, so that if the shooting is not appropriate, it is possible to retake the image on the spot and provide high-quality image data for synthesis. .

The method for registering eyeglasses images according to claim 30 is the method according to claim 29. In the image registration method, the step to be displayed has a step of designating the color and transmittance of the lens of the glasses image to be combined with the human image.

The eyeglass image registration method according to claim 31 is the eyeglass image registration method according to claim 29 or claim 30, wherein the displaying step is a step of adjusting the position of the eyeglass image to be combined with the human image. It has. This makes it possible to combine the captured eyeglasses image with a human image on the spot.

Since the relative position between the human image and the glasses can be changed, it is possible to set an optimal position for the composition.

The eyeglass image registration method according to claim 32 is the eyeglass image registration method according to any one of claims 19 to 31, wherein the registration unit is provided on a lens unit of the captured glasses. The method includes a step of reading a label.

As a result, it is possible to automatically read the label imprinted on the captured glasses image and register the product, preventing the occurrence of artificial registration errors and providing image data for composition more easily. it can. In addition, by detecting the outer circumference of the glasses and the outer circumference of the lens based on the read label information, erroneous detection due to noise such as shadows and reflections can be suppressed, and more reliable mask generation can be performed. Become.

The eyeglass image registration method according to claim 33 is the eyeglass image registration method according to any one of claims 19 to 32, wherein the step of transmitting the image data for synthesis registered by the registration unit is performed. It is provided with.

As a result, the registered data can be used online in the glasses sales system. Since it can be provided, it will be possible to sell eyeglasses more quickly and at lower cost.

The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments of the present invention with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a configuration diagram of a glasses image registration system according to an embodiment of the present invention.

FIG. 2 is an illustrative view of a front image capturing device of the glasses image registration system according to one embodiment of the present invention.

FIG. 3 is an illustrative view showing a maintenance door of the front image photographing device of the glasses image registration system according to one embodiment of the present invention.

FIG. 4 is an illustrative view of a bird's-eye view image capturing apparatus of the glasses image registration system according to one embodiment of the present invention.

FIG. 5 is an overall processing flowchart of a computer in the glasses image registration system according to the embodiment of the present invention.

FIG. 6 is a processing flowchart according to an embodiment of the mask automatic processing. FIG. 7 is a process flow chart according to an embodiment of the eyeglass periphery detection processing.

FIG. 8 is a processing flowchart according to an embodiment of the lens outer periphery detection processing.

FIG. 9 is a processing flow chart according to one embodiment of the composite image display processing. FIG. 10 is a processing flow chart according to one embodiment of the mask manual editing processing. is there.

FIG. 11 is a processing flowchart according to an embodiment of the data registration processing. FIG. 12 is an explanatory diagram of a starting point in the glasses outer periphery detection processing. FIG. 13 is an explanatory diagram of nearby points in the boundary tracking processing of glasses and lenses.

FIG. 14 is an explanatory diagram showing the tracking process of the eyeglasses and lens boundary tracking process.

FIG. 15 is an illustrative sectional view in the longitudinal direction of the front image capturing apparatus according to one embodiment of the present invention.

FIG. 16 is an illustrative view of eyeglasses viewed from the camera side of the front image photographing apparatus according to one embodiment of the present invention.

FIG. 17 is an explanatory view showing the principle of illumination for making the outline of the glasses clear.

FIG. 18 is an explanatory view showing the effect of lighting for making the outline of the glasses clear.

FIG. 19 shows an example of gray processing performed on the inner surfaces (〇 surface '11 surface) of the side panels 22 and 23 of the magenta accommodating portion of the front image photographing apparatus according to one embodiment of the present invention. FIG.

FIG. 20 is an explanatory diagram showing an example of gray processing performed on the inner surface (B surface) of the rear panel 25 of the front image photographing device according to one embodiment of the present invention. ,

FIG. 21 is an explanatory diagram showing an example of gradation processing performed on the inner surface (eight surfaces) of the front panel 20 of the camera housing section of the front image photographing apparatus according to one embodiment of the present invention.

FIG. 22 shows opening and closing of the front image capturing apparatus according to the embodiment of the present invention. FIG. 9 is an explanatory view showing an example of black processing applied to the inner surface (C surface) of the door upper panel 28.

FIG. 23 is an explanatory diagram showing an example of gray processing performed on the inner surface (B-F surface) of the rear part of the lower panel 12 of the front image photographing apparatus according to one embodiment of the present invention.

FIG. 24 is an example of an image of metal frame glasses taken by the front image photographing apparatus according to one embodiment of the present invention.

FIG. 25 is an example of an image obtained by photographing cell-frame glasses using the front image photographing apparatus according to the embodiment of the present invention.

FIG. 26 is an example of an image obtained by photographing Nyrol glasses using the front image photographing apparatus according to one embodiment of the present invention.

FIG. 27 is an example of an image of one-point glasses taken by the front image photographing apparatus according to one embodiment of the present invention.

FIG. 28 shows an example of a display screen (before the start of photographing) in photographing glasses images.

FIG. 29 is an example of a display screen in the imaging of the glasses image (after the end of the imaging). FIG. 30 is an example of a display screen in the mask processing (before the processing is started). FIG. 31 shows an example of a display screen in the mask processing (after the processing is completed). FIG. 32 is an example of a display screen in manual mask editing.

Fig. 33 shows an example of a display screen for data registration (before registration starts). Fig. 34 shows an example of a display screen during registration (after registration is completed). FIG. 35 shows a modification of the display screen in the mask processing. BEST MODE FOR CARRYING OUT THE INVENTION-FIG. 1 shows a glasses image registration system according to an embodiment of the present invention. A system configuration diagram is shown that is connected to a gane sales system via a network.

In the figure, 1 is a glasses image registration system, and 300 is a glasses sales system. The glasses image registration system 1 is composed of a front image photographing device 100 for photographing a front image of glasses, a bird's-eye image photographing device 100 for photographing a bird's-eye image of glasses, and image processing of photographed glasses images. And a computer 200 for processing image data.

First, an embodiment of a front image capturing apparatus 10 for capturing a front image of glasses to be subjected to image processing will be described.

As shown in FIG. 2, the front image capturing device 10 includes a capturing box 11 for blocking external light when capturing a glasses image, and a glasses fixing member for fixing the glasses 2 to be captured. Barrel Torso 30, fixed lamps 40, which are illumination means for illuminating shooting target glasses 2 fixed to torso 30, and fixed distances, facing shooting target glasses 2 fixed to torso 30 And a digital camera 50 as an image pickup means for obtaining digital image data by photographing glasses from the camera.

As shown in FIGS. 15 and 16, the photographing box 11 is provided on the lower panel 12 serving as an entire base plate, and at one end of the lower panel 12 at the front end thereof. A camera housing part 14 for housing the camera body 14, and an eyeglass housing which is provided at the other end of the lower panel 12 at the rear end, has an opening on the camera housing part 14 side, and is installed so as to surround the torso 30. It consists of a part 16 and an opening / closing door 18 that slides along a guide rail 13 provided on the lower panel 12.

The camera accommodating portion 14 has a front panel 20 on the torso 30 side, and has a lens hole 21 of the camera 50 at a position facing the glasses 2 fixed to the torso 30. The rear of the camera housing 14 (K side in Fig. 15) is used to supply power to the camera 50 and the illumination lamp 40. A power connector 52 for turning on / off the power of the camera 50 and the illumination lamp 40; and a USB connector 56 for connecting the camera 50 to the computer 200.

A pair of left and right side panels 22 and 23 surrounding the side of the torso 30, an upper panel 24 surrounding the upper side of the torso 30, and a rear panel surrounding the back of the torso 30 are provided. 2 5.

The opening / closing door 18 slides along the guide rail 13 and includes a pair of left and right side panels 26 and 27 surrounding the side surfaces, and an upper surface panel 28 surrounding the upper side. FIG. 2 shows a state in which the opening and closing door 18 is opened. In this state, the glasses 2 are set on the torso 30 and the opening and closing door 18 is moved until it comes into close contact with the camera accommodating section 14 and closed. As a result, the external light is completely blocked, and it is possible to photograph the glasses 2 with the force camera 50.

For convenience, the front panel 20 side of the shooting box 11 is called the front side, the rear panel 25 side is called the rear side, and the front panel 20 is the front of the shooting box 11 and the rear panel 25 is the shooting box. 1 1 Rear, side panel 2 2 and 2 3 of glasses housing 1 6 shooting box 1 1 side, bottom panel 1 2 glasses housing 16 side of shooting box 1 1 The upper panel 28 of 18 is referred to as the upper front side of the shooting box 11.

Lower panel 1 2 that constitutes shooting box 1 1, front panel 20 of camera housing 14, side panels 22 and 23 of glasses housing 16, and upper panel 2 4 of glasses housing 16 The interior of the rear panel 25, the side panels 26 and 27 of the door 18 and the top panel 28 of the door 18 is based on white, and the light of the lighting lamps described later By reflecting light from the camera, it was possible to illuminate from multiple directions without affecting the color tone of the glasses.

For this reason, in this embodiment, a white decorative panel is used for these panels. Although the photographing box 11 was formed as a result, it goes without saying that the inner surface may be painted white.

As shown in FIG. 3, the rear upper panel 24 is provided with an opening / closing mechanism (not shown) that can be opened and closed upward with the door 18 closed so that the illumination lamp 40 can be maintained.

Here, the opening / closing door 18 is of a sliding type here, but it goes without saying that a lid for taking in and out the glasses may be provided.

The torso 30 is a cylindrical body on which the glasses to be photographed are set, and is vertically fixed to the lower panel 12 of the magnet housing 16. In this embodiment, a cylindrical body formed by winding white rubber on a cylindrical core made of styrene foam or the like was used. The diameter of the cylinder is set so that it does not slip down with the widest interval of the temples in the glasses 2 to be photographed.

In front of the torso 30, that is, in the vicinity of the torso 30 on the side of the camera accommodating part 14 for accommodating the camera 50, in order to prevent the left and right sides of the glasses 2 from being tilted, a transparent acrylic on which the front parts of the glasses 2 are placed A glasses base 32 is provided. The glasses pedestal 32 has a width that is approximately the width of the glasses 2 and a minimum thickness that allows the front part of the glasses 2 to be placed, so that the height of the glasses 2 and the distance to the camera 50 are fixed within a certain range. In addition, it is configured to prevent left and right inclination (axis deviation) when setting. In addition, since the eyeglass pedestal 32 is formed of a transparent acrylic resin plate, the reflected light from the lower panel 12 is also supplied to the lower part of the lens of the eyeglass 2 so that the eyeglass pedestal 32 can be used for eyeglasses such as a nyrole bottle. The outline of the lens can be captured more clearly.

In the present embodiment, the eyeglass pedestal 32 is fixed to the lower panel 12, and formed so that the lens of the eyeglasses 2 placed on the upper surface of the glass pedestal 32 has a height directly opposite to the lens of the force lens 50. Have been. Also, the glasses base 3 2 In order to prevent the shadow of glasses 2 from appearing on the torso 30, it is fixed at a position slightly away from the torso. As a result, a glasses image in which the lens portion of the glasses 2 is more clearly captured is obtained.

In addition, a scale and a color patch are provided on the surface of the glasses base 32 on the camera 50 side, and the photographing size and the color tone can be checked by transferring the scale and the color patch to the camera 50.

The illumination lamp 40 is fixed to the inner surface of the upper panel 24 of the glasses housing 16 to directly illuminate the glasses 2 set in the torso 30 from above, and the reflected light from the inner surface of the shooting box 11 Indirect illumination from all directions.

As the illumination lamp 40, a circular fluorescent lamp was used. This is because illumination with a fluorescent lamp does not easily impair the color tone of the glasses 2, is easily available, and has excellent maintainability. In addition, the reason for using a circular fluorescent lamp is that since the light emitting area is large and the light is illuminated from a wide range, shadows are hardly formed and glasses images with less disturbance can be obtained. Since the upper panel 24 has the opening / closing mechanism as described above, the illumination lamp 40 can be easily replaced.

The illumination lamp 40 is provided behind the glasses pedestal 32 on the inner surface of the upper panel 24 (farther from the digital camera 50). This aims at obtaining a spectacle image in which the outline of the lens of the spectacle is more clearly captured by illuminating the camera 2 from behind the camera 50.

Fig. 17 explains the principle. The upper part of the glasses 2 reflects the direct light from the light source and is captured by the camera 50. The lower part of the glasses 2 receives the light from the illumination lamp 40 and the lower panel 1 The light is reflected by 2 and supplied through the transparent glasses base 32, and the reflected light is captured by the camera 50. This allows As shown in Fig. 18, it has become possible to more clearly capture the outline of the lens even with glasses such as nyrol and two-point glasses.

In the present embodiment, the inner surfaces of the side panels 22 and 23 of the glasses housing portion 16 (the G surface and the Η surface in FIG. 16) are subjected to 70% gray processing as shown in FIG. It has been subjected. The inner surface of the rear panel 25 (the upper surface in FIG. 15) is subjected to a 40% gray treatment as shown in FIG. This is because the front panel 20 on which the camera 50 is mounted needs to be at a certain distance from the glasses 2 in order to capture the front image of the glasses 2 as flat as possible, but from the glasses 2 to the side and rear. The distance is set as small as possible due to the miniaturization of the equipment, and the reflected light from the inner surface of the shooting box 11 supplied to the glasses becomes too bright on the side and rear compared to the front. This is because Therefore, the inner surfaces of the side panels 22 and 23 were treated with 70% gray, and the inner surface of the rear panel 25 was treated with 40% gray, to achieve uniform lighting conditions from all directions. . As a result, it was possible to take a flat image of the mound and obtain glasses images with less disturbance.

In the present embodiment, the gray processing is performed by printing black at a predetermined density on synthetic paper having a plastic texture called "upo" and pasting it on the inner surface of the panel.

It goes without saying that the gray treatment may be such that the inner surface of the panel is painted in a predetermined gray.

In the present embodiment, as shown in FIG. 15, the inner surface (upper surface) of the rear panel 25 is rounded toward the lower panel 12 (F surface). This is to prevent the boundary between the rear panel 25 and the lower panel 12 from being caught by the force melody and becoming a disturbance.

In the present embodiment, the process of providing roundness on the inner surface of the rear panel 25 is as described above. The 40% gray-treated rubosite was rounded and attached from the rear panel 25 to the lower panel 12.

It is needless to say that the rounding of the inner surface of the rear panel 25 may be performed, for example, by using a member processed to a predetermined curvature, or by forming the imaging box 11 itself with a rounded panel. .

As shown in FIG. 15, the front panel 20 is provided with a lens hole 21 of the camera 50, and the camera 50 is mounted in the camera accommodating portion 14. The lens hole 21 of the force lens 50 is provided so that the center of the image of the camera substantially coincides with the center of the height of the glasses when the glasses are placed on the glasses base 32. Therefore, the lens of the glasses 2 is directly opposed to the lens hole 21 of the camera 50, and there is a problem that the lens of the camera 50 is reflected on the lens of the glasses 2.

For this reason, in the present embodiment, the inner surface of the front panel 20 (the A surface in FIG. 15) is subjected to 40% gray processing as a whole as shown in FIG. The periphery of 21 is subjected to a gradation process that darkens the center and gradually brightens toward the periphery. As a result, the reflected light from the front panel is suppressed, and the lens portion of the camera 50 is prevented from being reflected on the lens of the glasses 2, and a gradation process is applied to the lens surface of the glasses 2 to give a shade to the glasses 2. This prevents the lens from becoming completely transparent when viewed from the front.

In the present embodiment, the gradation process is performed by printing black with a predetermined density change on synthetic paper called “rubo” and pasting it on the inner surface of the panel, similarly to the gray process. Here, the lens portion of the gradation-processed lens whose center is 100% and its periphery is 40% was cut out and used. The gradation is set so that the color around the lens hole 21 is almost the same as the color of the camera 500 lens. You.

It is needless to say that the gradation process may be such that the inner surface of the panel is painted gray with a predetermined density change.

In the present embodiment, the inner surface (the C surface in FIG. 15) of the door upper panel 28 is subjected to 100% gray processing (black processing) as shown in FIG. This is intended to prevent the boundary line between the front panel 20 and the lower panel 12 from being reflected by the lens of the gun 2 and being reflected on the camera 50. By making 8 black, the illumination of the border was suppressed, and almost no image was captured by the glasses 2 lens.

In this embodiment, the inner surface of the eyeglass accommodating portion 16 of the lower panel 12 (the surface in FIG. 15) is dark on the rear panel 25 side as shown in FIG. A gradation process is applied to increase the brightness toward the position. This is because the torso 30 is fixed to the lower panel 12, and the reflected light from the lower panel 12 causes an imbalance in the brightness of the top and bottom that the bottom of the torso 30 is bright and the top is dark. Because. For this reason, gradation processing is performed on the inner surface of the eyeglass accommodating section 16 of the lower panel 12 with 70% gray on the rear panel 25 side and 0% (white) on the installation position side of the torso 30. The brightness of the top and bottom of the torso 30 became uniform, and glasses images with less disturbance could be obtained.

As described above, the rear panel 25 is rounded with a 40% gray-processed adhesive toward the lower panel 12, and the lower panel 12 is bonded to the rear panel 25 with a rounded surface. It is set to be almost 40% gray.

As the camera 50, a 300,000-pixel class digital camera equipped with a lens having a focal length of 5 O mm was used. Also, camera 50 is USB It is connected to the computer 200 via the interface, sets the glasses 2 on the torso 30, closes the door 18, and takes a front image of the glasses by remote control from the computer 200.

FIGS. 24 to 27 show examples of images actually photographed by the front image photographing apparatus 10. Fig. 24 shows metal frame glasses, Fig. 25 shows self-frame glasses, Fig. 26 shows Nyrol glasses, and Fig. 27 shows two-point glasses, all of which have a uniform background and frame outline. The images are clearly captured, and the outline of the lens can be captured at a level that allows image processing even with Nyrol and two-point glasses.

On the other hand, the bird's-eye view image capturing device 100 captures a bird's-eye view image of the glasses in order to present the appearance of the glasses to the customer in the glasses sales system 300, and as shown in FIG. A shooting box 1 1 1 to block external light when shooting an image, an eye lamp 2 140 serving as a lighting means for illuminating the shooting target glasses 2 installed in the shooting box 1 1 1, and a shooting target It comprises a digital camera 150 as an imaging means for obtaining digital image data by photographing the glasses 2 from a certain distance.

As in the case of the photographing box 11 of the front image photographing apparatus 10 described above, the photographing box 1 1 1 is attached to the lower panel 1 1 2 serving as the entire base plate and one end of the lower panel 1 1 2 as the front part. And a camera housing section 114 for housing the camera 150, and a rear end provided at the other rear end of the lower panel 112, which has an opening in the direction of the camera housing section 114. 1 2 Glasses storage section 1 16 for storing glasses installed on top of 1 2 and opening and closing door 1 1 8 sliding along guide rail 1 1 3 provided on bottom panel 1 1 2 .

Further, the photographing box 111 is formed of a panel having a white color on the inner surface in the same manner as the photographing box 11 of the front image photographing apparatus 10 described above, and By using a circular fluorescent lamp as the bright lamp 140, white light is supplied to the glasses from multiple directions, so that a good external image of the glasses with less shadow and reflection can be taken.

Note that the overhead image photographing device 100 is also implemented by the above-described front image photographing device 100 according to the shadow condition, uneven brightness, reflection of the lens of the glasses, etc. on the inner surface of the photographing box 111. It goes without saying that the same gray processing / gradation processing as described above may be performed.

The camera accommodating section 1 14 and the opening / closing door 18 of the photographing box 1 1 1 are the same as those of the front image photographing apparatus 10, and therefore description thereof is omitted.

As the camera 150, a digital camera of 300,000 pixel class equipped with a zoom lens having a focal length of 28-105 mm was used. The camera 150 is connected to the computer 200 via a USB interface.Then, set the glasses 2 in the glasses storage section 116, close the door 118, and start from the computer 200. Take a bird's-eye view of the glasses with the remote control of.

The computer 200 is connected to the camera 50 of the front image capturing device 10 and the camera 150 of the overhead image capturing device 100 via a USB interface, and connects the camera 50 and the camera 150 with each other. Remotely control the camera to capture the composition glasses image and the appearance glasses image and capture them all at once.

The captured eyeglasses image for synthesis generates image data for synthesis to be synthesized with a human image by image processing.

Further, the computer 200 has a network connection function as a communication means, and transmits the generated image data to the glasses sales system 300.

The computer 200 that performs such processing includes a personal computer or a network with a USB interface and network connection function. Stations and the like can be used.

Next, the flow of image capturing and image processing by the computer 200 will be described.

The processing can be broadly divided into three stages: “shooting glasses images”, “mask processing”, and “data registration”.

First, a process in which a registrant of a glasses image registers a glasses image using the glasses image registration system 1 will be described using a screen display example displayed on the computer 200.

Figure 28 shows an example of the screen display before the start of shooting in “glasses image shooting”. As shown in the figure, there are a “front view” button and an “appearance view” button, respectively, for the digital camera 50 of the front view image capture device 10 and the digital camera 150 of the overhead view image capture device 100. A shooting command is sent, and the captured image data is loaded into the computer 200 via the USB interface.

Fig. 29 shows an example of the screen display after shooting in “Glasses image shooting”. As shown in the figure, the captured front image and overhead image are displayed on the screen. If the shot image is bad, you can reset the glasses and press the “Front shot” button or the “Appearance shot” button to take the shot again and again. Once a good image has been captured, you can press the “next frame” button to continue capturing many images of the glasses.

FIG. 30 shows an example of a screen display before the processing in the “mask processing”. As shown in the figure, the file name of the captured glasses image is displayed in the “original folder” section, and by pressing the “Start mask creation” button, automatic mask processing is performed on the captured glasses image.

Fig. 31 shows an example of the screen display after the processing in the "mask processing" is completed. As shown in the figure, the front image was captured by the front image capturing device 10 in the “original image” section. The front image is an image in which the scale etc. of the lower part of the front image are cut in the “display image” section, and the image in which the lens of the front image is colored in the desired color is the “frame mask” in the “composite image” section. A frame mask image in which only the frame portion of the front image is extracted is displayed in the "" portion, and a lens mask image in which only the lens portion of the front image is extracted is displayed in the "lens mask" portion. The color of the lens is selected from the colors provided at the lower left.

In addition, a test image in which the image for synthesis is synthesized with a standard human image is displayed in the “glasses check” section. At this time, if a numerical value is entered in the “lens transmittance” section, the image is synthesized with the human image at the set transmittance. Also, the position of the glasses when combining with the human image can be adjusted with the arrow button.

If the automatic masking process is not successful, you can press the "Adjust" button to manually edit the mask and perform "Mask Manual Edit". Fig. 32 shows an example of the screen display of “Manual editing of mask”. As shown in the figure, press the “RGB Display” button to display the image for compositing, and press the “Translucent Mask” button to display the image obtained by translucently combining the mask image with the image for compositing. Press to display the mask image. At this time, selecting the “Edit Lens” button uses the lens mask image, and selecting the “Edit Frame Mask” button uses the frame mask image.

Here, the polygon can be drawn by selecting the “Polygon Set” button and clicking on the screen with the mouse, so that the contour of the lens or frame can be set. This allows you to edit the mask manually.

If you check “Spline” at this time, a smooth curve is automatically applied, and a mask with a smooth outline can be generated. Fig. 33 shows the screen before registration of “Data registration” is started. The display example is shown. As shown Basically, set product code, glasses type, product number, color number, brand as basic items, and set gender, type, frame color, different colors, lens diameter, nose width, temple length as frame information. And other information such as price, sales status, product display flag, inventory management information, sales date, and registrant.

Fig. 34 shows an example of the screen display after registration of "data registration". As shown in the figure, the corresponding image file is selected according to the entered product code, and the display image, frame mask image, lens mask image, image for synthesis, and appearance image are displayed in the “image” section. By setting the required items and pressing the “Register” button, the setting data is registered and transmitted to the Megane sales system 300 via the network.

FIG. 5 shows an example of an overall processing flow of the computer 200 that performs the above-described registration processing.

In “Glasses image shooting”, the screen for shooting glasses images is displayed (S100), and when the “Front shot” button is pressed (S1002), the camera 50 of the front image shooting device 10 is displayed. A photographing command is sent, and the photographed image is acquired as a front image (S104). When the “appearance photographing” button is pressed (S 106), a photographing command is sent to the camera 150 of the bird's-eye image photographing device 100, and the photographed image is acquired as a bird's-eye image (S 110). 8). When the "next frame" button is pressed (S110), the shooting number is updated (S112), and this is repeated from S102 until the "End" button is pressed (S111) Four

In “Mask processing”, the screen of the mask processing is displayed (S116), and when the “Start masking” button is pressed (S118), the frame mask image and lens mask image of the captured front image are displayed. Activate “mask automatic processing” (S200). Next, a "synthesized image display process" (S300) for performing a synthesis test with a human image using the generated frame mask image, lens mask image, and synthesis image is started. When the “Adjust” button is pressed, “Mask Manual Editing” (S400) for manually editing the mask image is started, and “Synthetic Image Display Processing” is performed using the edited mask image. (S300). The above processing is repeated from S118 until the "End" button is pressed (S122)

In the “data registration”, the data registration screen is displayed (S124), and the “de-night registration process” (S500), which registers the image data together with the specifications of the glasses, is started and registered. The transmitted data is transmitted to the glasses sales system 300 (S126).

Next, the details of the “mask automatic processing” (S 200) will be described. Fig. 6 shows an example of the processing flow of "mask automatic processing". As shown in the figure, first, a “glass outline detection process” (S220) is performed to detect the outline of the glasses from the background toward the glasses, and the left lens position is estimated from the detected outline of the glasses. Then, a “lens outer circumference detection process” (S 250) for detecting the outer circumference of the lens for the left lens is performed. Similarly, the right lens position is estimated from the detected outer circumference of the glasses (S204), and a "lens outer circumference detection process" (S225) for detecting the outer circumference of the lens for the right lens is performed. The estimation of the left and right lens positions (S202, S204) will be described later, as will be described later, in the outer periphery data obtained by the eyeglass outer periphery detection processing (S220). Then, a lens outer periphery detection start point is set inside from the boundary at the predetermined feature point in consideration of the frame width of the glasses, and the outer periphery of the lens is detected from the inside of the lens toward the peripheral portion using the start point.

Next, a lens mask image is generated from the left and right lens outer periphery data obtained by the “lens outer periphery detection processing” (S250, S252) (S250). 6) Then, a left-right comparison correction process is performed on the generated lens mask image (S208).

In the generation of the lens mask image (S206), the area surrounded by the left and right lens outer peripheral data is painted out, and an image in which the inside is white and the outside is black is generated to be a lens mask image.

In the left / right comparison correction processing (S208), the generated lens mask image is turned back from the center of the stake, and the left and right images are compared to extract a difference. If the difference exceeds a predetermined size, it is determined which of the left and right is out of the line determined by curve approximation from the lens outer peripheral data, and the other side is added to the lens mask image on the other side. Write back the lens mask image of. This makes it possible to correctly generate a lens mask image even when a label is stuck near the outer periphery of one of the lenses, or when detection of the outer periphery of the lens fails due to a shadow or reflection. In addition, the perimeter data of the lens obtained by the “glass outer circumference detection processing” (S220) and the left and right lenses obtained by the “lens outer circumference detection processing” (S250, S252) A frame mask image is generated from the outer periphery data (S210), and the left and right sides of the generated frame mask image are compared for correction (S212).

In generating the frame mask image (S210), the area between the outer circumference data of the glasses and the outer circumference data of the left and right lenses is filled, and an image in which the corresponding area is white and the non-applicable area is black is generated as a frame mask image. . Here, in glasses that lack a part of the rim or the front part, such as a two-point two-point model, a thin line is formed due to the problem of the direction of boundary detection at the overlapping part of the outer periphery of the glasses and the outer periphery of the lens. There is a possibility that this region may occur, but this can be removed by using, for example, a contraction / expansion process that contracts and expands the generated frame mask image. In the left and right comparison and correction processing (S2122), the generated frame mask image is folded back from the center of the glasses, and the left and right images are compared to extract a difference. If the difference exceeds a predetermined size, it is determined which of the right and left is out of the line obtained by curve approximation from the outer periphery data of the lens and the outer periphery data of the lens, and the frame mask image on the off side is determined. The other side of the frame mask image is written back. This makes it possible to correctly generate a frame mask image even when the detection of the outer periphery of the glasses / lenses has failed due to a shadow or reflection.

FIG. 7 shows an example of a processing flow of the glasses outer periphery detection processing (S220). As shown in the figure, first, a boundary detection start point of the glasses is selected (S2222). For the boundary detection start point of the glasses, a line approaching the feature points of the glasses from the background to the glasses section is determined in advance, and the line is selected from the start point. In the front image capturing apparatus 10, the glasses 2 are set on the torso 30 and the lens portion is placed on the glasses pedestal 32, so that the position and the existence range of the glasses in the glasses image are limited to a certain range. Therefore, a process is performed to cut out the glasses present region 4 in the magenta image and detect the boundary of the glasses from the outer periphery of the glasses present region 4 toward the glasses.

For example, as shown in Fig. 12, as the characteristic points of the eyeglasses, points that can detect the outer periphery of the eyeglasses relatively stably are selected. On the other hand, P1 to P1 Set a starting point like 7

Here, the upper part of the glasses is detected from above to below, and the lower part of the glasses is detected from below to above, but the present invention is not limited to this. It is also possible to provide candidate points for approaching from a horizontal direction or an oblique direction.

Then, the following steps are repeated until the processing for all candidate points is completed (S2224). For the selected start point, glasses boundary detection processing for detecting a boundary from the background portion toward the glasses portion is performed (S226). To detect the boundary, the luminance of each pixel on the line is extracted, and points having a luminance change equal to or greater than a predetermined value are extracted. Since the frame and lens of the glasses may be brighter or darker than the background, here, the luminance change on both the plus side and the minus side with respect to the background is detected. If the boundary detection is not successful, the flow returns to S222, and the eyeglass boundary detection processing is performed for the next eyeglass boundary detection start point (S228).

If the detection of the boundary is successful, the eyeglasses boundary tracking process for sequentially tracking the boundaries of the glasses starting from the boundary in the following steps is performed. That is, the detected boundary is registered in the eyeglass periphery data (S230), it is detected whether or not a new boundary exists near the detected boundary (S232), and if the detection is successful (S23) 4), move to the detected new boundary (S2336), and repeat from S2330 until the tracking is completed (S238).

As shown in Fig. 13, eight neighboring points are selected on the basis of the current pixel, and the brightness of each pixel is extracted in a certain direction after the previous boundary. Is detected. For example, if the next boundary is detected clockwise from the adjacent point, the boundary of the glasses can be tracked clockwise.

If the next boundary cannot be detected (S2334), the next boundary is determined by curve approximation or the like from the outer circumference data detected so far until no detection is made continuously (S2400) until the predetermined boundary is not detected. Is estimated (S 2 42), and the process is repeated from S 2 32. If no detection has been performed continuously for a predetermined number of times (S240), the process returns to S222, selects the next eyeglass boundary detection start point, and repeats the eyeglass boundary tracking processing.

As described above, when the eyeglasses boundary tracking process is performed, The process is shown in FIG. As shown in the figure, for example, starting from the eyeglass boundary detection start point P3, the boundary F1 of the eyeglasses is detected first, and the boundaries are tracked clockwise from F2, F3, F4,…. Go.

Tracking is completed (S 238) in principle when returning to the starting point where tracking started, but here there is a possibility of making a U-turn because the outer circumference of the glasses is interrupted on the way, It was detected that tracking was continued at a certain distance from the already detected outer circumference, and tracking was also completed in this case. When the tracking is completed, the missing part of the glasses outer circumference data is interpolated (S244). This is because not all of the outer circumference of the glasses is clearly captured in the glasses image, and the glasses boundary detection processing for detecting the boundary from the background to the megane in S226. The outer peripheral portion of the glasses that could not be detected by the glasses boundary tracking processing of S230 to S238 is interpolated. In the interpolation, based on the detected outer circumference of the glasses, an outer circumference of the missing portion is generated by, for example, curve approximation, and added to the outer circumference data of the glasses.

Next, the illegal projections are removed from the eyeglass periphery data (S246). This is to remove illegal projections on the outer periphery of the glasses due to shadows or reflections on the background. The outer circumference data is estimated, and the detected eyewear outer circumference data is compared with the estimated eyewear outer circumference data. If the distance is more than a predetermined distance, the detected eyewear outer circumference data is determined to be an invalid projection, and the detected eyewear outer circumference data is estimated. Replace with data.

Finally, the outer circumference data of the glasses is smoothed (S248). This is to smooth out irregularities caused by unevenness in brightness at the boundary. For example, coordinate data is replaced with the average value of coordinate data in a certain range before and after. In addition, smoothing is applied to a curve estimated by least squares approximation from eyeglass circumference data. You may make it change.

The outer circumference of the glasses has corners such as a bridge, a nose pad, and a tongue, and if this part is smoothed, the corners become rounded, so the average azimuth changes between before and after The smoothed portion was not smoothed. FIG. 8 shows an example of the processing flow of the lens periphery detection processing (S250, S252). As shown in the figure, first, the starting point of the boundary detection of the lens is selected (S254). This is to extract the data related to the left and right 'lens portions among the above-mentioned eyeglass feature points from the eyeglass outer circumference data obtained by the eyeglass outer circumference detection processing (S220), and consider the eyeglass frame width from this Then, a position inside the predetermined width is set as a candidate point and selected from these. For example, in FIG. 12, among the eyeglass boundary detection start points, P2, P3, P4, PI1, PI2, and P13 are on the left lens unit, and the corresponding Q2, Q3, Q4, Qll, Q12, and Q13 are candidates for the left lens boundary detection start point, and similarly, among the eyeglass boundary detection start points, P6, P7, P8, P15, P16 and P17 cover the right lens section, and the corresponding Q6, Q7, Q8, Q15, Q16 and Q17 are candidates for the right lens boundary detection start point. It is. In this case, the left lens boundary detection start line 6 is set inside the left lens from the one around the left lens in the outer periphery of the glasses, and the right lens out of the right lens out of the glasses outer data. Alternatively, the right lens boundary detection start line 8 may be set at the right, and the left and right lens boundary detection start points may be selected from any positions on these boundary detection start lines.

Here, the outer periphery of the lens is detected from the upper part of the lens from below to the upper part, and the lower part of the lens is detected from the upper part to the lower part. However, the present invention is not limited to this. It is also possible to provide candidate points for approaching from a horizontal direction or an oblique direction. Then, in the same manner as the eyeglass periphery detection processing (S220), the following steps are repeated until the processing for all the candidate points is completed (S256). The lens boundary detection processing for detecting the boundary is performed toward the section (S258). In detecting the boundary, the luminance of each pixel on the line is extracted, and points having a luminance change of a predetermined value or more are extracted. Since the lens portion may be brighter or darker than the peripheral portion of the lens, the change in luminance on both the plus side and the minus side with respect to the lens portion is detected here. If the boundary detection is not successful, the flow returns to S254, and the lens boundary detection processing is performed for the next lens boundary detection start point (S260)

) o

If the detection of the boundary is successful, the lens boundary tracking process for sequentially tracking the boundary of the lens from the boundary as a starting point is performed by the following steps. Ie

Then, the detected boundary is registered in the lens outer circumference data (S2662), and whether a new boundary exists near the detected boundary is detected (S264), and if the detection is successful (S266) ), Move to the new boundary detected (S 268), and repeat from S 262 until tracking is completed (S 270). In the vicinity, eight neighbors are used in the same way as the eyeglass perimeter detection processing, and eight adjacent points are selected based on the current pixel as shown in Fig. 13, and the brightness of each pixel in a certain direction from the next point after the previous boundary Is extracted to detect the presence or absence of a boundary. For example, if the next boundary is detected clockwise from an adjacent point, the boundary of the lens can be tracked counterclockwise.

If the next boundary cannot be detected, the next boundary is estimated by curve approximation or the like from the perimeter data detected so far until no detection is performed continuously for a predetermined number of times (S272) (S27). 2 7 4) and repeat from S 2 64. When the detection is not performed continuously for a predetermined number of times (S272), the process returns to S254 and returns to the next level. The tracking process is repeated by selecting the start point of the boundary detection.

FIG. 14 shows the lens outer periphery tracking process when the above-described lens boundary tracking process is performed. As shown in the figure, with the left lens, for example, starting from the lens boundary detection start point Q3, the boundary L1 of the glasses is detected first, and then L2, L3, L4, ... And keep track of the boundaries. In the right lens, for example, starting from the lens boundary detection start point Q7, the boundary R1 of the glasses is detected first, and the boundaries are tracked clockwise from R2, R3, R4,…. Go.

Completion of tracking (S270) means, in principle, when returning to the starting point where tracking started, but here there is a possibility of making a U-turn because the outer periphery of the lens is interrupted on the way. It was detected that tracking was continued at a certain distance from the already detected outer circumference, and tracking was also completed in this case. When the tracking is completed, the missing portion of the lens outer circumference data is interpolated (S276). This is because not all of the outer periphery of the lens is clearly captured in the eyeglasses image, and the lens boundary detection processing that detects the boundary from the lens portion toward the lens periphery in S258 And the outer peripheral portion of the lens that could not be detected by the lens boundary tracking processing from S262 to S270. In the interpolation, based on the detected lens outer peripheral data, outer peripheral data of the defective portion is generated by, for example, curve approximation, and added to the lens outer peripheral data.

Next, the illegal projection is removed from the lens outer circumference data (S2778). This is to remove irregular projections generated on the outer periphery of the eyeglasses due to shadows and reflections. The detected lens outer circumference data is compared with the estimated lens outer circumference.If the distance is a predetermined distance or more, it is determined that the projection is incorrect, and the detected lens outer circumference data is estimated. Replaced with the lens outer circumference data.

Finally, the lens outer circumference data is smoothed (S280). This is to smooth out irregularities caused by unevenness in brightness at the boundary, for example, by replacing the coordinate data overnight with the average value of coordinate data in a certain range before and after. The smoothing may be replaced by a curve estimated from the lens outer circumference data by least squares approximation or the like.

FIG. 9 shows an example of the processing flow of the composite image display processing (S300). As shown in the figure, first, the designated lens color and the designated lens transmittance input on the mask processing screen are acquired (S302). Next, an image for synthesis in which the lens mask portion in the front image is colored with the designated lens color is generated (S304), and the front image, overhead image, image for synthesis, frame mask image, and lens mask image are displayed on the screen. (S306). Further, the CPU acquires the combined position information designated by the mouse (S308), and corrects the positions of the frame mask image and the lens mask image based on the acquired combined position information (S310).

Retrieves a prestored human image (S312), replaces the frame mask portion of the extracted human image with the image for synthesis (S312), and replaces the lens mask portion of the human image with the specified lens color and specified lens transmission. It is colored at a rate (S316), and the synthesized human image is displayed on the screen (S318).

The above processing is repeated from S302 when the "Resynthesis" button is pressed (S320), and terminated when the "End" button is pressed (S322).

FIG. 10 shows an example of the processing flow of the mask manual editing processing (S400). As shown in the figure, the mouse click position is obtained (S402), and a spline curve is generated based on the mouse click position information (S404). This is repeated from S402 until the "Setting" button is pressed (S406) When the “SET” button is pressed, the spline curve is registered in the eyeglass outer data or lens outer data (S408), and the lens mask image is created based on the registered eyeglass outer data and lens outer data. Is generated (S410), and a frame mask image is generated (S412). The generation of the lens mask image and the generation of the frame mask image are performed by the same processing as in S206 and S210 in the above-described mask automatic processing.

The above process is repeated from S402 until the "end" button is pressed (S414).

FIG. 11 shows an example of the processing flow of the data registration processing (S500). As shown in the figure, when the “register” button is pressed (S502), the input “basic item” data is obtained (S504), and the input “frame information” data is obtained ( S506), the input "other information" data is acquired (S508).

The front image, the bird's-eye view image, the composition image, the frame mask image, and the lens mask image are registered together with the spectacles specifications (S510).

The above processing is repeated from S502 until the "Close" button is pressed (S512).

Next, processing for selling glasses using the data registered by the glasses image registration system of the present invention will be described.

As shown in Fig. 1, the glasses image registration system 1 is connected to the glasses sales system 300 via a network, and the data generated by the glasses image registration system 1 is transmitted to the glasses sales system 300. And used to sell glasses.

The glasses sales system 300 executes a database server 310 for recording the specifications of the megane received from the computer 200, various power data, image data, etc., and an application program for selling glasses. A Prevent external intrusion when connecting the application server 320, the web server 330 that provides a user interface function with consumers, and the glasses sales system 300 to the Internet For firefighters, it consists of 340.

The consumer can use the Web browser to access the Web server 330 of this system, and order glasses based on the information stored in the database server 310. When the customer selects the desired glasses on the web server 330, an appearance image of the glasses is presented. In addition, by transmitting a face image taken by a consumer with a digital camera or the like, the application server 320 presents an image combined with a desired image for combining glasses.

The processing of combining the image for combining the glasses with the image of the consumer's face can be realized by the same processing as that from S312 to S318 in the synthetic image display processing (S300) described above. .

As a result, consumers can select and order glasses that suit themselves while staying at home, and dealers can send glasses directly from the manufacturing department without stocking them. In addition, by connecting online with the glasses image registration system 1 via a network, it is possible to realize rapid processing such as immediately selling megane prototypes at a manufacturing site, and to reduce the variety of glasses of various types. It can be provided at a price.

In the above embodiment, the glasses image registration system is configured such that the overhead image capturing device 100 that captures an overhead image for presenting the appearance of the glasses and the front image capturing device 1 that captures a front image for combining with a human image. However, the invention of the present application is not limited to this, and the torso 30 in the photographing box 11 of the front image photographing device 10, the glasses pedestal 32, and each panel are provided. Overhead image capture device with removable gray processing etc. May also be used.

In the above embodiment, the computer 200 has been described as being provided separately from the front image processing apparatus 100 or the overhead image capturing apparatus 100. However, the present invention is not limited to this. For example, a microcomputer may be incorporated in the image photographing apparatus 100 or the bird's-eye view image photographing apparatus 100 so as to execute the image processing function and the communication function with the glasses selling system. This makes it easy to install the eyeglass image registration system at the manufacturing site, enables communication with the eyeglass sales system by connecting the network directly to the imaging device, and builds an online eyeglass sales system.

In the above embodiment, the computer of the eyeglass image registration system and the eyeglass sales system are described as simply connected to a network. However, only when a private line or a public telephone line is used for private connection. It is needless to say that the connection is not limited, and the connection during this time may be communicated via the Internet.

In the above embodiment, the imaging means of the front image capturing apparatus 10 and the overhead image capturing apparatus 100 has been described as using a digital camera. However, the present invention is not limited to this. A television camera may be used, and the video signal may be captured by a video capture board provided in the computer 200 and taken into a digital still image as a digital still image. Thus, since a very small CCD camera can be used as the camera head, the front image capturing device 10 and the overhead image capturing device 100 can be further reduced in size.

In the above embodiment, the lighting means of the glasses image registration system has been described as using a fluorescent lamp. However, the present invention is not limited to this, and may be any range within a range that does not affect the color tone of the glasses. Like a lighting lamp May be used.

Further, in the above embodiment, the circular illumination lamp is described as being used, but the present invention is not necessarily limited to this, and a plurality of rod-shaped or egg-shaped illumination lamps may be provided. . As a result, the light is similarly illuminated from a wide range, and a shadow image is less likely to be formed, and a glasses image with less disturbance is obtained.

Note that the illumination lamp does not necessarily need to be a lamp that is continuously lit, and a flash or the like that emits light in synchronization with imaging by the imaging unit may be used, and the effects of the present invention are achieved.

In the above embodiment, in order to uniformly illuminate the glasses to be photographed from all directions, to equalize the brightness of the upper and lower portions of the glasses fixing member, and to suppress various disturbances from being reflected in the imaging means, Although it has been described that the inner surface is subjected to gray processing and gradation processing, the present invention is not necessarily limited to this, and the lighting means is dispersed and arranged, and the emission intensity is adjusted, and a reduction is provided by providing a cover or the like. It may be made to emit light, and the effect of the present invention is exhibited.

In the above embodiment, the description of the pre-processing of the front image targeted for the eyeglass boundary detection processing, the eyeglass boundary tracking processing, the lens boundary detection processing, and the lens boundary tracking processing is omitted. Adding shading correction processing to remove uneven brightness, adding smoothing processing to remove noise, and adding sharpening processing to enhance the outer contour of glasses and lenses. desirable. As a result, the mask automatic processing can be performed more stably.

In the above embodiment, the description has been made assuming that the eyeglasses boundary tracking processing and the lens boundary tracking processing are performed after the eyeglasses boundary detection processing and the lens boundary detection processing, respectively. However, the present invention is not limited to this. The pitch at which the lens boundary detection process is performed is made finer, and the outer periphery of the glasses and the outer periphery of the lens are determined by the outer periphery interpolation process and the outer periphery interpolation process of the eyeglasses and the lens boundaries. In the above embodiment, the eyeglass boundary detection processing in the eyeglass circumference detection processing and the lens boundary detection processing in the lens circumference detection processing take out the luminance of each pixel on the line and extract points having a luminance change equal to or more than a predetermined value. However, the present invention is not limited to this. For example, the luminance distribution (histogram) of the entire image or a specific portion is obtained, and the background portion is determined by a modal method, a sunset filter method, a discriminant analysis method, or the like. The boundary may be detected by calculating the threshold of the brightness level that separates the lens and the glasses or the lens from the surroundings. Initially attempts to boundary detection seeking histogram of a wide range, and a histogram narrow down from the obtained edge information, obtaining a more strict threshold, it may be used a hierarchical threshold setting method. This makes it possible to detect the boundary of an image with low contrast that is difficult to detect. In addition, for example, a method of sequentially performing differential processing, threshold processing, and thinning processing to obtain an outer peripheral contour may be employed.

In the above-described embodiment, the description has been given assuming that the outer circumference detection processing of the glasses and the outer circumference detection processing of the lens are performed by a single processing regardless of the type of the glasses. The outer circumference of the glasses or the outer circumference of the lens may be obtained by utilizing the model information of the glasses.

For example, as shown in Fig. 35, on the "Mask processing" display screen, there is a "Recreate individual mask" section with a model selection button for selecting the model of the glasses, and the mask for which the operating system is automatically generated cannot be changed. When deemed appropriate, By pressing the model selection button, information registered for each model may be extracted and the mask may be regenerated.

This is because, for example, in S120 of the processing flow of FIG. 5, it is determined whether or not the model selection button has been pressed, and if the model selection button has been pressed, it is registered in advance for each type of glasses. The information may be extracted, the threshold value of the luminance change in the eyeglasses boundary detection processing and the lens boundary detection processing may be changed, and the process may return to S200 and restart the mask automatic processing. Further, the mask automatic processing may be restarted by changing the detection range of the outer circumference of the glasses or the outer circumference of the lens based on information registered for each model of the glasses.

In this way, the outer periphery of the lens and the outer periphery of the lens can be detected more strictly in accordance with the model of each eyeglass, so even if automatic mask generation fails, the operator can automatically select the model selection button. Pressing can easily generate a mask.

In the eyeglass boundary detection process and the lens boundary detection process, when the method of detecting the boundary by the luminance level is adopted, the threshold of the luminance level may be changed depending on the type of the glasses, and other methods are employed. In such a case, the judgment method and various judgment criteria may be changed depending on the type of glasses.

In the above-described embodiment, the manual mask editing is described as editing the outer peripheral data from a point designated by the mouse. However, by dragging a part of the outer peripheral data generated by the automatic mask processing with the mouse, the outer peripheral data is edited. A function to correct the evening may be provided.

In the above embodiment, the overnight registration processing has been described as being performed in the computer 200. However, the registration may be directly performed in the database server 310 of the glasses sales system 300 via a network. Say things Not even.

In the above embodiment, the data registration process is described assuming that the registrant inputs basic items such as a product code and a product number, frame information, and other information. A reading unit may be provided so that a part or the front of the information is automatically input. The label may be read by, for example, providing the computer 200 with a 0 C R function or a bar code reading function to read characters or bar codes written on the label attached to the glasses. Alternatively, data on the frame shape or lens shape stored in advance may be extracted from the information read by the level reading means, and based on the data, the outer circumference detection processing for the glasses and the outer circumference detection processing may be performed. As a result, the detection range can be narrowed according to the shape of the glasses and the shape of the lens, and the boundary detection processing can be performed according to the shape, so that more reliable mask automatic processing can be performed. Industrial applicability

As described above, according to the present invention, a photographing means for photographing an image of glasses, a synthesizing image generating means for generating a synthesizing image for synthesizing a human image from an image photographed by the photographing means, Since there is provided a registration means for registering the composition image generated by the composition image generation means together with the specifications of the glasses, an image data which can be combined with a human image no matter when, where, and by whom. Can be obtained quickly at low cost.

Claims

The scope of the claims

1. Photographing means for photographing the images of the glasses, composition image producing means for producing a composition image for composition with a human image from the image photographed by the photography means, and production by the composition image producing means Registering means for registering the synthesized image together with the specifications of the glasses, and a glasses image registration system.

2. The photographing means has an opening / closing door, a photographing box for accommodating glasses for photographing, a lighting means for illuminating the glasses contained in the photographing box, and glasses for accommodating the glasses. Claims 1 characterized by comprising: an image pickup means for obtaining digital image data by photographing an image of a photographer, wherein glasses for photographing are housed in the photographing box, and the door is closed to photograph a glasses image. Glasses image registration system.

3. The glasses image registration system according to claim 1, wherein the photographing unit includes a front image photographing unit that photographs a front image of the glasses.

4. The Megane image registration system according to any one of claims 1 to 3, wherein the photographing means includes a bird's-eye image photographing means for photographing a bird's-eye image of glasses.

5. The image-for-synthesis generation means includes: a mask for generating a mask including a frame mask image specifying a frame portion of the captured glasses image; and a lens mask image specifying a lens portion of the captured glasses image. The glasses image registration system according to any one of claims 1 to 4, further comprising a generation unit.

6. The mask generation unit includes: a pair of spectacles perimeter detecting unit configured to detect a pair of spectacles perimeters of the captured spectacle image; Lens outer circumference detecting means for detecting the outer circumference of

Generating the frame mask image from the outer circumference of the glasses detected by the outer circumference detecting means of the glasses and the outer circumference of the lens detected by the outer circumference detecting means of the lens;

6. The eyeglass image registration system according to claim 5, further comprising a function of generating the lens mask image from an outer periphery of the lens detected by the lens outer periphery detection unit.

7. The spectacles perimeter detecting means includes spectacles boundary detecting means for detecting a boundary of spectacles from a background portion of the photographed spectacles image toward the spectacles portion, wherein the lens perimeter detecting means includes: A lens section estimating section for estimating a lens section from the detected outer periphery of the glasses; and a lens boundary detecting section for detecting a lens boundary from the lens section estimated by the lens section estimating section toward the lens peripheral section. The eyeglass image registration system according to claim 6, wherein the eyeglass image registration system includes:

8. The glasses outer circumference detecting means includes glasses boundary tracking means for sequentially detecting and tracking the boundaries of neighboring glasses starting from the glasses boundaries detected by the glasses boundary detecting means,

8. The method according to claim 7, wherein the lens outer circumference detecting unit includes a lens boundary tracking unit that sequentially detects and tracks the boundaries of neighboring lenses starting from the lens boundary detected by the lens boundary detecting unit. Megane image registration system described in.

9. The glasses outer circumference detecting means includes glasses outer circumference interpolating means for interpolating a missing portion of the detected outer circumference of the glasses,

The lens outer circumference detecting means includes a lens outer circumference interpolating means for interpolating a defective part of the detected outer circumference of the lens, wherein 9. The glasses image registration system according to claim 8.

10. The spectacles periphery detecting means includes spectacles periphery irregular protrusion removing means for removing the detected irregular protrusions on the periphery of the spectacles,

10. The lens outer periphery detecting unit according to claim 6, wherein the detected lens outer periphery detecting unit includes a lens outer periphery irregular protrusion removing unit that removes an illegal protrusion on the detected outer periphery of the lens. Glasses image registration system.

1 1. The mask generation means includes glasses type information obtaining means for obtaining glasses type information,

The eyeglass outer circumference detecting means and the lens outer circumference detecting means have a function of detecting the outer circumference of the eyeglasses and / or the outer circumference of the lens by utilizing the model information of the eyeglasses obtained by the eyeglass model information obtaining means. The megane image registration system according to any one of claims 6 to 10.

12. The mask generation unit includes a left and right comparison correction unit that compares and corrects left and right images of the generated frame mask image and / or the generated lens mask image. The glasses image registration system according to any one of claims 5 to 11.

13. The glasses according to claim 5, wherein the mask generating unit includes a mask manual editing unit that edits a mask from point data specified by a pointing device. 14. Image registration system.

14. The image generation means for synthesis includes a synthesis display means for synthesizing and displaying a pre-registered human image and the taken glasses image, and the composition display means is characterized in that: The glasses image registration system according to any of the above.

15. The combination display means includes a lens color of a glasses image to be combined with a human image. The eyeglasses image registration system according to claim 14, further comprising a function of designating a transmittance and a transmittance.

16. The glasses image registration system according to claim 14, wherein the combination display unit has a function of adjusting a position of a glasses image to be combined with a human image. 16.

17. The glasses according to any one of claims 1 to 16, wherein the registration unit includes a label reading unit that reads a label provided on a lens unit of the captured glasses. Image registration system.

18. The glasses image registration system according to any one of claims 1 to 17, further comprising a communication unit that transmits data registered by the registration unit to the glasses sales system.

1 9. A step of taking an image of the glasses, a step of generating a combining image for combining with the human image from the taken image, and registering the generated combining image together with the specifications of the glasses. A method of registering eyeglass images.

20. The step of generating the image for synthesis includes a step of generating a mask including a frame mask image specifying a frame portion of the captured glasses and a lens mask image specifying a lens portion of the captured glasses. The eyeglasses image registration method according to claim 19, comprising a step.

21. The step of generating the mask includes the steps of: detecting an outer circumference of the glasses of the captured glasses image; detecting an outer circumference of the lenses of the captured glasses image; A step of generating the frame mask image from an outer periphery and the detected outer periphery of the lens; and a step of generating the lens mask image from the detected outer periphery of the lens. Glasses image described in 20 Registration method.

22. The step of detecting the outer periphery of the glasses includes a step of detecting a boundary of the glasses from a background portion of the captured glasses image toward the glasses portion.

Detecting the outer circumference of the lens includes: estimating a lens portion from the outer circumference of the glasses detected by the step of detecting the outer circumference of the glasses; and 22. The eyeglass image registration method according to claim 21, further comprising: detecting a boundary of the lens.

23. The step of detecting the outer periphery of the glasses includes a step of sequentially detecting and tracking the boundaries of neighboring glasses starting from the boundaries detected by the steps of detecting the boundaries of the glasses,

The step of detecting the outer circumference of the lens includes the step of sequentially detecting and tracking the boundary of a nearby lens starting from the boundary detected by the step of detecting the boundary of the lens. The glasses image registration method described in Item 22.

24. The step of detecting the outer circumference of the glasses includes a step of interpolating a defective portion of the outer circumference of the detected glasses,

The glasses according to any one of claims 21 to 23, wherein the step of detecting the outer periphery of the lens includes a step of interpolating a defective portion of the detected outer periphery of the lens. Image registration method.

25. The step of detecting the outer periphery of the glasses includes a step of removing an illegal projection on the outer periphery of the detected glasses.

The step of detecting the outer periphery of the lens includes a step of removing an illegal projection on the outer periphery of the detected lens.

21. The eyeglass image registration method according to any one of claims 21 to 24.

26. The step of generating the mask includes a step of acquiring model information of glasses.

The step of detecting the outer circumference of the glasses and / or the step of detecting the outer circumference of the lens includes the step of detecting the outer circumference of the glasses and / or the outer circumference of the lens using the acquired model information of the glasses. 26. The megane image registration system according to claim 21, characterized in that:

27. The step of generating the mask includes a step of comparing and correcting left and right images of the generated frame mask image and / or the generated lens mask image. An eyeglass image registration method according to any one of claims 20 to 26.

28. The method according to any one of claims 20 to 27, wherein the step of generating the mask includes a step of editing the mask from point data specified by a pointing device. Glasses image registration method.

29. The method according to claim 19, wherein the step of generating the image for combination includes a step of combining and displaying a pre-registered human image and the captured glasses image. 28 The method for registering a magne image described in any of 8 above

30. The eyeglasses image registration method according to claim 29, wherein the displaying step includes a step of designating a color and a transmittance of a lens of the eyeglasses image to be combined with the human image.

31. The eyeglasses image registration method according to claim 29, wherein the displaying includes adjusting a position of the eyeglasses image to be combined with the human image.

3 2. The step of registering is set in a lens portion of the glasses that have been photographed. 31. The eyeglasses image registration method according to claim 19, further comprising a step of reading an applied label.

33. The glasses image registration method according to any one of claims 19 to 32, further comprising a step of transmitting data registered in the registration step to a glasses sales system.