WO2010015881A1

WO2010015881A1 - Position encoding using an invisible pattern data matrix

Info

Publication number: WO2010015881A1
Application number: PCT/IB2008/053145
Authority: WO
Inventors: Mehdy Davary Dolat Abady
Original assignee: Mehdy Davary Dolat Abady
Priority date: 2008-08-05
Filing date: 2008-08-05
Publication date: 2010-02-11

Abstract

The present invention is embodied in: A pattern said MD222 using Data Matrix bar code symbols printed with UV/IR invisible ink for encoding machine-readable information of a PC-screen pixel positions data onto a printable-surface. A Digital-pen which combines an infrared light module with a unique two dimensional CCD image sensor for reading an invisible Data Matrix bar code in real-time. The MD222 encodes the absolute X, Y positions of pixels of a PC-screen into thousands of tiny, individual Data Matrix bar code symbols. Each symbol of MD222 pattern consists of a small 12x12 Data Matrix, as small as 1.35 mm or less, depending on the resolution of the printing that is used. Depending directly on their physical position on the MD222 and so on the printable-surface, each Data Matrix symbol represents a single pixel and stores only the absolute X, Y position of that pixel of PC-screen. A 1280 x 1024 matrix (MD222 pattern) of these Data Matrix bar code symbols can be used to encode absolute X, Y positions of all pixels existing in a XVGA PC-screen.

Description

POSITION ENCODING USING AN INVISIBLE PATTERN DATA MATRIX

[1] This invention uses an IR invisible ink to print a pattern of 2D Data Matrix (12x12) bar codes (one per pixel) on a printable-surface (opaque or transparent). Each Data Matrix (12x12) can store max 10 numeric characters. We use only 8 numeric characters to store (X, Y) position of each pixel. The first four numeric characters are the real X position of the Data Matrix on the printable- surface and the second 4 numeric characters are the real Y position of it. Since each Data Matrix is printed in the exact position of the pixel it represents so we have a matrix of very small (0.8 mm in 1000 DPI and 1.35 mm in 600 DPI) printed Data Matrixes on the printable- surface which is equivalent to the Digital Screen's pixels.

[2] (FigureOl)

[3] To read the (X, Y) values from each Data Matrix in reel-time we use a wireless pen- like CCD Infrared camera device (100 frame per sec.) to send the captured images directly to the connected PC. Then each image is analyzed by Image Processing Software to extract the stored data (8 numeric characters) from the Data Matrix in the image. As explained earlier in the text (X, Y) values are directly readable from these 8 numeric characters; the first four numeric characters give the X position and the second 4 numeric characters give Y position. These values are used to set mouse pointer position on the Digital Display.

[4] Call with AX = 4

[5] CX = x coordinate

[6] DX = y coordinate

[7] Returns: Nothing

[8] To have much more flexibility we combine our pen-like CCD Infrared camera with a

RF wire less mouse. So the pen can be used as an ordinary mouse from up to 30 m and in the same time can be used as Interactive Whiteboard stylus to write and/or interact with the projected image (from a PC-screen) on the printable- surface Interactive Whiteboard.

[9] The pen-like camera can also be used to read any data stored in a Data Matrix which is printed with UV/IR invisible ink over a normal text or image (the visible images must be printed using a non-carbon ink) to provide supplementary but hidden data (such as hyper links, audio, answers to quizzes and etc.) for both readers and connected PC.

Technical Field

[10] The invention generally relates to computer input Human Interface Device for

Whiteboard with mouse control. A method and an apparatus for decoding encoded Data Matrix bar code symbols to a flexible or rigid printed media.

[11] The present invention relates to:

[12] A method to print, as a transparent second layer, over a flexible or rigid printed media said "printable- surface". Using this method we associate, on a one-to-one basis, the X, Y position of every pixel of a digital display said "PC-screen", by encoding them to a pattern of invisible, machine-readable, small 12x12 Data Matrix bar codes said "MD222" where the physical location of each individual Data Matrix of MD222 on the printable-surface corresponds exactly to the real position of the very pixel of the PC- screen that it represents.

[13] An apparatus said "Digital-pen", used as HID mouse controller, without having an integral image processing means on board, comprising an unique infrared two- dimensional CCD image sensor with high sample density to omni-directionally capture in real time, a two-dimensional image of an invisible machine-readable 12x12 Data Matrix, representing a pixel of the PC-screen, and transferring the image to the connected PC to process for outputting the X, Y position of that pixel and use this information to Set Mouse Pointer to Location with X, Y position. Background Art

[14] Anoto: Anoto functionality is regarded as the standard for digital pen and paper. It comprises three main parts, the paper, the pen and the service infrastructure.

[15] Various types of paper are printed with a unique background pattern that forms part of a huge "map", the total area of which is larger than the surface of Europe and Asia combined. Computer analysis of the pattern allows the absolute position of a digital pen on the pattern to be calculated with great accuracy.

[16] (FigureO2)

[17] 2D bar code: Two dimensional bar codes are advantageous because they can hold binary as well as alphanumeric data, and usually include an error correction function. In addition, a two dimensional bar code is significantly more efficient than a traditional one dimensional bar code, being able to store much more information in a smaller space.

[18] Data Matrix is an industry standard two dimensional bar code convention that includes both error detection and correction. Each Data Matrix symbol consists of data regions that contain nominally square modules set out in a rectangular array. A dark module is a binary 1 and a light module is a binary 0. There is no specified minimum or maximum for the X or Y dimension. The data region is surrounded by a finder pattern, a perimeter to the data region that is one module wide that is surrounded by a quite zone on all four sides of the symbol. Two adjacent sides are solid dark lines used primarily to define physical size, orientation, and symbol distortion. The two opposite sides consist of alternating dark and light modules. These are used primarily to define the cell structure but also assist in determining physical size and distortion. One popular form of Data Matrix symbology is known as ECC 200, which uses Reed- Solomon error correction.

[19] The compact size of the Data Matrix symbols makes them ideal for use in direct part marking. In direct part marking, the Data Matrix symbols are imprinted directly on paper being marked as opposed to being imprinted on labels that are applied to the part. Direct part marking presents challenges caused by the fact that the Data Matrix symbols may not necessarily be imprinted upon a perfectly flat surface or upon a surface with uniform reflectivity properties. In addition, the data modules may be imprinted as circular dots instead of square modules, such as in the case where metallic parts are marked by dot-peening the parts. Disclosure of Invention Advantageous Effects

[20] Very Low-Cost Whiteboard: Since we print the invisible Data Matrix pattern on a printable surface, the Interactive Whiteboard can be reproduced at a very Low-Cost rate.

[21] Higher Resolution: Although the maximum common display resolution for a portable projector is 1080p (1920x1080 pixels), but by using a Data Matrix (12x12) which stores up to ten numeric characters, in theory one can make a square Interactive Whiteboard with up to 100 OOO pixels per side (width and height).

[22] Smaller size and higher definition of Whiteboard: The printed size of a Data

Matrix of 12x12 is 1.35 mm in 600DPI and 0.8 mm in IOOODPI which is smaller than Anoto code standard that is 1.8 mm.

[23] Shorter time for analyse and position calculation (in Real-time): Since we store the exact (X, Y) position of each pixel in its equivalent Data Matrix bar code printed on the printable-surface, after image processing no calculation or algorithm is needed to calculate the (X, Y) values as it is necessary in Anoto.

[24] No need for calibration: The physical size of the projection surface is prefixed by the print size of each Data Matrix and is directly related to the DPI used to print the pattern. For example the size of projection area for a XGA (1024x768 pixels) is calculated as following: 1 Data Matrix (Code Size: 12, Module Size: 2, Quiet Zone: 2, Size : 32x32 pixels, DPI : 600 Size : 1.35 mm). Projection area size: 1024*1.35=1382 mm x 768*1.35=1036 mm. So the calibration is some how automatic because we project the Digital Image on a predefined surface according to the resolution of the video projector and PC.

[25] (Figure03)

[26] Low-Cost Digital-pen: Most of the digital pens used with Anoto code are equipped with intern Processor, Memory and normal Ink system. In our case we reduce the production price by using only the CCD camera, rechargeable battery and wireless device. The image processing is down directly via software on the connected PC.

[27] (FigureO4)

[28] Very white projection surface: Due to the fact of using an IR invisible ink to print the Data Matrix pattern on the printable- surface which is used as Interactive Whiteboard and so as projection surface, the Whiteboard can be really white. In Anoto standard, the code is printed with a very light grey ink which is less suitable to be used as a projection screen. Description of Drawings

[29] FigureOl: A MD222 pattern of 1280x1024 Data Matrix bar codes represents a standard digital display with SXGA (1280x1024 pixels).

[30] FigureO2: Anoto pattern

[31] FigureO3: The size of projection area in millimetres for a XGA PC-screen

(1024x768 pixels)

[32] FigureO4: Comparison of Anoto Digital-pen and MD222 Digital-pen

[33] FigureO5: The position of a series of printable-surfaces which will be used as an interactive whiteboard in Puzzle form

[34] FigureO6: Each puzzle piece of printable- surfaces such as "al" is a bitmap image of

20' 480 Data Matrix

[35] FigureO7: MD222 areas can be printed on a document as a second layer over its normal printing process using an UV/IR invisible ink Industrial Applicability

[36] Interactive Whiteboards: The main use of the MD222 technology will be to create a series of printable-surfaces which will be used as an interactive whiteboard in Puzzle form.

[37] (Figure05)

[38] Each puzzle piece (for example al) is a bitmap image of 160 x 128 = 20'480 2D code bars (12x12 Data Matrix, 32x32 pixels) with pixel dimension X of 160 x 32 = 5' 120 and Y of 128 x 32 = 4'096.

[39] (FigureO6)

[40] Hypertext links in traditional paper-based publication: Most printed surfaces are silent and static; they do not emit sounds or display visual information relating to the objects depicted on the surface. Examples of such surfaces include the pages of books, magazines, newspapers, board games and displays.

[41] Computer systems and programs are known to provide that content on a display.

Some computer programs highlight words as they are read. Other computer systems and programs allow a user to click on a word or image to provide additional information relating to the content. These conventional systems, however, are not part of the actual print medium and they lack the look and feel of the print medium.

[42] MD222 technology also allows to generate invisible hyperlinks and anchor in a printed media, an ordinary documents such as books, commercial brochures, newspapers, journals and manuals; to carry thousands of characters of information (such as URL, translations, synonyms, footnotes, hyperlinks, price information and etc.) hidden in these invisible Data Matrix patterns printed with UV/IR invisible ink as a second layer over any conventional text and graphic design elements. Being invisible their presence will go completely unnoticed.

[43] MD222 areas can be printed on a document as a second layer over its normal printing process using an UV/IR invisible ink. The information to be put in the MD222 is encoded as a sequence of individual Data Matrix bar code elements, and these can be printed either directly by the encoding software (for instance, by computer laser printer) or via a conventional printing process, such as offset.

[44] (FigureO7)

Claims

[I] An apparatus, said Digital-pen for reading in real-time a pattern of invisible 12x12 Data Matrix bar code symbols, said "MD222 pattern" having a two dimensional image capture module for capturing two dimensional images, said image capture module having a field of view defining an imaged area.

[2] A Digital-pen in accordance with claim 1 further comprises a program executed on said connected PC for processing the images captured by said image sensor starting substantially in the centre of the field of view and working outward, whereby a Data Matrix symbol close to the centre of said field of view is decoded first. [3] A Digital-pen in accordance with claim 1, wherein said reader further comprises a sensor which produces an electrical signal in response to reflected infrared light from said scan area. [4] A Digital-pen in accordance with claim 1 further comprises a Bluetooth or RF wireless connection means for transferring data from the Digital-pen to the connected PC. [5] A Digital-pen in accordance with claim 1 further comprises a rechargeable battery on board. [6] A Digital-pen in accordance with claim 1 functions in two modes: as an ordinary wireless mouse or in absolute mode, where the mouse pointer's position on the

PC-screen directly corresponds with Digital-pen's location on the interactive whiteboard. [7] A pattern of invisible 12x12 Data Matrix bar code symbols, said "MD222 pattern" which has exactly the same number of "12x12 Data Matrix bar codes" as the number of "the pixels on the PC-screen". [8] A MD222 pattern as claimed in claim 7 is printed over a flexible or rigid printed media said "printable-surface". [9] A printable-surface as claimed in claim 7 characterized in that said surfaces are made of an organic and/or inorganic material, metal, plastic, paper, wood. [10] A printable-surface as claimed in claim 7 can be already white such as a whiteboard or colour printed media such as newspaper, books, etc.

[I I] A MD222 pattern as claimed in claim 7 comprising: a two-dimensional matrix (1280x1024) of 12x12 Data Matrix bar code symbols.

[12] A MD222 pattern as claimed in claim 7 wherein said invisible ink is an infrared absorbing ink, and wherein said invisible radiation is infrared light.

[13] A Data Matrix as claimed in claim 7 is a Data Matrix bar code with the following parameters: Code Size: 12 (four equal sides), Module Size: 2, Quiet Zone: 2, Size in pixels: 32x32 pixels.

[14] A Data Matrix as claimed in claim 7 should be printed at 600 DPI (Size on the printable- surface : 1.35 mm) or at 1000 DPI (Size on the printable-surface : 0.80 mm)

[15] A method to print the MD222 pattern over a printable-surface.

[16] The method of claim 15, further comprising capturing data using a Digital-pen by detecting at least one of the Data Matrix bar codes of the MD222 pattern printed over the printable-surface when the Digital-pen is moved over the MD222 pattern.

[17] The method of claim 15, further comprising determining the position of the

Digital-pen, thereby capturing the data from each Data Matrix of MD222.

[18] The method of claim 15, wherein the data captured from a Data Matrix of

MD222 pattern printed over a printable-surface is directly used to output the X, Y position of the very pixel of the digital display said "PC-screen", which is represented by that Data Matrix.

[19] The X, Y position of each pixel of PC-screen is stored by using 9 alphanumeric characters (an X followed by 8 numeric characters) in a Data Matrix: Xxxxxyyyy, where X is a character used for programming purposes only and the xxxx and yyyy are numeric characters which represent the X and Y position of the pixel on the PC-screen. So for example X12791023 is used to represent the pixel on the extreme right bottom of a 1280x1024 PC-screen where X and Y position of that pixel are X= 1279 and Y= 1023.