WO1998022910A1

WO1998022910A1 - Method of and apparatus for displaying an image of a document on a display

Info

Publication number: WO1998022910A1
Application number: PCT/GB1997/003126
Authority: WO
Inventors: Paul Booth
Original assignee: Geoworks Corporation
Priority date: 1996-11-22
Filing date: 1997-11-21
Publication date: 1998-05-28
Also published as: GB9624281D0

Abstract

Displaying a facsimile message with bit-mapped text on a small screen computer such as a personal digital assistant is difficult because of the limited screen size. This invention teaches a scrollable text selection window extending the full width of a miniaturised and therefore illegible image of the received facsimile; the height of the text selection window is such that only a single line of bit-mapped text is covered by it. The text selection window is divided into four segments of equal width; the bit-mapped text in the far left segment is magnified to legibility and then displayed as a first row in a viewing window. The next segment, moving rightwards over the line of text, is also subject to magnification and placed in a row of the viewing window below the highest row. The process is repeated for all four segments; the original line is therefore broken into four sections which are displayed as four rows in the viewing window enabling the line to be easily read in the viewing window.

Description

Method of and apparatus for displaying an image of a document on a display

Field of the Invention

This invention relates to a method of and apparatus for displaying an image of a document on a display. In particular, it relates to a method of and apparatus for displaying an image of a bit-mapped text document on a display, in which the text document has been received from a physically remote transmitting device, such as a remote facsimile machine.

Description of the Prior Art

Many computers and personal communicators can be used with fax-modems that enable facsimile or other messages to be received. However, the small screen size and resolution of a typical display screen, particularly a portable computer or personal communicator display screen, can make reading messages difficult. For example, take a facsimile message sent from a conventional facsimile machine to a portable computer: The original message would be on A4 size paper or similar. The screen size of the portable computer is far less than A4, so that it is impossible to display the entire message at its original size. The problem can be put as one of being unable to see the necessary detail if one sees any significant part of the whole.

One conventional approach is to display a portion of the bit-mapped image at a reduced magnification, with the user being able to select a region of the displayed bit-mapped image for extra magnification. Often, the user can call up a window that acts as if it is a "magnifying glass", magnifying the text underlying the window so that it can be read. In this way, a user can read words falling within a typically rectangular frame and can scroll the magnifying glass over the displayed portion of the text so that the text can be viewed. One disadvantage with this approach is that it is inconvenient to the user since the user must carefully control the scrolling of the magnifying glass to read even a single line of text. Returning to the start of the next line is often difficult. This is particularly the case for computers or telephones with small screens, such as personal digital assistants or mobile communicators.

Another conventional approach is to attempt to recognise text represented by bitmapped images using character recognition software. Once the bit-mapped text is converted to data, then it is easy to display the entire message by enabling the user to scroll through it. However, this approach generally loses most or all of the format of the original image, which may be inconvenient. More significantly, character recognition software is not sufficiently accurate for this task, is expensive, slow in operation and requires significant processing capabilities that may be beyond simpler personal digital assistants.

Further reference may be made to GB 2298102 A to Samsung Aerospace Industries Ltd. This specification discloses an overhead projector apparatus. It does not however deal with the problem of displaying a bit-mapped text document on a display, in which the text document has been received from a physically remote transmitting device.

Statement of the Invention

In accordance with the present invention, a method of displaying an image of a document on a display screen comprises the steps of:

[ 1 ] receiving, from a remote transmitting device, a bit-mapped image of a document comprising text; [2] dividing a whole or part of a line of text of the document into segments;

[3] processing the segments so that text in each segment can be displayed at a legible size and re-arranging segments which are positioned adjacently in a line or a part of a line into two or more rows and [4] displaying the re-arranged segments on the display screen.

Using this approach, a bit mapped image of text, such as a facsimile image, can be displayed on a computer screen so that the text can be easily read.

For example, a single line of bit-mapped text could be divided into four segments of equal width, with the bit-mapped text from each segment being processed to a legible size and then displayed within a window divided into four rows, with the first row containing the processed text from the segment from the far left of the unmagnified line, the next row down containing the processed text from the next segment along and so forth.

It is not a requirement of this invention that the re-arrangement be done after the processing step: it is entirely equivalent to re-arrange and then process.

Preferably, the following step occurs after step [2]: displaying either some or all of the image either (i) at a magnification at which it is not readily legible or (ii) as a picture representative of the image. This approach allows the user to select the data which is to be extracted and processed from the document image. For example, where that data corresponds to an entire line of bit-mapped text, then the user can select which particular line is to be extracted and re-assembled in magnified form. This can be achieved by providing a scrollable window (a "text selection window") or marker which can be moved to overlie the low magnification, illegible, portion of the image to be re-assembled. Alternatively, the low magnification, illegible image can be scrolled past a fixed marker such as a window covering the entire width of the image, which overlies that portion of the image to be re-assembled in magnified form. It is convenient if only a single line of text is processed for re-assembly in magnified form. However, the height of characters, the line spacing etc. varies from document to document and may vary even within a document itself. It is therefore not possible to provide a fixed height narrow, rectangular text selection window [or other marker] for example, which will work with all documents. This problem can be solved by allowing the user to vary the height of such a window [or other marker]. For example, on the screen, the text selection window can be defined by a single line marking the centre of the short axis of the window; on screen buttons can then be provided which when selected either increase or decrease the height of the window. Another approach is to provide "grab handles" at some or all corners of the window; these can be dragged to enlarge or decrease the height of the window.

Yet another alternative is to provide pattern recognition which can determine the height of a line of text, by for example, looking at a pre-determined narrow region of the text selection window and decreasing the height of that window in incremental steps until an output from a pattern recognition engine indicates that it most closely matches the pattern associated with a single line of text, preferably with a small- amount of unfilled space above and below the line of text.

Alternatively, the window may be dispensed with entirely: the system automatically work out work out the positions of each line of text using a pattern recognition engine. This removes the need for manual selection via a text selection window. Either approach can be implemented by those skilled in the recognisor arts. Further, since many of the devices for which this invention will be most apt will anyway incorporate hand-writing recognition software, the extra burden this approach places on cost and performance is minimal. Some areas of the image may not be full width lines of text. Conveniently, the width of the window [or other marker] can be controlled by the user, for instance by dragging "grab handles" at the corners of the window until the appropriate width is reached.

Also, the window [or other marker] can be used to indicate, at the user's option, either a conventional "magnifying glass" region or the re-assembly and magnification approach of this invention. For example, if the image comprises both text and some pictures, then the user may wish to view the text using the method of the present invention, but switch to the conventional "magnifying glass" approach to view the picture. In either option, the magnified image portions may be arranged to appear within the same window. Further, on-screen control buttons can present the user with the option of selecting either mode.

In some devices, it may be impractical or undesirable to display the actual image, even in unmagnified form. This may be because, with the resolution available, the appearance of such an image is unattractive. Then, a picture merely representative of the image may be displayed and it would be against this picture that the user would place the text selection window or marker that indicated which portions were to be magnified. The picture would preferably have some detailing corresponding to the layout of the text in the image, although this is not essential.

In another aspect, an apparatus for displaying an image of a document on a screen comprises:

[ 1 ] a receiver for receiving, from a remote transmitting device, a bit-mapped image of a document comprising text;

[2] a processor programmed (i) to process the document by dividing a whole or part of a line of text into segments; (ii) to process the segments so that text in each segment can be displayed at a legible size and (iii) to re-arrange the segments which are positioned adjacently in a line or a part of a line into two or more rows so that their contents can be easily read and

[3] a display to display the re-arranged segments.

Brief Description of the Drawings

An embodiment of the invention will now be further described with reference to the accompanying drawings in which: -

Figure 1 is a schematic block diagram illustrating the major components of such an apparatus for displaying an image of a document on a screen in accordance with the present invention;

Figure 2 is a plan view of such an apparatus showing what might be displayed by such an apparatus.

Detailed Description

The invention will now be described with reference to a particular embodiment. It will be appreciated that this embodiment includes various features that are not essential to each and every implementation of the invention.

Referring now to Figure 1, a display means 2, comprising a LCD type display, is shown forming part of a computer indicated generally at 1. The computer comprises a translucent transducer pad 3 overlaying the display 2, together with a stylus 4. The pad 3 is an Indium Tin Oxide (ITO) device which is well known in this art. It is operable to supply a signal indicative of the position at which the stylus 4 touches the pad 3 and allows the user to easily and naturally input graphical information for storage and subsequent, although nearly immediate, display by the apparatus.

It will be appreciated that a variety of components, e.g. a power supply, is necessary for the functioning of the apparatus and will have to be present in the apparatus but these are not expressly detailed here in the interests of brevity and clarity.

The transducer ITO pad 3, also known as a digitiser, is shown schematically overlaying the LCD type display 2. The stylus 4 is touching the pad 3 and electrical signals indicative of the position (X Y co-ordinates) at which it contacts the pad 3 are supplied from the stylus 4 to an A to D converter 14 feeding a data path or system bus 10.

A CPU 12 is connected for bi-directional data flow to the bus 10, as is a RAM 17. A ROM 18 is also connected to bus 10, being programmed with the operating system for the apparatus. CPU 12 provides all control signals via bus 10. RAM 17 stores, inter alia, the data defining received bit-mapped facsimile messages. RAM 17 may be an EEPROM or other form of memory. CPU 12 may be a microprocessor such as an ARM RISC chip. In addition there is provided a serial data interface 19 connected to bus 10 for sending and receiving information. A display driver 11 is connected to bus 10 and drives the display 2.

In use, when the user receives a facsimile, data defining the bit-mapped facsimile image is sent via the serial interface 19 from a fax/modem [not shown] and is stored in RAM 17 or other storage medium. A reduced size version of the image, with much information lost, is displayed on the display 2 at 21. The text in facsimile image 21 is not legible. To read the text, the user selects the "Fax View" [TM] function from a drop down menu or other conventional system. When Fax View is selected, text selection window 22 appears on screen over a portion of the facsimile image 21. The user can move text selection window 22 around facsimile image 21 using the conventional 4-way movement controller 23. In addition, in a pen driven system, the user can simply drag the text selection window 22 to the desired location. The height of the text selection window 22 may be automatically varied to match the height of the text in the facsimile by a pattern recognition engine as described earlier, in those implementations where a recognisor engine is available. The details of the pattern recognition engine will be familiar to the skilled implementer. In addition, grab handles 24 are provided at each corner which enable the user to alter the height and width as necessary. Finally, marker 25 is provided that indicates the centre of the short axis of the text selection window 22; marker 26 is provided that indicates the centre of the long axis of the text selection window 22. The user can drag markers 25 and 26 to the appropriate position using either the conventional 4-way movement controller 23 or, where the computer is pen driven, by dragging marker 25 or 26 with the pen.

Text selection window 22 is divided into four equal width sections 27 A, 27B, 27 C and 27D. The bit maps in the segment of the facsimile image 21 which underlie each equal width section are processed by being subject to a simple magnification process which enlarges and maps the contents of each section 27A to 27D into a respective section of the viewing window 30; viewing window 30 is divided into four rows, 30A, 30B, 30C and 30D. Section 27 A of the facsimile image 21 is mapped into row 30A; section 27B of the facsimile image 21 is mapped into row 30B; section 27C of the facsimile image 21 is mapped into row 30C; section 27D of the facsimile image 21 is mapped into row 30D. In this way, an entire line of bit mapped text can be easily read in viewing window 30. To view the next line, the user simply selects the on-screen "return" key 31 if the computer is pen driven. Text selection window then automatically moves down the correct distance to overlie the next line of bit-mapped text. In some kinds of computer, for instance small personal organisers with integral keyboards, the actual "return" key is hit to move the text selection window to the next line down. In each case, the pattern recognition engine may be used to ensure that the window 22 is positioned at the optimum position to enable the next line of text to be read. Another approach to viewing the next line down is to simply manually re-position text selection window 22 using controller 23 or marker 26.

A further feature is that the viewing window can be used not only merely to display single lines of the original facsimile image 21 broken down into a number of rows as described above; in addition, it can be used to select an area of the image 21 to be viewed without being split into segments and then re-assembled. Hence, by selecting the magnifier option button 32, the viewing window 30 becomes a conventional "magnifying glass". In addition, the text selection window is shrunk down to a size which enables the whole of the image that it underlies to be displayed within the viewing window 30. This feature enables the user to switch quickly from the "Fax View" embodiment of this invention to a conventional "magnifying glass".

It is preferable if words in the facsimile image 21 are not chopped into two; this would happen if the boundary between adjacent sections 27 of text selection window 22 intersects words. The present embodiment addresses this problem by dynamically altering the width of each section 27 A, 27B, 27C and 27D so that no bit-mapped text is intersected in this way. A pattern recognition engine searches a narrow strip on the left hand border of each section 27 A, 27B, 27C and 27D for the presence of an underlying bit mapped text. If underlying text is indicated, the width of the relevant section is incrementally varied and the search of the narrow band repeated. The process continues until it is determined that no text underlies the narrow band. The process is then repeated for the next section along until all sections across the entire width of the line have been sized to ensure that no underlying word will be split into two when viewed in the viewing window 30.

Claims

1. A method of displaying an image of a document on a display screen comprising the steps of:

[ 1 ] receiving, from a remote transmitting device, a bit-mapped image of a document comprising text; [2] dividing a whole or part of a line of text of the document into segments; [3] processing the segments so that text in each segment can be displayed at a legible size and re-arranging segments which are positioned adjacently in a line or a part of a line into two or more rows and; [4] displaying the re-arranged segments on the display screen.

2. The method of Claim 1 wherein each segment is displayed within a viewing window divided into rows, with the first row of the viewing window containing processed text from the segment from the far left of the line, the next row down containing the processed text from the next segment along and subsequent rows down contain the processed text from adjacent segments along the line.

3. The method of Claim 2 wherein the user can select the text which is to be processed from the document image.

4. The method of Claim 1 wherein the following step occurs after step [2]: displaying either some or all of the image either (i) at a magnification at which it is not readily legible or (ii) as a picture representative of the image.

5. The method of Claim 4 wherein selecting the text is achieved using a movable selection window or cursor which can be moved over portions of the illegible image or the picture to the desired line to be viewed in the viewing window.

6. The method of Claim 5 wherein the height and/or width of the movable selection window can be varied.

7. The method of Claim 6 wherein the output of a pattern recognition engine is operable to control the height of the movable selection window so that it approximates to the height of a line of text.

8. The method of Claim 7 wherein the pattern recognition engine looks at a predetermined narrow region in the movable selection window and the height of that window is varied in incremental steps until an output is generated indicative that the narrow region does not overlie any text.

9. The method of Claim 7 wherein the pattern recognition engine looks at a predetermined narrow region in each segment of the movable selection window and the width of that segment is varied in incremental steps until an output is generated indicative that the narrow region does not overlie any text.

10. The method of Claim 5 wherein the movable selection window can also be used to select a portion of the image to be magnified and displayed in the viewing window without being divided into segments and then re-assembled in a different arrangement.

1 1. An apparatus for displaying an image of a document on a screen comprising:

[1] a receiver for receiving, from a remote transmitting device, a bit-mapped an image of a document comprising text; [2] a processor programmed (i) to process the document by dividing a whole or part of a line of text into segments; (ii) to process the segments so that their contents are of a legible size and (iii) to re-arrange the segments which are positioned adjacently in a line or a part of a line into two or more rows so that their contents can be easily read and

[3] a display to display the re-arranged segments.