WO1998047406A2

WO1998047406A2 - Interactive desk

Info

Publication number: WO1998047406A2
Application number: PCT/IL1998/000131
Authority: WO
Inventors: Moshe Shpitalni; Hod Lipson
Original assignee: Technion Research And Development Foundation
Priority date: 1997-04-23
Filing date: 1998-03-24
Publication date: 1998-10-29
Also published as: AU6417698A; WO1998047406A3

Abstract

An interactive desk system (10) comprising a horizontal desktop (14), part of which is at least partially light transmissive, the part having an underside surface, a top display surface (26), a user-positionable and operable locator providing an output indication of its location, a computer (32) receiving the output indication of the location of the locator, a display projector coupled to the computer for projecting the pattern through the underside surface of the desktop surface to be visible on the top display surface.

Description

INTERACTIVE DESK FIELD OF THE INVENTION The present invention relates to visual interactive devices generally and more particularly to interactive writing surfaces.

BACKGROUND OF THE INVENTION Various types of interactive display systems are known in the art. Touch sensitive display screens and boards are described in U.S. Patents 5,448,263 5,184,115 and 4,938,570. Digitization apparatus for generating absolute coordinates of pen-like probes such as light pens, acoustic locators, digitization boards, tablets and video cameras are described in U.S. Patents 5,530,207, 5,308,936, 5,239,373, 5,142,506, 5,115,230 and 5,023,408. Projection of live computer-generated images onto a large projection surface is described in U.S. Patent 4,846,694.

SUMMARY OF THE INVENTION

The present invention seeks to provide an interactive desk having an image generation functionality associated with the desk surface.

There is thus provided in accordance with a preferred embodiment of the present invention an interactive desk system including: a generally horizontal desktop surface, at least part of which is at least partially light transmissive, said part having an underside surface and a top display surface;

One or more user-positionable and operable locator providing an output indication of its location; a computer receiving the output indication of the location of the locator and generating a pattern at least partially in response to the output indication; and a display projector coupled to the computer for projecting the pattern on the underside of the desktop surface, so as to be visible on the top display surface. BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be understood and appreciated more fully from the following detailed description, taken in conjunction with the drawings in which:

Fig. 1 is a pictorial illustration of an interactive desk system constructed and operative in accordance with a preferred embodiment of the invention, being operated by a user;

Fig. 2 is a generalized flowchart illustrating information flow in the system of Fig. 1;

Fig. 3 is a flowchart illustrating start up operation of the system of Fig. 1;

Fig. 4 is a flowchart illustrating control operation of the system of Fig. 1;

Fig. 5 is a flowchart illustrating a preferred calibration procedure for use with the system of Fig. 1;

Fig. 6 is a schematic sectional side view illustration of part of the system of Fig. 1 in accordance with one embodiment of the present invention;

Fig. 7 is a schematic sectional side view illustration of part of the system of Fig. 1 in accordance with another embodiment of the present invention;

Fig. 8 is a schematic sectional side view illustration of part of the system of Fig. 1 in accordance with yet another embodiment of the present invention; and

Fig. 9 is a schematic sectional side view illustration of part of the system of Fig. 1 in accordance with one embodiment of the present invention.

Fig. 10 is a pictorial illustration of an interactive desk system of Fig. 1 in accordance with another embodiment of the present invention;

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS Reference is now made to Fig. 1, which is a pictorial illustration of an interactive desk system constructed and operative in accordance with a preferred embodiment of the invention, being operated by a user. The desk system preferably comprises a desk assembly, indicated generally by reference numeral 10, which includes a conventional base 12 and a desk top assembly 14.

In accordance with a preferred embodiment of the present the desk top assembly 14 may be adjustably oriented with respect to the base 12, as by an adjustable mounting mechanism 16. The desk top assembly 14 typically comprises a generally horizontal desktop 18, which may be maintained horizontal or tilted to a desired degree as illustrated. Desktop 18 preferably comprises a peripheral frame 20 and an interior, at least partially light transmissive portion 22, having an underside surface 24 and a top display surface 26.

Preferably, the partially light transmissive underside surface 24 includes a transparent glass, plastic (e.g. "Perspex" or "Plexi-glass" commercially available from Duppont) or any other general transparent substrate plate, thin but of sufficient strength to support desktop weight. The top display surface 26 is removably covered with a thin film of "transparent drafting paper", or any other coating, etching, roughening which is sufficiently transmissive to allow the formation of an image thereon. The use of such "transparent drafting paper" also provides the user with a true paper and pen feeling while using the interactive desk.

A user manipulates a digitizer stylus 30, such as a three dimensional mouse or any other suitable position locator. The stylus 30 is connected to a computer 32, via a cable 34 or alternatively by a wireless data link. The stylus 30 may be operable in two dimensions or three dimensions but is preferably operable in three dimensions. According to one embodiment of the invention, the digitizer stylus 30 comprises an acoustic digitizer stylus, such as "GP9" Digitizer commercially available from S.A.C. Several different locators 40 may be used alternatingly so as to emulate pens of different color, width and texture. Additionally several different pens may be used simultaneously by a single user or multiple users on the same desk, for example, one locator serving as a pen for drawing, and another locator as an eraser 42.

In accordance with a preferred embodiment of the present invention, the computer 32 is supplied with stylus responsive rendering software. Preferred software includes, but is not limited to, any of the below-listed commercially available software: Graphical operating system "MS-Windows" commercially available from Microsoft, document editing package "MS-Office" commercially available from Microsoft, image editing "Adobe Photoshop" commercially available from Adobe, graphic art package "FreeHand" commercially available from MacroMedia, computer aided design package "AutoCad" commercially available from AutoDesk, teleconferencing "Imperson" commercially available from Silicon Graphics.

Preferably, the computer 32 drives a display projector 36, coupled to the computer 32, for projecting a pattern 33 on the underside 24 of the desktop, so as to be visible on the top display surface 26. Various embodiments of display projectors are described hereinbelow with respect to Figs. 6 - 9 and incorporate a reflective surface 38 disposed below the desktop 18.

Reference is now made to Fig. 2, which is a generalized flowchart illustrating information flow in the system of Fig. 1. As seen in Fig. 2, the user holds and manipulates stylus 30. The stylus 30 location is digitized and assigned coordinates, preferably within a coordinate system defined with respect to the desktop 18. The computer 32 transforms the coordinates of the digitizer to internal coordinates with the result that a internal cursor is moved on the screen.

The internal cursor cues are processed by computer 32 into display cues which are converted by display projector 36 to light beams which impinge, leaving no gap between the stylus and the impinging image, on the underside surface 24 of the desktop 18, thus causing an image 37 to be viewable on the top display surface 26 of the desktop 18, in the view of the user. It is a particular feature of the present invention that in contrast to conventional touch screens which are pressure sensitive, the desktop 18 of the present invention has no pressure or weight sensitivity and may be used just as an ordinary desktop without interfering with its display functionality.

Reference is now made to Fig. 3, which is a flowchart illustrating start up operation of the system of Fig. 1. During start up, the computer first attempts to establish communication with the digitizer stylus 30. Upon sensed failure to establish communication, an error signal is generated and operation is terminated. The computer may then attempt to establish communication with the display projector 36. Upon sensed failure to establish communication, an error signal is generated and operation is terminated.

The computer then deals with calibration, which is described hereinbelow with reference to Fig. 5. Upon completion of start up, as described hereinabove with reference to Fig. 3, the computer executes a main loop control process, illustrated in Fig. 4, which executes repeatedly until the user stops its operation. The computer operates in a multitasking environment and therefore yields to concurrent programs on the computer at every iteration, in a time-sharing mode. At each iteration, the computer checks the communication line with the digitizer stylus 30 to determine whether input cues from the stylus are available. If not, another iteration takes place.

If stylus cues are available, they are processed as follows: First, it is determined whether the incoming cue indicates that the user requests re-execution of calibration. If so, the calibration procedure is rexecuted. In the absence of a stop cue, the computer extracts the digitizer coordinates from the incoming stylus cue and translates them into internal coordinates, using a calibration matrix obtain from the calibration process, which is described hereinbelow with reference to Fig. 5. Additional stylus cue information, such as pen up or pen down, drawing force, function keys, pen orientation and height, pen identification and pen type cues may also be processed.

The internal coordinates thus derived and the stylus cues are employed to move an internal cursor, much in the same way as if it is manipulated by a conventional two dimensional input device, such as a mouse. In this way, through multiple iterations, a display output is generated.

Reference is now made to Fig. 5, which is a flowchart illustrating a preferred calibration procedure for use with the system of Fig. 1. The calibration procedure begins by allocating empty storage space for a plurality of point pairs, designated P and P. P denotes a point in internal coordinates, while P' denotes a corresponding point in the coordinate system of the desktop 18 in digitizer coordinates.

A marker point P is generated at a random location on the screen and is displayed on the top display surface 26. The user is then prompted to indicate the position of the marker point P by positioning the stylus thereat and generating a stylus cue. The received digitizer coordinates are denoted as point P' and the two coordinate pairs P and P¹ are stored. Normally a total of between 4 - 8 coordinate pairs are stored at any time and are used to define a best-fit transformation matrix, by employing a mapping procedure described hereinbelow. The mapping procedure seeks to obtain a best-fit homogeneous transformation matrix of size 3 x 3 preferably using well-known least-squares techniques. Reference is made in this connection to "Principle of Interactive Computer Graphics", W.M. Newman, R.F Sproull, McGraw Hill, 1979; "Computational Geometry for Design and Manufacture", ID. Faux, M.J. Pratt, Ellis Horwood Publishers, 1980. This transformation can take into account non-isotropic scaling, translation, rotation, skewing and trapezoidal deformation which may be encountered in projection of the display onto the desktop 18. Sometimes a more complex mapping may be required to account for nonlinear optical distortion.

Using the mapping thus obtained, an error value is computed. Once the error value is below an acceptability threshold, typically of the order of 1 mm on desktop 18, the calibration is considered to be acceptable. Until such time as the error value falls below the acceptability threshold, the calibration procedure continues to iterate, each time discarding the earliest acquired pair of coordinates.

Reference is now made to Fig. 6, which is a schematic sectional side view illustration of part of the system of Fig. 1 in accordance with one embodiment of the present invention. It is seen that a display projector 60, driven by computer 32 (Fig. 1) projects a pattern onto a first mirror 62, which reflects the pattern via a second mirror 64 onto the underside surface 24 of the light transmissive portion 22 of desktop 18. Preferably first-surface mirrors are used to avoid double reflectance.

Mirrors 62 and 64 are typically planar mirrors which are arranged at angles such that the plane of the underside surface 24 is perpendicular to the principle ray 68 of the light beams impinging thereon and such that a sufficiently long optical path between the projector and the desktop is obtained thus allowing transmissive portion 22 to reside entirely within the image plane of the projector. The optical path should be enclosed in a light-tight cover 66, so that stray light from the surrounding will not interfere. It should be added that the principle ray 68 does not necessarily follow a planer path and the arrangement of projectors, mirrors and desktop may be spatial.

Mirror 64 is preferably located so as not to limit the leg room available to a user seated at the desk. The projector 60 is preferably located such that the rear panel controls are easily accessible to the user, while not interfering with the user's work zone on the desktop.

Reference is now made to Fig. 7, which is a schematic sectional side view illustration of part of the system of Fig. 1 in accordance with another embodiment of the present invention. In this embodiment, a display projector 70 is located beneath the desktop 18. Otherwise, the essentials of the structure and operation of the system remain the same.

Reference is now made to Fig. 8, which is a schematic sectional side view illustration of part of the system of Fig. 1 in accordance with yet another embodiment of the present invention. Here only a single mirror 80 is employed. The remaining essentials of the structure and operation of the system remain the same. This embodiment is particularly suitable for projectors lacking a rear projection option.

Reference is now made to Fig. 9, which is a schematic sectional side view illustration of part of a variation the system of Fig. 1 in accordance with one embodiment of the present invention. The system of Fig. 9 may be identical with that illustrated in Fig. 1 wherein an acoustic digitizer is replaced by a an optical digitizer and camera assembly 90.

In this case, a wireless pen 92 is employed as the stylus which forms a pattern of light such as a sharp point 93 when brought into propinquity with the top display surface 26 of the desktop 18. The camera 90 is preferably a high resolution video camera, which employs a beam splitter 94 associated with a general optical system 96 for intercepting light received from the desktop 18. The digitizer and camera output to computer 32 (Fig. 1).

It should be pointed out that the rays projected from the display projector in Fig. 9 are illustrated so as not to clutter the drawing, in such a manner that they appear to stop at the beam splitter 94. However in reality, they continue along their path to the desktop surface in the same manner as in Fig. 6.

Reference is now made to Fig. 10, which is a pictorial illustration of an interactive desk system of Fig. 1 in accordance with another embodiment of the present invention. In this embodiment the desktop image is created by projecting a preferably stereoscopic image onto a partially transmissive surface between the users eyes and the desktop, with appropriate perspective and focus so that it appears to the user to be residing on the desktop itself. In the preferred embodiment shown in Fig 10, the transmissive surface is fixed to the users head by means of a head-mounted display 102, such as the "Datavisor" commercially available from Nvision Inc. The image is updated in accordance with the user movements, with respect to the desktop, by means of a head tracking device such as "Flock of birds" commercially available from Polhemous.

It will be appreciated by persons skilled in the art that the present invention is not limited to what has been particularly shown and described hereinabove. Rather, the scope of the present invention includes both combinations and subcombinations of elements and features described hereinabove as well as obvious variations and extensions thereof.

Claims

C L A I M S

1. An interactive desk system including: a generally horizontal desktop, at least part of which is at least partially light transmissive, said part having an underside surface and a top display surface; a user-positionable and operable locator providing an output indication of its location which can be placed on the said top display surface; a computer receiving the output indication of the location of the locator and generating a pattern at least partially in response to the output indication; and a display projector coupled to the computer for projecting the pattern through the underside surface of the desktop surface, so as to be visible on the top display surface.

2. An interactive desk system according to claim 1 and wherein said display projector comprises a pattern generator and a plurality of mirrors for reflecting said pattern from said pattern generator to the underside of the desktop surface.

3. An interactive desk system according to either of claims 1 and 2 and wherein said locator comprises a wireless-three dimensional locator.

4. An interactive desk system according to claim 3 and wherein said locator comprises an acoustic pointer.

5. An interactive desk system according to claim 3 and wherein said locator comprises an electromagnetic positioning pointer.

6. An interactive desk system according to claim 3 and wherein said locator comprises a light emitting pen and an optical pen light sensor and digitizer.

7. An interactive desk system according to claim 3 and wherein said locator is operative to emulate pens of different color, width and texture.

8. An interactive desk system according to claim 3 wherein several said locators can be used simultaneously.

9. An interactive desk system according to claim 8 wherein several said locators may serve as at least one of a pen and an eraser.

10. An interactive desk system according to claim 1 wherein said top display surface comprises at least one portion which comprises a finish selected from the group consisting of a roughened surface, coated surface and etched surface.

11. An interactive desk system according to claim 1 and wherein said display projector comprises a screen selected from the group consisting of a plasma, polysilicon, TFT, CRT and LCD screen.

12. An interactive desk system according to claim 1 and wherein said desktop image is created by projecting the image onto a partially transmissive surface between the user's eyes and the desktop, with appropriate perspective and focus so that it appears to the user to be residing on the desktop itself.