WO1991009394A1

WO1991009394A1 - Orientational computer graphics paint and draw-graphics system

Info

Publication number: WO1991009394A1
Application number: PCT/US1990/007130
Authority: WO
Inventors: J. Cambridge Vivien
Original assignee: Automated Decisions, Inc.
Priority date: 1989-12-11
Filing date: 1990-12-10
Publication date: 1991-06-27
Also published as: EP0457894A4; JPH05500872A; EP0457894A1

Abstract

A system (40) for manipulating and editing image data displayed on a computer screen (58) by controlling display information such as cursors (56), paintbrushes, pointers, pixel images, and graphics objects, which display information is controlled by rotation and lateral movement of a handheld input device (44) such that said display information (56) turns as a handheld device (44) turns and said information (56) moves laterally as the handheld device (44) moves laterally.

Description

0RIENTATI0NAL COMPUTER GRAPHICS PAINT AND DRAW-GRAPHICS SYSTEM

RELATED APPLICATION

This application is a continuation-in-part of U.S. Patent Application Serial No. 07/416,142, filed October 1, 1989, and entitled "ORIENTATIONAL MOUSE COMPUTER INPUT SYSTEM"

TECHNICAL FIELD OF THE INVENTION

The present invention relates to interactive information handling systems, and, more particularly, to a system for controlling computer display information, such as cursors, paintbrushes, pointers, pixel (digital) images, and graphics objects, by rotation and lateral movement of a handheld input device to edit or control the display information.

BACKGROUND OF THE INVENTION

Computer graphics paint and draw-graphics systems allow users to edit information on computer screens with "tools" which are controlled by manipulating a handheld • input device. This interactive control of screen information is achieved through a mouse input subsystem which subsystem may be or may not be partially embedded in the paint or draw software program. In the mouse input subsystem, the movement of a handheld input device is detected across a flat surface with mechanical, photosensitive, electro-magnetic, or electronic sensing mechanisms. The handheld input device, which may be, for example, a mouse, a hand-cursor/tablet, or a stylus/tablet, sends data related to the extent of movement to a mouse interface which interface computes information used to move cursors, pointers, paintbrushes, pixel image bit maps, or draw-graphics objects and transmits this information to the paint or draw-graphics software program.

A mouse data entry system is generally illustrated in Figure 1 and allows a user to interactively enter data into a computer in an easy, natural manner. Such a system typically comprises a handheld input device 10, a mouse interface, cursor driver circuitry for controlling the cursors, pointers, and/or graphics objects 12 on a display screen 14, and software for transmitting mouse parameters such as device location to application programs. Movement of the hand held input device 10 across a flat surface causes cursor 12 on display screen 14 to move or variables in a computer application program to change. This data entry system allows a user to "point" on the display screen or to affect the execution of application programs. In paint systems the aforementioned "tool" is usually a paintbrush which moves across the computer display screen and edits the information displayed on the display screen and edits the information displayed on the display screen in accordance with the shape of the paintbrush and the present and previous positions of the paintbrush. For example, as shown in Figure 2, a typical paint system displays a paintbrush which has a certain, constant, shape such as a line or rectangle, which brush can be moved on the screen by moving the mouse across a flat surface. As the brush moves across the screen, the system edits the screen by altering the color or intensity value of pixels 16 (small colored or lighted elements that, together, make up the screen image) which pixels are positioned on what may be conceived to be the path along which the brush moved. This path can be determined by creating straight or curved lines 18 from the tips of the brush image in the present position 20 to the tips of the brush image in a previous position 22 and defining the area 24 enclosed by the brush's present and previous images and the lines 18. Alternatively, the brush may simply make an imprint at every position that it contacts. Paint systems usually allow the user to activate and deactivate the screen editing action of the paintbrush by pressing buttons on the mouse.

In draw-graphics systems, cursors or pointers are used to create, identify, move, or alter graphics objects on the computer screen. These graphics objects consist of graphics primitives, which primitives are usually lines, polylines, rectangles, circles, ellipses, and text. To create, edit, and move graphics objects, draw- graphics systems use strategic, pertinent, information in the graphics primitives. For example, a polyline is totally described by its vertices and therefore, the draw-graphics system needs only to identify the vertices of the polyline to have a permanent representation of the polyline. Whenever the polyline needs to be moved or altered, the system simply moves or alters the polyline's vertices and uses the new vertices to draw the new polyline.

Draw-graphics systems use the mouse input subsystem to display a moving cursor on the screen and use the cursor and buttons on the mouse to create, select, and change objects. Once an object is selected, it can be moved along with or instead of the cursor, or its shape, that is, its vertices or other pertinent parameters, can change along with or instead of cursor movement.

Currently, computer graphics paint and draw- graphics systems allow the user to directly control only variables that are related to the x and y coordinates of the device as it is moved across a flat surface. Hence a user can currently only change information on the computer screen by changing the location of the device. These systems do not allow users to rotationally control a paintbrush, cursor, pointer, or the orientation of a graphics object or pixel graphics image by rotating the movement sensitive device.

Interactive paint and draw-graphics systems exist where graphics objects and text can be rotated indirectly by moving the movement sensitive device in a circular motion so that the cursor on the screen describes an arc around the center of the object. In those systems the change in the orientation of the object is related to the change in the slope of a line connecting the cursor and the center of the rotating object, where the position of the cursor is related to the position of the movement sensitive device. The operation of such a system is illustrated in Figure 3 and is described in U.S. Patent Nos. 4,661,810 and 4,745,405. Since these methods operate in an indirect fashion their effectiveness is sub-optimal. For example, these existing systems do not allow changes in the position of the graphics objects to take place simultaneously with rotating action. Indirect operation may also increase tediousness, and the current systems do not accommodate rotating cursors and paintbrushes, but rather the angular orientation of cursors and paintbrushes in present paint and draw- graphics systems is fixed during painting and cursor movement operations.

Rotating paintbrushes would enable users to create effects which can be created in the "real world" situation (i.e. with a real paintbrush on paper) and to more effectively and selectively edit the screen.

A need has thus arisen for an interactive graphics system which permits the user to rotate a paintbrush, cursor, pointer, or graphics object and interactively enter data into a computer by turning a handheld, movement sensitive device on a flat surface area. Such a system must also provide for lateral movement, that is, changes in the position on the display screen, of paintbrush, cursor, pointer, or graphics objects (either simultaneously or non-simultaneously with turning motion) by moving the device laterally on the flat surface.

SUMMARY OF THE INVENTION

In accordance with the present invention, a computer paint system allows real time interactive control of the angular orientation of a paintbrush by turning a mouse, giving the user an option which is available in the "real world" situation for creating effects and for more flexible editing. This aspect of the present invention can significantly improve the quality and ease of use of these systems, as well as the output which they create. As do current systems, the system of the present invention allows lateral movement of the paintbrush by moving the mouse laterally.

In accordance with another aspect of the present invention, a computer draw-graphics system allows angular orientation of a cursor, pointer, graphics object or pixel image on the display screen to be changed by turning the data input mouse. Therefore, the draw- graphics system allows the user to move graphics objects by rotating the mouse as well as moving it. The draw- graphics system which allows movement and rotation of graphics objects and blocks of text can allow the user to perform this operation, move and rotate, in one step since the system of the present invention provides for the rotating action of manipulated objects by turning the mouse. The present invention renders the draw-graphics system more powerful and easier to use. The present invention, further, allows the creation and altering of graphics objects by rotating the mouse as well as moving the mouse. Furthermore, the angular orientation of a cursor, pointer, or some other form of graphics in the draw-graphic system can conveniently be used as an additional variable in interactive editing and data entry. BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the present invention and for further advantages thereof, reference is now made to the following Description of the Preferred Embodiments taken in conjunction with the accompanying Drawings in which:

Figure 1 illustrates the operation of a typical mouse data entry system;

Figure 2 illustrates the screen editing action of a paintbrush in a paint graphics system;

Figure 3 illustrates a system for interactively rotating paint and draw-graphics tools and objects on a computer screen in interactive draw-graphics systems;

Figure 4 illustrates the operation of the present orientational interactive graphics system;

Figure 5 is a block diagram of a mouse input sub¬ system of the present orientational interactive graphics syste ;

Figure 6 is a flowchart of the present interactive graphics system which embodies a paint graphics system; and

Figure 7 is a flowchart of the present interactive graphics system which embodies a draw-graphics system.

^yuwxn PCT/US90/07130

8 DETAILED DESCRIPTION OF THE INVENTION

Referring simultaneously to Figures 4, 5, 6, and 7, the present interactive graphics system is illustrated, and is generally identified by the numeral 38 having a mouse subsystem, generally identified by the numeral 40. In the embodiments described, the mouse subsystem 40 is principally the system described in U.S. Patent Application Serial No. 07/416,142. This mouse input subsystem 40 includes a movement sensitive device 42 which is sensitive to changes in its orientation as well as changes in its lateral position on a flat surface. Device 42 is designed to sense changes in its orientation and may comprise, for example, a handheld mouse 44 which is moved over a flat surface such as a tablet 46, or the device 42 may include a stylus and tablet, or a hand- cursor and tablet. The mouse subsystem 40 further includes a mouse interface 50 which may be hardware or software implemented, and functions to determine the angular orientation or changes therein of mouse 44 based on the output generated by mouse 44.

Determination of the angular orientation of the mouse 44 is preferably done through a search in a look¬ up-table 52 which contains precalculated values of change in angle vs. device output parameters, but it may be achieved by computations which may involve geometrical transformations. Mouse interface 50 further functions to keep track of the angular orientation of mouse 44 as well as the lateral position of mouse 44; determine and keep track of the angular orientation of an object 56 on a computer display screen 58 associated with a computer 60 as well as the lateral position of the cursor 56 on the computer display screen 58; transmit to a cursor driver 64 the coordinates of the cursor 56 as well as its orientation on the computer display screen 58; and transmit to the graphics system the orientation of the hand-held device 44 and of the object 56 in addition to other mouse parameters such as position and button states. As user herein, the term "image" or "cursor" displayed on a display screen 58 will include cursor 56, 5 paintbrushes, pointers, graphics objects, pixel images, i and test.

Cursor driver 64 is capable of displaying an image on display screen 58 at variable orientations, which cursor driver 64 used a cursor bit map look-up-table 70

10 to relate object 56 orientation to appearance of the object 56 on the display screen 58. For example if a desired cursor angle is determined to be 1 radian, the cursor driver 64 obtains a bit map from the cursor bit may look-up-table 70 which corresponds to this angle (1

15 radian) and uses this bit map to place a visual representation of object 56 on the display screen 58 which is oriented according to the angle. A keyboard, computer screen, and other peripherals which are part of the typical arrangement of interactive information

20 handling systems are included within computer 60.

Flowcharts of the draw-graphics and of the paint embodiments of the system of the present invention are illustrated in Figures 6 and 7 respectively. In this description it is assumed that the system 40 uses an 25 orientationally sensitive hand-held device to drive a rotating paintbrush or cursor. Reference is first made to the draw-graphics embodiment of the present invention which is depicted in Figure 6, with emphasis on the move/rotate aspect of this draw/graphic system. In ^«30 addition to the option to move and rotate objects, the system 40 includes, in a main menu, options to create and change objects and to load, save, clear, and shut down.

Once the move/rotate option is selected in the system's main menu, the system accepts input from the mouse 44 at block 80 and determines at block 82 from this input if the state of the mouse 44 has changed, that is, if the mouse 44 has moved or turned, or if buttons have been pressed or released.

If the system 40 determines a change in mouse 44 state, and it finds that no buttons were pressed, block 84, the system 40 checks at block 86 to see if there has been a change in the angular orientation of the mouse 44. If a change in the orientation of the mouse 44 is detected, the system 40 computes the change in the angular orientation at block 88 and updates the angle of the mouse at block 90 as well as the angle of the rotating object 56, block 92. At block 94, the current object image corresponding to the current object orientation is stored in a temporary buffer for the purpose of later erasing the current object image.

The system 40 uses the change in the object's orientation or the object's new orientation to transform the object and thereby obtain the image of the object which corresponds to the rotated object at block 96.

Alternatively, the system 40 may not transform and display the object until the move/rotate operation is complete, but rather the system 40 may transform and display a rotating box, convex hull, or some other simplified surrogate of the object. When the move/rotate operation is complete, the system 40 erases the surrogate image and uses the final orientation of the object to transform and display it.

At block 98 the system 40 uses the mouse 44 input to determine any change in the lateral position of the mouse 44, and the system 40 updates the object's coordinates using the computed change in the lateral position of the mouse 44 device. The system 40 erases the image of the object or its surrogate at block 100 and then places a new image on the screen which corresponds to the current location and angular orientation of the object or its surrogate at block 102. The system 40 interprets any button action at block 104 and takes required action. At block 106, based on user input at block 104, the system 40 may terminate the move/rotate operation and surrender control to the main menu of the system 40 or for continued move/rotate operations. Block 88, computation of object angle, preferentially uses a look-up-table previously explained.

A flowchart of the paint embodiment of the present invention is depicted in Figure 7, with emphasis on the paint aspect of the system. In addition to the option to paint on the computer display screen, the system 40 includes, in a main menu, options to change paint and paint brush attributes, load, save, clear, and shut down.

Referring now to Figure 7, once the paint option is selected in the system's main menu, the system 40 accepts input from the movement sensitive device such as a mouse 44 at block 80 and determines at block 82 from this input if the state of the device has changed, that is, if the device has moved or turned, or if buttons have been pressed or released.

If the system 40 determines a change in device state, and it finds that no buttons were pressed, block 84, the system 40 checks at block 86 to see if there has been a change in the angular orientation of the mouse 44. If a change in the orientation of the mouse 44 is detected, the system 40 computes the change in the angular orientation at block 88 and updates the angle of the mouse 44 at block 90 as well as the angle of the rotating paintbrush 56, block 92. At block 94, the current paint brush bit mapped image corresponding to the current object orientation is stored in a temporary buffer for the purpose of later erasing the current paintbrush image.

The system 40 uses the change in the paintbrush's orientation or the paintbrush's new orientation to transform the object 56 and thereby obtain the image of the object 56 which corresponds to the rotated paintbrush at block 96. At block 98 the system 40 uses the mouse 44 input to determine any change in the lateral position of the mouse 44, and it updates the paintbrush's coordinates using the computed change in the lateral position of the mouse 44 at block 100. The system 40 erases the image of the paintbrush at block 102 and then places a new image on the screen which corresponds to the current location and angular orientation of the paintbrush at block 103. If the system 40-finds at block 105 that the paintbrush is activated, that is, if the paintbrush's editing action is active, the system 40 uses the present and/or previous paintbrush positions and orientations to edit the computer display screen at block 107 according to the paintbrush and paint attributes.

The system 40 interprets any button action at block 104 and takes required action. At block 109, based on user input at block 104, the system 40 may stop or start painting. If user input demands stop/start painting, then the system 40 activates or deactivates the paintbrush, depending on the present mode of the paintbrush. If user input did not demand stop/start painting, the system 40 may terminate the paint operation and surrender control to the main menu of the system 40 at block 106, or the system 40 moves to block 110 to reset paint and paintbrush attributes. Blocks 88 and 96, computation of paintbrush angle and bit mapped image, preferentially use a look-up-tables as previously explained. It therefore can be seen that the present system provides for a display of a paintbrush, cursor, pointer, or graphics object which rotates as a mouse rotates, and the system therefore allows interactive painting and graphics data manipulation and editing where tools and graphics objects turn as the user turns the mouse and move laterally as the user moves the mouse laterally.

Whereas the present invention has been described with respect to specific embodiments thereof, it will be understood that various changes and modifications will be suggested to one skilled in the art and it is intended to encompass such changes and modifications as fall within the scope of the appended claims.

Claims

AMENDED CLAIMS[received by the International Bureau on 19 March 1991 (19.03.91); original claim 1 amended; new claims 6 and 7 added; other claims unchanged (2 pages)]

1. A system for editing and manipulating an image on a display screen which system includes tools and graphics images which tools and graphics images can move and rotate on a computer display screen, and which system comprises a mouse interactive data entry subsystem wherein said data entry subsystem comprises: a hand-held movement sensitive device sensitive to changes in orientation and lateral position on a surface; means for sensing changes in variable angular orientation and position of said device; and means responsive to said sensing means for changing the angular orientation and the location of the image on the display screen corresponding to changes in the angular orientation and lateral position of said device.

2. The system of Claim 1 wherein the system includes a paint program, which paint program includes screen images of drawing tools and/or raster images manipulated by said device, such that these tools can be turned on a computer display screen by turning said device.

3. The system of Claim 1 wherein the system includes a paint program, which paint program includes screen images of drawing tools and/or raster images manipulated by said device, such that these tools can be turned and moved laterally on a computer display screen by turning and moving said device laterally.

4. The system of Claim 1 wherein the system includes a draw-graphics program, which draw-graphics program includes screen images of objects manipulated by said device, such that these screen images can be turned on a computer display screen by turning said device.

5. The system of Claim 1 wherein the system includes a draw-graphics program, which draw-graphics program includes screen images of objects manipulated by said device, such that these screen images can be turned and moved laterally on a computer display screen by turning and moving said device laterally.