WO1991011775A1

WO1991011775A1 - A force feedback and texture simulating interface device

Info

Publication number: WO1991011775A1
Application number: PCT/US1991/000632
Authority: WO
Inventors: James F. Kramer
Original assignee: Kramer James F
Priority date: 1990-02-02
Filing date: 1991-01-30
Publication date: 1991-08-08
Also published as: JP3290436B2; AU671705B2; KR100252706B1; EP0513199A1; CA2075178C; ATE287555T1; AU5752394A; JPH05506736A; AU7319991A; EP0513199A4; AU649655B2; US5184319A; DE69133441D1; EP0513199B1; CA2075178A1

Abstract

A man-machine interface is disclosed which provides force and texture information to sensing body parts. The interface is comprised of a force actuating device (900) that produces a force which is transmitted to force applying device (902). The force applying device applies the generated force to a pressure sensing body part. A force sensor (909) on the force applying device measures the actual force applied to the pressure sensing body part, while angle sensors (911) measure the angles of relevant joint body parts. A computing device (911) uses the joint body part position information to determine a desired force value to be applied to pressure sensing body part. The computing device combines the joint body part position information with the force sensor information to calculate the force command which is sent to the force actuating device. In this manner, the computing device may control the actual force applied to a pressure sensing body part to a desired force which depends upon the positions of related joint body parts. In addition, the interface is comprised of a displacement actuating device (901) which produces a displacement which is transmitted to a displacement applying device (902) (e.g., a texture simulator). The displacement applying device applies the generated displacement to a pressure sensing body part. The force applying device and displacement applying device may be combined to simultaneously provide force and displacement information to a pressure sensing body part.

Description

AFORCE FEEDBACKAND TEXTURESIMULATINGINTERFACE

DEVICE

TECHNICAL FIELD

This invention relates to a man-machine interface and in particular to an interface that measures body part positions and provides force and texture feedback to a user.

BACKGROUND OF THE INVENTION

A new manner of computer interaction is now in its infancy. The words "virtual environment" or "virtual reality" will soon be commonplace. A virtual environment is an environment where some portion of the environment is artificially simulated, most often via a computer. A computer may create a graphic simulation of an environment, complete with graphic images of chairs, windows, doors, walls, etc., and even images of other people. The computer may also simulate environmental sounds. The generated objects may be viewed on a common two dimensional display, such as a computer screen, or, by viewing with special stereoscopic equipment, the objects may be made to appear three dimensional.

The most natural way for an individual to interact in a virtual environment is to directly control a graphical representation of himself. For example, if the individual turns his head, the display screen at which he is looking is appropriately updated. Also, if the individual reaches out and closes his hand, the computer generated image of his hand on the screen reaches out and closes. Such virtual environments have been discussed in the literature.

To create the sensation of a virtual reality, the computer should be able to generate and manipulate graphic images of real or imaginary objects in real time. Although generating a graphic representation of an environment may be time consuming and non- trivial to implement, much of the theory has been explored and is &, 11 understood by those skilled in the art of interactive 3-D computer graphics and solid modeling.- The invention described here pertains to the important related area in which relatively little research has been done, i.e., "How may a human user perceive grasping force and texture from his computer generated counterpart in the virtual environment?"

There are many peripheral devices which have been created to allow a user to enter

SUBSTITUTE SHEET information into the computer. The most notable of these is the standard QWERTY keyboard. Besides die numerous modifications of this "key input" concept, there are many other devices with their associated permutations. A partial list of such devices includes mice, joysticks, trackballs and Computer Aided Design (CAD) tablets. The main drawback of these computer input devices is that they don't permit human users to enter information in a manner which may be the most efficient and natural. For example, in a CAD software program, the human designer may wish to rotate a 3-D graphic representation of a block on a computer screen to view and modify the hidden side. Using currently available input devices, the designer must select the axis or a sequence of axes about which the object must be rotated to achieve titte desired orientation and view. After the desired axis is selected, the amount of angular rotation must be determined, usually by the linear motion of a mouse or by entering the desired amount of rotation as a decimal quantity via the keyboard. This whole procedure seems very awkward and unintuitive when compared to what a person would normally do when confronted with a similar task in the "real world," ie., he would simply reach out, pick up and rotate the objectl Providing feedback for this more natural approach to object/environment interaction is an object of this invention.

Instrumented gloves which provide digit position information to the computer have been used to manipulate simulated objects in virtual environments. Such gloves have also been used in telerobotics to control highly dextrous end effectors to grasp real objects. However, lack of force feedback to the glove wearer has reduced the effectiveness of these open-loop manipulation approaches. Imagine a 3-D graphic model of an egg on a computer screen. Suppose you are wearing a glove which maps your digit and hand motions to a graphic image of a hand on the same screen as the egg. As you move your hand and digits, the corresponding graphic images of die hand and digits move in a similar manner. The task is to move your own hand and digits to control the graphic hand on the computer screen to pick up the egg. To accomplish this task you must provide enough force to reliably grasp and lift the virtual egg, but not so much force such that the egg is crushed. Without some kind of grasping force and tactile feedback, this task would be extremely difficult.

Attempts have been made to provide information about simulated contact with virtual or telemanipulated objects to senses other than the corresponding tactile senses. One method of simulated feedback which has been tested uses audible cues. For example, the computer may beep when contact is made. Another simple method is to highlight the object once contact is made. Both these methods will require the user to re-leam hand-eye coordination. It may be frustrating and time consuming for the user to learn one of these "unnatural" methods of grasping an object, and die sensation of interacting in a virtual

SUBSTITUTE SHEET environment will be reduced.

SUMMARY OF THE INVENTION

An object of the invention is a man-machine interface which may be employed in interactive computer applications.

Another object of the invention is a force feedback control system capable of controlling a set force to a selected part of the body, e.g., the digit tip.

Still another object of die invention is a man-machine system capable of simulating textures on a selected part of the body, e.g., the digit tip.

Yet another object of the invention is a man-machine interface comprised of a glove capable of sensing digit and hand positions and hand orientation, which may exert, measure and dynamically vary and control the forces applied to each digit, and which may vary the tactile array pattern presented to each digit tip.

Another object of die invention is a digitial control system capable of sensing the force applied to the digit tip and capable of using this signal to control the digit tip force to a desired force set point which may vary as a function of digit position.

Still another object of the invention is a force and texture feedback system which may be employed in many different applications, such as virtual environments, telemanipulation and interactive 3-D graphics and Computer Aided Design (CAD).

A feature of die invention is the use of a flexible housing which may comprise one or more concentric flexible casings which guide a force-transmitting flexible elongated element such as a flexible, low friction/stiction, low modulus of elasticity thread or a shape memory alloy wire which serves as a tendon and is used in tension to apply force to a sensing body part or to actuate texture simulating elements.

Another feature of the invention is the use of a flexible housing which may comprise one or more concentric inelastic tubes to guide a force transmitting flexible elongated element such as pneumatic or hydraulic fluid to a sensing body part to be used by a force applicator to apply force to the sensing body part. Still another feature of the invention is the use of force actuators to generate fence which is transmitted to die sensing body part via flexible tendon cables, or pneumatic or hydraulic tubes, and used by a force applicator to apply force to the sensing body part

Yet another feature of the invention is the use of force or displacement actuators to generate displacement which is transmitted to a sensing body part via flexible tendon cables, or pneumatic or hydraulic tubes, and used by a texture simulator to simulate textures on the sensing body part

Yet another feature of the invention is the use of a support to which the flexible tendon cables or tubes are secured. The support may be a reinforced wrist-strap when the sensing body part is part of the hand.

Another feature of the invention is the use of a pressure, tension and/or force sensor to measure the force aplied to die force-sensing body part by the force actuator.

One embodiment of the invention presents, for the first time, the use of a glove incorporating not only sensors which provide analog values representing digit and overall hand motion, but also true force feedback to the wearer's digit tips relating the amount of force. a corresponding graphic (or actual) device is applying to a given virtual (or telemanipulated) object The invention also relates to a means whereby simulated texture and edge orientation may be presented to a user.

The invention, which senses one or more body part positions and provides force and texture feedback to one or more body parts, permits a relatively "natural" method of computer interaction. The subject device provides in a single unit (1) controlling body part position-sensing means employing a plurality of signal producing means associated with individual movable controlling body parts, where the signal is related to controlling body part position, with the individual signals analyzed to define a composite signal The signal producing means may be anything which provides body part position and/or orientation, including strain gage, electromagnetic, ultrasonic, piezoelectric, hall effect infrared emitter/detector pair, encoder/potentiometer, laser scanning or other optical position (and or orientation) sensors; (2) force-applying means which may be anything which provides force information to a sensing body part and (3) force-sensing means which may be anything which provides a force measurement signal; and (4) texture- applying means (e.g., an array of texture elements) which may be anything which provides surface pattern (e.g., texture) information to a sensing body part; and (5) force-generating means which may be any actuator which generates a force (or displacement), including

SUBSTITUTE SHEET electrical, electromagnetic, electromechanical, pneumatic, hydraulic, piezoelectric, shape memory alloy (e.g., Nickel Titanium alloys), vapor pressure actuators; and (6) force- transmitting means (eg., a flexible, inelastic tendon guided by a flexible, incompressible housing, or an incompressible fluid guided by an inelastic housing) which may be anything which transmits a force signal from a force-generating means to an applying means (e.g., a force-applying means or a texture-applying means); and (7) signal collection and producing means (e.g., a processor or computer) which may be anything which collects signals (e.g., from the position-sensing and/or force-sensing means) and produces signals (e.g., for the force-applying and/or texture-applying means); and (8) support structure (including clips, straps, clamps, guides, pockets, material, etc.) used to support the body part sensing means, the force-applying means, die texture-applying means, the force-generating means, the force-transmitting means and the signal collection and producing means.

The signal associated with the controlling body part position-sensing means may be coordinated with the force applied to a sensing body part and also with die texture applied to a sensing body par For example, the signal produced by the controlling body part position-sensing means may be used by a signal collection and producing means to manipulate a multiarticulated computer generated interactive entity in a virtual environment The force-applying means may apply force to a sensing body part in relation to the interaction between die interactive entity and a component of the virtual environment In addition, die texture-applying means may be associated with a surface pattern informative signal and apply a texture to a sensing body part to further enhance die sensation of reality in relation to the interaction of the entity and a component of the virtual environment

A particular application for the invention is to sense and provide force and texture feedback to die hand. A useful embodiment for the invention when used for die hand is a "feedback glove." The feedback glove embodiment is comprised of means for measuring position and orientation of the hand, means for measuring individual joint angles, means for applying force to various parts of the hand, means for sensing the applied force, and means for applying selected textures to various parts of the hand. Many of die specific descriptions of die invention will be centered around die feedback glove, however, the sensing and structures described for the glove may be easily translated to other body parts (e.g„ arms, legs, feet head, neck, waist etc.).

In a preferred embodiment of the feedback glove, the means for providing position and orientation of the hand is a Polhemus™ electromagnetic position sensor. The individual joint angle sensing means is comprised of two long flexible strain gages mounted back to back. The strain gage assemblies reside in guiding pockets sewn over

SUBSTITUTE SHEET each joint When a joint is flexed, one of the strain gages of die corresponding pair of gages is in tension, while the other strain gage is in compression. Each pair of two train gages comprise the two legs of a half bridge of a common Wheatstone bridge configuration. An analog multiplexer is used to select which of the half bridge voltages is to be sampled by an analog-to-digital converter. The maximum strain experienced by each gage is adjusted by varying the thickness and elastic modulus of die backing to which the gages are mounted. The backing is selected to maximize the signal output without significantly reducing the fatigue life of a gage. These joint angle strain gage sensors are disclosed in the Kramer et al. patent application number 07/258,204 and are incorpσrtated herein by reference.

The means for applying force to parts of die hand is comprised of a means (e.g., an electric motor) for generating a desired force, a means (e.g., a flexible tendon/casing assembly) for transmitting die generated force to a force-applying means, and a means (eg-, a force-applying platform) for transferring the force to a specific part of the hand (e.g., the digit tip). The feedback glove may also comprise a means (e.g., a force-sensing platform or load cell) for measuring the applied force. The means for applying texure to parts of the hand is comprised of a means (e.g., an electromechanical solenoid) for generating a desired displacement a means (e.g., a flexible tendon casing assembly) for transmitting the generated displacement to die hand, and a means (e.g., an array of texture elements) for applying a surface pattern to a specific part of the hand (e.g., die digit tip). The embodiment includes structure which supports both ends of die tendons and casings, and also supports the force and texture-applying means.

The force feedback glove, which embodies joint angle sensors and also the force and texture feedback apparatus, overcomes many of the problems of joint sensing devices which do not embody force and texture feedback. The feedback glove simulates contact and grasping information in a "natural" manner to a user and facilitates many tasks, such as those arising in interactive 3-D graphics and telerobotics. The feedback glove may be used to feedback texture information from "virtual" objects in a virtual environment or from distal "real" objects when used in telerobotic applications.

When used with appropriate animation and control software, the feedback glove provides joint angle sensing and sufficient tactile feedback for a user to control an interactive entity, such as a computer generated graphic representation of his hand to reliably grasp a virtual object such as a cup, or any object which appears as a graphic model on a display device. Some virtual objects are programmed to demonstrate physical properties similar to real objects, such as weight contour, stiffness and texture. These,

ET and other features, may be sensed and die virtual objects manipulated using the feedback glove. The force feedback incorporated into die glove relays the virtual grasping force information to the user, while a texture simulator allows die user to sense orientation and motion of edges simply by "touching" virtual objects with his own computer simulated virtual digits.

The feedback glove, which provides joint angle sensing and force and texture feedback, may also be used for telerobotics. For this application, die feedback glove provides joint angle information which is used to control an interactive entity, such as a robot manipulator, to grasp a distal real object The force and texture feedback of the glove provide the user witii the actual gripping force and the object contours sensed by the robot's gripper, so the real object may be reliably grasped and manipulated without dropping or crushing.

A glove using force feedback may also be programmed to teach digit dexterity, digit timing and even the motions necessary to learn some musical instruments. For example, if the user were learning the piano, as digits are flexed, the user would receive digit tip pressure form virtual keys signifying to the user that he had pressed die key. Tendons similar to those positioned on die dorsal side of die digits to restrict digit flexure may also be placed on fee palm side of die hand. These palm-side tendons may be used to force the digits into the desired flexed positions or to restrict the digits from extending. These tendons would be used in the case when die user wanted to be "taught" to play the piano and wanted his digits to be properly positioned and flexed for him at die proper times. The idea of this example may be extended from a virtual piano to other virtual instruments and even to other devices such as a virtual typewriter. The feedback glove could be used to teach someone to type, and when learned, to allow die user to generate text by "typing in the air."

More specifically, the invention is a man-machine system which, in addition to measuring actual human joint angles, provides two feedback sensations to die user. The first sensation is force. In a preferred embodiment a small device is attached to die digit tip of a joint-angle sensing glove and holds a force-applying platform in juxtaposition to the digit tip. The force-applying platform is displaced from the digit tip (by about 4 mm) by a retractable means (e.g., a leaf spring) when unactivated, but is capable of quickly contacting the digit tip and applying a dynamically selectable force when activated. The sudden impact of the force-applying platform provides a sensation similar to that perceived when die actual digit tip contacts an object Thereafter, the force-applying platform presses against the digit tip with a programmable force which may relate the amount of force that a virtual digit is pressing against a virtual object

In a preferred embodiment the force that is applied by die force-applying platform to the digit tip is transmitted from a force generating actuator (a c. servo motor) via a high tensile strength, flexible tendon enclosed in a flexible, non-compressible tubular casing. The function of tiiis assembly is similar to a bicycle brake cable. Other embodiments may employ force actuators based on electrical, electromagnetic, electromechanical, pneumatic, hydraulic, piezoelectric, shape memory alloy (e.g., Nickel/Titanium alloys), vapor pressure, or other suitable technologies. In choosing the appropriate actuator technology, various factors should be considered, such as speed of response, force output size, weight cost and power consumption.

One end of die tendon casing is secured near the force actuator and die other end is secured to a wristband near the feedback glove. As a tendon emerges from the end of the casing secured to the wristband, it is guided by sections of casing affixed to die glove material until the tendon reaches its designated final location. Tendons which are to provide a force to restrict the wearer from flexing a digit are guided from die wristband across the back side of the hand to the final location. A preferred embodiment has these tendons passing across the back of each digit and has them mechanically connected to die force-applying platform at die digit tip. In addition, a tendon may be terminated at any properly reinforced intermediate glove location.

As tension is increased, tendons which pass along the back side of a digit press against the joints and do not tend to pull die glove material away form the hand or digits. The tension of the tendon restricts the joints over which the tendon passes from flexing in a direction which attempts to extend die tendon further.

To provide a force to restrict the wearer from extending a digit or to actually drive a digit into a flexed position, tendons are guided across the palm side of the glove by sections of casing. In a preferred embodiment these tendons are guided to die digit tip where they are ultimately secured to a force-applying platform, but they may also terminate at properly reinforced intermediate positions. Unlike the case where the tendons are guided along the back-side of die hand, when die tendons which are guided along the palm-side of die hand are in tension, they tend to pull die casing sections (and hence the glove material) away form the hand. Although not necessary, if it is desired to guide these tendons along die surface of the palm and digits as they pass from where the casings are secured to the wristband to their final designated locations, die glove must be appropriately reinforced between each joint

SUBSTITUTE SHEET Where the tendons are routed and where they are ultimately secured to die glove will determine the forces applied to the hand by the tendon. Forces and torques applied to parts of the hand by a single tendon may not be controlled independently. Only die force applied to one part of the hand or the torque applied by die tendon to an individual joint may be controlled. In a preferred embodiment the tendons are fastened to die force- applying platforms at the digit tips, and the forces at the digit tips are measured and controlled, not die torques applied to die joints. To isolate die force and independendy restrict motion of a single intermediate joint, a separate tendon is used. Its casing is secured just prior to die joint, and the tendon is fastened to a force-applying platform just beyond die joint As previously mentioned, die glove is properly reinforced near the joint so the glove material doesn't unduly stretch under the force of the tendon.

When force is initially applied by a force actuator, die force will appear between die wristband and the intended digit Therefore, die wristband will tend to move towards die digit as the "slack" in the skin on the wrist is taken up. The tendency for this relative motion can be reduced by incorporating a means which initially takes up die slack in this skin. Once this slack is taken up, the wristband will stop moving, and die digit will experience the full tendon force (except for frictional losses). If the slack in the wrist skin s not initially taken up, to provide a realistic contact sensation, die force actuator must have sufficiently high bandwidth such that this slack take-up time is insignificant when compared to die bandwidth of digit motion.

In a preferred embodiment me actual force at the digit tip is sensed and fed back to a servo control system. The control system controls the output of the force actuator such that die force applied to die digit tip follows a desired force profile regardless of the undesireable compliance of die skin on the wrist The force profile for any digit is a function which produces a desired force set point for any given digit and hand position. That is. as either the digit or hand changes position, die force applied to the digits varies accordingly . For example, a force profile may be generated which simulates the force sensation of a push button switch that gradually increases its opposing force as the button is depressed until it reaches its toggle point, clicks, and releases most of its resistive force.

In addition to providing object contact and force information, the invention describes a means whereby object textures and edge orientations may be perceived. For one embodiment the previously described digit tip force applicator may be modified to include an array of small stimulators, called texture elements. These elements produce a surface pattern (e.g., a simulated texture) in addition to providing force feedback. Each texture element may be individually selected. The texure element may be a small pin which extends when selected and die amount of its extension may be programmed. The texture element may also be a pair of electrodes, and tactile sensation produced via electrocutaneous stimulation.

In a preferred embodiment, the texture elements are driven by a texture displacement actuator. A flexible bundle of force feedback and texture simulating tendons connect the glove to bodi die force and texture actuators. The type of displacement actuator for a texture element may vary. A particular embodiment may employ binary or linear displacement actuators and die actuators may be based on electrical, electromagnetic, electromechanical, pneumatic, hydraulic, piezoelectric, shape memory alloy, vapor pressure and other suitable technologies. In choosing the appropriate actuator technology, various factors should be considered, such as speed of response, force output size, weight, cost and power consumption. If pneumatics or hydraulics is used, a hermetically sealed flexible tubing assembly is used to connect die texture acmator to the texture element Otherwise, the connection may employ a cabling means comprised of a tendon inside a casing, similar to that used to transmit me force from the force actuator to the fence applicator.

If binary actuator (e.g., a two-state solenoid) is used, then die texture element will either be fully extended or fully retracted. __f a linear actuator is chosen (e.g., a d.c. servo motor) then the extension of the texture element may be continuously varied. The force witii which die texture is presented to die digit tip is determined by the force actuator. The pattern of die texture array may be varied widi time and reflect changes in die position of die joints or hand. For example, by dynamically varying the texture array, a user may perceive his virtual digit moving over various (e.g., smooth rough) virtual surfaces. Using the time varying texture array, a user may also determine die edge orientation of a virtual or telemanipulated object

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. la is a perspective view of a tendon/casing assembly.

FIG. lb is a cross-section for the perspective view of FIG. la.

FIG. 2a is the side view of an embodiment of die invention showing the force- transmitting tendon assembly affixed to a glove. FIG. 2b is a cross-section view of an embodiment of die invention which shows tendons affixed, via tendon guides, to material covering a digit one tendon to die back side and one tendon to die palm side of the digit

FIG.2c is an embodiment of die invention which shows tendons affixed to provide force feedback to other body parts (e.g., the arm).

FIG. 3 is the side view of an embodiment of the invention showing the texture simulating tendon assembly affixed to a glove.

FIGS. 4a and 4b show an embodiment of die invention where force tendons are affixed, via tendon casings, to bodi the palm and back side of die digit tip of a glove. One end of die tendon casing is secured to a wrist portion of the glove, and die otiier end is fastened to die force applicator assembly on me digit tip.

FIGs.5a - 5h, and FIGS.6a -6c show various force applicator embodiments.

FIGs.7a ami 7b show the force applicator modified to simulate, in addition, texture information.

FIGs.8a - 8m show various texture simulator embodiments.

FIG.9 is a schematic electrical mechanical signal propogation diagram.

FIG. 10 is a control system block diagram for control of the digit tip force.

FIGs. 11a - lid show a force applicator embodiment which employs a load cell to sense force applied to die digit tip.

FIGs. 12a and 12b show a force platform capable of pivoting to make the contact pressure between the platform and the digit tip uniform.

FIG. 13 is a side view of a force applying platform where the pressure distribution may be modified by adjusting tendon tensions differentially.

FIGs. 14a and 14b show the side and plan views of an embodiment where the force applying platform is capable of pivoting in any direction and tiius can move the location of

SUBSTITUTE SHEET the centroid of pressure.

FIG. 15 is a side view of an embodiment showing how die tension in die tendon may be measured prior to the platform contacting die digit tip.

FIGs. 16a and 16b are side views of two more methods to measure tendon tension.

FIGs. 17a and 17b are side views of two embodiments of a structure which supports both a bend sensor and a force-transmitting tendon.

FIGs. 18a and 18b are a perspective and plan view of an embodiment which provides a pre-tension between a force feedback glove and die casing support wristband.

HG. 19 is the block diagram of a three-loop force control system.

DETAILED DESCRIPTIONOF ILLUSTRATIVE EMBODIMENTS

FIGS, la and lb show how the force generated by a force actuator may be transmitted to a chosen location. More specifically, FIG. la shows a perspective view of a tendon assembly, and FIG. lb shows a cross-section view. The tendon assembly is comprised of a low friction, high modulus of elasticity and high tensile strength, flexible tendon cable

100 (e.g., Dacron™ 20 lb. test fishing line or Kevlar™ thread) inside an assembly employing one or more concentric flexible, low-compressibility tubular casings 101 (e.g, Teflon™ mbing). One end 102 of die casing assembly is secured near the force actuator and die otiier end 103 of die casing is secured near the location where the force is to be applied (e.g., for a feedack glove, the casing may be secured to die wristband, and die force applied to die digit tip). By using a plurality of concentric casings (e.g., a #20 Teflon tube inside a #14 tube) rather than simply increasing the thickness of the wall of a single casing, die resulting tendon casing is more flexible (since the casings may slide relative to each otiier) and still produces an overall compressive strength nearly equal to that of a single casing of equivalent wall thickness.

FIG.2a is a side view of a force feedback tendon assembly affixed to a glove 200. In this embodiment each tendon force is generated by a d.c. servo motor 201. The motor is driven by a current amplifier so that a motor torque is produced which is proportional to the amplifier current This torque is converted to a tendon force by the tendon pulley 202 on which the tendon cable 203 is wound. By securing one end 204 of die tendon casing

SUBSTITUTE SHEET near the motor and die otiier end 205 to the glove's reinforced wristband 206, the tendon force produced by die motor may be transmitted to me glove. In a preferred embodiment of die invention, die wristband is comprised of a sturdy, reinforced strap with Velcrc™ backing wrapped around a thin, rubber (e.g., polyurethane) intermediate layer. The rubber layer provides a comfortable interface between die reinforcing strap and die user's wrist The strap is made from a heavy-duty thread (e.g., canvas) which is woven to allow it to be flexed around a wrist but to otherwise provide a sturdy support In general, the wristband may be manufactured from a variety of materials such as foam padded injection molded plastic. The wristband may be manufactured as part of the glove or made as a separate unit

The tendon cable passes through a series of tendon guides 207 as it extends beyond die point where die casing is secured to die wristband on its way to the digit tip force applicator. In one embodiment the tendon guides for the back side of the hand are made from flexible, but incompressible casing (e.g., Teflon mbing) and fastened over die metacaipophalangeal (MP) 208 and proximal interphalangeal (PIP) 209 joints. These guides prevent die tendons from moving laterally and slipping off die sides of the joints as the knuckles protrude during flexure. In die embodiment where die glove has tendons 210 on die palm side of die hand, and it is desired to have die tendons remain close to the hand when tiiey are in tension, tendon guides 211 are located between die MP and PIP joints and also across the palm to keep die tendon from pulling away from the glove. The glove is also reinforced in a variety of places to prevent die glove from being pulled away from the hand by die tendon guides. Tendon guides may be affixed to die glove by such means as sewing or gluing, or the casings may be molded directly onto into the glove.

The digit tip force applicator 212 (shown generically by the cross-hatched portion of the digit tip) applies both back-side and palm-side tendon forces directly to the digit tip.

Also on the digit tip force applicator assembly is a force transducer for each tendon which senses the actual force applied to die digit tip. These force signals are fed back to the motor force control system which makes appropriate adjustments such mat die desired force profile is perceived by die user.

FIG. 2b is a cross-section view of an embodiment of die invention showing force feedback tendons 216 passing through guides on both die back 213 and palm 214 sides of a glove digit Bodi tendons are attached to die force applicator at the digit tip. In a preferred embodiment when the tendon guides are affixed to an elastic glove, only die palm-side tendon guides need reinforcement to ensure that they remain against the digit when die tendon is in tension. One way to accomplish the reinforcement is to fasten

S B T TU additional material 215 of low elasticity (e.g., nylon, plastic, or metal) around the digit at die base of the tendon guide.

FIG. 2c shows a force feedback tendon/casing assembly applied to the arm. Casings 217 may be secured to a reinforced strap 218 worn around die bicep. The strap is similar in construction to the wristband previously described and also employed here.

Bodi die tendons shown exit the casings on the bicep and are affixed to die wristband 219.

One tendon 220 provides a force which restricts the elbow from extending while the other tendon 221 provides a force which restricts the elbow from retracting. Assemblies similar to the ones shown in FIG.2a - 2c may be incorporated into a "feedback body suit" i.e., a suit which covers all, or portions of the body, and which can apply force and texture information to various parts of the body.

FIG. 3 is a side view of a texture simulating tendon assembly affixed to a glove. The tendon displacement in this embodiment is generated by a two-state electromechanical solenoid 300 and is transmitted to die digit tip via a tendon and casing assembly 301. The tendon assembly shown here is similar in function to die tendon assembly described earlier for FIGS, la and lb, however, die diameter of bodi die tendon and casing may be smaller since the forces these texture tendons need to exert are less than the forces exerted by the force feedback tendons.

One end 302 of die tendon casing for the texture simulator is secured near the displacement actuator, and a point 303 near the other end of die casing is secured to die glove's reinforced wristband. After the casing is affixed to die wristband, it continues on and is fastened to the glove at various locations 304 between the joints on its way to its designated final location, which in this embodiment is die digit tip texture simulator 305. Casings may be affixed to die glove by such means as sewing or gluing, or the casings may be molded directly onto into the glove. In die embodiment shown, there is slack 306 in the casing between points where it is affixed to die glove to allow for the tightening of the casing when the digits are bent The casings may also be guided along the sides of die digits widiout allowing for slack since they won't be stressed when the digits are bent

FIG. 4a shows a plurality of force feedback tendons 400 and dieir guides 401. Although the texture feedback discussed in FIG. 3 may be used simultaneously with force feedback, die texture producing tendons have been omitted from this drawing for clarity. The tendon casings 402 are shown secured to the reinforced wristband 403. In this embodiment mere is one tendon on die back of each digit to control the force applied to the digit tip. In addition, die figure provides an example of an abduction force feedback tendon

TUTE SHEET 404 on the thumb side of die index digit

Force is imparted to each tendon from a force actuator. In die embodiment shown, forces are transmitted to the glove via a tendon assembly similar to FIGS, la and lb. One end of die tendon casings is secured near the force actuator, and at die otiier end is fastened to die glove's reinforced wristband. Tendons 405 intended for the palm side of the glove extend around the wristband as shown. These tendons 400 intended for the back side of the hand emerge from the casing on die wristband and are guided along the back surface of the glove by sections of casing 401 until they reach the desired final location. In die embodiment shown the final tendon location is the digit tip force applicator 406.

FIG. 4b shows a force feedback tendons 405 guided around die wristband to die palm side of a glove. The palm-side tendons men emerge from their casings on the wristband and are guided through sections of casing 407 on their way to die digit tip force applicator.

One useful yet uncumbersome and inexpensive embodiment of die invention employs force feedback tendons only along die back of the hand to die tips of die thumb and index digits, and employs texture elements only on die index digit tip. This "reduced" embodiment is in contrast to employing bodi force feedback and texture simulation to each joint of all five digits. The reduced embodiment provides die wearer with sufficient force feedback information to grasp most virtual objects and also allows the wearer to sense virtual textures witii die index finger. Although, employing force feedback to all joints on all digits and texture simulation to all digits tips will provide die wearer with a more realistic simulation of his virtual environment the increase in realism may not outweigh the added cost and complexity of the system.

FIGS. 5a-5e shows a digit tip force feedback applicator which is comprised of a force-applying platform and a force-sensing platform. FIG. 5a is a perspective view, FIG. 5b is a front view, FIG. 5c is a bottom view, and FIGS. 5d and 5e are side views. Modifications may be made to this functional design without departing from the scope of the invention. The force feedback applicator may be manufactured directly into the glove material (as might be done if the glove were molded from a type of plastic). The applicator may also be affixed to die glove externally after both the applicator and glove are manufactured separately. The force applicator may also be a device which is simply clipped to die digit tip after the glove is put on.

In a preferred embodiment a force tendon 500 is guided from the force actuator to

SUBSTITUTE SHEET the force feedback applicator, splits into two tendons, each tendon passing by die force- applying platform 501 (e.g., though holes), and mechanically connected to die ends of die force-sensing means, which is a force-sensing platform 502. The force feedback applicator structure 519 provides support for holding die force-sensing and force-applying platforms in juxtapositon to die digit tip. The force-sensing platform is forced via die force of die tendon towards the digit tip. The force-sensing platform presses against the force- applying platform which then contacts and applies force to the digit tip (FIG Je). When there is little or no force in the tendon, die force-applying platform is displaced from the digit tip by about 4 mm and is held away by a retractable means such as small springs (FIG. 5d). Leaf springs 503 are employed in the embodiment shown. By keeping die force-applying platfoπn displaced from the digit tip in an unactivated position until force is applied, bandwidth requirements of die forcέ actuator are reduced. For example, when die invention is used to provide feedback from a virtual environment and a virtual object is grasped, die force-applying platfoπn assumes an activated position and contacts the digit tip with a non-zero relative velocity, as would a real object when contacting die digit tip. If the force-applying platform were always in contact with die digit tip, very large tendon velocities and accelerations would have to be generated to provide die same contact sensation to the user.

The force-sensing platform may be simply a strain gage beam which bends across a fulcrum 504 as tendon force is applied. The fulcrum shown in FIGS. 5a - 5f is thin and concentrates the applied force over a small area such that the induced strain is easily measured by die two strain gages 505, 506 mounted differentially to either side of this force-sensing platform.

Alternative fulcrum designs are possible such as shown in FIG.5g. By modifying die fulcrum shape and contour, various stress vs. tendon force profiles may be obtained. For example, the fulcrum design of FIG. 5g will provide a higher strain "gain" for low strains than die fulcrum of FIG. 5f, e., the detected strain will be large for small forces, but the strain gain will decrease as the force-sensing platform bends around die fulcrum. As the force-sensing platform bends around die fulcrum, the measured strain includes not only a component from bending but also includes a component from tension in the platform. By varying the contour, and thus the strain sensitivity of the force-sensing platform, small forces are detected with fine resolution, but die sensor will not saturate as quickly for higher strains. Further modifications of die fulcrum and platform geometries produce additional strain vs. force profiles.

As shown in FIGS.5a - 5g, when tension is applied to die tendon, strain gage 505

SUBSTITUTE SHEET is in tension and strain gage 506 is in compression. Both strain gages are active and cover the area of the platform experiencing strain. Together, the two strain gages form a half bridge for a common Wheatstone bridge circuit which provides temperature compensation. The fulcrum and all other parts of die force applying platform that touch die force sensing platform are made from a thermally insulating material to insulate the strain gages on die force-sensing platform from the temperature fluctuations of the digit

FIG. 5h shows a force-sensing means, comprised of two strain gages 507, 508, mounted to opposite sides of a flexible stress-sensing element 509 which is placed in series with the tendon and experiences a tensile force related to the tendon force. The stress-sensing element may be a flattened portion of die tendon itself. This stress-sensing element may be used to measure die tendon tension and/or die joint angles. One strain gage 507 is mounted to die top side of die element while the second strain gage 508 is mounted to the bottom side. In the embodiment shown, die stress-sensing element is used to measure bodi tendon tension and joint flexure. Therefore, the entire gage-element-gage "sandwich" is positioned in, and slides freely through, the casing guide 510, which has a rectangular cross-section in this region. Bodi gages are covered with a smooth, flexible encapsulation 511 (e.g., a type of plastic) which provides the surface that slides against die casing. The differential signal from the two gages is used to determine the joint angle, while the common mode signal from the same two gages provides a measure of die tendon tension. The stress-sensing element may be made from a non-flexible material and located between joints when only a measure of tendon tension is desired. The force in the tendon near the digit tip closely approximates die force applied by die force-applying platform to the digit tip. If the tendon tension is found using die stress-sensing element described here, the force-sensing platform previously described may be removed from the digit tip force applicator, and the mechanical design may be simplified to a single platform 512.

FIG.5i shows how a force may be focused to restrict flexure of a single joint (e.g., the PIP joint as shown in this figure). The tendon casing 513 is secured to a first reinforced section 514 of die glove just prior to the selected joint The tendon 515 exits the main casing and is guided over the joint by a section of casing 516, which is fastened to a second reinforced section 517 of die glove. The tendon exits the casing and forks into two tendon parts (as is shown 520 for the digit tip force-applying platform of FIG. 5a). The two tendon parts pass around opposite sides of die digit and are affixed to opposite ends of die force-applying platform 518, which is secured to die second reinforced section of the glove. The platform assembly contacts and presses against the digit when die tendon 515 is in tension.

UBSTITUTE SHEET The same method of operation can be applied to restrict the joint from extending as was described above to restrict the joint from flexing. A second tendon casing 521 is affixed to die first reinforced section of die glove. A second tendon 522 emerges from die casing and forks into two tendon parts. The two tendon parts pass around opposite sides of die digit and are affixed to opposite ends of the force-applying platform 523. The platform assembly contacts and presses against die digit when die tendon 522 is in tension.

In die case where it is undesireable to reinforce die glove to support sections such as 514 and 517, HG.5j shows a way to provide force feedback to an individual joint of an unreinforced glove. If the glove of HG.5h were not reinforced near sections 514 and 517, then when tendon 515 was in tension, die two sections would be drawn towards each other. A possible solution would be to place a hinge between the sections to prevent them from simply sliding togetiier. However, since the bend axis of a digit may translate and change orientation with bend angle, a single hinge would be uncomfortable for a glove wearer.

A preferred alternative to die "fixed binge" solution is shown in HG. 5j, where sections 524 and 525 are in contact with each other and produce a pivot surface 527 when tendon 526 (emerging from casing 530) is in tension. The pivot surface is created by the two mating flaps 528 and 529, which each have a characteristic surface contour designed to follow the average knuckle axis during flexure. As die tendon tension increases, the two sections press against each other and section 525 is forced to rotate clockwise, while section 524 rotates counter clockwise, each section rotating about the "moving-" contact pivot point The two sections are able to slide axially relative to die digit so they may contact each other when tendon tension is applied, and also so the same surface contours for the two sections will accomodate a variety of different knuckles. The two flaps, in addition to possessing a contour, may also have mating surfaces, such as mating groves, to prevent one surface from sliding off the other surface.

To keep the sections secured to die digits, die sections may be made from a solid, but elastic material (such as a plastic or spring metal), which is pro-formed to clip around die digit as shown in HG. 5j. The firm elastic strap 530 helps hold the two ends 531 of the clip together. One end of die elastic strap is permanently secured to one side of die clip, while the odier end 532 of the strap is secured to die otiier side of the clip by Velcio™ 533. The elasticity of die clip, together with die elastic strap, hold the section firmly to the digit but since the clip and strap are elastic, they allow the digit diameter to expand when the digit is flexed.

TITUTE SHEET In some instances, it may be preferred to have a linkage attached to die sections, such as is shown in HG. 5L For example, if a rotary goniometer (e.g., a potentiometer, an optical encoder, or a rotary Hall effect sensor) were attached to die linkage at die joint 534 between die two links 535 and 536, die value of die goniometer may be related to the joint angle of die knuckle. When the linkage is employed, die force feedback assembly of HG.5j may still be used, however, as shown in HG. 51, the tendons may also be affixed directly to die linkage. A first casing 537 is affixed to link 535 and tendon 538 is affixed to link 536. Similarly, a second casing 539 is affixed to link 535 and tendon 540 is affixed to link 536. When tendon 538 is in tension, link 536 is pulled to rotate clockwise, forcing the digit to extend. When tendon 540 is in tension, link 536 is pulled to rotate counter clockwise, forcing the digit to flex.

Note that in HG. 51, supporting sections similar to those used in HG. 5j are shown. If the glove is appropriately reinforced, otiier support sections, such as shown in HG. 5i, may be used. Also note that in HGS. 5J-51, force-applying platforms may be employed to focus die applied force to a particular region of the digit In addition, for clarity, force feedback tendons for the palm-side of the hand are not shown in HGS.5j-51, however, they may be employed in an obvious manner.

HGS. 6a - 6c show an embodiment of die force feedback applicator which produces force feedback from a tendon affixed to die palm side of die glove. This configuration provides a force which restricts the digit joints from extending and may also force them to flex. HGS. 6a and 6b show side views, while HG. 6c shows a top view. For clarity, only the apparatus specifically required for palm-side tendons is shown, but the force applicator may additionally include die apparatus shown in HGS. 5a - 5e. Tendon force is generated by an acmator and transmitted, as shown in HGS. 1 and 2, to die force feedback applicator. As shown in HG. 5a, the tendon 600 is guided past die force applying platform 601 (e.g., through holes), and is affixed to the force-sensing platform 602. The force-sensing platform again has two strain gages connected differentially in a half bridge configuration. The force-applying platform is also as before and has a stress concentrating, thermally insulating fulcrum on die side opposite to the digit The insulating fulcrum prevents heat conduction from the digit to the gages on the force-sensing platform. The force-applying platform is displaced above the digit nail by springs (HG. 6a) and contacts the digit nail only when a force is applied to die tendon (HG. 6b). In the embodiment shown the springs are leaf springs 603. The applied tendon force presses the force-sensing platform into the force-applying platform which then presses against die digit nail. As the force-sensing platform presses against the force-applying platform, the platform is bent around the fulcrum and produces a strain in the gages indicative of die force applied to die digit nail.

HGS. 7a and 7b show an embodiment of a digit tip texture simulator. HG. 7a shows the top view, while HG. 7b shows a view looking at the die texture simulator from the digit tip. The particular embodiment shows a 3x3 texture array 700, where the texture elements are spaced on 3 mm centers and extend 1 mm when activated. Texture arrays employing various numbers of texture elements may be constructed. The texture array is contained within a modified force-applying platform 701 and held in juxtaposition to die digit tip by the supporting structure 519. As shown, this texture simulator assembly may also provide force feedback by including die same force-sensing platform 702, fulcrum 703, and strain gages 704 as described in HGS. 5 and 6. In HGS. 7a and 7b, the actuating mechanism for the texture elements is not shown.

Displacement may be delivered to die digit tip texture simulator from the corresponding acmator as previously described in HG. 3 via a tendon cable/casing or mbing assembly, by electircal wires, or by pneumatic or hydraulic means. HG. 8a is a cross-section view where a tendon 800 enters die digit tip texture simulator 801, and when actuated, pulls on die base of a corresponding spring-loaded texture element 802 to raise it When raised, the texture element extends from within its enclosure and presses against the digit tip. When the tendon force is reduced, die spring 803 causes die element to retract back into die digit tip texture simulating enclosure. In all of- HGS. 8a -8m, the diagram on die left shows die unactivated state and die diagram on the right shows the activated state.

HG 8b is a cross-section view of a digit tip texture simulator where a tendon pulls on the L-shaped bracket 804, rotating it counter clockwise. As it rotates, the bracket pushes on the texture element which then extends from the digit tip texture simulator enclosure and presses against the digit tip. When tendon tension is removed, die spring 805 returns the texture element to its original, unextended position.

FIG. 8c is a cross-section view of a digit tip texture simulator when either pneumatics or hydraulics arc employed. A positive pneumatic or hydraulic pressure extends the texture element and a negative pressure retracts it

HG. 8d is a cross-section view of a digit tip texture simulator where another type of pneumatic actuator is used. When actuated, air enters the device and exits through the nozzle 806. This focused air stream creates a tactile sensation on the digit tip.

SUBSTITUTE SHEET HG. 8e is a cross-section view of a digit tip texture simulator where a tendon 807 pulls on the bar 808 causing it to pivot The pivot may eidier be a hinge with a return spring or a living hinge 809 (as shown). A texture element 810 is attached to the bar which protrudes from the enclosure and presses against the digit tip when die bar pivots.

HG. 8f is a cross-section view of a digit tip texture simulator where a tendon 811 pulls on a wedge 812 causing it to slide underneath and raise the texture element 813.

When tendon force is released, die spring 814 returns the wedge to its initial position.

HG. 8g is a cross-section view of a digit tip texture simulator where a tendon 815 pulls on the middle hinge 816 of die linkage 817, as shown, and raises the texture element 818. When tendon force is released, die spring 819 returns the hinge to its initial position.

HG. 8h functions similarly to HG. 8g, but the hinges and spring are replaced by a flexible beam 820. The beam is initially curved, as shown. When a tendon force is applied, die beam straightens, forcing the texture element up.

' HG. 8i is a cross-section view of a digit tip texture simulator where the texture element is raised by generating a pressure by heating eidier vapor, liquid or a combinaticxi of die two 821. Current is passed through die resistive heating coil 822, causing the vapor (or liquid) to heat up and expand and raise the texture element

HG. 8j is a cross-section view of a digit tip texture simulator where the texture element is raised by piezoelectric elements. A voltage applied to a piezoelectric element causes it to either expand or contract depending on the voltage polarity. In the figure, there are two separate pieces of piezoelectric material connected to form a "bimorph". The two element are wired with opposite polarities such that when a single voltage is applied, one piezoelectric element 823 expands while the otiier element 824 contracts. When one expands and die other contracts, the bimorph bends towards the direction of the element which contracts. A texture element 825 is attached to die free end of die bimoiph and protrudes from the enclosure when he bimorph bends.

HG. 8k is a cross-section view of a digit tip texture simulator where a texture element 826 acts as the plunger of a electromechanical solenoid. As current is applied to die coil 827, the texture element is raised. A spring 828 returns the texture element to its initial position when die current is removed. HG. 81 is a cross-section view of a digit tip texture simulator where a flexible, relatively incompressible fiber 829 (similar to a fiber optic wire) is used. The fiber resides in a flexible, but incompressible outer casing 830 (similar to the tendon casing assembly). The fiber transfers displacement generated at one location (possibly by a bulky or heavy displacement actuator) to a second location (e.g., the digit tip) by sliding relative to die outer casing. The principle of operation is similar to a catheter tube. The end of the fiber is die actual texture element which protrudes and presses against the digit tip. The difference between this "fiber" method and die tendon method is that die tendon is "active" in tension while the fiber is "active" in compression.

HG. 8m is a cross-section view of a digit tip texture simulator where a magnetic attraction, in this embodiment generated by electromagnet 834, pulls on die metal bar 832 causing it to pivot The pivot may either be a hinge with a return spring or a living hinge 831 (as shown). A texture element 833 is attached to die bar which protrudes from the enclosure and presses against the digit tip when die bar pivots. This texture simulator embodiment can be realized witii m aomotor/microactuator technology.

In the embodiments shown in HGS. 8i, j, k and m, the actuation displacement for the texture simulator is generated in die digit tip force applicator enclosure itself. Any of . these same actuator technologies may be employed, but positioned at an alternate location (e.g., on die wristband or at the same place as the force actuator). The displacement may then be transferred to the digit tip by a tendon or pneumatic/hydraulic tube and used by any appropriate texture simulator.

In addition to the actuator technologies shown in HGS. 8i, j, k and m, other, more standard force and displacement actuators such as electromechanical motors and pneumatic (hydraulic) compressors (pumps) may be used. Shape memory alloys (SMA, e.g., Nickel/Titanium alloys) may also be used to generate the tensile force or displacement of a tendon. SMA wire has die property that it contracts when heated. The wire may be heated simply by passing an electrical current through it

HG. 9 shows how the electrical and mechanical signals propogate through die force/texture feedback control system. HG. 10 is a diagram of the force and texture feedback control system in standard control tiieory block diagram form. The embodiment shown employs a d.c. servo motor 900 for force actuation and an electromechanical solenoid 901 to produce the displacement for a texture simulating element 902. A computer sends a digital value representing the desired force to a d.c. servomotor control circuit In the embodiment shown in HG.9, die desired force is presented to the digital-to-

SUBSTITUTE SHEET analog converter (DAC) 903. The analog output of the DAC is then amplified by a variable gain amplifier 904. This amplified force set point voltage is tiien converted into a current by a common voltage-to-current configuration of a power operational amplifier 905. This current drives the servo motor at a desired torque. Velocity damping of die servocontrolloopisperformedbytachometerfeedback906.

Torque generated by die motor is converted into a tensile force by a pulley 907 on die motor shaft The diameter of this pulley is selected to achieve the desired force and speed of response for a given motor. In a preferred embodiment a pulley diameter of 1/4 inch was used. The generated tensile force is transmitted to die digit tip force applicator from the force actuator via a tendon cable/casing assembly 908. The force applied to die digit tip is sensed by die two strain gages 909 mounted differentially to the strain sensing platform and wired into a half-bridge configuration. A full Wheatstone bridge is used to amplify the detected force. This amplified signal is digitized by an analog-to-digital converter910 andrcadinto the computer 911.

The computer implements a force control law 912 (e.g., Proportional-Integral- Derivative or state feedback) using well understood techniques from the field of digital control. The control law incorporates die feedback force information 913, and servos the motor to produce a desired force at die digit tip. Digitized values 914 from analog joint angle sensors provide the information the computer needs to determine the force set point 915. In a preferred embodiment die computer converts digit joint angles into actual digit positions. If erne of the digits is found to be intersecting a virtual object die computer calculates die fence to be applied to that digit using knowledge of die virtual object's shape and compliance 916. In a preferred embodiment differential strain gage angle sensors 917, as disclosed in the Kramer et aL patent application, are used to determine joint angles.

As shown in HG. 9, the computer also outputs commands to die displacement acmator of the texture simulating array. In the embodiment shown, the computer outputs digital values which control solenoid drive transistors 918. For example, a logical value of "1" turns the transistor "on," and a logical "0" turns the transistor "off." When the transistor is on, the solenoid coil is energized, and die plunger 919 is retracted. The retraction generates a displacement which is transmitted to the texture simulator 902 via a tendon cable casing assembly 920. The texture simulator uses the displacement to extend the texture elements beyond the surface of the digit tip force-applicator platform against die digit tip. When die transistor is turned off, the solenoid plunger is extended by die return spring and cable tension is released. When the tension is released, the texture element is retracted back into the texture array platform housing by its own return mechanism. HGs. lla-lld are functionally similar to HGs.5a-5e in that diey all poses a force- applying means and a force-sensing means. The difference is in die force-sensing means. In HGs. 5, the force-sensing means is shown as a force-sensing platform. In HGs. 11 the force-sensing means is shown to include a load cell The load cell 1100 may employ any of a wide variety of technologies, such as strain gage, capacitive or resistive sensing technologies, and die like. Besides die more common strain gage load cells, force sensor pads which use capacitive sensing technology are discussed in die literature by Fearing and resistive force sensing pads are available commercially by Interlink and TekScan. In HGs. 11_* the force-sensing means comprises part of the force-applying means. The force- sensing/applying structure comprises a platform 1101 which is affixed to support 1102. Support 1102 is connected to die digit tip clip 1103 by spring 1104. Force-transmitting tendon 1105 is affixed to platform 1101. Load cell 1100 is affixed to the digit side of platform 1101. For various reasons, such as when the load cell surface is not rugged or if the load cell is temperature sensative, a protective/temperature insulating platform 1106 is affixed to die digit side of die load cell When the tension in tendon 1105 is increased (HG. lie), platform 1101 presses on the load cell 1100 which in turn presses platform 1106 against die digit tip. The load cell measures the tension in tendon 1105 at die digit tip.

HGs. 12a and 12b are side and plan views of a force-applying platform which is capable of pivoting to make the contact pressure between the platform and the digit tip as uniform as possible. In this embodiment platform 1200 pivots on hinge 1201 which is connected by support 1202 to return spring 1203, which in turn is affixed to digit tip clip 1204. When tension is applyed to tendon 1205, platform 1200 contacts the digit tip and rotates on hinge 1201 to make the contact pressure uniform.

HG. 13 is a side view of an extension of HG. 12, with die addition tiiat die contact pressure distribution between platform 1300 and the digit tip may be modified by adjusting the tension in tendons 1301 and 1302, If the tension in tendon 1301 is greater than in tendon 1302, then the digit tip will detect greater contact force nearer the fingernail dian the bottom of the digit tip.

HGs. 14a and 14b are the side and plan view of yet another embodiment which is used to modify die pressure distribution sensed by the digit tip. In this embodiment, platform 1400 is capable of pivoting in any direction due to the connection to support

1401 via ball joint 1402. By varying the tension in tendons 1403 and 1404. the centroid of pressure may be shifted vertically, whereas varying the tension in tendons

SUBSTITUTE SHEET 1405 and 1406, the centroid of pressure may be shifted laterally. By uniformly varying the tension in all tendons, the magnitude of the pressure distribution may be changed accordingly without shifting the centroid. Although the embodiment provided only shows four tendons in a symmetric pattern, the concept obviously may be expanded to include more tendons and in more complex patterns.

HG. 15 is a side view of an embodiment showing how the tension in die tendon may be measured prior to die platform contacting die digit tip. Platform 1500 is affixed to support 1501 which is attached to to digit tip clip 1502 via flexible elastic member 1503. The extent of flexion of 1503 is a measure of the force applied to platform 1500 by tendon 1506 until die platform contacts the digit tip. With this capability, it can be sensed, among other tilings, when die tendon is slack. In the embodiment shown, the flexion is measured via differential strain gages 1504 and 1505.

HGs. 16a and 16b are side views of two more methods to measure tendon tension, and thus, force applied to die body part i the embodiments provided, die tension is being measured near the force-generating actuator. The same measurement principles may be used to sense tendon tension at the force-sensing body part, for example, at a feedback glove. In HG. 16a, tendon 1600 is wound on pulley 1601 which is in the shaft of force- generating actuator 1602, which in the embodiment provided is a motor. The tendon passes over pulley 1603, under fixed pulley 1604 and enters casing 1605. Pulley 1603 is affixed to the free end of cantilever 1606, while the otiier end of die cantilever is anchored securely. When tendon tension is increased, pulley 1603 is displaced downward, causing the cantilever also to displace downward. In die embodiment provided, this cantilever displacement is measured via differential strain gages 1607 and 1608. Other displacement sensing technologies may be substituted.

HG. 16b shows how die tendon tension may be measured by sensing the stress in the tendon casing. Tendon 1609 leaves the force-generating actuator 1610 and enters a tendon casing stress sensing sleeve 1611. This sleeve is affixed to casing support 1612 at one end, and not connected to anything at the odier end. At the free end, die sleeve presses against a spacer 1613 which then presses against the main section of die tendon casing 1614 which guides die tendon to its destination. The spacer is not connected to anything, but may rest idle on the tendon. Casing 1614 is guided and supported by structure 1615. The stress experienced by stress sensing sleeve 1611 is sensed, in the embodiment provided, by differential strain gages 1 16 and 1 17. The use of spacer 1613 and support 1615 reduces the influence that lateral motion of casing 1614 would otherwise have on the sensed stress. FIGs. 17a and 17b are side views of two embodiments of a structure which supports both a bend sensor (e.g., the strain gage bend sensor of Kramer et al) and a force-transmitting tendon. HG. 17a shows a cross sectional view of an embodiment where bend sensor 1700 is in guiding pocket 1701 in support structure 1702. The support structure is affixed in proximity to the joint whose angle is to be measured, shown in HGs. 17a and 17b to be die PIP joint Force-transmitting tendon 1703 is also supported over the body part by structure 1702. The tendon may reside in a trough or pass through a hole in structure 1702. Structure 1702 should move in relation to die body part during flexure and may be made of a variety of materials including plastic, RTV silicon rubber and die like.

HG. 17b is a side view of a tendon/bend sensor support structure similar to HG. 17a but has portions of material removed 1704 from the structure 1705 to permit easier bending. The dashed line outlines where the bend sensor 1706 may be positioned in the support structure. Although, in both HGs. 17a and 17b, die bend sensor is shown positioned in the support structure between tendon 1707 and die body part, other topologies may be used, such as the tendon between die bend sensor and die body part

HGs. 18a and 18b are a perspective and plan view of an embodiment which provides a pre-tension between a force feedback glove and die casing support wristband. The embodiment provided is a schematic representation and a variety of details may be added to support the functional parts. In this embodiment there are two pulleys mounted on wristband 1800, one on the top 1801, one on die bottom 1802. The pulleys are able to translate in either direction along the axis of the forearm, optionally in a slotted guide, but are pulled in the direction away from the glove by elastic members 1803 and 1804. The pulleys may also be allowed to slide in a direction that is not parallel to, but has a component along the axis of die forearm. The glove is reinforced on both the top 1805 and bottom 1806 (similar to top side reinforcement but not shown). The reinforced sections are connected to each other via pre-tension tendon 1807 which passes over pulley 1801, around the wrist (optionally over a bearing surface such as a series of roller bearings), and over pulley 1802. The reinforced glove sections serve to distribute die pre¬ tension force over the hand. The reinforcement may be extra material such as nylon, plastic or RTV silicon rubber. The wristband is strapped around the wrist at a location diat places the elastic members in tension. The tension serves to draw the wristband toward the glove, without allowing the wristband to slide relative to the skin, and thus taking up the slack in the forearm skin so tiiere is litde motion of the wristband later when a force-transmitting tendon is placed in tension. HG. 19 is the block diagram of a three-loop force control system. The diagram is very similar to HG. 10 with die addition of an inner servo loop that controls the force sensed at the output of the force acmator. This inner servo loop is a "fast loop" which may have a high gain to quickly adjust die force output by the force acmator based on sensing the output force near the force actuator itself. A computing device 1900 which _* has knowledge of, for example, the environment, object shape, position and complance, determines a force set point 1901 for die control system based on additional knowledge of digit tip position which may be sensed by die Kramer et al strain gage bend sensors 1902 or suitable substitute. This force set point is compared to actual force sensed at the digit tip by a suitable sensor 1903 , such as the force-sensing platform or appropriate load cell The error in die force signal is input to die "slow loop" controller 1904 which may be running a standard control law. This is called die slow loop because the gain shouldn't be too high since there are some nonlinear dynamics involved, if the cable force-transmission system 1905 is employed.

The output of the slow loop controller is the force set point 1906 to die "fast loop" control system. This fast loop set point is compared to a force sensed (e.g., by die previously discribed strain gage cantilever 1907 of HG. 16) at the output of the force actuator 1908 which produces the error signal input for the fast loop controller 1909 which also may be running a standard control law. The gain of the fast loop may be large compared to die gain of the slow loop controller since die dynamics of this loop are fairly linear and are relatively fast if a good quality servo motor were used. Therefore, the tension output of the motor can be controlled to a desired value very quickly, whereas the force sensed at the digit tip cannot be servoed to a desired value as quickly without increasing the possibility of oscillation due to the nonlinear transmission system.

By appropriately combining commands to the texture array and die force applicator, innumerous sensations may be applied to the digit tip. For example, by extending tiiree texture elements along a single column and tiien actuating the force platform to press against the digit tip, die sensation of touching the digit tip to die vertical edge of a virtual object is simulated. If the three extended texture elements of the column are retracted at the same time that the tiiree elements of the adjacent column are raised, a sensation that the object edge is moving across the digit tip will be produced. This sensation may be used either when an object edge is moving and die digit tip is remaining stationary, or when the object position is fixed and the digit tip is moving across the edge. With appropriate modifications, force and texture may be simulated at other parts of the body besides die digit tip, such as is shown for the arm in HG.2c. While the invention has been described witii reference to specific embodiments, die description is illustrative of the invention and is not to be construed as limiting the invention. Thus, various modifications and amplifications may occur to those skilled in the art without departing .form the true spirit and scope of the invention as defined by the appended claims.

Claims

CLAIMS What is claimed is

1. A device for producing a signal at a sensing body part simulating the interaction between an interactive entity and a distal virtual or real object said device comprising:

means for generating a force simulating the interaction between said interactive entity and said object; and

means for transmitting said generated force to said sensing body part

2. A device according to Claim 1, wherein said generating means comprises:

means for generating a force simulating die contact force of said interactive entity interacting with said object

3. A device according to Claim 1 , wherein said generating means comprises:

means for producing a surface pattern simulating a surface of said object

4. A device for producing a signal at a sensing body part simulating the interaction between an interactive entity and a distal virtual or real object said device comprising:

means for generating a force simulating the contact force of said interactive entity interacting with said object;

means for applying a force to said sensing body part, displaced from said sensing body part in a first unactivated position and touching said sensing body part in a second activated position;

transmitting means operably connecting said generating means to said applying means.

5. A device according to Claim 4, wherein said applying means comprises: a force-applying platform

a supporting structure for holding said platform in juxtaposition to said sensing body part;

retractable means for holding said force-applying platform in said first position.

6. A device according to Qaim 5, said applying means further comprising:

a force-sensing platform;

a fulcrum mounted on said force-applying platform and supporting said force- sensing platform;

mechanical means connecting said force-sensing platform to said transmitting means, wherein actuation of said force-sensing platform by said transmitting means moves said force-applying platform to said second position.

7. A device according to Qaim 4, said transmitting means comprising:

a force transmitting flexible elongated element

a flexible housing for guiding said element from said force-generating means to said force-applying means;

means for transferring said force from said element to said force-applying means.

8. A device according to Qaim 7, wherein said element comprises an inelastic tendon and said housing is incompressible.

9. A device according to Claim 7, wherein said element comprises an incompressible fluid and said housing is inelastic.

10. A device according to Claim 7, wherein said device further comprises:

force-sensing means attached to said element between said force-generating means and said force-applying means.

11. A device for producing a signal at a sensing body part simulating die surface interaction between an interactive entity and a distal virtual or real object surface, said device comprising:

a plurality of texture elements in a predetermined array;

means for holding said array of texture elements in juxtaposition to said sensing body part

means for generating forces to displace a plurality of texture elements simulating die interaction between said interactive entity and said object surface; and

means for transmitting said generated forces to said texture element array to simulate said object surface to said sensing body part

12. A device according to Claim 11, wherein said array is a 3 x 3 array of texture elements.

1 . A device for producing a signal at a sensing body part simulating the interaction between an interactive entity and a distal virtual or real object said device comprising:

first means for generating a force simulating die contact force of said interactive entity interacting with said object

first transmitting means operably connecting said first generating means to said applying means;

said applying means comprising a plurality of texture elements in a predetermined array in confronting relationship with said sensing body part

second means for generating force to displace a plurality of texture elements simulating the interaction between said interactive entity and said object surface; and second means for transmitting said second generated forces to said texture element array to simulate said object surface to said sensing body part

14. A device according to Qaim 13, wherein said applying means comprises;

a force-applying platform comprising said plurality of texture elements;

a supporting structure for holding said force-applying platform in juxtaposition to said sensing body part;

15. A device according to Claim 14, said applying means further comprising:

a force-sensing platform;

16. A man-machine interface device for producing a first signal to a sensing body part simulating the interaction between an interactive entity and a distal virtual or real object and manipulating said interactive entity in relation to said object said device comprising:

means for generating a force simulating the interaction between said interactive entity and said object

means for transmitting said generated force to said sensing body part as said first signal;

means for sensing position of a controlling body part and producing a second signal related to said position of said controlling body par

SUBSTITUTE SHEET signal collection and producing means for receiving said second signal controlling the interaction between said interactive entity and said object in relation to said second signal and actuating said generating means to produce said first signal in relation to the interaction of said interactive entity and said object

17. A man-machine interface according to Qaim 16,

wherein said generating means comprises first means for applying a fence to said sensing body part, displaced from said sensing body part in a first unactivated position and touching said sensing body part in a second activated position;

said transmitting means comprises applying means comprising a platform comprising a plurality of texture elements in a predetermined array in confronting relationship with said sensing body part; and further comprising:

second means for generating forces to displace a plurality of texture elements simulating the interaction between said interactive entity and said object surface; and

second means for transmitting said second generated forces to said texture element array to simulate said object surface to said sensing body part

18. A man-machine interface device for producing a first signal to a digit part simulating the interaction between an interactive entity and a distal virtual or real object and manipulating said interactive entity in relation to said object, said device comprising:

a support for attaching to said digit part;

a force applying means affixed to said support;

means for generating a force simulating die interaction between said interactive entity and said object

means for transmitting said generated force to said digit part as said first signal;

means for sensing position of a controlling digit part and producing a second signal related to said position of said controlling digit part

signal collection and producing means for receiving said second signal, controlling the interaction between said interactive entity and said object in relation to said second signal and actuating said generating means to produce said first signal in relation to the interaction of said interactive entity and said object

19. A man-machine interface device according to Claim 18, wherein said support is a glove, said digit part is die digit tip, said transmitting means comprising a housing and a force-transmitting flexible elongated element guided by said housing, and further comprising:

a guide member attached to said glove for directing said force transmitting element

awriststrap; and

said housing attached at one end to said wriststrap and at die other end attached proximal to said force generating means.

20. A man-machine interface device according to Claim 19, wherein said applying means comprises:

a force-applying platform movable from a first unactivated position to a second activated position in contact with said digit tip;

a supporting structure for holding said force applying platform in juxtaposition to said digit tip;

retractable means for holding said force-applying platform in said first position;

a force-sensing platform;

mechanical means connecting said force-sensing platform to said transmitting means, wherein acmation of said force-sensing platform by said transmitting means moves said force-applying platform to said second position.