US20110181547A1 - Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen - Google Patents

Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen Download PDF

Info

Publication number
US20110181547A1
US20110181547A1 US13/013,985 US201113013985A US2011181547A1 US 20110181547 A1 US20110181547 A1 US 20110181547A1 US 201113013985 A US201113013985 A US 201113013985A US 2011181547 A1 US2011181547 A1 US 2011181547A1
Authority
US
United States
Prior art keywords
physical model
finger
objects
graphical
tactile
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/013,985
Inventor
Pascal Joguet
Guillaume Largillier
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Stantum SAS
Original Assignee
Stantum SAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=34834225&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=US20110181547(A1) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Application filed by Stantum SAS filed Critical Stantum SAS
Priority to US13/013,985 priority Critical patent/US20110181547A1/en
Publication of US20110181547A1 publication Critical patent/US20110181547A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/045Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using resistive elements, e.g. a single continuous surface or two parallel surfaces put in contact
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0354Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/047Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means using sets of wires, e.g. crossed wires
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04847Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04886Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10HELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00Details of electrophonic musical instruments
    • G10H1/02Means for controlling the tone frequencies, e.g. attack or decay; Means for producing special musical effects, e.g. vibratos or glissandos
    • G10H1/04Means for controlling the tone frequencies, e.g. attack or decay; Means for producing special musical effects, e.g. vibratos or glissandos by additional modulation
    • G10H1/053Means for controlling the tone frequencies, e.g. attack or decay; Means for producing special musical effects, e.g. vibratos or glissandos by additional modulation during execution only
    • G10H1/055Means for controlling the tone frequencies, e.g. attack or decay; Means for producing special musical effects, e.g. vibratos or glissandos by additional modulation during execution only by switches with variable impedance elements
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10HELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00Details of electrophonic musical instruments
    • G10H1/32Constructional details
    • G10H1/34Switch arrangements, e.g. keyboards or mechanical switches specially adapted for electrophonic musical instruments
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/041Indexing scheme relating to G06F3/041 - G06F3/045
    • G06F2203/04104Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10HELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2220/00Input/output interfacing specifically adapted for electrophonic musical tools or instruments
    • G10H2220/091Graphical user interface [GUI] specifically adapted for electrophonic musical instruments, e.g. interactive musical displays, musical instrument icons or menus; Details of user interactions therewith
    • G10H2220/096Graphical user interface [GUI] specifically adapted for electrophonic musical instruments, e.g. interactive musical displays, musical instrument icons or menus; Details of user interactions therewith using a touch screen
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10HELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H2220/00Input/output interfacing specifically adapted for electrophonic musical tools or instruments
    • G10H2220/155User input interfaces for electrophonic musical instruments
    • G10H2220/221Keyboards, i.e. configuration of several keys or key-like input devices relative to one another
    • G10H2220/241Keyboards, i.e. configuration of several keys or key-like input devices relative to one another on touchscreens, i.e. keys, frets, strings, tablature or staff displayed on a touchscreen display for note input purposes

Definitions

  • the invention relates to musical controllers, particularly to an interface permitting, e.g., the control of music software or of a controller by a multi-contact tactile screen with the manipulation of virtual objects.
  • Manual-type software controllers are known. They include, e.g., potentiometers that can be manipulated by the user in the form of a console and control the different functions of music software. Such a console is disclosed in WO 01/69399.
  • WO 03/041006 and U.S. Pat. No. 6,570,078 disclose musical controllers with tactile control on a matrix sensor.
  • the technologies described therein permit tactile control of the multi-contact type in which all the fingers can intervene for the control of software.
  • US 2002/005108 discloses a system and a process for controlling in real time signal processors, synthesizers, musical instruments, MIDI processors, lights, video, and special effects during presentations, recordings or in compositional environments using images derived from tactile sensors, from matrices of pressure sensors, from matrices of optical transducers, from matrices of chemical sensors, matrices of body sensors and from digital processes.
  • That system furnishes touchpads, matrices of pressure sensors and matrices of body sensors as interfaces of tactile control, video cameras and matrices of light sensors such as optical transducers, matrices of chemical sensors and of other apparatuses for generating digital images from processes on computers or from digital simulations.
  • the tactile transducers can be arranged on the keys of conventional instruments, attached to existing instruments or also be used to create new instruments or new controllers.
  • the matrices of chemical sensors and the other apparatuses for generating digital images from computer processes or from digital simulations can be used to observe or simulate natural physical phenomena such as environmental conditions or self-organizing process behaviors.
  • Scalar matrices or vectors are processed to extract pattern limits, geometric properties of pixels within limits (geometric center, weighted moments, etc.) and information derived from a higher level (direction of rotation, segmented regions, pattern classification, syntax, grammars, sequences, etc.) that are used to create control signals to external video and visual equipment and for control or even algorithms. It also provides MIDI and non-MIDI control signals.
  • U.S. Pat. No. 5,027,689 discloses an apparatus for generating musical sounds. That apparatus comprises a device for generating positional information for generating information about the position of musical instruments (PS) as values of plane coordinates. This information (PS) is stored in a memory device or determined in a selective manner by a manual operation. The apparatus also comprises a device for the conversion of information for converting the information (PS) into information for controlling parameters of musical sounds (PD). This PD control information controls the source signals of musical sounds (S 11 , S 12 and S 13 ) for generating a sound field corresponding to the position of musical instruments arranged on a stage. This allows an operator to verify the positions of musical instruments on a stage, thus supplying the sensation of being in a true live performance.
  • the apparatus only functions linearly for the multipoint option and does not allow tracking (following of trajectory). Moreover, the apparatus requires a plurality of sensors specific to each of the instruments.
  • U.S. Pat. No. 5,559,301 discloses a solution of the musical controller type in the form of a tactile screen with visual return of the manipulated objects.
  • predefined objects essentially of the sliders type and circular potentiometer type.
  • object types are limiting and can prove to be not very ergonomic for special manipulations.
  • the acquisition mode described is not in real time. In fact, an icon must first be activated by a first contact with a finger, then the manipulated object, and the values are only updated after the icon has been released. That system does not allow management in real time of the parameters associated with the object.
  • the tactile sensor is a “mono-contact” sensor that permits the acquisition, e.g., only for a single finger and therefore the control of a single object at a time. This characteristic is very limiting for an efficient manipulation of objects.
  • This invention relates to a process for controlling computerized equipment with a device including a multi-contact bidimensional sensor that acquires tactile information and a calculator that generates command signals as a function of the tactile information, including generating graphical objects on a screen placed under a transparent multi-contact tactile sensor, each graphical object associated with at least one specific processing rule such that the sensor delivers during each acquisition phase a plurality of tactile information, and each piece of the tactile information forms an object of a specific processing determined by its localization relative to a position of one of the graphical objects.
  • This invention also relates to a device for controlling computerized equipment including a multi-contact bidimensional sensor for acquisition of tactile information, a viewing screen arranged under the bidimensional tactile sensor, a memory for recording graphical objects that are each associated with at least one processing rule, and a local calculator that analyzes positions of acquired tactile information and applies a processing rule as a function of the position relative to the position of the graphical objects.
  • FIG. 1A is a functional diagram of a controller
  • FIG. 1B represents the structure of the controller associated with the functional diagram
  • FIG. 1C represents the functional diagram of the different stages of the processes for the acquisition of data coming from the sensor, of the creation of cursors associated with the different fingers, of the interaction with the graphical objects and of the generation of control messages;
  • FIG. 2A is a description of the tactile matrix sensor
  • FIG. 2B describes the first stage of the scanning functioning of the sensor in order to obtain the multi-contact information
  • FIGS. 2C , 2 E and 2 F explain the resolution of problems of orthogonality
  • FIG. 2D is a functional diagram of the capture interface
  • FIGS. 3A to 3D are diagrams explaining the stages for the creation of cursors, filtering, calculation of barycenter, mapping and of the control of graphical objects;
  • FIGS. 4 and 5 represent different examples of graphical objects
  • FIGS. 6 to 10 represent different examples of combinations of graphical objects on the controller.
  • FIG. 11 illustrates the network use of the controller associated with the computer of the user.
  • multi-contact defines a tactile sensor that allows acquisition of contact zones of several fingers at a time in contrast to “mono-contact” sensors that only allow acquisition for a single finger or for a stylus as, e.g., in U.S. Pat. No. 5,559,301.
  • We also provide a process for controlling computerized equipment with a device comprising a multi-contact bidimensional sensor for the acquisition of tactile information as well as calculating means generating command signals as a function of this tactile information, and a stage for generating graphical objects on a screen placed under a transparent multi-contact tactile sensor, each of which graphical objects is associated with at least one specific processing law, wherein the sensor delivers during each acquisition phase a plurality of tactile information, and each piece of the tactile information forms the object of a specific processing determined by its localization relative to the position of one of these graphical objects.
  • the process steps may comprise a bounding zone detection of the contact zone of an object with the tactile sensor.
  • the process may also comprise a barycenter detection.
  • It may further comprise stages for refreshing graphical objects as a function of the process carried out during at least one previous acquisition stage.
  • the process may comprise a stage for editing graphical objects including generating a graphical representation from a library of graphical components and functions and determining an associated processing law.
  • the acquisition frequency of the tactile information may be greater than 50 Hz.
  • a device for controlling a computerized piece of equipment comprising a multi-contact bidimensional sensor for acquiring tactile information, a viewing screen arranged under the bidimensional tactile sensor, a memory for recording graphical objects that are each associated with at least one processing law, and a local calculator for analyzing the position of acquired tactile information and application of a processing law as a function of the position relative to the position of the graphical objects.
  • the device may be connected to a hub (multi-socket network) to form a network of controllers.
  • a hub multi-socket network
  • This multi-contact bidimensional tactile sensor is advantageously a resistive tile.
  • the device may comprise a network output suitable for receiving a network cable.
  • control is performed on a computerized piece of equipment that can be, e.g., a music software, a controller, audiovisual equipment or multimedia equipment.
  • the first basic element is the matrix sensor 101 necessary for acquisition (multi-contact manipulations) with the aid of a capture interface 102 .
  • the sensor 101 may be divided, if necessary, into several parts to accelerate capture, with each part being scanned simultaneously.
  • the general principle is to create as many cursors (such as a mouse cursor) as there are zones detected on the sensor and to follow their developments in time.
  • the sensor may be a resistive tactile matrix tile of a known type.
  • Resistive tactile matrix tiles are composed of 2 superposed faces on which tracks of ITO (indium tin oxide), that is a translucent conductive material, are organized.
  • the tracks are laid out in lines on the upper layer and in columns on the lower layer and a matrix as shown in FIG. 2A .
  • the two conductive layers are insulated from one another by spacing braces.
  • the intersection of the line with the column forms a contact point.
  • a column or columns situated on the upper layer are put in contact with a line or line situated on the lower layer, thus creating one or several contact points as shown in FIG. 2B .
  • braces by a transparent resistive material (e.g., a conductive polymer) whose resistance varies as a function of the pressure, which resistance drops if a sufficient pressure force is exerted. In this manner, it is also possible to extract the pressure exerted on the surface by performing a resistance measurement at each line-column intersection.
  • a transparent resistive material e.g., a conductive polymer
  • the state of the tile is measured at least 100 times per second, which tile can be divided into several zones to perform a parallel processing on these zones.
  • the sampling frequency of the tile may be at least 100 Hz.
  • Another basic element is the electronic device for scanning the tactile tile that allows the simultaneous detection of several contact points on the matrix sensor.
  • the known methods of acquisitions for this type of sensor do not allow the detection of several simultaneous contact points.
  • the columns are fed, e.g., at 5V in turn and the level of the lines (high or low level) measured sequentially.
  • column 1 is fed at first while the other columns are in high impedance.
  • the lines are measured sequentially, that is, one after the other.
  • the value on the first line is read initially while all the other lines are connected to ground. Then, line 1 is connected to ground and the value on line 2 is read and so forth until the value of all the lines has been read.
  • the scanning is performed in this manner up to the last column.
  • the total scanning of the matrix is carried out at an elevated frequency to obtain the value of each of the intersection points of the tile several times per second.
  • FIG. 2D representing the algorithm of the acquisition of a tile comprising 100 lines (L) and 135 columns (C).
  • the resistance of the transparent material (ITO) composing the columns and the lines increases proportionately to the length of the tracks.
  • the potential measured at the lower left corner of the sensor will be greater than the potential measured at the upper right corner.
  • the cloud of points absorbs a large part of the electrical potential of the fed column.
  • the potential measured at the isolated point is therefore too low to be detected.
  • a solution to this problem is in using a voltage comparator piloted digitally at the output of the line to determine whether the tension observed is sufficient for being considered as resulting from the action of a finger on the tactile tile.
  • the reference value of the comparator (comparison threshold) is decremented at each line measure.
  • the comparison values of the last lines are lower than those of the first lines, which allows the contact point located at the lower left or the upper right to be detected in the same manner.
  • a complete sampling of the tile is performed at least 100 times per second for the columns and the lines.
  • the data from capture interface 102 thus form an image representative of the totality of the sensor. This image is placed in memory so that a program can proceed to the filtering, the detection of the fingers and to the creation of the cursors as seen in FIG. 1 .
  • the filtering phase illustrated by FIG. 3B eliminates noise that might be generated by the acquisition interface or the sensor itself. It is considered that only the clouds of several contact points can correspond to the pressure of a finger. Therefore, a bounding zone detection is carried out to eliminate isolated contact points.
  • the following stage associates a cursor with each support point ( FIG. 3C ). To this end, the barycenter of each bounding zone is calculated. When a finger is released, the corresponding cursor is freed.
  • the program executed locally by the main processor allows these cursors to be associated with graphical objects that are displayed on screen 105 to manipulate them.
  • the local program uses these cursors for generating control messages addressed to the host computer or the controlled apparatus.
  • the program comprises a simulator of the physical models allowing modification of the interaction laws between the cursors and the graphical objects.
  • Different physical models can be employed: spring-loaded system, vibration of a string, management of collisions, the law of gravity, electromagnetic field and the like.
  • the program considers the positioning of the cursors and on which graphical object each is located.
  • a specific processing is supplied to the data coming from the sensor as a function of the object considered. For example, a pressure measurement (corresponding to a development of the spot made by the finger on the tactile tile in a short interval of time) can be interpreted.
  • Other parameters can be deduced as a function of the nature of the object: the acceleration, speed, trajectories, etc.
  • Algorithms of recognition of form can also be applied to differentiate different fingers.
  • the main program 103 also transmits the data to be displayed on screen 105 to graphical interface 104 .
  • this graphical interface is constituted of a graphical processor.
  • This graphical processor is, e.g., of a known type. The latter can be constituted of primitive graphical functions allowing, e.g., the displaying of bitmap, fonts of polygons and figures in 2 and 3 dimensions, vectorial design, antialiasing, texture mapping, transparency and interpolation of colors.
  • the main program may also comprise an analyzer of mathematical expressions that allows mathematical functions to be inputted and calculated in real time. These functions allow the values of any variable to be modified. For example, the coordinates (x, y) of a cursor inside an object can be considered as two variables comprised between 0 and 1.
  • the expression analyzer allows an expression of the type “x*1000+600” to be created to obtain a new variable whose value is comprised between 600 and 1600. The variable obtained allows the control, e.g., of the frequency of an oscillator comprised between 600 and 1600 hertz.
  • the expression analyzer is a tool that allows real-time calculations to be performed on the variables of objects.
  • Local program 103 also performs a formatting of data in the form of messages for network port 106 , that communicates it to the computer on which the computer applications are performed.
  • the network interface is, e.g., an Ethernet 10/100 baseT standard interface that communicates by packets with the protocol TCP/IP. It can also be a network interface of the wireless type.
  • Ethernet connection offers the user the possibility, by using a simple hub (multi-socket network), of indefinitely expanding the control apparatus by constituting a network of controllers.
  • the controller or controllers present in the network then communicate among themselves and with the host computer in the form of the reciprocal sending of messages.
  • the unit constituting the machine is fed by a battery (not shown) of a known type or by an AC adapter.
  • an interface editor 107 at the level of the computer of the user allows the interface, that is, the totality of the graphic objects displayed on screen 105 , to be programmed in a graphical manner.
  • the interfaces may themselves be organized in scenes, that are higher hierarchical structures. In fact, each scene comprises several interfaces. The user can interchange the interfaces with the aid of a button keyboard or a control pedal board connected to input-output port 109 .
  • Another function of the interface editor is to assign the control data to the parameters that the user wishes to control.
  • the user has at the user's disposal, e.g., a library of parameterable graphical objects allowing the composition of different interfaces according to the application desired.
  • FIGS. 4 and 5 represent different graphical objects placed at the disposition of the user.
  • a linear potentiometer 403 , 404 is particularly adapted to control continuous parameters such as the volume of a sound signal, the frequency of a filter.
  • a serrated wheel 401 can serve, e.g., to control the playing of an audio or video reader.
  • the objects can also be freely developed with a development kit (SDK) of a known type 109 .
  • SDK development kit
  • the development kit furnishes access to the primitive graphical functions of the controller.
  • Interface editor 107 thus allows the user to readily create personalized control interfaces. It is a software executed on the user's computer. It is composed of a main window representing the tactile surface of the tile on which graphical objects from a library of proposed objects can be placed. The manipulation and placing of objects on the surface are performed, e.g., with the mouse. The object placed on the window is displayed at the same time on the controller and the object is recorded in a memory of the controller. It can subsequently move or re-dimension the objects at its convenience.
  • a button 402 can also act as a switch or as a trigger.
  • a pressure measurement can optionally be performed.
  • Another example of a parameterable object is area 2D ( 503 , 544 ) of which the principle includes moving pawns inside a delimited zone. The number of pawns present in area 2D is a parameterable option.
  • the area can be configured in uniplan mode, a mode in which the pawns enter into collision with each other, or multi-plan, a mode in which the pawns are placed on distinct superposed planes.
  • Physical parameters can also be configured: the coefficient of friction of the pawns on the plane, the rebound and the attraction of the pawns on the edges and among themselves.
  • the editor also permits the objects present on the surface to be listed and the creation of functions and of variables with the expression analyzer.
  • the objects have by default a certain number of variables (x, y, z . . . ) corresponding to their primitive axes. These variables are always comprised between 0 and 1 and vary in the form of 32-bit numbers with floating comma. The user must be able to “connect” these variables to other values more representative of what he desires to control.
  • Status display options are also desired. They permit a visual control of the state of a parameter.
  • the further treatments to be applied to the objects at the level of the main calculating unit 103 by the manipulation on the tile are specific to each type of object.
  • a circular movement of the finger on a virtual linear potentiometer should not have an effect on the state of the potentiometer whereas it should modify the state in the case of a circular potentiometer 401 .
  • certain objects can only take into account a single finger (the linear potentiometer, for example) at a time whereas others can accept the interaction of several fingers (keyboard, area 2D).
  • the “area 2D” ( 503 , 504 ) is a rectangular surface containing a certain number of pawns, each with its own position. The pawns can be moved by the user.
  • the principle is to put in place a physical system for the totality of the objects, that is, e.g., that the pawns moved by the user acquire a speed of inertia that they retain when the user lets them go; the pawns subjected in this manner to their own speed will rebound on the edges of “area 2D” and also rebound among themselves. Furthermore, they will be subjected to forces of attraction/repulsion on the edges and on the other pawns as well as to a coefficient of friction on the surface of area 2D for stopping the pawns at the end of a certain time. All these parameters will be parameterable.
  • area 2D includes applying a physical law of the “spring-loaded” type.
  • a virtual rubber band is stretched between each cursor and each pawn. The user can modify the behavior of this object by configuring the friction and the interpolation factor. These properties can also be modified in real time with the aid of other objects.
  • Multislider 501 a table of cursors whose numbers can be configured.
  • the typical use is the controlling of a graphic equalizer or of a spectral envelope.
  • the difference between a “multislider” and several simple juxtaposed linear potentiometers is that the totality of the cursors can be modified in a single touch by sliding the finger.
  • the multislider can also be used as a discrete string. For this, it is sufficient to apply to it the physical model of a string whose tension is parameterable by the user.
  • FIGS. 6 to 9 A visualization of different examples of interfaces uniting different types of objects is illustrated by FIGS. 6 to 9 , in which several objects described above can be observed.
  • FIG. 6 shows an arrangement of 6 areas 2D ( 601 ) containing 1 pawn each.
  • This interface could control, e.g., six different filters assigned to one or several sound sources.
  • the abscissa movement of each pawn in each zone controls the frequency of the filter whereas the ordinate movement controls the quality factor or the width of the filter band.
  • FIG. 7 shows an example of the control of a synthesizer or of a sampler of a known type.
  • the interface is composed by a tempered keyboard 704 controlling the pitch of the sounds, by a group of four vertical potentiometers 703 allowing the control, e.g., of its dynamic envelope (attack time, hold level, release time).
  • An area 2D ( 701 ) containing 3 pawns allows the control, e.g., of effects applied to the sound (reverberation, echo, filters).
  • a matrix of 16 buttons 792 can, e.g., release 16 different recorded musical sequences or also call up 16 prerecorded configurations of the previously described controls.
  • FIG. 8 Another example is illustrated by FIG. 8 showing the control of a device for the broadcasting of different sound sources into space on a device constituted by several loudspeakers.
  • an area 2D ( 801 ) representing the broadcasting space contains 4 pawns 801 corresponding to four sound sources.
  • Area 2D also contains 5 icons 802 representing the position of five loudspeakers.
  • the level and/or the phase of each sound source relative to each enclosed space is regulated by moving the different pawns 802 , which determines its emplacement in the space.
  • a group of four linear potentiometers 803 allows the relative level of each source to be regulated.
  • a unit of four buttons 804 allows each sound source to be activated or deactivated.
  • FIG. 9 shows the control of a synthesizer or a sound generator according to a configuration different from that shown in FIG. 7 .
  • the frequency of the sound generator is controlled by four virtual strings 903 .
  • the initial tension (the pitch) of each string can itself be controlled, e.g., by a linear potentiometer 902 .
  • An area 2D 10 e.g., control other parameters of the sound generator such as the output level, the sound quality, the panning, etc.
  • FIG. 10 shows the control of equipment for audio and/or video editing of a known type.
  • a serrated wheel 1001 allows the rate of reading the audio and/or video sources to be controlled.
  • Status display object 1002 allows the positioning of the reading to be represented according to a format (hour, minute, second, image) of a known type.
  • a set of buttons 1003 allows access to the functions of reading and editing of the controlled apparatus.

Abstract

A device configured to acquire tactile information, including a transparent tactile matrix sensor; a display device; a measurement unit configured to simultaneously measure a first position of a first finger and a second position of a second finger that are placed on the transparent tactile matrix sensor; a graphical processing unit configured to generate a first and a second cursor that are associated with the first and second position, respectively; and a processor configured to execute an application that associates the first and second cursors with a graphical object that is displayed on the display device, to interact with the graphical object.

Description

    RELATED APPLICATION
  • This application is a divisional of U.S. patent application Ser. No. 10/590,306 filed Nov. 6, 2006, the entire contents being incorporated herein by reference. This application is being filed concurrently with the following present applications: U.S. patent application Ser. No. ______ (Attorney Docket No. 373691US41DIV); U.S. patent application Ser. No. ______ (Attorney Docket No. 373699US41DIV); and U.S. patent application Ser. No. ______ (Attorney Docket No. 373715US41DIV), all are filed on even date herewith. This application is also a §371 of International Application No. PCT/FR2005/000428, with an international filing date of Feb. 23, 2005 (WO 2005/091104 A2, published Sep. 29, 2005), which is based on French Patent Application No. 04/50329, filed Feb. 23, 2004, the entire contents being incorporated herein by reference.
  • TECHNICAL FIELD
  • The invention relates to musical controllers, particularly to an interface permitting, e.g., the control of music software or of a controller by a multi-contact tactile screen with the manipulation of virtual objects.
  • BACKGROUND
  • Manual-type software controllers are known. They include, e.g., potentiometers that can be manipulated by the user in the form of a console and control the different functions of music software. Such a console is disclosed in WO 01/69399.
  • One disadvantage of this type of controller is that they are not very ergonomic for an efficient manipulation of software. One thought has been to implement a tactile screen for the manipulation of and the access to software functions.
  • In the area of tactile controllers, WO 03/041006 and U.S. Pat. No. 6,570,078 disclose musical controllers with tactile control on a matrix sensor. The technologies described therein permit tactile control of the multi-contact type in which all the fingers can intervene for the control of software.
  • However, those publications do not contemplate a visual return of the manipulations since the different matrix sensors are of the opaque type.
  • US 2002/005108 discloses a system and a process for controlling in real time signal processors, synthesizers, musical instruments, MIDI processors, lights, video, and special effects during presentations, recordings or in compositional environments using images derived from tactile sensors, from matrices of pressure sensors, from matrices of optical transducers, from matrices of chemical sensors, matrices of body sensors and from digital processes. That system furnishes touchpads, matrices of pressure sensors and matrices of body sensors as interfaces of tactile control, video cameras and matrices of light sensors such as optical transducers, matrices of chemical sensors and of other apparatuses for generating digital images from processes on computers or from digital simulations. The tactile transducers can be arranged on the keys of conventional instruments, attached to existing instruments or also be used to create new instruments or new controllers. The matrices of chemical sensors and the other apparatuses for generating digital images from computer processes or from digital simulations can be used to observe or simulate natural physical phenomena such as environmental conditions or self-organizing process behaviors. Scalar matrices or vectors are processed to extract pattern limits, geometric properties of pixels within limits (geometric center, weighted moments, etc.) and information derived from a higher level (direction of rotation, segmented regions, pattern classification, syntax, grammars, sequences, etc.) that are used to create control signals to external video and visual equipment and for control or even algorithms. It also provides MIDI and non-MIDI control signals.
  • It does not contemplate a visual return of manipulations and does not mention a command law. Finally, it does not contemplate technical solution to the masking phenomena that intervene when several figures are aligned or placed in an orthogonal manner on the sensor. The resolution of these problems is indispensable for realizing a multi-contact tactile sensor.
  • U.S. Pat. No. 5,027,689 discloses an apparatus for generating musical sounds. That apparatus comprises a device for generating positional information for generating information about the position of musical instruments (PS) as values of plane coordinates. This information (PS) is stored in a memory device or determined in a selective manner by a manual operation. The apparatus also comprises a device for the conversion of information for converting the information (PS) into information for controlling parameters of musical sounds (PD). This PD control information controls the source signals of musical sounds (S11, S12 and S13) for generating a sound field corresponding to the position of musical instruments arranged on a stage. This allows an operator to verify the positions of musical instruments on a stage, thus supplying the sensation of being in a true live performance.
  • It mentions a multi-contact, but it is only two contacts on an axis and not in Cartesian coordinates. The apparatus only functions linearly for the multipoint option and does not allow tracking (following of trajectory). Moreover, the apparatus requires a plurality of sensors specific to each of the instruments.
  • U.S. Pat. No. 5,559,301 discloses a solution of the musical controller type in the form of a tactile screen with visual return of the manipulated objects. However, it describes predefined objects (essentially of the sliders type and circular potentiometer type). These object types are limiting and can prove to be not very ergonomic for special manipulations. Moreover, the acquisition mode described is not in real time. In fact, an icon must first be activated by a first contact with a finger, then the manipulated object, and the values are only updated after the icon has been released. That system does not allow management in real time of the parameters associated with the object. Finally, the tactile sensor is a “mono-contact” sensor that permits the acquisition, e.g., only for a single finger and therefore the control of a single object at a time. This characteristic is very limiting for an efficient manipulation of objects.
  • SUMMARY
  • This invention relates to a process for controlling computerized equipment with a device including a multi-contact bidimensional sensor that acquires tactile information and a calculator that generates command signals as a function of the tactile information, including generating graphical objects on a screen placed under a transparent multi-contact tactile sensor, each graphical object associated with at least one specific processing rule such that the sensor delivers during each acquisition phase a plurality of tactile information, and each piece of the tactile information forms an object of a specific processing determined by its localization relative to a position of one of the graphical objects.
  • This invention also relates to a device for controlling computerized equipment including a multi-contact bidimensional sensor for acquisition of tactile information, a viewing screen arranged under the bidimensional tactile sensor, a memory for recording graphical objects that are each associated with at least one processing rule, and a local calculator that analyzes positions of acquired tactile information and applies a processing rule as a function of the position relative to the position of the graphical objects.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The disclosure will be better understood with the aid of the following description given below solely by way of explanation of a selected, representative example with reference made to the attached figures in which:
  • FIG. 1A is a functional diagram of a controller;
  • FIG. 1B represents the structure of the controller associated with the functional diagram;
  • FIG. 1C represents the functional diagram of the different stages of the processes for the acquisition of data coming from the sensor, of the creation of cursors associated with the different fingers, of the interaction with the graphical objects and of the generation of control messages;
  • FIG. 2A is a description of the tactile matrix sensor;
  • FIG. 2B describes the first stage of the scanning functioning of the sensor in order to obtain the multi-contact information;
  • FIGS. 2C, 2E and 2F explain the resolution of problems of orthogonality;
  • FIG. 2D is a functional diagram of the capture interface;
  • FIGS. 3A to 3D are diagrams explaining the stages for the creation of cursors, filtering, calculation of barycenter, mapping and of the control of graphical objects;
  • FIGS. 4 and 5 represent different examples of graphical objects;
  • FIGS. 6 to 10 represent different examples of combinations of graphical objects on the controller; and
  • FIG. 11 illustrates the network use of the controller associated with the computer of the user.
  • DETAILED DESCRIPTION
  • The term “multi-contact” defines a tactile sensor that allows acquisition of contact zones of several fingers at a time in contrast to “mono-contact” sensors that only allow acquisition for a single finger or for a stylus as, e.g., in U.S. Pat. No. 5,559,301.
  • We provide a screen for multi-contact tactile musical control with visual return of the different actions of the user on parameterable objects.
  • We also provide a process for controlling computerized equipment with a device comprising a multi-contact bidimensional sensor for the acquisition of tactile information as well as calculating means generating command signals as a function of this tactile information, and a stage for generating graphical objects on a screen placed under a transparent multi-contact tactile sensor, each of which graphical objects is associated with at least one specific processing law, wherein the sensor delivers during each acquisition phase a plurality of tactile information, and each piece of the tactile information forms the object of a specific processing determined by its localization relative to the position of one of these graphical objects.
  • The process steps may comprise a bounding zone detection of the contact zone of an object with the tactile sensor.
  • The process may also comprise a barycenter detection.
  • It may further comprise stages for refreshing graphical objects as a function of the process carried out during at least one previous acquisition stage.
  • The process may comprise a stage for editing graphical objects including generating a graphical representation from a library of graphical components and functions and determining an associated processing law.
  • The acquisition frequency of the tactile information may be greater than 50 Hz. We also provide a device for controlling a computerized piece of equipment comprising a multi-contact bidimensional sensor for acquiring tactile information, a viewing screen arranged under the bidimensional tactile sensor, a memory for recording graphical objects that are each associated with at least one processing law, and a local calculator for analyzing the position of acquired tactile information and application of a processing law as a function of the position relative to the position of the graphical objects.
  • The device may be connected to a hub (multi-socket network) to form a network of controllers.
  • This multi-contact bidimensional tactile sensor is advantageously a resistive tile.
  • Furthermore, the device may comprise a network output suitable for receiving a network cable.
  • In the following description, the control is performed on a computerized piece of equipment that can be, e.g., a music software, a controller, audiovisual equipment or multimedia equipment.
  • As FIGS. 1A, 1B and 2A illustrate, the first basic element is the matrix sensor 101 necessary for acquisition (multi-contact manipulations) with the aid of a capture interface 102. The sensor 101 may be divided, if necessary, into several parts to accelerate capture, with each part being scanned simultaneously.
  • The general principle is to create as many cursors (such as a mouse cursor) as there are zones detected on the sensor and to follow their developments in time.
  • When the user removes the user's fingers from the sensor, the associated cursors are destroyed.
  • In this manner, the position and development of several fingers are captured simultaneously on the sensor. This is a multi-contact capture that is quite innovative for this type of controller.
  • The sensor may be a resistive tactile matrix tile of a known type.
  • Resistive tactile matrix tiles are composed of 2 superposed faces on which tracks of ITO (indium tin oxide), that is a translucent conductive material, are organized. The tracks are laid out in lines on the upper layer and in columns on the lower layer and a matrix as shown in FIG. 2A.
  • The two conductive layers are insulated from one another by spacing braces. The intersection of the line with the column forms a contact point. When a finger is placed on the tile, a column or columns situated on the upper layer are put in contact with a line or line situated on the lower layer, thus creating one or several contact points as shown in FIG. 2B.
  • It is possible to replace the braces by a transparent resistive material (e.g., a conductive polymer) whose resistance varies as a function of the pressure, which resistance drops if a sufficient pressure force is exerted. In this manner, it is also possible to extract the pressure exerted on the surface by performing a resistance measurement at each line-column intersection.
  • As concerns the musical or audiovisual use of these tiles, it is imperative to measure the activity of a finger with a maximum latency of 20 ms.
  • The state of the tile is measured at least 100 times per second, which tile can be divided into several zones to perform a parallel processing on these zones.
  • Thus, the sampling frequency of the tile may be at least 100 Hz.
  • Another basic element is the electronic device for scanning the tactile tile that allows the simultaneous detection of several contact points on the matrix sensor. In fact, the known methods of acquisitions for this type of sensor do not allow the detection of several simultaneous contact points.
  • The known methods do not allow the problems illustrated in FIG. 2C to be solved.
  • If a simultaneous measurement of all the lines is performed while feeding a column, problems of orthogonality arise. Contact point No. 1 will mask contact point No. 2. Likewise, if a line is measured when all the columns are fed, contact point No. 2 is masked by contact point No. 1. The solution for this problem is in performing a sequential scanning of the sensor.
  • The columns are fed, e.g., at 5V in turn and the level of the lines (high or low level) measured sequentially.
  • When one of the columns is placed under voltage, the others are in high impedance to prevent the propagation of current into the latter.
  • Thus, column 1 is fed at first while the other columns are in high impedance.
  • The lines are measured sequentially, that is, one after the other. The value on the first line is read initially while all the other lines are connected to ground. Then, line 1 is connected to ground and the value on line 2 is read and so forth until the value of all the lines has been read.
  • Column 1 then passes into the high impedance state and column 2 is fed. The reading of the state of each of the lines recommences.
  • The scanning is performed in this manner up to the last column.
  • As the goal is to form a multi-contact tile, the total scanning of the matrix is carried out at an elevated frequency to obtain the value of each of the intersection points of the tile several times per second.
  • The device permitting the acquisition of the tile data is illustrated in FIG. 2D, representing the algorithm of the acquisition of a tile comprising 100 lines (L) and 135 columns (C).
  • Certain problems in the masking of a point by one or several other points can appear.
  • In fact, the resistance of the transparent material (ITO) composing the columns and the lines increases proportionately to the length of the tracks. Thus, the potential measured at the lower left corner of the sensor will be greater than the potential measured at the upper right corner.
  • In FIGS. 2E and 2F, the cloud of points absorbs a large part of the electrical potential of the fed column. The potential measured at the isolated point is therefore too low to be detected.
  • A solution to this problem is in using a voltage comparator piloted digitally at the output of the line to determine whether the tension observed is sufficient for being considered as resulting from the action of a finger on the tactile tile. The reference value of the comparator (comparison threshold) is decremented at each line measure. Thus, the comparison values of the last lines are lower than those of the first lines, which allows the contact point located at the lower left or the upper right to be detected in the same manner.
  • Thus, e.g., a complete sampling of the tile is performed at least 100 times per second for the columns and the lines.
  • The data from capture interface 102 thus form an image representative of the totality of the sensor. This image is placed in memory so that a program can proceed to the filtering, the detection of the fingers and to the creation of the cursors as seen in FIG. 1.
  • The filtering phase illustrated by FIG. 3B eliminates noise that might be generated by the acquisition interface or the sensor itself. It is considered that only the clouds of several contact points can correspond to the pressure of a finger. Therefore, a bounding zone detection is carried out to eliminate isolated contact points.
  • The following stage associates a cursor with each support point (FIG. 3C). To this end, the barycenter of each bounding zone is calculated. When a finger is released, the corresponding cursor is freed.
  • The program executed locally by the main processor allows these cursors to be associated with graphical objects that are displayed on screen 105 to manipulate them. At the same time, the local program uses these cursors for generating control messages addressed to the host computer or the controlled apparatus.
  • Furthermore, the program comprises a simulator of the physical models allowing modification of the interaction laws between the cursors and the graphical objects. Different physical models can be employed: spring-loaded system, vibration of a string, management of collisions, the law of gravity, electromagnetic field and the like.
  • The program considers the positioning of the cursors and on which graphical object each is located. A specific processing is supplied to the data coming from the sensor as a function of the object considered. For example, a pressure measurement (corresponding to a development of the spot made by the finger on the tactile tile in a short interval of time) can be interpreted. Other parameters can be deduced as a function of the nature of the object: the acceleration, speed, trajectories, etc. Algorithms of recognition of form can also be applied to differentiate different fingers.
  • The main program 103 also transmits the data to be displayed on screen 105 to graphical interface 104. Moreover, this graphical interface is constituted of a graphical processor. This graphical processor is, e.g., of a known type. The latter can be constituted of primitive graphical functions allowing, e.g., the displaying of bitmap, fonts of polygons and figures in 2 and 3 dimensions, vectorial design, antialiasing, texture mapping, transparency and interpolation of colors.
  • The main program may also comprise an analyzer of mathematical expressions that allows mathematical functions to be inputted and calculated in real time. These functions allow the values of any variable to be modified. For example, the coordinates (x, y) of a cursor inside an object can be considered as two variables comprised between 0 and 1. The expression analyzer allows an expression of the type “x*1000+600” to be created to obtain a new variable whose value is comprised between 600 and 1600. The variable obtained allows the control, e.g., of the frequency of an oscillator comprised between 600 and 1600 hertz.
  • The mathematical expressions can be applied to scalar values as well as to vectors.
  • The expression analyzer is a tool that allows real-time calculations to be performed on the variables of objects.
  • Local program 103 also performs a formatting of data in the form of messages for network port 106, that communicates it to the computer on which the computer applications are performed.
  • The network interface is, e.g., an Ethernet 10/100 baseT standard interface that communicates by packets with the protocol TCP/IP. It can also be a network interface of the wireless type.
  • It should be noted as illustrated in FIG. 11 that the Ethernet connection offers the user the possibility, by using a simple hub (multi-socket network), of indefinitely expanding the control apparatus by constituting a network of controllers.
  • The controller or controllers present in the network then communicate among themselves and with the host computer in the form of the reciprocal sending of messages.
  • Furthermore, the unit constituting the machine is fed by a battery (not shown) of a known type or by an AC adapter.
  • Finally, an interface editor 107 at the level of the computer of the user allows the interface, that is, the totality of the graphic objects displayed on screen 105, to be programmed in a graphical manner. The interfaces may themselves be organized in scenes, that are higher hierarchical structures. In fact, each scene comprises several interfaces. The user can interchange the interfaces with the aid of a button keyboard or a control pedal board connected to input-output port 109.
  • Another function of the interface editor is to assign the control data to the parameters that the user wishes to control.
  • The user has at the user's disposal, e.g., a library of parameterable graphical objects allowing the composition of different interfaces according to the application desired. FIGS. 4 and 5 represent different graphical objects placed at the disposition of the user.
  • They can be predefined and dedicated quite particularly to music or to the control of audiovisual equipment or computerized apparatuses. For example, a linear potentiometer 403, 404 is particularly adapted to control continuous parameters such as the volume of a sound signal, the frequency of a filter. A serrated wheel 401 can serve, e.g., to control the playing of an audio or video reader. The objects can also be freely developed with a development kit (SDK) of a known type 109. The development kit furnishes access to the primitive graphical functions of the controller.
  • Interface editor 107 thus allows the user to readily create personalized control interfaces. It is a software executed on the user's computer. It is composed of a main window representing the tactile surface of the tile on which graphical objects from a library of proposed objects can be placed. The manipulation and placing of objects on the surface are performed, e.g., with the mouse. The object placed on the window is displayed at the same time on the controller and the object is recorded in a memory of the controller. It can subsequently move or re-dimension the objects at its convenience.
  • In addition to the positioning of graphical objects on the main window, other secondary windows allow the regulation of different parameters inherent in the objects (graphical properties, physical behavior). For example, a button 402 can also act as a switch or as a trigger. In the case of the trigger mode, a pressure measurement can optionally be performed. Another example of a parameterable object is area 2D (503, 544) of which the principle includes moving pawns inside a delimited zone. The number of pawns present in area 2D is a parameterable option. The area can be configured in uniplan mode, a mode in which the pawns enter into collision with each other, or multi-plan, a mode in which the pawns are placed on distinct superposed planes. Physical parameters can also be configured: the coefficient of friction of the pawns on the plane, the rebound and the attraction of the pawns on the edges and among themselves.
  • The editor also permits the objects present on the surface to be listed and the creation of functions and of variables with the expression analyzer.
  • Thus, the objects have by default a certain number of variables (x, y, z . . . ) corresponding to their primitive axes. These variables are always comprised between 0 and 1 and vary in the form of 32-bit numbers with floating comma. The user must be able to “connect” these variables to other values more representative of what he desires to control. Thus, the expression analyzer furnishes the possibility of creating new variables with the aid of simple mathematical expressions. For example, a rectilinear potentiometer has a primitive axis that is x. If the user wishes to control the frequency of 500 to 2500 Hz he must create a variable a=2000x+500.
  • Status display options are also desired. They permit a visual control of the state of a parameter.
  • The further treatments to be applied to the objects at the level of the main calculating unit 103 by the manipulation on the tile are specific to each type of object.
  • In fact, a circular movement of the finger on a virtual linear potentiometer (403, 404) should not have an effect on the state of the potentiometer whereas it should modify the state in the case of a circular potentiometer 401. Likewise, certain objects can only take into account a single finger (the linear potentiometer, for example) at a time whereas others can accept the interaction of several fingers (keyboard, area 2D).
  • For example, the “area 2D” (503, 504) is a rectangular surface containing a certain number of pawns, each with its own position. The pawns can be moved by the user.
  • The principle is to put in place a physical system for the totality of the objects, that is, e.g., that the pawns moved by the user acquire a speed of inertia that they retain when the user lets them go; the pawns subjected in this manner to their own speed will rebound on the edges of “area 2D” and also rebound among themselves. Furthermore, they will be subjected to forces of attraction/repulsion on the edges and on the other pawns as well as to a coefficient of friction on the surface of area 2D for stopping the pawns at the end of a certain time. All these parameters will be parameterable.
  • Another variant of area 2D includes applying a physical law of the “spring-loaded” type. A virtual rubber band is stretched between each cursor and each pawn. The user can modify the behavior of this object by configuring the friction and the interpolation factor. These properties can also be modified in real time with the aid of other objects.
  • Another example is the “Multislider” 501, a table of cursors whose numbers can be configured. The typical use is the controlling of a graphic equalizer or of a spectral envelope. The difference between a “multislider” and several simple juxtaposed linear potentiometers is that the totality of the cursors can be modified in a single touch by sliding the finger. The multislider can also be used as a discrete string. For this, it is sufficient to apply to it the physical model of a string whose tension is parameterable by the user.
  • A visualization of different examples of interfaces uniting different types of objects is illustrated by FIGS. 6 to 9, in which several objects described above can be observed.
  • FIG. 6 shows an arrangement of 6 areas 2D (601) containing 1 pawn each. This interface could control, e.g., six different filters assigned to one or several sound sources. In this instance, the abscissa movement of each pawn in each zone controls the frequency of the filter whereas the ordinate movement controls the quality factor or the width of the filter band.
  • FIG. 7 shows an example of the control of a synthesizer or of a sampler of a known type. The interface is composed by a tempered keyboard 704 controlling the pitch of the sounds, by a group of four vertical potentiometers 703 allowing the control, e.g., of its dynamic envelope (attack time, hold level, release time). An area 2D (701) containing 3 pawns allows the control, e.g., of effects applied to the sound (reverberation, echo, filters). A matrix of 16 buttons 792 can, e.g., release 16 different recorded musical sequences or also call up 16 prerecorded configurations of the previously described controls.
  • Another example is illustrated by FIG. 8 showing the control of a device for the broadcasting of different sound sources into space on a device constituted by several loudspeakers. In this configuration an area 2D (801) representing the broadcasting space contains 4 pawns 801 corresponding to four sound sources. Area 2D also contains 5 icons 802 representing the position of five loudspeakers. The level and/or the phase of each sound source relative to each enclosed space is regulated by moving the different pawns 802, which determines its emplacement in the space. Moreover, a group of four linear potentiometers 803 allows the relative level of each source to be regulated. A unit of four buttons 804 allows each sound source to be activated or deactivated.
  • Another example is illustrated in FIG. 9 that shows the control of a synthesizer or a sound generator according to a configuration different from that shown in FIG. 7. Here, the frequency of the sound generator is controlled by four virtual strings 903. The initial tension (the pitch) of each string can itself be controlled, e.g., by a linear potentiometer 902. An area 2D 10, e.g., control other parameters of the sound generator such as the output level, the sound quality, the panning, etc.
  • FIG. 10 shows the control of equipment for audio and/or video editing of a known type. A serrated wheel 1001 allows the rate of reading the audio and/or video sources to be controlled. Status display object 1002 allows the positioning of the reading to be represented according to a format (hour, minute, second, image) of a known type. A set of buttons 1003 allows access to the functions of reading and editing of the controlled apparatus.
  • The devices and methods described above are by way of example. It is understood that one skilled in the art is capable of realizing different variants of the devices and methods without departing from the scope of the appended claims.

Claims (10)

1. A device configured to acquire tactile information, comprising.
a transparent tactile matrix sensor;
a display device;
a measurement unit configured to simultaneously measure a first position of a first finger and a second position of a second finger that are placed on the transparent tactile matrix sensor;
a graphical processing unit configured to generate a first and a second cursor that are associated with the first and second position, respectively; and
a processor configured to execute an application that associates the first and second cursors with a graphical object that is displayed on the display device, to interact with the graphical object.
2. The device according to claim 1, wherein the graphical processing unit is configured to destroy the first and the second cursors, respectively, when the measurement unit detects that the first finger and the second finger, respectively, have been removed from the transparent tactile matrix sensor.
3. The device according to claim 1, wherein the modification of a movement of the graphical object by the processor follows an interaction rule as a function of the first and the second cursor, and the interaction rule is based on a physical model.
4. The device according to claim 3, wherein the physical model follows a spring-like behavior.
5. The device according to claim 3, wherein the physical model is based on a vibration of a string.
6. The device according to claim 3, wherein the physical model is configured to stimulate collisions of the graphical object.
7. The device according to claim 3, wherein the physical model is configured to simulate effects of gravity.
8. The device according to claim 3, wherein the physical model is configured to simulate effects of an electromagnetic field on the graphical object.
9. The device according to claim 3, wherein the graphical object includes a plurality of movable objects that are located inside a rectangular area, and the physical model is configured such that the first and the second cursors are moved by a movement of the first and second fingers, and will apply a inertia movement to the plurality of movable objects once at least one of the first or second finger is retracted from the transparent tactile matrix sensor.
10. The device according to claim 9, where physical model is configured such that the moving of the objects will rebound from a border of the rectangular area and will rebound if the objects collide with each other.
US13/013,985 2004-02-23 2011-01-26 Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen Abandoned US20110181547A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/013,985 US20110181547A1 (en) 2004-02-23 2011-01-26 Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
FR04/50329 2004-02-23
FR0450329A FR2866726B1 (en) 2004-02-23 2004-02-23 CONTROLLER BY HANDLING VIRTUAL OBJECTS ON A MULTI-CONTACT TOUCH SCREEN
US10/590,306 US8049730B2 (en) 2004-02-23 2005-02-23 Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen
PCT/FR2005/000428 WO2005091104A2 (en) 2004-02-23 2005-02-23 Controller involving manipulation of virtual objects on a multi-contact touch screen
US13/013,985 US20110181547A1 (en) 2004-02-23 2011-01-26 Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen

Related Parent Applications (2)

Application Number Title Priority Date Filing Date
PCT/FR2005/000428 Division WO2005091104A2 (en) 2004-02-23 2005-02-23 Controller involving manipulation of virtual objects on a multi-contact touch screen
US11/590,306 Division US20070050495A1 (en) 2000-09-18 2006-10-31 Distributed quality-of-service system

Publications (1)

Publication Number Publication Date
US20110181547A1 true US20110181547A1 (en) 2011-07-28

Family

ID=34834225

Family Applications (6)

Application Number Title Priority Date Filing Date
US10/590,306 Expired - Fee Related US8049730B2 (en) 2004-02-23 2005-02-23 Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen
US13/013,985 Abandoned US20110181547A1 (en) 2004-02-23 2011-01-26 Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen
US13/013,976 Abandoned US20110181546A1 (en) 2004-02-23 2011-01-26 Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen
US13/013,994 Expired - Fee Related US8659545B2 (en) 2004-02-23 2011-01-26 Device and method for controlling computerized equipment
US13/013,970 Expired - Fee Related US8665232B2 (en) 2004-02-23 2011-01-26 Device and method for acquiring tactile information with sequential scanning
US13/187,143 Abandoned US20110273392A1 (en) 2004-02-23 2011-07-20 Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US10/590,306 Expired - Fee Related US8049730B2 (en) 2004-02-23 2005-02-23 Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen

Family Applications After (4)

Application Number Title Priority Date Filing Date
US13/013,976 Abandoned US20110181546A1 (en) 2004-02-23 2011-01-26 Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen
US13/013,994 Expired - Fee Related US8659545B2 (en) 2004-02-23 2011-01-26 Device and method for controlling computerized equipment
US13/013,970 Expired - Fee Related US8665232B2 (en) 2004-02-23 2011-01-26 Device and method for acquiring tactile information with sequential scanning
US13/187,143 Abandoned US20110273392A1 (en) 2004-02-23 2011-07-20 Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen

Country Status (12)

Country Link
US (6) US8049730B2 (en)
EP (5) EP2306279A2 (en)
JP (2) JP4932500B2 (en)
KR (1) KR101197876B1 (en)
CN (1) CN100447723C (en)
AT (1) ATE402436T1 (en)
CA (1) CA2557063C (en)
DE (1) DE602005008384D1 (en)
ES (1) ES2311218T3 (en)
FR (1) FR2866726B1 (en)
PL (1) PL1719047T3 (en)
WO (1) WO2005091104A2 (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090237374A1 (en) * 2008-03-20 2009-09-24 Motorola, Inc. Transparent pressure sensor and method for using
US20110285666A1 (en) * 2010-05-21 2011-11-24 Ivan Poupyrev Electrovibration for touch surfaces
US8963874B2 (en) 2010-07-31 2015-02-24 Symbol Technologies, Inc. Touch screen rendering system and method of operation thereof
US8988191B2 (en) 2009-08-27 2015-03-24 Symbol Technologies, Inc. Systems and methods for pressure-based authentication of an input on a touch screen
US9018030B2 (en) 2008-03-20 2015-04-28 Symbol Technologies, Inc. Transparent force sensor and method of fabrication
US9122330B2 (en) 2012-11-19 2015-09-01 Disney Enterprises, Inc. Controlling a user's tactile perception in a dynamic physical environment

Families Citing this family (114)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6610917B2 (en) 1998-05-15 2003-08-26 Lester F. Ludwig Activity indication, external source, and processing loop provisions for driven vibrating-element environments
US9024884B2 (en) 2003-09-02 2015-05-05 Apple Inc. Touch-sensitive electronic apparatus for media applications, and methods therefor
FR2866726B1 (en) * 2004-02-23 2006-05-26 Jazzmutant CONTROLLER BY HANDLING VIRTUAL OBJECTS ON A MULTI-CONTACT TOUCH SCREEN
US8717301B2 (en) 2005-08-01 2014-05-06 Sony Corporation Information processing apparatus and method, and program
JP4201775B2 (en) * 2005-03-02 2008-12-24 株式会社コナミデジタルエンタテインメント Information processing apparatus, information processing apparatus control method, and program
EP1783593A3 (en) * 2005-10-07 2012-12-19 Sony Corporation Information processing apparatus with a user interface comprising a touch panel, method and program
DE502007000444D1 (en) * 2006-06-30 2009-04-02 Christian Iten Method for positioning a cursor on a touch-sensitive screen
US8970503B2 (en) * 2007-01-05 2015-03-03 Apple Inc. Gestures for devices having one or more touch sensitive surfaces
US8144129B2 (en) * 2007-01-05 2012-03-27 Apple Inc. Flexible touch sensing circuits
US7844915B2 (en) 2007-01-07 2010-11-30 Apple Inc. Application programming interfaces for scrolling operations
US20080168478A1 (en) 2007-01-07 2008-07-10 Andrew Platzer Application Programming Interfaces for Scrolling
US20080168402A1 (en) 2007-01-07 2008-07-10 Christopher Blumenberg Application Programming Interfaces for Gesture Operations
FR2914756B1 (en) * 2007-04-05 2012-09-21 Jazzmutant TRANSPARENT MULTI-TOUCH SENSOR.
US9052817B2 (en) 2007-06-13 2015-06-09 Apple Inc. Mode sensitive processing of touch data
KR101395780B1 (en) * 2007-07-27 2014-05-16 삼성전자주식회사 Pressure sensor arrary apparatus and method for tactility
CN101458586B (en) * 2007-12-11 2010-10-13 义隆电子股份有限公司 Method for operating objects on touch control screen by multi-fingers
AR064377A1 (en) 2007-12-17 2009-04-01 Rovere Victor Manuel Suarez DEVICE FOR SENSING MULTIPLE CONTACT AREAS AGAINST OBJECTS SIMULTANEOUSLY
FR2925716B1 (en) 2007-12-19 2010-06-18 Stantum ELECTRONIC ANALYSIS CIRCUIT WITH SCANNING CHARACTERISTIC MODULATION FOR A PASSIVE MATRIX MULTICONTACT TOUCH SENSOR
FR2925714B1 (en) 2007-12-19 2010-01-15 Stantum ELECTRONIC CAPACITIVE / RESISTIVE ALTERNATING ANALYSIS CIRCUIT FOR MULTICONTACT PASSIVE MATRIX TOUCH SENSOR
TWI358028B (en) * 2007-12-25 2012-02-11 Htc Corp Electronic device capable of transferring object b
US8797271B2 (en) 2008-02-27 2014-08-05 Microsoft Corporation Input aggregation for a multi-touch device
US8416196B2 (en) 2008-03-04 2013-04-09 Apple Inc. Touch event model programming interface
US8717305B2 (en) 2008-03-04 2014-05-06 Apple Inc. Touch event model for web pages
US8645827B2 (en) 2008-03-04 2014-02-04 Apple Inc. Touch event model
US8345014B2 (en) 2008-07-12 2013-01-01 Lester F. Ludwig Control of the operating system on a computing device via finger angle using a high dimensional touchpad (HDTP) touch user interface
FR2934921B1 (en) * 2008-08-05 2010-09-24 Stantum MULTICONTACT TOUCH SENSOR WITH VARIABLE SIZE AND IMPEDANCE SPACING MEANS
FR2934908B1 (en) 2008-08-05 2013-09-27 Stantum METHOD FOR ACQUIRING AND ANALYZING A MULTICONTACT TOUCH SENSOR FOLLOWING A DICHOTOMIC PRINCIPLE, ELECTRONIC CIRCUIT AND MULTICONTACT TOUCH SENSOR IMPLEMENTING SUCH A METHOD
FR2936326B1 (en) * 2008-09-22 2011-04-29 Stantum DEVICE FOR THE CONTROL OF ELECTRONIC APPARATUS BY HANDLING GRAPHIC OBJECTS ON A MULTICONTACT TOUCH SCREEN
US8749495B2 (en) * 2008-09-24 2014-06-10 Immersion Corporation Multiple actuation handheld device
US20100079400A1 (en) * 2008-09-26 2010-04-01 Sony Ericsson Mobile Communications Ab Touch sensitive display with conductive liquid
US8310464B2 (en) * 2008-10-16 2012-11-13 Texas Instruments Incorporated Simultaneous multiple location touch systems
US8477103B2 (en) 2008-10-26 2013-07-02 Microsoft Corporation Multi-touch object inertia simulation
US8466879B2 (en) 2008-10-26 2013-06-18 Microsoft Corporation Multi-touch manipulation of application objects
US20100141604A1 (en) * 2008-12-09 2010-06-10 Nokia Corporation Resistive multi touch screen
US8330732B2 (en) * 2008-12-19 2012-12-11 Honeywell International Inc. Method and apparatus for avionic touchscreen operation providing sensible feedback
US9342202B2 (en) * 2009-01-23 2016-05-17 Qualcomm Incorporated Conductive multi-touch touch panel
FR2942329B1 (en) 2009-02-17 2011-07-15 Stantum MULTIPOINT SENSOR
US9684521B2 (en) * 2010-01-26 2017-06-20 Apple Inc. Systems having discrete and continuous gesture recognizers
US8566045B2 (en) 2009-03-16 2013-10-22 Apple Inc. Event recognition
US9311112B2 (en) 2009-03-16 2016-04-12 Apple Inc. Event recognition
US8285499B2 (en) 2009-03-16 2012-10-09 Apple Inc. Event recognition
US9459734B2 (en) * 2009-04-06 2016-10-04 Synaptics Incorporated Input device with deflectable electrode
TWI466004B (en) * 2009-04-17 2014-12-21 Egalax Empia Technology Inc Method and device for resistive multi-point touch
CN101866253B (en) * 2009-04-20 2013-03-20 禾瑞亚科技股份有限公司 Resistance type multi-point touch device and method
KR101611511B1 (en) 2009-05-12 2016-04-12 삼성전자주식회사 A method of composing music in a portable terminal having a touchscreen
TWI402738B (en) * 2009-05-27 2013-07-21 Wintek Corp Touch apparatus and touch sensing method
CN102576277A (en) * 2009-05-29 2012-07-11 海普提克科技公司 Method for determining multiple touch inputs on a resistive touch screen and multiple touch controller
US9383881B2 (en) * 2009-06-03 2016-07-05 Synaptics Incorporated Input device and method with pressure-sensitive layer
IT1394716B1 (en) * 2009-06-09 2012-07-13 Corona DEVICE AND METHOD OF ACQUISITION OF DATA FROM AEROFONE MUSICAL INSTRUMENTS, IN PARTICULAR FOR LAUNEDDAS AND SIMILARS
CN101989143B (en) * 2009-07-31 2013-05-01 群康科技(深圳)有限公司 Touch control device and control method thereof
GB0913734D0 (en) * 2009-08-06 2009-09-16 Binstead Ronald P Masked touch sensors
JP5278766B2 (en) * 2009-09-07 2013-09-04 Nltテクノロジー株式会社 Touch position detection device, information input system, touch position detection method, touch position detection program
US8558722B2 (en) * 2009-12-10 2013-10-15 Industrial Technology Research Institute Touch apparatus, transparent scan electrode structure, and manufacturing method thereof
FR2954981A1 (en) * 2010-01-05 2011-07-08 Stantum MULTICONTACT BIDIMENSIONAL TOUCH SENSOR AND SENSOR WITH LONGITUDINAL ELEMENTS INCLINED CONDUCTORS
US20110163944A1 (en) * 2010-01-05 2011-07-07 Apple Inc. Intuitive, gesture-based communications with physics metaphors
FR2954982A1 (en) * 2010-01-05 2011-07-08 Stantum MULTICONTACT TOUCH SENSOR WITH HIGH ELECTRIC CONTACT RESISTANCE
TWI426427B (en) * 2010-02-12 2014-02-11 Wintek Corp Touch panel
DE102011006448A1 (en) 2010-03-31 2011-10-06 Tk Holdings, Inc. steering wheel sensors
DE102011006344B4 (en) 2010-03-31 2020-03-12 Joyson Safety Systems Acquisition Llc Occupant measurement system
DE102011006649B4 (en) 2010-04-02 2018-05-03 Tk Holdings Inc. Steering wheel with hand sensors
US8686960B2 (en) * 2010-04-23 2014-04-01 Lester F. Ludwig Piecewise-linear and piecewise-affine transformations for high dimensional touchpad (HDTP) output decoupling and corrections
US10216408B2 (en) 2010-06-14 2019-02-26 Apple Inc. Devices and methods for identifying user interface objects based on view hierarchy
JP5821170B2 (en) * 2010-08-31 2015-11-24 ヤマハ株式会社 Electronic music apparatus and program
US8330033B2 (en) 2010-09-13 2012-12-11 Apple Inc. Graphical user interface for music sequence programming
US9158369B2 (en) * 2010-10-12 2015-10-13 Tactonic Technologies, Llc Sensors having a connecting frame and method for composite sensors
FR2968102B1 (en) * 2010-11-26 2013-01-04 Stantum TOUCH SENSOR WITH MATRIX NETWORK OF CONDUCTIVE TRACKS AND TOUCH CONTROL SCREEN
FR2971068B1 (en) 2011-01-31 2013-09-27 Stantum MULTICONTACT TOUCH SENSOR WITH RESISTIVE INTERMEDIATE LAYER
FR2971346A1 (en) 2011-02-03 2012-08-10 Stantum METHOD AND DEVICE FOR ACQUIRING DATA OF A MULTICONTACT MATRIX TOUCH SENSOR
FR2972288B1 (en) 2011-03-04 2013-04-05 Stantum METHOD FOR MANUFACTURING A TRANSPARENT FILM, IN PARTICULAR FOR A TRANSPARENT MATRIX TOUCH SENSOR
US8743244B2 (en) 2011-03-21 2014-06-03 HJ Laboratories, LLC Providing augmented reality based on third party information
US9298363B2 (en) 2011-04-11 2016-03-29 Apple Inc. Region activation for touch sensitive surface
FR2974200B1 (en) 2011-04-15 2014-08-01 Stantum METHOD FOR MANUFACTURING MATRIX TOUCH SENSOR, AND MATRIX TOUCH SENSOR THEREFOR
TWI469024B (en) * 2011-08-03 2015-01-11 Raydium Semiconductor Corp Touch input device for switching driving signals
FR2979025A1 (en) 2011-08-12 2013-02-15 Stantum METHOD OF CHARACTERIZING TOUCH ON A TOUCH SCREEN
JP6115470B2 (en) * 2011-09-27 2017-04-19 日本電気株式会社 Portable electronic device, touch operation processing method and program
WO2013049816A1 (en) 2011-09-30 2013-04-04 Sensitronics, LLC Hybrid capacitive force sensors
FR2981765B1 (en) 2011-10-20 2013-12-27 Stantum METHOD FOR ACQUIRING DATA OF A MATRIX TOUCH SENSOR, IN PARTICULAR FOR A TOUCH SCREEN
US20150033161A1 (en) * 2012-03-30 2015-01-29 Richard James Lawson Detecting a first and a second touch to associate a data file with a graphical data object
WO2013154720A1 (en) 2012-04-13 2013-10-17 Tk Holdings Inc. Pressure sensor including a pressure sensitive material for use with control systems and methods of using the same
JP6086188B2 (en) * 2012-09-04 2017-03-01 ソニー株式会社 SOUND EFFECT ADJUSTING DEVICE AND METHOD, AND PROGRAM
WO2014043664A1 (en) 2012-09-17 2014-03-20 Tk Holdings Inc. Single layer force sensor
US9671943B2 (en) 2012-09-28 2017-06-06 Dassault Systemes Simulia Corp. Touch-enabled complex data entry
US20140239815A1 (en) * 2013-02-28 2014-08-28 Power One Data International, Inc. Method and system for controlling lighting operation
US9733716B2 (en) 2013-06-09 2017-08-15 Apple Inc. Proxy gesture recognizer
US11221706B2 (en) 2013-09-27 2022-01-11 Sensel, Inc. Tactile touch sensor system and method
KR102271637B1 (en) 2013-09-27 2021-07-02 센셀, 인크. Resistive touch sensor system and method
JP6358554B2 (en) * 2013-12-19 2018-07-18 カシオ計算機株式会社 Musical sound control device, musical sound control method and program
US9542027B2 (en) * 2014-04-16 2017-01-10 At&T Intellectual Property I, L.P. Pressure-based input method for user devices
US9336762B2 (en) * 2014-09-02 2016-05-10 Native Instruments Gmbh Electronic music instrument with touch-sensitive means
US9779709B2 (en) * 2014-11-05 2017-10-03 Roger Linn Polyphonic multi-dimensional controller with sensor having force-sensing potentiometers
EP3329484A4 (en) * 2015-07-29 2019-06-05 Sensel Inc. Systems and methods for manipulating a virtual environment
KR101784420B1 (en) 2015-10-20 2017-10-11 연세대학교 산학협력단 Apparatus and Method of Sound Modulation using Touch Screen with Pressure Sensor
CN109154872B (en) 2016-03-25 2020-06-30 森赛尔股份有限公司 System and method for detecting and characterizing force input on a surface
US20190291224A1 (en) * 2018-03-22 2019-09-26 Ford Motor Company Workpiece alignment system having pressure sensors for assessing alignment of a workpiece with a fixture
US10642435B2 (en) 2018-03-29 2020-05-05 Cirrus Logic, Inc. False triggering prevention in a resonant phase sensing system
US10725549B2 (en) * 2018-03-29 2020-07-28 Cirrus Logic, Inc. Efficient detection of human machine interface interaction using a resonant phase sensing system
US11092657B2 (en) 2018-03-29 2021-08-17 Cirrus Logic, Inc. Compensation of changes in a resonant phase sensing system including a resistive-inductive-capacitive sensor
US10921159B1 (en) 2018-03-29 2021-02-16 Cirrus Logic, Inc. Use of reference sensor in resonant phase sensing system
US10908200B2 (en) 2018-03-29 2021-02-02 Cirrus Logic, Inc. Resonant phase sensing of resistive-inductive-capacitive sensors
US11537242B2 (en) 2018-03-29 2022-12-27 Cirrus Logic, Inc. Q-factor enhancement in resonant phase sensing of resistive-inductive-capacitive sensors
WO2020037439A1 (en) * 2018-08-23 2020-02-27 Pontificia Universidad Católica De Chile Midi controller with customisable sensitivity
US11402946B2 (en) 2019-02-26 2022-08-02 Cirrus Logic, Inc. Multi-chip synchronization in sensor applications
US10935620B2 (en) 2019-02-26 2021-03-02 Cirrus Logic, Inc. On-chip resonance detection and transfer function mapping of resistive-inductive-capacitive sensors
US11536758B2 (en) 2019-02-26 2022-12-27 Cirrus Logic, Inc. Single-capacitor inductive sense systems
US10948313B2 (en) 2019-02-26 2021-03-16 Cirrus Logic, Inc. Spread spectrum sensor scanning using resistive-inductive-capacitive sensors
US11079874B2 (en) 2019-11-19 2021-08-03 Cirrus Logic, Inc. Virtual button characterization engine
US11579030B2 (en) 2020-06-18 2023-02-14 Cirrus Logic, Inc. Baseline estimation for sensor system
US11835410B2 (en) 2020-06-25 2023-12-05 Cirrus Logic Inc. Determination of resonant frequency and quality factor for a sensor system
US11868540B2 (en) 2020-06-25 2024-01-09 Cirrus Logic Inc. Determination of resonant frequency and quality factor for a sensor system
US11619519B2 (en) 2021-02-08 2023-04-04 Cirrus Logic, Inc. Predictive sensor tracking optimization in multi-sensor sensing applications
US11808669B2 (en) 2021-03-29 2023-11-07 Cirrus Logic Inc. Gain and mismatch calibration for a phase detector used in an inductive sensor
US11821761B2 (en) 2021-03-29 2023-11-21 Cirrus Logic Inc. Maximizing dynamic range in resonant sensing
US11507199B2 (en) 2021-03-30 2022-11-22 Cirrus Logic, Inc. Pseudo-differential phase measurement and quality factor compensation
US11854738B2 (en) 2021-12-02 2023-12-26 Cirrus Logic Inc. Slew control for variable load pulse-width modulation driver and load sensing

Citations (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4914624A (en) * 1988-05-06 1990-04-03 Dunthorn David I Virtual button for touch screen
US5027689A (en) * 1988-09-02 1991-07-02 Yamaha Corporation Musical tone generating apparatus
US5053585A (en) * 1990-10-12 1991-10-01 Interlink Electronics, Incorporated Multipurpose keyboard using digitizer pad featuring spatial minimization of a pressure contact area and method of making same
US5327163A (en) * 1991-11-28 1994-07-05 Sharp Kabushiki Kaisha Display integrated type position reading apparatus
US5559301A (en) * 1994-09-15 1996-09-24 Korg, Inc. Touchscreen interface having pop-up variable adjustment displays for controllers and audio processing systems
US5825352A (en) * 1996-01-04 1998-10-20 Logitech, Inc. Multiple fingers contact sensing method for emulating mouse buttons and mouse operations on a touch sensor pad
US5869791A (en) * 1995-04-18 1999-02-09 U.S. Philips Corporation Method and apparatus for a touch sensing device having a thin film insulation layer about the periphery of each sensing element
US6073036A (en) * 1997-04-28 2000-06-06 Nokia Mobile Phones Limited Mobile station with touch input having automatic symbol magnification function
US6107997A (en) * 1996-06-27 2000-08-22 Ure; Michael J. Touch-sensitive keyboard/mouse and computing device using the same
US6229529B1 (en) * 1997-07-11 2001-05-08 Ricoh Company, Ltd. Write point detecting circuit to detect multiple write points
US6281420B1 (en) * 1999-09-24 2001-08-28 Yamaha Corporation Method and apparatus for editing performance data with modifications of icons of musical symbols
US6323846B1 (en) * 1998-01-26 2001-11-27 University Of Delaware Method and apparatus for integrating manual input
US20020005108A1 (en) * 1998-05-15 2002-01-17 Ludwig Lester Frank Tactile, visual, and array controllers for real-time control of music signal processing, mixing, video, and lighting
US6762752B2 (en) * 2001-11-29 2004-07-13 N-Trig Ltd. Dual function input device and method
US20060026521A1 (en) * 2004-07-30 2006-02-02 Apple Computer, Inc. Gestures for touch sensitive input devices
US7015894B2 (en) * 2001-09-28 2006-03-21 Ricoh Company, Ltd. Information input and output system, method, storage medium, and carrier wave
US20060097991A1 (en) * 2004-05-06 2006-05-11 Apple Computer, Inc. Multipoint touchscreen
US7046235B2 (en) * 2002-05-20 2006-05-16 Sharp Kabushiki Kaisha Input device and touch area registration method
US20070198926A1 (en) * 2004-02-23 2007-08-23 Jazzmutant Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen
US20070279392A1 (en) * 1995-12-01 2007-12-06 Rosenberg Louis B Networked applications including haptic feedback
US7307623B2 (en) * 2002-02-28 2007-12-11 Sony Computer Entertainment Inc. Information processing device having detector capable of detecting coordinate values, as well as changes thereof, of a plurality of points on display screen

Family Cites Families (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5816504B2 (en) * 1979-09-27 1983-03-31 沖電気工業株式会社 Pressure-sensitive handwritten figure input device
US4296406A (en) * 1979-12-28 1981-10-20 Sperry Corporation Pressure sensitive switch structure
DE3220622A1 (en) * 1982-06-01 1983-12-15 M.A.N.- Roland Druckmaschinen AG, 6050 Offenbach DATA INPUT DEVICE ON PRINTING MACHINES
US4550221A (en) * 1983-10-07 1985-10-29 Scott Mabusth Touch sensitive control device
AU582181B2 (en) * 1984-12-28 1989-03-16 Wang Laboratories, Inc. Information display and entry device
US5153829A (en) * 1987-11-11 1992-10-06 Canon Kabushiki Kaisha Multifunction musical information processing apparatus
JP3089421B2 (en) * 1988-09-02 2000-09-18 ヤマハ株式会社 Sound processing device
US5113042A (en) * 1990-11-20 1992-05-12 Summagraphics Corporation Digitizer tablet with reduced radiation susceptibility
US5159159A (en) * 1990-12-07 1992-10-27 Asher David J Touch sensor and controller
JPH0743627B2 (en) * 1993-08-12 1995-05-15 セイコーエプソン株式会社 Input device
US5565657A (en) * 1993-11-01 1996-10-15 Xerox Corporation Multidimensional user interface input device
GB9406702D0 (en) * 1994-04-05 1994-05-25 Binstead Ronald P Multiple input proximity detector and touchpad system
JPH07281813A (en) * 1994-04-08 1995-10-27 Oki Electric Ind Co Ltd Touch panel device and its position detecting method
JP3239017B2 (en) * 1994-07-05 2001-12-17 シャープ株式会社 Anti-reflective tablet coordinate detection device
JPH08307954A (en) * 1995-05-12 1996-11-22 Sony Corp Device and method for coordinate input and information processor
US6255604B1 (en) * 1995-05-31 2001-07-03 Canon Kabushiki Kaisha Coordinate detecting device for outputting coordinate data when two points are simultaneously depressed, method therefor and computer control device
US5949501A (en) * 1996-02-13 1999-09-07 Kabushiki Kaisha Pilot Coordinates input device using liquid crystal sheet
JPH09282095A (en) * 1996-02-13 1997-10-31 Pilot Corp:The Coordinate input device using liquid crystal sheet
JPH11112254A (en) * 1997-10-01 1999-04-23 Sony Corp Remote controller and controller
JP3470995B2 (en) * 1997-10-31 2003-11-25 株式会社デジタル Display screen creation device for programmable display device
JP2000222130A (en) * 1999-02-02 2000-08-11 Toshiba Corp Input device and method and storage medium
TW501000B (en) * 2000-01-05 2002-09-01 Iomega Corp Storage device in an expansion slot of a computer input device
FR2806497B1 (en) 2000-03-17 2002-05-03 Naguy Caillavet HARDWARE AND SOFTWARE INTERFACE FOR MIDI MESSAGE CONTROL
DE10042300A1 (en) 2000-08-29 2002-03-28 Axel C Burgbacher Electronic musical instrument with tone generator contg. input members
US7030861B1 (en) * 2001-02-10 2006-04-18 Wayne Carl Westerman System and method for packing multi-touch gestures onto a hand
JP4686886B2 (en) * 2001-04-06 2011-05-25 ソニー株式会社 Information processing device
JP2003008808A (en) * 2001-06-19 2003-01-10 Fuji Xerox Co Ltd Touch panel input device, display device, and image recording device
TW565798B (en) * 2001-10-12 2003-12-11 High Tech Comp Corp Pressing point coordinate detecting method for resistor type touch panel
FR2831707B1 (en) * 2001-10-25 2004-10-29 Siemens Vdo Automotive TOUCH-SENSITIVE SURFACE AND PRESSURE LEVELS
US6995752B2 (en) 2001-11-08 2006-02-07 Koninklijke Philips Electronics N.V. Multi-point touch pad
US7038659B2 (en) * 2002-04-06 2006-05-02 Janusz Wiktor Rajkowski Symbol encoding apparatus and method
US6882337B2 (en) * 2002-04-18 2005-04-19 Microsoft Corporation Virtual keyboard for touch-typing using audio feedback
US6809280B2 (en) * 2002-05-02 2004-10-26 3M Innovative Properties Company Pressure activated switch and touch panel
US7461356B2 (en) * 2002-06-03 2008-12-02 Fuji Xerox Co., Ltd. Function control unit and method thereof
US7151532B2 (en) * 2002-08-09 2006-12-19 3M Innovative Properties Company Multifunctional multilayer optical film
US20040090429A1 (en) * 2002-11-12 2004-05-13 Geaghan Bernard O. Touch sensor and method of making
TWI231453B (en) * 2003-01-20 2005-04-21 Htc Corp Method and apparatus for avoiding pressing inaccuracies on a touch panel
US7213125B2 (en) 2004-07-31 2007-05-01 Hewlett-Packard Development Company, L.P. Method for patching virtually aliased pages by a virtual-machine monitor
WO2008028044A2 (en) * 2006-08-30 2008-03-06 Centocor, Inc. Markers and methods for assessing and treating ulcerative colitis and related disorders using 66 gene panel

Patent Citations (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4914624A (en) * 1988-05-06 1990-04-03 Dunthorn David I Virtual button for touch screen
US5027689A (en) * 1988-09-02 1991-07-02 Yamaha Corporation Musical tone generating apparatus
US5053585A (en) * 1990-10-12 1991-10-01 Interlink Electronics, Incorporated Multipurpose keyboard using digitizer pad featuring spatial minimization of a pressure contact area and method of making same
US5327163A (en) * 1991-11-28 1994-07-05 Sharp Kabushiki Kaisha Display integrated type position reading apparatus
US5559301A (en) * 1994-09-15 1996-09-24 Korg, Inc. Touchscreen interface having pop-up variable adjustment displays for controllers and audio processing systems
US5869791A (en) * 1995-04-18 1999-02-09 U.S. Philips Corporation Method and apparatus for a touch sensing device having a thin film insulation layer about the periphery of each sensing element
US20070279392A1 (en) * 1995-12-01 2007-12-06 Rosenberg Louis B Networked applications including haptic feedback
US5825352A (en) * 1996-01-04 1998-10-20 Logitech, Inc. Multiple fingers contact sensing method for emulating mouse buttons and mouse operations on a touch sensor pad
US6107997A (en) * 1996-06-27 2000-08-22 Ure; Michael J. Touch-sensitive keyboard/mouse and computing device using the same
US6073036A (en) * 1997-04-28 2000-06-06 Nokia Mobile Phones Limited Mobile station with touch input having automatic symbol magnification function
US6229529B1 (en) * 1997-07-11 2001-05-08 Ricoh Company, Ltd. Write point detecting circuit to detect multiple write points
US6888536B2 (en) * 1998-01-26 2005-05-03 The University Of Delaware Method and apparatus for integrating manual input
US6323846B1 (en) * 1998-01-26 2001-11-27 University Of Delaware Method and apparatus for integrating manual input
US20060238518A1 (en) * 1998-01-26 2006-10-26 Fingerworks, Inc. Touch surface
US20020005108A1 (en) * 1998-05-15 2002-01-17 Ludwig Lester Frank Tactile, visual, and array controllers for real-time control of music signal processing, mixing, video, and lighting
US6570078B2 (en) * 1998-05-15 2003-05-27 Lester Frank Ludwig Tactile, visual, and array controllers for real-time control of music signal processing, mixing, video, and lighting
US6281420B1 (en) * 1999-09-24 2001-08-28 Yamaha Corporation Method and apparatus for editing performance data with modifications of icons of musical symbols
US7015894B2 (en) * 2001-09-28 2006-03-21 Ricoh Company, Ltd. Information input and output system, method, storage medium, and carrier wave
US6762752B2 (en) * 2001-11-29 2004-07-13 N-Trig Ltd. Dual function input device and method
US7307623B2 (en) * 2002-02-28 2007-12-11 Sony Computer Entertainment Inc. Information processing device having detector capable of detecting coordinate values, as well as changes thereof, of a plurality of points on display screen
US7046235B2 (en) * 2002-05-20 2006-05-16 Sharp Kabushiki Kaisha Input device and touch area registration method
US20070198926A1 (en) * 2004-02-23 2007-08-23 Jazzmutant Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen
US8049730B2 (en) * 2004-02-23 2011-11-01 Stantum Devices and methods of controlling manipulation of virtual objects on a multi-contact tactile screen
US20060097991A1 (en) * 2004-05-06 2006-05-11 Apple Computer, Inc. Multipoint touchscreen
US20060026521A1 (en) * 2004-07-30 2006-02-02 Apple Computer, Inc. Gestures for touch sensitive input devices

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090237374A1 (en) * 2008-03-20 2009-09-24 Motorola, Inc. Transparent pressure sensor and method for using
US9018030B2 (en) 2008-03-20 2015-04-28 Symbol Technologies, Inc. Transparent force sensor and method of fabrication
US8988191B2 (en) 2009-08-27 2015-03-24 Symbol Technologies, Inc. Systems and methods for pressure-based authentication of an input on a touch screen
US20110285666A1 (en) * 2010-05-21 2011-11-24 Ivan Poupyrev Electrovibration for touch surfaces
US8963874B2 (en) 2010-07-31 2015-02-24 Symbol Technologies, Inc. Touch screen rendering system and method of operation thereof
US9310920B2 (en) 2010-07-31 2016-04-12 Symbol Technologies, Llc Touch screen rendering system and method of operation thereof
US9122330B2 (en) 2012-11-19 2015-09-01 Disney Enterprises, Inc. Controlling a user's tactile perception in a dynamic physical environment

Also Published As

Publication number Publication date
DE602005008384D1 (en) 2008-09-04
US20110115736A1 (en) 2011-05-19
FR2866726A1 (en) 2005-08-26
EP2306279A2 (en) 2011-04-06
EP1719047B1 (en) 2008-07-23
KR101197876B1 (en) 2012-11-05
FR2866726B1 (en) 2006-05-26
WO2005091104A3 (en) 2006-07-13
EP2278439A2 (en) 2011-01-26
WO2005091104A2 (en) 2005-09-29
CA2557063C (en) 2013-08-13
CA2557063A1 (en) 2005-09-29
CN1947087A (en) 2007-04-11
EP1950649A2 (en) 2008-07-30
ES2311218T3 (en) 2009-02-01
EP2278440A2 (en) 2011-01-26
US20110181546A1 (en) 2011-07-28
CN100447723C (en) 2008-12-31
ATE402436T1 (en) 2008-08-15
PL1719047T3 (en) 2009-03-31
KR20070031286A (en) 2007-03-19
US20070198926A1 (en) 2007-08-23
JP2007527061A (en) 2007-09-20
JP2012089150A (en) 2012-05-10
US8665232B2 (en) 2014-03-04
EP1719047A2 (en) 2006-11-08
US20110273392A1 (en) 2011-11-10
EP1950649A3 (en) 2008-12-24
JP4932500B2 (en) 2012-05-16
US20110119580A1 (en) 2011-05-19
US8049730B2 (en) 2011-11-01
US8659545B2 (en) 2014-02-25

Similar Documents

Publication Publication Date Title
US8659545B2 (en) Device and method for controlling computerized equipment
US7701438B2 (en) Design of force sensations for haptic feedback computer interfaces
US6292170B1 (en) Designing compound force sensations for computer applications
JP6074170B2 (en) Short range motion tracking system and method
US20110134039A1 (en) User interface device, such as a mouse, with a plurality of scroll wheels
US20020054021A1 (en) Designing force sensations for force feedback computer applications
US20040160415A1 (en) Designing force sensations for force feedback computer applications
JP2013037675A5 (en)
O’Modhrain et al. The moose: A haptic user interface for blind persons
WO2002023323A2 (en) Freely specifiable real-time control
Smith et al. Digital foam interaction techniques for 3D modeling
US20140266569A1 (en) Controlling music variables
CN117224952A (en) Display control method, display control device, storage medium and electronic equipment
JPH03175525A (en) Three-dimensional coordinate input device
O'Modhrain et al. A haptic interface for the digital sound studio
JPH10143155A (en) Formation of virtual musical instrument
JPS62132287A (en) Sequence preparing device

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION