US20130157761A1

US20130157761A1 - System amd method for a song specific keyboard

Info

Publication number: US20130157761A1
Application number: US13/645,692
Authority: US
Inventors: Bruce William Cichowlas
Original assignee: REAL KEYS MUSIC Inc
Current assignee: REAL KEYS MUSIC Inc
Priority date: 2011-10-05
Filing date: 2012-10-05
Publication date: 2013-06-20

Abstract

A melody based game system includes a controller, a computing unit, and a display. The controller has a limited number of programmable real keys configured to be manipulated by a player for playing one or more song segments and thereby providing game input. Each of the song segments includes one or more pitches and one or more of the programmable real keys are programmed to generate the one or more pitches comprised within each of the song segments, respectively. The computing unit communicates with the controller and includes a music application. The display is connected to the computing unit and is configured to display prompts for manipulating the one or more programmed real keys in coordination with playing of the one or more pitches of each of the song segments.

Description

CROSS REFERENCE TO RELATED CO-PENDING APPLICATIONS

This application claims the benefit of U.S. provisional application Ser. No. 61/543,393 filed on Oct. 5, 2011 and entitled SYSTEM AMD METHOD FOR A SONG SPECIFIC KEYBOARD which is commonly assigned and the contents of which are expressly incorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates to a system and a method for a simplified “song specific” keyboard, and in particular to an electronic game that utilizes an electronic board with a limited number of buttons/keys for providing game input as prompted by a game display and usually in coordination with a recorded recognized song or song segment.

BACKGROUND OF THE INVENTION

The idea of teaching the playing of an actual keyboard instrument through a computer has been discussed in prior art documents, such as US2006/0252503. There are also several prior art systems in the market that try to teach piano through a computer, notably the Miracle Piano Systems, published by Software Toolworks, Nintendo, Synthesia published byHDPiano.com, among others. However, these devices are primarily intended for education, and in general are not effective in having fun with music or learning to play actual phrases and parts of music.
There are also several electronic games that receive game input from musical instruments, such as Guitar Hero, published by RedOctane, Rock Band and Dance Dance Revolution. However, these systems are rhythm action games that utilize mock-type (i.e., not “real”) musical instruments and do not provide significant music education value. A “real” musical instrument, as defined here, has means for selecting the “pitches” of a song and includes both actual instruments and musical instrument simulators that have places on the instrument that can be associated with a “pitch”. “Pitch” represents the perceived fundamental frequency of a sound.
Accordingly, there is a need for a musical instrument teaching method and system that is fun to play as a game and more effective than presently available music teaching systems and methods.

SUMMARY OF THE INVENTION

The invention provides an electronic game that utilizes an electronic board, or a controller, or a real musical instrument for providing game input as prompted by a game display and usually in coordination with a recorded recognized song or song segment.
In general, in one aspect, the invention features a melody based game system configured to receive game input by playing one or more song segments including a controller, a computing unit, and a display. The controller has a limited number of programmable real keys configured to be manipulated by a player for playing one or more song segments and thereby providing game input. Each of the song segments includes one or more pitches and one or more of the programmable real keys are programmed to generate the one or more pitches comprised within each of the song segments, respectively. The computing unit communicates with the controller and includes a music application. The display is connected to the computing unit and is configured to display prompts for manipulating the one or more programmed real keys in coordination with playing of the one or more pitches of each of the song segments. The music application simultaneously causes a song segment to be played and provides prompts indicating a manipulation sequence of the one or more programmed real keys corresponding to the playing of the one or more pitches of the song segment. Manipulation of the one or more programmed real keys by the player coinciding with the displayed prompts in pitch, intensity, and timing during the playing of the one or more pitches of the song segment causes a positive scoring event.
Implementations of this aspect of the invention may include one or more of the following features. The computing unit may be a computer, a tablet computer, an iPad™, an iPhone™, a Smartphone a Playstation™, an Xbox™, a Wii™, a PlayStation™, a Nintendo DS™, a game controlling device or a handheld game controlling device. The game system may further includes a database comprising musical recordings and data specific to each musical recording that associate pitches of each musical recording with a specific manipulation sequence of the real keys. The musical recordings may be song segments, song fragments, musical arrangements, instrumental musical pieces, or vocal musical pieces. No manipulation of the one or more programmed real keys by the player coinciding with the displayed prompts or manipulation of the programmed real keys by the player not coinciding in pitch, intensity, and timing with the simultaneously displayed prompts during the playing of the one or more pitches of the song segment causes a negative scoring event. The number of the programmable real keys is in the range between 3 and 11. The number of the programmable real keys is seven. The programmable real keys are touch areas defined on a screen of a touch enabled display. The computing unit may be a tablet computer having a touch enabled display screen and the controller and the programmable real keys are touch areas defined in the touch enabled display screen. The one or more programmable real keys that are programmed to generate the one or more pitches comprised within each of the song segments are designated during the playing of each of the song segments with a designation comprising one of light, colors, shapes, numbers, letters, textures, font type, font size, or key relative position. A melody of any length including a plurality of song segments is played by sequential reprogramming of the programmable real keys. The game system further includes a bidirectional communication mechanism between the computing unit and the controller.
In general, in another aspect, the invention features a melody based game system configured to receive game input by playing one or more song segments including a tablet computer and a controller. The tablet computer has a touch enabled display screen and a music application. The controller includes a limited number of programmable real keys configured to be manipulated by a player for playing one or more song segments and thereby providing game input. Each of the song segments comprises one or more pitches and the one or more of the programmable real keys are programmed to generate the one or more pitches comprised within each of the song segments, respectively. The music application simultaneously causes a song segment to be played and provides prompts indicating a manipulation sequence of the one or more programmed real keys corresponding to the playing of the one or more pitches of the song segment. The controller and the programmable real keys comprise touch areas defined in the touch enabled display screen and the prompts are displayed on the touch enabled display screen. Manipulation of the one or more programmed real keys by the player coinciding with the prompts in pitch, intensity, and timing during the playing of the one or more pitches of the song segment causes a positive scoring event.
In general, in another aspect, the invention features a method for a melody based game system configured to receive game input by playing one or more song segments including the following. First, providing a controller including a limited number of programmable real keys configured to be manipulated by a player for playing one or more song segments and thereby providing game input. Each of the song segments comprises one or more pitches and one or more of the programmable real keys are programmed to generate the one or more pitches comprised within each of the song segments, respectively. Next, providing a computing unit communicating with the controller and comprising a music application. Next, providing a display connected to the computing unit and configured to display prompts for manipulating the one or more programmed real keys in coordination with playing of the one or more pitches of each of the song segments. The music application simultaneously causes a song segment to be played and provides prompts indicating a manipulation sequence of the one or more programmed real keys corresponding to the playing of the one or more pitches of the song segment. Manipulation of the one or more programmed real keys by the player coinciding with the displayed prompts in pitch, intensity, and timing during the playing of the one or more pitches of the song segment causes a positive scoring event.
Among the advantages of the invention may be one or more of the following. The present invention combines melody and rhythm action into a new generation of instrument games, expanding the class of musical genre offered, and the demographics to be reached. Real melody and its corresponding notes are associated with colors/patterns.
The corresponding colors/patterns are then associated with specific “keys”/positions on a musical instrument or a “controller”, so that when a “key”/position is pressed a recognizable frequency (i.e., a note) is generated which is associated with the desired note. When the right combinations of these notes are pressed at the right time or rhythm, then the melody is “played” correctly. The “played’ melody may be audibly heard by the player and/or scored for its correctness. In cases where the “controller” is not an actual musical instrument, but is able to select or generate different musical pitches (frequencies) from its various “keys”/buttons that are programmed by the game console, then in effect, the game has turned the controller into a musical instrument. The colors/patterns may be displayed on a screen/display and the screen/display may be integrated with the “controller’/musical instrument or may be separate from the “controller’/musical instrument.
Melody based action games are different from rhythm based games because they have different pitches (frequencies) that are actually being played, whereas rhythm only based games are associated only with the timing of the actions. Melody based action games have the advantage that they can be used for teaching music while allowing the players to have fun with the game. This is especially true, when the controller is a real musical instrument, such as a piano, keyboard, guitar, or wind instrument, among others. Melody based action games are also challenging and fun to play because they allow the game players to hear the fruit of their success in matching notes to colors/patterns to keys/positions and thereby to produce real melodies. Appendix C lists some further differences between this melody/rhythm game and existing games.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A depicts a schematic diagram of the hardware architecture for a simplified interactive musical instrument game of this invention;

FIG. 1B is a schematic diagram of the hardware architecture for another simplified interactive musical instrument game of this invention;

FIG. 2A depicts a set of rectangular touch areas on an electronic board of FIG. 1B;

FIG. 2B depicts a set of rectangular touch areas on an iPad™;

FIG. 3A depicts a song segment from “Tainted Love”;

FIG. 3B depicts a set of rectangular touch areas programmed for playing the song segment of FIG. 3A;

FIG. 4A depicts a song segment from “Mary had a little lamb”;

FIG. 4B depicts a set of rectangular touch areas programmed for playing the song segment of FIG. 4A;

FIG. 5A depicts a song segment from “96 Tears”;

FIG. 5B depicts a set of rectangular touch areas programmed for playing the song segment of FIG. 5A;

FIG. 6A-FIG. 7D depict various touch areas arrangements and designs;

FIG. 8A depicts a song (Doors “Light My Fire) with three different song segments;

FIG. 8B depicts the lighting of the keys in a keyboard for playing the song segments of FIG. 8A;

FIG. 8C depicts the lighting of the touch areas in a song specific board for playing the song segments of FIG. 8A;

FIG. 8D depicts an implementation of the interactive musical instrument game of this invention in an iPad™ tablet;

FIG. 8E-FIG. 8O depict various touch areas arrangements and designs in the implementation of FIG. 8D;

FIG. 9A depicts the notes for Sweet Child O'Mine song;

FIG. 9B depicts the lighting of the touch areas in a song specific board for playing the song segment of FIG. 9A;

FIG. 9C depicts the lighting of the keys in a keyboard for playing the song segment of FIG. 9A;

FIG. 10A depicts the notes for Bohemian Rhapsody song;

FIG. 10B depicts the lighting of the touch areas in a song specific board for playing the song segment of FIG. 10A;

FIG. 10C depicts the lighting of the keys in a keyboard for playing the song segment of FIG. 10A;

FIG. 11 depicts a block diagram of the process of developing the interactive musical instrument game of this invention;

FIG. 12 depicts a block diagram of the process of playing the interactive musical instrument game of this invention;

FIG. 13 depicts a schematic diagram of the hardware architecture for the interactive musical instrument game of this invention;

FIG. 14 is an another embodiment of the hardware architecture for the interactive musical instrument game of this invention; and

FIG. 15A-15B depict typical examples of song segments;

FIG. 16A depicts the “Mary Had A Little Lamb” song segment;

FIG. 16B-16C depict the corresponding multi-colour lighting of the focused keyboard for the song segment of FIG. 16A played in the C-key and Eb key, respectively;

FIG. 17-18 depict the lighting of focused keyboard for the song segments of FIG. 8A;

FIG. 19A depicts a song segment of “Two Hearts That Beat As One” by U2;

FIG. 19B depicts the lighting of the focused keyboard for the song segment of FIG. 19A;

FIG. 20A depicts a song segment (Roy Orbison's Blue Bayou);

FIG. 20B depicts the lighting of the focused keyboard for the song segment of FIG. 20A;

FIG. 21A-21C depict a simplified right-hand accompaniment pattern to John Lennon's “Imagine” and its representation on the focused keyboard of this invention;

FIG. 21E depicts left (572) and right (574) groups of notes accompaniment pattern to John Lennon's “Imagine” and their corresponding representations on a non-focused keyboard;

FIG. 21F depicts a scale-oriented system for a song in the key of C major and the corresponding non-focused keyboard with multi-color key lighting;

FIG. 22 depicts the display of FIG. 1A and FIG. 13;

FIG. 23 depicts a game scoring indication in the display of FIG. 22;

FIG. 24 depicts an alternative game scoring indication in the display of FIG. 22;

FIG. 25A depicts an alternative keyboard for game 100 of FIG. 13;

FIG. 25B depicts an arcade version of game 100 of FIG. 13.

FIG. 26 depicts a method of showing the players the audio balance between what they are playing and what the recorded band is playing;

FIG. 27A depicts a guitar with multi-color lighted fretboard for the song segment of FIG. 16A played in the C-key;

FIG. 27B depicts a bass guitar with multi-color lighted fretboard for the song segment of FIG. 16A played in the C-key;

FIG. 27C depicts a soprano sax with multi-color lighted keys for the song segment of FIG. 16A played in the C-key;

FIG. 27D depicts an alto sax with multi-color lighted keys for the song segment of FIG. 16A played in the C-key;

FIG. 27E depicts an Electronic Wind Instrument (EWI) with multi-color lighted key for the song segment of FIG. 16A played in the C-key;

DETAILED DESCRIPTION OF THE INVENTION

Referring to FIG. 1A, a simplified (or “song-specific”) interactive melody and rhythm action musical game 700 includes an electronic board/controller 720 and a computing unit 702 connected to the electronic board/controller 720 and to a display 706 and to speakers 703 a, 703 b. The computing unit 702 may be a computer, an Xbox™, a Wii™, a Playstation™, an iPad™, an iPhone™, a Smartphone, a tablet computer, or any other computing or game controlling device. In one example, musical game 700 is implemented in an iPad™ computer 702 that also includes the controller 720, the display 706 and the speakers 703 a, 703 b. In this case, display 706 is a touch enabled display and controller 720 is implemented to be an area of the touch enabled display. The computing unit 702 includes the Real Key Music (RKM) software 750 and a song library 760. Library 760 is a digital storage unit including recorded songs, song fragments, musical arrangements, instrumental musical pieces, and vocal musical pieces, among others. Examples of song segments are shown in FIG. 4A. In other embodiments, library 760 may be a database stored in an external storage device or may be downloaded from an online web-site via a network connection. Speakers 703 a, 703 b may be separate speakers or may be integrated within the computer 702, the electronic board 720, or the display 706. Speakers 703 a, 703 b broadcast sound generated from the playing of the recorded musical pieces and or the playing of the electronic board/controller 720. In some embodiments electronic board/controller 720 includes a second set of speakers dedicated for broadcasting music played on the electronic board/controller itself. In FIG. 1A, the electronic board/controller 720 is shown to be a rectangular board with four colored buttons 722 a, 722 b, 722 c and 722 d. Buttons 722 a, 722 b, 722 c, 722 d are color coded with a constant (non-changing) color. In the example of FIG. 1A button 722 a is green, button 722 b is red, button 722 c is yellow and button 722 d is blue. The buttons may be colored or they may have a colored dot, sticker, or light. In other embodiments, electronic board/controller 720 may have any shape including circular, oval, triangular, hexagonal, polygonal, curved, spherical, cubical, among others, or may have the shape of a keyboard, a Keytar (i.e., a keyboard or synthesizer supported by a strap around the neck and shoulders of the player), a typewriter keyboard, a computer keyboard in which the keys are software mapped to correspond to music notes, a saxophone or other wind-type instrument, a guitar, or any other musical instrument. In other embodiments, the electronic board/controller 720 is shaped as any game pad controller or is integrated with a game controlling device or in general with the computing unit 702.
Electronic board/controller 720 includes only as many buttons 722 a-722 d as are needed to play any of the included songs or a song segments. Some songs or song segments may not use all the buttons. In some embodiments there is an indication that only some of the buttons are needed for a particular song. In other embodiments the buttons that are not used are implied by not using buttons of certain color or shape. As was explained above, a song segment is defined as a partial segment of a song in time, a partial segment in terms of which instrumental or vocal parts were included and/or a partial segment in that it is a simplification of the actual musical notes played. In one example, an electronic board/controller 720 configured to play “Mary Had A Little Lamb” has only four buttons 722 a-722 d, as shown in FIG. 1A. In another example, an electronic board/controller to play “Mary Had A Little Lamb” has only three buttons, depending upon how the melody is written. Each button corresponds to a “pitch” used in the song or song segment being played. “Pitch” represents the perceived fundamental frequency of a sound and in this case the song pitches represent the set of perceived fundamental frequencies that characterize the song or song segment. The buttons may still be ordered by pitch, i.e., high to low or the reverse. As was described above, almost any piece of music can be broken down into song segments that involve a relatively small number of pitches and therefore a small number of buttons can be used to play the son or song segment.
One can play a melody of any length by using a sequence of simplified “song-specific” electronic boards. A sequence of “song-specific” electronic boards is actually a sequence of the same electronic board 720 shown in FIG. 1A in which buttons 722 a-722 d are reprogrammed to generate different pitches corresponding to pitches for a different song or different segment of the song. In one embodiment, electronic board/controller 720 recognizes (i.e., “knows”) it is time to reprogram itself by observing what buttons the player has played and therefore knows when the first segment of the song is over. Alternately, if the player is playing along with a recorded version of the song or even a prompting device as simple as a visual and/or audio metronome, the electronic board “knows” when to switch to a different board setup by knowing when the song started and the time elapsed. In yet another embodiment, the player “tells” the electronic board/controller 720 to switch to a different board setup by pushing a button or a foot-switch, making a gesture or speaking a certain word. The different “song-specific” electronic boards within the same sequence may use a different number of buttons. Therefore, sometimes there are some buttons that are not used. In one example the electronic board/controller has thirteen buttons, but one actually uses an average of seven buttons in a large number of songs or song segments. Song segments that are suitable for beginner players may only use four or five buttons and song segments that are suitable for advanced players may use as many as all thirteen buttons. The buttons do not need to be lighted and usually only need an electrical contact closure. Electronic boards with electrical contact buttons are usually manufactured inexpensively. The unused buttons may be identified or not. The unused buttons may be identified by their position, shape or color. In one example, the unused buttons are selected to be at one or the other “end” of the electronic board, or in a different row, or in a specifically marked area of the electronic board. In another example, it is indicated that the buttons/keys used for a specific song are only the black buttons/keys or the first five buttons/keys. In addition, the buttons/keys that are used may be repositioned so that they are, for instance, all next to one another or within easy reach. The group of buttons/ keys 722 a, 722 b, 722 d used in a song segment are shown in the display 706 as buttons/ keys 712 a, 712 b, 712 d, as shown in FIG. 1A.
Since exactly the same set of physical buttons/keys is used for the different sets of pitches of each song segment, it is possible to always have these buttons/keys coded the same way for identification purposes, regardless of the actual song or pitches, provided that we always order the pitches used in the song in an accommodating and corresponding way. In one example, within a song segment, we always associate “blue” with the lowest pitch actually used, “green” with the second lowest, “red” with the third lowest, and light blue with the fourth lowest. New colors are added as necessary. We then map those color/pitches to a linear set of buttons in which the first is colored “blue”, the second “green”, the third “red”, and the fourth “light blue”. In other embodiments, the white keys of a music keyboard, or the adjacent strings or the adjacent frets of an appropriately equipped stringed instrument or the fingerings of a wind instrument, are tinted in a similar way as “blue”, “green”, “red” and “light blue”. In yet other embodiments, footpads, button of a game controller, areas of a touch sensitive surface, areas pointed at (with appropriate detection) or even facial gestures detected are associated with the colors described above. In some embodiments, a series of tinted or otherwise differentiated light beams are used instead of buttons/keys and one plays melodies by entering fingers, hands, legs, another object or the shadow of any of the mentioned objects into the light beam. These various implementations could lead to some novel and dramatic ways to play melodies and other melodic parts on stage or accommodate the limited physically expressive possibilities of a handicapped person.
Library 160 includes, in addition to the recorded songs, data specific to each song which associate the pitches of the song with specific buttons in the electronic board, as will be described below. The display 706 may be a computer display, a TV, a video game console, an arcade machine display or any other display device. Display 706 displays the image of a virtual electronic board 708 corresponding to the actual electronic board 720, colored lines 711 a-711 c, prompts 714 a-714 c and a target bar (or landing area) 707. The virtual electronic board 708 is designed to have the same number of buttons as the actual electronic board/controller 720. The displayed image is controlled by the RKM software 750. The RKM software 750 causes specific buttons 712 a-712 c of the virtual electronic board 708 to become highlighted with different colors depending on the pitches of the song that is played. The programming of the electronic board buttons 722 a-722 d is also controlled by the RKM software 750. Lines 711 a-711 d diverge from a common starting point and end at a point directly over the buttons 712 a-712 d, that are highlighted. They are spaced apart by non-equal spaces and the common starting point may be off the screen. As different buttons are highlighted during the playing of a song, the end points of path lines 711 a-711 d move to point toward the buttons that are highlighted while their starting point remains fixed. In this way they act as visual pointers toward the buttons that are hit on the virtual electronic board 708 and therefore need to be hit on the actual electronic board 720. Prompts 714 a-714 c run along path lines 711 a-711 c, respectively, and extinguish after they arrive at the target bar/landing area 707. The time prompts 714 a-714 d hit the target bar 707 corresponds to the time the player needs to hit the corresponding buttons on the actual electronic board/controller 720 in order to have a successful score. Buttons 712 a-712 d, path lines 711 a-711 d and prompts 714 a-714 d have the same color, respectively. In one example buttons 712 a, path line 711 a and prompt 714 a are blue, button 712 b, path line 711 b, and prompt 714 b are green and button 712 c, path line 711 c, and prompt 714 c are red. Display 706 also includes a list of the available songs and song segments 118, as shown in FIG. 22. In some embodiments, the target bar/landing area 707 has the shape of the electronic board/musical instrument and lights up if it is hit at the right time to indicate the buttons/keys that are targeted. In yet other embodiments, the target bar/landing area 707 is omitted or coincides with the image of the electronic board 708. In these cases the prompts 714 a, 714 b, 714 d indicating the notes that are to be played continue to move along the path lines 711 a, 711 b, 711 d onto the buttons/ keys 712 a, 712 b, 712 d, respectively, or change direction after hitting the top of the board image 708 or target bar 707. Path lines 711 a-711 d may continue and be visible under the buttons/keys 712 a-712 d. In other embodiments, lines 712 a-712 d are parallel to each other. In any of these embodiments, the score for the note could simply appear on or near the related key rather than following the path of the note.
Referring to FIG. 1B, in another embodiment, the simplified (or “song-specific”) interactive melody and rhythm action musical game includes an electronic board/controller 720 and a computing unit 702 connected to the electronic board/controller 720. No display is included in this embodiment. The buttons/keys 722 a-722 d that need to be played are indicated by lighted prompts 754 a-754 d that move along paths 751 a-751 d, respectively, on the electronic board 720. In one example, lighted prompts 754 a-754 b are mini light emitting diodes (LEDs). In this embodiment, the electronic board/controller 720, the indication prompts 754 a-754 d, the computing unit 702 and the speakers 703 a, 703 b are integrated into one unit 750.
In some embodiments, the system also includes a bidirectional communication mechanism 704 between the computer 702 and the electronic board/controller 720, similar to the one shown in FIG. 1A. Mechanism 704 carries signals that program buttons 722 a-722 d on the actual electronic board/controller 720 to correspond to the specific pitches of a song or a song segment. Mechanism 704 also transmits signals from the electronic board buttons 722 a-722 d to the computer 702. In one example, mechanism 704 is implement via bidirectional musical instrument digital interface (MIDI) communications. MIDI is an industry standard protocol that enables electronic musical instruments and computers to communicate, control, synchronize and exchange data with each other. A MIDI instrument or controller transmits “event messages” such as the pitch and intensity of the musical notes that are played, control signals for parameters such a volume and vibrato and clock signals to set the tempo. When a musical performance is played on a MIDI instrument (or controller) it transmits MIDI channel messages from its MIDI Out connector.
A reverse MIDI keyboard system may also be used to light up buttons/keys 722 a-722 d in the electronic board/controller 720 from a game controller. MIDI keyboards normally transmit pitch and velocity (among other things) to a processor. The pitch is an integer between 0 and 127 and corresponds to a chromatic pitch. Similarly the velocity is also in that range with ‘0’ being used to indicate off. If we reverse this and consider the velocity number to be a color instead (for instance a color selection from a list of up to 127 colors with 0 representing off), then the processor controlling the button lighting is analogous to a keyboard sending data to the processor, except that it is in the reverse direction. Both include pitch. Instead of 127 possible degrees of velocity we send up to 127 variations of color, using the same bits in the message format that would have been used to send velocity. It is almost as if the computer is pressing a note in return and the harder it presses the note, the more different colors it is able to achieve.
In other embodiments, keys wired in a way similar to a computer keyboard are used. In particular, most Windows PC and Mac keyboards do an adequate job of relaying both key presses and releases to the processor in a timely manner. In these cases, the key-caps of keyboard are replaced with colored key caps. In one example, a row of keys on a computer keyboard is programmed to correspond to the pitches used in a song. In this example, the row A-S-D-F-G-H-J-K-L-;- is used. The keys are marked with colored stickers or are replaced with colored key-caps. One or more players can use the programmed computer keyboard to play alone or in cooperation with each other or against each other using the songs stored in the library 760.
As was mentioned above, the computing unit 702 may be an Xbox™, a Wii™, a Play station™, an iPad™, an iPhone™, a Smartphone, a tablet computer, or any other computing or game controlling device. These type of controllers have very limited digital output bits, which makes it difficult to integrate them with other gaming systems including the systems of this invention. This integration problem is addressed by utilizing the motor output signals of the controlling device. In particular, an Xbox™ controller has two motor outputs. However, the output to these motors is not a steady DC signal that could be considered to be a zero or one. Instead, if a motor is on, the output to it is a steady pulse wave with controllable duty cycle (pulse width). The software specifies a value between 0.0 and 1.0 to the controller and it delivers a wave to the motor with a duty cycle from about 5% to 50%. A table shows what output values produce what duty cycle. By measuring the width of the resulting pulses on the controller, using digital or analog logic or a microprocessor, it is possible to retrieve digital data. Approximately 40 pulses occur per second. By timing the pulses, each pulse can be categorized into several length categories. If there are, for example, eight distinguishable length categories, that yields three bits of information, meaning that we could use each of the two motor signals to transfer reliably up to 120 bits/second, even if we distinguish only eight different lengths of pulses. Combining the two motor signals gives us 240 bits/second. For the lighted keyboard system 700, only one bit of data need be transmitted for each key position since we need only distinguish between on and off. If colors are used in a predetermined successive order (as they can be by the game), then we simply give each “on” key the next successive color. This means that for a 25-key keyboard, we can relay the necessary information for a change of lighting in about 1/10 second, which is timely enough for these purposes.
Another scheme that allows us to control the keyboard lighting with about 1/10 second response, even if the output connection to the keyboard was limited to 10 bits/second includes the following. Since the one or more successive lighting changes for a particular song are known in advance of playing the song, they could each be transmitted before the playing of the song began, even if it took several seconds to do so. In this case, the keyboard has a small amount of memory that remembers each of the lighting configuration for each time a change in lighting occurs, and then, by simply transmitting one bit to start the song and a bit each time a lighting change is to occur the processor can control the lighting of the keyboard even under extremely low data communication conditions. These considerations are relevant for most gaming platforms because they frequently only support a very limited number of controllers. The easiest way to add a new piece of peripheral hardware to such a system is to have it emulate a controller that the gaming platform already supports.
The advantages of the simplified interactive melody and rhythm action musical game includes the following. An electronic board, or a controller or a musical instrument with a small number of buttons/keys is less intimidating than one with many more buttons/keys. An electronic board or a controller or a musical instrument with a small number of buttons/keys is similar to a simple video game control in complexity and therefore easy to use. A player with relatively little experience can play a melody of any length by using a succession of programmed simplified “song-specific electronic keyboard pitch arrangement”. In some embodiments, the electronic board is designed to be easily used by players with many types of physical disabilities. The electronic board/controller/musical instrument is portable and is also adaptable to other interesting interpretations (for instance, waving arms or feet to get different pitches).
Referring to FIG. 2A and FIG. 2B, in this example a touchscreen 120 (such as the Apple iPad™) is used in the prototype implementation of the electronic board. In touchscreen 120, touch areas 122 a-122 d are defined on the screen through a software. These areas 122 a-122 d may be of any size, shape and arrangement. In this example, areas 122 a-122 d are a series of rectangular areas in a row. In this example, touch areas 122 a-122 d are colored to make the illustrations cleaner and because it can be an aid to someone playing a game or learning music. However, it is not necessary to use a color for each button or even to use color at all. And it is also possible to use other differentiators than color, such as shape or size.
It is assumed that a “song segment” is a subset of the note pitches of a piece of music, delimited optionally by time section (such as “verse”, “chorus”, “intro” . . . ) within the performance, instrument being used or some other criteria (such as “right hand notes of the piano part”). A “song segment” from the well known keyboard synthesizer “riff” of Tainted Love (made famous by Soft Cell and others) is shown in FIG. 3A. This “song segment” spans a range from G to Eb on the treble clef. This spans 8 chromatic pitches (G-Ab-A-Bb-B-C-Db-D-Eb) and would normally be performed on a keyboard of at least 25 keys (two octaves) since it spans two C-to-C octaves. However, upon examination of the song segment, we find that there are actually only 4 pitches played within this particular song segment:

G-Bb-C-Eb
Thus, if the performer has at least four different keys to press (or gestures to make or pads to stomp or anything else discernibly different), they can perform this song segment accurately provided that those four keys or other mechanisms are tuned to play G, Bb, C and Eb. The other keys are simply not required for this song segment and can therefore be omitted. Using the example above, we could use the setup in FIG. 3B to play the song of FIG. 3A.

In another example, if we were using the same physical or touchscreen keyboard to play Mary Had A Little Lamb, shown in FIG. 4A, we could tune it as shown in FIG. 4B. If a song segment requires a different number of pitches, we can programmatically add additional buttons or use additional switches on a physical device. For instance, this combo organ part from 96 Tears (made famous by Question Mark and the Mysterians) uses 6 pitches from a range of 10 chromatic pitches and would typically be performed on a keyboard of two octaves (25 keys) or more, as shown in FIG. 5A. However, here we do it with just six buttons and still can accomplish the original performance, as shown in FIG. 5B.
As was mentioned above, these “buttons” need not be actual buttons or be colored or follow a regular pattern. For example, they could be arranged in other ways in order to be more appealing or more dramatic to a particular audience, to fit into a theme or to coordinate with other activities or needs. Examples of the various button shapes and arrangements are shown in FIG. 6A-FIG. 7C.
FIG. 8D shows an implementation on an iPad™, where we take advantage of its self-contained touch screen, computer and audio system. In FIG. 8D only the pitches used in the song have colored keys 622 a-622 e. In FIG. 8E, this effect is enhanced by fading the keys that are not used in the song somewhat, though still maintaining a look similar to a standard keyboard. In FIG. 8F, the keys not used are almost shadow-like. In FIG. 8G, they are not visible at all. Each step along the way makes it more obvious to the player at the quickest glance which keys are going to be used and which are not. Since this lighting is shown even before the song begins, it also helps the player to mentally and physically prepare for the song, all of which results in a better performance and better learning retention by the player.
In FIG. 8H, we have completely removed the keys not used in the song and adopted a simple setup and spacing for the keys 622 a-622 f used in the song. It is important to note that, unlike a rhythm game, the player is still actually playing the melody when they use this song specific keyboard because each key always plays the same pitch for the duration of the song and all pitches used in the melody are still present. Because this is a very distinct visual simplification from the original piano-style keyboard, novices play the songs much better. And since they are still hearing themselves play and learning the timing and relative pitches of the notes, they are able to transfer these skills backwards with relative ease back to the conventional keyboard, especially with the confidence they have gained in actually playing the song.
In FIG. 8I, we see one further benefit of the song-specific keyboard. Since only the pitches present in the song are used and since the keys have a regular and simplified layout, we can transfer this screen setup to a much smaller device such as an iPhone™ and still leave the touch points far enough away from one another that the player is able to select them easily and accurately.
For some songs, the simplification is even more drastic, eading to positive player experiences even more quickly. FIG. 81-FIG. 80 show the same succession of keyboard styles that would be used for the melodic synthesizer riff of Tainted Love, which only involved four distinct pitches and therefore only four keys 622 a-622 d are used.
FIG. 7A and FIG. 7B depicts an image of structures on a stage set. The performer may stand in front of them and play on one structure with each hand. Or, arranged differently, a performer might touch one with a hand and the other with a foot. Or they could be touched by two different performers. Even though the colors might match from one to another, they could still be different sets of pitches used in a particular song segment. And since these are song-specific, they could be automatically reprogrammed to different pitches depending on the song segment being played at that particular time. For example, the performer would be playing one melody at one part of the show and a different melody at another part.
The coloring need not be different for each pitch and may follow a color scheme based on some other performance or aesthetic need. For instance, the implementation of FIG. 7C has only two colors, but is decorative and could perform any song-segment with six different pitches or less.
The song-specific concept can be used for song-segments with any number of pitches. For example, FIG. 7D represents a decorative touch pad designed to be able to play song segments with up to ten pitches. The principle remains that the keyboard or other structure need only be implemented and programmed to control as many pitches as are used in a particular segment of a song rather than a larger set of more general scalar or chromatic pitches. Then in another segment, possibly following immediately after this one, the keys are immediately reprogrammed to a different set of pitches corresponding to the next song segment played.
People who have limited range of motions, such as having only eye movement, moving only a few fingers or other limitations often have limited chance for creative and/or playful activities. The pitches of a song-specific keyboard are mapped by sensors or other means to the few motions that a person can make, and this allows the person with the limited range of motion to play musical games or experiment with creating music using a set of chosen pitches or using the same pitches as in a preferred song-segment. The motion detection need not be through a physical device. A mechanism such as Microsoft's Kinect™ may be used to detect the motions and interpret them as different keys of a song-specific keyboard. In one example, a patient's leg motions are converted to the notes of a scale, similar to a slide trombone. However, adding an implementation of song-specific technology, means that instead of being limited to ‘n’ notes of the range of a particular modal or chromatic scale, the patient can now play any song segment with up to ‘n’ pitches, thereby greatly increasing the amount of music he can perform.
A song-specific implementation can dramatically reduce the number of keys needed for a particular song-passage, thereby reducing the number of keys from what might be found on a professional music keyboard to what might be inexpensively found on a cereal box toy.
Referring to FIG. 8A, the well-known opening to “Light My Fire” made popular by The Doors is divided into four “song segments”: 532 a, 532 b, 532 c and 532 d. FIG. 8B shows the keys needed to play each part on a conventional 25-key keyboard. As can be seen, this passage requires the larger part of a 25-key keyboard. It uses chromatic pitches from C# (122 a) to D an octave higher (122 c) for a range of 14 chromatic pitches (C#-D-D#-E-F-F#-G-G#-A-A#-B-C-C#-D), 8 white keys and 6 black keys. The corresponding song-specific keyboard is shown in FIG. 8C. The pitches under the keys are for reference and are not displayed. They show how the pitches change for each of the buttons during the four sequential song segments (532 a, 532 b, 532 c and 532 d). The performer need not be aware of the actual pitches or even that they have changed. As long as they perform the correct buttons in time, their performance will sound exactly as if they had performed it on the 25-key keyboard shown. Far less awareness of key position and agility is required on their part and a far less inexpensive keyboard can be used. This could allow a performer who is not a musician to more easily play a musical part as part of a production. They could be actually playing the notes, adding their own expressiveness in volume and note timing and duration as opposed to a recording played from off-stage, for example.
As used in an iPad™ implementation, the song-specific keyboard allows non-musician players to more easily play song segments they might be required to play as part of a game. It also makes it possible to do far more in the limited screen size of an iPad™ as compared to the relative ease in positioning pitches on a full-sized music keyboard (about 9″ wide for the iPad™ compared to about 19″ wide for a conventional 25-key keyboard). This is especially important because in a game the limited space of the iPad™ screen must be shared with other elements of game play.
The game display of Rock Band 3™, Guitar Hero™ or Tap Tap™ may at first appear to be similar to the song-specific keyboard. However, there is a fundamental and important difference. The song-specific keyboard defined here has a one-to-one correspondence between the buttons (or other mechanisms or implementations) of the keyboard and the pitches being used in the song segment being performed. That is not true of the three other game displays mentioned above. This is very important because it means that once someone has learned to play a song segment on the song specific keyboard, that same song segment can be played on a piano or similar instrument simply by putting one's fingers on top of the keys for the pitches used in the song segment and repeating the finger motions used on the song-specific keyboard. In other words, once one's fingers are in place, that song-segment can be played on a real piano keyboard in the exact same way it was performed on the song-specific keyboard. That is not true of the other products mentioned because one button on those games results in the playing of different pitches at different times. If the same motions from one of these games were attempted on a real piano keyboard, the fingers would have to move from place to place within the performance in order to get the multiple pitches corresponding with the button. Not only is this more difficult, but the performer would generally not have any idea of which piano key corresponded to a particular button at any particular time.
To be specific, in the song game around “Sweet Child O′ Mine” in Expert mode Guitar Hero 2, within the first five seconds of the performance sometimes the first button from the left results in a musical ‘Db’ pitch and sometimes it results in a musical ‘Gb pitch. A video performance of this song in the game was available at http://www.youtube.com/watch?v=VY_DOeCCbCg. The first two seconds, approximately, of the performance described is shown in FIG. 9. The apostrophe indicates pitches in the higher octave.
Similarly, a Tap Tap Revenge™ 3 performance of Fireflies now exists at http://www.youtube.com/watch?v=1A9-sgX7qMg. In the first part of the intro, note that the number of pitches used is much more than the three buttons used by the player.
Similarly, in Rock Band 3™ a performance of Bohemian Rhapsody, at about 2:50 a guitar lead line starts as shown below. It has many more pitches than the five buttons available to the player, but we will just consider the first five notes, shown in FIG. 10A.
The numbers indicate the buttons numbered from the left, or respectively green, red and yellow in color.
Note that button 2 plays an F at one point and then perhaps a second later plays a Bb. Also note that the pitch of ‘G’ is played by button 3 at one point and by button 1 at another. Clearly there is no one-to-one correspondence between buttons in the game and pitches in the music.
In any of the last three existing products shown, it is not possible to play the passage on a piano by merely placing your fingers on the appropriate keys and making the same finger motions in rhythm that you did in the game. Compare that to the two examples of the same two passages illustrated above as they might be performed on a song specific keyboard, each with an illustration of a song-specific keyboard that could be used and the corresponding finger positions to play the same passage on a piano, as shown in FIGS. 9A, 9B and 9C and in FIGS. 10A, 10B and 10C.
Referring to FIG. 11, the process 200 of developing the game includes the following steps. First a song or a song segment is selected (210) and the main pitches of the song or song segment are associated with specific keys in the electronic keyboard (202). The data that associate the song “pitches” with keys are stored in a data file (203) and a mechanism is provided for lighting specific keys on the keyboard based on signals received from a computer/controller (204). An electronic display is also provided that receives signals from the computer and displays an image of a virtual electronic keyboard and prompts directed to specific keys of the displayed virtual keyboard (205).
Referring to FIG. 12, the process of playing of the game 210 involves selecting a song and recalling the song and its associated pitches/keys data from the library (211). Next, the computer sends signal to the electronic keyboard that activate lighting in specific keys that are associated with the pitches of the selected song (212). Each key is lighted with a different color. The image of a virtual electronic keyboard is displayed on the display, as well as prompts directed to the specific keys of the virtual keyboard associated with the pitches fort he selected song (213). Next, the playing of the song on the computer/controller is initiated (214) and during the playing of the song the player attempts to hit the specific lighted keys on the keyboard that associated with the specific song pitches that are played (actual hit) at the same time the electronic prompt hits the corresponding specific lighted keys of the virtual keyboard (virtual hit) (215). The player is rewarded based on the timing and duration of the actual hit as compared with the virtual hit.
Referring to FIG. 13, in another embodiment, an interactive melody and rhythm action musical game 100 includes an actual musical instrument 120 and a computing unit 102 connected to the musical instrument 120 and to a display 106 and to speakers 103 a, 103 b. The computing unit 102 may be a computer, an Xbox™, a Wii™, a Playstation™, an iPad™, an iPhone™, an Android™ phone or any other computing or game controlling device. The computing unit 102 includes the Real Key Music (RKM) software 150 and a library 160. Library 160 is a digital storage unit including recorded songs, song fragments, musical arrangements, instrumental musical pieces, and vocal musical pieces, among others. Examples of song segments are shown in FIG. 4A. In other embodiments, library 160 may be a database stored in an external storage device or may be downloaded from an online web-site via a network connection. Speakers 103 a, 103 b may be separate speakers or may be integrated within the computer 102, the musical instrument 120, or the display 106. Speakers 103 a, 103 b broadcast sound generated from the playing of the recorded musical pieces and or the playing of the actual musical instrument 120. In the embodiment of FIG. 14, musical instrument 120 includes a second set of speakers 123 a, 123 b dedicated for broadcasting music played on the musical instrument itself. In FIG. 13, the musical instrument 120 is shown to be an actual electronic keyboard. In this example, musical instrument 120 is a 25-key electronic keyboard manufactured by M-Audio. It includes 15 white keys and 10 black keys. In other embodiments, musical instrument 120 may be any type of electronic keyboard or synthesizer having different number of keys and controls, a Keytar (i.e., a keyboard or synthesizer supported by a strap around the neck and shoulders of the player) a computer keyboard in which the keys are software mapped to correspond to music notes, a saxophone, a guitar, or any other musical instrument. Library 160 includes, in addition to the recorded songs, data specific to each song which associate the pitches of the song with specific keys in the keyboard, as will be described below. “Pitch” represents the perceived fundamental frequency of a sound and in this case the song pitches represent the set of perceived fundamental frequencies that characterize the song or song segment. The display 106 may be a computer display, a TV, a video game console, an arcade machine display or any other display device. Display 106 displays the image of a virtual keyboard 108 corresponding to the actual keyboard 120, colored lines 111 a-111 c, prompts 114 a-114 c and a target bar 107. The virtual keyboard 108 is designed to have the same number of keys as the actual keyboard 120. The displayed image is controlled by the RKM software 150. The RKM software 150 causes specific keys 112 a-112 c of the virtual keyboard 108 to become highlighted with different colors depending on the pitches of the song that is played. Lines 111 a-111 c diverge from a common starting point and end at a point directly over the keys that are highlighted. They are spaced apart by non-equal spaces and the common starting point may be off the screen. As different keys are highlighted during the playing of a song, the end points of lines 111 a-111 c move to point toward the keys that are highlighted while their starting point remains fixed. In this way they act as visual pointers toward the keys that are hit on the virtual keyboard 108 and therefore need to be hit on the actual keyboard 120. Prompts 114 a-114 c run along lines 111 a-111 c, respectively, and extinguish after they arrive at the target bar 107. The time prompts 114 a-114 c hit the target 107 corresponds to the time the player needs to hit the corresponding keys on the actual keyboard 120 in order to have a successful score. Keys 112 a-112 c, lines 111 a-111 c and prompts 114 a-114 c have the same color respectively. In one example key 112 a, line 111 a and prompt 114 a are red, key 112 b, line 111 b, and prompt 114 b are green and key 112 c, line 111 c, and prompt 114 c are blue. Display 106 also includes a list of the available songs and song segments 118, as shown in FIG. 22.
The system also includes a bidirectional communication mechanism 104 between the computer 102 and the instrument 120. Mechanism 104 causes specific keys 122 a-122 c on the actual keyboard 120 to get highlighted based on signals controlled by the RKM software 150 and received from the computer 102 during the playing of a song. Mechanism 104 also transmits signals from the keyboard keys to the computer 102. The highlighted specific keys 122 a-122 c on the keyboard 120 correspond to the pitches of the song that is played. In one example, mechanism 104 is implement via bidirectional musical instrument digital interface (MIDI) communications. MIDI is an industry standard protocol that enables electronic musical instruments and computers to communicate, control, synchronize and exchange data with each other. A MIDI instrument or controller transmits “event messages” such as the pitch and intensity of the musical notes that are played, control signals for parameters such a volume and vibrato and clock signals to set the tempo. When a musical performance is played on a MIDI instrument (or controller) it transmits MIDI channel messages from its MIDI Out connector.
A typical MIDI channel message sequence corresponding to a key being struck and released on a keyboard includes the following:
The user presses the middle C key with a specific velocity (which is usually translated into the volume of the note) and the instrument sends one Note-On message.
The user changes the pressure applied on the key while holding it down—a technique called Aftertouch and the instrument sends one or more Aftertouch messages.
The user releases the middle C key, again with the possibility of velocity of release controlling some parameters and the instrument sends one Note-Off message.
Note-On, Aftertouch, and Note-Off are all channel messages. For the Note-On and Note-Off messages, the MIDI specification defines a number (from 0-127) for every possible note pitch (C, C #, D etc.), and this number is included in the message.
In one example, a MIDI channel message includes the following four parts:
(1) the “command” - - - in this case NoteOn (0x80) or NoteOff(0x90)
(2) the “channel” - - - any channel could be consistently used, for example channel 1.
(3) data1 - - - this is the MIDI pitch
(4) data2 - - - for NoteOn, this is the “velocity”, which in the present invention is used to indicate the type of highlighting. NoteOff is then used to turn off the highlighting for that note.
Many keyboards do not generate “Aftertouch” messages and sometimes a NoteDown message with a velocity of zero actually indicates a Note release, making it a substitute for the NoteUp messages. In this implementation the velocity is used as the note highlighting for messages that the processor sends to the keyboard. If the velocity were non-zero, the velocity could be interpreted as an indication of how a note should be highlighted, selecting one of up to 127 possible colors, for example. Alternately, the seven bits of the velocity could be interpreted as a color, using for instance two bits for red intensity, three bits for green intensity and two bits for blue intensity. This would result in three possible red illumination levels (besides off), seven possible green intensity levels besides off, and three possible blue levels besides off, which can then be combined in a red-green-blue (RGB) fashion into one of 127 possible colors. Since the colors need to be easily distinguishable from one another in the described application, it is hard to imagine a situation in which more than 127 color shades might be needed. But if that should become necessary, additional bits could be conveyed. For instance, if two notes on messages for the key were sent in rapid succession, the velocity data of the second message could be interpreted as additional bits, giving a total of 14 bits instead of 7, and 17,367 shades besides completely off.
In implementations where independent signals are used for the three RGB components, one MIDI channel is used for red intensity, one for green intensity and one for blue. Similarly, the highlighting may utilize a subtractive color technology rather than an additive one, and the intensities of concern will be cyan, yellow and magenta instead of red, green and blue.
In alternate implementations, the “highlighting” method may not use color at all and it may have very few or even just one level (or color) other than off. For instance, keys may be physically moved slightly to show that they were highlighted or made to vibrate or physically “hum” or be of a different temperature or conduct heat differently, which one could easily discern through touch.
Another way to convey the control information through MIDI is to convey it as “controller” information rather than as “note channel” information. “SysX” technology may be used in this implementation. The exact use of channels and message content is defined almost arbitrarily and combined in various ways with other messages, but the meaning attached to the messages includes the highlighting message info above.
In any of these systems, there may be “panic” messages which causes all highlighting to reset to off. In addition to or in place of the messages above instructing single keys to highlight, “bulk” messages may be used which would give the keyboard a new highlighting scheme to use in place of the current one. These messages may be implemented in MIDI, XML, JSON or some other technology.
In one particular implementation, the music keys are considered to be numbered from 0 successively left to right, regardless of their MIDI pitch. In this implementation the f instruction: “2,0;3,1;4,2;7,3;10,4;11,5” tells the keyboard to discard any previous lighting and to
Highlight key number 2 in color 0
Highlight key number 3 in color 1
Highlight key number 4 in color 2
Highlight key number 7 in color 3
Highlight key number 10 in color 4
Highlight key number 11 in color 5
The remainder of the keys are not to be highlighted. In this system, the actual colors are taken from a predefined, fixed list of colors. For example, 0:blue, 1:green, 2:red, 3:yellow, 4:cyan, 5:orange. If the keyboard starts with a ‘C’, the corresponding pitches will be D, D#, E, G, A# and B, respectively. However, it would also be possible to have a system which downloaded the list of colors at the beginning or periodically as necessary or desirable.
In the embodiment of FIG. 13 mechanism 104 includes light emitting diodes (LEDs) or any other lighting source used to light keys 122 a-122 c in response to electronic signals received from the computer 102. In the embodiment of FIG. 14, keys 122 a-122 c are lighted by projecting light onto them through a projector 130. Projector 130 receives the image of the lighted virtual keyboard 108 from the display 106 and projects it onto the actual keyboard 120. The image of the virtual keyboard is aligned with the actual keyboard so that their corresponding keys coincide. The projector beam may be directed directly onto the keys or may be reflected onto the keys by a mirror. In other embodiments where multiple keyboards are used, multiple projectors may be used or a single projector with a multiple beam splitting. In yet other embodiments the beam of a single projector is separated into multiple beams with mirrors or prisms.
Referring to FIG. 15A and FIG. 15B, examples of song segments used in playing the interactive game 100 include the synthesizer part of the song “Whip it” by Devo 500 and the “bomp bomp bomp” ending in the song “Sweet Caroline” by Neil Diamond 510. The concept of a song segment, as used here, means a musical “part” played by one or more instruments or voices for part or the duration of a musical selection.
The division of a song and its vocal and instrumental parts into song segments is arbitrary. A person or process breaking a song into song segments can make choices to please the needs and abilities of an intended audience, opting to try to keep the number of pitches used near or below a particular number. In one example, a simple automated process, could decide to use 16-bar segments of tracks in a Musical Instrument Digital Interface (MIDI) file for the song segments. That would work, but better results can be achieved by considering the aspects of particular songs in a more detailed manner, whether manually or in an automated process. In one embodiment, represented by the examples above, the number of keys used within a song segment is kept around seven. In other song segments between three and eleven keys are used. This requires skill and discretion in choosing the song segments. Once, the song segments are defined, each can be seen to include certain pitches (or “perceived fundamental frequencies” or keys) and not others and that defines how it will appear and be used on the keyboard.
A “song segment” may contain a certain number of pitches, some of which may be played more than once. In one example, the melody to “Mary Had A Little Lamb”, as commonly played, has four different pitches, but all are used more than once, as shown in FIG. 16A. This concept is similar to what would occur on one line of a music score for a certain amount of musical time (possibly the entire selection), as may be measured in musical beats or measures or by Musical Instrument Digital Interface (MIDI) ticks or other timekeeping system.
It is not necessary that this target song segment be performed in the original work by the same type of instrument used by the player. For instance, a keyboard could be used to play a vocal melody line, a synthesizer used to play a brass line or a guitar lead, a guitar used to play the notes of a string section line, among others This sort of substitution often happens in real musical ensembles, so it is not even unnatural to do it here.
The song segment may contain more than one pitch played at a time. For instance, it may be a left or right hand part of a piano selection. Alternatively, the pitches played by both hands maybe considered together as the pitches of a song segment. Although, as we shall see, there are advantages to defining the song segment in such a way as to include a limited number of pitches, that restriction is not included in how we are defining a song segment.
It might even be played by more than one instrument or player if, in the mind of the listener, it might meld together into a musical concept. For example, a set of tympani is a set of individual tuned drums that are physically independent from one another, but the melodic line produced by using all of these drums together might be considered to be a single song segment. In another example, classical composers at times have written melodic lines that run from the violins through the violas ending, for instance, on the cello. The composer may conceive and the listener may hear this as a single melodic line, even though it is played serially by different players on different instruments. The part played by the player may be transposed from the original. It might even be simplified from the original.
As was mentioned above, the keyboard 120 has some means to highlight a set of keys on the keyboard (or fret positions on the fretboard). Though it is of some use to highlight all of them in the same way, greater benefit can be obtained by varying each highlighted key in a different way. For example, on a keyboard with lighted keys, the non-highlighted pitches might be unlit and the highlighted pitches lit. Though it is of some use for them to be all one color, it would be a functional improvement if more than one color were used and better still if each color used was unique and easily distinguishable.
For instance, using the “Mary Had a Little Lamb” example in the key of C, the pitches would be C, D, E and G from lowest pitch to highest pitch. We might light them respectively blue, green, red and yellow 122 a 122 b, 122 c, 122 d, respectively, as shown in FIG. 16B. Alternatively, if only two colors were available, we might light them bright green, bright red, dim green and dim red. Alternatively, even simply green, red, green, red if necessary, referring to them when necessary as lowest green, lowest red, highest green, highest red. If they were all one color, they could be lowest selected pitch, 2nd lowest selected pitch, 2nd highest selected pitch, and highest pitch. It is desirable to easily distinguish each pitch uniquely, but some benefit can be derived when designations on selected pitches need to be duplicated or even in simply being able to very quickly distinguish the current selected pitches from those not selected.
Using the four color representation, one could think of “Mary Had a Little Lamb” as red-green-blue-green-red. However, on keyboard 120, not every key is colored. In fact, all of the keys which are not selected are not colored or colored the same or identified by some unique changeable characteristic. The keys that are used are colored, as was described above, and remain colored during the entire duration of the song segment. Therefore at a glance, the player can tell which keys do not have to be considered for a song segment and which ones will be used. The keyboard 120 ideally illustrates this for all the keys even before the playing of the song segment begins. This allows the player to quickly consider how to place his hand or hands, or feet for a pedal clavier, in such a way as to be able to easily and quickly access those keys, ideally even before the playing of the song passage begins. This described the concept of the “focused” keyboard.
This is very different from approaches that indicate one or sometimes more notes on a keyboard or on a keyboard representation just before they are used, such as Piano Wizard or Synthesia. In these prior art cases, the keys have all the same or different colors and there is not any form of constant highlighting of the keys used in the song and, more importantly, constant non-highlighting of all the keys not used in the song, even when they occur physically between keys that are used, such as the F in “Mary Has A Little Lamb”.
In the focused keyboard 120 of this invention, it would be considered functionally better if the same colors were used. The present approach does not use one color for C's, another for D's, and so on. Instead we use one color for the lowest note actually used in a particular song segment, another for the second lowest note actually used, and so on. Therefore, if we use a different song segment, or a song segment played on a different key, we may use the same colors, but they will generally be on different keys since the keys used by the new song segment or the song segment played on a different key would be different. In the example of FIG. 16C the song segment of “Mary Has A Little Lamb” is shown played in D-#(Eb) key and the pitches would be D#(Eb),F,G and A#(Bb) from lowest pitch to highest pitch. We light them again as blue, green, red and yellow 122 a 122 b, 122 c, 122 d, respectively, but are on different keys than in FIG. 16B, where it was played in the key of C major.
It is possible that a song segment might encompass an entire song from start to finish, such as in “Mary Had A Little Lamb” above. However, in many cases, more than one song segment might be used in succession. For instance, there might be one song segment for the intro, one for each verse, one for the first part of the chorus, one for the remainder of the chorus and one for the ending. Since it is desirable to have not more than a few pitches in each song segment (for example, less than twelve and, when reasonable, no more than six), in complicated musical passages or those involving chords, it might be desirable to change to a new song segment quite often, sometimes even within a measure.
Referring to FIG. 8A, in an example from the organ intro to the Doors' “Light My Fire” there are four song segments 532 a (blue), 532 b (green), 532 c (pink), 532 d (yellow). The colors used are to distinguish one song segment from another and do not relate to the keyboard color lighting. For each of the four song segments, the keyboard lighting may respectively be 531 a, 531 b, 531 c, 531 d, as shown in FIG. 8B. The coloring strategy here is to assign colors consistently from left to right. As shown in FIG. 8B, the same key does not always have the same color though out the different segments.
Many alternative coloring strategies can be accommodated by the focused keyboard 120. Referring to FIG. 17, here is the result of a strategy that keeps a key the same color if it appears in two successive song segments. However, even in this strategy, a color is not constantly associated with a key or pitch name. For instance, the ‘A’ is green 122 e at the start 531 b and red 122 c at the bottom 531 d. In other embodiments, the focused keyboard 120 may employ a strategy in which each pitch remained the same color with the colors perhaps chromatically assigned. That would result in something similar to the set of pictures 531 a-531 d, shown in FIG. 18.
A focused keyboard's 120 color (or other designation) selection could in some cases be set up specially for something highly dependent on the particular song segment, for instance making things easier to grasp by capitalizing on some symmetry in the music. Referring to FIG. 19A, a passage played on bass synthesizer in U2's pop song called “Two Hearts That Beat As One” 540 is represented on the focused keyboard 120, as shown in FIG. 19B. Here there are duplicated colors (which is usually a disadvantage), but in this case they simplify the situation for the player. If the player decides to play with two hands, the player can see how to place the two hands and guess that they will be led to play notes in similar ways. Even if the player uses only one hand (as would be done in more professional situations), the layout in this case still helps to explain how the notes are used in this case. If this were played with the left hand, a professional would generally use the thumb on the right three notes and other fingers for the lower notes, possibly just the little finger, given the nature of this example. In any event, the focused keyboard 120 helps the player to quickly see what is expected in performance.
Referring to FIG. 20A, and FIG. 20B, in the example from the marimba part of Roy Orbison's Blue Bayou 550, there's a similar usage, but in this case identical or similar colors are used to cue the user of keys that will be played together, even though they are not the same pitch name.
There may be circumstances that make one scheme preferential over another, such as cost considerations or a principle in a music lesson being taught. For example, the scheme of FIG. 20A, FIG. 20B, has the advantage that each key need only display one color. For instance, twelve distinguishable shades are needed and similar shades are on adjacent keys where, perhaps, contrasting colors would be preferable for quick recognition, particular if coordinating with rapidly moving colors on an associated computer display. Also, there are only four notes highlights, but two of them are F's and therefore have the same color. Also note that in the first scheme shown in FIG. 19B, if one is playing the song segments with right hand only, positioning one's thumb on the blue note quickly starts the user towards what is often a favorable hand position for that particular song segment. The user would get used to associating ‘blue’ with the lowest note. Yet, adjacently used notes have contrasting colors whether they are very close in pitch or not.
The controlling software or hardware 150 used with the focused keyboard 120 may always consistently use the same approach to key coloring. Alternately, it may use different strategies at different times, either chosen by the user's preference or chosen within the usage context, for instance, keeping pitch colors the same when pitch-oriented lessons were given and keeping the color for the thumb key the same when teaching lessons about fingering.
Ideally the game or other mechanism allows the player to first work with each song segment individually, possibly even allowing it to be slowed down or advanced a note at a time. That way, when the song segments are used together and the highlighting changes (ideally as the last note of the previous segment is being played), the user is ready and expecting the “focus” change since they have already played the song segments individually. In another possible embodiment, a focused keyboard 120 or fretboard could be used without playing with recorded sounds or even trying to play in rhythm. It would still help the player to only focus on the keys used for the particular passage.
The present system doesn't lead the user through, note by note. Instead, in system 100 the notes actually played during a song segment are highlighted, whether or not they are a part of a particular chord or scale. Rarely this might coincide to come up with the same set of keys, but usually it would be quite different. That's because many of the song segments are melody or counterpoint lines and these usually do not simply follow the notes of a chord or scale up and down. In one example, a song segment covers a chordal accompaniment pattern that includes playing a C and F chord, possibly in “broken fashion”. The notes of the C-chord are C-E-G, the notes of the F-chord are C-F-A and the notes for the entire C-Major scale are C-D-E-F-G-A-B-C. The highlighting of the notes on the keyboard is based on the notes that are actually played in the song segment, i.e., C-E-F-G-A, as shown in FIG. 21B and FIG. 21C. This is neither the notes of a chord nor the notes of a scale.
Expanding further with a musical example, a simplified right-hand accompaniment pattern to John Lennon's “Imagine” is shown in FIG. 21A. Here we have included about twenty seconds of a useful musical thing to play and will only need to highlight five notes to cover that duration. Anybody will easily understand what is being indicated and how to play it and most will succeed, in this case even at full tempo, within a few tries of playing. Even without any background recording, the playing of this song segment at the tempo of the song with a piano sound is distinctive enough that many listeners might quickly identify the song as John Lennon's Imagine. That's unusual considering that the player may not have any real musical experience, but it seems to be a common occurrence with song segments learned from the game in this style. On the focused keyboard instrument 120, the keys for these pitches C-E-F-G-A are highlighted because these are exactly the ones included in the above song segment, as shown in FIG. 21B. Ideally, each of these keys 122 a-122 d are lit in a different color, i.e. C-blue, E-green, F-red, G-yellow, A-cyan, respectively. These color selections depend on the song segment, not the note name of the key, i.e. C's would not always be blue every time they are used in a song segment. A different song segment may likely result in a different color for ‘middle C’ (the lowest note in this example) if it happened to use ‘middle C’. In the example of FIG. 21D each of these keys 122 a-122 d are lit in the same color, i.e. C-red, E-green, F-red, G-red, A-red, respectively. The keys that are not used are not lit.
In contrast, a non-focused keyboard which highlights chords, may display the left group of notes 572 for the odd-numbered measures C-E-G and the right group of notes 574 for the even measures C-F-A, shown in FIG. 21E. FIG. 21F depicts the notes in a scale-oriented non-focused keyboard 578, even though ‘D’ for example, is not used in this song segment. In summary, the focused keyboard instantly shows the player which keys will be played during the song segment and which need not be considered at all, thus showing the player where to focus their attention.
The scoring of the game is described with reference to FIG. 23 and FIG. 24. Each note can be worth up to 1000 points if it is played accurately. Accuracy is judged both in pitch and in time. A slightly wrong note played at the right time is worth approximately as much as a right note played at a slightly wrong time. If a note is neglected entirely, in the current implementation it turns into an open circle ‘O’ 117 c when it becomes too late to play. If “extra” notes are played that cannot be matched well enough with any note in the song, an ‘X’ 117 e appears on or between the lines showing this added note played. The software is programmable as to how close a note must be in time and pitch to be a possible match. At most, each song note can only be matched once. If it seems to have been attempted more than once, the game attempts to match it with the best match and the remaining notes are considered to be“extra” notes and result in a penalty, similar to the penalty for notes missed entirely.
If a note is matched pretty well, it results in an explosion 117 a, as shown in FIG. 23. This note was close to ideal, but perhaps off by a few milliseconds, so it scored 802 out of 1000 in this implementation. If a note is matched, but not particularly well, a tail 117 b of various sizes is drawn on the note, as shown in FIG. 23. In this case, the direction of the tail indicates that the player played a note with a slightly lower pitch and played it slightly late. This note was therefore not as good as the previous one (117 a), but still close enough to get 736 out of a possible 1000 points. In other embodiments scoring scales with a different maximum point are used.
Referring to FIG. 24, in another embodiment the target bar 107 moves and the notes do not. It is also a horizontal, non-perspective mode, though those attributes are not necessarily associated with the target bar moving instead of the notes. In the case shown here, the target bar 107 moves from left to right and a short song segment has just completed. One advantage of this particular mode is that it is easy for the user to look back after the end of the song to see exactly what happened. In variations of this mode, we alternate between two or more sets of lines, similar to going from one line to the next in sheet music. That allows more notes to be displayed at a time than would easily fit on one line and also makes the rhythm clearer to the player. The “alpha” 117 d scoring notation is actually superposition of a missed note scoring notation ‘O’ 117 c followed by an extra note scoring notation ‘X’ 117 e. The criteria for whether a note is a good enough match could be altered by user preferences and also by such factors as the speed and difficulty rating of the particular song.
Game 100 is able to accurately match and score notes in real-time, as fast as they are played. Prior art games may appear to do a similar thing in their scoring, but there is an important difference. While prior art games may allow graduated scoring for notes (or steps) that are slightly off in time, they do nothing that compares to graduated scoring for notes that are different but close in pitch. In the present game, any of the twenty-five keys on the standard two-octave keyboard might be pressed at any time, whether lit or not and partial score and matching will be considered for even unlit keys that are played at nearly the right time as the lighted note. This is doing matching in two dimensions (time and pitch) rather than just in time, i.e., the problem space is geometrically larger.
Since this is a music game, what the player hears when playing is of paramount importance. There is actually some flexibility in what sound might best be offered. Because the audio sounds live and involving and closely matches what the player is playing on the game keyboard, the player feels as if they are part of making the sound.
This can be the case even when the sounds heard do not reflect or include what the player is doing at all, particularly if the player is doing well. A similar situation can happen with real musicians in a group. If one is playing clarinet in a concert band with a lot of clarinets and the nearby other clarinets are playing the same part, it can be hard to discern one's own playing and even if one stops playing, it may not sound that much different. All the parts playing the same notes blend in together and it is difficult to discern the instruments individually. Comparably, if the sound played from the player's keyboard is very similar to the sound on the recording and they are doing well at playing the same part with the recording, they may feel as if they are hearing themselves play, even when only the recording is being heard. This effect is strengthened by the fact that the ear hears sound intensity in a logarithmic way. When two instruments play instead of one, the combined sound is three decibels louder, but that is not heard as a doubling of the sound volume.
There are situations in which the player might really want to blend in well and want the sound of their own “instrument” to closely match the sound qualities of the associated instrument on the recording and want to hear their part only quietly or not at all. For instance, when showing off for friends, one might want to sound just like the player in the recording, be heard distinctively a little, but not be too obvious when wrong or mistimed notes were played. On the other hand, when the player is alone and seriously wanting to do better, they might want to be able to hear their part clearly and distinctly by having it loud compared to the recording and possibly of a somewhat different tone quality. For example, the recording might be using an electric piano sound but the player chooses to hear a different and distinguishable model of electric piano (say Wurlitzer-like instead of Fender Rhodes-like) or even chooses to hear their playing with an acoustic piano or perhaps a marimba sound. This can be very helpful when learning, since the player can relate to both the similarities between their playing and the audio track as well as the differences. Another effect that can be controlled by the computer and used separately or in combination with this is to play the players part and the recorded part in two different places spatially, for instance the audio track in the left stereo channel and the player in the right stereo channel.
In cases where the game is using an audio recording in which it can separate the instrument that corresponds with the player from the other instruments in the recording, there are even more useful possibilities. For instance, the corresponding instrument in the audio track may be dropped altogether, leaving only the player playing that part in the result.
If the player can control the mix, they could also turn the audio recording off altogether, only hearing their own playing. A player might also find it fun, interesting and useful to remix the levels of the instruments and voices in the recording, to the extent the audio recording made it possible to do so. In the current implementation, the player can change the mix between game audio and their own playing at any time. On the screen, they can see the current mix represented by the brightness of the keyboard icon compared to the brightness of the band icon, as shown in FIG. 26.
In one embodiment of game 100, there is an audio track (stereo pair of tracks, actually) that contains what the player is to hear during the game. Most often, this is the original audio of the song, as performed in a well known version by the creating artists without modification. This is one reason why players and bystanders seem to love the sound of the game. In some cases, this audio is slowed down or even slightly speeded up for the benefit of game ease or difficulty. In other cases, this audio may be transposed (pitch changed) so that the player's part on the often two-octave keyboard might correspond properly to the notes. The only sound that is added in this case are a few “lead-in” beats that alert the player that the audio is about to begin. The sound of these metronome-like lead-in beats is generally percussive in nature, though it could be a voice (“one, two, three, four”) like a band leader. Currently the game uses one hi-hat cymbal sound for the downbeat and another for the other beats in the lead-in, though other prototypes used bongo sounds or other instruments.
An alternative to supplying an audio track in the above manner is to supply MIDI-tracks (played through some MIDI-compatible synthesis method) or a combination of the two. One advantage to the audio is that it sounds more natural to the player and avoids the need to supply software and/or hardware to play the described MIDI-tracks. In one embodiment, the above-mentioned audio and the generated lead-in beats is all that the player hears during the actual playing of the song during the game. Between songs, the keyboard does other things, as described elsewhere. When the audio needs to be slowed down excessively, it may sound unrealistic and even grotesque. In these cases when a usable and pleasant slowed down audio track is not available, it may be preferable to substitute with a MIDI equivalent. That does require the presence of MIDI software or hardware within the system or external to it which might not be necessary otherwise, since the simple lead-in beats can easily be done by other means. Another possibility to support playing very slow (typically less than 50%) of the original tempo is to use alternate audio tracks specifically designed to be slower performances of the same songs. These might be recorded by the original artists or by others.
There are other aural things that can be played along with the playing of the song. The existence, strength and mix of these might be determined as a player preference or fixed values might be tested and built into the game. These possibilities include the following:
a) Hearing what the player is actually playing on a similar or dissimilar instrument. A similar instrument blends in better. A dissimilar instrument makes it easier to hear the differences between what the player is playing and the actual part in the song. Either may be a good choice depending on circumstances. In some embodiments, we have played the player's part in a different octave to make it similar, but distinct that way.
b) Hearing the desired part played on a synthesized similar instrument. In this case, audio may be used that deliberately does not include the part of the desired instrument, which might require audio specially produced for the game.
c) Hearing a deliberately somewhat dissonant version of the desired track played if the player's performance has errors which trigger this. This “error” track could be accomplished in either audio or MIDI. After the player has met some criteria and/or after some delay has passed and/or after some new part of the song is reached, the playing of this “error” track would stop and the sound would revert to what it had been.
d) Hearing some rhythm parts made louder or with more emphasis in some or the entire song excerpt to make it easier to follow the beat, though with current music this seldom seems necessary.
e) Reinforcing or otherwise making the featured (often keyboard) part of the song more emphatic and easily heard, possibly by changing its strength or tone color or quieting or silencing less relevant instruments or voices.
There are a lot of possible choices here. They could be dictated by player preferences, considered preset choices, circumstances within the game or some combination of these.
In the embodiment of FIG. 13, while the song is not actually being played, the keyboard reverts to acting like a sampled keyboard, using a “sample” that is similar to the instrument part played in the previous or in the coming song. There are varying degrees of sophistication possible in sampled keyboards and which could be used to make this state of the game better. This embodiment uses the same sound across the keyboard, only changing it in pitch. Some sounds are looped, whereas others are not.
As was mentioned above, in the embodiment of FIG. 13, keyboard 120 is a 25 key C to C keyboard. In other embodiments, one may use a smaller or a larger keyboard. However, twenty-five keys allow very many song parts to be easily played without much modification. It is, of course, possible and somewhat desirable to have even more keys, but that makes the keyboard more expensive, especially if all the keys have all the lighting abilities and this makes the instrument bigger and heavier. Keyboard 120 in FIG. 13 has full-size keys, though there is nothing that would prevent the game from being played with keys of any size. FIG. 25A depicts two examples of non-full- size keys 120 a, 120 b as are common on smaller keyboards and even on some keyboard instruments, such as an accordion. Game 100 is not dependent on the size of the keys and any of these keyboard sized, along with other sizes, may be used. However, playing the game with full-size keys provides more realism and makes it easier to transfer any musical skills learned to other instruments. Keyboard 120 b may also be limiting because it has only fifteen notes. Keyboards with limited number of notes, such as 120 b, or keyboards using ordinary type buttons, are used in cases where a low cost instrument option is desired. The game automatically “transposes” the game music part to the octaves of the attached keyboard. So a keyboard used for a bass part on one song can play the high part on the next song without touching any buttons on the keyboard.
Referring to FIG. 25B, in an arcade version of game 100 with likely limited functionality, it might be more practical to light up fewer keys. In an application like this where durability and resilience to vandalism are important, the keys may even be large lighted buttons on a panel that only depress slightly or even touch sensitive lighted pads possibly flush with the panel surface. Arcade machines generally use ruggedized panels as shown in FIG. 25B. Nonetheless, suitable buttons or touch surfaces are arranged into a keyboard pattern and lit in multi-color style to implement an arcade version of game 100.
Some keyboards have small speakers built-in, as shown n FIG. 25A, and others do not. Almost all have a headphone jack and most have “line out” or “balanced line out” in mono, stereo or even quad to external amplification and recording devices. Some keyboards offer several power options. Any of these options are possible for a keyboard 120 for game 100, though none are mandatory. It is a matter of desirability vs. size and cost.
A minimum of four colors or other distinguishing mechanism would be desired for a reasonable game. If only a small number of colors were used, they could be repeated again in order on higher notes without adding an intolerable amount of confusion. However, the game works significantly better, when each key that is lit has its own color. In some embodiments, between one and nine colors per song excerpt have been used, but there is nothing magical about the number “nine” and more colors could be available and used on some songs.
From an electronic point of view, keyboard 120 (or any other musical instrument) signals the pressing of notes electronically to the game processor. It is highly desirable that it signals their release as well, but one could still have a good game without that feature. Keyboard 120 also accepts electronic signals telling it to make a particular key a particular color.
Other desirable features of game 100 include the following among others:

- Having a small display could be useful as a means of communicating to the user, particularly if the keyboard supports usage away from the game system.
- Having additional controls, such as the pitch-bend and tonal effect controls available on many keyboards could be used as an added feature in some games and in general add to the enjoyment of playing the instrument with or without the game attached.
- Touch sensitivity and pressure sensitivity would be desirable and could be used within the game, but these are not necessary.
- Having an internal sound generation system with speakers and/or an earphone jack would be useful particularly when the instrument is used away from the game.
- If the sound generation system is based on samples (such as PCM) or other means of capturing an instrument sound, this is useful in making the keyboard sound like it is associated with the game even when it is disconnected from the game.
- Appropriate batteries and/or ac power adapter would be necessary for use away from the game. The instrument could be powered that way when used with the game, but it might more likely be powered through the cable attaching it to the game system.
- Wireless capability would add to the flexibility and enjoyment of the game.
- The interface to the computer gaming system might be through a standard compatible with many platforms, such as USB, or some other standard could be used with the game possibly connecting in a different way to the gaming system.

Keyboard 120 may have a small amount of memory that would allow it to store the sounds of instruments recently used in the game and/or instruments in the game specified by the player. This will be useful in cases keyboard 120 is temporarily removed from the game. It may also have other built-in sounds as well. With enough memory and appropriate onboard intelligence, the keyboard 120 may even be able to download and store audio song samples from the game so that the player could play along with them even without the game. In some embodiments, the keyboard 120 incorporates the computer 102 and/or display 106.
Other embodiments of the invention may include one or more of the following. The keys in the focused keyboard 120 may be designated with shapes, numbers, letters, textures or type font size, instead of color. Another possibility is to reposition keys to show their status in the song segment. For instance, the keys not used could be slightly lowered. This may be done with or without another designation system, such as colored keys. The “prompting” display of the keys, as was described above, shows the keys used by colored dots arranged in order of pitch in one dimension and in order of times in the perpendicular or another dimension. These dots move towards a target position 107, as shown in FIG. 14 and FIG. 13. In other configurations the notes may be steady and the target bar 107 may move across them instead. The entire part being played could be represented by one continuous line of these musical notes or several lines could be used in sequence, proceeding to the next (or back to the first) when a line has finished playing. This is particularly useful in cases where the target moves instead of the notes because then it more resembles the idiom of sheet music. The colored lines 111 may be evenly spaced and instead of a model virtual keyboard 106 being pictured at the bottom, two keyboards may be pictured at the left as shown in FIG. 23. In this case the upper keyboard 108 a shows which colors are for which keys and the keys on this keyboard may move up and down as the notes are played. The lower keyboard 108 b in this case echoes what notes are being played on the player's actual keyboard 120 so that it is easily judged whether a correct key is being used. Alternately, the upper keyboard 108 a could just show the colors and the lower keyboard 108 b could show the timing of the notes, i.e. whether they should be up or down at the moment. A thin target bar 107 may be used on which the note should be played when it is centered, as shown in FIG. 23. An arbitrary scoring system may be used to score a game or even no scoring system at all. Scoring systems could give bonuses, for instance, for having played so many notes without error or for achieving, say, exceptional timing accuracy. Mechanism 104 may be implement via other means of transmitting music-related messages, such as XML, JSON or a proprietary format. Game 100 may be played without the actual instrument 120, but rather using the computer keyboard. Game 100 may also be played with a keyboard that doesn't light up by simply watching the lighting on the screen carefully and matching notes to the keyboard by comparing it by shape with the colored keys pictured on the screen. However, the ability to be able to match the notes on the screen to those of the physical color by color at any instant is something that gives particularly starting players confidence and security and adds a lot to the appeal and easy understanding of the game.
Furthermore, in other embodiments, two (or more) song segments for the same song are shown and used simultaneously together. For example, something with both a piano part and synthesizer part, as is common in Elton John recordings. Alternatively, it may be two or more synthesizers, or a piano and organ, or even two or more string, horn or vocal parts. Both parts are shown on the computer display or on two different displays or even on two displays in two or more different locations connected by wiring or some sort of network. Games/learning exercises that can be played in this setup include the following among others:
One player plays two parts on a larger keyboard, one with each hand. A third or even a fourth part could be played with pedals in the style of a classical organist or some synthesizer artists. One player plays two parts with a hand on each of two keyboards. Again, adding pedal parts is possible. The players may use the game to work up to this complexity one step at a time. Two or more players may each play one part on a larger keyboard or instruments. Two or more players may each play one or more parts on separate keyboards or instruments.
Combinations of different instrument may be used, for instance, keyboard and fretboard. The fact that each part is presented in a focused manner to the player involved would help keep the complexity manageable, and this would, of course, very much be like real musicians playing together in ensemble.
In other variations, two or more players are involved with the same song segment or segments. Some variations include the following. The players alternately play the song segment in a competitive manner. They might use the same keyboard (or instrument) or they might each use their own, particularly if they were in different locations. It is also possible to set up a competitive situation if they are not playing the same song segment scenario and/or level, but to be fair one would have to handicap if the difficulties involved were different. Similarly, a handicap system could give a weaker player a chance to compete meaningfully against a stronger player. It would also be possible for a player on a keyboard to compete with one on a fretboard given a handicap system or other means to compensate, even though the ease of playing the part might favor one type of instrument over another.
In other embodiments, instruments other than keyboards are used. They include guitar with lighted fretboard, bass guitar lighted fretboard, soprano sax lighted keys, alto sax lighted keys and EWI lighted keys, among others, as shown in FIG. 27A-FIG. 27E, respectively.
The present invention also relates to a method and a system that associates musical notes to colors and to “colored” keys/buttons on an electronic board. The notes are associated with the melody at specific frequencies (pitches) of a song or a song segment. The notes are then associated with specific colors displayed on a screen and the key/buttons are associated with the specific colors. In one implementation, the system is used in a Melody Rhythm Action Game application. The game is played by hitting the keys/buttons corresponding to a specific color at the same time the specific color is displayed on the screen, thereby playing (matching) the notes that are associated with the melody of the song or song segment. The key/buttons on the electronic board are colored by light of a specific color (either reflected or transmitted), or by paint.
The invention also relates to a method and a system that converts any game controller/electronic board with a few buttons into a musical instrument (“simplified interactive musical instrument”). By definition a musical instrument is simply something that generates notes/frequencies when struck or played at different positions/keys. Thus when specific frequencies are assigned to specific positions/keys/buttons, and sound comes out when the keys/buttons are hit (i.e. played), then by definition a musical instrument is created. In one example, a game controller with 5 keys/buttons is used to play any pattern of melodies or “music”.
Several embodiments of the present invention have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention. Accordingly, other embodiments are within the scope of the following claims.
What is claimed is:

APPENDIX A

The notion of using an ordinary appearing piano (or other) keyboard (or fretboard, etc.) as a song-specific keyboard (or fretboard, etc.) might seem alien at first. However, considering it from another point of view, it can actually be quite simple and be viewed as a way of “tuning” a keyboard to a song, much like how guitarists sometimes retune their instruments for different songs or how a pedal steel guitar, orchestral harp or tympani are all dynamically tuned by means of pedals.
I'm going to talk about one and two octave keyboards here though the principles can be extended to keyboards larger and smaller other variants.
Here's a conventioanl two-octave keyboard. On it, I have written how the keys are ordinarily “tuned”.
Note that as commonly used, the black keys can be called by two different names, depending on whether they are referenced to the white key to the left (in which case they are “sharps” or to the white key to the right (in which case they are “flats”.) The notation G# is pronounced G-sharp. The notation Gb is pronounced G-flat. Nothing new here so far. This is just how keys are usuallly named on a piano keyboard. A keyboard with sharps and flats is sometimes called “chromatic”. Fretboard and wind instrument fingering is similar in some ways and different in others but accomplishes the same purpose.
Not all instruments can produce all of the chromatic scale tones shown above. For instance, most harmonicas, bugles, and tympani cannot without going to extraordinary measures within a song. So it is possible for an instrument to be melodic without being fully chromatic. In fact, to play a melody only really requires being able to play the notes that are used within that particular melody and that will be our point of departure from typical keyboard usage, as we will see below.
On most musical instruments, the same key usually produces the same pitch all the time, with the exceptions for some instruments noted above. For some wind instruments, the wind pressure and mouth position may also affect which pitch is produced.
It is common for instruments to go from lower to higher pitch from left to right and we will generally follow that convention (though some artists such as Joe Zawinul arenoted for setting up their keyboard from right to left, partly to encourage experimentation).

The Song-Specific Keyboard

Let's consider the notes in what I have defined elsewhere as a song segment. It could be a whole melody, a part of a melody or some other significant and recognizable part using notes with specific pitches.
We can consider all of the notes that are used anywhere within a particular song segment. We can order them in pitch from low to high and number the lowest one “1”, the second lowest one “2”, etc. There could be dozens of pitches involved within a song segment, but that would be unusual. If we limit how we define the song segment in time, complexity or other means, we can scope it small enough so that the number of pitches involved is thirteen or less. In fact, very recognizable song segments often only have about seven pitches used and some have as few as three or four. (One well known samba has only two used in the entire chorus - - - that's interesting because it means that we can produce recozniable music sometimes with as few as two keys - - - so using up to thirteen will actually cover a large amount of ground! Even seven will allow us to do many interesting and recognizable things.)
Frequently we'd used colors instead of numbers, such as blue for the lowest note, red for the second lowest, etc. following some arbitrary scheme, but here I will continue using numbers because it will make it clearer to illustrate the principles. But keep in mind that if we used a coloring scheme such as the one just mentioned, the coloring and key position for the lowest note, the second lowest note, etc. would remain the same for each song, so keys could simply be painted particular colors and then used for song after song provided something like a computer “retuned” what pitches the keys actually played for each song or song segment.
I'm going to use conventional piano-style keyboard to show some of the possibilities for laying out the pitches in a logical and playable manner but we could use the same principles on a fretboard, wind instrument or anything at all that could be logically divided into designated “pieces” that could be “played” in some fashion.
Keyboards often run from “C to C”, meaning that if pitches were assigned in a conventional manner to the keyboard, it would start and end with a C. For instance, here is a two octave C to C keyboard, though I'm not going to label the keys because I want to emphasize their pattern.
Note that in many of these examples, the final key on the right is retangular, even though if the same pitch were in the middle of the keyboard, it would have a chunk taken out of an upper corner (or two) to accommodate the positioning of a black key.
There can also be a one octave C to C keyboard.
A one octave F to F keyboard is also a common possibility and looks almost the same except for the positioning of the black keys.
In fact, you can logically see a one octave F to F keyboard as a subset of a two octave C to C keyboard.
To make things easier for the player/beginner musician, we can logically pick out which keys to use. Usually we'll only use about seven, so even on a one octave keyboard there will be unused keys. In fact, even though I will usually show about a dozen keys numbered, for a particular song we would only use as many as were necessary. For instance, for “Mary Had A Little Lamb”, a four-pitch song, we would only use the keys numbered 1, 2, 3 and 4.
That said, here is one simple way to use a two-octave keyboard. In fact, we will only use white keys:
I'll call this a two-octave linear mapping.
Since we generally only use seven or so notes, one might comment that when we played a song-segment it would usually look as if we were always playing down on the left side of the keyboard. Since we have an excess or notes, let's do a mapping starting nearer the center. If we run out of notes, we can always add a few black keys or even go back and use some of the white keys that we skipped:
If we only use about seven notes, it will seem as if we are playing near the center of the keyboard most of the time. Ill call this “Linear in G” since I started on a G with key number 1. I would have started anywhere, but I liked how this was centered and how I started with a group of two black keys between the first thrree notes.
We could also have started one key to the left, for example. When I do that, I call it Linear in F.
With two octave, we can usually get by just using the white keys. If we were trying to save money on the design or construction or coloring issues of such an instrument, the black keys would not necessarily have to move or function at all! They might be merely decorative or help the player find a particular key position by pattern easier. (Of course, we don't neccessariy have to use the two black key, three black key pattern of a piano. The keys could be any fashionable pattern or might even be arranged like the 144-bass buttons of an accoridion or some entirely new arrangement.
For size, economy or other reasons, it might be desirable to scale down to a one octave or possibly even smaller keyboard. In that case, merely using the white keys might still be enough since there are eight of them, but sometimes we might need a few more pitches or perhaps some players might enjoy using black keys as well as white keys. We could use any arbitrary system to assign pitches, but here are a few I have tried:
One octave C to C Chromatic order
This is what I call “Downstairs Upstairs in C” order, first using the white keys, then moving up to use the black keys.
One octave F to F chromatic. (This can also be another way to use the center section of a two-octave C to C keyboard.)
One Octave Downstairs Upstairs In F
I've also done it with typewriter keyboards and even with an Xbox controller. If these are done with colored keycaps or buttons instead of numbers they are much easier for a player to match, but I am showing numbers here to be clear about order.
Note that there are so many keys on a Qwerty keyboard or on a multi-octave music keyboard, that multiple players could play at once on the same keyboard competitively or cooperatively, with the same sound or different sounds. Or multiple players could each have their own keyboard (music, qwerty or even Xbox).
Here's an Xbox controller mapping that I've used to play melodies. (It felt pretty strange and reminded me of playing an ocarina, but I think I could get used to it with a little practive.)
Many other variations are possible. One might ask, why might want to have a two-octave C to C keyboard when one is only using the center F to F section. A number of reasons are possible. For one, the player/musician, espcially “wannabe's”, might like the appearance of the larger keyboard. Another reason is that the game or instruction might later advance to also using the keyboard in a conventional way, in which case having the full two octaves or even more might be a very handy thing. Or perhaps, at a later time, possibly as an upgrade, the full two octave keyboard could be used with added lighting in the way described in the Focused Keyboard, teaching fingering skills for melodies that diectly transfer to most keyboard instruments.

APPENDIX B

How this Melody/Rhythm Game Differs From Existing Music Games
This is a summary of the major differences in our approach to a music game compared to existing technology and games. Note that in this discussion “key” refers to a key on a keyboard (or a fret position on a fretboard or other physical position on other instrument) where the key produces a musical “note” or a particular “pitch” (roughly, frequency). Therefore, in this discussion, the terms “key”, “note” and “pitch” are closely associated and are often interchangeable. (Note that “key” here does not refer to key signature, i.e. C Major or F minor.)

Narrowing the Attention to the Keys Actually Used in a Song or Song Segment

There have been a lot of keyboard games of various types in which result in the player playing a melody or melodic part. Most notably, there has been The Miracle Piano System (known by several other names) and more recently Piano Wizard. Some of these games, such as Piano Wizard, use a color coding that is related to the pitch of the note (i.e. the keys with a C pitch are one color, the keys with a C# pitch another, the keys with D still another, etc.). Note that in a game like this the colors are not related to the particular song played. Other methods, such as Piano Wizard and Casio lighted keyboards, use a single color for all the keys or use no color at all. Note that none of these systems distinguish between the keys actually used for a particular song or song part and those not used within that song or song part.
The games described here, on the other hand, do distinguish between the keys actually used in a song or song segment and those which are not. Accordingly, the designations on the keyboard and/or designations referring to the keyboard change depending on the song or song part played. In some cases, they may change within a song from one part of the song to another. This differentiates this game from others in this category.
Also note that in embodiments of these games that support it (by the hardware used), the colors used are not related to the key pitch. For instance, in one embodiment described, the lowest key is always one particular color (say “Blue”) regardless of whether the lowest pitch is a C, a D, or an E-flat. This also differentiates this game from others.
Melody/Rhythm Games compared to Rhythm Games
There are some games currently available, such as Guitar Hero and Rock Band, in which the player taps buttons or performs some similar action in time with and response to the rhythm of the game in most cases also prompted with a visual display. While in these game the player might tap out a melody of, say, a guitar solo in rhythm with a recording of the guitar solo, there is not a one-to-one correspondence between the pitches of the melody and the buttons on the instrument, i.e. the same button might be used to respond to a F pitch at one point and shortly thereafter respond to a F-sharp, G, or some other pitch, generally without the player being aware that anything had changed. In some cases, the same pitch may be represented by a different button in another part of the song or one button might correspond to more than one pitch being played. In the games described here, the same key or button always corresponds to the same pitch within the given song or song segment. Within a song segment, one key means one particular pitch and that particular pitch is only mapped to that one key.
Melody/Rhythm Games compared to Melody Games
There are some games, in particular on some Casio keyboards, where a key is lit and when the player plays that key the next key used in the song is lit up. Thus, the player is led note by note through a song. (In some cases, more than one light might go on indicating that two keys are to be played simultaneously). The only lights that are on are, at most, the key or keys being played and the key or keys to be played next. The game described here differs from that in two significant ways: all the keys used in a song or segment are designated at once for the length of the song or song segment, not just immediately before they are needed - - - and - - - in many of these other games, the player sets the rhythm by the speed that they play speeding up or slowing down at their will whereas in our game the rhythm of the song is steady and the player must play at the same speed to get the best score—the speed of the song does not vary with the speed of the player.

An Actual Musical Instrument May be Used in Some But Not All Embodiments

The distinction between game controllers and actual musical instruments can get quite fuzzy these days, particularly where computers are involved enhancing and transforming either. Most electronic games these days now involve a computer processor and most electronic musical instruments also involve a computer processor.
Therefore, what the player actually “plays” in the game described can range from a game controller to a computer keyboard to a toy musical keyboard to an actual musical keyboard to a fretboard to the keys of a wind instrument or to the fingerboard of a bowed instrument to moving ones hands in the air to other embodiments. The game could be played using any distinguishable movements that could be associated with pitches. The associated instrument need not even be electronic, since its sound could be detected, for instance, with a microphone and/or the player's movements by a camera or other motion sensing mechanism.
Most of the descriptions here will concentrate on a game being played using the keys of an electronic musical keyboard, but in each case, it could be a fretboard, a toy instrument, a computer keyboard, a game controller, a touchscreen or any of the embodiments mentioned above. In the process of testing, this game has already been played on a music keyboard, a computer keyboard, a game controller and toy musical instruments models. This represents only a few of the possible ways to implement this game.
It is also worth noting as a summary point that in some implementations that keyboard (or fretboard, etc.) is used only as a passive input device to the game, but in other embodiments, the game controls features of the keyboard in a way to correspond with game usage, most notably to illuminate or otherwise highlight (or further highlight) the keys that are actually used within the current song or song segment. Depending on the particular embodiment, all the keys so used might highlight in the same way or, alternatively, several means of highlight (such as several colors) might be used to further distinguish one highlighted key (or button, etc.) from another. Further, in embodiments where highlighted keys are further distinguished from one another, the highlight for each individual key might be unique (such as a unique color) or it may be one of several types of highlighting (for instance, when the hardware is incapable of supplying a unique color for every highlighted key). In these latter cases, the highlighting system may or may not correspond with the pitch “letter” of the key, i.e. the same color of highlighting may or may not be used for all pitch “C's” and if it is necessary to use the same type of color or highlighting more than once, the keys highlighted with one particular color may not necessarily be the same pitch.
Appendix C illustrates several types of highlight schemes possible for the song segment consisting of the first two lines of the popular song “Take Me Out To The Ball Game”.

APPENDIX C

Examples of Several Possible Highlighting Schemes for the Focused Keyboard

The following examples use the melody of the first two lines of the popular song Take Me Out To The Ball Game as a song segment. Though the actual music notation and pitch letter names generally do not appear directly in the game, here they are for reference:
If the keyboard used for the game embodiment is one only capable of highlight keys one way, the highlighting of the keyboard for the playing of this song segment might look like this:
In another embodiment of the game, a keyboard might be used that could light each key any one of three colors. For that embodiment, this song segment might look like this:
Another embodiment of the game might have a rotating pattern of, say, five colors repeated in a chromatic pattern, e.g.

C Blue

C# Purple

D Green

D# Orange

E Red

F Blue

F# Purple

G Green

G# Orange

A Red

A# Blue

B Purple

C Green

On such a keyboard, this song segment would look like:
On still another embodiment, there might be six colors repeated in a chromatic pattern:

C Blue

C# Purple

D Green

D# Orange

E Red

F Yellow

F# Blue

G Purple

G# Green

A Orange

A# Red

B Yellow

C Blue

The song segment might then appear like this:
It might also appear that same way on an envisioned seven color scheme that repeats on each octave yet gives each white key pitch a distinct color:

C Blue

C# Purple

D Green

D# Orange

E Red

F Cyan

F# Blue

G Purple

G# Green

A Orange

A# Red

B Yellow

C Blue

The most important thing to note is that the keys actually used in the song or song segment are highlighted in some way to easily distinguish them from those that are not used. Also note that the highlighting remains present throughout the entire song or segment.
The most important effect of this method of highlighting is to eliminate all of the keys that are not used in the song or song segment from the player's consideration. Since the number of pitches actually used in the song or song segment is generally much smaller than the total number of pitches available, the player is able to focus only on this smaller number of keys and therefore success at playing the melody and rhythm involved is greatly improved and the player experiences more satisfaction.
The keys or buttons need not be arranged in a way corresponding to an actual musical instrument, but if they are, the player will find it easy to transfer the patterns learned in playing the game to performance on an actual conventional musical instrument.

APPENDIX D

Using Keyboards or Buttons that are Permanently Colored and do not in Themselves “Highlight”

For cost considerations or other reasons, it may be desirable to use a keyboard (or buttons, fretboard, etc.) on which keys are differentiated by color or some other means but on which this highlighting is static and does not change.
For example, here is a one-octave “painted” keyboard on which only the “white” keys are able to be used (we would not use this as a “focused” keyboard for any songs or song segments that required the use of any “black” keys if we were trying to use a layout similar to an actual musical instrument).
We could still use this as a focused keyboard if we are able to indicate which keys are actually used by some other distinctive means. Following the “Take Me Out to the Ball Game” example, we might for instance use this keyboard with a display that indicates all the notes actually used in the song or song segment.
For instance, here is the above keyboard associated with a screen or other display for the game:
Note that there are six keys used within this song segment (the first two lines of Take Me Out To The Ball Game) and we can easily see which ones they are by both the fact that the line colors match those keys and by the fact that the lines are arranged in the same order as the keys and physically approach them if the keyboard is aligned below the display. Only six of the eight possible “white” keys are used and none of the possible black keys are used, thereby limited the player's consideration to only 6 of 13 possible pitches in the range of the keyboard. If the keyboard were larger than an octave, this consideration would be even more dramatic, being 6 of 25 for a two-octave keyboard or 6 of 88 for a full sized piano. Even someone with no musical experience generally has no trouble keeping track of only six possibilities, especially when they are already familiar with the melody and rhythm of the song. This level of concentration is similar to conventional video games which might require the player to operate six different buttons or parts of buttons within a short time period.
With some reduction in perceptual ease, this setup could be reduced to even simpler hardware or logical embodiments. For instance, we could eliminate the use of slanted lines, perhaps making it possible to use a simpler display:
Note that the display (or panel) might be attached directly to the keyboard. Also note that since the use of each color on this display is restricted to a unique area, this display could, for example, be achieved by a panel that selectively illuminate lines and dots in white with a multi-color tinted filter placed over it. (The display of the previous example could also be produced that way, though the tinted areas would be trapezoids instead of rectangles.)
In a more primitive form, the only important distinction between notes might be whether they are included in the song or not. This might lead to a very basic display like this:
Note that this form of display still makes it possible for the player to easily tell which keys are included in the song or song segment and which are not.
Also note that the examples used so far in this appendix used white keys only and showed how a keyboard could be used realistically even if the “black keys” did not operate. But the examples could have used instead a painted keyboard with colored black keys of unique colors or repeated colors, for instance this one:
Please note that all of the examples in this appendix operate the keyboard as a “focused keyboard”, rather than a “song specific” keyboard. Recall that on a “focused keyboard” the keys (pitches) that are not used are not highlighted and the configuration of pitches found on the keyboard resembles that of a conventional musical keyboard. On the “song specific” keyboard, pitches that are not used do not appear as keys at all on the keyboard, the remaining keys are adjacent to one another and the arrangement of pitches on the keys changes with each song or song segment and does not resemble the arrangement of pitches on a conventional music keyboard. The “song specific” keyboard is fully described elsewhere in this document.

Claims

1. A melody based game system configured to receive game input by playing one or more song segments comprising:

a controller comprising a limited number of programmable real keys configured to be manipulated by a player for playing one or more song segments and thereby providing game input, and wherein each of said song segments comprises one or more pitches and wherein one or more of the programmable real keys are programmed to generate the one or more pitches comprised within each of the song segments, respectively;

a computing unit communicating with said controller and comprising a music application;

a display connected to said computing unit and configured to display prompts for manipulating the one or more programmed real keys in coordination with playing of the one or more pitches of each of the song segments;

wherein said music application simultaneously causes a song segment to be played and provides prompts indicating a manipulation sequence of the one or more programmed real keys corresponding to the playing of the one or more pitches of the song segment; and

wherein manipulation of the one or more programmed real keys by the player coinciding with the displayed prompts in pitch, intensity, and timing during the playing of the one or more pitches of the song segment causes a positive scoring event.

2. The melody based game system of claim 1, wherein said computing unit comprises one of a computer, a tablet computer, an iPad™, an iPhone™, a Smartphone, a Playstation™, an Xbox™, a Wii™, a PlayStation™, a Nintendo DS™, a game controlling device or a handheld game controlling device.

3. The melody based game system of claim 1, further comprising a database comprising musical recordings and data specific to each musical recording that associate pitches of each musical recording with a specific manipulation sequence of said real keys.

4. The melody based game system of claim 1, wherein said musical recordings comprise one of song segments, song fragments, musical arrangements, instrumental musical pieces, or vocal musical pieces.

5. The melody based game system of claim 1, wherein no manipulation of the one or more programmed real keys by the player coinciding with the displayed prompts or manipulation of the programmed real keys by the player not coinciding in pitch, intensity, and timing with the simultaneously displayed prompts during the playing of the one or more pitches of the song segment causes a negative scoring event.

6. The melody based game system of claim 1, wherein the number of the programmable real keys is in the range between 3 and 11.

7. The melody based game system of claim 1, wherein the number of the programmable real keys is seven.

8. The melody based game system of claim 1, wherein the programmable real keys comprise touch areas defined on a screen of a touch enabled display.

9. The melody based game system of claim 1 wherein said computing unit comprises a tablet computer having a touch enabled display screen and wherein said controller and said programmable real keys comprise touch areas defined in said touch enabled display screen.

10. The melody base game system of claim 1, wherein said one or more programmable real keys that are programmed to generate the one or more pitches comprised within each of the song segments are designated during the playing of each of the song segments with a designation comprising one of light, colors, shapes, numbers, letters, textures, font type, font size, or key relative position.

11. The melody based game system of claim 1, wherein a melody of any length comprising a plurality of song segments is played by sequential reprogramming of the programmable real keys.

12. The melody based game system of claim 1, further comprising a bidirectional communication mechanism between said computing unit and said controller.

13. The melody based game system of claim 1, wherein said prompts comprise lighted prompts that move along paths terminating onto the programmed real keys.

14. A melody based game system configured to receive game input by playing one or more song segments comprising:

a tablet computer comprising a touch enabled display screen and a music application;

wherein said music application simultaneously causes a song segment to be played and provides prompts indicating a manipulation sequence of the one or more programmed real keys corresponding to the playing of the one or more pitches of the song segment;

wherein said controller and said programmable real keys comprise touch areas defined in said touch enabled display screen and wherein said prompts are displayed on said touch enabled display screen; and

wherein manipulation of the one or more programmed real keys by the player coinciding with the prompts in pitch, intensity, and timing during the playing of the one or more pitches of the song segment causes a positive scoring event.

15. A method for a melody based game system configured to receive game input by playing one or more song segments comprising:

providing a controller comprising a limited number of programmable real keys configured to be manipulated by a player for playing one or more song segments and thereby providing game input, and wherein each of said song segments comprises one or more pitches and wherein one or more of the programmable real keys are programmed to generate the one or more pitches comprised within each of the song segments, respectively;

providing a computing unit communicating with said controller and comprising a music application;

providing a display connected to said computing unit and configured to display prompts for manipulating the one or more programmed real keys in coordination with playing of the one or more pitches of each of the song segments;

16. The method of claim 15, wherein said computing unit comprises one of a computer, a tablet computer, an iPad™, an iPhone™, a Smartphone a Playstation™, an Xbox™, a Wii™, a PlayStation™, a Nintendo DS™, a game controlling device or a handheld game controlling device.

17. The method of claim 15, further comprising providing a database comprising musical recordings and data specific to each musical recording that associate pitches of each musical recording with a specific manipulation sequence of said real keys.

18. The method of claim 15, wherein said musical recordings comprise one of song segments, song fragments, musical arrangements, instrumental musical pieces, or vocal musical pieces.

19. The method of claim 15, wherein no manipulation of the one or more programmed real keys by the player coinciding with the displayed prompts or manipulation of the programmed real keys by the player not coinciding in pitch, intensity, and timing with the simultaneously displayed prompts during the playing of the one or more pitches of the song segment causes a negative scoring event.

20. The method of claim 15, wherein the number of the programmable real keys is in the range between 3 and 11.

21. The method of claim 15, wherein the number of the programmable real keys is seven.

22. The method of claim 15, wherein said one or more programmable real keys that are programmed to generate the one or more pitches comprised within each of the song segments are designated during the playing of each of the song segments with a designation comprising one of light, colors, shapes, numbers, letters, textures, font type, font size, or key relative position.

23. The method of claim 15, wherein a melody of any length comprising a plurality of song segments is played by sequential reprogramming of the programmable real keys.

24. The method of claim 15, wherein said computing unit comprises a tablet computer having a touch enabled display screen and wherein said controller and said programmable real keys comprise touch areas defined in said touch enabled display screen.