US20050264857A1 - Binaural horizontal perspective display - Google Patents
Binaural horizontal perspective display Download PDFInfo
- Publication number
- US20050264857A1 US20050264857A1 US11/141,606 US14160605A US2005264857A1 US 20050264857 A1 US20050264857 A1 US 20050264857A1 US 14160605 A US14160605 A US 14160605A US 2005264857 A1 US2005264857 A1 US 2005264857A1
- Authority
- US
- United States
- Prior art keywords
- display
- image
- projection
- horizontal perspective
- sound
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000002156 mixing Methods 0.000 claims abstract description 11
- 238000000034 method Methods 0.000 claims description 59
- 230000033001 locomotion Effects 0.000 claims description 13
- 238000001514 detection method Methods 0.000 claims description 3
- 238000012986 modification Methods 0.000 claims description 3
- 230000004048 modification Effects 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 3
- 230000002093 peripheral effect Effects 0.000 claims 2
- 239000000284 extract Substances 0.000 claims 1
- 238000004088 simulation Methods 0.000 abstract description 26
- 210000001508 eye Anatomy 0.000 description 95
- 210000003128 head Anatomy 0.000 description 45
- 210000005069 ears Anatomy 0.000 description 21
- 230000004044 response Effects 0.000 description 16
- 230000004438 eyesight Effects 0.000 description 12
- 210000004556 brain Anatomy 0.000 description 11
- 230000008447 perception Effects 0.000 description 11
- 230000000694 effects Effects 0.000 description 10
- 239000011521 glass Substances 0.000 description 10
- 230000000007 visual effect Effects 0.000 description 10
- 230000006870 function Effects 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 238000000926 separation method Methods 0.000 description 7
- 230000004886 head movement Effects 0.000 description 6
- 230000004807 localization Effects 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000009877 rendering Methods 0.000 description 6
- 238000002592 echocardiography Methods 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 5
- 230000002452 interceptive effect Effects 0.000 description 5
- 230000008859 change Effects 0.000 description 4
- 238000012937 correction Methods 0.000 description 4
- 210000000613 ear canal Anatomy 0.000 description 4
- 210000000883 ear external Anatomy 0.000 description 4
- 238000009416 shuttering Methods 0.000 description 4
- 230000015572 biosynthetic process Effects 0.000 description 3
- 238000007654 immersion Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000005236 sound signal Effects 0.000 description 3
- 241000282412 Homo Species 0.000 description 2
- 230000001419 dependent effect Effects 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 238000010521 absorption reaction Methods 0.000 description 1
- 230000004308 accommodation Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 208000003464 asthenopia Diseases 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 210000005252 bulbus oculi Anatomy 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000003750 conditioning effect Effects 0.000 description 1
- 235000009508 confectionery Nutrition 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013016 damping Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 238000004091 panning Methods 0.000 description 1
- 230000035484 reaction time Effects 0.000 description 1
- 230000003362 replicative effect Effects 0.000 description 1
- 210000001525 retina Anatomy 0.000 description 1
- 230000002207 retinal effect Effects 0.000 description 1
- 230000004256 retinal image Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000035807 sensation Effects 0.000 description 1
- 210000002832 shoulder Anatomy 0.000 description 1
- 230000007480 spreading Effects 0.000 description 1
- 238000003892 spreading Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 210000003454 tympanic membrane Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
- G02B30/40—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images giving the observer of a single two-dimensional [2D] image a perception of depth
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
- G02B30/50—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images the image being built up from image elements distributed over a 3D volume, e.g. voxels
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B30/00—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images
- G02B30/50—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images the image being built up from image elements distributed over a 3D volume, e.g. voxels
- G02B30/56—Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images the image being built up from image elements distributed over a 3D volume, e.g. voxels by projecting aerial or floating images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/10—Geometric effects
- G06T15/20—Perspective computation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformation in the plane of the image
- G06T3/40—Scaling the whole image or part thereof
- G06T3/4038—Scaling the whole image or part thereof for image mosaicing, i.e. plane images composed of plane sub-images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/20—Image signal generators
- H04N13/275—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals
- H04N13/279—Image signal generators from 3D object models, e.g. computer-generated stereoscopic image signals the virtual viewpoint locations being selected by the viewers or determined by tracking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/356—Image reproducers having separate monoscopic and stereoscopic modes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/361—Reproducing mixed stereoscopic images; Reproducing mixed monoscopic and stereoscopic images, e.g. a stereoscopic image overlay window on a monoscopic image background
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/363—Image reproducers using image projection screens
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/366—Image reproducers using viewer tracking
- H04N13/383—Image reproducers using viewer tracking for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/388—Volumetric displays, i.e. systems where the image is built up from picture elements distributed through a volume
- H04N13/395—Volumetric displays, i.e. systems where the image is built up from picture elements distributed through a volume with depth sampling, i.e. the volume being constructed from a stack or sequence of 2D image planes
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
- H04S7/303—Tracking of listener position or orientation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
- H04R2499/10—General applications
- H04R2499/15—Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
- H04S7/303—Tracking of listener position or orientation
- H04S7/304—For headphones
Definitions
- This invention relates to a three-dimensional display system, and in particular, to a multiple view display system.
- the answer is three dimensional illusions.
- the two dimensional pictures must provide a numbers of cues of the third dimension to the brain to create the illusion of three dimensional images.
- This effect of third dimension cues can be realistically achievable due to the fact that the brain is quite accustomed to it.
- the three dimensional real world is always and already converted into two dimensional (e.g. height and width) projected image at the retina, a concave surface at the back of the eye.
- the brain through experience and perception, generates the depth information to form the three dimension visual image from two types of depth cues: monocular (one eye perception) and binocular (two eye perception).
- binocular depth cues are innate and biological while monocular depth cues are learned and environmental.
- the major binocular depth cues are convergence and retinal disparity.
- the brain measures the amount of convergence of the eyes to provide a rough estimate of the distance since the angle between the line of sight of each eye is larger when an object is closer.
- the disparity of the retinal images due to the separation of the two eyes is used to create the perception of depth.
- the effect is called stereoscopy where each eye receives a slightly different view of a scene, and the brain fuses them together using these differences to determine the ratio of distances between nearby objects.
- Binocular cues are very powerful perception of depth. However, there are also depth cues with only one eye, called monocular depth cues, to create an impression of depth on a flat image.
- the major monocular cues are: overlapping, relative size, linear perspective and light and shadow. When an object is viewed partially covered, this pattern of blocking is used as a cue to determine that the object is farther away. When two objects known to be the same size and one appears smaller than the other, this pattern of relative size is used as a cue to assume that the smaller object is farther away.
- the cue of relative size also provides the basis for the cue of linear perspective where the farther away the lines are from the observer, the closer together they will appear since parallel lines in a perspective image appear to converge towards a single point. The light falling on an object from a certain angle could provide the cue for the form and depth of an object.
- the distribution of light and shadow on a objects is a powerful monocular cue for depth provided by the biologically correct assumption that light comes from above.
- Perspective drawing is most often used to achieve the illusion of three dimension depth and spatial relationships on a flat (two dimension) surface, such as paper or canvas.
- a flat (two dimension) surface such as paper or canvas.
- three dimension objects are depicted on a two dimension plane, but “trick” the eye into appearing to be in three dimension space.
- the first theoretical treatise for constructing perspective, Depictura was published in the early 1400's by the architect, Leone Battista Alberti. Since the introduction of his book, the details behind “general” perspective have been very well documented. However, the fact that there are a number of other types of perspectives is not well known. Some examples are military, cavalier, isometric, and dimetric, as shown at the top of FIG. 1 .
- FIG. 1 Central perspective, also called one-point perspective, is the simplest kind of “genuine” perspective construction, and is often taught in art and drafting classes for beginners.
- FIG. 2 further illustrates central perspective.
- the chess board and chess pieces look like three dimension objects, even though they are drawn on a two dimensional flat piece of paper.
- Central perspective has a central vanishing point, and rectangular objects are placed so their front sides are parallel to the picture plane. The depth of the objects is perpendicular to the picture plane. All parallel receding edges run towards a central vanishing point. The viewer looks towards this vanishing point with a straight view.
- an architect or artist creates a drawing using central perspective they must use a single-eye view. That is, the artist creating the drawing captures the image by looking through only one eye, which is perpendicular to the drawing surface.
- the angle between the viewing surface and the line of vision is preferrably 45° but can be almost any angle, and the viewing surface is perferrably horizontal (wherein the name “horizontal perspective”), but it can be any surface, as long as the line of vision forming a not-perpendicular angle to it.
- Horizontal perspective images offer realistic three dimensional illusion, but are little known primarily due to the narrow viewing location (the viewer's eyepoint has to be coincide precisely with the image projection eyepoint), and the complexity involving in projecting the two dimensional image or the three dimension model into the horizontal perspective image.
- the present invention recognizes that the personal computer is perfectly suitable for horizontal perspective display. It is personal, thus it is designed for the operation of one person, and the computer, with its powerful microprocessor, is well capable of rendering various horizontal perspective images to the viewer.
- the present invention display system discloses a three dimension display system comprising at least a display surface displaying a three dimensional horizontal perspective images.
- the other display surfaces can display two dimensional images, or preferably three dimensional central perpective images.
- the display surfaces can have a curvilinear blending display section to merge the various images.
- the multi-plane display system can comprise various camera eyepoints, one for the horizontal perspective images, one for the central perspective images, and optionally one for the curvilinear blending display surface.
- the multi-plane display surface can further adjust the various images to accommodate the position of the viewer.
- the display can accept manual input such as a computer mouse, trackball, joystick, tablet, etc. to re-position the horizontal perspective images.
- the display can also automatically re-position the images based on an input device automatically providing the viewer's viewpoint location.
- the display is also included three dimensional audio such as binaural simulation to lend realism to the three dimensional display.
- FIG. 1 shows the various perspective drawings.
- FIG. 2 shows a typical central perspective drawing.
- FIG. 3 shows the comparison of central perspective (Image A) and horizontal perspective (Image B).
- FIG. 4 shows the central perspective drawing of three stacking blocks.
- FIG. 5 shows the horizontal perspective drawing of three stacking blocks.
- FIG. 6 shows the method of drawing a horizontal perspective drawing.
- FIG. 7 shows a horizontal perspective display and an viewer input device.
- FIG. 8 shows a horizontal perspective display, a computational device and an viewer input device.
- FIG. 9 shows mapping of the 3-d object onto the horizontal plane.
- FIG. 10 shows the projection of 3-d object by horizontal perspective.
- FIG. 11 shows the simulation time of the horizontal perspective.
- FIG. 12 shows an embodiment of the present invention multi-plane display.
- FIG. 13 shows the horizontal perspective and central perspective projection on the present invention multi-plane display.
- the present invention discloses a multi-plane display system comprising at least two display surfaces, one of which capable of projecting three dimensional illusion based on horizontal perspective projection.
- the present invention multi-plane display system can be used to display three dimensional images and has obvious utility to many industrial applications such as manufacturing design reviews, ergonomic simulation, safety and training, video games, cinematography, scientific 3D viewing, and medical and other data displays.
- horizontal perspective Normally, as in central perspective, the plane of vision, at right angle to the line of sight, is also the projected plane of the picture, and depth cues are used to give the illusion of depth to this flat image.
- the plane of vision remains the same, but the projected image is not on this plane. It is on a plane angled to the plane of vision. Typically, the image would be on the ground level surface. This means the image will be physically in the third dimension relative to the plane of vision.
- horizontal perspective can be called horizontal projection.
- the object In horizontal perspective, the object is to separate the image from the paper, and fuse the image to the three dimension object that projects the horizontal perspective image.
- the horizontal perspective image must be distorted so that the visual image fuses to form the free standing three dimensional figure. It is also essential the image is viewed from the correct eye points, otherwise the three dimensional illusion is lost.
- the horizontal perspective images In contrast to central perspective images which have height and width, and project an illusion of depth, and therefore the objects are usually abruptly projected and the images appear to be in layers, the horizontal perspective images have actual depth and width, and illusion gives them height, and therefore there is usually a graduated shifting so the images appear to be continuous.
- FIG. 3 compares key characteristics that differentiate central perspective and horizontal perspective.
- Image A shows key pertinent characteristics of central perspective
- Image B shows key pertinent characteristics of horizontal perspective.
- Image A the real-life three dimension object (three blocks stacked slightly above each other) was drawn by the artist closing one eye, and viewing along a line of sight perpendicular to the vertical drawing plane.
- the resulting image when viewed vertically, straight on, and through one eye, looks the same as the original image.
- Image B the real-life three dimension object was drawn by the artist closing one eye, and viewing along a line of sight 45° to the horizontal drawing plane.
- the resulting image when viewed horizontally, at 45° and through one eye, looks the same as the original image.
- central perspective showing in Image A and horizontal perspective showing in Image B is the location of the display plane with respect to the projected three dimensional image.
- the display plane can be adjusted up and down, and therefore the projected image can be displayed in the open air above the display plane, i.e. a physical hand can touch (or more likely pass through) the illusion, or it can be displayed under the display plane, i.e. one cannot touch the illusion because the display plane physically blocks the hand.
- This is the nature of horizontal perspective, and as long as the camera eyepoint and the viewer eyepoint is at the same place, the illusion is present.
- the three dimensional illusion is likely to be only inside the display plane, meaning one cannot touch it.
- the central perspective would need elaborate display scheme such as surround image projection and large volume.
- FIGS. 4 and 5 illustrate the visual difference between using central and horizontal perspective.
- FIG. 4 drawn with central perspective, through one open eye. Hold the piece of paper vertically in front of you, as you would a traditional drawing, perpendicular to your eye. You can see that central perspective provides a good representation of three dimension objects on a two dimension surface.
- FIG. 5 drawn using horizontal perspective, by sifting at your desk and placing the paper lying flat (horizontally) on the desk in front of you. Again, view the image through only one eye. This puts your one open eye, called the eye point at approximately a 45° angle to the paper, which is the angle that the artist used to make the drawing. To get your open eye and its line-of-sight to coincide with the artist's, move your eye downward and forward closer to the drawing, about six inches out and down and at a 45° angle. This will result in the ideal viewing experience where the top and middle blocks will appear above the paper in open space.
- both central and horizontal perspective not only define the angle of the line of sight from the eye point; they also define the distance from the eye point to the drawing.
- FIGS. 4 and 5 are drawn with an ideal location and direction for your open eye relative to the drawing surfaces.
- the use of only one eye and the position and direction of that eye relative to the viewing surface are essential to seeing the open space three dimension horizontal perspective illusion.
- FIG. 6 is an architectural-style illustration that demonstrates a method for making simple geometric drawings on paper or canvas utilizing horizontal perspective.
- FIG. 6 is a side view of the same three blocks used in FIG. 5 . It illustrates the actual mechanics of horizontal perspective. Each point that makes up the object is drawn by projecting the point onto the horizontal drawing plane. To illustrate this, FIG. 6 shows a few of the coordinates of the blocks being drawn on the horizontal drawing plane through projection lines. These projection lines start at the eye point (not shown in FIG.
- FIG. 6 one of the three blocks appears below the horizontal drawing plane.
- points located below the drawing surface are also drawn onto the horizontal drawing plane, as seen from the eye point along the line-of-site. Therefore when the final drawing is viewed, objects not only appear above the horizontal drawing plane, but may also appear below it as well-giving the appearance that they are receding into the paper. If you look again at FIG. 5 , you will notice that the bottom box appears to be below, or go into, the paper, while the other two boxes appear above the paper in open space.
- the present invention multi-plane display system promotes horizontal perspective projection viewing by providing the viewer with the means to adjust the displayed images to maximize the illusion viewing experience.
- the horizontal perspective display of the present invention is shown in FIG. 7 , comprising a real time electronic display 100 capable of re-drawing the projected image, together with a viewer's input device 102 to adjust the horizontal perspective image.
- the horizontal perspective display of the present invention can ensure the minimum distortion in rendering the three dimension illusion from the horizontal perspective method.
- the input device can be manually operated where the viewer manually inputs his or her eyepoint location, or change the projection image eyepoint to obtain the optimum three dimensional illusion.
- the input device can also be automatically operated where the display automatically tracks the viewer's eyepoint and adjust the projection image accordingly.
- the multi-plane display system removes the constraint that the viewers keeping their heads in relatively fixed positions, a constraint that create much difficulty in the acceptance of precise eyepoint location such as horizontal perspective or hologram display.
- the horizontal perspective display system shown in FIG. 8 , can further a computation device 110 in addition to the real time electronic display device 100 and projection image input device 112 providing input to the computational device 110 to calculating the projectional images for display to providing a realistic, minimum distortion three dimensional illusion to the viewer by coincide the viewer's eyepoint with the projection image eyepoint.
- the system can further comprise an image enlargement/reduction input device 115 , or an image rotation input device 117 , or an image movement device 119 to allow the viewer to adjust the view of the projection images.
- the input device can be operated manually or automatically.
- the input device can detect the position and orientation of the viewew eyepoint, to compute and to project the image onto the display according to the detection result.
- the input device can be made to detect the position and orientation of the viewer's head along with the orientation of the eyeballs.
- the input device can comprise an infrared detection system to detect the position the viewer's head to allow the viewer freedom of head movement.
- Other embodiments of the input device can be the triangulation method of detecting the viewer eyepoint location, such as a CCD camera providing position data suitable for the head tracking objectives of the invention.
- the input device can be manually operated by the viewer, such as a keyboard, mouse, trackball, joystick, or the like, to indicate the correct display of the horizontal perspective display images.
- the head or eye-tracking system can comprise a base unit and a head-mounted sensor on the head of the viewer.
- the head-mounted sensor produces signals showing the position and orientation of the viewer in response to the viewer's head movement and eye orientation. These signals can be received by the base unit and are used to compute the proper three dimensional projection images.
- the head or eye tracking system can be infrared cameras to capture images of the viewer's eyes. Using the captured images and other techniques of image processing, the position and orientation of the viewer's eyes can be determined, and then provided to the base unit. The head and eye tracking can be done in real time for small enough time interval to provide continous viewer's head and eye tracking.
- the multi-plane display system comprises a number of new computer hardware and software elements and processes, and together with existing components creates a horizontal perspective viewing simulator.
- the computer hardware viewing surface is preferrably situated horizontally, such that the viewer's line of sight is at a 45° angle to the surface. Typically, this means that the viewer is standing or seated vertically, and the viewing surface is horizontal to the ground.
- the viewer can experience hands-on simulations at viewing angles other than 45° (e.g. 55°, 30° etc.), it is the optimal angle for the brain to recognize the maximum amount of spatial information in an open space image. Therefore, for simplicity's sake, we use “45°” throughout this document to mean “an approximate 45 degree angle”.
- any viewing surface could offer similar three dimensional illusion experience.
- the horizontal perspective illusion can appear to be hanging from a ceiling by projecting the horizontal perspective images onto a ceiling surface, or appear to be floating from a wall by projecting the horizontal perspective images onto a vertical wall surface.
- the viewing simulations are generated within a three dimensional graphics view volume, both situated above and below the physical viewing surface.
- the computer-generated x, y, z coordinates of the Angled Camera point form the vertex of an infinite “pyramid”, whose sides pass through the x, y, z coordinates of the Reference/Horizontal Plane.
- FIG. 9 illustrates this infinite pyramid, which begins at the Angled Camera point and extending through the Far Clip Plane.
- the viewing volume is defined by a Comfort Plane, a plabe on top of the viewing volume, and is appropriately named because its location within the pyramid determines the viewer's personal comfort, i.e. how their eyes, head, body, etc. are situated while viewing and interacting with simulations.
- the viewer For the viewer to view open space images on their physical viewing device it must be positioned properly, which usually means the physical Reference Plane is placed horizontally to the ground. Whatever the viewing device's position relative to the ground, the Reference/Horizontal Plane must be at approximately a 45° angle to the viewer's line-of-site for optimum viewing.
- One way the viewer might perform this step is to position their CRT computer monitor on the floor in a stand, so that the Reference/Horizontal Plane is horizontal to the floor.
- This example uses a CRT-type television or computer monitor, but it could be any type of viewing device, display screen, monochromic or color display, luminescent, TFT, phosphorescent, computer projectors and other method of image generation in general, providing a viewing surface at approximately a 45° angle to the viewer's line-of-sight.
- the display needs to know the view's eyepoint to proper display the horizontal perspective images.
- One way to do this is for the viewer to supply the horizontal perspective display with their eye's real-world x, y, z location and line-of-site information relative to the center of the physical Reference/Horizontal Plane. For example, the viewer tells the horizontal perspective display that their physical eye will be located 12 inches up, and 12 inches back, while looking at the center of the Reference/Horizontal Plane.
- the horizontal perspective display maps the computer-generated Angled Camera point to the viewer's eyepoint physical coordinates and line-of-site.
- Another way is for the viewer to manually adjusting an input device such as a mouse, and the horizontal perspective display adjust its image projection eyepoint until the proper eyepoint location is experienced by the viewer.
- Another way way way is using triangulation with infrared device or camera to automatically locate the viewer's eyes locations.
- FIG. 10 is an illustration of the horizontal perspective display that includes all of the new computer-generated and real physical elements as described in the steps above. It also shows that a real-world element and its computer-generated equivalent are mapped 1:1 and together share a common Reference Plane.
- the full implementation of this horizontal perspective display results in a real-time computer-generated three dimensional graphics appearing in open space on and above a viewing device's surface, which is oriented approximately 45° to the viewer's line-of-sight.
- the present invention also allows the viewer to move around the three dimensional display and yet suffer no great distortion since the display can track the viewer eyepoint and re-display the images correspondingly, in contrast to the conventional pior art three dimensional image display where it would be projected and computed as seen from a singular viewing point, and thus any movement by the viewer away from the intended viewing point in space would cause gross distortion.
- the display system can further comprise a computer capable of re-calculate the projected image given the movement of the eyepoint location.
- the horizontal perspective images can be very complex, tedious to create, or created in ways that are not natural for artists or cameras, and therefore require the use of a computer system for the tasks.
- To display a three-dimensional image of an object with complex surfaces or to create an animation sequences would demand a lot of computational power and time, and therefore it is a task well suited to the computer.
- Three dimensional capable electronics and computing hardware devices and real-time computer-generated three dimensional computer graphics have advanced significantly recently with marked innovations in visual, audio and tactile systems, and have producing excellent hardware and software products to generate realism and more natural computer-human interfaces.
- the multi-plane display system of the present invention are not only in demand for entertainment media such as televisions, movies, and video games but are also needed from various fields such as education (displaying three-dimensional structures), technological training (displaying three-dimensional equipment).
- entertainment media such as televisions, movies, and video games
- various fields such as education (displaying three-dimensional structures), technological training (displaying three-dimensional equipment).
- three-dimensional image displays which can be viewed from various angles to enable observation of real objects using object-like images.
- the horizontal perspective display system is also capable of substitute a computer-generated reality for the viewer observation.
- the systems may include audio, visual, motion and inputs from the user in order to create a complete experience of three dimensional illusion.
- the input for the horizontal perspective system can be two dimensional image, several images combined to form one single three dimensional image, or three dimensional model.
- the three dimensional image or model conveys much more information than that a two dimensional image and by changing viewing angle, the viewer will get the impression of seeing the same object from different perspectives continuously.
- the multi-plane display system can further provide multiple views or “Multi-View” capability.
- Multi-View provides the viewer with multiple and/or separate left-and right-eye views of the same simulation.
- Multi-View capability is a significant visual and interactive improvement over the single eye view.
- Multi-View mode both the left eye and right eye images are fused by the viewer's brain into a single, three-dimensional illusion.
- the problem of the discrepancy between accommodation and convergence of eyes, inherent in stereoscopic images, leading to the viewer's eye fatigue with large discrepancy, can be reduced with the horizontal perspective display, especially for motion images, since the position of the viewer's gaze point changes when the display scene changes.
- Multi-View devices that can be used in the present invention include methods with glasses such as anaglyph method, special polarized glasses or shutter glasses, methods without using glasses such as a parallax stereogram, a lenticular method, and mirror method (concave and convex lens).
- a display image for the right eye and a display image for the left eye are respectively superimpose-displayed in two colors, e.g., red and blue, and observation images for the right and left eyes are separated using color filters, thus allowing a viewer to recognize a stereoscopic image.
- the images are displayed using horizontal perspective technique with the viewer looking down at an angle.
- the eyepoint of the projected images has to be coincide with the eyepoint of the viewer, and therefore the viewer input device is essential in allowing the viewer to observe the three dimensional horizontal perspective illusion. From the early days of the anaglyph method, there are much improvements such as the spectrum of the red/blue glasses and display to generate much more realizm and comfort to the viewers.
- the left eye image and the right eye image are separated by the use of mutually extinguishing polarizing filters such as orthogonally linear polarizer, circular polarizer, elliptical polarizer.
- the images are normally projected onto screens with polarizing filters and the viewer is then provided with corresponding polarized glasses.
- the left and right eye images appear on the screen at the same time, but only the left eye polarized light is transmitted through the left eye lens of the eyeglasses and only the right eye polarized light is transmitted through the right eye lens.
- stereocopic display is the image sequential system.
- the images are displayed sequentially between left eye and right eye images rather than superimposing them upon one another, and the viewer's lenses are synchronized with the screen display to allow the left eye to see only when the left image is displayed, and the right eye to see only when the right image is displayed.
- the shuttering of the glasses can be achieved by mechanical shuttering or with liquid crystal electronic shuttering.
- display images for the right and left eyes are alternately displayed on a CRT in a time sharing manner, and observation images for the right and left eyes are separated using time sharing shutter glasses which are opened/closed in a time sharing manner in synchronism with the display images, thus allowing an observer to recognize a stereoscopic image.
- optical method Other way to display stereoscopic images is by optical method.
- display images for the right and left eyes which are separately displayed on a viewer using optical means such as prisms, mirror, lens, and the like, are superimpose-displayed as observation images in front of an observer, thus allowing the observer to recognize a stereoscopic image.
- Large convex or concave lenses can also be used where two image projectors, projecting left eye and right eye images, are providing focus to the viewer's left and right eye respectively.
- a variation of the optical method is the lenticular method where the images form on cylindrical lens elements or two dimensional array of lens elements.
- FIG. 11 is a horizontal perspective display focusing on how the computer-generated person's two eye views are projected onto the Horizontal Plane and then displayed on a stereoscopic 3D capable viewing device.
- FIG. 11 represents one complete display time period. During this display time period, the horizontal perspective display needs to generate two different eye views, because in this example the stereoscopic 3D viewing device requires a separate left- and right-eye view.
- the illustration in the upper left of FIG. 11 shows the Angled Camera point for the right eye after the first (right) eye-view to be generated.
- the horizontal perspective display starts the process of rendering the computer-generated person's second eye (left-eye) view.
- the illustration in the lower left of FIG. 11 shows the Angled Camera point for the left eye after the completion of this time.
- the horizontal perspective display makes an adjustment to the Angled Camera point. This is illustrated in FIG. 11 by the left eye's x coordinate being incremented by two inches. This difference between the right eye's x value and the left eye's x+2′′ is what provides the two-inch separation between the eyes, which is required for stereoscopic 3D viewing.
- the rendering continues by displaying the second (left-eye) view.
- the horizontal perspective display continues to display the left- and right-eye images, as described above, until it needs to move to the next display time period.
- An example of when this may occur is if the bear cub moves his paw or any part of his body. Then a new and second Simulated Image would be required to show the bear cub in its new position.
- This new Simulated Image of the bear cub in a slightly different location, gets rendered during a new display time period. This process of generating multiple views via the nonstop incrementing of display time continues as long as the horizontal perspective display is generating real-time simulations in stereoscopic 3D.
- the display rate is the number of images per second that the display uses to completely generate and display one image. This is similar to a movie projector where 24 times a second it displays an image. Therefore, 1/24 of a second is required for one image to be displayed by the projector. But the display time could be a variable, meaning that depending on the complexity of the view volumes it could take 1/12 or 1 ⁇ 2 a second for the computer to complete just one display image. Since the display was generating a separate left and right eye view of the same image, the total display time is twice the display time for one eye image.
- FIG. 12 illustrates an example of the present invention Multi-Plane display in which the Multi-Plane display is a computer monitor that is approximately “L” shaped when open.
- the end-user views the L-shaped computer monitor from its concave side and at approximately a 45° angle to the bottom of the “L,” as shown in FIG. 12 .
- From the end-user's point of view the entire L-shaped computer monitor appears as one single and seamless viewing surface.
- the edge is the two display segments is preferably smoothly joined and can also having a curvilinear projection to connect the two displays of horizontal perspective and central perspective.
- the Multi-Plane display can be made with one or more physical viewing surfaces.
- the vertical leg of the “L” can be one physical viewing surface, such as flat panel display
- the horizontal leg of the “L” can be a separate flat panel display.
- the edge of the two display segments can be a non-display segment and therefore the two viewing surface are not continuous.
- Each leg of a Multi-Plane display is called a viewing plane and as you can see in the upper left of FIG. 25 there is a vertical viewing plane and a horizontal viewing plane where a central perspective image is generated on the vertical plane and a horizontal perspective image is generated on the horizontal plane, and then blend the two images where the planes meet, as illustrated in the lower right of FIG. 12 .
- FIG. 12 also illustrates that a Multi-Plane display is capable of generating multiple views. Meaning that it can display single-view images, i.e. a one-eye perspective like the simulation in the upper left, and/or multi-view images, i.e. separate right and left eye views like the simulation in the lower right. And when the L-shaped computer monitor is not being used by the end-user it can be closed and look like the simulation in the lower left.
- FIG. 13 is a simplified illustration of the present invention Multi-Plane display.
- the upper right of FIG. 13 is an example of a single-view image of a bear cub that is displayed on an L-shaped computer monitor.
- Normally a single-view or one eye image would be generated with only one camera point, but as you can see there are at least two camera points for the Multi-Plabe display even though this is a single-view example.
- One camera point is for the horizontal perspective image, which is displayed on the horizontal surface, and the other camera point is for the central perspective image, which is displayed on the vertical surface.
- the vertical viewing plane of the L-shaped monitor is the display surface for the central perspective images, and thus there is a need to define another common reference plane for this surface.
- the common reference plane is the plane where the images are display, and the computer need to keep track of this plane for the synchronization of the locations of the displayed images and the real physical locations.
- the multi-plane display system can further include a curvilinear connection display section to blend the horizontal perspective and the central perspective images together at the location of the seam in the “L,” as shown at the bottom of FIG. 13 .
- the multi-plane display system can continuously update and display what appears to be a single L-shaped image on the L-shaped Multi-Plane device.
- the multi-plane display system can comprise multiple display surfaces together with multiple curvilinear blending sections as shown in FIG. 14 .
- the multiple display surfaces can be a flat wall, multiple adjacent flat walls, a dome, and a curved wraparound panel.
- the present invention multi-plane display system thus can simultaneously projecting a plurality of three dimensional images onto multiple display surfaces, one of which is a horizontal perspective image. Further, it can be a stereoscopic multiple display system allowing viewers to use their stereoscopic vision for three dimensional image presentation.
- the multi-plane display system comprises at least two display surfaces
- various requirements need to be addressed to ensure high fidelity in the three dimensional image projection.
- the display requirements are typically geometric accuracy, to ensure that objects and features of the image to be correctly positioned, edge match accuracy, to ensure continuity between display surfaces, no blending variation, to ensure no variation in luminance in the blending section of various display surfaces, and field of view, to ensure a continuous image from the eyepoint of the viewer.
- the blending section of the multi-plane display system is preferably a curve surface
- some distortion correction could be applied in order for the image projected onto the blending section surface to appear correct to the viewer.
- There are various solutions for providing distortion correction to a display system such as using a test pattern image, designing the image projection system for the specific curved blending display section, using special video hardware, utilizing a piecewise-linear approximation for the curved blending section.
- Still another distortion correction solution for the curve surface projection is to automatically computes image distortion correction for any given position of the viewer eyepoint and the projector.
- the multi-plane display system comprises more than one display surface, care should be taken to minimize the seams and gaps between the edges of the respective displays.
- the overlapped image is calculated by an image processor to ensure that the projected pixels in the overlapped areas are adjusted to form the proper displayed images.
- Other solutions are to control the degree of intensity reduction in the overlapping to create a smooth transition from the image of one display surface to the next.
- binaural or three dimensional audio simulation is also included.
- the present invention also provide the means to adjust the binaural or 3D audio to ensure proper sound simulation.
- Hearing can provide the direction of the sound sources but with poorer resolution than vision, the identity and content of a sound source such as speech or music, and the nature of the environment via echoes, reverberation such as a normal room or an open field.
- the brain can interprete properly the various kinds of sound arriving at the head such as direct sounds, diffractive sounds around the head and by interaction with the outer ears and shoulder, different sound amplitudes and different arrival time of the sounds. These acoustic modifications are called ‘sound cues’ and serve to provide us the directional acoustis information of the sounds.
- the sound cues are related to timing, volume, frequency and reflection.
- timing cues the ears recognize the time the sound arrives and assume that the sound comes from the closest source. Further, with two ears separated about 8 inches apart, the delay of the sound reaching one ear with respect to the other ear can give a cue about the location of the sound source. The timing cue is stronger than the level cue in the sense that the listener locates the sound based on the first wave that reaches the ear, regardless of the loudness of any later arriving waves.
- volume (or level) cues the ears recognize the volume (or loudness) of the sound and assume that the sound coming from the loudest direction.
- the amplitude difference between the ears is a strong cue for the localization of the sound source.
- frequency (or equalization) cues the ears recognize the frequency balance of the sound as it arrives in each ear since frontal sounds are directed into the eardrums, while rear sounds bounce off the external ear and thus having a high frequency roll off.
- reflection cue the sound bounces off various surfaces and are either dispersed or absorbed in varying degrees before reaching the ears multiple times. This reflections off the walls of the room and the foreknowledge of the difference between the way various floor coverings sound also contribute to localization.
- the body especially the head, can move relative to the sound source to help in locate the sound.
- ITDs interaural time differences
- ILDs interaural level differences
- HRTFs head-related transfer functions
- the HRTFs are a collection of spatial cues for a particular listener, including ITDs, ILDs and the reflections, diffractions and damping caused by the listener's body, head, outer ears and shoulder.
- the external ear, or pinna has a significant contribution to the HRTFs.
- Higher frequency sounds are filtered by the pinna to provide the brain a way as to perceive the lateral position, or azimuth, and elevation of the sound source since the response of the pinna filter is highly dependent on the overall direction of the sound source.
- the head can account for a reduced amplitude of various frequencies of the sounds since the sound has to go through or around the head in order to reach the ear.
- HRTF is the frequency response of the sound waves as received by the ears.
- the sounds can be delivered to a listener's ears via headphones or loudspeakers to create a virtual sound image in three dimensions.
- the sound transformation to the ear canal i.e. HRTF frequency response
- HRTF frequency response can be measured accurately by using small microphones in the ear canals.
- the measured signal is then processed by a computer to derive the HRTF frequency responses for the left and right ears corresponding to the sound source location.
- a 3D audio system works by using the measured HRTFs as the audio filters or equalizers.
- the HRTFs filters When a sound signal is processed by the HRTFs filters, the sound localization cues are reproduced, and the listener should perceive the sound at the location specified by the HRTFs.
- This method of binaural synthesis works extremely well when the listener's own HRTFs are used to synthesize the localization cues.
- measuring HRTFs is a complicated procedure, so 3D audio systems typically use a single set of HRTFs previously measured from a particular human or manikin subject.
- the HRTF sometimes needs to be changed to accurately respond to a perticular listener.
- the tuning of a HRTF function can be accomplished by providing various sound source locations and environments and asking the listener to identify.
- a 3D audio system should provide the ability for the listener to define a three-dimensional space, to position multiple sound sources and that listener in that 3D space, and to do it all in real-time, or interactively.
- Beside 3D audio system other technologies such stereo extension and surround sound could offer some aspects of 3D positioning or interactivity.
- Extended stereo processes an existing stereo (two channel) soundtrack to add spaciousness and to make it appear to originate from outside the left/right speaker locations through fairly straight-forward methods.
- Some of the characteristics of the extended stereo technology include the size of the listening area (called sweet spot), the amount of spreading of stereo images, the amount of tonal changes, the amount of lost stereo panning information, and the ability to achieve effect on headphones as well as speakers.
- the surround sound create a larger-than-stereo sound stage with a surround sound 5-speaker setup.
- virtual surround sound systems use 3D audio technology to create the illusion of five speakers emanating from a regular set of stereo speakers, therefore enabling a surround sound listening experience without the need for a five speaker setup.
- the characteristics of the surround sound technology include the presentation accuracy, the clarity of spatial imaging, and the size of the listening area
- audio technology needs to create a life-like listening experience by replicating the 3D audio cues that the ears hear in the real world for allowing non-interactive and interactive listening and positioning of sounds anywhere in the three-dimensional space surrounding a listener.
- the head tracker function is also very important to provide perceptual room constancy to the listener. In other words, when the listener move their heads around, the signals would change so that the perceived auditory world maintain its spatial position.
- the simulation system needs to know the head position in order to be able to control the binaural impulse responses adequately. Head position sensors have therefore to be provided.
- the impression of being immersed is of particular relevance for applications in the context of virtual reality.
- a replica of a sound field can be produced by putting an infinite number of microphones everywhere. After being stored on a recorder with an infinite number of channels, this recording can then be played back through an infinite number of point-source loudspeakers, each placed exactly as its corresponding microphone was placed.
- Headphone reproduction is thus differed from loudspeaker reproduction since headphone microphones should be spaced about seven inches apart for a normal ear separation, and loudspeaker microphones separation should be about seven feet apart. Further loudspeakers suffer from crosstalk and therefore some signal conditioning such as crosstalk cancellation will be needed for 3D loudspeaker setup.
- Loudspeaker 3D audio systems are extremely effective in desktop computing environments. This is because there is usually only a single listener (the computer user) who is almost always centered between the speakers and facing forward towards the monitor. Thus, the primary user gets the full 3D effect because the crosstalk is properly cancelled. In typical 3D audio applications, like video gaming, friends may gather around to watch. In this case, the best 3D audio effects are heard by others when they are also centered with respect to the loudspeakers. Off-center listeners may not get the full effect, but they still hear a high quality stereo program with some spatial enhancements.
- the speakers are typically arranged surrounding the listener in about the same horizontal plane, but could be arranged to completely surround the listener, from the ceiling to the floor to the surrounding walls.
- the speakers can also be put on the ceiling, on the floor, arranged in an overhead dome configuration, or arranged in a vertical wall configuration.
- beam transmitted speakers can be used instead of headphone. Beam transmitted speaker offers the freedom of movement for the listener and without the crosstalk between speakers since beam transmitted speaker provide a tight beam of sound.
- crosstalk cancellers are based on a highly simplified model of crosstalk, for example modeling crosstalk as a simple delay and attenuation process, or a delay and a lowpass filter.
- Other crosstalk cancellers have been based on a spherical head model. As with binaural synthesis, crosstalk cancellation performance is ultimately limited by the variation in the size and shape of human heads.
- 3D audio simulation can be accomplished by the following steps:
- the simulation of an acoustic environment involves one or more of the following functions:
- Binaural simulation is generally carried out using the sound source material free from any unwanted echoes or noise.
- the sound source material can then be replayed to a subject, using the appropriate HRTF filters, to create the illusion that the source audio is originating from a particular direction.
- the HRTF filtering is achieved by simply convolving the audio signal with the pair of HRTF responses (one HRTF filter for each channel of the headphone).
- Improved playback through headphones can be achieved through the use of head tracking. This technique makes use of continuous measurements of the orientation of a subject's head, and adapts the audio signals being fed to the headphones appropriately.
- Binaural signal should allow a subject to easily discriminate between left and right sound source locations easily, but the ability to discriminate between front and back, and high and low sound sources is generally only possible if head movement is permitted. Whilst multiple speaker playback methods solve this problem to a large degree, there are still many applications where headphone playback is preferable, and head tracking can then be used as a valuable tool for improving the quality of the 3-D playback.
- the simplest form of head tracking binaural system is one which simply simulates anechoic HRTFs, and changes the HRTF functions rapidly in response to the subjects head movements.
- This HRTF switching can be achieved through a lookup table, with interpolation used to resolve angles that are not represented in the HRTF table.
- An important feature of any head tracking playback system is the delay from the subject head movement to the change in the audio response at the headphones. If this delay is excessive, the subject can experience a form of virtual motion sickness and general disorientation.
- head motion to resolve ambiguities: a faint, low sound could be either in front or back of us, so we quickly and sub-consciously turn our head a small fraction to the left, and we know if the sound is now off to the right, it is in the front, otherwise it is in the back.
- One of the reasons why interactive audio is more realistic than pre-recorded audio (soundtracks) is the fact that the listeners head motion can be properly simulated in an interactive system (using inputs from a joystick, mouse, or head-tracking system).
- the HRTF function are performed using digital signal processing (DSP) hardware for real time performance.
- DSP digital signal processing
- Typical feature of DSP are that the direct sound must be processed to give the correct amplitude and perceived direction, the early echoes must arrive at the listener with appropriate time, amplitude and frequency response to give the perception of the size of the spaces (as well as the acoustic nature of the room surfaces), and the late reverberation must be natural and correctly distributed in 3-D around the listener.
- the relative amplitude of the direct sound compared to the remainder of the room response helps to provide the sensation of distance.
- 3D audio simulation can provides a binaural gain so that the exact same audio content is more audible and intelligible in the binaural case, because the brain can localize and therefore “single out” the binaural signal, while the non-binaural signal gets washed into the noise. Further the listener would still be able to tune into and understand individual conversations, because they are still spatially separated, and “amplified by” binaural gain, an effect called the cocktail party effct. Binaural simulation also can provide faster reaction time because such a signal mirrors the ones received in the real world.
- binaural signals can convey positional information: a binaural radar warning sound can warn a user about a specific object that is approaching (with a sound that is unique to that object), and naturally indicate where that object is coming from. Also listening to binaural simulation can be less fatigue since we are used to hearing sounds that originate outside of their heads, as is the case with binaural signals. Mono or stereo signals appear to come from inside a listener's head when using headphones, and produce more strain than a natural sounding, binaural signal. An lastly, 3D binaural simulation can provide an increased perception and immersion in higher quality 3D environment when visuals are shown in synch with binaural sound.
Abstract
The present invention display system discloses a three dimension display system comprising a three dimensional horizontal perspective display and a 3-D audio system such as binaural simulation to lend realism to the three dimensional display. The three dimensional display system can futher comprise a second display, together with a curvilinear blending display section to merge the various images. The multi-plane display surface can accommodate the viewer by adjusting the various images and 3-D sound according to the viewer's eyepoint and earpoint locations.
Description
- This application claims priority from U.S. provisional applications Ser. No. 60/576,187 filed Jun. 1, 2004, entitled “Multi plane horizontal perspective display”; Ser. No. 60/576,189 filed Jun. 1, 2004, entitled “Multi plane horizontal perspective hand on simulator”; Ser. No. 60/576,182 filed Jun. 1, 2004, entitled “Binaural horizontal perspective display”; and Ser. No. 60/576,181 filed Jun. 1, 2004, entitled “Binaural horizontal perspective hand on simulator” which are incorporated herein by reference.
- This application is related to co-pending application Ser. No. 11/098,681 filed Apr. 4, 2005, entitled “Horizontal projection display”; Ser. No. 11/098,685 filed Apr. 4, 2005, entitled “Horizontal projection display”, Ser. No. 11/098,667 filed Apr. 4, 2005, entitled “Horizontal projection hands-on simulator”; Ser. No. 11/098,682 filed Apr. 4, 2005, entitled “Horizontal projection hands-on simulator”; “Multi plane horizontal perspective display” filed May 27, 2005; “Multi plane horizontal perspective hand on simulator” filed May 27, 2005; “Binaural horizontal perspective display” filed May 27, 2005; and “Binaural horizontal perspective hand on simulator” filed May 27, 2005.
- This invention relates to a three-dimensional display system, and in particular, to a multiple view display system.
- Ever since humans began to communicate through pictures, they faced a dilemma of how to accurately represent the three-dimensional world they lived in. Sculpture was used to successfully depict three-dimensional objects, but was not adequate to communicate spatial relationships between objects and within environments. To do this, early humans attempted to “flatten” what they saw around them onto two-dimensional, vertical planes (e.g. paintings, drawings, tapestries, etc.). Scenes where a person stood upright, surrounded by trees, were rendered relatively successfully on a vertical plane. But how could they represent a landscape, where the ground extended out horizontally from where the artist was standing, as far as the eye could see?
- The answer is three dimensional illusions. The two dimensional pictures must provide a numbers of cues of the third dimension to the brain to create the illusion of three dimensional images. This effect of third dimension cues can be realistically achievable due to the fact that the brain is quite accustomed to it. The three dimensional real world is always and already converted into two dimensional (e.g. height and width) projected image at the retina, a concave surface at the back of the eye. And from this two dimensional image, the brain, through experience and perception, generates the depth information to form the three dimension visual image from two types of depth cues: monocular (one eye perception) and binocular (two eye perception). In general, binocular depth cues are innate and biological while monocular depth cues are learned and environmental.
- The major binocular depth cues are convergence and retinal disparity. The brain measures the amount of convergence of the eyes to provide a rough estimate of the distance since the angle between the line of sight of each eye is larger when an object is closer. The disparity of the retinal images due to the separation of the two eyes is used to create the perception of depth. The effect is called stereoscopy where each eye receives a slightly different view of a scene, and the brain fuses them together using these differences to determine the ratio of distances between nearby objects.
- Binocular cues are very powerful perception of depth. However, there are also depth cues with only one eye, called monocular depth cues, to create an impression of depth on a flat image. The major monocular cues are: overlapping, relative size, linear perspective and light and shadow. When an object is viewed partially covered, this pattern of blocking is used as a cue to determine that the object is farther away. When two objects known to be the same size and one appears smaller than the other, this pattern of relative size is used as a cue to assume that the smaller object is farther away. The cue of relative size also provides the basis for the cue of linear perspective where the farther away the lines are from the observer, the closer together they will appear since parallel lines in a perspective image appear to converge towards a single point. The light falling on an object from a certain angle could provide the cue for the form and depth of an object. The distribution of light and shadow on a objects is a powerful monocular cue for depth provided by the biologically correct assumption that light comes from above.
- Perspective drawing, together with relative size, is most often used to achieve the illusion of three dimension depth and spatial relationships on a flat (two dimension) surface, such as paper or canvas. Through perspective, three dimension objects are depicted on a two dimension plane, but “trick” the eye into appearing to be in three dimension space. The first theoretical treatise for constructing perspective, Depictura, was published in the early 1400's by the architect, Leone Battista Alberti. Since the introduction of his book, the details behind “general” perspective have been very well documented. However, the fact that there are a number of other types of perspectives is not well known. Some examples are military, cavalier, isometric, and dimetric, as shown at the top of
FIG. 1 . - Of special interest is the most common type of perspective, called central perspective, shown at the bottom left of
FIG. 1 . Central perspective, also called one-point perspective, is the simplest kind of “genuine” perspective construction, and is often taught in art and drafting classes for beginners.FIG. 2 further illustrates central perspective. Using central perspective, the chess board and chess pieces look like three dimension objects, even though they are drawn on a two dimensional flat piece of paper. Central perspective has a central vanishing point, and rectangular objects are placed so their front sides are parallel to the picture plane. The depth of the objects is perpendicular to the picture plane. All parallel receding edges run towards a central vanishing point. The viewer looks towards this vanishing point with a straight view. When an architect or artist creates a drawing using central perspective, they must use a single-eye view. That is, the artist creating the drawing captures the image by looking through only one eye, which is perpendicular to the drawing surface. - The vast majority of images, including central perspective images, are displayed, viewed and captured in a plane perpendicular to the line of vision. Viewing the images at angle different from 90° would result in image distortion, meaning a square would be seen as a rectangle when the viewing surface is not perpendicular to the line of vision. However, there is a little known class of images that we called it “horizontal perspective” where the image appears distorted when viewing head on, but displaying a three dimensional illusion when viewing from the correct viewing position. In horizontal perspective, the angle between the viewing surface and the line of vision is preferrably 45° but can be almost any angle, and the viewing surface is perferrably horizontal (wherein the name “horizontal perspective”), but it can be any surface, as long as the line of vision forming a not-perpendicular angle to it.
- Horizontal perspective images offer realistic three dimensional illusion, but are little known primarily due to the narrow viewing location (the viewer's eyepoint has to be coincide precisely with the image projection eyepoint), and the complexity involving in projecting the two dimensional image or the three dimension model into the horizontal perspective image.
- The generation of horizontal perspective images require considerably more expertise to create than conventional perpendicular images. The conventional perpendicular images can be produced directly from the viewer or camera point. One need simply open one's eyes or point the camera in any direction to obtain the images. Further, with much experience in viewing three dimensional depth cues from perpendicular images, viewers can tolerate significant amount of distortion generated by the deviations from the camera point. In contrast, the creation of a horizontal perspective image does require much manipulation. Conventional camera, by projecting the image into the plane perpendicular to the line of sight, would not produce a horizontal perspective image. Making a horizontal drawing requires much effort and very time consuming. Further, since human has limited experience with horizontal perspective images, the viewer's eye must be positioned precisely where the projection eyepoint point is to avoid image distortion. And therefore horizontal perspective, with its difficulties, has received little attention.
- For realistic three dimensional display, binaural or three dimensional audio simulation is also needed.
- The present invention recognizes that the personal computer is perfectly suitable for horizontal perspective display. It is personal, thus it is designed for the operation of one person, and the computer, with its powerful microprocessor, is well capable of rendering various horizontal perspective images to the viewer.
- Thus the present invention display system discloses a three dimension display system comprising at least a display surface displaying a three dimensional horizontal perspective images. The other display surfaces can display two dimensional images, or preferably three dimensional central perpective images. Further, the display surfaces can have a curvilinear blending display section to merge the various images. The multi-plane display system can comprise various camera eyepoints, one for the horizontal perspective images, one for the central perspective images, and optionally one for the curvilinear blending display surface. The multi-plane display surface can further adjust the various images to accommodate the position of the viewer. By changing the displayed images to keep the camera eyepoints of the horizontal perspective and central perspective images in the same position as the viewer's eye point, the viewer's eye is always positioned at the proper viewing position to perceive the three dimensional illusion, thus minimizing viewer's discomfort and distortion. The display can accept manual input such as a computer mouse, trackball, joystick, tablet, etc. to re-position the horizontal perspective images. The display can also automatically re-position the images based on an input device automatically providing the viewer's viewpoint location.
- Further, the display is also included three dimensional audio such as binaural simulation to lend realism to the three dimensional display.
-
FIG. 1 shows the various perspective drawings. -
FIG. 2 shows a typical central perspective drawing. -
FIG. 3 shows the comparison of central perspective (Image A) and horizontal perspective (Image B). -
FIG. 4 shows the central perspective drawing of three stacking blocks. -
FIG. 5 shows the horizontal perspective drawing of three stacking blocks. -
FIG. 6 shows the method of drawing a horizontal perspective drawing. -
FIG. 7 shows a horizontal perspective display and an viewer input device. -
FIG. 8 shows a horizontal perspective display, a computational device and an viewer input device. -
FIG. 9 shows mapping of the 3-d object onto the horizontal plane. -
FIG. 10 shows the projection of 3-d object by horizontal perspective. -
FIG. 11 shows the simulation time of the horizontal perspective. -
FIG. 12 shows an embodiment of the present invention multi-plane display. -
FIG. 13 shows the horizontal perspective and central perspective projection on the present invention multi-plane display. - The present invention discloses a multi-plane display system comprising at least two display surfaces, one of which capable of projecting three dimensional illusion based on horizontal perspective projection.
- In general, the present invention multi-plane display system can be used to display three dimensional images and has obvious utility to many industrial applications such as manufacturing design reviews, ergonomic simulation, safety and training, video games, cinematography, scientific 3D viewing, and medical and other data displays.
- Horizontal perspective is a little-known perspective, of which we found only two books that describe its mechanics: Stereoscopic Drawing (©1990) and How to Make Anaglyphs (©1979, out of print). Although these books describe this obscure perspective, they do not agree on its name. The first book refers to it as a “free-standing anaglyph,” and the second, a “phantogram.” Another publication called it “projective anaglyph” (U.S. Pat. No. 5,795,154 by G. M. Woods, Aug. 18, 1998). Since there is no agreed-upon name, we have taken the liberty of calling it “horizontal perspective.” Normally, as in central perspective, the plane of vision, at right angle to the line of sight, is also the projected plane of the picture, and depth cues are used to give the illusion of depth to this flat image. In horizontal perspective, the plane of vision remains the same, but the projected image is not on this plane. It is on a plane angled to the plane of vision. Typically, the image would be on the ground level surface. This means the image will be physically in the third dimension relative to the plane of vision. Thus horizontal perspective can be called horizontal projection.
- In horizontal perspective, the object is to separate the image from the paper, and fuse the image to the three dimension object that projects the horizontal perspective image. Thus the horizontal perspective image must be distorted so that the visual image fuses to form the free standing three dimensional figure. It is also essential the image is viewed from the correct eye points, otherwise the three dimensional illusion is lost. In contrast to central perspective images which have height and width, and project an illusion of depth, and therefore the objects are usually abruptly projected and the images appear to be in layers, the horizontal perspective images have actual depth and width, and illusion gives them height, and therefore there is usually a graduated shifting so the images appear to be continuous.
-
FIG. 3 compares key characteristics that differentiate central perspective and horizontal perspective. Image A shows key pertinent characteristics of central perspective, and Image B shows key pertinent characteristics of horizontal perspective. - In other words, in Image A, the real-life three dimension object (three blocks stacked slightly above each other) was drawn by the artist closing one eye, and viewing along a line of sight perpendicular to the vertical drawing plane. The resulting image, when viewed vertically, straight on, and through one eye, looks the same as the original image.
- In Image B, the real-life three dimension object was drawn by the artist closing one eye, and viewing along a line of sight 45° to the horizontal drawing plane. The resulting image, when viewed horizontally, at 45° and through one eye, looks the same as the original image.
- One major difference between central perspective showing in Image A and horizontal perspective showing in Image B is the location of the display plane with respect to the projected three dimensional image. In horizontal perspective of Image B, the display plane can be adjusted up and down, and therefore the projected image can be displayed in the open air above the display plane, i.e. a physical hand can touch (or more likely pass through) the illusion, or it can be displayed under the display plane, i.e. one cannot touch the illusion because the display plane physically blocks the hand. This is the nature of horizontal perspective, and as long as the camera eyepoint and the viewer eyepoint is at the same place, the illusion is present. In contrast, in central perspective of Image A, the three dimensional illusion is likely to be only inside the display plane, meaning one cannot touch it. To bring the three dimensional illusion outside of the display plane to allow viewer to touch it, the central perspective would need elaborate display scheme such as surround image projection and large volume.
-
FIGS. 4 and 5 illustrate the visual difference between using central and horizontal perspective. To experience this visual difference, first look atFIG. 4 , drawn with central perspective, through one open eye. Hold the piece of paper vertically in front of you, as you would a traditional drawing, perpendicular to your eye. You can see that central perspective provides a good representation of three dimension objects on a two dimension surface. - Now look at
FIG. 5 , drawn using horizontal perspective, by sifting at your desk and placing the paper lying flat (horizontally) on the desk in front of you. Again, view the image through only one eye. This puts your one open eye, called the eye point at approximately a 45° angle to the paper, which is the angle that the artist used to make the drawing. To get your open eye and its line-of-sight to coincide with the artist's, move your eye downward and forward closer to the drawing, about six inches out and down and at a 45° angle. This will result in the ideal viewing experience where the top and middle blocks will appear above the paper in open space. - Again, the reason your one open eye needs to be at this precise location is because both central and horizontal perspective not only define the angle of the line of sight from the eye point; they also define the distance from the eye point to the drawing. This means that
FIGS. 4 and 5 are drawn with an ideal location and direction for your open eye relative to the drawing surfaces. However, unlike central perspective where deviations from position and direction of the eye point create little distortion, when viewing a horizontal perspective drawing, the use of only one eye and the position and direction of that eye relative to the viewing surface are essential to seeing the open space three dimension horizontal perspective illusion. -
FIG. 6 is an architectural-style illustration that demonstrates a method for making simple geometric drawings on paper or canvas utilizing horizontal perspective.FIG. 6 is a side view of the same three blocks used inFIG. 5 . It illustrates the actual mechanics of horizontal perspective. Each point that makes up the object is drawn by projecting the point onto the horizontal drawing plane. To illustrate this,FIG. 6 shows a few of the coordinates of the blocks being drawn on the horizontal drawing plane through projection lines. These projection lines start at the eye point (not shown inFIG. 6 due to scale), intersect a point on the object, then continue in a straight line to where they intersect the horizontal drawing plane, which is where they are physically drawn as a single dot on the paper When an architect repeats this process for each and every point on the blocks, as seen from the drawing surface to the eye point along the line-of-sight the horizontal perspective drawing is complete, and looks likeFIG. 5 . - Notice that in
FIG. 6 , one of the three blocks appears below the horizontal drawing plane. With horizontal perspective, points located below the drawing surface are also drawn onto the horizontal drawing plane, as seen from the eye point along the line-of-site. Therefore when the final drawing is viewed, objects not only appear above the horizontal drawing plane, but may also appear below it as well-giving the appearance that they are receding into the paper. If you look again atFIG. 5 , you will notice that the bottom box appears to be below, or go into, the paper, while the other two boxes appear above the paper in open space. - The generation of horizontal perspective images require considerably more expertise to create than central perspective images. Even though both methods seek to provide the viewer the three dimension illusion that resulted from the two dimensional image, central perspective images produce directly the three dimensional landscape from the viewer or camera point. In contrast, the horizontal perspective image appears distorted when viewing head on, but this distortion has to be precisely rendered so that when viewing at a precise location, the horizontal perspective produces a three dimensional illusion.
- The present invention multi-plane display system promotes horizontal perspective projection viewing by providing the viewer with the means to adjust the displayed images to maximize the illusion viewing experience. By employing the computation power of the microprocessor and a real time display, the horizontal perspective display of the present invention is shown in
FIG. 7 , comprising a real timeelectronic display 100 capable of re-drawing the projected image, together with a viewer'sinput device 102 to adjust the horizontal perspective image. By re-display the horizontal perspective image so that its projection eyepoint coincides with the eyepoint of the viewer, the horizontal perspective display of the present invention can ensure the minimum distortion in rendering the three dimension illusion from the horizontal perspective method. The input device can be manually operated where the viewer manually inputs his or her eyepoint location, or change the projection image eyepoint to obtain the optimum three dimensional illusion. The input device can also be automatically operated where the display automatically tracks the viewer's eyepoint and adjust the projection image accordingly. The multi-plane display system removes the constraint that the viewers keeping their heads in relatively fixed positions, a constraint that create much difficulty in the acceptance of precise eyepoint location such as horizontal perspective or hologram display. - The horizontal perspective display system, shown in
FIG. 8 , can further acomputation device 110 in addition to the real timeelectronic display device 100 and projectionimage input device 112 providing input to thecomputational device 110 to calculating the projectional images for display to providing a realistic, minimum distortion three dimensional illusion to the viewer by coincide the viewer's eyepoint with the projection image eyepoint. The system can further comprise an image enlargement/reduction input device 115, or an imagerotation input device 117, or animage movement device 119 to allow the viewer to adjust the view of the projection images. - The input device can be operated manually or automatically. The input device can detect the position and orientation of the viewew eyepoint, to compute and to project the image onto the display according to the detection result. Alternatively, the input device can be made to detect the position and orientation of the viewer's head along with the orientation of the eyeballs. The input device can comprise an infrared detection system to detect the position the viewer's head to allow the viewer freedom of head movement. Other embodiments of the input device can be the triangulation method of detecting the viewer eyepoint location, such as a CCD camera providing position data suitable for the head tracking objectives of the invention. The input device can be manually operated by the viewer, such as a keyboard, mouse, trackball, joystick, or the like, to indicate the correct display of the horizontal perspective display images.
- The head or eye-tracking system can comprise a base unit and a head-mounted sensor on the head of the viewer. The head-mounted sensor produces signals showing the position and orientation of the viewer in response to the viewer's head movement and eye orientation. These signals can be received by the base unit and are used to compute the proper three dimensional projection images. The head or eye tracking system can be infrared cameras to capture images of the viewer's eyes. Using the captured images and other techniques of image processing, the position and orientation of the viewer's eyes can be determined, and then provided to the base unit. The head and eye tracking can be done in real time for small enough time interval to provide continous viewer's head and eye tracking.
- The multi-plane display system comprises a number of new computer hardware and software elements and processes, and together with existing components creates a horizontal perspective viewing simulator. For the viewer to experience these unique viewing simulations the computer hardware viewing surface is preferrably situated horizontally, such that the viewer's line of sight is at a 45° angle to the surface. Typically, this means that the viewer is standing or seated vertically, and the viewing surface is horizontal to the ground. Note that although the viewer can experience hands-on simulations at viewing angles other than 45° (e.g. 55°, 30° etc.), it is the optimal angle for the brain to recognize the maximum amount of spatial information in an open space image. Therefore, for simplicity's sake, we use “45°” throughout this document to mean “an approximate 45 degree angle”. Further, while horizontal viewing surface is preferred since it simulates viewers' experience with the horizontal ground, any viewing surface could offer similar three dimensional illusion experience. The horizontal perspective illusion can appear to be hanging from a ceiling by projecting the horizontal perspective images onto a ceiling surface, or appear to be floating from a wall by projecting the horizontal perspective images onto a vertical wall surface.
- The viewing simulations are generated within a three dimensional graphics view volume, both situated above and below the physical viewing surface. Mathematically, the computer-generated x, y, z coordinates of the Angled Camera point form the vertex of an infinite “pyramid”, whose sides pass through the x, y, z coordinates of the Reference/Horizontal Plane.
FIG. 9 illustrates this infinite pyramid, which begins at the Angled Camera point and extending through the Far Clip Plane. The viewing volume is defined by a Comfort Plane, a plabe on top of the viewing volume, and is appropriately named because its location within the pyramid determines the viewer's personal comfort, i.e. how their eyes, head, body, etc. are situated while viewing and interacting with simulations. - For the viewer to view open space images on their physical viewing device it must be positioned properly, which usually means the physical Reference Plane is placed horizontally to the ground. Whatever the viewing device's position relative to the ground, the Reference/Horizontal Plane must be at approximately a 45° angle to the viewer's line-of-site for optimum viewing.
- One way the viewer might perform this step is to position their CRT computer monitor on the floor in a stand, so that the Reference/Horizontal Plane is horizontal to the floor. This example uses a CRT-type television or computer monitor, but it could be any type of viewing device, display screen, monochromic or color display, luminescent, TFT, phosphorescent, computer projectors and other method of image generation in general, providing a viewing surface at approximately a 45° angle to the viewer's line-of-sight.
- The display needs to know the view's eyepoint to proper display the horizontal perspective images. One way to do this is for the viewer to supply the horizontal perspective display with their eye's real-world x, y, z location and line-of-site information relative to the center of the physical Reference/Horizontal Plane. For example, the viewer tells the horizontal perspective display that their physical eye will be located 12 inches up, and 12 inches back, while looking at the center of the Reference/Horizontal Plane. The horizontal perspective display then maps the computer-generated Angled Camera point to the viewer's eyepoint physical coordinates and line-of-site. Another way is for the viewer to manually adjusting an input device such as a mouse, and the horizontal perspective display adjust its image projection eyepoint until the proper eyepoint location is experienced by the viewer. Another way way is using triangulation with infrared device or camera to automatically locate the viewer's eyes locations.
-
FIG. 10 is an illustration of the horizontal perspective display that includes all of the new computer-generated and real physical elements as described in the steps above. It also shows that a real-world element and its computer-generated equivalent are mapped 1:1 and together share a common Reference Plane. The full implementation of this horizontal perspective display results in a real-time computer-generated three dimensional graphics appearing in open space on and above a viewing device's surface, which is oriented approximately 45° to the viewer's line-of-sight. - The present invention also allows the viewer to move around the three dimensional display and yet suffer no great distortion since the display can track the viewer eyepoint and re-display the images correspondingly, in contrast to the conventional pior art three dimensional image display where it would be projected and computed as seen from a singular viewing point, and thus any movement by the viewer away from the intended viewing point in space would cause gross distortion.
- The display system can further comprise a computer capable of re-calculate the projected image given the movement of the eyepoint location. The horizontal perspective images can be very complex, tedious to create, or created in ways that are not natural for artists or cameras, and therefore require the use of a computer system for the tasks. To display a three-dimensional image of an object with complex surfaces or to create an animation sequences would demand a lot of computational power and time, and therefore it is a task well suited to the computer. Three dimensional capable electronics and computing hardware devices and real-time computer-generated three dimensional computer graphics have advanced significantly recently with marked innovations in visual, audio and tactile systems, and have producing excellent hardware and software products to generate realism and more natural computer-human interfaces.
- The multi-plane display system of the present invention are not only in demand for entertainment media such as televisions, movies, and video games but are also needed from various fields such as education (displaying three-dimensional structures), technological training (displaying three-dimensional equipment). There is an increasing demand for three-dimensional image displays, which can be viewed from various angles to enable observation of real objects using object-like images. The horizontal perspective display system is also capable of substitute a computer-generated reality for the viewer observation. The systems may include audio, visual, motion and inputs from the user in order to create a complete experience of three dimensional illusion.
- The input for the horizontal perspective system can be two dimensional image, several images combined to form one single three dimensional image, or three dimensional model. The three dimensional image or model conveys much more information than that a two dimensional image and by changing viewing angle, the viewer will get the impression of seeing the same object from different perspectives continuously.
- The multi-plane display system can further provide multiple views or “Multi-View” capability. Multi-View provides the viewer with multiple and/or separate left-and right-eye views of the same simulation. Multi-View capability is a significant visual and interactive improvement over the single eye view. In Multi-View mode, both the left eye and right eye images are fused by the viewer's brain into a single, three-dimensional illusion. The problem of the discrepancy between accommodation and convergence of eyes, inherent in stereoscopic images, leading to the viewer's eye fatigue with large discrepancy, can be reduced with the horizontal perspective display, especially for motion images, since the position of the viewer's gaze point changes when the display scene changes.
- In Multi-View mode, the objective is to simulate the actions of the two eyes to create the perception of depth, namely the left eye and the right right sees slightly different images. Thus Multi-View devices that can be used in the present invention include methods with glasses such as anaglyph method, special polarized glasses or shutter glasses, methods without using glasses such as a parallax stereogram, a lenticular method, and mirror method (concave and convex lens).
- In anaglyph method, a display image for the right eye and a display image for the left eye are respectively superimpose-displayed in two colors, e.g., red and blue, and observation images for the right and left eyes are separated using color filters, thus allowing a viewer to recognize a stereoscopic image. The images are displayed using horizontal perspective technique with the viewer looking down at an angle. As with one eye horizontal perspective method, the eyepoint of the projected images has to be coincide with the eyepoint of the viewer, and therefore the viewer input device is essential in allowing the viewer to observe the three dimensional horizontal perspective illusion. From the early days of the anaglyph method, there are much improvements such as the spectrum of the red/blue glasses and display to generate much more realizm and comfort to the viewers.
- In polarized glasses method, the left eye image and the right eye image are separated by the use of mutually extinguishing polarizing filters such as orthogonally linear polarizer, circular polarizer, elliptical polarizer. The images are normally projected onto screens with polarizing filters and the viewer is then provided with corresponding polarized glasses. The left and right eye images appear on the screen at the same time, but only the left eye polarized light is transmitted through the left eye lens of the eyeglasses and only the right eye polarized light is transmitted through the right eye lens.
- Another way for stereocopic display is the image sequential system. In such a system, the images are displayed sequentially between left eye and right eye images rather than superimposing them upon one another, and the viewer's lenses are synchronized with the screen display to allow the left eye to see only when the left image is displayed, and the right eye to see only when the right image is displayed. The shuttering of the glasses can be achieved by mechanical shuttering or with liquid crystal electronic shuttering. In shuttering glass method, display images for the right and left eyes are alternately displayed on a CRT in a time sharing manner, and observation images for the right and left eyes are separated using time sharing shutter glasses which are opened/closed in a time sharing manner in synchronism with the display images, thus allowing an observer to recognize a stereoscopic image.
- Other way to display stereoscopic images is by optical method. In this method, display images for the right and left eyes, which are separately displayed on a viewer using optical means such as prisms, mirror, lens, and the like, are superimpose-displayed as observation images in front of an observer, thus allowing the observer to recognize a stereoscopic image. Large convex or concave lenses can also be used where two image projectors, projecting left eye and right eye images, are providing focus to the viewer's left and right eye respectively. A variation of the optical method is the lenticular method where the images form on cylindrical lens elements or two dimensional array of lens elements.
-
FIG. 11 is a horizontal perspective display focusing on how the computer-generated person's two eye views are projected onto the Horizontal Plane and then displayed on a stereoscopic 3D capable viewing device.FIG. 11 represents one complete display time period. During this display time period, the horizontal perspective display needs to generate two different eye views, because in this example the stereoscopic 3D viewing device requires a separate left- and right-eye view. There are existing stereoscopic 3D viewing devices that require more than a separate left- and right-eye view, and because the method described here can generate multiple views it works for these devices as well. - The illustration in the upper left of
FIG. 11 shows the Angled Camera point for the right eye after the first (right) eye-view to be generated. Once the first (right) eye view is complete, the horizontal perspective display starts the process of rendering the computer-generated person's second eye (left-eye) view. The illustration in the lower left ofFIG. 11 shows the Angled Camera point for the left eye after the completion of this time. But before the rendering process can begin, the horizontal perspective display makes an adjustment to the Angled Camera point. This is illustrated inFIG. 11 by the left eye's x coordinate being incremented by two inches. This difference between the right eye's x value and the left eye's x+2″ is what provides the two-inch separation between the eyes, which is required for stereoscopic 3D viewing. The distances between people's eyes vaty but in the above example we are using the average of 2 inches. It is also possible for the view to supply the horizontal perspective display with their personal eye separation value. This would make the x value for the left and right eyes highly accurate for a given viewer and thereby improve the quality of their stereoscopic 3D view. - Once the horizontal perspective display has incremented the Angled Camera point's x coordinate by two inches, or by the personal eye separation value supplied by the viewer, the rendering continues by displaying the second (left-eye) view.
- Depending on the stereoscopic 3D viewing device used, the horizontal perspective display continues to display the left- and right-eye images, as described above, until it needs to move to the next display time period. An example of when this may occur is if the bear cub moves his paw or any part of his body. Then a new and second Simulated Image would be required to show the bear cub in its new position. This new Simulated Image of the bear cub, in a slightly different location, gets rendered during a new display time period. This process of generating multiple views via the nonstop incrementing of display time continues as long as the horizontal perspective display is generating real-time simulations in stereoscopic 3D.
- By rapidly display the horizontal perspective images, three dimensional illusion of motion can be realized. Typically, 30 to 60 images per second would be adequate for the eye to perceive motion. For stereocopy, the same display rate is needed for superimposed images, and twice that amount would be needed for time sequential method.
- The display rate is the number of images per second that the display uses to completely generate and display one image. This is similar to a movie projector where 24 times a second it displays an image. Therefore, 1/24 of a second is required for one image to be displayed by the projector. But the display time could be a variable, meaning that depending on the complexity of the view volumes it could take 1/12 or ½ a second for the computer to complete just one display image. Since the display was generating a separate left and right eye view of the same image, the total display time is twice the display time for one eye image.
- The present invention further discloses a Multi-Plane display comprising a horizontal perspective display together with a non-horizontal central perspective display.
FIG. 12 illustrates an example of the present invention Multi-Plane display in which the Multi-Plane display is a computer monitor that is approximately “L” shaped when open. The end-user views the L-shaped computer monitor from its concave side and at approximately a 45° angle to the bottom of the “L,” as shown inFIG. 12 . From the end-user's point of view the entire L-shaped computer monitor appears as one single and seamless viewing surface. The bottom L of the display, positioned horizontally, shows horizontal perspective image, and the other branch of the L display shows central perspective image. The edge is the two display segments is preferably smoothly joined and can also having a curvilinear projection to connect the two displays of horizontal perspective and central perspective. - The Multi-Plane display can be made with one or more physical viewing surfaces. For example, the vertical leg of the “L” can be one physical viewing surface, such as flat panel display, and the horizontal leg of the “L” can be a separate flat panel display. The edge of the two display segments can be a non-display segment and therefore the two viewing surface are not continuous. Each leg of a Multi-Plane display is called a viewing plane and as you can see in the upper left of
FIG. 25 there is a vertical viewing plane and a horizontal viewing plane where a central perspective image is generated on the vertical plane and a horizontal perspective image is generated on the horizontal plane, and then blend the two images where the planes meet, as illustrated in the lower right ofFIG. 12 . -
FIG. 12 also illustrates that a Multi-Plane display is capable of generating multiple views. Meaning that it can display single-view images, i.e. a one-eye perspective like the simulation in the upper left, and/or multi-view images, i.e. separate right and left eye views like the simulation in the lower right. And when the L-shaped computer monitor is not being used by the end-user it can be closed and look like the simulation in the lower left. -
FIG. 13 is a simplified illustration of the present invention Multi-Plane display. In the upper right ofFIG. 13 is an example of a single-view image of a bear cub that is displayed on an L-shaped computer monitor. Normally a single-view or one eye image would be generated with only one camera point, but as you can see there are at least two camera points for the Multi-Plabe display even though this is a single-view example. This is because each viewing plane of a Multi-Plane device requires its own rendering perspective. One camera point is for the horizontal perspective image, which is displayed on the horizontal surface, and the other camera point is for the central perspective image, which is displayed on the vertical surface. - To generate both the horizontal perspective and central perspective images requires the creation of two camera eyepoints (which can be the same or different) as shown in
FIG. 13 for two different and separate camera points labeled OSI and CPI. The vertical viewing plane of the L-shaped monitor, as shown at the bottom ofFIG. 13 , is the display surface for the central perspective images, and thus there is a need to define another common reference plane for this surface. As discussed above, the common reference plane is the plane where the images are display, and the computer need to keep track of this plane for the synchronization of the locations of the displayed images and the real physical locations. With the L-shaped Multi-Plane device and the two display surfaces, the Simulation can to generate the three dimansional images, a horizontal perspective image using (OSI) camera eyepoint, and a central perspective image using (CPI) camera eyepoint. - The multi-plane display system can further include a curvilinear connection display section to blend the horizontal perspective and the central perspective images together at the location of the seam in the “L,” as shown at the bottom of
FIG. 13 . The multi-plane display system can continuously update and display what appears to be a single L-shaped image on the L-shaped Multi-Plane device. - Furthermore, the multi-plane display system can comprise multiple display surfaces together with multiple curvilinear blending sections as shown in
FIG. 14 . The multiple display surfaces can be a flat wall, multiple adjacent flat walls, a dome, and a curved wraparound panel. - The present invention multi-plane display system thus can simultaneously projecting a plurality of three dimensional images onto multiple display surfaces, one of which is a horizontal perspective image. Further, it can be a stereoscopic multiple display system allowing viewers to use their stereoscopic vision for three dimensional image presentation.
- Since the multi-plane display system comprises at least two display surfaces, various requirements need to be addressed to ensure high fidelity in the three dimensional image projection. The display requirements are typically geometric accuracy, to ensure that objects and features of the image to be correctly positioned, edge match accuracy, to ensure continuity between display surfaces, no blending variation, to ensure no variation in luminance in the blending section of various display surfaces, and field of view, to ensure a continuous image from the eyepoint of the viewer.
- Since the blending section of the multi-plane display system is preferably a curve surface, some distortion correction could be applied in order for the image projected onto the blending section surface to appear correct to the viewer. There are various solutions for providing distortion correction to a display system such as using a test pattern image, designing the image projection system for the specific curved blending display section, using special video hardware, utilizing a piecewise-linear approximation for the curved blending section. Still another distortion correction solution for the curve surface projection is to automatically computes image distortion correction for any given position of the viewer eyepoint and the projector.
- Since the multi-plane display system comprises more than one display surface, care should be taken to minimize the seams and gaps between the edges of the respective displays. To avoid seams or gaps problem, there could be at least two image generators generating adjacent overlapped portions of an image. The overlapped image is calculated by an image processor to ensure that the projected pixels in the overlapped areas are adjusted to form the proper displayed images. Other solutions are to control the degree of intensity reduction in the overlapping to create a smooth transition from the image of one display surface to the next.
- For realistic three dimensional display, binaural or three dimensional audio simulation is also included. The present invention also provide the means to adjust the binaural or 3D audio to ensure proper sound simulation.
- Similar to vision, hearing using one ear is called monoaural and hearing using two ears is called binaural. Hearing can provide the direction of the sound sources but with poorer resolution than vision, the identity and content of a sound source such as speech or music, and the nature of the environment via echoes, reverberation such as a normal room or an open field.
- The head and ears, and sometime the shoulder, function as an antenna system to provide information about the location, distance and environment of the sound sources. The brain can interprete properly the various kinds of sound arriving at the head such as direct sounds, diffractive sounds around the head and by interaction with the outer ears and shoulder, different sound amplitudes and different arrival time of the sounds. These acoustic modifications are called ‘sound cues’ and serve to provide us the directional acoustis information of the sounds.
- Basically, the sound cues are related to timing, volume, frequency and reflection. In timing cues, the ears recognize the time the sound arrives and assume that the sound comes from the closest source. Further, with two ears separated about 8 inches apart, the delay of the sound reaching one ear with respect to the other ear can give a cue about the location of the sound source. The timing cue is stronger than the level cue in the sense that the listener locates the sound based on the first wave that reaches the ear, regardless of the loudness of any later arriving waves. In volume (or level) cues, the ears recognize the volume (or loudness) of the sound and assume that the sound coming from the loudest direction. With the binaural (two ears) effect, the amplitude difference between the ears is a strong cue for the localization of the sound source. In frequency (or equalization) cues, the ears recognize the frequency balance of the sound as it arrives in each ear since frontal sounds are directed into the eardrums, while rear sounds bounce off the external ear and thus having a high frequency roll off. In reflection cue, the sound bounces off various surfaces and are either dispersed or absorbed in varying degrees before reaching the ears multiple times. This reflections off the walls of the room and the foreknowledge of the difference between the way various floor coverings sound also contribute to localization. In addition, the body, especially the head, can move relative to the sound source to help in locate the sound.
- The above various sound cues are scientifically classified into three types of spatial hearing cues: interaural time differences (ITDs), interaural level differences (ILDs), and head-related transfer functions (HRTFs). ITDs relate to the time for a sound to reach the ears and the time difference for reaching both ears. ILDs refer to the amplitude in the frequency spectrum of sound reaching the ears and also the amplitude differences of the sound frequencies as heard in both ears. HRTFs can provide the perception of distance by the changes in the timbre and distance dependencies, the time delay and directions of direct sound and reflections in echoic environments.
- The HRTFs are a collection of spatial cues for a particular listener, including ITDs, ILDs and the reflections, diffractions and damping caused by the listener's body, head, outer ears and shoulder. The external ear, or pinna, has a significant contribution to the HRTFs. Higher frequency sounds are filtered by the pinna to provide the brain a way as to perceive the lateral position, or azimuth, and elevation of the sound source since the response of the pinna filter is highly dependent on the overall direction of the sound source. The head can account for a reduced amplitude of various frequencies of the sounds since the sound has to go through or around the head in order to reach the ear. The overall effects of head shadowing contribute to the perception of linear distance and direction of a sound source. Further, sound frequencies in the range of 1-3 kHz are reflected from the shoulder to produce echoes representing a time delay dependent on the elevation of the sound source. The reflections from surfaces in the world and the reverberation also seem to affect the localization judgement of sound distance and direction.
- In addition to these cues, the movement of the head to help in locate the location of a sound source is a key factor, together with the vision to confirm the sound direction. For a 3D immersion, all mechanisms to localize the sounds are always in play and should normally agree. If not, there would be some discomfort and confusion.
- Although we can hear with one ear, hearing with two ears is clearly better. Many of the sound cues are related to the binaural perception depending on both the relative loudness of sound and the relative time of arrival of sound at each ear. And thus the binaural performance is clear superior for the localization of single or multiple sound sources and for the formation of the room environment, for the separation of signals coming from multiple incoherent and coherent sound sources; and the enhancement of a chosen signal in a reverberant environment.
- Mathematically speaking, HRTF is the frequency response of the sound waves as received by the ears. By measuring the HRTF of a particular listener, and by synthesised electronically using digital signal processing, the sounds can be delivered to a listener's ears via headphones or loudspeakers to create a virtual sound image in three dimensions.
- The sound transformation to the ear canal, i.e. HRTF frequency response, can be measured accurately by using small microphones in the ear canals. The measured signal is then processed by a computer to derive the HRTF frequency responses for the left and right ears corresponding to the sound source location.
- Thus a 3D audio system works by using the measured HRTFs as the audio filters or equalizers. When a sound signal is processed by the HRTFs filters, the sound localization cues are reproduced, and the listener should perceive the sound at the location specified by the HRTFs. This method of binaural synthesis works extremely well when the listener's own HRTFs are used to synthesize the localization cues. However, measuring HRTFs is a complicated procedure, so 3D audio systems typically use a single set of HRTFs previously measured from a particular human or manikin subject. Thus the HRTF sometimes needs to be changed to accurately respond to a perticular listener. The tuning of a HRTF function can be accomplished by providing various sound source locations and environments and asking the listener to identify.
- A 3D audio system should provide the ability for the listener to define a three-dimensional space, to position multiple sound sources and that listener in that 3D space, and to do it all in real-time, or interactively. Beside 3D audio system, other technologies such stereo extension and surround sound could offer some aspects of 3D positioning or interactivity.
- Extended stereo processes an existing stereo (two channel) soundtrack to add spaciousness and to make it appear to originate from outside the left/right speaker locations through fairly straight-forward methods. Some of the characteristics of the extended stereo technology include the size of the listening area (called sweet spot), the amount of spreading of stereo images, the amount of tonal changes, the amount of lost stereo panning information, and the ability to achieve effect on headphones as well as speakers.
- The surround sound create a larger-than-stereo sound stage with a surround sound 5-speaker setup. Additionally, virtual surround sound systems use 3D audio technology to create the illusion of five speakers emanating from a regular set of stereo speakers, therefore enabling a surround sound listening experience without the need for a five speaker setup. The characteristics of the surround sound technology include the presentation accuracy, the clarity of spatial imaging, and the size of the listening area For better 3D audio system, audio technology needs to create a life-like listening experience by replicating the 3D audio cues that the ears hear in the real world for allowing non-interactive and interactive listening and positioning of sounds anywhere in the three-dimensional space surrounding a listener.
- The head tracker function is also very important to provide perceptual room constancy to the listener. In other words, when the listener move their heads around, the signals would change so that the perceived auditory world maintain its spatial position.
- To this end, the simulation system needs to know the head position in order to be able to control the binaural impulse responses adequately. Head position sensors have therefore to be provided. The impression of being immersed is of particular relevance for applications in the context of virtual reality.
- A replica of a sound field can be produced by putting an infinite number of microphones everywhere. After being stored on a recorder with an infinite number of channels, this recording can then be played back through an infinite number of point-source loudspeakers, each placed exactly as its corresponding microphone was placed.
- As the number of microphones and speakers is reduced, the quality of the sound field being simulated suffers. By the time we are down to two channels, height cues have certainly been lost and instead of a stage that is audible from anywhere in the room we find that sources on the stage are now only localizable if we listen along a line equidistant from the last two remaining speakers and face them.
- However, only two channels should be adequate, since if we deliver the exact sound required to simulate a live performance at the entrance to each ear canal, then since we only have two ear canals, we should only need to generate two such sound fields. In other words, since we can hear three-dimensionally in the real world using just two ears, it must be possible to achieve the same effect from just two speakers or a set of headphones.
- Headphone reproduction is thus differed from loudspeaker reproduction since headphone microphones should be spaced about seven inches apart for a normal ear separation, and loudspeaker microphones separation should be about seven feet apart. Further loudspeakers suffer from crosstalk and therefore some signal conditioning such as crosstalk cancellation will be needed for 3D loudspeaker setup.
-
Loudspeaker 3D audio systems are extremely effective in desktop computing environments. This is because there is usually only a single listener (the computer user) who is almost always centered between the speakers and facing forward towards the monitor. Thus, the primary user gets the full 3D effect because the crosstalk is properly cancelled. In typical 3D audio applications, like video gaming, friends may gather around to watch. In this case, the best 3D audio effects are heard by others when they are also centered with respect to the loudspeakers. Off-center listeners may not get the full effect, but they still hear a high quality stereo program with some spatial enhancements. - To achieve 3D audio, the speakers are typically arranged surrounding the listener in about the same horizontal plane, but could be arranged to completely surround the listener, from the ceiling to the floor to the surrounding walls. Optionally, the speakers can also be put on the ceiling, on the floor, arranged in an overhead dome configuration, or arranged in a vertical wall configuration. Further, beam transmitted speakers can be used instead of headphone. Beam transmitted speaker offers the freedom of movement for the listener and without the crosstalk between speakers since beam transmitted speaker provide a tight beam of sound.
- Generally, a minimum of four loudspeakers are required to achieve a convincing 3-D audio experience, while some researchers are using twenty or more speakers in an anechoic chamber to recreate acoustic environments with much greater precision.
- The main advantages of multi-speaker playback are:
-
- There is no dependence on the individual subject's HRTF, since the sound field is created without any reference to individual listeners.
- The subject is free to turn their head, and even move about within a limited range.
- In some cases, more than one subject can listen to the system simultaneously.
- Many crosstalk cancellers are based on a highly simplified model of crosstalk, for example modeling crosstalk as a simple delay and attenuation process, or a delay and a lowpass filter. Other crosstalk cancellers have been based on a spherical head model. As with binaural synthesis, crosstalk cancellation performance is ultimately limited by the variation in the size and shape of human heads.
- 3D audio simulation can be accomplished by the following steps:
-
- Input the characteristics of the acoustic space.
- Determine the sequence of sound arrivals that occur at the listening position. Each sound arrival will have the following characteristics: (a) time of arrival, based on the distance travelled by the echo-path, (b) direction of arrival, (c) attenuation (as a function of frequency) of the sound due to the absorption properties of the surfaces encountered by the echo-path.
- Compute the impulse response of the acoustic space incorporating the multiple sound arrivals.
- The results from the FIR filter are played back to a listener. In the case where the impulse responses were computed using a dummy head response, the results are played over headphones to the listener. In this case, the equalisation required for the particular headphones is also applied.
- The simulation of an acoustic environment involves one or more of the following functions:
-
- Processing an audio source input and presenting it to the subject through a number of loudspeakers (or headphones) with the intention of making the sound source appear to be located at a particular position in space.
- Processing multiple input audio sources in such a way that each source is independently located in space around the subject.
- Enhanced processing to simulate some aspects of the room acoustics, so that the user can acoustically sense the size of the room and the nature of the floor and wall coverings.
- The capability for the subject to move (perhaps within a limited range) and turn his/her head so as to focus attention on some aspects of the sound source characteristics or room acoustics.
- Binaural simulation is generally carried out using the sound source material free from any unwanted echoes or noise. The sound source material can then be replayed to a subject, using the appropriate HRTF filters, to create the illusion that the source audio is originating from a particular direction. The HRTF filtering is achieved by simply convolving the audio signal with the pair of HRTF responses (one HRTF filter for each channel of the headphone).
- The eyes and ears often perceive an event at the same time. Seeing a door close, and hearing a shutting sound, are interpreted as one event if they happen synchronously. If we see a door shut without a sound, or we see a door shut in front of us, and hear a shutting sound to the left, we get alarmed and confused. In another scenario, we might hear a voice in front of us, and see a hallway with a corner; the combination of audio and visual cues allows us to figure out that a person might be standing around the corner. Together, synchronized 3D audio and 3D visual cues provide a very strong immersion experience.
- Both 3D audio and 3D graphics systems can be greatly enhanced by such synchronization.
- Improved playback through headphones can be achieved through the use of head tracking. This technique makes use of continuous measurements of the orientation of a subject's head, and adapts the audio signals being fed to the headphones appropriately.
- Binaural signal should allow a subject to easily discriminate between left and right sound source locations easily, but the ability to discriminate between front and back, and high and low sound sources is generally only possible if head movement is permitted. Whilst multiple speaker playback methods solve this problem to a large degree, there are still many applications where headphone playback is preferable, and head tracking can then be used as a valuable tool for improving the quality of the 3-D playback.
- The simplest form of head tracking binaural system is one which simply simulates anechoic HRTFs, and changes the HRTF functions rapidly in response to the subjects head movements. This HRTF switching can be achieved through a lookup table, with interpolation used to resolve angles that are not represented in the HRTF table.
- Simulation of room acoustics over headphones with head tracking becomes more difficult because the direction of arrival of the early reflections is also important in making the result sound realistic. Many researchers believe that the echoes in the reverberant tail of the room response are generally so diffuse that there is no requirement for this part of the room response to be tracked with the subject's head movements.
- An important feature of any head tracking playback system is the delay from the subject head movement to the change in the audio response at the headphones. If this delay is excessive, the subject can experience a form of virtual motion sickness and general disorientation.
- Audio cues change dramatically when a listener tilts or rotates his or her head. For example, quickly turning the head 90 degrees to look to the side is the equivalent of a sound traveling from the listener's side to the front in a split second. We often use head motion to track sounds or to search for them. The ears alert the brain about an event outside of the area that the eyes are currently focused on, and we automatically turn to redirect our attention. Additionally, we use head motion to resolve ambiguities: a faint, low sound could be either in front or back of us, so we quickly and sub-consciously turn our head a small fraction to the left, and we know if the sound is now off to the right, it is in the front, otherwise it is in the back. One of the reasons why interactive audio is more realistic than pre-recorded audio (soundtracks) is the fact that the listeners head motion can be properly simulated in an interactive system (using inputs from a joystick, mouse, or head-tracking system).
- The HRTF function are performed using digital signal processing (DSP) hardware for real time performance. Typical feature of DSP are that the direct sound must be processed to give the correct amplitude and perceived direction, the early echoes must arrive at the listener with appropriate time, amplitude and frequency response to give the perception of the size of the spaces (as well as the acoustic nature of the room surfaces), and the late reverberation must be natural and correctly distributed in 3-D around the listener. The relative amplitude of the direct sound compared to the remainder of the room response helps to provide the sensation of distance.
- Thus 3D audio simulation can provides a binaural gain so that the exact same audio content is more audible and intelligible in the binaural case, because the brain can localize and therefore “single out” the binaural signal, while the non-binaural signal gets washed into the noise. Further the listener would still be able to tune into and understand individual conversations, because they are still spatially separated, and “amplified by” binaural gain, an effect called the cocktail party effct. Binaural simulation also can provide faster reaction time because such a signal mirrors the ones received in the real world. In addition, binaural signals can convey positional information: a binaural radar warning sound can warn a user about a specific object that is approaching (with a sound that is unique to that object), and naturally indicate where that object is coming from. Also listening to binaural simulation can be less fatigue since we are used to hearing sounds that originate outside of their heads, as is the case with binaural signals. Mono or stereo signals appear to come from inside a listener's head when using headphones, and produce more strain than a natural sounding, binaural signal. An lastly, 3D binaural simulation can provide an increased perception and immersion in
higher quality 3D environment when visuals are shown in synch with binaural sound.
Claims (20)
1. A method of three dimensional image display by horizontal perspective projection and 3-D audio system projection, the horizontal perspective projection comprising a display of horizontal perspective images according to a predetermined projection eyepoint, and the 3-D audio system projection comprising providing 3-D sound according to a predertermined projection earpoint, the method comprising the steps of:
detecting an eyepoint location of a viewer;
displaying a horizontal perspective image using the detected eyepoint location as the projection eyepoint; and
projecting a 3-D sound to the viewer using the eyepoint location as the projection earpoint.
2. A method as in claim 2 wherein the 3-D sound comprises two sound channels and a HRTF (head related transfer function) filter.
3. A method as in claim 2 wherein the 3-D sound comprises a 3-D loudspeaker audio system or a 3-D headphone audio system.
4. A method of three dimensional image display by horizontal perspective projection and 3-D audio system projection, the horizontal perspective projection comprising a display of horizontal perspective images according to a predetermined projection eyepoint, and the 3-D audio system projection comprising providing 3-D sound according to a predertermined projection earpoint, the method comprising the steps of:
continuously scanning to detect an eyepoint location of a viewer;
calculating a new horizontal perspective image using the detected eyepoint location as the projection eyepoint;
displaying the new image;
calculating a new 3-D sound using the detected eyepoint location as the projection earpoint; and
displaying the new 3-D sound.
5. A method as in claim 4 wherein the 3-D sound comprises two sound channels and a HRTF (head related transfer function) filter.
6. A method as in claim 4 wherein the 3-D sound comprises a 3-D loudspeaker audio system or a 3-D headphone audio system.
7. A method as in claim 4 wherein the horizontal perspective image is stereoscopic images.
8. A method as in claim 4 wherein the horizontal perspective image is calculated from a flat two dimensional picture.
9. A method as in claim 4 wherein the horizontal perspective image is calculated from a three dimensional model.
10. A method as in claim 4 wherein detecting a viewer eyepoint location is by manually inputting the location through a manual input device.
11. A method as in claim 10 wherein the manual input device is a computer peripheral or a wireless computer peripheral.
12. A method as in claim 10 wherein the manual input device is selected from a group consisted of a keyboard, a stylus, a keypad, a computer mouse, a computer trackball, a tablet, a pointing device.
13. A method as in claim 4 wherein the detection of a viewer eyepoint location is through an automatic input device whereby the automatic input device automatically extracts the eyepoint location from the viewer.
14. A method as in claim 13 wherein the automatic input device is selected from a group consisted of radio-frequency tracking device, infrared tracking device, camera tracking device.
15. A method as in claim 4 further comprising the step of manipulating the image.
16. A method as in claim 4 wherein the manipulation of the image comprises the modification of the displayed image or the generation of a new image.
17. A method as in claim 4 wherein the modification of the displayed image comprises the movement, zooming or rotation of the image.
18. A method of image display by horizontal perspective projection and 3-D audio system projection, the horizontal perspective projection comprising a display of horizontal perspective images according to a predetermined projection eyepoint, and the 3-D audio system projection comprising providing 3-D sound according to a predertermined projection earpoint, the method comprising the steps of:
displaying a first image onto a first display;
continuously scanning to detect an eyepoint location of a viewer;
calculating a second horizontal perspective image using the detected eyepoint location as the projection eyepoint;
displaying the second image onto a second horizontal perspective display; and
projecting a 3-D sound to the viewer using the eyepoint location as the projection earpoint.
19. A method as in claim 18 further comprising a step of display a third image onto a third curvilinear display, the curvilinear display blending the first display and the second display.
20. A method as in claim 18 further comprising a step of calculating the third image using the eyepoint location as the projection eyepoint before displaying.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US11/141,606 US20050264857A1 (en) | 2004-06-01 | 2005-05-31 | Binaural horizontal perspective display |
Applications Claiming Priority (5)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US57618704P | 2004-06-01 | 2004-06-01 | |
US57618904P | 2004-06-01 | 2004-06-01 | |
US57618104P | 2004-06-01 | 2004-06-01 | |
US57618204P | 2004-06-01 | 2004-06-01 | |
US11/141,606 US20050264857A1 (en) | 2004-06-01 | 2005-05-31 | Binaural horizontal perspective display |
Publications (1)
Publication Number | Publication Date |
---|---|
US20050264857A1 true US20050264857A1 (en) | 2005-12-01 |
Family
ID=35462954
Family Applications (8)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/141,649 Active 2028-04-22 US7796134B2 (en) | 2004-06-01 | 2005-05-31 | Multi-plane horizontal perspective display |
US11/141,650 Abandoned US20050275915A1 (en) | 2004-06-01 | 2005-05-31 | Multi-plane horizontal perspective display |
US11/141,651 Abandoned US20050264558A1 (en) | 2004-06-01 | 2005-05-31 | Multi-plane horizontal perspective hands-on simulator |
US11/141,541 Abandoned US20050275914A1 (en) | 2004-06-01 | 2005-05-31 | Binaural horizontal perspective hands-on simulator |
US11/141,540 Abandoned US20050275913A1 (en) | 2004-06-01 | 2005-05-31 | Binaural horizontal perspective hands-on simulator |
US11/141,652 Abandoned US20050264559A1 (en) | 2004-06-01 | 2005-05-31 | Multi-plane horizontal perspective hands-on simulator |
US11/141,606 Abandoned US20050264857A1 (en) | 2004-06-01 | 2005-05-31 | Binaural horizontal perspective display |
US11/141,640 Abandoned US20050281411A1 (en) | 2004-06-01 | 2005-05-31 | Binaural horizontal perspective display |
Family Applications Before (6)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/141,649 Active 2028-04-22 US7796134B2 (en) | 2004-06-01 | 2005-05-31 | Multi-plane horizontal perspective display |
US11/141,650 Abandoned US20050275915A1 (en) | 2004-06-01 | 2005-05-31 | Multi-plane horizontal perspective display |
US11/141,651 Abandoned US20050264558A1 (en) | 2004-06-01 | 2005-05-31 | Multi-plane horizontal perspective hands-on simulator |
US11/141,541 Abandoned US20050275914A1 (en) | 2004-06-01 | 2005-05-31 | Binaural horizontal perspective hands-on simulator |
US11/141,540 Abandoned US20050275913A1 (en) | 2004-06-01 | 2005-05-31 | Binaural horizontal perspective hands-on simulator |
US11/141,652 Abandoned US20050264559A1 (en) | 2004-06-01 | 2005-05-31 | Multi-plane horizontal perspective hands-on simulator |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/141,640 Abandoned US20050281411A1 (en) | 2004-06-01 | 2005-05-31 | Binaural horizontal perspective display |
Country Status (5)
Country | Link |
---|---|
US (8) | US7796134B2 (en) |
EP (2) | EP1759379A2 (en) |
JP (2) | JP2008507006A (en) |
KR (2) | KR20070052261A (en) |
WO (2) | WO2005119376A2 (en) |
Cited By (19)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20090125801A1 (en) * | 2007-11-10 | 2009-05-14 | Cherif Atia Algreatly | 3D windows system |
US7796134B2 (en) | 2004-06-01 | 2010-09-14 | Infinite Z, Inc. | Multi-plane horizontal perspective display |
US20100309390A1 (en) * | 2009-06-03 | 2010-12-09 | Honeywood Technologies, Llc | Multimedia projection management |
US7907167B2 (en) | 2005-05-09 | 2011-03-15 | Infinite Z, Inc. | Three dimensional horizontal perspective workstation |
US20110187706A1 (en) * | 2010-01-29 | 2011-08-04 | Vesely Michael A | Presenting a View within a Three Dimensional Scene |
US20110248987A1 (en) * | 2010-04-08 | 2011-10-13 | Disney Enterprises, Inc. | Interactive three dimensional displays on handheld devices |
US20120002828A1 (en) * | 2010-06-30 | 2012-01-05 | Sony Corporation | Audio processing device, audio processing method, and program |
US20120022842A1 (en) * | 2009-02-11 | 2012-01-26 | Arkamys | Test platform implemented by a method for positioning a sound object in a 3d sound environment |
US20120148055A1 (en) * | 2010-12-13 | 2012-06-14 | Samsung Electronics Co., Ltd. | Audio processing apparatus, audio receiver and method for providing audio thereof |
US8315443B2 (en) | 2010-04-22 | 2012-11-20 | Qualcomm Incorporated | Viewpoint detector based on skin color area and face area |
US20130069880A1 (en) * | 2010-04-13 | 2013-03-21 | Dean Stark | Virtual product display |
US20130342536A1 (en) * | 2012-06-22 | 2013-12-26 | Canon Kabushiki Kaisha | Image processing apparatus, method of controlling the same and computer-readable medium |
US8717423B2 (en) | 2005-05-09 | 2014-05-06 | Zspace, Inc. | Modifying perspective of stereoscopic images based on changes in user viewpoint |
US8786529B1 (en) | 2011-05-18 | 2014-07-22 | Zspace, Inc. | Liquid crystal variable drive voltage |
US9265458B2 (en) | 2012-12-04 | 2016-02-23 | Sync-Think, Inc. | Application of smooth pursuit cognitive testing paradigms to clinical drug development |
US9380976B2 (en) | 2013-03-11 | 2016-07-05 | Sync-Think, Inc. | Optical neuroinformatics |
US9492122B2 (en) | 2011-06-28 | 2016-11-15 | Kabushiki Kaisha Toshiba | Medical image processing apparatus |
US10503457B2 (en) | 2017-05-05 | 2019-12-10 | Nvidia Corporation | Method and apparatus for rendering perspective-correct images for a tilted multi-display environment |
US11170563B2 (en) * | 2018-01-04 | 2021-11-09 | 8259402 Canada Inc. | Immersive environment with digital environment to enhance depth sensation |
Families Citing this family (127)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CA2482240A1 (en) * | 2004-09-27 | 2006-03-27 | Claude Choquet | Body motion training and qualification system and method |
JP4046121B2 (en) * | 2005-03-24 | 2008-02-13 | セイコーエプソン株式会社 | Stereoscopic image display apparatus and method |
EP1889171A4 (en) * | 2005-04-07 | 2012-11-28 | Visionsense Ltd | Method for reconstructing a three- dimensional surface of an object |
US20060285832A1 (en) * | 2005-06-16 | 2006-12-21 | River Past Corporation | Systems and methods for creating and recording digital three-dimensional video streams |
KR101309176B1 (en) * | 2006-01-18 | 2013-09-23 | 삼성전자주식회사 | Apparatus and method for augmented reality |
JP4940671B2 (en) * | 2006-01-26 | 2012-05-30 | ソニー株式会社 | Audio signal processing apparatus, audio signal processing method, and audio signal processing program |
US9104195B2 (en) | 2006-12-20 | 2015-08-11 | Lincoln Global, Inc. | Welding job sequencer |
US10994358B2 (en) | 2006-12-20 | 2021-05-04 | Lincoln Global, Inc. | System and method for creating or modifying a welding sequence based on non-real world weld data |
US9937577B2 (en) | 2006-12-20 | 2018-04-10 | Lincoln Global, Inc. | System for a welding sequencer |
GB2447060B (en) | 2007-03-01 | 2009-08-05 | Magiqads Sdn Bhd | Method of creation of a virtual three dimensional image to enable its reproduction on planar substrates |
US8665180B2 (en) | 2007-03-02 | 2014-03-04 | Nec Corporation | Image display device |
WO2008108389A1 (en) * | 2007-03-07 | 2008-09-12 | Nec Corporation | Image display |
JP2008226400A (en) * | 2007-03-15 | 2008-09-25 | Sony Computer Entertainment Inc | Audio reproducing system and audio reproducing method |
US8269822B2 (en) * | 2007-04-03 | 2012-09-18 | Sony Computer Entertainment America, LLC | Display viewing system and methods for optimizing display view based on active tracking |
US8400493B2 (en) * | 2007-06-25 | 2013-03-19 | Qualcomm Incorporated | Virtual stereoscopic camera |
US8355019B2 (en) * | 2007-11-02 | 2013-01-15 | Dimension Technologies, Inc. | 3D optical illusions from off-axis displays |
US8452052B2 (en) * | 2008-01-21 | 2013-05-28 | The Boeing Company | Modeling motion capture volumes with distance fields |
JP4991621B2 (en) * | 2008-04-17 | 2012-08-01 | キヤノン株式会社 | Imaging device |
CN101610360A (en) * | 2008-06-19 | 2009-12-23 | 鸿富锦精密工业(深圳)有限公司 | The camera head of automatically tracking sound source |
KR101908033B1 (en) | 2008-07-10 | 2018-12-10 | 리얼 뷰 이미징 리미티드 | Broad viewing angle displays and user interfaces |
US8884177B2 (en) | 2009-11-13 | 2014-11-11 | Lincoln Global, Inc. | Systems, methods, and apparatuses for monitoring weld quality |
US9280913B2 (en) | 2009-07-10 | 2016-03-08 | Lincoln Global, Inc. | Systems and methods providing enhanced education and training in a virtual reality environment |
US9318026B2 (en) | 2008-08-21 | 2016-04-19 | Lincoln Global, Inc. | Systems and methods providing an enhanced user experience in a real-time simulated virtual reality welding environment |
US8915740B2 (en) | 2008-08-21 | 2014-12-23 | Lincoln Global, Inc. | Virtual reality pipe welding simulator |
US8851896B2 (en) | 2008-08-21 | 2014-10-07 | Lincoln Global, Inc. | Virtual reality GTAW and pipe welding simulator and setup |
US8911237B2 (en) | 2008-08-21 | 2014-12-16 | Lincoln Global, Inc. | Virtual reality pipe welding simulator and setup |
US9330575B2 (en) | 2008-08-21 | 2016-05-03 | Lincoln Global, Inc. | Tablet-based welding simulator |
US9483959B2 (en) | 2008-08-21 | 2016-11-01 | Lincoln Global, Inc. | Welding simulator |
US9196169B2 (en) | 2008-08-21 | 2015-11-24 | Lincoln Global, Inc. | Importing and analyzing external data using a virtual reality welding system |
US8657605B2 (en) | 2009-07-10 | 2014-02-25 | Lincoln Global, Inc. | Virtual testing and inspection of a virtual weldment |
US8747116B2 (en) | 2008-08-21 | 2014-06-10 | Lincoln Global, Inc. | System and method providing arc welding training in a real-time simulated virtual reality environment using real-time weld puddle feedback |
US8834168B2 (en) * | 2008-08-21 | 2014-09-16 | Lincoln Global, Inc. | System and method providing combined virtual reality arc welding and three-dimensional (3D) viewing |
US20110187844A1 (en) * | 2008-09-12 | 2011-08-04 | Kabushiki Kaisha Toshiba | Image irradiation system and image irradiation method |
US20100128112A1 (en) * | 2008-11-26 | 2010-05-27 | Samsung Electronics Co., Ltd | Immersive display system for interacting with three-dimensional content |
US20100156907A1 (en) * | 2008-12-23 | 2010-06-24 | Microsoft Corporation | Display surface tracking |
US8274013B2 (en) * | 2009-03-09 | 2012-09-25 | Lincoln Global, Inc. | System for tracking and analyzing welding activity |
TW201039099A (en) * | 2009-04-22 | 2010-11-01 | Song-Yuan Hu | Electronic device |
US8279269B2 (en) * | 2009-04-29 | 2012-10-02 | Ke-Ou Peng | Mobile information kiosk with a three-dimensional imaging effect |
US20100306825A1 (en) * | 2009-05-27 | 2010-12-02 | Lucid Ventures, Inc. | System and method for facilitating user interaction with a simulated object associated with a physical location |
KR101647536B1 (en) * | 2009-05-29 | 2016-08-10 | 휴렛-팩커드 디벨롭먼트 컴퍼니, 엘.피. | Multi-projector system and method |
JP4681690B2 (en) * | 2009-06-11 | 2011-05-11 | 新日本製鐵株式会社 | Manufacturing method of thick high strength steel plate with excellent heat input heat affected zone toughness, and thick high strength steel plate with high heat input heat affected zone toughness |
CN102804789B (en) | 2009-06-23 | 2015-04-29 | Lg电子株式会社 | Receiving system and method of providing 3D image |
KR20120039703A (en) | 2009-07-07 | 2012-04-25 | 엘지전자 주식회사 | Method for displaying three-dimensional user interface |
US9773429B2 (en) | 2009-07-08 | 2017-09-26 | Lincoln Global, Inc. | System and method for manual welder training |
US9221117B2 (en) | 2009-07-08 | 2015-12-29 | Lincoln Global, Inc. | System for characterizing manual welding operations |
US10748447B2 (en) | 2013-05-24 | 2020-08-18 | Lincoln Global, Inc. | Systems and methods providing a computerized eyewear device to aid in welding |
US9011154B2 (en) | 2009-07-10 | 2015-04-21 | Lincoln Global, Inc. | Virtual welding system |
US9294751B2 (en) | 2009-09-09 | 2016-03-22 | Mattel, Inc. | Method and system for disparity adjustment during stereoscopic zoom |
EP2478492B1 (en) * | 2009-09-16 | 2019-03-20 | Sydac Pty Ltd | Visual presentation system |
US8569655B2 (en) | 2009-10-13 | 2013-10-29 | Lincoln Global, Inc. | Welding helmet with integral user interface |
RU2524834C2 (en) * | 2009-10-14 | 2014-08-10 | Нокиа Корпорейшн | Autostereoscopic rendering and display apparatus |
KR101370925B1 (en) | 2009-10-16 | 2014-03-07 | 엘지전자 주식회사 | Method for indicating a 3d contents and apparatus for processing a signal |
US8569646B2 (en) | 2009-11-13 | 2013-10-29 | Lincoln Global, Inc. | Systems, methods, and apparatuses for monitoring weld quality |
US9468988B2 (en) | 2009-11-13 | 2016-10-18 | Lincoln Global, Inc. | Systems, methods, and apparatuses for monitoring weld quality |
WO2011071929A2 (en) * | 2009-12-07 | 2011-06-16 | Photon-X, Inc. | 3d visualization system |
US8532962B2 (en) | 2009-12-23 | 2013-09-10 | Honeywell International Inc. | Approach for planning, designing and observing building systems |
DE102010009737A1 (en) * | 2010-03-01 | 2011-09-01 | Institut für Rundfunktechnik GmbH | Method and arrangement for reproducing 3D image content |
US20110234591A1 (en) * | 2010-03-26 | 2011-09-29 | Microsoft Corporation | Personalized Apparel and Accessories Inventory and Display |
TWI407773B (en) * | 2010-04-13 | 2013-09-01 | Nat Univ Tsing Hua | Method and system for providing three dimensional stereo image |
US8995678B2 (en) | 2010-04-30 | 2015-03-31 | Honeywell International Inc. | Tactile-based guidance system |
US8990049B2 (en) | 2010-05-03 | 2015-03-24 | Honeywell International Inc. | Building structure discovery and display from various data artifacts at scene |
US8538687B2 (en) | 2010-05-04 | 2013-09-17 | Honeywell International Inc. | System for guidance and navigation in a building |
JP2011259373A (en) * | 2010-06-11 | 2011-12-22 | Sony Corp | Stereoscopic image display device and stereoscopic image display method |
JP5488306B2 (en) * | 2010-07-29 | 2014-05-14 | 船井電機株式会社 | projector |
US20120056875A1 (en) * | 2010-08-11 | 2012-03-08 | Lg Electronics Inc. | Method for operating image display apparatus |
KR101695819B1 (en) * | 2010-08-16 | 2017-01-13 | 엘지전자 주식회사 | A apparatus and a method for displaying a 3-dimensional image |
US8704879B1 (en) | 2010-08-31 | 2014-04-22 | Nintendo Co., Ltd. | Eye tracking enabling 3D viewing on conventional 2D display |
US20120062551A1 (en) * | 2010-09-13 | 2012-03-15 | Lg Electronics Inc. | Image display apparatus and method for operating image display apparatus |
CN103098459B (en) * | 2010-09-22 | 2016-11-09 | 富士胶片株式会社 | Stereo photographic device and shadow correction method |
US9001053B2 (en) | 2010-10-28 | 2015-04-07 | Honeywell International Inc. | Display system for controlling a selector symbol within an image |
WO2012082105A1 (en) | 2010-12-13 | 2012-06-21 | Edison Welding Institute, Inc. | Welding training system |
ES2768033T3 (en) | 2010-12-22 | 2020-06-19 | Zspace Inc | Three-dimensional tracking of a user control device in a volume |
US8773946B2 (en) | 2010-12-30 | 2014-07-08 | Honeywell International Inc. | Portable housings for generation of building maps |
US8982192B2 (en) | 2011-04-07 | 2015-03-17 | Her Majesty The Queen In Right Of Canada As Represented By The Minister Of Industry, Through The Communications Research Centre Canada | Visual information display on curvilinear display surfaces |
US9766698B2 (en) | 2011-05-05 | 2017-09-19 | Nokia Technologies Oy | Methods and apparatuses for defining the active channel in a stereoscopic view by using eye tracking |
JP5746908B2 (en) * | 2011-05-06 | 2015-07-08 | 株式会社東芝 | Medical image processing device |
US9560314B2 (en) | 2011-06-14 | 2017-01-31 | Microsoft Technology Licensing, Llc | Interactive and shared surfaces |
US9342928B2 (en) | 2011-06-29 | 2016-05-17 | Honeywell International Inc. | Systems and methods for presenting building information |
DE102011112618A1 (en) * | 2011-09-08 | 2013-03-14 | Eads Deutschland Gmbh | Interaction with a three-dimensional virtual scenario |
CN103828359B (en) * | 2011-09-29 | 2016-06-22 | 杜比实验室特许公司 | For producing the method for the view of scene, coding system and solving code system |
US9606992B2 (en) * | 2011-09-30 | 2017-03-28 | Microsoft Technology Licensing, Llc | Personal audio/visual apparatus providing resource management |
AU2012326218B2 (en) | 2011-10-17 | 2017-03-09 | Butterfly Network, Inc. | Transmissive imaging and related apparatus and methods |
US9106903B2 (en) | 2011-11-18 | 2015-08-11 | Zspace, Inc. | Head tracking eyewear system |
US9292184B2 (en) | 2011-11-18 | 2016-03-22 | Zspace, Inc. | Indirect 3D scene positioning control |
FR2983330B1 (en) * | 2011-11-24 | 2014-06-20 | Thales Sa | METHOD AND DEVICE FOR REPRESENTING SYNTHETIC ENVIRONMENTS |
ITTO20111150A1 (en) | 2011-12-14 | 2013-06-15 | Univ Degli Studi Genova | PERFECT THREE-DIMENSIONAL STEREOSCOPIC REPRESENTATION OF VIRTUAL ITEMS FOR A MOVING OBSERVER |
JP6017795B2 (en) * | 2012-02-10 | 2016-11-02 | 任天堂株式会社 | GAME PROGRAM, GAME DEVICE, GAME SYSTEM, AND GAME IMAGE GENERATION METHOD |
US20160093233A1 (en) | 2012-07-06 | 2016-03-31 | Lincoln Global, Inc. | System for characterizing manual welding operations on pipe and other curved structures |
US9767712B2 (en) | 2012-07-10 | 2017-09-19 | Lincoln Global, Inc. | Virtual reality pipe welding simulator and setup |
US20140125763A1 (en) * | 2012-11-07 | 2014-05-08 | Robert Nathan Cohen | 3d led output device and process for emitting 3d content output for large screen applications and which is visible without 3d glasses |
KR101916663B1 (en) * | 2012-12-18 | 2018-11-08 | 삼성전자주식회사 | Device of displaying 3d image using at least one of gaze direction of user or gravity direction |
US9667889B2 (en) | 2013-04-03 | 2017-05-30 | Butterfly Network, Inc. | Portable electronic devices with integrated imaging capabilities |
US10019130B2 (en) | 2013-04-21 | 2018-07-10 | Zspace, Inc. | Zero parallax drawing within a three dimensional display |
EP2806404B1 (en) | 2013-05-23 | 2018-10-10 | AIM Sport AG | Image conversion for signage |
US10930174B2 (en) | 2013-05-24 | 2021-02-23 | Lincoln Global, Inc. | Systems and methods providing a computerized eyewear device to aid in welding |
US9426598B2 (en) | 2013-07-15 | 2016-08-23 | Dts, Inc. | Spatial calibration of surround sound systems including listener position estimation |
US20150072323A1 (en) | 2013-09-11 | 2015-03-12 | Lincoln Global, Inc. | Learning management system for a real-time simulated virtual reality welding training environment |
US10083627B2 (en) | 2013-11-05 | 2018-09-25 | Lincoln Global, Inc. | Virtual reality and real welding training system and method |
JP2017505933A (en) * | 2013-11-26 | 2017-02-23 | シェフィ,ヨァヴ | Method and system for generating a virtual image fixed on a real object |
US9836987B2 (en) | 2014-02-14 | 2017-12-05 | Lincoln Global, Inc. | Virtual reality pipe welding simulator and setup |
US9667951B2 (en) | 2014-02-18 | 2017-05-30 | Cisco Technology, Inc. | Three-dimensional television calibration |
WO2015142936A1 (en) | 2014-03-17 | 2015-09-24 | Meggitt Training Systems Inc. | Method and apparatus for rendering a 3-dimensional scene |
JP6687543B2 (en) | 2014-06-02 | 2020-04-22 | リンカーン グローバル,インコーポレイテッド | System and method for hand welding training |
US9841844B2 (en) * | 2014-06-20 | 2017-12-12 | Funai Electric Co., Ltd. | Image display device |
JP2016012889A (en) * | 2014-06-30 | 2016-01-21 | 株式会社リコー | Image projection system |
WO2016018929A1 (en) | 2014-07-28 | 2016-02-04 | Beam Authentic, LLC | Mountable display devices |
WO2016025853A1 (en) | 2014-08-15 | 2016-02-18 | Beam Authentic, LLC | Systems for displaying media on display devices |
US10235807B2 (en) | 2015-01-20 | 2019-03-19 | Microsoft Technology Licensing, Llc | Building holographic content using holographic tools |
US20170084084A1 (en) * | 2015-09-22 | 2017-03-23 | Thrillbox, Inc | Mapping of user interaction within a virtual reality environment |
CN106067967B (en) * | 2016-06-29 | 2018-02-09 | 汇意设计有限公司 | Deformable Volumetric hologram implementation method |
EP3319066A1 (en) | 2016-11-04 | 2018-05-09 | Lincoln Global, Inc. | Magnetic frequency selection for electromagnetic position tracking |
US10878591B2 (en) | 2016-11-07 | 2020-12-29 | Lincoln Global, Inc. | Welding trainer utilizing a head up display to display simulated and real-world objects |
US10913125B2 (en) | 2016-11-07 | 2021-02-09 | Lincoln Global, Inc. | Welding system providing visual and audio cues to a welding helmet with a display |
CN107193372B (en) * | 2017-05-15 | 2020-06-19 | 杭州一隅千象科技有限公司 | Projection method from multiple rectangular planes at arbitrary positions to variable projection center |
JP2018197844A (en) * | 2017-05-23 | 2018-12-13 | ヘ−ヨン・チョイ | Apparatus for spatially separated images |
US10373536B2 (en) | 2017-05-26 | 2019-08-06 | Jeffrey Sherretts | 3D signage using an inverse cube illusion fixture |
US10997872B2 (en) | 2017-06-01 | 2021-05-04 | Lincoln Global, Inc. | Spring-loaded tip assembly to support simulated shielded metal arc welding |
CN107333121B (en) * | 2017-06-27 | 2019-02-26 | 山东大学 | The immersion solid rendering optical projection system and its method of moving view point on curve screens |
US11475792B2 (en) | 2018-04-19 | 2022-10-18 | Lincoln Global, Inc. | Welding simulator with dual-user configuration |
US11557223B2 (en) | 2018-04-19 | 2023-01-17 | Lincoln Global, Inc. | Modular and reconfigurable chassis for simulated welding training |
CN110858464A (en) * | 2018-08-24 | 2020-03-03 | 财团法人工业技术研究院 | Multi-view display device and control simulator |
CN109407329B (en) * | 2018-11-06 | 2021-06-25 | 三亚中科遥感研究所 | Space light field display method and device |
US11113879B2 (en) | 2018-11-16 | 2021-09-07 | Geomni, Inc. | Systems and methods for generating augmented reality environments from two-dimensional drawings |
IL264032B (en) * | 2018-12-30 | 2020-06-30 | Elbit Systems Ltd | Systems and methods for reducing image artefacts in binocular displays |
US11367361B2 (en) * | 2019-02-22 | 2022-06-21 | Kyndryl, Inc. | Emulating unmanned aerial vehicle (UAV) |
US11508131B1 (en) * | 2019-11-08 | 2022-11-22 | Tanzle, Inc. | Generating composite stereoscopic images |
CN112929636A (en) * | 2019-12-05 | 2021-06-08 | 北京芯海视界三维科技有限公司 | 3D display device and 3D image display method |
Citations (94)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US1592034A (en) * | 1924-09-06 | 1926-07-13 | Macy Art Process Corp | Process and method of effective angular levitation of printed images and the resulting product |
US4182053A (en) * | 1977-09-14 | 1980-01-08 | Systems Technology, Inc. | Display generator for simulating vehicle operation |
US4291380A (en) * | 1979-05-14 | 1981-09-22 | The Singer Company | Resolvability test and projection size clipping for polygon face display |
US4677576A (en) * | 1983-06-27 | 1987-06-30 | Grumman Aerospace Corporation | Non-edge computer image generation system |
US4763280A (en) * | 1985-04-29 | 1988-08-09 | Evans & Sutherland Computer Corp. | Curvilinear dynamic image generation system |
US4795248A (en) * | 1984-08-31 | 1989-01-03 | Olympus Optical Company Ltd. | Liquid crystal eyeglass |
US4984179A (en) * | 1987-01-21 | 1991-01-08 | W. Industries Limited | Method and apparatus for the perception of computer-generated imagery |
US5079699A (en) * | 1987-11-27 | 1992-01-07 | Picker International, Inc. | Quick three-dimensional display |
US5276785A (en) * | 1990-08-02 | 1994-01-04 | Xerox Corporation | Moving viewpoint with respect to a target in a three-dimensional workspace |
US5287437A (en) * | 1992-06-02 | 1994-02-15 | Sun Microsystems, Inc. | Method and apparatus for head tracked display of precomputed stereo images |
US5327285A (en) * | 1990-06-11 | 1994-07-05 | Faris Sadeg M | Methods for manufacturing micropolarizers |
US5381158A (en) * | 1991-07-12 | 1995-01-10 | Kabushiki Kaisha Toshiba | Information retrieval apparatus |
US5381127A (en) * | 1993-12-22 | 1995-01-10 | Intel Corporation | Fast static cross-unit comparator |
US5400177A (en) * | 1993-11-23 | 1995-03-21 | Petitto; Tony | Technique for depth of field viewing of images with improved clarity and contrast |
US5438623A (en) * | 1993-10-04 | 1995-08-01 | The United States Of America As Represented By The Administrator Of National Aeronautics And Space Administration | Multi-channel spatialization system for audio signals |
US5515079A (en) * | 1989-11-07 | 1996-05-07 | Proxima Corporation | Computer input system and method of using same |
US5537144A (en) * | 1990-06-11 | 1996-07-16 | Revfo, Inc. | Electro-optical display system for visually displaying polarized spatially multiplexed images of 3-D objects for use in stereoscopically viewing the same with high image quality and resolution |
US5652617A (en) * | 1995-06-06 | 1997-07-29 | Barbour; Joel | Side scan down hole video tool having two camera |
US5745164A (en) * | 1993-11-12 | 1998-04-28 | Reveo, Inc. | System and method for electro-optically producing and displaying spectrally-multiplexed images of three-dimensional imagery for use in stereoscopic viewing thereof |
US5795154A (en) * | 1995-07-07 | 1998-08-18 | Woods; Gail Marjorie | Anaglyphic drawing device |
US5862229A (en) * | 1996-06-12 | 1999-01-19 | Nintendo Co., Ltd. | Sound generator synchronized with image display |
US5880733A (en) * | 1996-04-30 | 1999-03-09 | Microsoft Corporation | Display system and method for displaying windows of an operating system to provide a three-dimensional workspace for a computer system |
US5945985A (en) * | 1992-10-27 | 1999-08-31 | Technology International, Inc. | Information system for interactive access to geographic information |
US5956046A (en) * | 1997-12-17 | 1999-09-21 | Sun Microsystems, Inc. | Scene synchronization of multiple computer displays |
US6028593A (en) * | 1995-12-01 | 2000-02-22 | Immersion Corporation | Method and apparatus for providing simulated physical interactions within computer generated environments |
US6034717A (en) * | 1993-09-23 | 2000-03-07 | Reveo, Inc. | Projection display system for viewing displayed imagery over a wide field of view |
US6064354A (en) * | 1998-07-01 | 2000-05-16 | Deluca; Michael Joseph | Stereoscopic user interface method and apparatus |
US6069649A (en) * | 1994-08-05 | 2000-05-30 | Hattori; Tomohiko | Stereoscopic display |
US6072495A (en) * | 1997-04-21 | 2000-06-06 | Doryokuro Kakunenryo Kaihatsu Jigyodan | Object search method and object search system |
US6100903A (en) * | 1996-08-16 | 2000-08-08 | Goettsche; Mark T | Method for generating an ellipse with texture and perspective |
US6108005A (en) * | 1996-08-30 | 2000-08-22 | Space Corporation | Method for producing a synthesized stereoscopic image |
US6139434A (en) * | 1996-09-24 | 2000-10-31 | Nintendo Co., Ltd. | Three-dimensional image processing apparatus with enhanced automatic and user point of view control |
US6195205B1 (en) * | 1991-12-18 | 2001-02-27 | Reveo, Inc. | Multi-mode stereoscopic imaging system |
US6198524B1 (en) * | 1999-04-19 | 2001-03-06 | Evergreen Innovations Llc | Polarizing system for motion visual depth effects |
US6208346B1 (en) * | 1996-09-18 | 2001-03-27 | Fujitsu Limited | Attribute information presenting apparatus and multimedia system |
US6211848B1 (en) * | 1998-05-15 | 2001-04-03 | Massachusetts Institute Of Technology | Dynamic holographic video with haptic interaction |
US6226008B1 (en) * | 1997-09-04 | 2001-05-01 | Kabushiki Kaisha Sega Enterprises | Image processing device |
US6241609B1 (en) * | 1998-01-09 | 2001-06-05 | U.S. Philips Corporation | Virtual environment viewpoint control |
US6252707B1 (en) * | 1996-01-22 | 2001-06-26 | 3Ality, Inc. | Systems for three-dimensional viewing and projection |
US6346938B1 (en) * | 1999-04-27 | 2002-02-12 | Harris Corporation | Computer-resident mechanism for manipulating, navigating through and mensurating displayed image of three-dimensional geometric model |
US6351280B1 (en) * | 1998-11-20 | 2002-02-26 | Massachusetts Institute Of Technology | Autostereoscopic display system |
US20020041327A1 (en) * | 2000-07-24 | 2002-04-11 | Evan Hildreth | Video-based image control system |
US6373482B1 (en) * | 1998-12-23 | 2002-04-16 | Microsoft Corporation | Method, system, and computer program product for modified blending between clip-map tiles |
US6392689B1 (en) * | 1991-02-21 | 2002-05-21 | Eugene Dolgoff | System for displaying moving images pseudostereoscopically |
US20020080094A1 (en) * | 2000-12-22 | 2002-06-27 | Frank Biocca | Teleportal face-to-face system |
US6417867B1 (en) * | 1999-05-27 | 2002-07-09 | Sharp Laboratories Of America, Inc. | Image downscaling using peripheral vision area localization |
US20020113752A1 (en) * | 1998-04-20 | 2002-08-22 | Alan Sullivan | Multi-planar volumetric display system and method of operation using psychological vision cues |
US6452593B1 (en) * | 1999-02-19 | 2002-09-17 | International Business Machines Corporation | Method and system for rendering a virtual three-dimensional graphical display |
US20020140698A1 (en) * | 2001-03-29 | 2002-10-03 | Robertson George G. | 3D navigation techniques |
US20030006943A1 (en) * | 2000-02-07 | 2003-01-09 | Seiji Sato | Multiple-screen simultaneous displaying apparatus, multi-screen simultaneous displaying method, video signal generating device, and recorded medium |
US20030011535A1 (en) * | 2001-06-27 | 2003-01-16 | Tohru Kikuchi | Image display device, image displaying method, information storage medium, and image display program |
US6529210B1 (en) * | 1998-04-08 | 2003-03-04 | Altor Systems, Inc. | Indirect object manipulation in a simulation |
US6556197B1 (en) * | 1995-11-22 | 2003-04-29 | Nintendo Co., Ltd. | High performance low cost video game system with coprocessor providing high speed efficient 3D graphics and digital audio signal processing |
US20030085896A1 (en) * | 2001-11-07 | 2003-05-08 | Freeman Kyle G. | Method for rendering realistic terrain simulation |
US20030085866A1 (en) * | 2000-06-06 | 2003-05-08 | Oliver Bimber | Extended virtual table: an optical extension for table-like projection systems |
US6593934B1 (en) * | 2000-11-16 | 2003-07-15 | Industrial Technology Research Institute | Automatic gamma correction system for displays |
US6614427B1 (en) * | 1999-02-01 | 2003-09-02 | Steve Aubrey | Process for making stereoscopic images which are congruent with viewer space |
US6618049B1 (en) * | 1999-11-30 | 2003-09-09 | Silicon Graphics, Inc. | Method and apparatus for preparing a perspective view of an approximately spherical surface portion |
US6680735B1 (en) * | 2000-10-04 | 2004-01-20 | Terarecon, Inc. | Method for correcting gradients of irregular spaced graphic data |
US6690337B1 (en) * | 1999-06-09 | 2004-02-10 | Panoram Technologies, Inc. | Multi-panel video display |
US20040037459A1 (en) * | 2000-10-27 | 2004-02-26 | Dodge Alexandre Percival | Image processing apparatus |
US6715620B2 (en) * | 2001-10-05 | 2004-04-06 | Martin Taschek | Display frame for album covers |
US20040066384A1 (en) * | 2002-09-06 | 2004-04-08 | Sony Computer Entertainment Inc. | Image processing method and apparatus |
US20040066376A1 (en) * | 2000-07-18 | 2004-04-08 | Max Donath | Mobility assist device |
US20040125103A1 (en) * | 2000-02-25 | 2004-07-01 | Kaufman Arie E. | Apparatus and method for volume processing and rendering |
US20040130525A1 (en) * | 2002-11-19 | 2004-07-08 | Suchocki Edward J. | Dynamic touch screen amusement game controller |
US20040135744A1 (en) * | 2001-08-10 | 2004-07-15 | Oliver Bimber | Virtual showcases |
US20040135780A1 (en) * | 2002-08-30 | 2004-07-15 | Nims Jerry C. | Multi-dimensional images system for digital image input and output |
US20040164956A1 (en) * | 2003-02-26 | 2004-08-26 | Kosuke Yamaguchi | Three-dimensional object manipulating apparatus, method and computer program |
US20040169649A1 (en) * | 2000-12-11 | 2004-09-02 | Namco Ltd. | Method, apparatus, storage medium, program, and program product for generating image data of virtual three-dimensional space |
US20040196359A1 (en) * | 2002-05-28 | 2004-10-07 | Blackham Geoffrey Howard | Video conferencing terminal apparatus with part-transmissive curved mirror |
US20040208358A1 (en) * | 2002-11-12 | 2004-10-21 | Namco Ltd. | Image generation system, image generation method, program, and information storage medium |
US20050024331A1 (en) * | 2003-03-26 | 2005-02-03 | Mimic Technologies, Inc. | Method, apparatus, and article for force feedback based on tension control and tracking through cables |
US20050030308A1 (en) * | 2001-11-02 | 2005-02-10 | Yasuhiro Takaki | Three-dimensional display method and device therefor |
US20050057579A1 (en) * | 2003-07-21 | 2005-03-17 | Young Mark J. | Adaptive manipulators |
US20050093876A1 (en) * | 2002-06-28 | 2005-05-05 | Microsoft Corporation | Systems and methods for providing image rendering using variable rate source sampling |
US20050093859A1 (en) * | 2003-11-04 | 2005-05-05 | Siemens Medical Solutions Usa, Inc. | Viewing direction dependent acquisition or processing for 3D ultrasound imaging |
US6898307B1 (en) * | 1999-09-22 | 2005-05-24 | Xerox Corporation | Object identification method and system for an augmented-reality display |
US20050151742A1 (en) * | 2003-12-19 | 2005-07-14 | Palo Alto Research Center, Incorporated | Systems and method for turning pages in a three-dimensional electronic document |
US20050156881A1 (en) * | 2002-04-11 | 2005-07-21 | Synaptics, Inc. | Closed-loop sensor on a solid-state object position detector |
US20050162447A1 (en) * | 2004-01-28 | 2005-07-28 | Tigges Mark H.A. | Dynamic width adjustment for detail-in-context lenses |
US6943754B2 (en) * | 2002-09-27 | 2005-09-13 | The Boeing Company | Gaze tracking system, eye-tracking assembly and an associated method of calibration |
US20050219695A1 (en) * | 2004-04-05 | 2005-10-06 | Vesely Michael A | Horizontal perspective display |
US20050219240A1 (en) * | 2004-04-05 | 2005-10-06 | Vesely Michael A | Horizontal perspective hands-on simulator |
US20050219693A1 (en) * | 2004-04-02 | 2005-10-06 | David Hartkop | Scanning aperture three dimensional display device |
US6956576B1 (en) * | 2000-05-16 | 2005-10-18 | Sun Microsystems, Inc. | Graphics system using sample masks for motion blur, depth of field, and transparency |
US20050231532A1 (en) * | 2004-03-31 | 2005-10-20 | Canon Kabushiki Kaisha | Image processing method and image processing apparatus |
US20060126927A1 (en) * | 2004-11-30 | 2006-06-15 | Vesely Michael A | Horizontal perspective representation |
US20060170652A1 (en) * | 2005-01-31 | 2006-08-03 | Canon Kabushiki Kaisha | System, image processing apparatus, and information processing method |
US7102635B2 (en) * | 1998-07-17 | 2006-09-05 | Sensable Technologies, Inc. | Systems and methods for sculpting virtual objects in a haptic virtual reality environment |
US20070035511A1 (en) * | 2005-01-25 | 2007-02-15 | The Board Of Trustees Of The University Of Illinois. | Compact haptic and augmented virtual reality system |
US20070043466A1 (en) * | 2005-08-18 | 2007-02-22 | Vesely Michael A | Stereoscopic display using polarized eyewear |
US20070040905A1 (en) * | 2005-08-18 | 2007-02-22 | Vesely Michael A | Stereoscopic display using polarized eyewear |
US20070109296A1 (en) * | 2002-07-19 | 2007-05-17 | Canon Kabushiki Kaisha | Virtual space rendering/display apparatus and virtual space rendering/display method |
Family Cites Families (37)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5361386A (en) | 1987-12-04 | 1994-11-01 | Evans & Sutherland Computer Corp. | System for polygon interpolation using instantaneous values in a variable |
US4853592A (en) * | 1988-03-10 | 1989-08-01 | Rockwell International Corporation | Flat panel display having pixel spacing and luminance levels providing high resolution |
US5168531A (en) | 1991-06-27 | 1992-12-01 | Digital Equipment Corporation | Real-time recognition of pointing information from video |
JPH07325934A (en) | 1992-07-10 | 1995-12-12 | Walt Disney Co:The | Method and equipment for provision of graphics enhanced to virtual world |
US5574835A (en) * | 1993-04-06 | 1996-11-12 | Silicon Engines, Inc. | Bounding box and projections detection of hidden polygons in three-dimensional spatial databases |
GB2288523B (en) * | 1993-09-28 | 1998-04-01 | Namco Ltd | Clipping processing device, three-dimensional simulator device, and clipping processing method |
US5686975A (en) | 1993-10-18 | 1997-11-11 | Stereographics Corporation | Polarel panel for stereoscopic displays |
US5684460A (en) * | 1994-04-22 | 1997-11-04 | The United States Of America As Represented By The Secretary Of The Army | Motion and sound monitor and stimulator |
AUPN003894A0 (en) | 1994-12-13 | 1995-01-12 | Xenotech Research Pty Ltd | Head tracking system for stereoscopic display apparatus |
US5574836A (en) | 1996-01-22 | 1996-11-12 | Broemmelsiek; Raymond M. | Interactive display apparatus and method with viewer position compensation |
US6317127B1 (en) | 1996-10-16 | 2001-11-13 | Hughes Electronics Corporation | Multi-user real-time augmented reality system and method |
US6115022A (en) * | 1996-12-10 | 2000-09-05 | Metavision Corporation | Method and apparatus for adjusting multiple projected raster images |
DE59805342D1 (en) * | 1997-10-30 | 2002-10-02 | Baldeweg Ag Dr | METHOD AND DEVICE FOR PROCESSING IMAGE OBJECTS |
US6047201A (en) * | 1998-04-02 | 2000-04-04 | Jackson, Iii; William H. | Infant blood oxygen monitor and SIDS warning device |
US20020163482A1 (en) | 1998-04-20 | 2002-11-07 | Alan Sullivan | Multi-planar volumetric display system including optical elements made from liquid crystal having polymer stabilized cholesteric textures |
US6215403B1 (en) * | 1999-01-27 | 2001-04-10 | International Business Machines Corporation | Wireless monitoring system |
US6593924B1 (en) * | 1999-10-04 | 2003-07-15 | Intel Corporation | Rendering a non-photorealistic image |
US6669346B2 (en) * | 2000-05-15 | 2003-12-30 | Darrell J. Metcalf | Large-audience, positionable imaging and display system for exhibiting panoramic imagery, and multimedia content featuring a circularity of action |
US6643124B1 (en) * | 2000-08-09 | 2003-11-04 | Peter J. Wilk | Multiple display portable computing devices |
US6553256B1 (en) * | 2000-10-13 | 2003-04-22 | Koninklijke Philips Electronics N.V. | Method and apparatus for monitoring and treating sudden infant death syndrome |
US20020180727A1 (en) * | 2000-11-22 | 2002-12-05 | Guckenberger Ronald James | Shadow buffer control module method and software construct for adjusting per pixel raster images attributes to screen space and projector features for digital warp, intensity transforms, color matching, soft-edge blending, and filtering for multiple projectors and laser projectors |
GB2375699B (en) * | 2001-05-16 | 2003-08-13 | Nibble Ltd | Information management system and method |
JP4722379B2 (en) | 2001-05-22 | 2011-07-13 | シエフイ,ヨアブ | Method and system for displaying visible content in a virtual 3D space |
US7259747B2 (en) | 2001-06-05 | 2007-08-21 | Reactrix Systems, Inc. | Interactive video display system |
TW579019U (en) | 2001-06-13 | 2004-03-01 | Eturbotouch Technology Inc | Flexible current type touch film |
US6478432B1 (en) | 2001-07-13 | 2002-11-12 | Chad D. Dyner | Dynamically generated interactive real imaging device |
CA2386702A1 (en) | 2002-05-17 | 2003-11-17 | Idelix Software Inc. | Computing the inverse of a pdt distortion |
US7190331B2 (en) * | 2002-06-06 | 2007-03-13 | Siemens Corporate Research, Inc. | System and method for measuring the registration accuracy of an augmented reality system |
JP2006513503A (en) | 2002-11-29 | 2006-04-20 | ブラッコ イメージング ソチエタ ペル アチオニ | Apparatus and method for managing a plurality of locations in a three-dimensional display |
US7495638B2 (en) | 2003-05-13 | 2009-02-24 | Research Triangle Institute | Visual display with increased field of view |
US20050248566A1 (en) | 2004-04-05 | 2005-11-10 | Vesely Michael A | Horizontal perspective hands-on simulator |
US7796134B2 (en) | 2004-06-01 | 2010-09-14 | Infinite Z, Inc. | Multi-plane horizontal perspective display |
US20060250390A1 (en) | 2005-04-04 | 2006-11-09 | Vesely Michael A | Horizontal perspective display |
US20060221071A1 (en) | 2005-04-04 | 2006-10-05 | Vesely Michael A | Horizontal perspective display |
JP4738870B2 (en) | 2005-04-08 | 2011-08-03 | キヤノン株式会社 | Information processing method, information processing apparatus, and remote mixed reality sharing apparatus |
WO2006121956A1 (en) | 2005-05-09 | 2006-11-16 | Infinite Z, Inc. | Biofeedback eyewear system |
WO2006121957A2 (en) | 2005-05-09 | 2006-11-16 | Michael Vesely | Three dimensional horizontal perspective workstation |
-
2005
- 2005-05-31 US US11/141,649 patent/US7796134B2/en active Active
- 2005-05-31 KR KR1020077000038A patent/KR20070052261A/en not_active Application Discontinuation
- 2005-05-31 WO PCT/US2005/019068 patent/WO2005119376A2/en active Application Filing
- 2005-05-31 US US11/141,650 patent/US20050275915A1/en not_active Abandoned
- 2005-05-31 US US11/141,651 patent/US20050264558A1/en not_active Abandoned
- 2005-05-31 EP EP05758077A patent/EP1759379A2/en not_active Withdrawn
- 2005-05-31 US US11/141,541 patent/US20050275914A1/en not_active Abandoned
- 2005-05-31 WO PCT/US2005/019069 patent/WO2005118998A1/en active Application Filing
- 2005-05-31 KR KR1020077000037A patent/KR20070052260A/en not_active Application Discontinuation
- 2005-05-31 US US11/141,540 patent/US20050275913A1/en not_active Abandoned
- 2005-05-31 US US11/141,652 patent/US20050264559A1/en not_active Abandoned
- 2005-05-31 JP JP2007515486A patent/JP2008507006A/en active Pending
- 2005-05-31 US US11/141,606 patent/US20050264857A1/en not_active Abandoned
- 2005-05-31 EP EP05757162A patent/EP1781893A1/en not_active Withdrawn
- 2005-05-31 US US11/141,640 patent/US20050281411A1/en not_active Abandoned
- 2005-05-31 JP JP2007515485A patent/JP2008506140A/en active Pending
Patent Citations (99)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US1592034A (en) * | 1924-09-06 | 1926-07-13 | Macy Art Process Corp | Process and method of effective angular levitation of printed images and the resulting product |
US4182053A (en) * | 1977-09-14 | 1980-01-08 | Systems Technology, Inc. | Display generator for simulating vehicle operation |
US4291380A (en) * | 1979-05-14 | 1981-09-22 | The Singer Company | Resolvability test and projection size clipping for polygon face display |
US4677576A (en) * | 1983-06-27 | 1987-06-30 | Grumman Aerospace Corporation | Non-edge computer image generation system |
US4795248A (en) * | 1984-08-31 | 1989-01-03 | Olympus Optical Company Ltd. | Liquid crystal eyeglass |
US4763280A (en) * | 1985-04-29 | 1988-08-09 | Evans & Sutherland Computer Corp. | Curvilinear dynamic image generation system |
US4984179A (en) * | 1987-01-21 | 1991-01-08 | W. Industries Limited | Method and apparatus for the perception of computer-generated imagery |
US5079699A (en) * | 1987-11-27 | 1992-01-07 | Picker International, Inc. | Quick three-dimensional display |
US5515079A (en) * | 1989-11-07 | 1996-05-07 | Proxima Corporation | Computer input system and method of using same |
US5327285A (en) * | 1990-06-11 | 1994-07-05 | Faris Sadeg M | Methods for manufacturing micropolarizers |
US6384971B1 (en) * | 1990-06-11 | 2002-05-07 | Reveo, Inc. | Methods for manufacturing micropolarizers |
US5537144A (en) * | 1990-06-11 | 1996-07-16 | Revfo, Inc. | Electro-optical display system for visually displaying polarized spatially multiplexed images of 3-D objects for use in stereoscopically viewing the same with high image quality and resolution |
US5276785A (en) * | 1990-08-02 | 1994-01-04 | Xerox Corporation | Moving viewpoint with respect to a target in a three-dimensional workspace |
US6392689B1 (en) * | 1991-02-21 | 2002-05-21 | Eugene Dolgoff | System for displaying moving images pseudostereoscopically |
US5381158A (en) * | 1991-07-12 | 1995-01-10 | Kabushiki Kaisha Toshiba | Information retrieval apparatus |
US6195205B1 (en) * | 1991-12-18 | 2001-02-27 | Reveo, Inc. | Multi-mode stereoscopic imaging system |
US5287437A (en) * | 1992-06-02 | 1994-02-15 | Sun Microsystems, Inc. | Method and apparatus for head tracked display of precomputed stereo images |
US5945985A (en) * | 1992-10-27 | 1999-08-31 | Technology International, Inc. | Information system for interactive access to geographic information |
US6034717A (en) * | 1993-09-23 | 2000-03-07 | Reveo, Inc. | Projection display system for viewing displayed imagery over a wide field of view |
US5438623A (en) * | 1993-10-04 | 1995-08-01 | The United States Of America As Represented By The Administrator Of National Aeronautics And Space Administration | Multi-channel spatialization system for audio signals |
US5745164A (en) * | 1993-11-12 | 1998-04-28 | Reveo, Inc. | System and method for electro-optically producing and displaying spectrally-multiplexed images of three-dimensional imagery for use in stereoscopic viewing thereof |
US5400177A (en) * | 1993-11-23 | 1995-03-21 | Petitto; Tony | Technique for depth of field viewing of images with improved clarity and contrast |
US5381127A (en) * | 1993-12-22 | 1995-01-10 | Intel Corporation | Fast static cross-unit comparator |
US6069649A (en) * | 1994-08-05 | 2000-05-30 | Hattori; Tomohiko | Stereoscopic display |
US5652617A (en) * | 1995-06-06 | 1997-07-29 | Barbour; Joel | Side scan down hole video tool having two camera |
US5795154A (en) * | 1995-07-07 | 1998-08-18 | Woods; Gail Marjorie | Anaglyphic drawing device |
US6556197B1 (en) * | 1995-11-22 | 2003-04-29 | Nintendo Co., Ltd. | High performance low cost video game system with coprocessor providing high speed efficient 3D graphics and digital audio signal processing |
US6028593A (en) * | 1995-12-01 | 2000-02-22 | Immersion Corporation | Method and apparatus for providing simulated physical interactions within computer generated environments |
US6252707B1 (en) * | 1996-01-22 | 2001-06-26 | 3Ality, Inc. | Systems for three-dimensional viewing and projection |
US5880733A (en) * | 1996-04-30 | 1999-03-09 | Microsoft Corporation | Display system and method for displaying windows of an operating system to provide a three-dimensional workspace for a computer system |
US5862229A (en) * | 1996-06-12 | 1999-01-19 | Nintendo Co., Ltd. | Sound generator synchronized with image display |
US6100903A (en) * | 1996-08-16 | 2000-08-08 | Goettsche; Mark T | Method for generating an ellipse with texture and perspective |
US6108005A (en) * | 1996-08-30 | 2000-08-22 | Space Corporation | Method for producing a synthesized stereoscopic image |
US6208346B1 (en) * | 1996-09-18 | 2001-03-27 | Fujitsu Limited | Attribute information presenting apparatus and multimedia system |
US6139434A (en) * | 1996-09-24 | 2000-10-31 | Nintendo Co., Ltd. | Three-dimensional image processing apparatus with enhanced automatic and user point of view control |
US6072495A (en) * | 1997-04-21 | 2000-06-06 | Doryokuro Kakunenryo Kaihatsu Jigyodan | Object search method and object search system |
US6226008B1 (en) * | 1997-09-04 | 2001-05-01 | Kabushiki Kaisha Sega Enterprises | Image processing device |
US5956046A (en) * | 1997-12-17 | 1999-09-21 | Sun Microsystems, Inc. | Scene synchronization of multiple computer displays |
US6241609B1 (en) * | 1998-01-09 | 2001-06-05 | U.S. Philips Corporation | Virtual environment viewpoint control |
US6529210B1 (en) * | 1998-04-08 | 2003-03-04 | Altor Systems, Inc. | Indirect object manipulation in a simulation |
US20020113752A1 (en) * | 1998-04-20 | 2002-08-22 | Alan Sullivan | Multi-planar volumetric display system and method of operation using psychological vision cues |
US6211848B1 (en) * | 1998-05-15 | 2001-04-03 | Massachusetts Institute Of Technology | Dynamic holographic video with haptic interaction |
US6064354A (en) * | 1998-07-01 | 2000-05-16 | Deluca; Michael Joseph | Stereoscopic user interface method and apparatus |
US7102635B2 (en) * | 1998-07-17 | 2006-09-05 | Sensable Technologies, Inc. | Systems and methods for sculpting virtual objects in a haptic virtual reality environment |
US6351280B1 (en) * | 1998-11-20 | 2002-02-26 | Massachusetts Institute Of Technology | Autostereoscopic display system |
US6373482B1 (en) * | 1998-12-23 | 2002-04-16 | Microsoft Corporation | Method, system, and computer program product for modified blending between clip-map tiles |
US6614427B1 (en) * | 1999-02-01 | 2003-09-02 | Steve Aubrey | Process for making stereoscopic images which are congruent with viewer space |
US6452593B1 (en) * | 1999-02-19 | 2002-09-17 | International Business Machines Corporation | Method and system for rendering a virtual three-dimensional graphical display |
US6198524B1 (en) * | 1999-04-19 | 2001-03-06 | Evergreen Innovations Llc | Polarizing system for motion visual depth effects |
US6346938B1 (en) * | 1999-04-27 | 2002-02-12 | Harris Corporation | Computer-resident mechanism for manipulating, navigating through and mensurating displayed image of three-dimensional geometric model |
US6417867B1 (en) * | 1999-05-27 | 2002-07-09 | Sharp Laboratories Of America, Inc. | Image downscaling using peripheral vision area localization |
US6690337B1 (en) * | 1999-06-09 | 2004-02-10 | Panoram Technologies, Inc. | Multi-panel video display |
US6898307B1 (en) * | 1999-09-22 | 2005-05-24 | Xerox Corporation | Object identification method and system for an augmented-reality display |
US6618049B1 (en) * | 1999-11-30 | 2003-09-09 | Silicon Graphics, Inc. | Method and apparatus for preparing a perspective view of an approximately spherical surface portion |
US20030006943A1 (en) * | 2000-02-07 | 2003-01-09 | Seiji Sato | Multiple-screen simultaneous displaying apparatus, multi-screen simultaneous displaying method, video signal generating device, and recorded medium |
US20040125103A1 (en) * | 2000-02-25 | 2004-07-01 | Kaufman Arie E. | Apparatus and method for volume processing and rendering |
US6956576B1 (en) * | 2000-05-16 | 2005-10-18 | Sun Microsystems, Inc. | Graphics system using sample masks for motion blur, depth of field, and transparency |
US20030085866A1 (en) * | 2000-06-06 | 2003-05-08 | Oliver Bimber | Extended virtual table: an optical extension for table-like projection systems |
US20040066376A1 (en) * | 2000-07-18 | 2004-04-08 | Max Donath | Mobility assist device |
US20020041327A1 (en) * | 2000-07-24 | 2002-04-11 | Evan Hildreth | Video-based image control system |
US6680735B1 (en) * | 2000-10-04 | 2004-01-20 | Terarecon, Inc. | Method for correcting gradients of irregular spaced graphic data |
US20040037459A1 (en) * | 2000-10-27 | 2004-02-26 | Dodge Alexandre Percival | Image processing apparatus |
US6912490B2 (en) * | 2000-10-27 | 2005-06-28 | Canon Kabushiki Kaisha | Image processing apparatus |
US6593934B1 (en) * | 2000-11-16 | 2003-07-15 | Industrial Technology Research Institute | Automatic gamma correction system for displays |
US20040169649A1 (en) * | 2000-12-11 | 2004-09-02 | Namco Ltd. | Method, apparatus, storage medium, program, and program product for generating image data of virtual three-dimensional space |
US20020080094A1 (en) * | 2000-12-22 | 2002-06-27 | Frank Biocca | Teleportal face-to-face system |
US20020140698A1 (en) * | 2001-03-29 | 2002-10-03 | Robertson George G. | 3D navigation techniques |
US6987512B2 (en) * | 2001-03-29 | 2006-01-17 | Microsoft Corporation | 3D navigation techniques |
US20030011535A1 (en) * | 2001-06-27 | 2003-01-16 | Tohru Kikuchi | Image display device, image displaying method, information storage medium, and image display program |
US20040135744A1 (en) * | 2001-08-10 | 2004-07-15 | Oliver Bimber | Virtual showcases |
US6715620B2 (en) * | 2001-10-05 | 2004-04-06 | Martin Taschek | Display frame for album covers |
US20050030308A1 (en) * | 2001-11-02 | 2005-02-10 | Yasuhiro Takaki | Three-dimensional display method and device therefor |
US20030085896A1 (en) * | 2001-11-07 | 2003-05-08 | Freeman Kyle G. | Method for rendering realistic terrain simulation |
US20050156881A1 (en) * | 2002-04-11 | 2005-07-21 | Synaptics, Inc. | Closed-loop sensor on a solid-state object position detector |
US20040196359A1 (en) * | 2002-05-28 | 2004-10-07 | Blackham Geoffrey Howard | Video conferencing terminal apparatus with part-transmissive curved mirror |
US20050093876A1 (en) * | 2002-06-28 | 2005-05-05 | Microsoft Corporation | Systems and methods for providing image rendering using variable rate source sampling |
US20070109296A1 (en) * | 2002-07-19 | 2007-05-17 | Canon Kabushiki Kaisha | Virtual space rendering/display apparatus and virtual space rendering/display method |
US20040135780A1 (en) * | 2002-08-30 | 2004-07-15 | Nims Jerry C. | Multi-dimensional images system for digital image input and output |
US20040066384A1 (en) * | 2002-09-06 | 2004-04-08 | Sony Computer Entertainment Inc. | Image processing method and apparatus |
US6943754B2 (en) * | 2002-09-27 | 2005-09-13 | The Boeing Company | Gaze tracking system, eye-tracking assembly and an associated method of calibration |
US20040208358A1 (en) * | 2002-11-12 | 2004-10-21 | Namco Ltd. | Image generation system, image generation method, program, and information storage medium |
US20040130525A1 (en) * | 2002-11-19 | 2004-07-08 | Suchocki Edward J. | Dynamic touch screen amusement game controller |
US20040164956A1 (en) * | 2003-02-26 | 2004-08-26 | Kosuke Yamaguchi | Three-dimensional object manipulating apparatus, method and computer program |
US20050024331A1 (en) * | 2003-03-26 | 2005-02-03 | Mimic Technologies, Inc. | Method, apparatus, and article for force feedback based on tension control and tracking through cables |
US20050057579A1 (en) * | 2003-07-21 | 2005-03-17 | Young Mark J. | Adaptive manipulators |
US20050093859A1 (en) * | 2003-11-04 | 2005-05-05 | Siemens Medical Solutions Usa, Inc. | Viewing direction dependent acquisition or processing for 3D ultrasound imaging |
US20050151742A1 (en) * | 2003-12-19 | 2005-07-14 | Palo Alto Research Center, Incorporated | Systems and method for turning pages in a three-dimensional electronic document |
US20050162447A1 (en) * | 2004-01-28 | 2005-07-28 | Tigges Mark H.A. | Dynamic width adjustment for detail-in-context lenses |
US20050231532A1 (en) * | 2004-03-31 | 2005-10-20 | Canon Kabushiki Kaisha | Image processing method and image processing apparatus |
US20050219693A1 (en) * | 2004-04-02 | 2005-10-06 | David Hartkop | Scanning aperture three dimensional display device |
US20050219694A1 (en) * | 2004-04-05 | 2005-10-06 | Vesely Michael A | Horizontal perspective display |
US20050219240A1 (en) * | 2004-04-05 | 2005-10-06 | Vesely Michael A | Horizontal perspective hands-on simulator |
US20050219695A1 (en) * | 2004-04-05 | 2005-10-06 | Vesely Michael A | Horizontal perspective display |
US20060126927A1 (en) * | 2004-11-30 | 2006-06-15 | Vesely Michael A | Horizontal perspective representation |
US20060126926A1 (en) * | 2004-11-30 | 2006-06-15 | Vesely Michael A | Horizontal perspective representation |
US20070035511A1 (en) * | 2005-01-25 | 2007-02-15 | The Board Of Trustees Of The University Of Illinois. | Compact haptic and augmented virtual reality system |
US20060170652A1 (en) * | 2005-01-31 | 2006-08-03 | Canon Kabushiki Kaisha | System, image processing apparatus, and information processing method |
US20070043466A1 (en) * | 2005-08-18 | 2007-02-22 | Vesely Michael A | Stereoscopic display using polarized eyewear |
US20070040905A1 (en) * | 2005-08-18 | 2007-02-22 | Vesely Michael A | Stereoscopic display using polarized eyewear |
Cited By (33)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7796134B2 (en) | 2004-06-01 | 2010-09-14 | Infinite Z, Inc. | Multi-plane horizontal perspective display |
US9684994B2 (en) | 2005-05-09 | 2017-06-20 | Zspace, Inc. | Modifying perspective of stereoscopic images based on changes in user viewpoint |
US7907167B2 (en) | 2005-05-09 | 2011-03-15 | Infinite Z, Inc. | Three dimensional horizontal perspective workstation |
US9292962B2 (en) | 2005-05-09 | 2016-03-22 | Zspace, Inc. | Modifying perspective of stereoscopic images based on changes in user viewpoint |
US8717423B2 (en) | 2005-05-09 | 2014-05-06 | Zspace, Inc. | Modifying perspective of stereoscopic images based on changes in user viewpoint |
US20090125801A1 (en) * | 2007-11-10 | 2009-05-14 | Cherif Atia Algreatly | 3D windows system |
US20120022842A1 (en) * | 2009-02-11 | 2012-01-26 | Arkamys | Test platform implemented by a method for positioning a sound object in a 3d sound environment |
US8612187B2 (en) * | 2009-02-11 | 2013-12-17 | Arkamys | Test platform implemented by a method for positioning a sound object in a 3D sound environment |
US20100309390A1 (en) * | 2009-06-03 | 2010-12-09 | Honeywood Technologies, Llc | Multimedia projection management |
US8269902B2 (en) * | 2009-06-03 | 2012-09-18 | Transpacific Image, Llc | Multimedia projection management |
US20110187706A1 (en) * | 2010-01-29 | 2011-08-04 | Vesely Michael A | Presenting a View within a Three Dimensional Scene |
US9824485B2 (en) | 2010-01-29 | 2017-11-21 | Zspace, Inc. | Presenting a view within a three dimensional scene |
US9202306B2 (en) | 2010-01-29 | 2015-12-01 | Zspace, Inc. | Presenting a view within a three dimensional scene |
US8717360B2 (en) * | 2010-01-29 | 2014-05-06 | Zspace, Inc. | Presenting a view within a three dimensional scene |
US20110248987A1 (en) * | 2010-04-08 | 2011-10-13 | Disney Enterprises, Inc. | Interactive three dimensional displays on handheld devices |
US8581905B2 (en) * | 2010-04-08 | 2013-11-12 | Disney Enterprises, Inc. | Interactive three dimensional displays on handheld devices |
US9704299B2 (en) | 2010-04-08 | 2017-07-11 | Disney Enterprises, Inc. | Interactive three dimensional displays on handheld devices |
US20130069880A1 (en) * | 2010-04-13 | 2013-03-21 | Dean Stark | Virtual product display |
US9733699B2 (en) * | 2010-04-13 | 2017-08-15 | Dean Stark | Virtual anamorphic product display with viewer height detection |
US8315443B2 (en) | 2010-04-22 | 2012-11-20 | Qualcomm Incorporated | Viewpoint detector based on skin color area and face area |
US20120002828A1 (en) * | 2010-06-30 | 2012-01-05 | Sony Corporation | Audio processing device, audio processing method, and program |
US9351092B2 (en) * | 2010-06-30 | 2016-05-24 | Sony Corporation | Audio processing device, audio processing method, and program |
US20120148055A1 (en) * | 2010-12-13 | 2012-06-14 | Samsung Electronics Co., Ltd. | Audio processing apparatus, audio receiver and method for providing audio thereof |
US9134556B2 (en) | 2011-05-18 | 2015-09-15 | Zspace, Inc. | Liquid crystal variable drive voltage |
US8786529B1 (en) | 2011-05-18 | 2014-07-22 | Zspace, Inc. | Liquid crystal variable drive voltage |
US9958712B2 (en) | 2011-05-18 | 2018-05-01 | Zspace, Inc. | Liquid crystal variable drive voltage |
US9492122B2 (en) | 2011-06-28 | 2016-11-15 | Kabushiki Kaisha Toshiba | Medical image processing apparatus |
US20130342536A1 (en) * | 2012-06-22 | 2013-12-26 | Canon Kabushiki Kaisha | Image processing apparatus, method of controlling the same and computer-readable medium |
US9265458B2 (en) | 2012-12-04 | 2016-02-23 | Sync-Think, Inc. | Application of smooth pursuit cognitive testing paradigms to clinical drug development |
US9380976B2 (en) | 2013-03-11 | 2016-07-05 | Sync-Think, Inc. | Optical neuroinformatics |
US10503457B2 (en) | 2017-05-05 | 2019-12-10 | Nvidia Corporation | Method and apparatus for rendering perspective-correct images for a tilted multi-display environment |
US10503456B2 (en) | 2017-05-05 | 2019-12-10 | Nvidia Corporation | Method and apparatus for rendering perspective-correct images for a tilted multi-display environment |
US11170563B2 (en) * | 2018-01-04 | 2021-11-09 | 8259402 Canada Inc. | Immersive environment with digital environment to enhance depth sensation |
Also Published As
Publication number | Publication date |
---|---|
US7796134B2 (en) | 2010-09-14 |
US20050264858A1 (en) | 2005-12-01 |
KR20070052260A (en) | 2007-05-21 |
EP1781893A1 (en) | 2007-05-09 |
US20050264558A1 (en) | 2005-12-01 |
JP2008506140A (en) | 2008-02-28 |
WO2005118998A8 (en) | 2006-04-27 |
US20050275913A1 (en) | 2005-12-15 |
US20050275915A1 (en) | 2005-12-15 |
US20050275914A1 (en) | 2005-12-15 |
US20050264559A1 (en) | 2005-12-01 |
EP1759379A2 (en) | 2007-03-07 |
WO2005119376A2 (en) | 2005-12-15 |
WO2005119376A3 (en) | 2006-04-27 |
US20050281411A1 (en) | 2005-12-22 |
KR20070052261A (en) | 2007-05-21 |
WO2005118998A1 (en) | 2005-12-15 |
JP2008507006A (en) | 2008-03-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20050264857A1 (en) | Binaural horizontal perspective display | |
US9684994B2 (en) | Modifying perspective of stereoscopic images based on changes in user viewpoint | |
US7907167B2 (en) | Three dimensional horizontal perspective workstation | |
CN101006492A (en) | Hrizontal perspective display | |
US20050219240A1 (en) | Horizontal perspective hands-on simulator | |
US20060126927A1 (en) | Horizontal perspective representation | |
de Bruijn | Application of wave field synthesis in videoconferencing | |
EP1740998A2 (en) | Horizontal perspective hand-on simulator | |
US20050248566A1 (en) | Horizontal perspective hands-on simulator | |
EP3595337A1 (en) | Audio apparatus and method of audio processing | |
JP4955718B2 (en) | Stereoscopic display control apparatus, stereoscopic display system, and stereoscopic display control method | |
JP7402185B2 (en) | Low frequency interchannel coherence control | |
NL2030186B1 (en) | Autostereoscopic display device presenting 3d-view and 3d-sound | |
KR20190064394A (en) | 360 degree VR partition circle vision display apparatus and method thereof | |
Richie | Audio and visual distance perception of familiar and unfamiliar objects using Wave Field Synthesis and a stereoscopic display | |
Simon et al. | Sonic interaction with a virtual orchestra of factory machinery |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: INFINITE Z, LLC, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:VESELY, MICHAEL A.;CLEMENS, NANCY L.;REEL/FRAME:019456/0438 Effective date: 20060317 |
|
AS | Assignment |
Owner name: INFINITE Z, INC., CALIFORNIA Free format text: CHANGE OF NAME;ASSIGNOR:INFINITE Z, LLC;REEL/FRAME:019464/0909 Effective date: 20061026 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |