US20030229735A1 - Navigable telepresence method and system utilizing an array of cameras - Google Patents
Navigable telepresence method and system utilizing an array of cameras Download PDFInfo
- Publication number
- US20030229735A1 US20030229735A1 US10/391,509 US39150903A US2003229735A1 US 20030229735 A1 US20030229735 A1 US 20030229735A1 US 39150903 A US39150903 A US 39150903A US 2003229735 A1 US2003229735 A1 US 2003229735A1
- Authority
- US
- United States
- Prior art keywords
- user
- camera
- image
- array
- processing element
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/2625—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect
- H04N5/2627—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect for providing spin image effect, 3D stop motion effect or temporal freeze effect
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Studio Devices (AREA)
- Closed-Circuit Television Systems (AREA)
- Navigation (AREA)
Abstract
A telepresence system for providing a first user with a first display of an environment and a second user with a second display of the environment includes an array of cameras. Each camera has an associated view of the environment and an associated camera output representing the associated view. The array also includes at least one camera path. A first user interface device has first user inputs associated with movement along a first path in the array, and a second user interface device has second user inputs associated with along a second path in the array. The user interface devices are coupled to at least one processing element that receives and interprets the user inputs. The processing element interprets the first inputs and selects outputs of cameras in the first path, and interprets second inputs and selects outputs of cameras in the second path independently of the first inputs, thereby allowing a first user and a second user to navigate simultaneously and independently through the array. In another embodiment, a telepresence system includes at least one processing element that distinguishes between permissible cameras in the array and impermissible cameras in the array.
Description
- This U.S. Non-Provisional Application claims the benefit of U.S. Provisional Application Serial No. 60/080,413, filed on Apr. 2, 1998, herein incorporated by reference.
- 1. Field of the Invention
- The present invention relates to a telepresence system and, more particularly, to a navigable camera array telepresence system and method of using same.
- 2. Description of Related Art
- In general, a need exists for the development of telepresence systems suitable for use with static venues, such as museums, and dynamic venues or events, such as a music concerts. The viewing of such venues is limited by time, geographical location, and the viewer capacity of the venue. For example, potential visitors to a museum may be prevented from viewing an exhibit due to the limited hours the museum is open. Similarly, music concert producers must turn back fans due to the limited seating of an arena. In short, limited access to venues reduces the revenue generated.
- In an attempt to increase the revenue stream from both static and dynamic venues, such venues have been recorded for broadcast or distribution. In some instances, dynamic venues are also broadcast live. While such broadcasting increases access to the venues, it involves considerable production effort. Typically, recorded broadcasts must be cut and edited, as views from multiple cameras are pieced together. These editorial and production efforts are costly.
- In some instances, the broadcast resulting from these editorial and production efforts provides viewers with limited enjoyment. Specifically, the broadcast is typically based on filming the venue from a finite number of predetermined cameras. Thus, the broadcast contains limited viewing angles and perspectives of the venue. Moreover, the viewing angles and perspectives presented in the broadcast are those selected by a producer or director during the editorial and production process; there is no viewer autonomy. Furthermore, although the broadcast is often recorded for multiple viewings, the broadcast has limited content life because each viewing is identical to the first. Because each showing looks and sounds the same, viewers rarely come back for multiple viewings.
- A viewer fortunate enough to attend a venue in person will encounter many of the same problems. For example, a museum-goer must remain behind the barricades, viewing exhibits from limited angles and perspectives. Similarly, concert-goers are often restricted to a particular seat or section in an arena. Even if a viewer were allowed free access to the entire arena to videotape the venue, such a recording would also have limited content life because each viewing would be the same as the first. Therefore, a need exists for a telepresence system that preferably provides user autonomy while resulting in recordings with enhanced content life at a reduced production cost.
- Apparently, attempts have been made to develop telepresence systems to satisfy some of the foregoing needs. One telepresence system is described in U.S. Pat. No. 5,708,469 for Multiple View Telepresence Camera Systems Using A Wire Cage Which Surrounds A Polarity Of Multiple Cameras And Identifies The Fields Of View, issued Jan. 13, 1998. The system disclosed therein includes a plurality of cameras, wherein each camera has a field of view that is space-contiguous with and at a right angle to at least one other camera. In other words, it is preferable that the camera fields of view do not overlap each other. A user interface allows the user to jump between views. In order for the user's view to move through the venue or environment, a moving vehicle carries the cameras.
- This system, however, has several drawbacks. For example, in order for a viewer's perspective to move through the venue, the moving vehicle must be actuated and controlled. In this regard, operation of the system is complicated. Furthermore, because the camera views are contiguous, typically at right angles, changing camera views results in a discontinuous image.
- Other attempts at providing a telepresence system have taken the form of a 360 degree camera systems. One such system is described in U.S. Pat. No. 5,745,305 for Panoramic Viewing Apparatus, issued Apr. 28, 1998. The system described therein provides a 360 degree view of environment by arranging multiple cameras around a pyramid shaped reflective element. Each camera, all of which share a common virtual optical center, receives an image from a different side of the reflective pyramid. Other types of 360 degree camera systems employ a parabolic lens or a rotating camera.
- Such 360 degree camera systems also suffer from drawbacks. In particular, such systems limit the user's view to 360 degrees from a given point perspective. In other words, 360 degree camera systems provide the user with a panoramic view from a single location. Only if the camera system was mounted on a moving vehicle could the user experience simulated movement through an environment.
- U.S. Pat. No. 5,187,571 for Television System For Displaying Multiple Views of A Remote Location issued Feb. 16, 1993, describes a camera system similar to the 360 degree camera systems described above. The system described provides a user to select an arbitrary and continuously variable section of an aggregate field of view. Multiple cameras are aligned so that each camera's field of view merges contiguously with those of adjacent cameras thereby creating the aggregate field of view. The aggregate field of view may expand to cover 360 degrees. In order to create the aggregate field of view, the cameras' views must be contiguous. In order for the camera views to be contiguous, the cameras have to share a common point perspective, or vertex. Thus, like the previously described 360 degree camera systems, the system of U.S. Pat. No. 5,187,571 limits a user's view to a single point perspective, rather than allowing a user to experience movement in perspective through an environment.
- Also, with regard to the system of U.S. Pat. No. 5,187,571, in order to achieve the contiguity between camera views, a relatively complex arrangement of mirrors is required. Additionally, each camera seemingly must also be placed in the same vertical plane.
- Thus, a need still exists for an improved telepresence system that provides the ability to better simulate a viewer's actual presence in a venue, preferably in real time.
- These and other needs are satisfied by the present invention. A telepresence system according to one embodiment of the present invention includes an array of cameras, each of which has an associated view of an environment and an associated output representing the view. The system also includes a first user interface device having first user inputs associated with movement along a first path in the array. The system further includes a second user interface device having second user inputs associated with movement along a second path in the array. A processing element is coupled to the user interface devices. The processing the element receives and interprets the first inputs and selects outputs of cameras in the first path. Similarly, the processing element receives and interprets the second inputs and selects outputs of cameras in the second path independently of the first inputs. Thus, a first user and a second user are able to navigate simultaneously and independently through the array. In another embodiment of the present invention the telepresence system distinguishes between permissible cameras in the array and impermissible cameras in the array.
- FIG. 1 is an overall schematic of one embodiment of the present invention.
- FIG. 2a is a perspective view of a camera and a camera rail section of the array according to one embodiment of the present invention.
- FIGS. 2b-2 d are side plan views of a camera and a camera rail according to one embodiment of the present invention.
- FIG. 2e is a top plan view of a camera rail according to one embodiment of the present invention.
- FIG. 3 is a perspective view of a portion of the camera array according to one embodiment of the present invention.
- FIG. 4 is a perspective view of a portion of the camera array according to an alternate embodiment of the present invention.
- FIG. 5 is a flowchart illustrating the general operation of the user interface according to one embodiment of the present invention.
- FIG. 6 is a flowchart illustrating in detail a portion of the operation shown in FIG. 5.
- FIG. 7a is a perspective view of a portion of one embodiment of the present invention illustrating the arrangement of the camera array relative to objects being viewed.
- FIGS. 7b-7 g illustrate views from the perspectives of selected cameras of the array in FIG. 7a.
- FIG. 8 is a schematic view of an alternate embodiment of the present invention.
- FIG. 9 is a schematic view of a server according to one embodiment of the present invention.
- FIG. 10 is a schematic view of a server according to an alternate embodiment of the present invention.
- 1. General Description of Preferred Embodiments
- The present invention relates to a telepresence system that, in preferred embodiments, uses modular, interlocking arrays of microcameras. The cameras are on rails, with each rail holding a plurality of cameras. These cameras, each locked in a fixed relation to every adjacent camera on the array and dispersed dimensionally in a given environment, enable remote viewers to navigate through such environment with the same spatial and visual cues (the changing perspective lines, the moving light reflections and shadows) that characterize an actual in-environment transit.
- In another preferred embodiment, the outputs of these microcameras are linked by tiny (less than half the width of a human hair) Vertical Cavity Surface Emitting Lasers (VCSELs) to optical fibers, fed through area net hubs, buffered on server arrays or server farms (either for recording or (instantaneous) relay) and sent to viewers at remote terminals, interactive wall screens, or mobile image appliances (like Virtual Retinal Displays). Each remote viewer, through an intuitive graphical user interface (GUI), can navigate effortlessly through the environment, enabling seamless movement through the event.
- This involves a multiplexed, electronic switching process (invisible to the viewer) which moves the viewer's point perspective from camera to camera. Rather than relying, per se, on physically moving a microcamera through space, the system uses the multiplicity of positioned microcameras to move the viewer's perspective from microcamera node to adjacent microcamera node in a way that provides the viewer with a sequential visual and acoustical path throughout the extent of the array. This allows the viewer to fluidly track or dolly through a 3-dimensional remote environment, to move through an event and make autonomous real-time decisions about where to move and when to linger.
- Instead of investing the viewer with the capacity to physically move a robotic camera, which would immediately limit the number of viewers that could simultaneously control their own course and navigate via a given camera, the System allows the viewer to float between a multiplicity of microcamera outputs in a way that, via electronic switching (and thus movement through the array), merges their fields of view into a seamless motion path.
-
- Certain embodiments of the present invention will now be described in greater detail with reference to the drawings. It is understood that the operation and functionality of many of the components of the embodiments described herein are known to one skilled in the art and, as such, the present description does not go into detail into such operative and functionality.
- A telepresence system100 according to the present invention is shown in FIG. 1. The telepresence system 100 generally includes an
array 10 ofcameras 14 coupled to aserver 18, which in turn is coupled to one or more users 22 each having a user interfaced/display device 24. As will be understood to one skilled it the art, the operation and functionality of the embodiment described herein is provided, in part, by the server and user interface/display device. While the operation of these components is not described by way of particular code listings or logic diagrams, it is to be understood that one skilled in the art will be able to arrive at suitable implementations based on the functional and operational details provided herein. Furthermore, the scope of the present invention is not to be construed as limited to any particular code or logic implementation. - In the present embodiment, the
camera array 10 is conceptualized as being in an X, Z coordinate system. This allows each camera to have an associated, unique node address comprising an X, and Z coordinate (X, Z). In the present embodiment, for example, a coordinate value corresponding to an axis of a particular camera represents the number of camera positions along that axis the particular camera is displaced from a reference camera. In the present embodiment, from the user's perspective the X axis runs left and right, and the Z axis runs down and up. Eachcamera 14 is identified by its X, Z coordinate. It is to be understood, however, that other methods of identifyingcameras 14 can be used. For example, other coordinate systems, such as those noting angular displacement from a fixed reference point as well as coordinate systems that indicate relative displacement from the current camera node may be used. In another alternate embodiment, the array is three dimensional, located in an X, Y, Z coordinate system. - The
array 10 comprises a plurality ofrails 12, eachrail 12 including a series ofcameras 14. In the present preferred embodiment, thecameras 14 are microcameras. The output from themicrocameras 14 are coupled to theserver 18 by means oflocal area hubs 16. Thelocal area hubs 16 gather the outputs and, when necessary, amplify the outputs for transmission to theserver 18. In an alternate embodiment, thelocal area hubs 16 multiplex the outputs for transmission to theserver 18. Although the figure depicts the communication links 15 between thecameras 14 and theserver 18 as being hardwired, it is to be understood that wireless links may be employed. Thus, it is within the scope of the present invention for the communication links 15 to take the form of fiber optics, cable, satellite, microwave transmission, internet, and the like. - Also coupled to the
server 18 is anelectronic storage device 20. Theserver 18 transfers the outputs to theelectronic storage device 20. The electronic (mass)storage device 20, in turn, transfers each camera's output onto a storage medium or means, such as CD-ROM, DVD, tape, platter, disk array, or the like. The output of eachcamera 14 is stored in a particular location on the storage medium associated with thatcamera 14 or is stored with an indication to whichcamera 14 each stored output corresponds. For example, the output of eachcamera 14 is stored in contiguous locations on a separate disk, tape, CD-ROM, or platter. As is known in the art, the camera output may be stored in a compressed format, such as JPEG, MPEG1, MPEG2, and the like. Having stored each output allows a user to later view the environment over and over again, each time moving through thearray 10 in a new path, as described below. In some embodiments of the present invention, such as those providing only real-time viewing, no storage device is required. - As will be described in detail below, the
server 18 receives output from thecameras 14 in the array. Theserver 18 processes these outputs for either storage in theelectronic storage device 20, transmission to the users 22 or both. - It is to be understood that although the
server 18 is configured to provide the functionality of the system 100 in the present embodiment, it is to be understood that other processing elements may provide the functionality of the system 100. For example, in alternate embodiments, the user interface device is a personal computer programmed to interpret the user input and transmit an indication of the desired current node address, buffer outputs from the array, and provide other of the described functions. - As shown, the system100 can accommodate (but does not require) multiple users 22. Each user 22 has associated therewith a user interface device including a user display device (collectively 24). For example, user 22-1 has an associated user interface device and a user display device in the form of a computer 24-1 having a monitor and a keyboard. User 22-2 has associated therewith an interactive wall screen 24-2 which serves as a user interface device and a user display device. The user interface device and the user display device of user 22-3 includes a mobile audio and image appliance 24-3. A digital interactive TV 24-4 is the user interface device and user display device of user 22-4. Similarly, user 22-5 has a voice recognition unit and monitor 24-5 as the user interface and display devices. It is to be understood that the foregoing user interface devices and user display devices are merely exemplary; for example, other interface devices include a mouse, touch screen, biofeedback devices, as well as those identified in U.S. Provisional Patent Application Serial No. 60/080,413 and the like.
- As described in detail below, each
user interface device 24 has associated therewith user inputs. These user inputs allow each user 22 to move or navigate independently through thearray 10. In other words, each user 22 enters inputs to generally select which camera outputs are transferred to the user display device. Preferably, each user display device includes a graphical representation of thearray 10. The graphical representation includes an indication of which camera in the array the output of which is being viewed. The user inputs allow each user to not only select particular cameras, but also to select relative movement or navigational paths through thearray 10. - As shown in FIG. 1, each user22 may be coupled to the
server 18 by an independent communication link. Furthermore, each communication link may employ different technology. For example, in alternate embodiments, the communication links include an internet link, a microwave signal link, a satellite link, a cable link, a fiber optic link, a wireless link, and the like. - It is to be understood that the
array 10 provides several advantages. For example, because thearray 10 employs a series ofcameras 14, no individual camera, or theentire array 10 for that matter, need be moved in order to obtain a seamless view of the environment. Instead, the user navigates through thearray 10, which is strategically placed through and around the physical environment to be viewed. Furthermore, because thecameras 14 of thearray 10 are physically located at different points in the environment to be viewed, a user is able to view changes in perspective, a feature unavailable to a single camera that merely changes focal length. - Each
camera 14 is preferably a microcamera. The microcameras—microlenses mounted on thumbnail-sized CMOS active pixel sensor (APS) microchips—are arranged in patterns that enable viewers to move radically, in straight lines, or in fluid combinations thereof. The cameras are produced in a mainstream manufacturing process, by several companies, including Photobit, Pasadena, Calif.; Sarnoff Corporation, Princeton, N.J.; and VLSI Vision, Ltd., Edinburgh, Scotland. - The structure of the
array 10 will now be described in greater detail with reference to FIGS. 2a-2 e. In general, thecamera array 10 of the present embodiment comprises a series ofmodular rails 12 carryingmicrocameras 14. The structure of therails 12 andcameras 14 will now be discussed in greater detail with reference to FIGS. 2a through 2 d. Eachcamera 14 includes registration pins 34. In the preferred embodiment, thecameras 14 utilize VCSELs to transfer their outputs to therail 12. It is to be understood that the present invention is not limited to any particular type ofcamera 14, however, or even to anarray 10 consisting of only one type ofcamera 14. - Each
rail 12 includes two sides, 12 a, 12 b, at least one of which 12 b is hingeably connected to the base 12 c of therail 12. The base 12 c includesdocking ports 36 for receiving the registration pins 34 of thecamera 14. When thecamera 14 is seated on arail 12 such that the registration pins 34 are fully engaged in thedocking ports 36, the hingedside 12 b of therail 12 is moved against thebase 32 of thecamera 14, thereby securing thecamera 14 to therail 12. - Each
rail 12 further includes afirst end 38 and asecond end 44. Thefirst end 38 includes, in the present embodiment, two locking pins 40 and a protectedtransmission relay port 42 for transmitting the camera outputs. Thesecond end 44 includes twoguide holes 46 for receiving the locking pins 40, and atransmission receiving port 48. Thus, thefirst end 38 of onerail 12 is engagable with asecond end 44 of anotherrail 12. Therefore, eachrail 12 is modular and can be functionally connected to another rail to create thearray 10. - Once the
camera 14 is securely seated to therail 12, thecamera 14 is positioned such that the camera output may be transmitted via the VCSEL to therail 12. Eachrail 12 includes communication paths for transmitting the output from eachcamera 14. - Although the
array 10 is shown having a particular configuration, it is to be understood that virtually any configuration ofrails 12 andcameras 14 is within the scope of the present invention. For example, thearray 10 may be a linear array ofcameras 14, a 2-dimensional array ofcameras 14, a 3-dimensional array ofcameras 14, or any combination thereof. Furthermore, thearray 10 need not be comprised solely of linear segments, but rather may include curvilinear sections. - The
array 10 is supported by any of a number of support means. For example, thearray 10 can be fixedly mounted to a wall or ceiling; thearray 10 can be secured to a moveable frame that can be wheeled into position in the environment or supported from cables. - FIG. 3 illustrates an example of a portion of the
array 10. As shown, thearray 10 comprises five rows ofrails 12 a, through 12 e. Each of theserails 12 a-12 e is directed towards a central plane, which substantially passes through thecenter row 12 c. Consequently, for any object placed in the same plane as themiddle row 12 c, a user would be able to view the object essentially from the bottom, front, and top. - As noted above, the
rails 12 of thearray 10 need not have the same geometry. For example, some of therails 12 may be straight while others may be curved. For example, FIG. 4 illustrates the camera alignment that results from utilizing curved rails. It should be noted that rails in FIG. 4 have been made transparent so that the arrangement ofcameras 14 may be easily seen. - In an alternate embodiment, each rail is configured in a step-like fashion with each camera above and in front of a previous camera. In such an arrangement, the user has the option of moving forward through the environment.
- It is to be understood that the spacing of the
microcameras 14 depends on the particular application, including the objects being viewed, the focal length of themicrocameras 14, and the speed of movement through thearray 10. In one embodiment the distance betweenmicrocameras 14 can be approximated by analogy to a conventional movie reel recording projector. In general, the speed of movement of a projector through an environment divided by the frames per unit of time second results in a frame-distance ratio. -
- A frame of the projector is analogous to a
camera 14 in the present invention. Thus, where one frame per inch results in a movie having a seamless view of the environment, so too does onecamera 14 per inch. Thus, in one embodiment of the present invention thecameras 14 are spaced approximately one inch apart, thereby resulting in a seamless view of the environment. - The general operation of the present embodiment will now be described with reference to FIG. 5 and continuing reference to FIG. 1. As shown in
step 110, the user is presented with a predetermined starting view of the environment corresponding to a starting camera. It is to be understood that the operation of the system is controlled, in part, by software residing in the server. As noted above, the system associates each camera in the array with a coordinate. Thus, the system is able to note the coordinates of the starting camera node. The camera output and, thus the corresponding view, changes only upon receiving a user input. - When the user determines that they want to move or navigate through the array, the user enters a user input through the
user interface device 24. As described below, the user inputs of the present embodiment generally include moving to the right, to the left, up, or down in the array. Additionally, a user may jump to a particular camera in the array. In alternate embodiments, a subset of these or other inputs, such as forward, backward, diagonal, over, and under, are used. The user interface device, in turn, transmits the user input to the server instep 120. - Next, the server receives the user input in
step 130 and proceeds to decode the input. In the present embodiment, decoding the input generally involves determining whether the user wishes to move to the right, to the left, up, or down in the array. - On the other hand, if the received user input does not correspond to backward, then The
server 18 proceeds to determine whether the input corresponds to moving to the user's right in thearray 10. This determination is shown instep 140. If the received user input does correspond to moving to the right, the current node address is incremented along the X axis instep 150 to obtain an updated address. - If the received user input does not correspond to moving to the right in the array, the
server 18 then determines whether the input corresponds to moving to the user's left in thearray 10 instep 160. Upon determining that the input does correspond to moving to the left, theserver 18 then decrements the current node address along the X axis to arrive at the updated address. This is shown instep 170. - If the received user input does not correspond to either moving to the right or to the left, the
server 18 then determines whether the input corresponds to moving up in the array. This determination is made instep 180. If the user input corresponds to moving up, instep 190, theserver 18 increments the current node address along the Z axis, thereby obtaining an updated address. - Next, the
server 18 determines whether the received user input corresponds to moving down in thearray 10. This determination is made instep 200. If the input does correspond to moving down in thearray 10, instep 210 theserver 18 decrements the current node address along the Z axis. - Lastly, in
step 220 theserver 18 determines whether the received user input corresponds to jumping or changing the view to aparticular camera 14. As indicated in FIG. 5, if the input corresponds to jumping to aparticular camera 14, theserver 18 changes the current node address to reflect the desired camera position. Updating the node address is shown asstep 230. In an alternate embodiment, the input corresponds to jumping to a particular position in thearray 10, not identified by the user as being a particular camera but by some reference to the venue, such as stage right. - It is to be understood that the
server 18 may decode the received user inputs in any of a number of ways, including in any order. For example, in an alternate embodiment theserver 18 first determines whether the user input corresponds to up or down. In another alternate, preferred embodiment, user navigation includes moving forward, backward, to the left and right, and up and down through a three dimensional array. - If the received user input does not correspond to any of the recognized inputs, namely to the right, to the left, up, down, or jumping to a particular position in the
array 10 then instep 240, theserver 18 causes a message signal to be transmitted to theuser display device 24, causing a message to be displayed to the user 22 that the received input was not understood. Operation of the system 100 then continues withstep 120, and theserver 18 awaits receipt of the next user input. - After adjusting the current node address, either by incrementing or decrementing the node address along an axis or by jumping to a particular node address, the
server 18 proceeds instep 250 to adjust the user's view. Once the view is adjusted, operation of the system 100 continues again withstep 120 as theserver 18 awaits receipt of the next user input. - In an alternate embodiment, the
server 18 continues to update the node address and adjust the view based on the received user input. For example, if the user input corresponded to “moving to the right”, then operation of the system 100 would continuously loop throughsteps server 18 continuously updates the view accordingly. - It is to be understood that the foregoing user inputs, namely, to the right, to the left, up, and down, are merely general descriptions of movement through the array. Although the present invention is not so limited, in the present preferred embodiment, movement in each of these general directions is further defined based upon the user input.
- Accordingly, FIG. 6 is a more detailed diagram of the operation of the system according to
steps array 10 at different speeds, moving to the right into a composited additional source output at different speeds, and having the user input overridden by the system 100. - The present invention allows a user22 to navigate through the
array 10 at the different speeds. Depending on the speed (i.e. number of camera nodes transversed per unit of time) indicated by the user's input, such as movement of a pointing device (or other interface device), theserver 18 will apply an algorithm that controls the transition between camera outputs either at critical speed (n nodes/per unit of time), under critical speed (n−1 nodes/per unit of time), or over critical speed (n+1 nodes/per unit of time). - It is to be understood that speed of movement through the
array 10 can alternatively be expressed as the time to switch from onecamera 14 to anothercamera 14. - Specifically, as shown in
step 140 a, theserver 18 makes the determination whether the user input corresponds to moving to the right at a critical speed. The critical speed is preferably a predetermined speed of movement through thearray 10 set by the system operator or designer depending on the anticipated environment being viewed. Further, the critical speed depends upon various other factors, such as focal length, distance between cameras, distance between the cameras and the viewed object, and the like. The speed of movement through thearray 10 is controlled by the number ofcameras 14 traversed in a given time period. Thus, the movement through thearray 10 at critical speed corresponds to traversing some number, “n”, camera nodes per millisecond, or taking some amount of time, “s”, to switch from onecamera 14 to another. It is to be understood that in the same embodiment the critical speed of moving through thearray 10 in one dimension need not equal the critical speed of moving through the array in another dimension. Consequently, theserver 18 increments the current node address along the X axis at n nodes per millisecond. - In the present preferred embodiment the user traverses twenty-four
cameras 14 per second. As discussed above, a movie projector records twenty-four frames per second. Analogizing between the movie projector and the present invention, at critical the user traverses (and theserver 18 switches between) approximately twenty-fourcameras 14 per second, or acamera 14 approximately every 0.04167 seconds. - As shown in FIG. 6, the user22 may advance not only at critical speed, but also at over the critical speed, as shown in
step 140 b, or at under the critical speed, as shown instep 140 c. Where the user input “I” indicates movement through thearray 10 at over the critical speed, theserver 18 increments the current node address along the X axis by a unit of greater than n, for example, at n+2 nodes per millisecond. The step of incrementing the current node address at n+1 nodes per millisecond along the X axis is shown instep 150 b. Where the user input “I” indicates movement through thearray 10 at under the critical speed, theserver 18 proceeds to increment the current node address at a variable less than n, for example, at n−1 nodes per millisecond. This operation is shown asstep 150 c. - The shape of the
array 10 can also be electronically scaled and the system 100 designed with a “center of gravity” that will ease a user's image path back to a “starting” or “critical position” node or ring of nodes, either when the user 22 releases control or when the system 100 is programmed to override the user's autonomy; that is to say, the active perimeter or geometry of thearray 10 can be pre-configured to change at specified times or intervals in order to corral or focus attention in a situation that requires dramatic shaping. The system operator can, by real-time manipulation or via a pre-configured electronic proxy sequentially activate or deactivate designated portions of thecamera array 10. This is of particular importance in maintaining authorship and dramatic pacing in theatrical or entertainment venues, and also for implementing controls over how much freedom a user 22 will have to navigate through thearray 10. - In the present embodiment, the system100 can be programmed such that certain portions of the
array 10 are unavailable to the user 22 at specified times or intervals. Thus, continuing withstep 140 d of FIG. 6, theserver 18 makes the determination whether the user input corresponds to movement to the right through the array but is subject to a navigation control algorithm. The navigation control algorithm causes theserver 18 to determine, based upon navigation control factors, whether the user's desired movement is permissible. - More specifically, the navigation control algorithm, which is programmed in the
server 18, determines whether the desired movement would cause the current node address to fall outside the permissible range of node coordinates. In the present embodiment, the permissible range of node coordinates is predetermined and depends upon the time of day, as noted by theserver 18. Thus, in the present embodiment, the navigation control factors include time. As will be appreciated by those skilled in the art, permissible camera nodes and control factors can be correlated in a table stored in memory. - In an alternate embodiment, the navigation control factors include time as measured from the beginning of a performance being viewed, also as noted by the server. In such an embodiment, the system operator can dictate from where in the array a user will view certain scenes. In another alternate embodiment, the navigation control factor is speed of movement through the array. For example, the faster a user22 moves or navigates through the array, the wider the turns must be. In other alternate embodiments, the permissible range of node coordinates is not predetermined. In one embodiment, the navigation control factors and, therefore, the permissible range, is dynamically controlled by the system operator who communicates with the server via an input device.
- Having determined that the user input is subject to the navigation control algorithm, the
server 18 further proceeds, instep 150 d, to increment the current node address along a predetermined path. By incrementing the current node address along a predetermined path, the system operator is able to corral or focus the attention of the user 22 to the particular view of thepermissible cameras 14, thereby maintaining authorship and dramatic pacing in theatrical and entertainment venues. - In an alternate embodiment where the user input is subject to a navigation control algorithm, the
server 18 does not move the user along a predetermined path. Instead, theserver 18 merely awaits a permissible user input and holds the view at the current node. Only when theserver 18 receives a user input resulting in a permissible node coordinate will theserver 18 adjust the user's view. - In addition to moving through the
array 10, the user 22 may, at predetermined locations in thearray 10, choose to leave the real world environment being viewed. More specifically, additional source outputs, such as computer graphic imagery, virtual world imagery, applets, film clips, and other artificial and real camera outputs, are made available to the user 22. In one embodiment, the additional source output is composited with the view of the real environment. In an alternate embodiment, the user's view transfers completely from the real environment to that offered by the additional source output. - More specifically, the additional source output is stored (preferably in digital form) in the
electronic storage device 20. Upon the user 22 inputting a desire to view the additional source output, theserver 18 transmits the additional source output to the user interface/display device 24. The present embodiment, theserver 18 simply transmits the additional source output to theuser display device 24. In an alternate embodiment, theserver 18 first composites the additional source output with the camera output and then transmits the composited signal to the user interface/display device 24. - As shown in
step 140 e, theserver 18 makes the determination whether the user input corresponds to moving in the array into the source output. If the user 22 decides to move into the additional source output, theserver 18 adjusts the view by substituting the additional source output for the updated camera output identified in either ofsteps 150 a-d. - Once the current node address is updated in either of
steps 150 a-d, theserver 18 proceeds to adjust the user's view instep 250. When adjusting the view, theserver 18 “mixes” the existing or current camera output being displayed with the output of thecamera 14 identified by the updated camera node address. Mixing the outputs is achieved differently in alternate embodiments of the invention. In the present embodiment, mixing the outputs involves electronically switching at a particular speed from the existing camera output to the output of thecamera 14 having the new current node address. - It is to be understood that in this and other preferred embodiments disclosed herein, the camera outputs are synchronized. As is well known in the art, a synchronizing signal from a “sync generator” is supplied to the cameras. The sync generator may take the form of those used in video editing and may comprise, in alternate embodiments, part of the server, the hub, and/or a separate component coupled to the array.
- As described above, at critical speed, the
server 18 switches camera outputs approximately at a rate of 24 per second, or one every 0.04167 seconds. If the user 22 is moving through thearray 10 at under the critical speed, the outputs of theintermediate cameras 14 are each displayed for a relatively longer duration than if the user is moving at the critical speed. Similarly, each output is displayed for a relatively shorter duration when a user navigates at over the critical speed. In other words, theserver 18 adjusts the switching speed based on the speed of the movement through thearray 10. - Of course, it is to be understood that in a simplified embodiment of the present invention, the user may navigate at only the critical speed.
- In another alternate embodiment, mixing the outputs is achieved by compositing the existing or current output and the updated camera node output. In yet another embodiment, mixing involves dissolving the existing view into the new view. In still another alternate embodiment, mixing the outputs includes adjusting the frame refresh rate of the user display device. Additionally, based on speed of movement through the array, the server may add motion blur to convey the realistic sense of speed.
- In yet another alternate embodiment, the server causes a black screen to be viewed instantaneously between camera views. Such an embodiment is analogous to blank film between frames in a movie reel. Furthermore, although not always advantageous, such black screens reduce the physiologic “carrying over” of one view into a subsequent view.
- It is to be understood that the user inputs corresponding to movements through the array at different speeds may include either different keystrokes on a keypad, different positions of a joystick, positioning a joystick in a given position for a predetermined length of time, and the like. Similarly, the decision to move into an additional source output may be indicated by a particular keystroke, joystick movement, or the like.
- In an alternate embodiment, although not always necessary, to ensure a seamless progression of views, the
server 18 also transmits to theuser display device 24 outputs from some or all of the intermediate cameras, namely those located between the current camera node and the updated camera node. Such an embodiment will now be described with reference to FIGS. 7a-7 g. Specifically, FIG. 7a illustrates a curvilinear portion of anarray 10 that extends along the X axis or to the left and right from the user's perspective. Thus, the coordinates that theserver 18 associates with thecameras 14 differ only in the X coordinate. More specifically, for purposes of the present example, thecameras 14 can be considered sequentially numbered, starting with theleft-most camera 14 being the first, i.e., number “1”. The X coordinate of eachcamera 14 is equal to the camera's position in the array. For illustrative purposes, particular cameras will be designate 14-X, where X equals the camera's position in thearray 10 and, thus, its associated X coordinate. - In general, FIGS. 7a-7 g illustrate possible user movement through the
array 10. The environment to be viewed includes three objects 602, 604, 606, the first and second of which include numbered surfaces. As will be apparent, these numbered surface allow a better appreciation of the change in user perspective. - In FIG. 7a, six cameras 14-2, 14-7, 14-11, 14-14, 14-20, 14-23 of the
array 10 are specifically identified. The boundaries of each camera's view is identified by the pair of lines 14-2 a, 14-7 a, 14-11 a, 14-14 a, 14-20 a, 14-23 a, radiating from each identified camera 14-2, 14-7, 14-11, 14-14, 14-20, 14-23, respectively. As described below, in the present example the user 22 navigates through thearray 10 along the X axis such that the images or views of the environment are those corresponding to the identified cameras 14-2, 14-7, 14-11, 14-14, 14-20, 14-23. - The present example provides the user22 with the starting view from camera 14-2. This view is illustrated in FIG. 7b. The user 22, desiring to have a better view of the
object 702, pushes the “7” key on the keyboard. This user input is transmitted to and interpreted by theserver 18. - Because the
server 18 has been programmed to recognized the “7” key as corresponding to moving or jumping through the array to camera 14-7. Theserver 18 changes the X coordinate of the current camera node address to 7, selects the output of camera 14-7, and adjusts the view or image sent to the user 22. Adjusting the view, as discussed above, involves mixing the outputs of the current and updated camera nodes. Mixing the outputs, in turn, involves switching intermediate camera outputs into the view to achieve the seamless progression of the discrete views of cameras 14-2 through 14-7, which gives the user 22 the look and feel of moving around the viewed object. The user 22 now has another view of thefirst object 702. The view from camera 14-7 is shown in FIG. 7c. As noted above, if the jump in camera nodes is greater than a predetermined limit, theserver 18 would omit some or all of the intermediate outputs. - Pressing the “right arrow” key on the keyboard, the user22 indicates to the system 100 a desire to navigate to the right at critical speed. The
server 18 receives and interprets this user input as indicating such and increments the current camera node address by n=4. Consequently, the updated camera node address is 14-11. Theserver 18 causes the mixing of the output of camera 14-11 with that of camera 14-7. Again, this includes switching into the view the outputs of the intermediate cameras (i.e., 14-8, 14-9, and 14-10) to give the user 22 the look and feel of navigating around the viewed object. The user 22 is thus presented with the view from camera 14-11, as shown in FIG. 7d. - Still interested in the
first object 702, the user 22 enters a user input, for example, “alt-right arrow,” indicating a desire to move to the right at less than critical speed. Accordingly, theserver 18 increments the updated camera node address by n−1 nodes, namely 3 in the present example, to camera 14-14. The outputs from cameras 14-11 and 14-14 are mixed, and the user 22 is presented with a seamless view associated with cameras 14-11 through 14-14. FIG. 7e illustrates the resulting view of camera 14-14. - With little to see immediately after the
first object 702, the user 22 enters a user input such as “shift-right arrow,” indicating a desire to move quickly through thearray 10, i.e., at over the critical speed. Theserver 18 interprets the user input and increments the current node address by n+2, or 6 in the present example. The updated node address thus corresponds to camera 14-20. Theserver 18 mixes the outputs of cameras 14-14 and 14-20, which includes switching into the view the outputs of the intermediate cameras 14-15 through 14-19. The resulting view of camera 14-20 is displayed to the user 22. As shown in FIG. 7f, the user 22 now views thesecond object 704. - Becoming interested in the
third object 704, the user 22 desires to move slowly through thearray 10. Accordingly, the user 22 enters “alt-right arrow” to indicate moving to the right at below critical speed. Once theserver 18 interprets the received user input, it updates the current camera node address along the X axis by 3 to camera 14-23. Theserver 18 then mixes the outputs of camera 14-20 and 14-23, thereby providing the user 22 with a seamless progression of views through camera 14-23. The resulting view 14-23 a is illustrated in FIG. 7g. - It is to be understood that devices other than cameras may be interspersed in the array. These other devices, such as motion sensors and microphones, provide data to the server(s) for processing. For example, in alternate embodiments output from motion sensors or microphones are fed to the server(s) and used to scale the array. More specifically, permissible camera nodes (as defined in a table stored in memory) are those near the sensor or microphone having a desired output e.g., where there is motion or sound. As such, navigation control factors include output from other such devices. Alternatively, the output from the sensors or microphones are provided to the user.
- An alternate embodiment in which the array of cameras includes multiple microphones interspersed among the viewed environment and the cameras will now be described with reference to FIG. 8. The
system 800 generally includes an array ofcameras 802 coupled to aserver 804, which, in turn, is coupled to one or more user interface anddisplay devices 806 and anelectronic storage device 808. Ahub 810 collects and transfers the outputs from thearray 802 to theserver 804. More specifically, thearray 802 comprisesmodular rails 812 that are interconnected. Eachrail 812 carriesmultiple microcameras 814 and amicrophone 816 centrally located atrail 812. Additionally, thesystem 800 includesmicrophones 818 that are physically separate from thearray 802. The outputs of both thecameras 814 andmicrophones server 804 for processing. - In general, operation of the
system 800 proceeds as described with respect to system 100 of FIGS. 1-2 d and 5-6. Beyond the operation of the previously described system 100, however, theserver 804 receives the sound output from themicrophones server 804 updates the current camera node address and changes the user's view, it also changes the sound output transmitted to the user. In the present embodiment, theserver 804 has stored in memory an associated range of camera nodes with a given microphone, namely thecameras 814 on eachrail 810 are associated with themicrophone 816 on thatparticular rail 810. In the event a user attempts to navigate beyond the end of thearray 802, theserver 804 determines the camera navigation is impermissible and instead updates the microphone node output to that of themicrophone 818 adjacent to thearray 802. - In an alternate embodiment, the
server 804 might include a database in which camera nodes in a particular area are associated with a given microphones. For example, a rectangle defined by the (X, Y, Z) coordinates (0,0,0), (10,0,0), (10,5,0), (0,5,0), (0,0,5), (10,0,5), (10,5,5) and (0,5,5) are associated with a given microphone. It is to be understood that selecting one of the series of microphones based on the user's position (or view) in the array provides the user with a sound perspective of the environment that coincides with the visual perspective. - It is to be understood that the server of the embodiments discussed above may take any of a number of known configurations. Two examples of server configurations suitable for use with the present invention will be described with reference to FIGS. 9 and 10. Turning first to FIG. 9, the
server 902,electronic storage device 20,array 10, users (1,2,3, . . . N) 22-1-22-N, and associated user interface/display devices 24-1-24-N are shown therein. - The
server 902 includes, among other components, a processing means in the form of one or more central processing units (CPU) 904 coupled to associated read only memory (ROM) 906 and a random access memory (RAM) 908. In general,ROM 906 is for storing the program that dictates the operation of theserver 902, and theRAM 908 is for storing variables and values used by theCPU 904 during operation. Also coupled to theCPU 904 are the user interface/display devices 24. It is to be understood that the CPU may, in alternate embodiments, comprise several processing units, each performing a discrete function. - Coupled to both the
CPU 904 and theelectronic storage device 20 is amemory controller 910. Thememory controller 910, under direction of theCPU 904, controls accesses (reads and writes) to thestorage device 20. Although thememory controller 910 is shown as part of theserver 902, it is to be understood that it may reside in thestorage device 20. - During operation, the
CPU 904 receives camera outputs from thearray 10 viabus 912. As described above, theCPU 904 mixes the camera outputs for display on the user interface/display device 24. Which outputs are mixed depends on the view selected by each user 22. Specifically, each user interface/display devices 24 transmits acrossbus 914 the user inputs that define the view to be displayed. Once theCPU 904 mixes the appropriate outputs, it transmits the resulting output to the user interface/display device 24 viabus 916. As shown, in the present embodiment, each user 22 is independently coupled to theserver 902. - The
bus 912 also carries the camera outputs to thestorage device 20 for storage. When storing the camera outputs, theCPU 904 directs thememory controller 910 to store the output of eachcamera 14 in a particular location of memory in thestorage device 20. - When the image to be displayed has previously been stored in the
storage device 20, theCPU 904 causes thememory controller 910 to access thestorage device 20 to retrieve the appropriate camera output. The output is thus transmitted to theCPU 904 viabus 918 where it is mixed.Bus 918 also carries additional source output to theCPU 904 for transmission to the users 22. As with outputs received directly from thearray 10, theCPU 904 mixes these outputs and transmits the appropriate view to the user interface/display device 24. - FIG. 10 shows a server configuration according to an alternate embodiment of the present invention. As shown therein, the
server 1002 generally comprises a control central processing unit (CPU) 1004, a mixingCPU 1006 associated with each user 22, and amemory controller 1008. Thecontrol CPU 1004 has associatedROM 1010 andRAM 1012. Similarly, each mixingCPU 1006 has associatedROM 1014 andRAM 1016. - To achieve the functionality described above, the camera outputs from the
array 10 are coupled to each of the mixingCPUs 1 through N 1006-1, 1006-N viabus 1018. During operation, each user 22 enters inputs in the interface/display device 24 for transmission (via bus 1020) to thecontrol CPU 1004. Thecontrol CPU 1004 interprets the inputs and, via buses 1022-1, 1022-N, transmits control signals to the mixing CPUs 1006-1, 1006-N instructing them which camera outputs received onbus 1018 to mix. As the name implies, the mixing CPUs 1006-1, 1006-N mix the outputs in order to generate the appropriate view and transmit the resulting view via buses 1024-1, 1024-N to the user interface/display devices 24-1, 24-N. - In an alternate related embodiment, each mixing
CPU 1006 multiplexes outputs to more than one user 22. Indications of which outputs are to mixed and transmitted to each user 22 comes from thecontrol CPU 1004. - The
bus 1018 couples the camera outputs not only to the mixing CPUs 1006-1, 1006-N, but also to thestorage device 20. Under control of thememory controller 1008, which in turn is controlled by thecontrol CPU 1004, thestorage device 20 stores the camera outputs in known storage locations. Where user inputs to thecontrol CPU 1004 indicate a users' 22 desire to view stored images, thecontrol CPU 1004 causes thememory controller 1008 to retrieve the appropriate images from thestorage device 20. Such images are retrieved into the mixingCPUs 1006 viabus 1026. Additional source output is also retrieved to the mixing CPUs 1006-1, 1006-N viabus 1026. Thecontrol CPU 1004 also passes control signals to the mixing CPUs 1006-1, 1006-N to indicate which outputs are to be mixed and displayed. - It is to be understood that it is within the scope of the present invention to employ stereoscopic views of the environment. To achieve the stereoscopic view, the system retrieves from the array (or the electronic storage device) and simultaneously transmits to the user at least portions of outputs from two cameras. The server processing element mixes these camera outputs to achieve a stereoscopic output. Each view provided to the user is based on such a stereoscopic output. In one stereoscopic embodiment, the outputs from two adjacent cameras in the array are used to produce one stereoscopic view. Using the notation of FIGS. 7a-7 g, one view is the stereoscopic view from cameras 14-1 and 14-2. The next view is based on the stereoscopic output of cameras 14-2 and 14-3 or two other cameras. Thus, in such an embodiment, the user is provided the added feature of a stereoscopic seamless view of the environment.
- As described above, the present invention allows multiple users to simultaneously navigate through the array independently of each other. To accommodate multiple users, the systems described above distinguish between inputs from the multiple users and selects a separate camera output appropriate to each user's inputs. In one such embodiment, the server tracks the current camera node address associated with each user by storing each node address in a particular memory location associate with that user. Similarly, each user's input is differentiated and identified as being associated with the particular memory location with the use of message tags appended to the user inputs by the corresponding user interface device.
- In an alternate embodiment, two or more users may choose to be linked, thereby moving in tandem and having the same view of the environment. In such an embodiment, each includes identifying another user by his/her code to serve as a “guide”. In operation, the server provides the outputs and views selected by the guide user to both the guide and the other user selecting the guide. Another user input causes the server to unlink the users, thereby allowing each user to control his/her own movement through the array.
- Although the present invention has been described in terms of certain preferred embodiments, other embodiments that are apparent to those of ordinary skill in the art are also intended to be within the scope of this invention. Accordingly, the scope of the present invention is intended to be limited only by the claims appended hereto.
Claims (135)
1. A telepresence system for providing a first user with a first display of an environment and a second user with a second display of the environment, the system comprising:
an array of cameras, each camera having an associated view of the environment and an associated camera output representing the associated view, the array including at least one camera path;
a first user interface device associated with the first user having first user inputs associated with movement along a first path in the array;
a second user interface device associated with the second user having second user inputs associated with movement along a second path in the array;
at least one processing element coupled to the user interface devices for receiving user inputs, the processing element configured to interpret received first inputs and select outputs of cameras in the first path, and interpret received second inputs and select outputs of cameras in the second path independently of the first inputs, thereby allowing the first user and second user to navigate simultaneously and independently through the array.
2. The system of claim 1 further including:
a first display device associated with the first user and coupled to the processing element; and
a second display device associated with the second user and coupled to the processing element,
wherein the processing element is further configured to select a first output based on the received first inputs and send the first output to the first the first display device, the first display device configured to display the view associated with the selected first output, and
select a second output based on the received second inputs and send the second output to the second display device, the second display device configured to display the view associated with the selected second output.
3. The system of claim 2 wherein the processing element is configured to select outputs in real time and the display devices are configure to display views in real time.
4. The system of claim 1 wherein the array includes a plurality of navigable camera paths and the first path is different than the second path.
5. The system of claim 1 wherein the processing element includes a control processing element and a plurality of mixing processing elements.
6. The system of claim 1 wherein the processing element is also coupled to the array.
7. The system of claim 1 wherein the processing element includes a plurality of processing elements.
8. The system of claim 2 wherein the first user interface device and the first display device are coupled to the processing element by different types of communication links.
9. The system of claim 1 wherein the user interface device is coupled to the processing element by one of the following communication links: an internet link, a microwave link, a satellite link, a wireless link, or a fiber optics link.
10. The system of claim 1 wherein the first user interface device and second user interface device are coupled to the processing element by different communication links.
11. The system of claim 1 wherein the first user interface device and second user interface device are different.
12. The system of claim 1 wherein the array of cameras includes a plurality of modular rails having a male end and a female end, the male end of one rail connectable to the female end of another rail, and a plurality of cameras connectable to each rail.
13. The system of claim 1 wherein the array of cameras includes hollow ball joints, each of which houses a camera, and hollow rods interconnecting the ball joints, the rods housing communication paths for the outputs.
8.
14. The system of claim 1 wherein the cameras in the array are positioned at a density of approximately one camera every inch.
8.
15. The system of claim 1 wherein the view of the first camera overlaps the view of the second camera.
16. The system of claim 1 wherein the array includes a first camera having a first output and a second camera having a second output and wherein the processing element is further configured to mix the first and second outputs in accordance with the received first user inputs.
16.
17. The system of claim 16 wherein the received first user inputs indicate a desire to move from the first camera to the second camera, and the processing element is configured to mix the first and second outputs by electronically switching between the first and second outputs at a switching speed.
16.
18. The system of claim 17 wherein the processing element is further configured to mix the outputs by adjusting the switching speed.
16.
19. The system of claim 1 wherein the first user inputs include an indication of a speed of movement through the first path.
16.
20. The system of claim 19 wherein the processing element is further configured to compare the indication of speed to a predetermined threshold and to select a number of outputs corresponding to fewer than all cameras in the path upon determining the indication of speed exceeds the predetermined threshold.
16.
21. The system of claim 19 wherein the number of outputs is inversely proportional to the speed.
16.
22. The system of claim 16 wherein the first user inputs include an indication of relative speed of movement through the array and the processing element is configured to mix the first output and second out by switching from the first output to the second output of a switching speed, the processing element further configured to adjust the switching speed in accordance with the indication of relative speed.
16.
23. The system of claim 19 wherein the indication of relative speed of movement through the array includes an indication of moving relatively slower through the array and the processing element adjusts the switching speed by causing the view associated with the first camera to be displayed for a relatively longer period of time in accordance with the indication of moving slower.
16.
24. The system of claim 16 wherein the processing element is further configured to mix the outputs by dissolving the view associated with the first camera with the view of the second camera.
16.
25. The system of claim 16 further comprising a first user display device associated with the first user and coupled to the processing element and wherein the first display device has a frame refresh rate associated therewith and the processing element is configured to mix the outputs by adjusting the frame refresh rate.
16.
26. The system of claim 16 wherein the processing element is configured to mix the outputs by creating a composite output from the first and second outputs.
16.
27. The system of claim 16 wherein the array further includes at least one other camera interposed between the first camera and the second camera and wherein the processor is configured to mix the outputs of the first camera and the second camera by switching from the first output, to an output of the at least one camera, and then to the second output.
16.
28. The system of claim 16 wherein the processing element is configured to mix outputs by adding motion blur.
29. The system of claim 1 wherein the user interface device includes a keyboard, wherein user inputs are keystrokes.
29.
30. The system of claim 1 wherein the user interface device includes a joystick wherein user inputs are movements of the joystick.
29.
31. The system of claim 1 wherein the user interface device includes a mouse, wherein user inputs are movements of the mouse.
32. The system of claim 1 wherein the user interface device includes a touchscreen.
33. The system of claim 1 wherein the user interface device includes a biofeedback device.
34. The system of claim 1 wherein the user interface device includes a voice recognition device.
35. The system of claim 1 wherein the user inputs include an indication of a particular camera in the array.
36. The system of claim 1 wherein the user inputs include moving up, down, to the right, and to the left.
37. The system of claim 34 wherein the user inputs further include moving forward and backward.
38. The system of claim 1 wherein the processing element operates in accordance with a navigation control algorithm, the navigation control algorithm distinguishing between permissible movement and impermissible movement in the array.
39. The system of claim 38 further including navigation control factors, wherein the navigation control algorithm controls movement through the array in accordance with the navigation control factors.
40. The system of claim 39 wherein the navigation control factors include time of day, time duration, speed of navigation through the array, output of a device in the array, output of a motion sensor, and output of a microphone.
41. The system of claim 1 wherein the processing element is further configured to disregard certain user input.
42. The system of claim 41 wherein the processing element is further configured to disregard user input by selecting at least one predetermined camera output, thereby directing the user to a predetermined view.
43. The system of claim 42 wherein the processing element is configured to select the predetermined output based on time of day.
44. The system of claim 42 wherein the processing element is configured to select the predetermined output based on a duration of time.
45. The system of claim 41 wherein the processing element is further configured to disregard user input by waiting for a permissible user input.
46. The system of claim 1 further including an operator input device coupled to the processing element for receiving from the operator input device operator input, and wherein the processing element is controlled to override user input based on the operator input.
47. The system of claim 1 wherein the system further includes a memory storing additional source output, wherein the user inputs include an indication of viewing the additional source output, and the processing element is further configured to mix camera output and the additional source output upon receiving the indication to view the additional source output.
48. The system of claim 47 wherein the indication of viewing the additional source output is available to the user in connection with a predetermined camera.
49. The system of claim 47 wherein the processing element is configured to mix camera output and the additional source output by compositing camera output and the additional source output.
50. The system of claim 47 wherein the processing element is configured to mix camera output and the additional source output by switching from camera output to the additional source output.
51. The system of claim 47 wherein the additional source output includes output from the group of outputs including: computer graphic imagery, virtual world imagery, applets, film clips, and animation.
52. The system of claim 1 further including a plurality of microphones, each microphone having an output, wherein the processing element is further configured to select a microphone output based on received first user input for transmission to the first user.
53. The system of claim 52 further including a memory coupled to the processing element, the memory correlating cameras and microphones, the processing element selecting a microphone by using the memory to correlate the selected camera output with a microphone output.
54. A method of providing users with real time views of a remote environment, the method comprising:
receiving electronic images of the environment from an array of cameras, the array including at least one camera path through the environment;
receiving a first input from a first user interface device associated with a first user, the first input indicating movement along a first path;
receiving a second input from a second user interface device associated with a second user, the second input indicating movement along a second path;
obtaining a first mixed image by mixing, with a first processing element, a first image with a second image in accordance with the first input;
obtaining a second mixed image by mixing, with a second processing element, a third image with a fourth image in accordance with the second input;
providing the first user with the first mixed image in substantially real time, thereby simulating movement along the first path; and
providing the second user with the second mixed image substantially in real time and simultaneous to providing the first user with the first mixed image, thereby independently and simulating movement along the second path.
55. The method of claim 54 wherein the first processing element is different than the second processing element.
56. The method of claim 54 wherein the array includes a plurality of navigable paths and the first path differs from the second path.
57. The method of claim 54 wherein the first image is different than the third image.
58. The method of claim 54 wherein mixing the first image with the second image includes switching from the first image to the second image at a first switching speed, the first mixed image including the second image, and mixing the third image with the fourth image includes switching from the third image to the fourth image at a second switching speed, the second mixed image including the fourth image.
59. The method of claim 58 wherein receiving the first input includes receiving an indication of a first speed of movement along the first path and wherein the first switching speed is based on the indication of the first speed of movement.
60. The method of claim 59 wherein receiving the second input includes receiving an indication of a second speed of movement along the second path, the second speed greater than the first speed, and wherein the second switching speed is based on the indication of the second speed of movement, the second switching speed greater than the first switching speed.
61. The method of claim 60 wherein mixing the third and fourth images involves providing the second user the third image for a relatively longer duration than the first image is provided to the first user.
62. The method of claim 60 wherein mixing the third and fourth images involves providing the second user the fourth image for a relatively longer duration than the second image is provided to the first user.
63. The method of claim 54 wherein mixing the first image with the second image includes compositing the first image with the second image.
64. The method of claim 63 wherein mixing the third image with the fourth image includes switching between the third image and the fourth image.
65. The method of claim 54 wherein mixing the first image with the second image includes dissolving the first image into the second image.
66. The method of claim 54 wherein the first image is from a first camera, the second image is from a second camera, the array includes another camera interposed between the first and second cameras, and mixing the first image with the second image includes providing the first user with an image from the interposed camera.
67. The method of claim 54 further including obtaining a third mixed image by mixing the second image with a fifth image in accordance with the first input and providing the first use the third mixed image.
68. The method of claim 54 further including receiving a third input from the first user interface device and overriding the third input.
69. The method of claim 68 wherein overriding the received third input includes presenting the first user with a predetermined image.
70. The method of claim 68 wherein overriding the received third input includes continuing to provide the first user with the first mixed image and awaiting a fourth input.
71. The method of claim 54 wherein receiving the first input includes receiving an indication of mixing an additional source output, the method further including obtaining a third mixed output by mixing the second image with the additional source output.
72. The method of claim 71 wherein mixing the second image with the additional source output includes switching from the second image to the additional source output.
73. A device for providing a user with a display of an environment, the device comprising:
an array of cameras, each camera having an associated view of the environment and an associated camera output representing the associated view; and
at least one processing element coupled to the array, the processing element configured to identify permissible cameras, the outputs of which may be provided to the user, and impermissible cameras, the outputs of which are not provided to the user.
74. The device of claim 73 wherein the permissible cameras change over time.
75. The device of claim 74 wherein the permissible cameras change based on the time of day.
76. The device of claim 73 wherein the environment being viewed is a performance and the permissible cameras change based on the duration of time from the start of the performance.
77. The device of claim 73 wherein the permissible cameras are predetermined.
78. The device of claim 77 further including memory coupled to the processing element, the memory storing an indication of the predetermined permissible cameras.
79. The device of claim 73 further including a user interface device associated with the user and coupled to the processing element, the user interface device providing user inputs to the processing element indicative of movement through the array, the processing element selecting permissible camera output based on the user input.
80. The device of claim 79 wherein the processing element is configured to disregard user input corresponding to a impermissible camera.
81. The device of claim 80 wherein the processing element is configured to disregard user input corresponding to a impermissible camera by awaiting new user input corresponding to a permissible camera.
82. The device of claim 80 wherein the processing element is configured to disregard user input corresponding to a impermissible camera by selecting a permissible camera output.
83. The device of claim 73 further including a data device other than a camera having an output coupled to the at least one processing element, the permissible cameras based on the data device output.
84. The device of claim 83 wherein the data device is a motion sensor.
85. The device of claim 83 wherein the data device is a microphone.
86. A method for providing a user with views of an environment from a plurality of cameras in accordance with user inputs, the method comprising:
electronically receiving user inputs indicative of a view from a desired camera;
identifying permissible cameras, the views from which are capable of being provided to the user; and
determining whether the desired camera is one of the permissible cameras; and
if the desired camera is not one of the permissible cameras, preventing the user from obtaining the view from the desired camera.
87. The method of claim 86 wherein the permissible cameras change while the user views the environment.
88. The method of claim 86 wherein identifying permissible cameras is based on time of day.
89. The method of claim 86 further including noting when the user begins to view the environment and wherein the permissible cameras are based on a period of time from when the user begins to view the environment.
90. The method of claim 86 wherein the environment is a performance and the permissible cameras are based on the performance.
91. A device for providing a user with a display of an environment in response to user inputs, the system comprising:
an array of cameras, each camera having an associated view of the environment and an associated camera output representing the associated view;
memory storing an additional source output; and
at least one processing element coupled to the memory for receiving the additional source output, the processing element configured to interpret user inputs and select, based on the user inputs, a camera output to provide to the user, the additional source output to provide to the user, or both a camera view and the additional source output to provide to the user.
92. The device of claim 91 wherein the processing element is configured to mix the additional source output with camera output.
93. The device of claim 92 wherein the processing element is configured to mix the additional source output and camera output by switching from the camera output to the additional source output.
94. The system of claim 91 further including a plurality of user interface devices coupled to the processing element, each associated with a different user and having user inputs associated with movement among cameras in the array.
95. The system of claim 91 wherein the array of cameras includes a plurality of modular rails having a male end and a female end, the male end of one rail connectable to the female end of another rail, and a plurality of cameras connectable to each rail.
96. The system of claim 91 wherein the array includes at least one user navigable path of cameras and the user inputs include an indication of a speed of movement through the path.
97. The system of claim 91 wherein the processing element is further configured to override user inputs and select at least one predetermined camera output, thereby directing the user to a predetermined view.
98. The system of claim 97 wherein the processing element is configured to select the predetermined output based on a navigation control factor.
99. A method of providing a user with a view of an environment, the method comprising:
receiving electronic images of the environment from an array of cameras;
receiving via a telecommunication link user inputs indicative of movement through the array;
selecting camera images based on the user inputs;
retrieving additional source output from memory based on user input indicating a desire to view the additional source output; and
providing the user with an additional source output at one or more points in the array.
100. A system for remote seamless viewing of an environment from an array of cameras, each having an output representing an image of the environment, the device comprising:
an interface device having inputs for selecting a path through at least a portion of the array from which to view the environment, the path including a sequence of cameras, each camera in the sequence having a different point perspective and a field of view that overlaps that of an adjacent cameras; and
a display device for sequentially displaying the image from each camera in the sequence, thereby providing the user a seamless view of the environment.
101. The system of claim 100 wherein the images from the selected cameras are displayed at a rate of approximately twenty-four images per second.
102. The system of claim 100 wherein the user interface device includes at least one of the following devices: a keyboard; a joystick; a mouse; a touchscreen; a biofeedback device; and a voice recognition device.
103. The system of claim 100 wherein the images displayed by the display device are mixed.
104. The system of claim 103 wherein the images displayed are mixed by continuously switching from the image of a current camera in the sequence to the image of a next camera in the sequence.
105. The system of claim 104 wherein the inputs are for selecting a speed of progressing through the path and the images are switched in accordance with the selected speed.
106. The system of claim 103 wherein the images displayed are mixed by continuously compositing the image from a current camera in the sequence with the image of a next camera in the array.
107. The system of claim 100 wherein the inputs are for selecting a speed of progressing through the path.
108. The system of claim 100 wherein the display device displays a plurality of images stereoscopically.
109. A method for seamless viewing of an environment, the method comprising:
receiving electronically a first image from an array of cameras, the first image having a first field of view;
receiving electronically a second image from the array, the second image having a second field of view that overlaps the first field of view;
receiving electronically a third image from the array, the third image having a third field of view that overlaps the second field of view;
displaying the first, second, and third images in sequence to obtaining a seamless view through the environment.
110. The method of claim 109 wherein the displaying includes switching from the first image to the second image to the third image at a given speed.
111. The method of claim 110 further including selecting a speed of movement through the array, wherein the given speed is based on the selected speed.
112. The method of claim 109 wherein the displaying includes applying a given frame refresh rate to the images.
113. The method of claim 109 wherein the displaying includes compositing the first image with the second image and then compositing the second image with the third image.
114. The method of claim 109 wherein the displaying includes dissolving the first image into the second image and then dissolving the second image into the third image.
115. The method of claim 109 further including selecting an additional source output to be displayed and mixing the additional source output with the third image.
116. The method of claim 109 wherein the first, second and third images correspond to first, second, and third cameras, respectively.
117. The method of claim 116 wherein the second camera is adjacent to the first and third cameras.
118. The method of claim 109 wherein at least one of the images is a stereoscopic image obtained from a plurality of cameras in the array.
119. A telepresence system for providing a user with a display of an environment, the system comprising:
an array of cameras, each camera having an associated view of the environment and an associated camera output representing the associated view;
an electronic storage device; and
at least one processing element coupled to the array for receiving camera outputs, the processing element also coupled to the electronic storage device, the processing element configured to electronically store outputs from cameras and an indication of which camera each output is associated with on the electronic storage device, thereby allowing subsequent retrieval of the stored outputs and viewing of the environment.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US10/391,509 US20030229735A1 (en) | 1998-04-02 | 2003-03-17 | Navigable telepresence method and system utilizing an array of cameras |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US8041398P | 1998-04-02 | 1998-04-02 | |
US09/283,413 US6535226B1 (en) | 1998-04-02 | 1999-04-01 | Navigable telepresence method and system utilizing an array of cameras |
US10/391,509 US20030229735A1 (en) | 1998-04-02 | 2003-03-17 | Navigable telepresence method and system utilizing an array of cameras |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/283,413 Continuation US6535226B1 (en) | 1998-04-02 | 1999-04-01 | Navigable telepresence method and system utilizing an array of cameras |
Publications (1)
Publication Number | Publication Date |
---|---|
US20030229735A1 true US20030229735A1 (en) | 2003-12-11 |
Family
ID=22157205
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/283,413 Expired - Lifetime US6535226B1 (en) | 1998-04-02 | 1999-04-01 | Navigable telepresence method and system utilizing an array of cameras |
US10/391,509 Abandoned US20030229735A1 (en) | 1998-04-02 | 2003-03-17 | Navigable telepresence method and system utilizing an array of cameras |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US09/283,413 Expired - Lifetime US6535226B1 (en) | 1998-04-02 | 1999-04-01 | Navigable telepresence method and system utilizing an array of cameras |
Country Status (11)
Country | Link |
---|---|
US (2) | US6535226B1 (en) |
EP (1) | EP1068730A4 (en) |
JP (1) | JP2004502321A (en) |
KR (1) | KR20010074470A (en) |
CN (1) | CN1298607A (en) |
AU (1) | AU761950B2 (en) |
BR (1) | BR9909429A (en) |
CA (1) | CA2326731A1 (en) |
IL (1) | IL138808A0 (en) |
MX (1) | MXPA00009686A (en) |
WO (1) | WO1999052288A1 (en) |
Cited By (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20050179942A1 (en) * | 2000-07-31 | 2005-08-18 | Stavely Donald J. | Method for introduction and linking of imaging appliances |
US20140341484A1 (en) * | 2013-05-20 | 2014-11-20 | Steven Sebring | Systems and methods for producing visual representations of objects |
WO2014189927A3 (en) * | 2013-05-20 | 2015-01-15 | Sebring Steven | Systems and methods for producing visual representations of objects |
US8937646B1 (en) * | 2011-10-05 | 2015-01-20 | Amazon Technologies, Inc. | Stereo imaging using disparate imaging devices |
EP3062148A1 (en) * | 2015-02-24 | 2016-08-31 | Nokia Technologies Oy | Device with an adaptive camera array |
USD781948S1 (en) | 2015-12-03 | 2017-03-21 | Durst Sebring Revolution, Llc | Photographic imaging system |
USD782559S1 (en) | 2015-12-03 | 2017-03-28 | Durst Sebring Revolution, Llc | Photo booth |
USD798936S1 (en) | 2015-12-03 | 2017-10-03 | Durst Sebring Revolution, Llc | Photo booth |
USD812671S1 (en) | 2015-12-03 | 2018-03-13 | Durst Sebring Revolution, Llc | 3D imaging system |
USD822746S1 (en) | 2016-02-05 | 2018-07-10 | Durst Sebring Revolution, Llc | Photo booth |
Families Citing this family (45)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB9929870D0 (en) * | 1999-12-18 | 2000-02-09 | Roke Manor Research | Improvements in or relating to security camera systems |
US7101988B2 (en) * | 2000-10-12 | 2006-09-05 | Marical, Inc. | Polyvalent cation-sensing receptor in Atlantic salmon |
US7027083B2 (en) | 2001-02-12 | 2006-04-11 | Carnegie Mellon University | System and method for servoing on a moving fixation point within a dynamic scene |
WO2002065763A2 (en) | 2001-02-12 | 2002-08-22 | Carnegie Mellon University | System and method for manipulating the point of interest in a sequence of images |
US20020190991A1 (en) * | 2001-05-16 | 2002-12-19 | Daniel Efran | 3-D instant replay system and method |
US7091989B2 (en) * | 2001-08-10 | 2006-08-15 | Sony Corporation | System and method for data assisted chroma-keying |
US7339609B2 (en) * | 2001-08-10 | 2008-03-04 | Sony Corporation | System and method for enhancing real-time data feeds |
US7173672B2 (en) * | 2001-08-10 | 2007-02-06 | Sony Corporation | System and method for transitioning between real images and virtual images |
US20030030658A1 (en) * | 2001-08-10 | 2003-02-13 | Simon Gibbs | System and method for mixed reality broadcast |
US7091931B2 (en) * | 2001-08-17 | 2006-08-15 | Geo-Rae Co., Ltd. | Method and system of stereoscopic image display for guiding a viewer's eye motion using a three-dimensional mouse |
US6940538B2 (en) * | 2001-08-29 | 2005-09-06 | Sony Corporation | Extracting a depth map from known camera and model tracking data |
GB2380885A (en) * | 2001-10-11 | 2003-04-16 | Hewlett Packard Co | Multiple camera arrangement |
US7239345B1 (en) * | 2001-10-12 | 2007-07-03 | Worldscape, Inc. | Camera arrangements with backlighting detection and methods of using same |
US20030076413A1 (en) * | 2001-10-23 | 2003-04-24 | Takeo Kanade | System and method for obtaining video of multiple moving fixation points within a dynamic scene |
US20030187820A1 (en) | 2002-03-29 | 2003-10-02 | Michael Kohut | Media management system and process |
CA2390621C (en) * | 2002-06-13 | 2012-12-11 | Silent Witness Enterprises Ltd. | Internet video surveillance camera system and method |
JP2004264492A (en) * | 2003-02-28 | 2004-09-24 | Sony Corp | Photographing method and imaging apparatus |
US20040199622A1 (en) * | 2003-04-07 | 2004-10-07 | Huscher Anthony Alan | eRoom operations console |
US20080129818A1 (en) * | 2004-02-11 | 2008-06-05 | Jacob James Miller | Methods for practically simulatnig compact 3d environments for display in a web browser |
US20060114251A1 (en) * | 2004-02-11 | 2006-06-01 | Miller Jacob J | Methods for simulating movement of a computer user through a remote environment |
US20060146132A1 (en) * | 2005-01-05 | 2006-07-06 | Hy Mayerson | Video system having multiple video cameras for capturing events |
US7982777B2 (en) * | 2005-04-07 | 2011-07-19 | Axis Engineering Technologies, Inc. | Stereoscopic wide field of view imaging system |
US7884848B2 (en) * | 2005-05-25 | 2011-02-08 | Ginther Mark E | Viewing environment and recording system |
US20070038945A1 (en) * | 2005-08-10 | 2007-02-15 | Miller Jacob J | System and method allowing one computer system user to guide another computer system user through a remote environment |
TW200718457A (en) * | 2005-08-18 | 2007-05-16 | Aruze Corp | Gaming machine and terminal device included in gaming machine |
KR100719097B1 (en) | 2005-11-03 | 2007-05-28 | (주)이즈미디어 | Camera arrangement method image acquisition system |
US8125481B2 (en) | 2008-03-21 | 2012-02-28 | Google Inc. | Lightweight three-dimensional display |
US8886669B2 (en) * | 2008-03-21 | 2014-11-11 | Trimble Navigation Limited | File access via conduit application |
US8730863B2 (en) * | 2008-09-09 | 2014-05-20 | The Charles Stark Draper Laboratory, Inc. | Network communication systems and methods |
JP5604921B2 (en) * | 2010-03-19 | 2014-10-15 | セイコーエプソン株式会社 | Image display system, image display method, and image display apparatus |
US8863208B2 (en) | 2012-06-18 | 2014-10-14 | Micropower Technologies, Inc. | Synchronizing the storing of streaming video |
US9497380B1 (en) | 2013-02-15 | 2016-11-15 | Red.Com, Inc. | Dense field imaging |
US9294757B1 (en) | 2013-03-15 | 2016-03-22 | Google Inc. | 3-dimensional videos of objects |
EP2887328B1 (en) * | 2013-12-19 | 2016-04-20 | Axis AB | Monitoring devices slidably mounted on a rail releasably locked to predetermined positions |
US20150195644A1 (en) * | 2014-01-09 | 2015-07-09 | Microsoft Corporation | Structural element for sound field estimation and production |
US20160335862A1 (en) * | 2014-01-17 | 2016-11-17 | Alexander Jaber | Tactical personal surveillance and monitoring system |
US10033992B1 (en) | 2014-09-09 | 2018-07-24 | Google Llc | Generating a 3D video of an event using crowd sourced data |
US9706187B2 (en) * | 2014-10-06 | 2017-07-11 | Electronics And Telecommunications Research Institute | Camera rig for shooting multi-view images and videos and image and video processing method for use with same |
ES2538012B1 (en) * | 2014-10-23 | 2016-03-30 | Digital Centre, S.L. | MULTI-CAMERA PHOTOMATON OF THREE-DIMENSIONAL EFFECT AND OPERATING PROCEDURE OF THE SAME |
US10178325B2 (en) | 2015-01-19 | 2019-01-08 | Oy Vulcan Vision Corporation | Method and system for managing video of camera setup having multiple cameras |
US9848212B2 (en) * | 2015-07-10 | 2017-12-19 | Futurewei Technologies, Inc. | Multi-view video streaming with fast and smooth view switch |
EP3369073A4 (en) | 2015-10-29 | 2019-06-26 | OY Vulcan Vision Corporation | Video imaging an area of interest using networked cameras |
DE102016223171A1 (en) | 2016-11-23 | 2018-05-24 | Robert Bosch Gmbh | Method and system for detecting a raised object located within a parking lot |
US10778906B2 (en) * | 2017-05-10 | 2020-09-15 | Grabango Co. | Series-configured camera array for efficient deployment |
NO346392B1 (en) | 2020-08-05 | 2022-07-04 | Muybridge As | Multiple camera sensor system and method of providing a target image by a virtual camera |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5495576A (en) * | 1993-01-11 | 1996-02-27 | Ritchey; Kurtis J. | Panoramic image based virtual reality/telepresence audio-visual system and method |
US5729471A (en) * | 1995-03-31 | 1998-03-17 | The Regents Of The University Of California | Machine dynamic selection of one video camera/image of a scene from multiple video cameras/images of the scene in accordance with a particular perspective on the scene, an object in the scene, or an event in the scene |
US5850352A (en) * | 1995-03-31 | 1998-12-15 | The Regents Of The University Of California | Immersive video, including video hypermosaicing to generate from multiple video views of a scene a three-dimensional video mosaic from which diverse virtual video scene images are synthesized, including panoramic, scene interactive and stereoscopic images |
US6020931A (en) * | 1996-04-25 | 2000-02-01 | George S. Sheng | Video composition and position system and media signal communication system |
US6154251A (en) * | 1994-12-21 | 2000-11-28 | Taylor; Dayton V. | System for producing time-independent virtual camera movement in motion pictures and other media |
US6208379B1 (en) * | 1996-02-20 | 2001-03-27 | Canon Kabushiki Kaisha | Camera display control and monitoring system |
US6522325B1 (en) * | 1998-04-02 | 2003-02-18 | Kewazinga Corp. | Navigable telepresence method and system utilizing an array of cameras |
Family Cites Families (41)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4355328A (en) | 1981-02-23 | 1982-10-19 | The United States Of America As Represented By The Secretary Of The Navy | 360 Degree closed circuit television system |
US4463380A (en) | 1981-09-25 | 1984-07-31 | Vought Corporation | Image processing system |
US4847700A (en) | 1987-07-16 | 1989-07-11 | Actv, Inc. | Interactive television system for providing full motion synched compatible audio/visual displays from transmitted television signals |
US5023725A (en) | 1989-10-23 | 1991-06-11 | Mccutchen David | Method and apparatus for dodecahedral imaging system |
GB9001468D0 (en) | 1990-01-23 | 1990-03-21 | Sarnoff David Res Center | Computing multiple motions within an image region |
US5130794A (en) | 1990-03-29 | 1992-07-14 | Ritchey Kurtis J | Panoramic display system |
US5257349A (en) | 1990-12-18 | 1993-10-26 | David Sarnoff Research Center, Inc. | Interactive data visualization with smart object |
US5187571A (en) | 1991-02-01 | 1993-02-16 | Bell Communications Research, Inc. | Television system for displaying multiple views of a remote location |
US5185808A (en) | 1991-06-06 | 1993-02-09 | Eastman Kodak Company | Method for merging images |
GB9119964D0 (en) | 1991-09-18 | 1991-10-30 | Sarnoff David Res Center | Pattern-key video insertion |
US5259040A (en) | 1991-10-04 | 1993-11-02 | David Sarnoff Research Center, Inc. | Method for determining sensor motion and scene structure and image processing system therefor |
US5724091A (en) | 1991-11-25 | 1998-03-03 | Actv, Inc. | Compressed digital data interactive program system |
US5253168A (en) * | 1991-12-06 | 1993-10-12 | Berg Jacqueline L | System for creative expression based on biofeedback |
US5343871A (en) * | 1992-03-13 | 1994-09-06 | Mindscope Incorporated | Method and apparatus for biofeedback |
US5325449A (en) | 1992-05-15 | 1994-06-28 | David Sarnoff Research Center, Inc. | Method for fusing images and apparatus therefor |
ES2151550T3 (en) | 1993-06-04 | 2001-01-01 | Sarnoff Corp | SYSTEM AND PROCEDURE FOR ELECTRONIC STABILIZATION OF IMAGES. |
US5497188A (en) | 1993-07-06 | 1996-03-05 | Kaye; Perry | Method for virtualizing an environment |
GB2282505A (en) | 1993-09-23 | 1995-04-05 | Sharp Kk | Three dimensional imaging apparatus |
US5562572A (en) * | 1995-03-10 | 1996-10-08 | Carmein; David E. E. | Omni-directional treadmill |
US5999641A (en) | 1993-11-18 | 1999-12-07 | The Duck Corporation | System for manipulating digitized image objects in three dimensions |
JPH07222202A (en) | 1994-02-02 | 1995-08-18 | Rohm Co Ltd | Stereoscopic vision camera |
US5537141A (en) | 1994-04-15 | 1996-07-16 | Actv, Inc. | Distance learning system providing individual television participation, audio responses and memory for every student |
US5508734A (en) | 1994-07-27 | 1996-04-16 | International Business Machines Corporation | Method and apparatus for hemispheric imaging which emphasizes peripheral content |
US5632007A (en) | 1994-09-23 | 1997-05-20 | Actv, Inc. | Interactive system and method for offering expert based interactive programs |
US5598208A (en) * | 1994-09-26 | 1997-01-28 | Sony Corporation | Video viewing and recording system |
US5600368A (en) | 1994-11-09 | 1997-02-04 | Microsoft Corporation | Interactive television system and method for viewer control of multiple camera viewpoints in broadcast programming |
US5649032A (en) | 1994-11-14 | 1997-07-15 | David Sarnoff Research Center, Inc. | System for automatically aligning images to form a mosaic image |
US5644694A (en) * | 1994-12-14 | 1997-07-01 | Cyberflix Inc. | Apparatus and method for digital movie production |
US5703961A (en) * | 1994-12-29 | 1997-12-30 | Worldscape L.L.C. | Image transformation and synthesis methods |
US5581629A (en) | 1995-01-30 | 1996-12-03 | David Sarnoff Research Center, Inc | Method for estimating the location of an image target region from tracked multiple image landmark regions |
US5745305A (en) | 1995-04-28 | 1998-04-28 | Lucent Technologies Inc. | Panoramic viewing apparatus |
GB2301216A (en) * | 1995-05-25 | 1996-11-27 | Philips Electronics Uk Ltd | Display headset |
US5963664A (en) | 1995-06-22 | 1999-10-05 | Sarnoff Corporation | Method and system for image combination using a parallax-based technique |
US5682196A (en) | 1995-06-22 | 1997-10-28 | Actv, Inc. | Three-dimensional (3D) video presentation system providing interactive 3D presentation with personalized audio responses for multiple viewers |
JPH11509064A (en) | 1995-07-10 | 1999-08-03 | サーノフ コーポレイション | Methods and systems for representing and combining images |
US5706416A (en) | 1995-11-13 | 1998-01-06 | Massachusetts Institute Of Technology | Method and apparatus for relating and combining multiple images of the same scene or object(s) |
DE69632384T2 (en) * | 1995-12-19 | 2005-05-04 | Canon K.K. | Apparatus and method for controlling a plurality of remote cameras |
US5708469A (en) * | 1996-05-03 | 1998-01-13 | International Business Machines Corporation | Multiple view telepresence camera system using a wire cage which surroundss a plurality of movable cameras and identifies fields of view |
US6084979A (en) | 1996-06-20 | 2000-07-04 | Carnegie Mellon University | Method for creating virtual reality |
US6151009A (en) | 1996-08-21 | 2000-11-21 | Carnegie Mellon University | Method and apparatus for merging real and synthetic images |
US6384820B2 (en) * | 1997-12-24 | 2002-05-07 | Intel Corporation | Method and apparatus for automated dynamics of three-dimensional graphics scenes for enhanced 3D visualization |
-
1999
- 1999-04-01 AU AU34646/99A patent/AU761950B2/en not_active Ceased
- 1999-04-01 CN CN99805539A patent/CN1298607A/en active Pending
- 1999-04-01 EP EP99916295A patent/EP1068730A4/en active Pending
- 1999-04-01 KR KR1020007011208A patent/KR20010074470A/en not_active Application Discontinuation
- 1999-04-01 US US09/283,413 patent/US6535226B1/en not_active Expired - Lifetime
- 1999-04-01 WO PCT/US1999/007255 patent/WO1999052288A1/en not_active Application Discontinuation
- 1999-04-01 IL IL13880899A patent/IL138808A0/en unknown
- 1999-04-01 BR BR9909429-0A patent/BR9909429A/en not_active IP Right Cessation
- 1999-04-01 CA CA002326731A patent/CA2326731A1/en not_active Abandoned
- 1999-04-01 JP JP2000542921A patent/JP2004502321A/en active Pending
- 1999-04-01 MX MXPA00009686A patent/MXPA00009686A/en unknown
-
2003
- 2003-03-17 US US10/391,509 patent/US20030229735A1/en not_active Abandoned
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5495576A (en) * | 1993-01-11 | 1996-02-27 | Ritchey; Kurtis J. | Panoramic image based virtual reality/telepresence audio-visual system and method |
US6154251A (en) * | 1994-12-21 | 2000-11-28 | Taylor; Dayton V. | System for producing time-independent virtual camera movement in motion pictures and other media |
US5729471A (en) * | 1995-03-31 | 1998-03-17 | The Regents Of The University Of California | Machine dynamic selection of one video camera/image of a scene from multiple video cameras/images of the scene in accordance with a particular perspective on the scene, an object in the scene, or an event in the scene |
US5850352A (en) * | 1995-03-31 | 1998-12-15 | The Regents Of The University Of California | Immersive video, including video hypermosaicing to generate from multiple video views of a scene a three-dimensional video mosaic from which diverse virtual video scene images are synthesized, including panoramic, scene interactive and stereoscopic images |
US6208379B1 (en) * | 1996-02-20 | 2001-03-27 | Canon Kabushiki Kaisha | Camera display control and monitoring system |
US6020931A (en) * | 1996-04-25 | 2000-02-01 | George S. Sheng | Video composition and position system and media signal communication system |
US6522325B1 (en) * | 1998-04-02 | 2003-02-18 | Kewazinga Corp. | Navigable telepresence method and system utilizing an array of cameras |
US20030132951A1 (en) * | 1998-04-02 | 2003-07-17 | Scott Sorokin | Navigable telepresence method and system utilizing an array of cameras |
US20060146123A1 (en) * | 1998-04-02 | 2006-07-06 | Scott Sorokin | Navigable telepresence method and systems utilizing an array of cameras |
Cited By (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US7349117B2 (en) * | 2000-07-31 | 2008-03-25 | Hewlett-Packard Development Company, L.P. | Method for introduction and linking of imaging appliances |
US20050179942A1 (en) * | 2000-07-31 | 2005-08-18 | Stavely Donald J. | Method for introduction and linking of imaging appliances |
US8937646B1 (en) * | 2011-10-05 | 2015-01-20 | Amazon Technologies, Inc. | Stereo imaging using disparate imaging devices |
US9325968B2 (en) | 2011-10-05 | 2016-04-26 | Amazon Technologies, Inc. | Stereo imaging using disparate imaging devices |
WO2014189927A3 (en) * | 2013-05-20 | 2015-01-15 | Sebring Steven | Systems and methods for producing visual representations of objects |
US9123172B2 (en) * | 2013-05-20 | 2015-09-01 | Steven Sebring | Systems and methods for producing visual representations of objects |
US20140341484A1 (en) * | 2013-05-20 | 2014-11-20 | Steven Sebring | Systems and methods for producing visual representations of objects |
US9473707B2 (en) | 2013-05-20 | 2016-10-18 | Durst Sebring Revolution, Llc | Systems and methods for producing visual representations of objects |
EP3062148A1 (en) * | 2015-02-24 | 2016-08-31 | Nokia Technologies Oy | Device with an adaptive camera array |
US9996934B2 (en) | 2015-02-24 | 2018-06-12 | Nokia Technologies Oy | Device with an adaptive camera array |
USD781948S1 (en) | 2015-12-03 | 2017-03-21 | Durst Sebring Revolution, Llc | Photographic imaging system |
USD782559S1 (en) | 2015-12-03 | 2017-03-28 | Durst Sebring Revolution, Llc | Photo booth |
USD798936S1 (en) | 2015-12-03 | 2017-10-03 | Durst Sebring Revolution, Llc | Photo booth |
USD812671S1 (en) | 2015-12-03 | 2018-03-13 | Durst Sebring Revolution, Llc | 3D imaging system |
USD822746S1 (en) | 2016-02-05 | 2018-07-10 | Durst Sebring Revolution, Llc | Photo booth |
Also Published As
Publication number | Publication date |
---|---|
WO1999052288A1 (en) | 1999-10-14 |
CN1298607A (en) | 2001-06-06 |
JP2004502321A (en) | 2004-01-22 |
US6535226B1 (en) | 2003-03-18 |
KR20010074470A (en) | 2001-08-04 |
AU761950B2 (en) | 2003-06-12 |
AU3464699A (en) | 1999-10-25 |
BR9909429A (en) | 2001-09-11 |
MXPA00009686A (en) | 2004-09-10 |
CA2326731A1 (en) | 1999-10-14 |
EP1068730A1 (en) | 2001-01-17 |
EP1068730A4 (en) | 2005-03-30 |
IL138808A0 (en) | 2001-10-31 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US6535226B1 (en) | Navigable telepresence method and system utilizing an array of cameras | |
US6522325B1 (en) | Navigable telepresence method and system utilizing an array of cameras | |
US6675386B1 (en) | Apparatus for video access and control over computer network, including image correction | |
Jouppi | First steps towards mutually-immersive mobile telepresence | |
US8913143B2 (en) | Panoramic experience system and method | |
CA2302616C (en) | Apparatus for video access and control over computer network, including image correction | |
US20060114251A1 (en) | Methods for simulating movement of a computer user through a remote environment | |
EP1224798A2 (en) | Method and system for comparing multiple images utilizing a navigable array of cameras | |
US11703942B2 (en) | System and method for interactive 360 video playback based on user location | |
JPH08279054A (en) | Video generation and display system | |
WO1995019093A1 (en) | Viewing imaged objects from selected points of view | |
JPH1070740A (en) | Stereoscopic camera and video transmission system | |
JPH08279058A (en) | Video generation and display system | |
JPH08279055A (en) | Video generation and display system | |
JP2003060996A (en) | Broadcast device, receiver and recording medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |