WO2001010128A1 - Instant video messenger - Google Patents

Instant video messenger Download PDF

Info

Publication number
WO2001010128A1
WO2001010128A1 PCT/US2000/021214 US0021214W WO0110128A1 WO 2001010128 A1 WO2001010128 A1 WO 2001010128A1 US 0021214 W US0021214 W US 0021214W WO 0110128 A1 WO0110128 A1 WO 0110128A1
Authority
WO
WIPO (PCT)
Prior art keywords
computer
receiver
video
sender
video message
Prior art date
Application number
PCT/US2000/021214
Other languages
French (fr)
Inventor
Gad Liwerant
Christopher Dodge
Guillaume Boissiere
Original Assignee
Videoshare, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Videoshare, Inc. filed Critical Videoshare, Inc.
Priority to AU65156/00A priority Critical patent/AU6515600A/en
Publication of WO2001010128A1 publication Critical patent/WO2001010128A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81Monomedia components thereof
    • H04N21/816Monomedia components thereof involving special video data, e.g 3D video
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/04Real-time or near real-time messaging, e.g. instant messaging [IM]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80Responding to QoS
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21Server components or server architectures
    • H04N21/222Secondary servers, e.g. proxy server, cable television Head-end
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/231Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion
    • H04N21/23106Content storage operation, e.g. caching movies for short term storage, replicating data over plural servers, prioritizing data for deletion involving caching operations
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/25Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
    • H04N21/254Management at additional data server, e.g. shopping server, rights management server
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/478Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/16Analogue secrecy systems; Analogue subscription systems
    • H04N7/173Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
    • H04N7/17309Transmission or handling of upstream communications
    • H04N7/17336Handling of requests in head-ends

Definitions

  • This invention relates generally to transmitting video files over a network. More particularly, the invention relates to transmitting streaming video files over a network without the intervention of a server computer in the streaming process.
  • Some of the problems that are associated with communications over a computer network operated according to the client/server model include, but are not limited to, the large expense of building, maintaining, and operating servers generally, and in particular the expense of installing and maintaining servers that perform functions related to data storage and hosting; the problems associated with loss of service if the server is inoperative, and degradation of service if the server is overloaded; delays, inconvenience, and bandwidth capacity issues associated with uploading information from a sender computer to the server for retransmission from the server to a recipient computer; and difficulties in increasing the scale of the network as the number of client computers increases. Because video files in particular tend to be quite large, the problems enumerated above can be particularly severe in handling video files in a client/server network.
  • the present invention provides a method and system of transmitting a video that mitigates the problems associated with the lengthy transmission time associated with a file attached to an e-mail, that eliminates the necessity to have multiple software packages to carry out the transmission, and that eliminates the necessity to invoke the services of a hosting computer for storing the video.
  • the method and system can be transparent to the user.
  • full motion video can be automatically prepared and transmitted by a sender from a sender computer to a viewer using a receiver computer, when the two computers are simultaneously connected to a computer network, including such networks as a global computer network, for example, the Internet or the World Wide Web ("Web").
  • IVM Instant Video Messenger
  • IVM is a rapid, asynchronous messaging system/software that supports symmetric, peer-to-peer communication of digital information (e.g., files containing video data such as files in the jpg, .avi, .mpeg, QuickTime, .wav, Windows Media Format, Real Media, and other formats) on a computer network, such as the Internet.
  • digital information e.g., files containing video data such as files in the jpg, .avi, .mpeg, QuickTime, .wav, Windows Media Format, Real Media, and other formats
  • Each user computer equipped with the IVM software is capable of sending and receiving streaming and non-streaming audio, streaming and non-streaming video, text, attachments, and meta-data.
  • the network structure of IVM reduces the burden on a central server (which traditionally has been used to host and stream video), as IVM is a peer-to-peer application and thus does not require a hosting facility.
  • a server can be used in an IVM system to perform functions that include, but are not limited to, monitoring which IVM users are on-line, providing communications driven by an entity that has permission to use the server (for example, an advertiser who has contracted for services), and hosting messages for users who have contracted for that service.
  • IVM supports an Instant Video Messenger Channel capability that allows for community building as well as affiliate opportunities.
  • the IVM distribution system allows for "self-replication.”
  • the invention features a method of sending a streaming video message over a network.
  • the method includes receiving at a server computer an indication that the operator of a sender computer intends to send a streaming video message to a receiver computer over a network, determining whether the receiver computer is online, and communicating from the server computer to the sender computer whether the receiver computer is online, wherein an indication that the receiver computer is online signals to the sender computer that streaming the video message directly from the sender computer to the receiver computer over the network is appropriate.
  • the invention can include a supervisory computer communication module operable on the server computer that performs the step of receiving at the server computer an indication that the operator of a sender computer intends to send a streaming video message to a receiver computer.
  • the supervisory computer communication module can also perform the step of determining whether the receiver computer is online, and can perform the step of communicating from the server computer to the sender computer whether the receiver computer is online.
  • the invention features a method of sending a streaming video message over a network.
  • the method includes sending to a server computer an indication that the operator of a sender computer intends to send a streaming video message to a receiver computer over a network, receiving at the sender computer an indication of whether the receiver computer is online, and in response to an indication that the receiver computer is online, streaming the video message from the sender computer to the receiver computer over the network independent of the operation of the server computer.
  • the step of streaming the video message from the sender computer to the receiver computer includes providing on the sender computer a computer communication module operable on each of the sender computer and the receiver computer, and streaming the video message from the communication module of the sender computer to the receiver computer over the network independent of the operation of the server computer.
  • the step of providing a computer communication module operable on each of the sender computer and the receiver computer can include providing individual copies of the same communication module to at least one of the sender computer and the receiver computer, each copy operable on each of the sender computer and the receiver computer.
  • the method can include providing communication modules that are symmetric in operation.
  • the method can include providing communication modules that are modular, and can include providing communication modules that can determine the features present in another communication module operating on another computer.
  • the step of streaming the video message from the sender computer to the receiver computer over the network independent of the operation of the server computer can include performing the streaming operation using only communication modules present on the sender computer and the receiver computer.
  • the method can additionally include the step of, in response to an indication that the receiver computer is not online, optionally sending the video message from the sender computer to a host computer, for storage and later transmission to the receiver computer, independent of the operation of the server computer.
  • the video message can include video information.
  • the video message can additionally include information selected from audio information, a thumbnail image, a text identifying a subject, a message body text, a document attachment, a slide and meta-data.
  • the invention features a computer program recorded on a machine-readable medium.
  • the computer program is adapted to transmit and receive streaming video messages over a network independent of a server computer.
  • the computer program operates on a sender computer, it performs the steps of requesting from a server computer the status of a receiver computer, the receiver computer being the intended recipient of a video message, receiving an indication of the online status of the receiver computer, and in response to an indication that the receiver computer is online, streaming the video message from the computer program of the sender computer to the computer program of the receiver computer independent of the operation of the server computer.
  • the computer program when it is operating on a receiver computer, is further capable of determining that a video message is being sent by a sender computer.
  • the program is additionally capable of displaying to a viewer of the receiver computer information about the video message, the information comprising at least one of an identifier of the sender of the message, a subject of the message, and a title of the message.
  • the computer program when it is operating on a receiver computer, is capable of receiving the streaming video message, and displaying the streaming video message to a viewer of the receiver computer.
  • the step of receiving the streaming video message can include receiving the streaming video message, and presenting to a viewer of the receiver computer an opportunity to select one of accepting the streaming video message, and rejecting the streaming video message.
  • the step of receiving the streaming video message can include accepting a response of the viewer, and processing the message in accordance with the response of the viewer.
  • the step of accepting the streaming video message can include at least one of accepting the streaming video message for immediate display, and accepting the streaming video message for storage in a machine-readable memory, and for display at a later time.
  • the computer program when it is operating on a receiver computer, the computer program is capable of receiving the streaming video message is accomplished without the necessity of an action by a viewer of the computer.
  • the program can accomplish displaying the streaming video message without the necessity of an action by a viewer of the computer.
  • the computer program can perform displaying the video message by invoking the operation of a video display module.
  • the computer program can, in response to an indication that the receiver computer is not online, optionally send the video message from the communication module of the sender computer to a host computer, for storage and later transmission to the communication module of the receiver computer, independent of the operation of the server computer.
  • the video message can include video information.
  • the video message can further include information selected from audio information, a thumbnail image, a text identifying a subject, a message body text, a document attachment, a slide and meta-data.
  • the invention features a computer program recorded on a machine-readable medium.
  • the computer program is adapted to operate a server computer and to provide server functionality including database connectivity and communication protocols.
  • the computer program When operating on a server computer, the computer program performs the steps of determining the online status of the receiver computer, and providing an indication of the status to the sender computer, such that in response to an indication that the receiver computer is online, streaming the video message from the computer program of the sender computer to the computer program of the receiver computer is accomplished independent of the operation of the server computer.
  • the computer program when in operation, is capable of accepting a request from a sender computer about the status of a receiver computer, the receiver computer being the intended recipient of a video message.
  • the computer program when in operation, is capable of providing communication protocols for exchanging information between interconnected computers.
  • the computer program is capable of providing a database in which to store information, including information about sender and receiver computers authorized to communicate with the server computer.
  • FIG. 1 shows the relationship between the time that elapses between the sending and the receiving of a message ("immediacy") and data delivery solutions of some technology such as that described in copending U.S.S.N. 09/497,587.
  • FIG. 2A depicts the interactions in a system including a sender computer, a server computer, and a receiver, such as the system described in copending U.S.S.N. 09/497,587.
  • FIG. 2B presents in overview a flowchart showing the steps involved in the interaction depicted in FIG. 2A.
  • FIG. 3 shows the relationship between immediacy and data delivery solutions of FIG. 1 and additionally including an embodiment of a method and system according to the invention.
  • FIG. 4A depicts an embodiment of the interactions between and among a sender computer, a server computer, and a receiver computer in a network according to the invention.
  • FIG. 4B presents in overview a flowchart showing the steps involved in the interaction depicted in FIG. 4A.
  • FIG. 5 depicts another embodiment of the interactions between and among a sender computer, a server computer, and a receiver computer in a network according to the invention.
  • FIG. 6 depicts an embodiment of the interactions between and among a sender computer, a server computer, and a receiver computer in a network having Instant Video Messenger Channels according to the invention.
  • FIG. 7 depicts an embodiment of the interactions between and among a sender computer, a server computer, and a receiver computer in a network having Instant Video Messenger Channels and the capacity to store message data according to the invention.
  • FIG. 8 is a flow diagram that depicts schematically an alternative embodiment of the messaging systems and methods, according to the principles of the invention.
  • FIG. 9 is a schematic embodiment of a process and system according to the copending U.S.S.N. 09/497,587.
  • FIG. 10 is an embodiment of a system according to the copending U.S.S.N. 09/497,587, including the interactions and interrelationships within the system.
  • FIG. 11 is a functional block and flow diagram of an embodiment of the copending U.S.S.N. 09/497,587.
  • FIG. 12 is a login screen on a user's computer, in one embodiment of the invention.
  • FIG. 13 is a record/playback screen as seen by the user, in accordance with an embodiment of the invention.
  • FIG. 14A is a flow diagram of an embodiment of the invention in which software automates a number of steps in connection with the uploading of a video segment.
  • FIG. 14B is a flow diagram of another embodiment of the invention in which software automates a number of steps in connection with the uploading of a video segment.
  • FIG. 14C is a flow diagram of an embodiment of the invention in which software automates a number of steps in connection with the formatting of a video segment.
  • FIG. 14D shows the relationship of some of the files created in the flow diagram of FIG. 14C.
  • FIG. 14E is a flow diagram of a method by which an optimally formatted video segment is sent to a user according to the invention.
  • FIG. 15 is a screen as seen by the user, the screen indicating that file processing is occurring.
  • FIG. 16 is a video playback screen seen by the user.
  • FIG. 17 is a screen used by the user to control the status of a video queue.
  • FIG. 18 is a screen used by the user to control the operational settings of equipment associated with the user's computer.
  • a one-dimensional measure termed "immediacy” is an indication of the amount of time (possibly related to a level of effort) that elapses between a user sending a Video Message and a receiver receiving the Video Message.
  • FIG. 1 depicts such a relationship between a method of sending messages called NetMeeting, that is known in the prior art, and another technology that is described in copending U.S.S.N. 09/497,587.
  • Real-time meaning an absolute minimum lag time between video capture and viewing.
  • Definered-time meaning that the amount of lag time between video capture and viewing is unknown.
  • the lag time can in principle be infinite (i.e., the video may never be viewed), for example because the intended recipient has no interest in viewing the video.
  • NetMeeting is software that is available in version 3 as part of Windows 2000 from Microsoft Corporation. Information about Netmeeting can be found at Microsoft's website, www.microsoft.com. Netmeeting provides an immediate coupling between the sender of information and the receiver of the information. Once a messaging connection is made using the NetMeeting technology, the receiver renders the video sent by the sender. The receiver cannot turn off the video (or "opt-out" of viewing the video) except by completely terminating the NetMeeting session.
  • the NetMeeting technology has the problematic features of being computationally intensive and requiring a large transmission bandwidth.
  • the NetMeeting technology requires that the video must be processed in real time, because it is analogous to two- way television broadcasting over a network.
  • the real time processing includes capturing the video, compressing the video, streaming the video, decompressing the video and rendering the video. Because all of these processes must be performed in real time, the NetMeeting real-time technology is beyond the capacity of the computers available to many non-professional or consumer users.
  • a receiver of the video message In a "deferred-time" application such as the VideoShare Producer which is described in the copending U.S.S.N. 09/497,587, a receiver of the video message must formally acknowledge the receipt of the message and actively initiate the downloading and viewing process.
  • a potential viewer must activate his or her email browser, receive a new email message announcing the availability of the video, read the subject body, have interest in the email body content, activate the associated tag link (for example, by clicking on it), be connected to the World Wide Web ("Web"), and activate a "play" button in a viewer software application.
  • Web World Wide Web
  • the "Sender” uses a sender computer provided with one set of software tools to capture, edit, and otherwise generate streaming video content.
  • Prior art software products that perform the "sending” operations include the Microsoft Media Encoder and the Real G2 Producer.
  • the VideoShare Producer of the copending U.S.S.N. 09/497,587 is another system and method for sending a streaming video from a sender to a server and then on to a viewer.
  • the "Receiver” uses a receiver computer that is provided with a different set of software tools to receive, manipulate, and display the video.
  • Prior art software products that implement the "receiver” functionality include the Windows Media Player and Real G2 Player.
  • the sending computer In asymmetric systems, the sending computer often cannot determine important features of the receiving computer.
  • the receiving computer also often cannot determine important features of the sending computer.
  • the sending computer generally does not have any information about the kind of software that the receiver computer has available.
  • the sending computer generally cannot determine the capabilities of the receiver computer, such as CPU power or available network connection speeds.
  • the sending computer generally knows nothing about the default control setting of the receiving computer as regards security protocols, preferred formats, and the like.
  • the application running on the sending computer and the application running on the receiving computer do not communicate directly.
  • the "Sender" transmits a message for receipt by the "Receiver," without assurance that the "Receiver" is available to receive the message, or that the "Receiver” can decipher the message that it does receive.
  • FIG. 2A depicts the structure of the VideoShare system and method as represented by the VideoShare Producer embodiment.
  • a "sender A" using sender computer 302 generates or obtains a video that the "sender” wishes to share with or publish to a "receiver B.”
  • sender computer 302 has a copy of the VideoShare Producer software that is used to obtain, prepare and transmit the video.
  • the sender computer 302 of "sender A” appears to have a direct connection to the receiver computer 304 of "receiver B.”
  • the dotted arrow from “A” to “B” denotes the apparent connection.
  • the sender computer 302 of "sender A” transmits the video segment to a server 306 that is operated by a hosting service, such as the VideoShare service. This upload is denoted by the arrow extending from the sender computer 302 to the VideoShare server 306.
  • the VideoShare server 306 in turn notifies the receiver computer 304 of "receiver B" that a video is available for viewing.
  • the VideoShare server 306 Upon an acknowledgment from B that the video is desired by B, the VideoShare server 306 streams the video to the receiver computer 304 for viewing by B.
  • the streaming of the video is denoted by the arrow extending from the VideoShare server 306 to the receiver computer 304.
  • the receiver computer 304 uses software designed to display a streaming video, such as the Microsoft Media Player, for example, in order to display the video to "receiver B."
  • a video message for example, a Video Greeting Card
  • the apparent connection is between the computers 302, 304 of A and B directly.
  • the actual data connection is between A's sender computer 302 and the VideoShare server 306 and then between the server 306 and B's receiver computer 304.
  • Sender A records a video message using sender computer 302 and associated hardware and software.
  • Sender A uploads the video message to VideoShare Server 306.
  • VideoShare Server 306 notifies Receiver B, who operates receiver computer 304, that a video message is available.
  • Receiver B acknowledges the notification (and requests delivery of the message).
  • the VideoShare Server 306 streams the video message to receiver computer 304, where Receiver B views the message.
  • the apparent transaction that of Sender A sending a video message to Receiver B, does not in fact occur directly, but only through the intermediation of the VideoShare Server 306, which is called on to both receive the video message and send on the video message.
  • the operational costs, computational load, and bandwidth requirements to service a growing user base become ever larger as the number of users and the amount of traffic increases.
  • videos are uploaded to, stored on, and streamed from a server system, such as server computer 306, which can comprise one hosting facility at one location or multiple facilities at multiple locations. Each hosting facility can service a large number of concurrent users.
  • the VideoShare Producer system and method can be considered topologically to be a "Star Network" where all clients machines that are operating in the system are in constant communication with a node, and all video traffic passes through the server.
  • Such topologies have three main limitations.
  • the number of interactions is given by the theory of the number of combinations of N things, where N is an integer greater than 1 , taken two at a time. This number increases at a rate of the order of the square of the number N of client computers and their users. As the load grows, the server's finite resources must be ever more thinly distributed to each client computer. In such a case, increased success in sharing video over a network requires more infrastructure to continue to perform the necessary tasks at an acceptable level of performance.
  • a failure on the single node of the network causes a large disruption of service to the connections of many client computers to the server. It is not uncommon that a single server problem can create communication problems for tens-of-thousands of client computers (and their users) at once. In addition, disturbances of a server that are less than complete failures can seriously degrade the service of many, for example by lengthening the time required to process requests that pass through the server.
  • the costs to build, to operate, and to maintain the server can be large.
  • the costs to maintain the ability of the server to perform at an acceptable level can escalate quickly.
  • a network that employs a server to handle data that flows between sender computers and receiver computers will in general require additional storage capacity in order to hold the messages that are in transmission to receiver computers that cannot immediately accept the messages. This situation involves having one or more caches or storage such as hard disks to hold the messages for later transmission from the server to the intended recipient computer.
  • the server will then have to deal with the overhead of storing and retrieving messages in addition to the demands of receiving and sending the messages. If the server is also responsible for routing the messages sent between a sender computer and a receiver computer, the server will spend time communicating with other computers to determine whether sufficient bandwidth exists on a particular connection to carry the intended message, as well as time spent by the server in sending the message itself.
  • Extensibility refers to the ease or difficulty (and in the extreme, the very possibility) of expanding a system, for example by adding additional capacity and/or additional operational features.
  • extensibility is accomplished through different versions that are sold separately as complete packages or through upgrades. The user often is charged for upgrades to software packages.
  • version 1.0 of a word processor generally has fewer features than version 2.0 of the same software.
  • the end user who wants to upgrade a software package must install the new version of software manually, typically either through an Internet download or through a purchase.
  • COM Component Object Model
  • Runtime binding means that when the software application is run, a number of software objects, each comprising one core functionality, can become active and can create connections to one another.
  • a particular object can be replaced with a different version of that object, with an associated change only in the function that the particular object provides. Therefore, in the Component Object Model, a software release is a collection of individual, replaceable, objects, rather than a single monolithic piece of software that must be reassembled in order to change some feature of the code.
  • API Application Programming Interface
  • the present invention relates to methods and systems that provide advantages in Immediacy, Symmetry, Network Structure, Caching & Routing, and Extensibility.
  • Immediacy is a registered trademark of Microsoft Corporation.
  • Symmetry is a registered trademark of Microsoft Corporation.
  • Network Structure is a registered trademark of Microsoft Corporation.
  • Caching & Routing is a registered trademark of Microsoft Corporation.
  • Extensibility is a registered trademark of Microsoft Corporation.
  • FIG. 3 shows the relationship of an embodiment of the invention to the prior art NetMeeting technology and to the VideoShare Producer technology of the copending U.S.S.N. 09/497,587.
  • the embodiment is called the Instant Video Messenger (IVM).
  • IVM falls into an intermediate position along the horizontal "immediacy" axis because the time for delivery from Sender A to Receiver B may range from nearly instantaneous to a lengthy interval, as will be described in more detail below.
  • IVM is neither “Real-time” nor is it fully “Deferred-time”.
  • IVM allows content creators the ability (and the possibility) to send streaming video content with little delay to viewers with few chances to "opt-out.”
  • IVM can be used in conjunction with both free personal use or paid-for professional services. Svmmetry
  • VideoShare Producer described in copending U.S.S.N. 09/497,587
  • functionality can be added to the Producer (e.g., video sending) and the web site (e.g., storage, notification, and the like).
  • functionality generally cannot be added to the playback features that are available to a user who employs a third party playback application, such as the Windows Media Player, for example.
  • a communication link that exists when a new task begins may have to slow down to allow the computer's CPU to have an opportunity to handle the new task, and conversely, when a task ends, the CPU is freed of a requirement to service the terminating task, and can devote more effort to the communication, link which may then be capable of speeding up.
  • the network configuration that supports the Instant Video Messenger is completely symmetric.
  • the software running on the sender computer is exactly the same as the software running on the receiver computer, with the possible exception for differences due to version updates.
  • the "Send/Receive" package can be prepared in a modular form comprising one or more Dynamic Link Library (.DLL) files, so that updating an installed copy of the software can be carried out by replacing an older version of a .DLL file with the most current version.
  • the Instant Video Messenger is able to address the "knowledge” issues (i.e., the sender computer "knows” about the software operating on the receiver computer), because both computers operate the same software.
  • FIG. 4A shows a topology in accordance with one embodiment of the invention.
  • the IVM client software that operates on a sender computer 302 has the capacity to transfer the contents of a video message directly to other IVM client software operating on a receiver computer 304.
  • Each Sender/Receiver is capable of streaming video itself, without reliance on a network centralized server.
  • servers In the IVM system servers have a more limited role than in the traditional client/server network model.
  • servers such as the server 406, are responsible for database transactions, which require limited bandwidth and computational power in comparison to the higher bandwidth and computational requirements for streaming video from a server, in the traditional client server model.
  • the server 406 has only limited database requirements. Such limited requirements for transmission bandwidth and for limited database manipulation simplify the replication of the server 406, because the server 406 need not be expensive or complicated. Ease of replication of the server 406 is useful to provide redundancy in case of critical operational failure of the server 406, and to allow the addition of more such servers as the IVM system grows (i.e., more IVM client computers are added) and user demand for the IVM service/system increases.
  • the embodiment called the Instant Video Messenger does not rely on a video streaming center 306.
  • FIG. 4B shows the steps in the process in overview in flowchart 401.
  • Sender A using sender computer 302, asks the IVM Server 406 if Receiver B is online via receiver computer 304. IVM Server 406 responds that Receiver B is online.
  • Sender A records a video message using sender computer 302 and associated hardware and software.
  • Sender A using sender computer 302, notifies Receiver B that a video message is ready for transmission to Receiver B via receiver computer 304.
  • Receiver B acknowledges the notification (and requests delivery of the message).
  • Sender A using sender computer 302, streams the video message to receiver computer 304, where Receiver B views the message.
  • the apparent transaction that of Sender A sending a video message to Receiver B, occurs directly without the intermediation of the IVM Server 406, which not called on to either receive the video message or send on the video message.
  • the exemplary process that occurs as described in FIG. 4A is distinct from the process as described in FIG. 2A, which requires the intermediation of the VideoShare Server 306 .
  • the Instant Video Messenger dynamically routes video messages across its network.
  • the IVM server 406 can determine whether any particular sender computer 302 and/or receiver computer 304 is/are on-line and available to send or receive video information (e.g., video files).
  • the IVM server 406 also can re-route messages if network (e.g., Internet) traffic volume creates a problem with transmission between the sender computer 302 and the receiver computer 304.
  • the IVM system can employ nodes that can store the messages that pass through them, enabling archiving applications, as described in more detail below.
  • a pool of video content can automatically build up by using such archiving applications.
  • a node used to store such content can later be used to redistribute the material.
  • the Instant Video Messenger uses the COM technology to provide extensibility.
  • a new version of an object that uses the COM technology is developed, a user who has the old version is notified of the availability of the software object upgrade.
  • the user can also be notified of the availability of new objects that provide additional functionality.
  • VideoShare has created a video effect COM API for use by software developers. This API provides an interface for the addition of new and improved video effects. Examples of video effects that are available in expensive, professional quality systems, and that can be provided in the IVM software, include cross-fading, titling, and blue-screening. As objects that incorporate or provide such functionality become available from the development community, they can immediately be available to the IVM end user, without the need for purchasing an upgrade or downloading the entire IVM software package.
  • the IVM software thus provides the feature of extensibility.
  • COM objects that provide new features can be prepared for both the sender computer (i.e., generation of material for transmission that enables a new feature) and for the receiver computer (i.e., the presentation of the material of the new feature to the recipient).
  • the upgraded COM objects for the sender and for the receiver are released together.
  • new features include capabilities such as customized streams and meta-data that associate information with URL links and other ties to electronic commerce (e-commerce), as well as other hooks that enable a wider range of uses of video rather than just watching the images.
  • a hook is a code segment that presently performs no useful function other than being a placeholder, but that can be substituted with another COM object that performs a desired function.
  • An example of a hook is a subroutine that includes only the command "return.”
  • the Instant Video Messenger permits, but does not limit, users to include additional information streams that are synchronous with the audio/video content. These peripheral synchronous information streams will be sent simultaneously with the primary audio and video streams. For example, the user can use these additional synchronous information streams to display business presentation slides. When the audio and video stream reaches a particular time location in the stream, for example when a CEO describes his or her business, a business presentation slide appears on the Instant Video Message receiver's computer.
  • Additional information streams associates "clickable-regions" (i.e., regions that are linked to an action that is activated by using a pointing device such as a mouse to highlight the region and then activating or clicking a mouse button) within a video frame, enabling the receiver of the Instant Video Message to bring up additional information (e.g., text descriptions such as a help screen, hyperlink URLs, and the like) by merely clicking on different areas of the video frame while watching the video.
  • “clickable-regions” i.e., regions that are linked to an action that is activated by using a pointing device such as a mouse to highlight the region and then activating or clicking a mouse button
  • additional information e.g., text descriptions such as a help screen, hyperlink URLs, and the like
  • the Instant Video Messenger uses a direct peer-to-peer connection between the computer 302 of A and the computer 304 of B. If A wants to send B a video message, the computer 302 of A sends the video content directly to the computer 304 of B.
  • the video can be in either streaming or non-streaming format.
  • IVMS 406 an Instant Video Messenger Server 406 that coordinates the run-time state of all members of the IVM Network (IVMN).
  • IVMS 406 Instant Video Messenger Server 406
  • the Instant Video Messenger Server 406 is notified when a given IVM user is on- or off-line, where the user is located (e.g., the user's IP address), what the user's Internet connections are (e.g.
  • the Instant Video Messenger Server 406 informs the sender computer 302 either that the receiver computer 304 is on-line (and a transmission is thus possible) or that the receiver computer 304 is not on-line. However, the Instant Video Messenger Server 406 does not have to deal with the details of, or the control of, a subsequent transmission. Furthermore, as depicted in FIG. 5, the IVM Server 406 is capable of sending messages, for example video messages, on its own account to the computers 302, 304 of both users A and B.
  • the IVM Server 406 can send its own video message (or messages) to either or both of Sender A and Receiver B via each user's computer 302, 304.
  • a message can be, for example, video advertising, video promotion material, or other videos related to a particular theme as part of the services provided by the IVM Network.
  • the message sent by the IVM Server 406 appears in a window such as that depicted in FIG. 16, for example when either or both of the Sender A or the Receiver B is using Microsoft Outlook for e-mail service.
  • the Windows Media Player 910 is displayed with the message pre- loaded.
  • the recipient of this embedded message only needs to activate the play button 920 on the Windows Media Player to see the video segment, rather than going to a URL hyper-link.
  • the embodiment includes the conventional dialog boxes for entry of an email address for a recipient (box 902), a "carbon copy" ("cc") address (box 904), and a subject (box 906).
  • instructions are presented below the Windows Media Player 910 for the convenience of the recipient.
  • This embodiment can also be used by a Receiver B to receive and view a message sent by Sender A.
  • IVM Channels can represent areas of commonality of interest, such as hobbies, business interests, sports interests, and the like. IVM Channels can be used to form a convenient set of well-organized users, for receipt of video messages such as advertisement, product/service offerings, and special offers (for example, special saver airline fares). In one embodiment, this feature of the IVM network allows the IVM Server 406 to deliver instant video messages to a well-organized base of users. The users who are members of a Channel may also use the Channel to contact each other.
  • a sender using sender computer 302 who desires to send a video message to a receiver using receiver computer 304 communicates with one or more IVM Channels IVC1 602, IVC2 604, IVC3 606, IVC4 608 in the set of IVM Channels 600 located on the network, such as the Internet, to find out if the receiver computer 304 is on-line.
  • IVM Channels IVC1 602, IVC2 604, IVC3 606, IVC4 608 in the set of IVM Channels 600 located on the network, such as the Internet These communications are denoted by the bidirectional arrow that interconnects the sender A and receiver B with the set of IVM Channels 600. If the receiver computer 304 is on-line, the sender computer 302 can transmit the video message directly to the receiver computer 304, as denoted by the arrow from A to B.
  • the software when operating on the receiver computer 304, performs a number of useful actions.
  • the software can determine that the sender computer 302, or another computer, has sent a video message to it.
  • the software can receive and display the video message for the viewer of the receiver computer 304 without any activity or intervention by the viewer, and without the necessity of an action by a viewer of the receiver computer 304.
  • the software can invoke other software modules to accomplish the display of the video message, again without any activity or intervention by the viewer, and without the necessity of an action by a viewer of the receiver computer 304.
  • the software can issue a notification message to "Receiver B" (e.g., the operator or viewer of receiver computer 304) that a message is being sent to receiver computer 304.
  • the software can detect and display information about the sender and the message.
  • the software can display for "Receiver B" any or all of an identifier of the sender of the video message, a subject of the video message, and a title of the video message. This information can be used by "Receiver B" to decide if, and when, he or she wants to view the video message.
  • the software can additionally present to a viewer of the receiver computer 304 an opportunity to select one of accepting the streaming video message, and rejecting the streaming video message.
  • the software accepts the response of the viewer and processes the message in accordance with the response of the viewer.
  • the software can present to the viewer any combination of choices that includes rejecting the streaming video message and at least one of accepting the streaming video message for immediate display, and accepting the streaming video message for storage in a machine-readable memory, for display at a later time.
  • the software can, for example, direct the video to be saved on a machine-readable medium, such as a hard drive or any machine- readable medium that makes the video message available to the viewer at a later time.
  • the machine-readable medium can be, for example, a computer floppy disk, a computer hard drive, a magnetic tape or the like, a CD-ROM, computer memory such as static or dynamic RAM, ROM, PROM, EPROM or the like, and/or any other mechanism or medium for storing machine-readable files, instructions, data or software.
  • the machine-readable medium can be physically attached to a computer different from one on which the data may be used, or the software may operate.
  • an archival copy of software can reside on one computer and a copy can be copied to another computer, where the copy is executed or otherwise used.
  • a file containing data or information may reside on the same computer as the one that received the file, or on a different computer that stores the file for the convenience of the viewer.
  • Each IVM client computer maintains a personal IVM Contact List for the user of the computer, which is a list of other IVM users that are allowed to send messages to and to receive messages from the user.
  • a user employs one of several notification methods such as email or Instant Messaging forums such as ICQ to invite another individual to communicate.
  • the user can invite other individuals who have computers connected to the network, but who are unregistered with the IVM Network and whose computers do not have installed IVM software, to become users through an install executable that is sent in an email notification.
  • the receiver of the email message can agree to accept the message by opening the email data attachment.
  • This email data attachment contains the initial software codes needed to install the Instant Video Messenger software on the receiver's machine. His or her computer automatically executes the installation program, and the most recent version of the IVM software is installed and prepared for immediate use.
  • the installed software enables the transmission and receipt of message components, which can include, for example,
  • a video message includes at least a video stream.
  • FIG. 7 depicts an embodiment of a system and method that provides such functionality according to the invention.
  • One embodiment involves a connection with the VideoShare technology described in copending U.S.S.N. 09/497,587.
  • the sender computer 302 has a video message that the operator of the computer 302 wishes to transmit to the receiver computer 304. However, receiver computer 304 is offline.
  • One of the IVM Channel Servers 602, 604, 606, 608 informs the sender computer 302 of this situation.
  • the sender computer 302 is prompted by the IVM Channel Server 602 (for example) of the possibility of storing that video in the sender's VideoCenter supported by VideoShare server computer 710 for later delivery to the intended recipient.
  • the sender computer 302 can send the video message for storage at server computer 710.
  • Server computer 710 can transmit the video message, for example, via e-mail or via Web retrieval as described in copending
  • the Instant Video Messenger system includes a Instant Video Messenger Channel Provider Kit that enables third parties (e.g., affiliates, partners, co-developers and the like) to provide additional Instant Video Messenger Channels to the IVM community.
  • the Instant Video Messenger Channel Provider Kit includes all of the components for hosting an IVM Channel on a server computer.
  • the components include, but are not limited to, software, database connectivity, and communication protocols.
  • the IVM Channel Provider dedicates a server computer to manage all of the communications between the server and the computers of the channel users (or channel members).
  • the IVM Channel Provider installs database software such as Microsoft SQL 7 on the server to allow the server to handle a database that identifies the users and their computers.
  • the IVM Channel Provider uses the database software to manage information to be communicated to the channel members, and configures the IVM Channel Provider software with an IVM Channel name and description.
  • the IVM Channel Provider (or Host) can decide how it wishes to implement the service based on the expected demand for its IVM Channel services.
  • IVM Channels can be directed towards a particular customer segment, for example a segment based on thematic areas or common interests, such as for example, travel, auctions, or computers.
  • IVM users can be notified of their introduction.
  • the IVM Channel Provider Kit enables third party organizations to insert additional video streams before and/or after video messages that are sent from one user to another, including, but not limited to, advertisements in the form of video, still images, audio, and text.
  • An alternative embodiment of the messaging systems and methods of the invention is depicted schematically in a flow diagram 800 shown in FIG. 8.
  • the sender i.e., Sender A
  • the sender begins by launching the computer communication module 3000, known in this exemplary embodiment as the Videoshare 2Peer application software 3000, on his or her computer 302, as denoted at box 805.
  • the sender uses the 2peer application software 3000, and so is also referred to as the "user" in FIG. 8.
  • the sender computer 302 uses the sender computer 302, the user logs into a 2peer channel, as indicated at box 810.
  • a 2peer channel is an embodiment of an IVM channel, generally 600, of FIGs. 6 and 7.
  • the user records a message comprising video and possible audio or other components, as indicated at box 815, which is reached from box 810 by travelling through Node A.
  • the user can attach meta-data to the video message.
  • the user selects at least one recipient.
  • the user can optionally select more than one recipient. For the purpose of the description, it is assumed that only one recipient is selected. However, if a plurality of recipients are selected, each recipient is handled individually, without regard for the situation of any other recipient.
  • the user communicates, using the 2Peer application software 3000 and the sender computer 302, with the IVM server 406, and requests the online status of the intended recipient (i.e. Receiver B).
  • IVM server 406 checks whether the receiver computer 304 of Receiver B is online via an IVM channel. The result of the inquiry is communicated to the sender. All of the request, the check, and the communication of results of the check is denoted by the box 830. Based on the online status of the receiver computer 304, either of two paths are selected.
  • the Receiver B is notified, as indicated at box 835, using receiver computer 304 and a copy of the 2Peer application software 3000 thereon, that Sender A wishes to send a video message to Receiver B.
  • the Receiver B selects whether he or she wishes to accept the video message, decline the video message, or defer receipt of the message. If the receiver accepts the message, as denoted by box 845, the sender, using sender computer 302 and the 2Peer application software 3000 thereon, steams the video to receiver computer 304, where the Receiver B can view the video message. The user is then carried in the system and method back to Node A, where the process can be repeated, or the user can terminate the process as indicated by the box 850.
  • the system and method operate by informing the user that the recipient has decline the message, as indicated at box 855.
  • the user is then carried in the system and method back to Node A, where the process can be repeated, or the user can terminate the process as indicated by the box 850.
  • the Recipient B defers receipt of the message, as denoted by the arrow from box 840 to box 860.
  • Box 860 indicates the action that the sender is notified that receipt of the message is being deferred by the recipient. If the recipient computer 304 is offline, the sender is informed that the receiver computer is not available to accept a video message, as indicated at box 865.
  • the systems and methods of the invention allow a substantially similar series of steps to take place.
  • the sender using the sender computer 302 and the 2Peer application software
  • the IVM server 406 is asked by the IVM server 406 whether the sender wishes to store the message that cannot be delivered immediately. If the sender declines to store the message, the systems and methods of the invention carry the sender back to Node A, where the process can be repeated, or the user can terminate the process as indicated by the box 850. If the sender does wish to store the message, the sender's status as a registered member of the VideoShare service is checked, as indicated at box 875. If the sender is not a registered member of the VideoShare service, the sender is registered for the service, as indicated at box
  • sender If the sender is a registered member, or has just registered, the sender logs on to the
  • VideoShare service as denoted at box 885.
  • the sender then transfers the video message, using sender computer 302, to the VideoShare server 710, as indicated at box 890.
  • the systems and methods of the invention carry the sender back to Node A, where the process can be repeated, or the user can terminate the process as indicated by the box 850.
  • VideoShare server 710 determines that the receiver computer 304 of Recipient B is online, it streams the video message file to the receiver computer 304 in a manner that is transparent to the sender, who may not be online when such streaming occurs.
  • the rich feature set and wide range of video and audio messaging is also suited to a corporate environment, particularly enabling remote communications such as intra-office video memos as well as messages sent between remote and home offices.
  • the Instant Video Messenger can, for example, be used by co-workers to keep in touch with each other while working on a project, including such activities as sending text, audio, and video messages to one another while sharing collaborative documents.
  • the entire Instant Video Messenger software system can be implemented in C++ within the Microsoft Foundation Class (MFC), ATL/COM, and ActiveX application frameworks.
  • Each functional component (capture, compression, saving, contact list, etc.) can be a separate .DLL file.
  • Such modularity allows easy upgrading, as users only have to download the components that have changed (generally files measured in the 10s of Kbytes).
  • the Instant Video Messenger software is distributed and installed on a user's computer that has a connection, either temporary or permanent, to a network, such as, but not limited to, the Internet.
  • the user may have a video capture device, such as a Web Camera or a video digitizing card, but it is not required for operation of the Instant Video Messenger.
  • Users that do not have a video source can still elect to load in pre-existing media files and can send those media files as Instant Video Messages. Communications between each member of the communication session and the Instant
  • Video Messenger Network can be performed via Transmission Control Protocol/Internet Protocol (TCP/IP) socket connections or User Datagram Protocol (UDP) connectionless datagrams.
  • TCP/IP Transmission Control Protocol/Internet Protocol
  • UDP User Datagram Protocol
  • the user can select which TCP/IP or UDP network Port to use for communications as well as video streaming.
  • the software structure is organized as a three-tier software system.
  • the lowest level tier is a COM based set of functional components that implement the core functionality such as video capture, audio capture, network communication, and contact list management.
  • the middle software tier exists as an interface layer which binds the lower level software tier to a graphical user interface such as windows, buttons, text, etc. This software layer is written as ActiveX components.
  • the top software tier is written as an Microsoft Foundation Class (MFC) desktop application, grouping together the ActiveX components into a final product offering.
  • MFC Microsoft Foundation Class
  • This software installs itself (upon startup of the computer) into the Windows Tool Bar as a small icon that can be opened and closed at the user's will. The software runs on the user's machine until it is shut down or manually terminated.
  • a user of the system such as a private individual working from home, or a professional working from a business, employs a computer system 302.
  • the user has an intention of sending a video message to a viewer of the message who is situated at another computer, that will be described in more detail below.
  • the user of the computer system 302 may also be referred to as the "sender” or "Sender A”, and computer system 302 may be referred to as the sender computer 302.
  • the computer system 302 can include a computer which can be a personal computer of conventional type such as a desktop or laptop computer, a hand held device such as a PDA, or a more powerful computer such as a workstation, a server, a minicomputer, a mainframe, or the like.
  • the computer system 302 can operate software including a web browser such as Microsoft Internet Explorer or Netscape Navigator or Communicator or the like, for communication over a network such as the Internet via the World Wide Web (hereinafter "the Web"), or to permit wireless communication.
  • the computer system 302 can operate software that can manipulate video segment files.
  • the computer system 302 can communicate with video sources, such a video cameras and video recording machines, if the user wishes to employ such sources. Conventional commercially available personal computers typically have sufficient capability to meet these requirements.
  • the computer system 302 can also employ video segments generated digitally by the computer and appropriate software, or by another computer, if the user wishes to employ such techniques.
  • the computer system 302 operates one or more modules that are part of a computer communication module 3000 called VideoShare 2Peer.
  • the VideoShare 2Peer 3000 is a software application package that the user can download from the Web site www.2peer.com or that the user can obtain in other formats such as on a CD- ROM or bundled with other software or hardware.
  • one or more modules that are components of the VideoShare 2Peer 3000 are present in software that the sender computer 302 controls.
  • the one or more modules of the VideoShare 2Peer software can be operated by the user under his or her control on his or her computer, in the computer system 302, in order to provide the capability of recording, converting, and optionally, compressing video segments, creating one or more identifiers for a video segment, and transmitting a video segment with one or more of the identifiers to a receiver computer 304 when the receiver computer 304 is simultaneously online with the sender computer. If the receiver computer 304 is not online, the sender computer can optionally transmit the video segment to a host computer 60 for storage at a location under the control of the host computer 60. The host computer 60 will be described further below.
  • the software modules can provide the capability to make a request of a server computer 406 that server computer 406 determine the online status of a receiver computer 304 to which the sender would like to send a video message, and that the status be reported to the sender computer 302 so that the sender computer 302 can determine whether sending the video message directly to the receiver computer 304 independent of the server computer 406 is appropriate, or whether the video message should be sent to a host computer 60 for storage and for later transmission to the receiver computer 304.
  • the computer in the computer system 302 of the user one can be connected to one or more kinds of equipment for generating video segments, such as a video camera such as a Web cam 12 or another type of video camera such as a professional quality video camera.
  • the computer in the computer system 302 of the user can be connected to one or more kinds of equipment for providing prerecorded video segments, such as a video recorder 14, or another computer that can create digital video segments through the use of suitable software, such as for example digital video segments that have been created for various commercial films, or the like.
  • the video segment with one or more identifiers is transmitted to the receiver computer 304 directly if the receiver computer 304 is online, or the video is sent the host computer 60 for storage and for retransmission to the receiver computer 304 at a later time.
  • the host computer 60 can include one or more server computers 62, 62', 62" that communicate via a network such as the Web with other computers, such as the computer in the user's computer system 302.
  • the one or more server computers 62, 62', 62" also communicate with a storage array 64, or optionally with a plurality of storage arrays substantially similar to storage array 64.
  • the storage array 64 can be any convenient storage system, such as a redundant array of magnetic storage disks, one or more readable and writeable CD-ROMs, random access semiconductor memory, any combination of such storage devices, or the like.
  • the host computer 60 can connect via the Web to one or more computers that comprise the Web, conceptually denoted by the box 70.
  • a video segment that is stored and controlled by the host computer 60 may be delivered to and displayed for a viewer in a variety of formats, and through a variety of methods, as denoted generally by the box 80.
  • a video segment can be displayed as: a video greeting card 81, such as a person wishing another a happy birthday; as video email 82, as video that can be viewed on a remote website 83 (e.g., a video segment embedded into the remote website so that a viewer who visits the remote website sees the video segment as part of the page that is presented); as video commerce 84, for example a video that depicts a person describing his or her experience and training as part of a resume submitted on-line; or as a video advertisement 85, for example a video depicting the benefits or showing the use of a product.
  • a video greeting card 81 such as a person wishing another a happy birthday
  • video email 82 as video that can be viewed on a remote website 83 (e.g., a video segment embedded into
  • the video segment can be made available to the viewer as a streaming video that is sent to the viewer.
  • the viewer can use a hand held device such as a PDA or a cellular telephone that can connect to a network such as the Internet to view the video segment.
  • the computer 16 of the user's computer system 302 is shown.
  • the box 18 is intended to schematically depict a user of a computer video input device, which device can be the computer 16 operating suitable software to generate digital video, or can be another such computer, or can be the web cam or video camera 12, or can be the video recording device 14, or the like.
  • the user begins by producing and/or recording a video segment on the hard disk of the computer 16 or within the temporary memory of a handheld device.
  • the video segment of step 1 can optionally be compressed and /or can be changed as regards the computer file format in which it is recorded on the hard disk.
  • the sender uses the computer 16 to request from the server computer 406 an indication of the online status of the receiver computer 304.
  • the video segment recorded on the hard drive of the computer 16 is transmitted with one or more identifiers to the receiver computer 304.
  • the identifiers can include information comprising at least one of an identifier of the sender of the video message, a subject of the video message, and a title of the video message.
  • the fourth step is schematically depicted by the arrow pointing generally from the computer 16 to the computer 90 of the viewer.
  • the box 92 is intended to schematically depict a user of a display device.
  • the display device can be the computer 90, or the display device can be a display device such as a Web TV, or can be a video output device such as a television set with a suitable decoder, or the like.
  • the display device can also be a wireless hand held device such as a PDA or a cellular telephone or the like.
  • the streaming video segment can in one embodiment be delivered as part of a video greeting card 81. In an alternative embodiment, the video can be delivered as a streaming video directly to the viewer from the sender computer , without the viewer having to activate the receiver computer 304 .
  • the user can obtain a copy of the VideoShare 2Peer software by downloading a copy of the software from the Website wwwNideoShare.com 50, as indicated by the picture at numeral 1.
  • the user can obtain a copy of the VideoShare 2Peer software on machine readable media such as a CD-ROM or the like.
  • the VideoShare 2Peer software can be bundled with one or more utility or application programs that are useful for a user to have, such as a "container" application so that the VideoShare 2Peer software can be operated on a desktop computer.
  • the user can install the VideoShare 2Peer software on his or her computer 16and can register with the IVM service at no charge.
  • the user In registering for the IVM service, the user obtains a username and a password that can be used to identify the user.
  • the activity of installing the IVM software comprising the computer communication module on the user's personal computer or the like and registering with the IVM system is indicated by the picture at the numeral 2.
  • the user In order to use the system, the user first obtains a video segment.
  • the user can create the video segment, for example with a Web cam 12, or the user can use an existing video segment obtained from a video recorder 16, as indicated by the picture at the numeral 3.
  • the VideoShare 2Peer 3000 software includes a module that has direct capture capabilities that permit the user to create the video segment.
  • the user can employ the VideoShare 2Peer 3000 software modules to optionally compress the video; to determine if a video segment is in a format that is compatible with streaming video; to convert the video to a file format that is compatible with streaming video if the video segment is not already in a file format that is compatible with streaming video; and to transmit the video segment together with one or more identifiers that represent selections that the user can make (for example, a still image selected from the series of images that comprise the video segment, and an identifier of the sender of the video segment (e.g., the user).
  • the activities carried out in conjunction with the VideoShare 2Peer 3000 software modules are generally indicated by the graphic at numeral 4.
  • the video segment and the identifier(s) can be transmitted to the receiver computer 304 directly if the receiver computer 304 is online, or if the receiver computer 304 is not online, can be transmitted to the host computer 60 for storage and for later distribution to the receiver computer 304.
  • the video message is transmitted in a streaming video file format. This transmission activity is denoted by the graphic at numeral 5.
  • the host computer 60 Upon a determination that the intended receiver computer 304 is online the host computer 60 sends the video in streaming video format to the receiver computer 304, where a viewer can observe the video in real time.
  • the activity of serving the video segment as a streaming video is denoted by the graphic at numeral 8.
  • the sender computer 302 can send the video message to a host computer, as indicated at numeral 6, which host can then send the video on to the receiver computer when it is online, as shown at numeral 7.
  • VideoShare 2Peer software was developed as a Windows 95, Windows 98, and Windows 2000 (“Windows 9x/2000") compatible ActiveX control (e.g. an .OCX file), with additional components existing as active template library (ATL) component object model (COM) components that are instantiated during runtime.
  • a "container application,” named “2peer.exe,” allows the VideoShare 2Peer ActiveX Control to be executed from the Windows 9x/2000 desktop.
  • the VideoShare 2Peer Active X Control can also be embedded into a web page, as is done within the www.2peer.com web site.
  • the custom written VideoShare 2Peer software includes the following binary/source code components: (1) VideoShare 2Peer ActiveX Control (2peer.ocx), (2) Streaming media rendering engine (vsrender.dll), (3) Streaming media network object (vsnetwork.dll), (4) Buddy-list Contact management (vscontact.dll), (5) 2Peer Channel management (vschannel.dll), (6) VideoCapture object (DirectShowRecord.dll and VFWrecord.dll), (7) VideoPlayback object (DSPlay.dll), and (8) Media management (vsmedia.dll)
  • VideoShare 2Peer software is built upon the following third-party technologies that provide lower-level device support, document sharing, and file format conversion: (1) Microsoft's DirectShow; (2) Microsoft's Windows Media Technologies; (3) Microsoft's Video for Windows; (4) MAPI; AND (5) ICQ.
  • Microsoft's DirectShow When the user launches the VideoShare 2Peer software comprising the computer communication module, he or she will see the window depicted in FIG. 12 appear on his or her computer 16 operating the Win9x/2000 operating system.
  • the login screen can be made optional for repeat users by providing a unique identifier for the user, such as a password, or by installing on the user's computer or the like a record similar to the "cookies" used by some interactive computer systems operating on a network such as the Internet.
  • VideoShare 2Peer 3000 software opens a TCP/IP socket connection to the VideoShare Upload/Database Server via port 80 in order to avoid typical Firewall and/or Proxy Server problems. If the box 430 labeled Remember password is checked, the VideoShare 2Peer 3000 software will remember the user's password, eliminating the necessity to type in that information each time the software is started.
  • VideoShare 2Peer 3000 software will notify the user if there is a more recent version of the software available, giving him or her the opportunity to automatically download and install the new software.
  • the user can also receive help, by activating the "Help" button 450, taking the user to a web page on the VideoShare web site.
  • the login dialog box can also be used to create a new VideoShare user account, by clicking the "Create Another Account” button 460.
  • the VideoShare 2Peer 3000 software looks for available DirectShow audio and video capture devices. These available devices are enumerated and listed within the "Settings Tab" as described later.
  • the VideoShare 2Peer 3000 software initializes the audio and video capture device, by recalling as a default the device that was used most recently.
  • VideoShare 2Peer software displays the window depicted in FIG. 13.
  • the image 510 in the middle of the window is the video input stream from the initialized, default video capture source.
  • the image in FIG. 13 is that of an employee of the assignee of the present invention, in the offices of the assignee.
  • the VideoShare 2Peer software automatically builds a DirectShow "preview graph" where the video stream from the video device is displayed on the screen, but is not saved to disk. This gives the user the opportunity to adjust the camera, e.g. an opportunity to correct the camera position, the camera focus, the camera angle, the magnification of the image, and the like.
  • the user is presented with five different "tabs", each presenting the user with different aspects of the VideoShare 2Peer software comprising the computer communication module.
  • the tab labeled "Record/Playback" 520 is active, indicating that the VideoShare 2Peer software comprising the computer communication module is ready to acquire and/or display a video segment.
  • the status message 522 that displays the current operation of the VideoShare 2Peer software comprising the computer communication module.
  • the status message 522 prompts the user to either activate the Record button 531 to create a new video segment, or to import an existing video segment by activating the Import Video button 535, both of which are described in more detail below.
  • Capture/Playback Control Panel 530 that includes the following items:
  • Record button 531 which begins a new audio/video capture; Stop button 532 which terminates an active audio/video capture operation; - Play button 533 which initiates the playing back of the last recorded or imported video;
  • Delete button 534 which cancels the last record or import operation and begins a new video preview
  • Import Video button 535 which allows the user to select a pre-existing video file from his or her hard drive
  • Save and Share button 536 which in the present embodiment activates software modules that convert the current video file into a compressed streaming format, and that send the video;
  • Shuttle Bar 537 which is used to control the current position of the playback file together with forward button 537 and reverse button 538, allowing the user to rewind and fast forward through the current video.
  • the software modules that operate upon the activation of Save and Share button 536 will be covered in a subsequent section in this document in detail.
  • the VideoShare 2Peer software When the user begins to record a video, the VideoShare 2Peer software builds a new "Capture Graph” that renders the video stream to both the display window as well as to a temporary .AVI file on the user's hard drive. The audio/video capturing continues until the user activates the "Stop” button 532 at which point the VideoShare 2Peer software stops the "Capture Graph", destroys the DirectShow filter, builds a Direct Show "Playback Graph", and displays the first frame of the captured video as video preview image 510. When the user activates the Play button 533 the DirectShow "Playback Graph" is put into running mode, playing back the entire recorded video from beginning to end.
  • the user can also choose to import a pre-existing video, which in one embodiment can be a file format selected from the AVI, MPEG, or QuickTime file formats, by activating the Import Video button 535.
  • the VideoShare 2Peer software automatically renders the correct DirectShow filter to display an imported video correctly.
  • Save and Share Process Once a video segment has been recorded or imported into the user's computer 16 that is running the VideoShare 2Peer software, the user can choose to process the video segment with various optional alternatives by activating the Save and Share button 536. When the Save and Share button 536 is activated, the video segment is archived and distributed automatically.
  • the VideoShare 2Peer software greatly simplifies the entire process by seamlessly automating the following steps that are depicted in FIG. 14A:
  • Logging the transactions and managing the user's storage account including causing the generation of an identification tag that the server computers 62, 62' can employ to retrieve the video segment for viewing.
  • FIG. 14A shows a flow diagram 600 of an embodiment of the invention in which the VideoShare 2Peer software automates a number of steps in connection with uploading a video segment by activation of the Save and Share button 536 described in FIG. 13.
  • a user first obtains and selects a video segment for processing for distribution.
  • the box 605 schematically encapsulates all of the actions that a user takes as described in relation to FIGs. 4 and 5 above.
  • the Save and Share button 536 the actions described below that are enclosed by the dotted line 607 are automatically carried out under the control of the VideoShare 2Peer software.
  • the VideoShare 2Peer software subjects the selected video segment to analysis to determine whether the selected video segment is or is not in a file format that is compatible with a streaming video format, as indicated at diamond 610.
  • Formats that are compatible with streaming media formats include formats such as MPEGs and QuickTime videos. If the selected video segment is not compatible with a streaming video format, it is converted to a compatible format, as depicted by the arrow labeled "NO" that points from the diamond 610 to the box 615, "Convert to compatible file format.”
  • the conversion process performed by the VideoShare 2Peer software creates a DirectShow filter graph that decompresses the video file into a temporary, uncompressed AVI file.
  • the video segment file in a format that is compatible with streaming video is then temporarily stored in the user's computer 16, for example as a file on the hard drive of computer 16.
  • This storing step is performed if the file was originally in a format compatible with streaming video by following the arrow marked "YES” that points from the diamond 610 to the box 620, "Temporarily store file.”
  • the storing step is performed if the file was originally not in a format compatible with streaming video by following the arrow that points from the box 615 to the box 620.
  • the stored temporary file representing the selected video is then analyzed by the VideoShare 2Peer software , as represented by diamond 625, "Should file be compressed?" to determine if the temporarily stored file should be compressed. If the software determines that the file should be compressed, as indicated by the arrow labeled "YES" that points from the diamond 625 to the box 630, labeled "Compress file,” the file is compressed.
  • the compression involves compressing the video file to a user-specified bitrate, or the bandwidth that is required to view the video without disruption in the transmission. The user can select the desired bitrate via the "Settings Tab" that is described in more detail below.
  • the file is then converted to a streaming multimedia format file as indicated by the box 635, labeled "Convert file to streaming multimedia format (“SMF”) file,” as denoted by the arrow pointing from the box 630 to the box 635. If the file is not to be compressed, the flow follows the arrow labeled "NO" pointing from the diamond 625 to the box 635, and the file is then converted to a streaming multimedia format file as schematically represented by the box 635.
  • the process that is performed by the VideoShare 2Peer software as denoted by the box 635 involves reading in the video file, frame by frame, and converting the video into a streaming multimedia format.
  • the VideoShare 2Peer software uses the Windows Media Streaming Format, known as ASF or WMF, but it is not technologically restricted to this choice.
  • the Windows Media Streaming Format comprises MPEG 4 v3 for the video stream and the Windows Media Audio format for the audio stream.
  • the output of this file is stored as a temporary file on the user's hard drive, in one embodiment.
  • the flow diagram indicates that the process makes a "thumbnail" of the video file, as represented schematically by the box 640, labeled “Create and temporarily store JPEG "thumbnail” identifier.”
  • the VideoShare 2Peer software produces a JPEG still image that is used as a reference image to the entire video file. It is an identifier of the subject matter or content of the video that a user or a viewer can readily recognize, as compared to an alphanumeric string such as a typical string used to identify a file by its drive, directory (and one or more subdirectories) and filename. Such alphanumeric identifiers are useful, but may be totally uninformative as to the content or subject matter contained in the identified file or video segment.
  • the VideoShare 2Peer software creates the "thumbnail” by taking the “middle” image of the entire video file, as measured by the temporal duration of the file.
  • the selection of an image from which to make the "thumbnail” can be left to the discretion of the user.
  • This JPEG file is also stored as a temporary file on the user's hard drive, in one embodiment.
  • the next part of the process is either sending the video message to the receiver computer 304, if the receiver computer 304 is online, or optionally sending the video message to the host computer 60 if the receiver computer 304 is not on line.
  • the sender 40 can send the video message directly to the receiver computer 304 if the receiver computer 304 is online, as indicated at box 641.
  • Sender A using sender computer 302, asks IVM Server 406 if Receiver B is online via receiver computer 304.
  • IVM Server 406 responds that Receiver B is online.
  • Sender A records, or already has recorded, a video message using sender computer 302 and the associated hardware and software.
  • Sender A using sender computer 302, notifies Receiver B via receiver computer 304 that a video message is ready for transmission.
  • Receiver B Assuming that Receiver B wishes to accept the video message, Receiver B acknowl edges notification using receiver computer 304 and requests delivery of the message.
  • Sender A using sender computer 302, streams the video message to Receiver B via receiver computer 304.
  • the sender 40 (or Sender A) can send the video message directly to the receiver computer 304 if the receiver computer 304 is online, as indicated at box 641.
  • the IVM Server 406 informs sender computer 302 that the receiver computer 304 is not online. Rather than sending a message to the receiver computer 304, the sender computer 320 sends the video message to a VideoShare server computer 710 (or host computer 60) for storage at server computer 710, and for later delivery to the intended recipient.
  • Server computer 710 can transmit the video message, for example, via e-mail or via Web retrieval as described in copending U.S.S.N. 09/497,587, to the intended recipient computer 304 when that computer 304 is again on-line. These steps are shown in boxes 645, 650, 655, 660 and 670.
  • the optional part of the process is the upload operation, in which the VideoShare 2Peer software contacts the host computer 60, which in one embodiment is the VideoShare
  • VideoShare 2Peer software notifies the host computer 60 that the user wishes to place his or her video into a repository maintained by the host computer 60, which in one embodiment can be the VideoShare VideoCenter, which is a repository of all recorded and uploaded videos to date. This upload is performed automatically via a direct TCP/IP socket connection over a specific connection port of the user's computer known as port 80.
  • the VideoShare 2Peer software uses a standard communications protocol to perform this transfer to the host computer 60.
  • a proprietary protocol can be used, for example if one wants to maintain the security of information contained in the video segment.
  • the video segment can be encrypted in order to provide enhanced security. Both the compressed video streaming multimedia file and the thumbnail image are uploaded at substantially the same time.
  • VideoShare 2Peer software removes all of the temporary files that were created in the course of the automated processing described above. This feature provides for the user a convenient, secure, and transparent process, with the benefit that the user's computer storage device(s), for example one or more hard drives, do not become cluttered with unnecessary and obsolete files.
  • the VideoShare 2Peer software and the host computer 60 will update the user's account to account for the required storage space that the video requires.
  • the necessary logging, creation of an identification tag, and storing of the video and the associated identifier or identifiers is also performed automatically, as schematically depicted by box 655.
  • the user can optionally add additional identification and control information about the user, and about how and under what conditions the video is to be made available for distribution, as schematically indicated by box 660.
  • the user is automatically prompted to provide this information, but has the option to forego making a decision immediately.
  • the transmission of video segment files to viewers is discussed in more detail below, and is represented in FIG. 14A by the box 670 labeled "Transmit file to viewer" which is outside the region 607 as an indication that the transmission of files to viewers is an action beyond the material discussed above in conjunction with the Save and Share button 536 of FIG. 13.
  • FIG. 14B shows a flow diagram 601 of another embodiment of the invention in which software automates a number of steps in connection with uploading a video segment. Many of the steps already described in connection with FIG. 14A also occur in the embodiment depicted in FIG. 14B, and are numbered in the same manner as in FIG. 14A.
  • FIG. 14B there is first an optional step indicated by the box 604 labeled "Optional: User authentication with server” in which the User is optionally required to provide identification, such as a user name and password, that authenticates the identity of the user to the server or host computer 60.
  • the user then obtains and selects a video segment for processing for distribution, as indicated at box 605 that schematically encapsulates all of the actions that a user takes as described in relation to FIGs. 4 and 5 above.
  • the Save and Share button 536 the actions described below that are enclosed by the dotted line 608 are automatically carried out under the control of the VideoShare 2PeerSoftware comprising the computer communication module.
  • the VideoShare 2Peer software subjects the selected video segment to analysis to determine whether the selected video segment is or is not in a file format that is compatible with a streaming video format, as indicated at diamond 610. If the selected video segment is not compatible with a streaming video format, it is converted to a compatible format, as depicted by the arrow labeled "NO" that points from the diamond 610 to the box 615, "Convert to compatible file format.”
  • the conversion process performed by the VideoShare 2Peer software creates a DirectShow filter graph that decompresses the video file into a temporary, uncompressed AVI file.
  • the video segment file in a format that is compatible with streaming video is then temporarily stored in the user's computer 16, for example as a file on the hard drive of computer 16.
  • This storing step is performed if the file was originally in a format compatible with streaming video by following the arrow marked "YES” that points from the diamond 610 to the box 620, "Temporarily store file.”
  • the storing step is performed if the file was originally not in a format compatible with streaming video by following the arrow that points from the box 615 to the box 620.
  • the stored temporary file representing the selected video is then analyzed by the VideoShare 2Peer software, and optionally compressed as represented by the box 623 labeled "Optional compression of file.”
  • the file is then converted to a streaming multimedia format file as indicated by the box 635, labeled "Convert file to streaming multimedia format ("SMF") file.”
  • a file from the box 620 can be uploaded to the host computer 60 without being converted to a streaming format, and the conversion to a streaming video format can be accomplished at the host computer 60.
  • the process that is performed by the VideoShare 2Peer software as denoted by the box 635 involves reading in the video file, frame by frame, and converting the video into a streaming multimedia format.
  • the flow diagram indicates that the process makes a "thumbnail" of the video file, as represented schematically by the box 640, labeled "Create and temporarily store JPEG "thumbnail” identifier.”
  • the sender 40 (or Sender A) can send the video message directly to the receiver computer 304 if the receiver computer 304 is online, as indicated at box 641.
  • the IVM Server 406 informs sender computer 302 that the receiver computer 304 is not online. Rather than sending a message to the receiver computer 304, the sender computer 320 sends the video message to a VideoShare server computer 710 (or host computer 60) for storage at server computer 710, and for later delivery to the intended recipient.
  • Server computer 710 can transmit the video message, for example, via e-mail or via Web retrieval as described in copending U.S.S.N. 09/497,587, to the intended recipient computer 304 when that computer 304 is again on-line. These steps are shown in boxes 645, 650, 655, 660 and 670.
  • the optional part of the process is the upload operation, in which the VideoShare 2Peer software contacts the host computer 60, which in one embodiment is the VideoShare Upload/Database Server at the VideoShare hosting facility.
  • This portion of the automated process is denoted by the box 645 labeled "Transfer ("upload") temporarily stored SMF file and JPEG thumbnail identifier to host computer 60.”
  • the VideoShare 2Peer software notifies the host computer 60 that the user wishes to place his or her video into a repository maintained by the host computer 60, which in one embodiment can be the VideoShare VideoCenter, which is a repository of all recorded and uploaded videos to date.
  • This upload is performed automatically via a direct TCP/IP socket connection over a specific connection port of the user's computer known as port 80.
  • the VideoShare 2Peer software uses a standard communications protocol to perform this transfer to the host computer 60.
  • a proprietary protocol can be used, for example if one wants to maintain the security of information contained in the video segment.
  • the video segment can be encrypted in order to provide enhanced security. Both the compressed video streaming multimedia file and the thumbnail image are uploaded at substantially the same time.
  • the next part of the process is the upload operation, in which the VideoShare 2Peer software contacts the host computer 60, which in one embodiment is the VideoShare Upload/Database Server at the VideoShare hosting facility.
  • This portion of the automated process is denoted by the box 645 labeled "Transfer ("upload") temporarily stored SMF file and JPEG thumbnail identifier to host computer 60.” Both the compressed video streaming multimedia file and the thumbnail image are uploaded at substantially the same time.
  • the VideoShare 2Peer software removes all of the temporary files that were created in the course of the automated processing described above.
  • This feature provides for the user a convenient, secure, and transparent process, with the benefit that the user's computer storage device(s), for example one or more hard drives, do not become cluttered with unnecessary and obsolete files.
  • the VideoShare 2Peer software and the host computer 60 will update the user's account to account for the required storage space that the video requires.
  • the necessary logging, creation of an identification tag, and storing of the video and the associated identifier or identifiers is also performed automatically, as schematically depicted by box 655.
  • the user can optionally add additional identification and control information about the user, and about how and under what conditions the video is to be made available for distribution, as schematically indicated by box 660.
  • the user is automatically prompted to provide this information, but has the option to forego making a decision immediately.
  • the transmission of video segment files to viewers is discussed in more detail below, and is represented in FIG. 14B by the box 670 labeled "Transmit file to viewer" which is outside the region 608 as an indication that the transmission of files to viewers is an action beyond the material discussed above in conjunction with the Save and Share button 536 of FIG. 13.
  • FIG. 14C shows a flow diagram 602 of an embodiment of the invention in which software automates a number of steps in the formatting of a video segment.
  • the video segment that the user wishes to provide in streaming video format is compressed into a plurality of formats, each of which is encoded for optimal display at a different transmission bitrate.
  • a casual viewer may have only a slow speed modem, such as a 28.8 kilobaud (kB) modem.
  • the slow transmission speed can make the size of a file a critical feature.
  • Such a user can view a video in real time if it is formatted for a 28.8 kB modem, but not if it is formatted for appreciably higher transmission speeds.
  • Another user for example, one who has a Tl connection that can handle transmission speeds up to approximately 1.5 megabaud, could successfully receive a version of the same video segment that is formatted for higher transmission speeds, with the possibility of having a better quality image and higher resolution, perhaps with better audio as well.
  • the Tl user could see the version of the video segment intended for 28.8 kB transmission if he or she wanted to, but might prefer to see a video segment that appeared to be more professional in quality.
  • the embodiment allows each user to view a version of the video segment that is optimally configured for the user's hardware.
  • the steps of the method enclosed within the dotted rectangle 609 are automated by software that embodies the present invention.
  • the user obtains and selects a video segment for processing for distribution, as indicated at box 605 that schematically encapsulates all of the actions that a user takes as described in relation to FIGs. 4 and 5 above.
  • the Save and Share button 536 the actions described below that are enclosed by the dotted line 609 are automatically carried out under the control of the VideoShare 2PeerSoftware comprising the computer communication module.
  • the VideoShare 2Peer software subjects the selected video segment to analysis to determine whether the selected video segment is or is not in a file format that is compatible with a streaming video format, as indicated at diamond 610. If the selected video segment is not compatible with a streaming video format, it is converted to a compatible format, as depicted by the arrow labeled "NO" that points from the diamond 610 to the box 615, "Convert to compatible file format.”
  • the conversion process performed by the VideoShare 2Peer software creates a DirectShow filter graph that decompresses the video file into a temporary, uncompressed AVI file.
  • the video segment file in a format that is compatible with streaming video is then temporarily stored in the user's computer 16, for example as a file on the hard drive of computer 16.
  • This storing step is performed if the file was originally in a format compatible with streaming video by following the arrow marked "YES” that points from the diamond 610 to the box 620, "Temporarily store file.”
  • the storing step is performed if the file was originally not in a format compatible with streaming video by following the arrow that points from the box 615 to the box 620.
  • the temporarily stored file is then compressed in multiple streaming multimedia formats, as denoted by the box 633.
  • three files will be used to describe the process, but it should be understood that more or fewer than three formats may be created at substantially the same time.
  • the resulting multiple files are denoted by the three boxes 634, 636 and 638 labeled "Bandwidth Target A,” “Bandwidth Target B,” and “Bandwidth Target C,” respectively.
  • Each file is optimally encoded for play as a streaming video segment at a particular transmission rate and bandwidth, such as 28.8 kB, 56 kB, lOOkB, 300kB, or other transmission rates.
  • the method includes a step of creating and temporarily storing a "thumbnail" identifier, as denoted by the box 640.
  • the embodiment of FIG. 14C transmits all the files 634, 636 and 638 in association with the single thumbnail and any other identifiers that are selected as appropriate.
  • each SMF file can be identified as to its bandwidth.
  • the system transmits only a single SMF file with its associated identifiers, including the JPEG "thumbnail," and the multiple bandwidth variants of the SMF file are generated at the host computer 60. This embodiment may be advantageous when the user has only a slow speed modem, and would be severely time constrained by having to upload multiple files.
  • the sender 40 (or Sender A) can send the video message directly to the receiver computer 304 if the receiver computer 304 is online, as indicated at box 641.
  • the IVM Server 406 informs sender computer 302 that the receiver computer 304 is not online. Rather than sending a message to the receiver computer 304, the sender computer 320 sends the video message to a VideoShare server computer 710 (or host computer 60) for storage at server computer 710, and for later delivery to the intended recipient.
  • Server computer 710 can transmit the video message, for example, via e-mail or via Web retrieval as described in copending U.S.S.N. 09/497,587, to the intended recipient computer 304 when that computer 304 is again on-line. These steps are shown in boxes 645, 650, 655, 660 and 670.
  • the remaining steps of this embodiment correspond substantially to the steps in FIG. 14A represented by the boxes identified with the corresponding numerals. It should be noted that the precise order of some of the steps, for example, the step denoted by the box 655 and the step denoted by the box 650, can be interchanged without a different outcome of the overall process. Other such interchanges in sequence are possible as well, again without a different outcome of the overall process.
  • FIG. 14D depicts an embodiment of the database 64 of the host computer 60 on which are recorded the three exemplary bandwidth target files 634, 636 and 638 for FIG. 14C. These files are available for delivery over a computer network to a viewer.
  • the files 634, 636 and 638 represent three versions of the same video segment in streaming multimedia format, each suitable for optimal viewing by a user having hardware operating at the transmission rate corresponding to the format of one of the files.
  • the user transmits to the host computer 60 a request for a particular video segment, denoted by the arrow from the box labeled "USER” to the box 960 labeled "Connection Speed Detector.”
  • Host computer 60 can include hardware that can sense the transmission speed of a user computer 16, or of a computer used by a person desiring to view a video segment. Alternatively, the host computer 60 can inquire of the computer on the network that is connected to the user computer 16 or the computer of a viewer about the speed of connection that is being maintained.
  • the host computer 60 can determine which file of the files exemplified by 634, 636 and 638 is most appropriate to serve to the user or viewer, as denoted by the box 692 labeled "Logic to select and serve SMF file to User.” The host computer 60 then transmits the appropriate file to the user, as denoted by the arrow from the box 692 to the box 694 labeled "User receives and views SMF file.” Alternatively, the viewer can request the transmission of a file encoded at a specific bitrate.
  • the "Progress Dialog" screen 700 depicted in FIG. 15 is presented, reflecting the status of the process in real time.
  • the "Progress Dialog" screen 700 notifies the user about the total number of bytes that have to be uploaded to perform the transfer and it also informs the user of the number of bytes and the percentage of the file that have been uploaded in real time.
  • FIG. 16 shows one embodiment of a screen, discussed above, that displays a video message.
  • FIG. 17 shows a screen 1000 used to control the status of a video queue.
  • the VideoShare 2Peer software performs the first three steps of the "Save and Share Process," namely, the video file format conversion represented by box 615 of FIG. 14A, the compression of the video segment to a streaming multimedia format at a user-specified bitrate represented by the box 635 of FIG. 14 A, and the creation of a "Thumbnail” JPEG snapshot of the video file represented by the box 640 of FIG. 14A.
  • the resulting output files are stored in a local database for later use in the "Sharing Queue," which is an operation similar to the temporary storage of files depicted in FIG. 14A.
  • a dialog box 1010 that displays a list of video segments that are ready to be uploaded to the VideoShare Web site.
  • the small "Preview” window 1020 in the upper left corner of FIG. 17 is a DirectShow playback graph that allows the user to review the stored video segment that is highlighted in the dialog box 1010. The user can use this window to preview the video segment file by activating the "Preview” button 1030, to delete the video segment file by activating the "Delete” button 1040, and to upload and publish the video by activating the "Save and Share Now" button 1050.
  • the "Save and Share Now” button 1050 performs the uploading process on each of the queued videos, creating a TCP/IP connection to the VideoShare Upload/Database Server, transferring the file to the VideoShare web site, and updating the user's VideoShare account, in a manner substantially similar to the method employed by the Save and Share button 536 of FIG. 13 to accomplish the same activities.
  • FIG. 18 shows a screen 1100 used to control the operational settings of equipment connected to the user's computer.
  • Another feature of the VideoShare 2Peer software the ability of the user to change the configuration of the audio, video, and compression devices through the use of the "Settings" tab 1110.
  • the screen 1100 is active.
  • the user can select the "bitrate” at which the streaming multimedia files will be compressed by using the set of radio buttons 1120 at the upper left corner of the screen 1100.
  • the default setting is "56k Modem” which corresponds to a user using a 56k modem. This default setting is denoted by the 56k Modem radio button 1120 appearing with a dot, while the remaining radio buttons for bitrate 1120 are blank.
  • the pie graph 1130 that appears at the upper right corner of screen 1100 indicates the percentage of the user's VideoShare storage space that is full. In the embodiment shown, the user has filled approximately 3.13% of the available storage capacity available for storing files.
  • the screen 1100 also provides to the user the current working directory information in a the box 1180 and the current queue directory information in the box 1190, which the user can optionally change by entering new values in either or both boxes 1180 and 1190.

Abstract

Methods and systems for a sender of a video message to send the message over a computer network from the sender's computer directly to a receiver computer used by a recipient, without the intermediation of a server computer. A server computer determines whether the receiver computer is online. In response to a request from the sender computer, the server informs the sender computer of the online status of the receiver computer. The sender computer prepares a viedo message, including a video segment, and optionally audio, text an other information. The sender computer informs the receiver computer that a video message is available to be sent. Depending on the response of the receiver computer, the video message is streamed from the sender computer to the receiver computer, the video message is sent to a host computer for storage and later streaming to the receiver computer, or is not sent. If the receiver computer is not online, the video message is optionally sent to the host computer for storage and later streaming to the receiver computer.

Description

INSTANT VIDEO MESSENGER
Cross-References to Related Applications
This application claims priority to and the benefit of U.S. Provisional Patent Application Serial Number 60/147,029, filed August 3, 1999, and of U.S. Provisional Patent Application Serial Number 60/196,069, filed April 10, 2000, which applications are incorporated herein in their entirety by reference. This application is a Continuation-in-Part of U.S. Patent Application Serial No. 09/497,587 (hereinafter "U.S.S.N. 09/497,587"), filed February 3, 2000, entitled "Method And System For Sharing Video Over A Network", which application is incorporated herein in its entirety by reference.
Field of the Invention This invention relates generally to transmitting video files over a network. More particularly, the invention relates to transmitting streaming video files over a network without the intervention of a server computer in the streaming process.
Background of the Invention Communication of information in the form of digital or computer files, such as documents, videos, and the like, by attachment to electronic messages such as electronic mail is well known. With this type of transmission, the entire video file must be transmitted and received before the receiver can view the video. For large files, the time required to complete such transmissions can be longer than the actual playing time of the video. Also, this type of transmission typically requires multiple computer programs to perform all of the necessary functions, including an e-mail application program to send or receive the video in computer file form, and a second program to play or display the video from the received file attachment. While these forms of communication represent an advance over the transmission of hard copies of documents, videos, and the like, there are still many limitations and problems that are associated with such communications.
Some of the problems that are associated with communications over a computer network operated according to the client/server model (and thus including one or more server computers) include, but are not limited to, the large expense of building, maintaining, and operating servers generally, and in particular the expense of installing and maintaining servers that perform functions related to data storage and hosting; the problems associated with loss of service if the server is inoperative, and degradation of service if the server is overloaded; delays, inconvenience, and bandwidth capacity issues associated with uploading information from a sender computer to the server for retransmission from the server to a recipient computer; and difficulties in increasing the scale of the network as the number of client computers increases. Because video files in particular tend to be quite large, the problems enumerated above can be particularly severe in handling video files in a client/server network.
Summary of the Invention The present invention provides a method and system of transmitting a video that mitigates the problems associated with the lengthy transmission time associated with a file attached to an e-mail, that eliminates the necessity to have multiple software packages to carry out the transmission, and that eliminates the necessity to invoke the services of a hosting computer for storing the video. In some embodiments, the method and system can be transparent to the user. In accordance with the present invention, full motion video can be automatically prepared and transmitted by a sender from a sender computer to a viewer using a receiver computer, when the two computers are simultaneously connected to a computer network, including such networks as a global computer network, for example, the Internet or the World Wide Web ("Web").
The invention is described in terms of an exemplary embodiment called the Instant Video Messenger (IVM). IVM is a rapid, asynchronous messaging system/software that supports symmetric, peer-to-peer communication of digital information (e.g., files containing video data such as files in the jpg, .avi, .mpeg, QuickTime, .wav, Windows Media Format, Real Media, and other formats) on a computer network, such as the Internet. Each user computer equipped with the IVM software is capable of sending and receiving streaming and non-streaming audio, streaming and non-streaming video, text, attachments, and meta-data. The network structure of IVM reduces the burden on a central server (which traditionally has been used to host and stream video), as IVM is a peer-to-peer application and thus does not require a hosting facility. A server can be used in an IVM system to perform functions that include, but are not limited to, monitoring which IVM users are on-line, providing communications driven by an entity that has permission to use the server (for example, an advertiser who has contracted for services), and hosting messages for users who have contracted for that service. IVM supports an Instant Video Messenger Channel capability that allows for community building as well as affiliate opportunities. The IVM distribution system allows for "self-replication."
In one aspect, the invention features a method of sending a streaming video message over a network. The method includes receiving at a server computer an indication that the operator of a sender computer intends to send a streaming video message to a receiver computer over a network, determining whether the receiver computer is online, and communicating from the server computer to the sender computer whether the receiver computer is online, wherein an indication that the receiver computer is online signals to the sender computer that streaming the video message directly from the sender computer to the receiver computer over the network is appropriate.
In one embodiment, the invention can include a supervisory computer communication module operable on the server computer that performs the step of receiving at the server computer an indication that the operator of a sender computer intends to send a streaming video message to a receiver computer. The supervisory computer communication module can also perform the step of determining whether the receiver computer is online, and can perform the step of communicating from the server computer to the sender computer whether the receiver computer is online.
In one aspect, the invention features a method of sending a streaming video message over a network. The method includes sending to a server computer an indication that the operator of a sender computer intends to send a streaming video message to a receiver computer over a network, receiving at the sender computer an indication of whether the receiver computer is online, and in response to an indication that the receiver computer is online, streaming the video message from the sender computer to the receiver computer over the network independent of the operation of the server computer. In one embodiment, the step of streaming the video message from the sender computer to the receiver computer includes providing on the sender computer a computer communication module operable on each of the sender computer and the receiver computer, and streaming the video message from the communication module of the sender computer to the receiver computer over the network independent of the operation of the server computer. The step of providing a computer communication module operable on each of the sender computer and the receiver computer can include providing individual copies of the same communication module to at least one of the sender computer and the receiver computer, each copy operable on each of the sender computer and the receiver computer. The method can include providing communication modules that are symmetric in operation. The method can include providing communication modules that are modular, and can include providing communication modules that can determine the features present in another communication module operating on another computer. In another embodiment, the step of streaming the video message from the sender computer to the receiver computer over the network independent of the operation of the server computer can include performing the streaming operation using only communication modules present on the sender computer and the receiver computer.
In a further embodiment, the method can additionally include the step of, in response to an indication that the receiver computer is not online, optionally sending the video message from the sender computer to a host computer, for storage and later transmission to the receiver computer, independent of the operation of the server computer.
In still another embodiment, the video message can include video information. The video message can additionally include information selected from audio information, a thumbnail image, a text identifying a subject, a message body text, a document attachment, a slide and meta-data.
In one aspect, the invention features a computer program recorded on a machine-readable medium. The computer program is adapted to transmit and receive streaming video messages over a network independent of a server computer. When the computer program operates on a sender computer, it performs the steps of requesting from a server computer the status of a receiver computer, the receiver computer being the intended recipient of a video message, receiving an indication of the online status of the receiver computer, and in response to an indication that the receiver computer is online, streaming the video message from the computer program of the sender computer to the computer program of the receiver computer independent of the operation of the server computer.
In one embodiment, when it is operating on a receiver computer, the computer program is further capable of determining that a video message is being sent by a sender computer. The program is additionally capable of displaying to a viewer of the receiver computer information about the video message, the information comprising at least one of an identifier of the sender of the message, a subject of the message, and a title of the message.
In another embodiment, when it is operating on a receiver computer, the computer program is capable of receiving the streaming video message, and displaying the streaming video message to a viewer of the receiver computer. The step of receiving the streaming video message can include receiving the streaming video message, and presenting to a viewer of the receiver computer an opportunity to select one of accepting the streaming video message, and rejecting the streaming video message. The step of receiving the streaming video message can include accepting a response of the viewer, and processing the message in accordance with the response of the viewer. The step of accepting the streaming video message can include at least one of accepting the streaming video message for immediate display, and accepting the streaming video message for storage in a machine-readable memory, and for display at a later time.
In still another embodiment, when it is operating on a receiver computer, the computer program is capable of receiving the streaming video message is accomplished without the necessity of an action by a viewer of the computer. In this embodiment, the program can accomplish displaying the streaming video message without the necessity of an action by a viewer of the computer. The computer program can perform displaying the video message by invoking the operation of a video display module. In another embodiment, the computer program can, in response to an indication that the receiver computer is not online, optionally send the video message from the communication module of the sender computer to a host computer, for storage and later transmission to the communication module of the receiver computer, independent of the operation of the server computer. In yet another embodiment, the video message can include video information. The video message can further include information selected from audio information, a thumbnail image, a text identifying a subject, a message body text, a document attachment, a slide and meta-data.
In one aspect, the invention features a computer program recorded on a machine-readable medium. The computer program is adapted to operate a server computer and to provide server functionality including database connectivity and communication protocols. When operating on a server computer, the computer program performs the steps of determining the online status of the receiver computer, and providing an indication of the status to the sender computer, such that in response to an indication that the receiver computer is online, streaming the video message from the computer program of the sender computer to the computer program of the receiver computer is accomplished independent of the operation of the server computer.
In one embodiment, when in operation, the computer program is capable of accepting a request from a sender computer about the status of a receiver computer, the receiver computer being the intended recipient of a video message. When in operation, the computer program is capable of providing communication protocols for exchanging information between interconnected computers. When in operation, the computer program is capable of providing a database in which to store information, including information about sender and receiver computers authorized to communicate with the server computer.
The foregoing and other objects, aspects, features, and advantages of the invention will become more apparent from the following description and from the claims.
Brief Description of the Drawings The objects and features of the invention can be better understood with reference to the drawings described below, and the claims. The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the drawings, like numerals are used to indicate like parts throughout the various views.
FIG. 1 shows the relationship between the time that elapses between the sending and the receiving of a message ("immediacy") and data delivery solutions of some technology such as that described in copending U.S.S.N. 09/497,587.
FIG. 2A depicts the interactions in a system including a sender computer, a server computer, and a receiver, such as the system described in copending U.S.S.N. 09/497,587.
FIG. 2B presents in overview a flowchart showing the steps involved in the interaction depicted in FIG. 2A.
FIG. 3 shows the relationship between immediacy and data delivery solutions of FIG. 1 and additionally including an embodiment of a method and system according to the invention. FIG. 4A depicts an embodiment of the interactions between and among a sender computer, a server computer, and a receiver computer in a network according to the invention. FIG. 4B presents in overview a flowchart showing the steps involved in the interaction depicted in FIG. 4A.
FIG. 5 depicts another embodiment of the interactions between and among a sender computer, a server computer, and a receiver computer in a network according to the invention. FIG. 6 depicts an embodiment of the interactions between and among a sender computer, a server computer, and a receiver computer in a network having Instant Video Messenger Channels according to the invention.
FIG. 7 depicts an embodiment of the interactions between and among a sender computer, a server computer, and a receiver computer in a network having Instant Video Messenger Channels and the capacity to store message data according to the invention.
FIG. 8 is a flow diagram that depicts schematically an alternative embodiment of the messaging systems and methods, according to the principles of the invention.
FIG. 9 is a schematic embodiment of a process and system according to the copending U.S.S.N. 09/497,587.
FIG. 10 is an embodiment of a system according to the copending U.S.S.N. 09/497,587, including the interactions and interrelationships within the system. FIG. 11 is a functional block and flow diagram of an embodiment of the copending U.S.S.N. 09/497,587.
FIG. 12 is a login screen on a user's computer, in one embodiment of the invention. FIG. 13 is a record/playback screen as seen by the user, in accordance with an embodiment of the invention.
FIG. 14A is a flow diagram of an embodiment of the invention in which software automates a number of steps in connection with the uploading of a video segment.
FIG. 14B is a flow diagram of another embodiment of the invention in which software automates a number of steps in connection with the uploading of a video segment. FIG. 14C is a flow diagram of an embodiment of the invention in which software automates a number of steps in connection with the formatting of a video segment.
FIG. 14D shows the relationship of some of the files created in the flow diagram of FIG. 14C.
FIG. 14E is a flow diagram of a method by which an optimally formatted video segment is sent to a user according to the invention.
FIG. 15 is a screen as seen by the user, the screen indicating that file processing is occurring.
FIG. 16 is a video playback screen seen by the user. FIG. 17 is a screen used by the user to control the status of a video queue. FIG. 18 is a screen used by the user to control the operational settings of equipment associated with the user's computer.
Detailed Description Introduction
The features of a communication network may be described in terms of the following five parameters or qualities:
1. Immediacy
2. Symmetry
3. Network Structure
4. Caching & Routing 5. Extensibility Immediacv
A one-dimensional measure termed "immediacy" is an indication of the amount of time (possibly related to a level of effort) that elapses between a user sending a Video Message and a receiver receiving the Video Message. FIG. 1 depicts such a relationship between a method of sending messages called NetMeeting, that is known in the prior art, and another technology that is described in copending U.S.S.N. 09/497,587. On the left end of the diagram is "Real-time," meaning an absolute minimum lag time between video capture and viewing. On the right end of the diagram is "Deferred-time," meaning that the amount of lag time between video capture and viewing is unknown. The lag time can in principle be infinite (i.e., the video may never be viewed), for example because the intended recipient has no interest in viewing the video. NetMeeting is software that is available in version 3 as part of Windows 2000 from Microsoft Corporation. Information about Netmeeting can be found at Microsoft's website, www.microsoft.com. Netmeeting provides an immediate coupling between the sender of information and the receiver of the information. Once a messaging connection is made using the NetMeeting technology, the receiver renders the video sent by the sender. The receiver cannot turn off the video (or "opt-out" of viewing the video) except by completely terminating the NetMeeting session. The NetMeeting technology has the problematic features of being computationally intensive and requiring a large transmission bandwidth. The NetMeeting technology requires that the video must be processed in real time, because it is analogous to two- way television broadcasting over a network. The real time processing includes capturing the video, compressing the video, streaming the video, decompressing the video and rendering the video. Because all of these processes must be performed in real time, the NetMeeting real-time technology is beyond the capacity of the computers available to many non-professional or consumer users. In a "deferred-time" application such as the VideoShare Producer which is described in the copending U.S.S.N. 09/497,587, a receiver of the video message must formally acknowledge the receipt of the message and actively initiate the downloading and viewing process. For example, a potential viewer must activate his or her email browser, receive a new email message announcing the availability of the video, read the subject body, have interest in the email body content, activate the associated tag link (for example, by clicking on it), be connected to the World Wide Web ("Web"), and activate a "play" button in a viewer software application. With some deferred-time applications, there are several stages at which the user can choose to ignore the message, or can avoid taking a necessary step to receive and view the video. The sequence of actions necessary to view a video may result in a receiver not viewing the video. Some commercial senders of videos, such as advertisers, may want to limit the possibility that a receiver might ignore or "opt out" of viewing a video. Symmetry
Applications that involve the transmission of videos over a network, such as the Internet, generally use an asymmetric system design that divides the supporting technologies into two distinct groups: the video messaging "Sender" and the video messaging "Receiver." The "Sender" uses a sender computer provided with one set of software tools to capture, edit, and otherwise generate streaming video content. Prior art software products that perform the "sending" operations include the Microsoft Media Encoder and the Real G2 Producer. The VideoShare Producer of the copending U.S.S.N. 09/497,587 is another system and method for sending a streaming video from a sender to a server and then on to a viewer. The "Receiver" uses a receiver computer that is provided with a different set of software tools to receive, manipulate, and display the video. Prior art software products that implement the "receiver" functionality include the Windows Media Player and Real G2 Player.
In asymmetric systems, the sending computer often cannot determine important features of the receiving computer. The receiving computer also often cannot determine important features of the sending computer. As an example, the sending computer generally does not have any information about the kind of software that the receiver computer has available. The sending computer generally cannot determine the capabilities of the receiver computer, such as CPU power or available network connection speeds. The sending computer generally knows nothing about the default control setting of the receiving computer as regards security protocols, preferred formats, and the like. In general, the application running on the sending computer and the application running on the receiving computer do not communicate directly. The "Sender" transmits a message for receipt by the "Receiver," without assurance that the "Receiver" is available to receive the message, or that the "Receiver" can decipher the message that it does receive. Furthermore, as a practical matter it is difficult, if not impossible, to extend the features of one of the "Sender" and the "Receiver" without a corresponding improvement or addition of capabilities by the other. Software for use in such applications is written and developed by various organizations. In the absence of uniform standards, each developer can prepare software that has different features from the software of others. Some of the features can be proprietary. As one example, it is difficult to extend the features of both the video sending and receiving aspects at the same time. In such instances, a "Sender" and a "Receiver" may find that they cannot communicate. An example of such a failure to be able to communicate that is well known is the inability of some older Web browser software to correctly render Web pages prepared for use with different browser software because the embedded commands in the pages were not understood or were incorrectly interpreted by browsers other than the intended browser software. Network Structure
The VideoShare Producer system and method, which is described as one embodiment in copending U.S.S.N. 09/497,587, is a solution that depends on the operation of a server. FIG. 2A depicts the structure of the VideoShare system and method as represented by the VideoShare Producer embodiment. A "sender A" using sender computer 302 generates or obtains a video that the "sender" wishes to share with or publish to a "receiver B." In one embodiment described in copending U.S.S.N. 09/497,587, sender computer 302 has a copy of the VideoShare Producer software that is used to obtain, prepare and transmit the video. In the VideoShare system and method, the sender computer 302 of "sender A" appears to have a direct connection to the receiver computer 304 of "receiver B." The dotted arrow from "A" to "B" denotes the apparent connection. In practice, the sender computer 302 of "sender A" transmits the video segment to a server 306 that is operated by a hosting service, such as the VideoShare service. This upload is denoted by the arrow extending from the sender computer 302 to the VideoShare server 306. The VideoShare server 306 in turn notifies the receiver computer 304 of "receiver B" that a video is available for viewing. Upon an acknowledgment from B that the video is desired by B, the VideoShare server 306 streams the video to the receiver computer 304 for viewing by B. The streaming of the video is denoted by the arrow extending from the VideoShare server 306 to the receiver computer 304. The receiver computer 304 uses software designed to display a streaming video, such as the Microsoft Media Player, for example, in order to display the video to "receiver B." In the VideoShare Producer system and method, when A, using sender computer 302, wants to send B, who uses receiver computer 304, a video message (for example, a Video Greeting Card), the apparent connection is between the computers 302, 304 of A and B directly. However, the actual data connection is between A's sender computer 302 and the VideoShare server 306 and then between the server 306 and B's receiver computer 304.
The steps in the process are outlined in overview in flowchart 201 in FIG. 2B, and as described in U.S.S.N. 09/497,587. At step 202, Sender A records a video message using sender computer 302 and associated hardware and software. At step 203, Sender A uploads the video message to VideoShare Server 306. In step 204, VideoShare Server 306 notifies Receiver B, who operates receiver computer 304, that a video message is available. In step 205, Receiver B acknowledges the notification (and requests delivery of the message). At step 206, the VideoShare Server 306 streams the video message to receiver computer 304, where Receiver B views the message. The apparent transaction, that of Sender A sending a video message to Receiver B, does not in fact occur directly, but only through the intermediation of the VideoShare Server 306, which is called on to both receive the video message and send on the video message. There is a computational load and transmission bandwidth requirement for each transmission of the video file. The operational costs, computational load, and bandwidth requirements to service a growing user base become ever larger as the number of users and the amount of traffic increases.
In the model embodied in the VideoShare service, videos are uploaded to, stored on, and streamed from a server system, such as server computer 306, which can comprise one hosting facility at one location or multiple facilities at multiple locations. Each hosting facility can service a large number of concurrent users. The VideoShare Producer system and method can be considered topologically to be a "Star Network" where all clients machines that are operating in the system are in constant communication with a node, and all video traffic passes through the server.
Such topologies have three main limitations. First, as the number of client computers concurrently using a server grows, the load on the single server node of the network increases rapidly. For example, in the system of FIG. 2, in which only A's sender computer 302 and B's receiver computer 304 are connected via server 306, A can send a video only to B. However, if there are many additional users operating computers connected to the server 306, for example users C through Z, A can now send the same video to a plurality of recipients, namely B through Z. That is, the number of potential interactions (i.e., transmissions) increases at a rate far faster than the rate of increase of users. In the limit, the number of interactions is given by the theory of the number of combinations of N things, where N is an integer greater than 1 , taken two at a time. This number increases at a rate of the order of the square of the number N of client computers and their users. As the load grows, the server's finite resources must be ever more thinly distributed to each client computer. In such a case, increased success in sharing video over a network requires more infrastructure to continue to perform the necessary tasks at an acceptable level of performance.
Second, a failure on the single node of the network causes a large disruption of service to the connections of many client computers to the server. It is not uncommon that a single server problem can create communication problems for tens-of-thousands of client computers (and their users) at once. In addition, disturbances of a server that are less than complete failures can seriously degrade the service of many, for example by lengthening the time required to process requests that pass through the server.
Third, in a network that has a server, the costs to build, to operate, and to maintain the server can be large. In particular, when the size of the typical message is large, such as is the case with video files, and the amount of traffic increases, the costs to maintain the ability of the server to perform at an acceptable level can escalate quickly. Caching & Routing
A network that employs a server to handle data that flows between sender computers and receiver computers will in general require additional storage capacity in order to hold the messages that are in transmission to receiver computers that cannot immediately accept the messages. This situation involves having one or more caches or storage such as hard disks to hold the messages for later transmission from the server to the intended recipient computer. The server will then have to deal with the overhead of storing and retrieving messages in addition to the demands of receiving and sending the messages. If the server is also responsible for routing the messages sent between a sender computer and a receiver computer, the server will spend time communicating with other computers to determine whether sufficient bandwidth exists on a particular connection to carry the intended message, as well as time spent by the server in sending the message itself. Extensibility
Extensibility refers to the ease or difficulty (and in the extreme, the very possibility) of expanding a system, for example by adding additional capacity and/or additional operational features. Normally, when a piece of software is released to the public, extensibility is accomplished through different versions that are sold separately as complete packages or through upgrades. The user often is charged for upgrades to software packages. For example, version 1.0 of a word processor generally has fewer features than version 2.0 of the same software. The end user who wants to upgrade a software package must install the new version of software manually, typically either through an Internet download or through a purchase. With the advent of the Component Object Model (COM) in 1995, operating systems, such as the Windows operating systems, made available runtime binding of software objects. Runtime binding means that when the software application is run, a number of software objects, each comprising one core functionality, can become active and can create connections to one another. In the COM model, a particular object can be replaced with a different version of that object, with an associated change only in the function that the particular object provides. Therefore, in the Component Object Model, a software release is a collection of individual, replaceable, objects, rather than a single monolithic piece of software that must be reassembled in order to change some feature of the code. As long as software developers maintain a consistent Application Programming Interface (API), it is possible to exchange one version of a software object for another at runtime (i.e., perform an upgrade of a selected functionality). The Invention and the Attributes of Immediacy. Symmetry, Network Structure, Caching & Routing, and Extensibility
The present invention, one embodiment of which is called the Instant Video Messenger (IVM), relates to methods and systems that provide advantages in Immediacy, Symmetry, Network Structure, Caching & Routing, and Extensibility. Immediacy
FIG. 3 shows the relationship of an embodiment of the invention to the prior art NetMeeting technology and to the VideoShare Producer technology of the copending U.S.S.N. 09/497,587. The embodiment is called the Instant Video Messenger (IVM). IVM falls into an intermediate position along the horizontal "immediacy" axis because the time for delivery from Sender A to Receiver B may range from nearly instantaneous to a lengthy interval, as will be described in more detail below. IVM is neither "Real-time" nor is it fully "Deferred-time". IVM allows content creators the ability (and the possibility) to send streaming video content with little delay to viewers with few chances to "opt-out." IVM can be used in conjunction with both free personal use or paid-for professional services. Svmmetry
With the VideoShare Producer described in copending U.S.S.N. 09/497,587, functionality can be added to the Producer (e.g., video sending) and the web site (e.g., storage, notification, and the like). However, functionality generally cannot be added to the playback features that are available to a user who employs a third party playback application, such as the Windows Media Player, for example.
Even if there is an agreement among software providers as to the features to be accommodated, there can still be "run-time" issues relating to compatibility. For example, a change of preferences or status by one of the sender computer and the receiver computer (e.g., changing connection speeds) needs to be communicated to the other, and there may be no mechanism to do so. For example, if a computer begins a new task, in particular in a multithreaded operating environment, the capacity available to those tasks that were running when the new task begins may have to be reduced. As an example, a communication link that exists when a new task begins may have to slow down to allow the computer's CPU to have an opportunity to handle the new task, and conversely, when a task ends, the CPU is freed of a requirement to service the terminating task, and can devote more effort to the communication, link which may then be capable of speeding up.
According to one embodiment of the invention, the network configuration that supports the Instant Video Messenger is completely symmetric. The software running on the sender computer is exactly the same as the software running on the receiver computer, with the possible exception for differences due to version updates. The "Send/Receive" package can be prepared in a modular form comprising one or more Dynamic Link Library (.DLL) files, so that updating an installed copy of the software can be carried out by replacing an older version of a .DLL file with the most current version. The Instant Video Messenger is able to address the "knowledge" issues (i.e., the sender computer "knows" about the software operating on the receiver computer), because both computers operate the same software. The Instant Video Messenger is able to address the "feature extensibility" issues, (i.e., issues that relate to improvements that appear in both the sender software and the receiver software at the same time) because the software has a common origin, and is modular, thereby allowing updates of the corresponding transmit and receive modules at the same time. Network Structure FIG. 4A shows a topology in accordance with one embodiment of the invention. According to the invention, the IVM client software that operates on a sender computer 302 has the capacity to transfer the contents of a video message directly to other IVM client software operating on a receiver computer 304. Each Sender/Receiver is capable of streaming video itself, without reliance on a network centralized server. In the IVM system servers have a more limited role than in the traditional client/server network model. In an IVM system, servers, such as the server 406, are responsible for database transactions, which require limited bandwidth and computational power in comparison to the higher bandwidth and computational requirements for streaming video from a server, in the traditional client server model. In one embodiment, the server 406 has only limited database requirements. Such limited requirements for transmission bandwidth and for limited database manipulation simplify the replication of the server 406, because the server 406 need not be expensive or complicated. Ease of replication of the server 406 is useful to provide redundancy in case of critical operational failure of the server 406, and to allow the addition of more such servers as the IVM system grows (i.e., more IVM client computers are added) and user demand for the IVM service/system increases. According to the present invention, the embodiment called the Instant Video Messenger does not rely on a video streaming center 306.
FIG. 4B shows the steps in the process in overview in flowchart 401. At step 402, Sender A, using sender computer 302, asks the IVM Server 406 if Receiver B is online via receiver computer 304. IVM Server 406 responds that Receiver B is online. At step 403, Sender A records a video message using sender computer 302 and associated hardware and software. At step 404, Sender A, using sender computer 302, notifies Receiver B that a video message is ready for transmission to Receiver B via receiver computer 304. In step 405, Receiver B acknowledges the notification (and requests delivery of the message). At step 206, Sender A, using sender computer 302, streams the video message to receiver computer 304, where Receiver B views the message. The apparent transaction, that of Sender A sending a video message to Receiver B, occurs directly without the intermediation of the IVM Server 406, which not called on to either receive the video message or send on the video message. The exemplary process that occurs as described in FIG. 4A is distinct from the process as described in FIG. 2A, which requires the intermediation of the VideoShare Server 306 . Caching & Routing
The Instant Video Messenger dynamically routes video messages across its network. The IVM server 406 can determine whether any particular sender computer 302 and/or receiver computer 304 is/are on-line and available to send or receive video information (e.g., video files). The IVM server 406 also can re-route messages if network (e.g., Internet) traffic volume creates a problem with transmission between the sender computer 302 and the receiver computer 304. In addition, the IVM system can employ nodes that can store the messages that pass through them, enabling archiving applications, as described in more detail below. In one embodiment, a pool of video content can automatically build up by using such archiving applications. A node used to store such content can later be used to redistribute the material. Extensibility
The Instant Video Messenger uses the COM technology to provide extensibility. When a new version of an object that uses the COM technology is developed, a user who has the old version is notified of the availability of the software object upgrade. The user can also be notified of the availability of new objects that provide additional functionality. For example, VideoShare has created a video effect COM API for use by software developers. This API provides an interface for the addition of new and improved video effects. Examples of video effects that are available in expensive, professional quality systems, and that can be provided in the IVM software, include cross-fading, titling, and blue-screening. As objects that incorporate or provide such functionality become available from the development community, they can immediately be available to the IVM end user, without the need for purchasing an upgrade or downloading the entire IVM software package. The IVM software thus provides the feature of extensibility.
Since the Instant Video Messenger is symmetric, COM objects that provide new features can be prepared for both the sender computer (i.e., generation of material for transmission that enables a new feature) and for the receiver computer (i.e., the presentation of the material of the new feature to the recipient). To maintain complete symmetry, the upgraded COM objects for the sender and for the receiver are released together. Examples of new features include capabilities such as customized streams and meta-data that associate information with URL links and other ties to electronic commerce (e-commerce), as well as other hooks that enable a wider range of uses of video rather than just watching the images. A hook is a code segment that presently performs no useful function other than being a placeholder, but that can be substituted with another COM object that performs a desired function. An example of a hook is a subroutine that includes only the command "return." These new and improved COM objects can be added to the IVM software at any time.
The Instant Video Messenger permits, but does not limit, users to include additional information streams that are synchronous with the audio/video content. These peripheral synchronous information streams will be sent simultaneously with the primary audio and video streams. For example, the user can use these additional synchronous information streams to display business presentation slides. When the audio and video stream reaches a particular time location in the stream, for example when a CEO describes his or her business, a business presentation slide appears on the Instant Video Message receiver's computer. Another example of additional information streams associates "clickable-regions" (i.e., regions that are linked to an action that is activated by using a pointing device such as a mouse to highlight the region and then activating or clicking a mouse button) within a video frame, enabling the receiver of the Instant Video Message to bring up additional information (e.g., text descriptions such as a help screen, hyperlink URLs, and the like) by merely clicking on different areas of the video frame while watching the video. Instant Video Messenger Solution
As shown in FIG. 4, the Instant Video Messenger uses a direct peer-to-peer connection between the computer 302 of A and the computer 304 of B. If A wants to send B a video message, the computer 302 of A sends the video content directly to the computer 304 of B. The video can be in either streaming or non-streaming format. This entire transaction is under the control of an Instant Video Messenger Server 406 (IVMS 406) that coordinates the run-time state of all members of the IVM Network (IVMN). The Instant Video Messenger Server 406 is notified when a given IVM user is on- or off-line, where the user is located (e.g., the user's IP address), what the user's Internet connections are (e.g. port address, firewall, proxy server configurations, and the like), and what the user's general activity has been (e.g., number of videos sent and/or received). The Instant Video Messenger Server 406 informs the sender computer 302 either that the receiver computer 304 is on-line (and a transmission is thus possible) or that the receiver computer 304 is not on-line. However, the Instant Video Messenger Server 406 does not have to deal with the details of, or the control of, a subsequent transmission. Furthermore, as depicted in FIG. 5, the IVM Server 406 is capable of sending messages, for example video messages, on its own account to the computers 302, 304 of both users A and B. For example, after Sender A, using sender computer 302, streams a video message to Receiver B, using receiver computer 304, the IVM Server 406 can send its own video message (or messages) to either or both of Sender A and Receiver B via each user's computer 302, 304. Such a message can be, for example, video advertising, video promotion material, or other videos related to a particular theme as part of the services provided by the IVM Network. In one embodiment, the message sent by the IVM Server 406 appears in a window such as that depicted in FIG. 16, for example when either or both of the Sender A or the Receiver B is using Microsoft Outlook for e-mail service.
As shown in FIG. 16, the Windows Media Player 910 is displayed with the message pre- loaded. The recipient of this embedded message only needs to activate the play button 920 on the Windows Media Player to see the video segment, rather than going to a URL hyper-link. The embodiment includes the conventional dialog boxes for entry of an email address for a recipient (box 902), a "carbon copy" ("cc") address (box 904), and a subject (box 906). In the embodiment shown, instructions are presented below the Windows Media Player 910 for the convenience of the recipient. This embodiment can also be used by a Receiver B to receive and view a message sent by Sender A.
In one embodiment, depicted in FIG. 6, TVM users are grouped together by IVM Channels. IVM Channels can represent areas of commonality of interest, such as hobbies, business interests, sports interests, and the like. IVM Channels can be used to form a convenient set of well-organized users, for receipt of video messages such as advertisement, product/service offerings, and special offers (for example, special saver airline fares). In one embodiment, this feature of the IVM network allows the IVM Server 406 to deliver instant video messages to a well-organized base of users. The users who are members of a Channel may also use the Channel to contact each other. In accordance with the invention, a sender using sender computer 302 who desires to send a video message to a receiver using receiver computer 304 communicates with one or more IVM Channels IVC1 602, IVC2 604, IVC3 606, IVC4 608 in the set of IVM Channels 600 located on the network, such as the Internet, to find out if the receiver computer 304 is on-line. These communications are denoted by the bidirectional arrow that interconnects the sender A and receiver B with the set of IVM Channels 600. If the receiver computer 304 is on-line, the sender computer 302 can transmit the video message directly to the receiver computer 304, as denoted by the arrow from A to B. The software, when operating on the receiver computer 304, performs a number of useful actions. The software can determine that the sender computer 302, or another computer, has sent a video message to it. In one embodiment, the software can receive and display the video message for the viewer of the receiver computer 304 without any activity or intervention by the viewer, and without the necessity of an action by a viewer of the receiver computer 304. The software can invoke other software modules to accomplish the display of the video message, again without any activity or intervention by the viewer, and without the necessity of an action by a viewer of the receiver computer 304.
The software can issue a notification message to "Receiver B" (e.g., the operator or viewer of receiver computer 304) that a message is being sent to receiver computer 304. In one embodiment, the software can detect and display information about the sender and the message. The software can display for "Receiver B" any or all of an identifier of the sender of the video message, a subject of the video message, and a title of the video message. This information can be used by "Receiver B" to decide if, and when, he or she wants to view the video message. In this embodiment, the software can additionally present to a viewer of the receiver computer 304 an opportunity to select one of accepting the streaming video message, and rejecting the streaming video message. When the viewer makes a choice, the software accepts the response of the viewer and processes the message in accordance with the response of the viewer. As an additional feature, the software can present to the viewer any combination of choices that includes rejecting the streaming video message and at least one of accepting the streaming video message for immediate display, and accepting the streaming video message for storage in a machine-readable memory, for display at a later time. In response to a selection of the choice of accepting the message for storage in a machine-readable memory, the software can, for example, direct the video to be saved on a machine-readable medium, such as a hard drive or any machine- readable medium that makes the video message available to the viewer at a later time.
The machine-readable medium can be, for example, a computer floppy disk, a computer hard drive, a magnetic tape or the like, a CD-ROM, computer memory such as static or dynamic RAM, ROM, PROM, EPROM or the like, and/or any other mechanism or medium for storing machine-readable files, instructions, data or software. In a network, the machine-readable medium can be physically attached to a computer different from one on which the data may be used, or the software may operate. For example, in a network, an archival copy of software can reside on one computer and a copy can be copied to another computer, where the copy is executed or otherwise used. If transfer time is not an issue, as when a viewer of a video puts off viewing to a later time, a file containing data or information (such as a video, a text file, a database file, a spreadsheet template or the like) may reside on the same computer as the one that received the file, or on a different computer that stores the file for the convenience of the viewer. Each IVM client computer maintains a personal IVM Contact List for the user of the computer, which is a list of other IVM users that are allowed to send messages to and to receive messages from the user. A user employs one of several notification methods such as email or Instant Messaging forums such as ICQ to invite another individual to communicate. Furthermore, the user can invite other individuals who have computers connected to the network, but who are unregistered with the IVM Network and whose computers do not have installed IVM software, to become users through an install executable that is sent in an email notification. The receiver of the email message can agree to accept the message by opening the email data attachment. This email data attachment contains the initial software codes needed to install the Instant Video Messenger software on the receiver's machine. His or her computer automatically executes the installation program, and the most recent version of the IVM software is installed and prepared for immediate use.
The installed software enables the transmission and receipt of message components, which can include, for example,
1) Video stream
2) Audio stream
3) Thumbnail image
4) Subject text
5) Message body text 6) Document attachments (file transfers such as WinWord, and the like)
7) Power-point slide presentations
8) Meta-data
A video message includes at least a video stream.
When a user operating a sender computer 302 wishes to send a video message to a receiver who operates a receiver computer 304 that is not on-line at that time, the Instant Video Messenger Server 406 (or an IVM Channel 602, 604, 606, 608) allows the sender to transmit a message for later reception and viewing by the receiver. FIG. 7 depicts an embodiment of a system and method that provides such functionality according to the invention. One embodiment involves a connection with the VideoShare technology described in copending U.S.S.N. 09/497,587. The sender computer 302 has a video message that the operator of the computer 302 wishes to transmit to the receiver computer 304. However, receiver computer 304 is offline. One of the IVM Channel Servers 602, 604, 606, 608 informs the sender computer 302 of this situation. The sender computer 302 is prompted by the IVM Channel Server 602 (for example) of the possibility of storing that video in the sender's VideoCenter supported by VideoShare server computer 710 for later delivery to the intended recipient. The sender computer 302 can send the video message for storage at server computer 710. Server computer 710 can transmit the video message, for example, via e-mail or via Web retrieval as described in copending
U.S.S.N. 09/497,587, to the intended recipient computer 304 when that computer 304 is again on-line.
The Instant Video Messenger system includes a Instant Video Messenger Channel Provider Kit that enables third parties (e.g., affiliates, partners, co-developers and the like) to provide additional Instant Video Messenger Channels to the IVM community. The Instant Video Messenger Channel Provider Kit includes all of the components for hosting an IVM Channel on a server computer. The components include, but are not limited to, software, database connectivity, and communication protocols. The IVM Channel Provider dedicates a server computer to manage all of the communications between the server and the computers of the channel users (or channel members). The IVM Channel Provider installs database software such as Microsoft SQL 7 on the server to allow the server to handle a database that identifies the users and their computers. The IVM Channel Provider uses the database software to manage information to be communicated to the channel members, and configures the IVM Channel Provider software with an IVM Channel name and description. The IVM Channel Provider (or Host) can decide how it wishes to implement the service based on the expected demand for its IVM Channel services.
These channels can be directed towards a particular customer segment, for example a segment based on thematic areas or common interests, such as for example, travel, auctions, or computers. As additional IVM Channels are developed by Instant Video Messenger Channel Providers, IVM users can be notified of their introduction. Furthermore, the IVM Channel Provider Kit enables third party organizations to insert additional video streams before and/or after video messages that are sent from one user to another, including, but not limited to, advertisements in the form of video, still images, audio, and text. An alternative embodiment of the messaging systems and methods of the invention is depicted schematically in a flow diagram 800 shown in FIG. 8. The sender (i.e., Sender A) of the video message begins by launching the computer communication module 3000, known in this exemplary embodiment as the Videoshare 2Peer application software 3000, on his or her computer 302, as denoted at box 805. The sender uses the 2peer application software 3000, and so is also referred to as the "user" in FIG. 8. Using the sender computer 302, the user logs into a 2peer channel, as indicated at box 810. A 2peer channel is an embodiment of an IVM channel, generally 600, of FIGs. 6 and 7. The user records a message comprising video and possible audio or other components, as indicated at box 815, which is reached from box 810 by travelling through Node A. In one or more optional steps, generally denoted by dotted box 820, the user can attach meta-data to the video message. At box 825, the user selects at least one recipient. The user can optionally select more than one recipient. For the purpose of the description, it is assumed that only one recipient is selected. However, if a plurality of recipients are selected, each recipient is handled individually, without regard for the situation of any other recipient.
The user communicates, using the 2Peer application software 3000 and the sender computer 302, with the IVM server 406, and requests the online status of the intended recipient (i.e. Receiver B). IVM server 406 checks whether the receiver computer 304 of Receiver B is online via an IVM channel. The result of the inquiry is communicated to the sender. All of the request, the check, and the communication of results of the check is denoted by the box 830. Based on the online status of the receiver computer 304, either of two paths are selected.
If the receiver computer 304 is online, corresponding to the arrow labeled "yes" exiting box 830, the Receiver B is notified, as indicated at box 835, using receiver computer 304 and a copy of the 2Peer application software 3000 thereon, that Sender A wishes to send a video message to Receiver B. At box 840, the Receiver B selects whether he or she wishes to accept the video message, decline the video message, or defer receipt of the message. If the receiver accepts the message, as denoted by box 845, the sender, using sender computer 302 and the 2Peer application software 3000 thereon, steams the video to receiver computer 304, where the Receiver B can view the video message. The user is then carried in the system and method back to Node A, where the process can be repeated, or the user can terminate the process as indicated by the box 850.
If instead, the Recipient B declines the video message, the system and method operate by informing the user that the recipient has decline the message, as indicated at box 855. The user is then carried in the system and method back to Node A, where the process can be repeated, or the user can terminate the process as indicated by the box 850.
In the third alternative, the Recipient B defers receipt of the message, as denoted by the arrow from box 840 to box 860. Box 860 indicates the action that the sender is notified that receipt of the message is being deferred by the recipient. If the recipient computer 304 is offline, the sender is informed that the receiver computer is not available to accept a video message, as indicated at box 865.
In either the case of a recipient deferring acceptance of a video message, or the case of a receiver computer 304 that is not online, the systems and methods of the invention allow a substantially similar series of steps to take place. At box 870, the sender, using the sender computer 302 and the 2Peer application software
3000, is asked by the IVM server 406 whether the sender wishes to store the message that cannot be delivered immediately. If the sender declines to store the message, the systems and methods of the invention carry the sender back to Node A, where the process can be repeated, or the user can terminate the process as indicated by the box 850. If the sender does wish to store the message, the sender's status as a registered member of the VideoShare service is checked, as indicated at box 875. If the sender is not a registered member of the VideoShare service, the sender is registered for the service, as indicated at box
880. If the sender is a registered member, or has just registered, the sender logs on to the
VideoShare service, as denoted at box 885. The sender then transfers the video message, using sender computer 302, to the VideoShare server 710, as indicated at box 890. The systems and methods of the invention carry the sender back to Node A, where the process can be repeated, or the user can terminate the process as indicated by the box 850.
As indicated ay box 895, when the VideoShare server 710 determines that the receiver computer 304 of Recipient B is online, it streams the video message file to the receiver computer 304 in a manner that is transparent to the sender, who may not be online when such streaming occurs. Sample Applications
The rich feature set and wide range of video and audio messaging is also suited to a corporate environment, particularly enabling remote communications such as intra-office video memos as well as messages sent between remote and home offices. The Instant Video Messenger can, for example, be used by co-workers to keep in touch with each other while working on a project, including such activities as sending text, audio, and video messages to one another while sharing collaborative documents. Implementation Details
The entire Instant Video Messenger software system can be implemented in C++ within the Microsoft Foundation Class (MFC), ATL/COM, and ActiveX application frameworks. Each functional component (capture, compression, saving, contact list, etc.) can be a separate .DLL file. Such modularity allows easy upgrading, as users only have to download the components that have changed (generally files measured in the 10s of Kbytes).
The Instant Video Messenger software is distributed and installed on a user's computer that has a connection, either temporary or permanent, to a network, such as, but not limited to, the Internet. The user may have a video capture device, such as a Web Camera or a video digitizing card, but it is not required for operation of the Instant Video Messenger. Users that do not have a video source can still elect to load in pre-existing media files and can send those media files as Instant Video Messages. Communications between each member of the communication session and the Instant
Video Messenger Network can be performed via Transmission Control Protocol/Internet Protocol (TCP/IP) socket connections or User Datagram Protocol (UDP) connectionless datagrams. The user can select which TCP/IP or UDP network Port to use for communications as well as video streaming. In one embodiment according to the invention, the software structure is organized as a three-tier software system. The lowest level tier is a COM based set of functional components that implement the core functionality such as video capture, audio capture, network communication, and contact list management. The middle software tier exists as an interface layer which binds the lower level software tier to a graphical user interface such as windows, buttons, text, etc. This software layer is written as ActiveX components. The top software tier is written as an Microsoft Foundation Class (MFC) desktop application, grouping together the ActiveX components into a final product offering. This software installs itself (upon startup of the computer) into the Windows Tool Bar as a small icon that can be opened and closed at the user's will. The software runs on the user's machine until it is shut down or manually terminated.
Referring to FIG. 9, a user of the system, such as a private individual working from home, or a professional working from a business, employs a computer system 302. The user has an intention of sending a video message to a viewer of the message who is situated at another computer, that will be described in more detail below. The user of the computer system 302 may also be referred to as the "sender" or "Sender A", and computer system 302 may be referred to as the sender computer 302. The computer system 302 can include a computer which can be a personal computer of conventional type such as a desktop or laptop computer, a hand held device such as a PDA, or a more powerful computer such as a workstation, a server, a minicomputer, a mainframe, or the like.
The computer system 302 can operate software including a web browser such as Microsoft Internet Explorer or Netscape Navigator or Communicator or the like, for communication over a network such as the Internet via the World Wide Web (hereinafter "the Web"), or to permit wireless communication. The computer system 302 can operate software that can manipulate video segment files. The computer system 302 can communicate with video sources, such a video cameras and video recording machines, if the user wishes to employ such sources. Conventional commercially available personal computers typically have sufficient capability to meet these requirements. The computer system 302 can also employ video segments generated digitally by the computer and appropriate software, or by another computer, if the user wishes to employ such techniques. In one embodiment, the computer system 302 operates one or more modules that are part of a computer communication module 3000 called VideoShare 2Peer. The VideoShare 2Peer 3000 is a software application package that the user can download from the Web site www.2peer.com or that the user can obtain in other formats such as on a CD- ROM or bundled with other software or hardware. In the present invention, one or more modules that are components of the VideoShare 2Peer 3000 are present in software that the sender computer 302 controls. The one or more modules of the VideoShare 2Peer software can be operated by the user under his or her control on his or her computer, in the computer system 302, in order to provide the capability of recording, converting, and optionally, compressing video segments, creating one or more identifiers for a video segment, and transmitting a video segment with one or more of the identifiers to a receiver computer 304 when the receiver computer 304 is simultaneously online with the sender computer. If the receiver computer 304 is not online, the sender computer can optionally transmit the video segment to a host computer 60 for storage at a location under the control of the host computer 60. The host computer 60 will be described further below. In addition, the software modules can provide the capability to make a request of a server computer 406 that server computer 406 determine the online status of a receiver computer 304 to which the sender would like to send a video message, and that the status be reported to the sender computer 302 so that the sender computer 302 can determine whether sending the video message directly to the receiver computer 304 independent of the server computer 406 is appropriate, or whether the video message should be sent to a host computer 60 for storage and for later transmission to the receiver computer 304.
The computer in the computer system 302 of the user one can be connected to one or more kinds of equipment for generating video segments, such as a video camera such as a Web cam 12 or another type of video camera such as a professional quality video camera. The computer in the computer system 302 of the user can be connected to one or more kinds of equipment for providing prerecorded video segments, such as a video recorder 14, or another computer that can create digital video segments through the use of suitable software, such as for example digital video segments that have been created for various commercial films, or the like. Once the sender has obtained a video segment, and has manipulated it according to the procedures described below with regard to the operation of the software comprising the computer communication module, or its equivalent, the video segment with one or more identifiers is transmitted to the receiver computer 304 directly if the receiver computer 304 is online, or the video is sent the host computer 60 for storage and for retransmission to the receiver computer 304 at a later time. The host computer 60 can include one or more server computers 62, 62', 62" that communicate via a network such as the Web with other computers, such as the computer in the user's computer system 302. The one or more server computers 62, 62', 62" also communicate with a storage array 64, or optionally with a plurality of storage arrays substantially similar to storage array 64. The storage array 64 can be any convenient storage system, such as a redundant array of magnetic storage disks, one or more readable and writeable CD-ROMs, random access semiconductor memory, any combination of such storage devices, or the like. The host computer 60 can connect via the Web to one or more computers that comprise the Web, conceptually denoted by the box 70.
A video segment that is stored and controlled by the host computer 60 may be delivered to and displayed for a viewer in a variety of formats, and through a variety of methods, as denoted generally by the box 80. In different embodiments, a video segment can be displayed as: a video greeting card 81, such as a person wishing another a happy birthday; as video email 82, as video that can be viewed on a remote website 83 (e.g., a video segment embedded into the remote website so that a viewer who visits the remote website sees the video segment as part of the page that is presented); as video commerce 84, for example a video that depicts a person describing his or her experience and training as part of a resume submitted on-line; or as a video advertisement 85, for example a video depicting the benefits or showing the use of a product. Many other like applications of the technology can be envisioned. The video segment can be made available to the viewer as a streaming video that is sent to the viewer. In some embodiments, the viewer can use a hand held device such as a PDA or a cellular telephone that can connect to a network such as the Internet to view the video segment.
In FIG. 10, the computer 16 of the user's computer system 302 is shown. The box 18 is intended to schematically depict a user of a computer video input device, which device can be the computer 16 operating suitable software to generate digital video, or can be another such computer, or can be the web cam or video camera 12, or can be the video recording device 14, or the like. The user begins by producing and/or recording a video segment on the hard disk of the computer 16 or within the temporary memory of a handheld device. As a second step, the video segment of step 1 can optionally be compressed and /or can be changed as regards the computer file format in which it is recorded on the hard disk. As a third step, the sender uses the computer 16 to request from the server computer 406 an indication of the online status of the receiver computer 304. As a fourth step, if the receiver computer 304 is determined to be online, the video segment recorded on the hard drive of the computer 16 is transmitted with one or more identifiers to the receiver computer 304. In different embodiments, the identifiers can include information comprising at least one of an identifier of the sender of the video message, a subject of the video message, and a title of the video message. The fourth step is schematically depicted by the arrow pointing generally from the computer 16 to the computer 90 of the viewer. The box 92 is intended to schematically depict a user of a display device. In one embodiment, the display device can be the computer 90, or the display device can be a display device such as a Web TV, or can be a video output device such as a television set with a suitable decoder, or the like. The display device can also be a wireless hand held device such as a PDA or a cellular telephone or the like. The streaming video segment can in one embodiment be delivered as part of a video greeting card 81. In an alternative embodiment, the video can be delivered as a streaming video directly to the viewer from the sender computer , without the viewer having to activate the receiver computer 304 .
As shown in FIG. 11 , the user can obtain a copy of the VideoShare 2Peer software by downloading a copy of the software from the Website wwwNideoShare.com 50, as indicated by the picture at numeral 1. Alternatively, the user can obtain a copy of the VideoShare 2Peer software on machine readable media such as a CD-ROM or the like. The VideoShare 2Peer software can be bundled with one or more utility or application programs that are useful for a user to have, such as a "container" application so that the VideoShare 2Peer software can be operated on a desktop computer. The user can install the VideoShare 2Peer software on his or her computer 16and can register with the IVM service at no charge. In registering for the IVM service, the user obtains a username and a password that can be used to identify the user. The activity of installing the IVM software comprising the computer communication module on the user's personal computer or the like and registering with the IVM system is indicated by the picture at the numeral 2.
In order to use the system, the user first obtains a video segment. The user can create the video segment, for example with a Web cam 12, or the user can use an existing video segment obtained from a video recorder 16, as indicated by the picture at the numeral 3. The VideoShare 2Peer 3000 software includes a module that has direct capture capabilities that permit the user to create the video segment.
The user can employ the VideoShare 2Peer 3000 software modules to optionally compress the video; to determine if a video segment is in a format that is compatible with streaming video; to convert the video to a file format that is compatible with streaming video if the video segment is not already in a file format that is compatible with streaming video; and to transmit the video segment together with one or more identifiers that represent selections that the user can make (for example, a still image selected from the series of images that comprise the video segment, and an identifier of the sender of the video segment (e.g., the user). The activities carried out in conjunction with the VideoShare 2Peer 3000 software modules are generally indicated by the graphic at numeral 4. The video segment and the identifier(s) can be transmitted to the receiver computer 304 directly if the receiver computer 304 is online, or if the receiver computer 304 is not online, can be transmitted to the host computer 60 for storage and for later distribution to the receiver computer 304. In one embodiment, the video message is transmitted in a streaming video file format. This transmission activity is denoted by the graphic at numeral 5.
Upon a determination that the intended receiver computer 304 is online the host computer 60 sends the video in streaming video format to the receiver computer 304, where a viewer can observe the video in real time. The activity of serving the video segment as a streaming video is denoted by the graphic at numeral 8. Alternatively, if the receiver computer 304 is not online, the sender computer 302 can send the video message to a host computer, as indicated at numeral 6, which host can then send the video on to the receiver computer when it is online, as shown at numeral 7.
The majority of the VideoShare 2Peer software was developed as a Windows 95, Windows 98, and Windows 2000 ("Windows 9x/2000") compatible ActiveX control (e.g. an .OCX file), with additional components existing as active template library (ATL) component object model (COM) components that are instantiated during runtime. A "container application," named "2peer.exe," allows the VideoShare 2Peer ActiveX Control to be executed from the Windows 9x/2000 desktop. The VideoShare 2Peer Active X Control can also be embedded into a web page, as is done within the www.2peer.com web site. The custom written VideoShare 2Peer software includes the following binary/source code components: (1) VideoShare 2Peer ActiveX Control (2peer.ocx), (2) Streaming media rendering engine (vsrender.dll), (3) Streaming media network object (vsnetwork.dll), (4) Buddy-list Contact management (vscontact.dll), (5) 2Peer Channel management (vschannel.dll), (6) VideoCapture object (DirectShowRecord.dll and VFWrecord.dll), (7) VideoPlayback object (DSPlay.dll), and (8) Media management (vsmedia.dll)
All components were entirely written by VideoShare Inc. The VideoShare 2Peer software is built upon the following third-party technologies that provide lower-level device support, document sharing, and file format conversion: (1) Microsoft's DirectShow; (2) Microsoft's Windows Media Technologies; (3) Microsoft's Video for Windows; (4) MAPI; AND (5) ICQ. When the user launches the VideoShare 2Peer software comprising the computer communication module, he or she will see the window depicted in FIG. 12 appear on his or her computer 16 operating the Win9x/2000 operating system. The login screen can be made optional for repeat users by providing a unique identifier for the user, such as a password, or by installing on the user's computer or the like a record similar to the "cookies" used by some interactive computer systems operating on a network such as the Internet.
When the user enters in his or her username in the box 410 labeled VideoShare Login Name and his or her password in the box 415 labeled VideoShare Password and activates the "Start VideoShare 2Peer" button 420, the VideoShare 2Peer 3000 software opens a TCP/IP socket connection to the VideoShare Upload/Database Server via port 80 in order to avoid typical Firewall and/or Proxy Server problems. If the box 430 labeled Remember password is checked, the VideoShare 2Peer 3000 software will remember the user's password, eliminating the necessity to type in that information each time the software is started. The VideoShare
Upload/Database Server then verifies the validity of the username/password. Furthermore, the VideoShare 2Peer 3000 software will notify the user if there is a more recent version of the software available, giving him or her the opportunity to automatically download and install the new software. With this login dialog, the user can also receive help, by activating the "Help" button 450, taking the user to a web page on the VideoShare web site. The login dialog box can also be used to create a new VideoShare user account, by clicking the "Create Another Account" button 460. Once the login process has been completed, the VideoShare 2Peer 3000 software looks for available DirectShow audio and video capture devices. These available devices are enumerated and listed within the "Settings Tab" as described later. The VideoShare 2Peer 3000 software initializes the audio and video capture device, by recalling as a default the device that was used most recently. VideoShare 2Peer Software Preview/Capture/Import Process
After the capture device initialization, the VideoShare 2Peer software displays the window depicted in FIG. 13.
The image 510 in the middle of the window is the video input stream from the initialized, default video capture source. The image in FIG. 13 is that of an employee of the assignee of the present invention, in the offices of the assignee. The VideoShare 2Peer software automatically builds a DirectShow "preview graph" where the video stream from the video device is displayed on the screen, but is not saved to disk. This gives the user the opportunity to adjust the camera, e.g. an opportunity to correct the camera position, the camera focus, the camera angle, the magnification of the image, and the like. At the top of this window, the user is presented with five different "tabs", each presenting the user with different aspects of the VideoShare 2Peer software comprising the computer communication module. In FIG. 13, the tab labeled "Record/Playback" 520 is active, indicating that the VideoShare 2Peer software comprising the computer communication module is ready to acquire and/or display a video segment.
At the bottom of the window, there is a status message 522 that displays the current operation of the VideoShare 2Peer software comprising the computer communication module. In FIG. 13, the status message 522 prompts the user to either activate the Record button 531 to create a new video segment, or to import an existing video segment by activating the Import Video button 535, both of which are described in more detail below.
Directly below the video preview image 510 is a Capture/Playback Control Panel 530 that includes the following items:
Record button 531 which begins a new audio/video capture; Stop button 532 which terminates an active audio/video capture operation; - Play button 533 which initiates the playing back of the last recorded or imported video;
Delete button 534 which cancels the last record or import operation and begins a new video preview;
Import Video button 535 which allows the user to select a pre-existing video file from his or her hard drive;
Save and Share button 536, which in the present embodiment activates software modules that convert the current video file into a compressed streaming format, and that send the video; and
Shuttle Bar 537 which is used to control the current position of the playback file together with forward button 537 and reverse button 538, allowing the user to rewind and fast forward through the current video. The software modules that operate upon the activation of Save and Share button 536 will be covered in a subsequent section in this document in detail.
When the user begins to record a video, the VideoShare 2Peer software builds a new "Capture Graph" that renders the video stream to both the display window as well as to a temporary .AVI file on the user's hard drive. The audio/video capturing continues until the user activates the "Stop" button 532 at which point the VideoShare 2Peer software stops the "Capture Graph", destroys the DirectShow filter, builds a Direct Show "Playback Graph", and displays the first frame of the captured video as video preview image 510. When the user activates the Play button 533 the DirectShow "Playback Graph" is put into running mode, playing back the entire recorded video from beginning to end. The user can also choose to import a pre-existing video, which in one embodiment can be a file format selected from the AVI, MPEG, or QuickTime file formats, by activating the Import Video button 535. The VideoShare 2Peer software automatically renders the correct DirectShow filter to display an imported video correctly. Save and Share Process Once a video segment has been recorded or imported into the user's computer 16 that is running the VideoShare 2Peer software, the user can choose to process the video segment with various optional alternatives by activating the Save and Share button 536. When the Save and Share button 536 is activated, the video segment is archived and distributed automatically. The VideoShare 2Peer software greatly simplifies the entire process by seamlessly automating the following steps that are depicted in FIG. 14A:
- Video file format conversion, as required;
- Compression to a streaming multimedia format at a user-specified bitrate;
- Creating a "Thumbnail" JPEG snapshot of the video file, as an identifier that a user or a viewer can observe in order to assess the content of the video segment; - if RC on line, sending video message to RC;
- if RC not on line, optionally sending the resultant video and thumbnail files to the VideoShare server computers 62, 62'; and
Logging the transactions and managing the user's storage account, including causing the generation of an identification tag that the server computers 62, 62' can employ to retrieve the video segment for viewing.
FIG. 14A shows a flow diagram 600 of an embodiment of the invention in which the VideoShare 2Peer software automates a number of steps in connection with uploading a video segment by activation of the Save and Share button 536 described in FIG. 13. As indicated at box 605, a user first obtains and selects a video segment for processing for distribution. The box 605 schematically encapsulates all of the actions that a user takes as described in relation to FIGs. 4 and 5 above. When the user activates the Save and Share button 536 the actions described below that are enclosed by the dotted line 607 are automatically carried out under the control of the VideoShare 2Peer software.
The VideoShare 2Peer software subjects the selected video segment to analysis to determine whether the selected video segment is or is not in a file format that is compatible with a streaming video format, as indicated at diamond 610. Formats that are compatible with streaming media formats include formats such as MPEGs and QuickTime videos. If the selected video segment is not compatible with a streaming video format, it is converted to a compatible format, as depicted by the arrow labeled "NO" that points from the diamond 610 to the box 615, "Convert to compatible file format." The conversion process performed by the VideoShare 2Peer software creates a DirectShow filter graph that decompresses the video file into a temporary, uncompressed AVI file.
The video segment file in a format that is compatible with streaming video is then temporarily stored in the user's computer 16, for example as a file on the hard drive of computer 16. This storing step is performed if the file was originally in a format compatible with streaming video by following the arrow marked "YES" that points from the diamond 610 to the box 620, "Temporarily store file." Alternatively, the storing step is performed if the file was originally not in a format compatible with streaming video by following the arrow that points from the box 615 to the box 620.
The stored temporary file representing the selected video is then analyzed by the VideoShare 2Peer software , as represented by diamond 625, "Should file be compressed?" to determine if the temporarily stored file should be compressed. If the software determines that the file should be compressed, as indicated by the arrow labeled "YES" that points from the diamond 625 to the box 630, labeled "Compress file," the file is compressed. The compression involves compressing the video file to a user-specified bitrate, or the bandwidth that is required to view the video without disruption in the transmission. The user can select the desired bitrate via the "Settings Tab" that is described in more detail below.
The file is then converted to a streaming multimedia format file as indicated by the box 635, labeled "Convert file to streaming multimedia format ("SMF") file," as denoted by the arrow pointing from the box 630 to the box 635. If the file is not to be compressed, the flow follows the arrow labeled "NO" pointing from the diamond 625 to the box 635, and the file is then converted to a streaming multimedia format file as schematically represented by the box 635. The process that is performed by the VideoShare 2Peer software as denoted by the box 635 involves reading in the video file, frame by frame, and converting the video into a streaming multimedia format. In one embodiment, the VideoShare 2Peer software uses the Windows Media Streaming Format, known as ASF or WMF, but it is not technologically restricted to this choice. The Windows Media Streaming Format comprises MPEG 4 v3 for the video stream and the Windows Media Audio format for the audio stream. The output of this file is stored as a temporary file on the user's hard drive, in one embodiment.
The flow diagram indicates that the process makes a "thumbnail" of the video file, as represented schematically by the box 640, labeled "Create and temporarily store JPEG "thumbnail" identifier." The VideoShare 2Peer software produces a JPEG still image that is used as a reference image to the entire video file. It is an identifier of the subject matter or content of the video that a user or a viewer can readily recognize, as compared to an alphanumeric string such as a typical string used to identify a file by its drive, directory (and one or more subdirectories) and filename. Such alphanumeric identifiers are useful, but may be totally uninformative as to the content or subject matter contained in the identified file or video segment. In one embodiment, the VideoShare 2Peer software creates the "thumbnail" by taking the "middle" image of the entire video file, as measured by the temporal duration of the file. In another embodiment, the selection of an image from which to make the "thumbnail" can be left to the discretion of the user. This JPEG file is also stored as a temporary file on the user's hard drive, in one embodiment.
The next part of the process is either sending the video message to the receiver computer 304, if the receiver computer 304 is online, or optionally sending the video message to the host computer 60 if the receiver computer 304 is not on line.
Using the symmetric software present on both the sender computer 302 and the receiver computer 304, the sender 40 (or Sender A) can send the video message directly to the receiver computer 304 if the receiver computer 304 is online, as indicated at box 641. As shown in Fig. 4B, Sender A, using sender computer 302, asks IVM Server 406 if Receiver B is online via receiver computer 304. For explanatory purposes, we will assume that IVM Server 406 responds that Receiver B is online. Sender A records, or already has recorded, a video message using sender computer 302 and the associated hardware and software. Sender A, using sender computer 302, notifies Receiver B via receiver computer 304 that a video message is ready for transmission. Assuming that Receiver B wishes to accept the video message, Receiver B acknowl edges notification using receiver computer 304 and requests delivery of the message. Sender A, using sender computer 302, streams the video message to Receiver B via receiver computer 304. Using the symmetric software present on both the sender computer 302 and the receiver computer 304, the sender 40 (or Sender A) can send the video message directly to the receiver computer 304 if the receiver computer 304 is online, as indicated at box 641.
In the event that the receiver computer 304 is not online, the IVM Server 406 informs sender computer 302 that the receiver computer 304 is not online. Rather than sending a message to the receiver computer 304, the sender computer 320 sends the video message to a VideoShare server computer 710 (or host computer 60) for storage at server computer 710, and for later delivery to the intended recipient. Server computer 710 can transmit the video message, for example, via e-mail or via Web retrieval as described in copending U.S.S.N. 09/497,587, to the intended recipient computer 304 when that computer 304 is again on-line. These steps are shown in boxes 645, 650, 655, 660 and 670.
The optional part of the process is the upload operation, in which the VideoShare 2Peer software contacts the host computer 60, which in one embodiment is the VideoShare
Upload/Database Server at the VideoShare hosting facility. This portion of the automated process is denoted by the box 645 labeled "Transfer ("upload") temporarily stored SMF file and JPEG thumbnail identifier to host computer 60." The VideoShare 2Peer software notifies the host computer 60 that the user wishes to place his or her video into a repository maintained by the host computer 60, which in one embodiment can be the VideoShare VideoCenter, which is a repository of all recorded and uploaded videos to date. This upload is performed automatically via a direct TCP/IP socket connection over a specific connection port of the user's computer known as port 80. The VideoShare 2Peer software uses a standard communications protocol to perform this transfer to the host computer 60. In another embodiment, a proprietary protocol can be used, for example if one wants to maintain the security of information contained in the video segment. In another embodiment, the video segment can be encrypted in order to provide enhanced security. Both the compressed video streaming multimedia file and the thumbnail image are uploaded at substantially the same time.
As schematically depicted by box 650, labeled "Delete temporary file to conserve storage space on user's computer," the VideoShare 2Peer software removes all of the temporary files that were created in the course of the automated processing described above. This feature provides for the user a convenient, secure, and transparent process, with the benefit that the user's computer storage device(s), for example one or more hard drives, do not become cluttered with unnecessary and obsolete files.
In the event that the video message has been sent to the host, the VideoShare 2Peer software and the host computer 60 (for example, the VideoShare Upload/Database Server) will update the user's account to account for the required storage space that the video requires. The necessary logging, creation of an identification tag, and storing of the video and the associated identifier or identifiers is also performed automatically, as schematically depicted by box 655. The user can optionally add additional identification and control information about the user, and about how and under what conditions the video is to be made available for distribution, as schematically indicated by box 660. The user is automatically prompted to provide this information, but has the option to forego making a decision immediately. The transmission of video segment files to viewers is discussed in more detail below, and is represented in FIG. 14A by the box 670 labeled "Transmit file to viewer" which is outside the region 607 as an indication that the transmission of files to viewers is an action beyond the material discussed above in conjunction with the Save and Share button 536 of FIG. 13.
FIG. 14B shows a flow diagram 601 of another embodiment of the invention in which software automates a number of steps in connection with uploading a video segment. Many of the steps already described in connection with FIG. 14A also occur in the embodiment depicted in FIG. 14B, and are numbered in the same manner as in FIG. 14A. In FIG. 14B, there is first an optional step indicated by the box 604 labeled "Optional: User authentication with server" in which the User is optionally required to provide identification, such as a user name and password, that authenticates the identity of the user to the server or host computer 60. The user then obtains and selects a video segment for processing for distribution, as indicated at box 605 that schematically encapsulates all of the actions that a user takes as described in relation to FIGs. 4 and 5 above. When the user activates the Save and Share button 536 the actions described below that are enclosed by the dotted line 608 are automatically carried out under the control of the VideoShare 2PeerSoftware comprising the computer communication module.
As discussed in relation to FIG. 14A, the VideoShare 2Peer software subjects the selected video segment to analysis to determine whether the selected video segment is or is not in a file format that is compatible with a streaming video format, as indicated at diamond 610. If the selected video segment is not compatible with a streaming video format, it is converted to a compatible format, as depicted by the arrow labeled "NO" that points from the diamond 610 to the box 615, "Convert to compatible file format." The conversion process performed by the VideoShare 2Peer software creates a DirectShow filter graph that decompresses the video file into a temporary, uncompressed AVI file.
The video segment file in a format that is compatible with streaming video is then temporarily stored in the user's computer 16, for example as a file on the hard drive of computer 16. This storing step is performed if the file was originally in a format compatible with streaming video by following the arrow marked "YES" that points from the diamond 610 to the box 620, "Temporarily store file." Alternatively, the storing step is performed if the file was originally not in a format compatible with streaming video by following the arrow that points from the box 615 to the box 620.
The stored temporary file representing the selected video is then analyzed by the VideoShare 2Peer software, and optionally compressed as represented by the box 623 labeled "Optional compression of file." The file is then converted to a streaming multimedia format file as indicated by the box 635, labeled "Convert file to streaming multimedia format ("SMF") file." Alternatively, a file from the box 620 can be uploaded to the host computer 60 without being converted to a streaming format, and the conversion to a streaming video format can be accomplished at the host computer 60. The process that is performed by the VideoShare 2Peer software as denoted by the box 635 involves reading in the video file, frame by frame, and converting the video into a streaming multimedia format. The flow diagram indicates that the process makes a "thumbnail" of the video file, as represented schematically by the box 640, labeled "Create and temporarily store JPEG "thumbnail" identifier."
Using the symmetric software present on both the sender computer 302 and the receiver computer 304, the sender 40 (or Sender A) can send the video message directly to the receiver computer 304 if the receiver computer 304 is online, as indicated at box 641.
In the event that the receiver computer 304 is not online, the IVM Server 406 informs sender computer 302 that the receiver computer 304 is not online. Rather than sending a message to the receiver computer 304, the sender computer 320 sends the video message to a VideoShare server computer 710 (or host computer 60) for storage at server computer 710, and for later delivery to the intended recipient. Server computer 710 can transmit the video message, for example, via e-mail or via Web retrieval as described in copending U.S.S.N. 09/497,587, to the intended recipient computer 304 when that computer 304 is again on-line. These steps are shown in boxes 645, 650, 655, 660 and 670.
The optional part of the process is the upload operation, in which the VideoShare 2Peer software contacts the host computer 60, which in one embodiment is the VideoShare Upload/Database Server at the VideoShare hosting facility. This portion of the automated process is denoted by the box 645 labeled "Transfer ("upload") temporarily stored SMF file and JPEG thumbnail identifier to host computer 60." The VideoShare 2Peer software notifies the host computer 60 that the user wishes to place his or her video into a repository maintained by the host computer 60, which in one embodiment can be the VideoShare VideoCenter, which is a repository of all recorded and uploaded videos to date. This upload is performed automatically via a direct TCP/IP socket connection over a specific connection port of the user's computer known as port 80. The VideoShare 2Peer software uses a standard communications protocol to perform this transfer to the host computer 60. In another embodiment, a proprietary protocol can be used, for example if one wants to maintain the security of information contained in the video segment. In another embodiment, the video segment can be encrypted in order to provide enhanced security. Both the compressed video streaming multimedia file and the thumbnail image are uploaded at substantially the same time.
The next part of the process is the upload operation, in which the VideoShare 2Peer software contacts the host computer 60, which in one embodiment is the VideoShare Upload/Database Server at the VideoShare hosting facility. This portion of the automated process is denoted by the box 645 labeled "Transfer ("upload") temporarily stored SMF file and JPEG thumbnail identifier to host computer 60." Both the compressed video streaming multimedia file and the thumbnail image are uploaded at substantially the same time.
As schematically depicted by box 650, labeled "Delete temporary file to conserve storage space on user's computer," the VideoShare 2Peer software removes all of the temporary files that were created in the course of the automated processing described above. This feature provides for the user a convenient, secure, and transparent process, with the benefit that the user's computer storage device(s), for example one or more hard drives, do not become cluttered with unnecessary and obsolete files. In the event that the video message has been sent to the host, the VideoShare 2Peer software and the host computer 60 (for example, the VideoShare Upload/Database Server) will update the user's account to account for the required storage space that the video requires. The necessary logging, creation of an identification tag, and storing of the video and the associated identifier or identifiers is also performed automatically, as schematically depicted by box 655. The user can optionally add additional identification and control information about the user, and about how and under what conditions the video is to be made available for distribution, as schematically indicated by box 660. The user is automatically prompted to provide this information, but has the option to forego making a decision immediately. The transmission of video segment files to viewers is discussed in more detail below, and is represented in FIG. 14B by the box 670 labeled "Transmit file to viewer" which is outside the region 608 as an indication that the transmission of files to viewers is an action beyond the material discussed above in conjunction with the Save and Share button 536 of FIG. 13.
FIG. 14C shows a flow diagram 602 of an embodiment of the invention in which software automates a number of steps in the formatting of a video segment. In particular, in this embodiment, the video segment that the user wishes to provide in streaming video format is compressed into a plurality of formats, each of which is encoded for optimal display at a different transmission bitrate. There can be a benefit to recording the same video segment in multiple formats. For example, a casual viewer may have only a slow speed modem, such as a 28.8 kilobaud (kB) modem. For such a viewer, the slow transmission speed can make the size of a file a critical feature. Such a user can view a video in real time if it is formatted for a 28.8 kB modem, but not if it is formatted for appreciably higher transmission speeds. Another user, for example, one who has a Tl connection that can handle transmission speeds up to approximately 1.5 megabaud, could successfully receive a version of the same video segment that is formatted for higher transmission speeds, with the possibility of having a better quality image and higher resolution, perhaps with better audio as well. The Tl user could see the version of the video segment intended for 28.8 kB transmission if he or she wanted to, but might prefer to see a video segment that appeared to be more professional in quality. By using a system that can automatically discriminate the transmission speed capabilities of the hardware that the user employs, the embodiment allows each user to view a version of the video segment that is optimally configured for the user's hardware.
In particular, the steps of the method enclosed within the dotted rectangle 609 are automated by software that embodies the present invention. As described above, the user obtains and selects a video segment for processing for distribution, as indicated at box 605 that schematically encapsulates all of the actions that a user takes as described in relation to FIGs. 4 and 5 above. When the user activates the Save and Share button 536 the actions described below that are enclosed by the dotted line 609 are automatically carried out under the control of the VideoShare 2PeerSoftware comprising the computer communication module.
As discussed in relation to FIG. 14A, the VideoShare 2Peer software subjects the selected video segment to analysis to determine whether the selected video segment is or is not in a file format that is compatible with a streaming video format, as indicated at diamond 610. If the selected video segment is not compatible with a streaming video format, it is converted to a compatible format, as depicted by the arrow labeled "NO" that points from the diamond 610 to the box 615, "Convert to compatible file format." The conversion process performed by the VideoShare 2Peer software creates a DirectShow filter graph that decompresses the video file into a temporary, uncompressed AVI file.
The video segment file in a format that is compatible with streaming video is then temporarily stored in the user's computer 16, for example as a file on the hard drive of computer 16. This storing step is performed if the file was originally in a format compatible with streaming video by following the arrow marked "YES" that points from the diamond 610 to the box 620, "Temporarily store file." Alternatively, the storing step is performed if the file was originally not in a format compatible with streaming video by following the arrow that points from the box 615 to the box 620.
The temporarily stored file is then compressed in multiple streaming multimedia formats, as denoted by the box 633. In the present example, three files will be used to describe the process, but it should be understood that more or fewer than three formats may be created at substantially the same time. The resulting multiple files are denoted by the three boxes 634, 636 and 638 labeled "Bandwidth Target A," "Bandwidth Target B," and "Bandwidth Target C," respectively. Each file is optimally encoded for play as a streaming video segment at a particular transmission rate and bandwidth, such as 28.8 kB, 56 kB, lOOkB, 300kB, or other transmission rates.
As described above, the method includes a step of creating and temporarily storing a "thumbnail" identifier, as denoted by the box 640. Rather than transmitting one video segment in one SMF with one thumbnail, the embodiment of FIG. 14C transmits all the files 634, 636 and 638 in association with the single thumbnail and any other identifiers that are selected as appropriate. For example, each SMF file can be identified as to its bandwidth. In an alternative embodiment, the system transmits only a single SMF file with its associated identifiers, including the JPEG "thumbnail," and the multiple bandwidth variants of the SMF file are generated at the host computer 60. This embodiment may be advantageous when the user has only a slow speed modem, and would be severely time constrained by having to upload multiple files.
Using the symmetric software present on both the sender computer 302 and the receiver computer 304, the sender 40 (or Sender A) can send the video message directly to the receiver computer 304 if the receiver computer 304 is online, as indicated at box 641.
In the event that the receiver computer 304 is not online, the IVM Server 406 informs sender computer 302 that the receiver computer 304 is not online. Rather than sending a message to the receiver computer 304, the sender computer 320 sends the video message to a VideoShare server computer 710 (or host computer 60) for storage at server computer 710, and for later delivery to the intended recipient. Server computer 710 can transmit the video message, for example, via e-mail or via Web retrieval as described in copending U.S.S.N. 09/497,587, to the intended recipient computer 304 when that computer 304 is again on-line. These steps are shown in boxes 645, 650, 655, 660 and 670. The remaining steps of this embodiment, as denoted by the boxes 650, 655, 660 and 670, correspond substantially to the steps in FIG. 14A represented by the boxes identified with the corresponding numerals. It should be noted that the precise order of some of the steps, for example, the step denoted by the box 655 and the step denoted by the box 650, can be interchanged without a different outcome of the overall process. Other such interchanges in sequence are possible as well, again without a different outcome of the overall process.
FIG. 14D depicts an embodiment of the database 64 of the host computer 60 on which are recorded the three exemplary bandwidth target files 634, 636 and 638 for FIG. 14C. These files are available for delivery over a computer network to a viewer. The files 634, 636 and 638 represent three versions of the same video segment in streaming multimedia format, each suitable for optimal viewing by a user having hardware operating at the transmission rate corresponding to the format of one of the files.
As shown in FIG. 14E, the user (or the viewer) transmits to the host computer 60 a request for a particular video segment, denoted by the arrow from the box labeled "USER" to the box 960 labeled "Connection Speed Detector." Host computer 60 can include hardware that can sense the transmission speed of a user computer 16, or of a computer used by a person desiring to view a video segment. Alternatively, the host computer 60 can inquire of the computer on the network that is connected to the user computer 16 or the computer of a viewer about the speed of connection that is being maintained. When the information is available to the host computer 60, the host computer 60 can determine which file of the files exemplified by 634, 636 and 638 is most appropriate to serve to the user or viewer, as denoted by the box 692 labeled "Logic to select and serve SMF file to User." The host computer 60 then transmits the appropriate file to the user, as denoted by the arrow from the box 692 to the box 694 labeled "User receives and views SMF file." Alternatively, the viewer can request the transmission of a file encoded at a specific bitrate.
When the user begins the process described in relation to FIG. 14A, in one embodiment, the "Progress Dialog" screen 700 depicted in FIG. 15 is presented, reflecting the status of the process in real time. The "Progress Dialog" screen 700 notifies the user about the total number of bytes that have to be uploaded to perform the transfer and it also informs the user of the number of bytes and the percentage of the file that have been uploaded in real time.
FIG. 16 shows one embodiment of a screen, discussed above, that displays a video message. FIG. 17 shows a screen 1000 used to control the status of a video queue. When the user, after recording or importing a video, clicks the "Save and Share" button 536 of FIG. 13 while in "offline mode", the VideoShare 2Peer software performs the first three steps of the "Save and Share Process," namely, the video file format conversion represented by box 615 of FIG. 14A, the compression of the video segment to a streaming multimedia format at a user-specified bitrate represented by the box 635 of FIG. 14 A, and the creation of a "Thumbnail" JPEG snapshot of the video file represented by the box 640 of FIG. 14A. The resulting output files are stored in a local database for later use in the "Sharing Queue," which is an operation similar to the temporary storage of files depicted in FIG. 14A. In the middle of FIG. 17 is a dialog box 1010 that displays a list of video segments that are ready to be uploaded to the VideoShare Web site. The small "Preview" window 1020 in the upper left corner of FIG. 17 is a DirectShow playback graph that allows the user to review the stored video segment that is highlighted in the dialog box 1010. The user can use this window to preview the video segment file by activating the "Preview" button 1030, to delete the video segment file by activating the "Delete" button 1040, and to upload and publish the video by activating the "Save and Share Now" button 1050. The "Save and Share Now" button 1050 performs the uploading process on each of the queued videos, creating a TCP/IP connection to the VideoShare Upload/Database Server, transferring the file to the VideoShare web site, and updating the user's VideoShare account, in a manner substantially similar to the method employed by the Save and Share button 536 of FIG. 13 to accomplish the same activities.
Audio/Video Setting Process FIG. 18 shows a screen 1100 used to control the operational settings of equipment connected to the user's computer. Another feature of the VideoShare 2Peer software the ability of the user to change the configuration of the audio, video, and compression devices through the use of the "Settings" tab 1110. Upon activation of the Settings tab 1110, the screen 1100 is active. The user can select the "bitrate" at which the streaming multimedia files will be compressed by using the set of radio buttons 1120 at the upper left corner of the screen 1100. The default setting is "56k Modem" which corresponds to a user using a 56k modem. This default setting is denoted by the 56k Modem radio button 1120 appearing with a dot, while the remaining radio buttons for bitrate 1120 are blank. In one embodiment, the pie graph 1130 that appears at the upper right corner of screen 1100 indicates the percentage of the user's VideoShare storage space that is full. In the embodiment shown, the user has filled approximately 3.13% of the available storage capacity available for storing files. Two pull-down menus, "Camera source device" box 1140 and "Audio source device" box 1150, list all of the available video and audio capture sources that the user has available on his or her Win9x/2000 machine. The user can select a source of audio or video by activating the appropriate pull-down menu box and locating a device of his or her choosing. To the right of these pull-down menus, there are two buttons, "Video Settings..." 1160 and "Audio Settings..." 1170 that allow the user to change the properties of the currently selected audio and video device. Such properties include image size, capture compression, lighting conditions, and the like. The screen 1100 also provides to the user the current working directory information in a the box 1180 and the current queue directory information in the box 1190, which the user can optionally change by entering new values in either or both boxes 1180 and 1190.
Equivalents While the invention has been particularly shown and described with reference to specific preferred embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. What is claimed is:

Claims

Claims 1. A method of sending a streaming video message over a network, comprising the steps of: receiving at a server computer an indication that the operator of a sender computer intends to send a streaming video message to a receiver computer over a network; determining whether said receiver computer is online; and communicating from said server computer to said sender computer whether said receiver computer is online, wherein an indication that said receiver computer is online signals to said sender computer that streaming said video message directly from said sender computer to said receiver computer over the network is appropriate. 2. The method of claim 1 , wherein the step of receiving at a server computer an indication that the operator of a sender computer intends to send a streaming video message to a receiver computer is performed by a supervisory computer communication module operable on said server computer. 3. The method of claim 1, wherein the step of determining whether said receiver computer is online is performed by a supervisory computer communication module operable on said server computer. 4. The method of claim 1 , wherein the step of communicating from said server computer to said sender computer whether said receiver computer is online is performed by a supervisory computer communication module operable on said server computer. 5. A method of sending a streaming video message over a network, comprising the steps of: sending to a server computer an indication that the operator of a sender computer intends to send a streaming video message to a receiver computer over a network; receiving at said sender computer an indication of whether said receiver computer is online; and in response to an indication that said receiver computer is online, streaming said video message from said sender computer to said receiver computer over the network independent of the operation of said server computer. 6. The method of claim 5, wherein the step of streaming said video message from said sender computer to said receiver computer comprises: providing on said sender computer a computer communication module operable on each of said sender computer and said receiver computer; and streaming said video message from said communication module of said sender computer to said receiver computer over the network independent of the operation of said server computer. 7. The method of claim 6, wherein providing a computer communication module operable on each of said sender computer and said receiver computer comprises providing individual copies of the same communication module to at least one of the sender computer and the receiver computer, each copy operable on each of said sender computer and said receiver computer. 8. The method of claim 7, wherein the step of providing individual copies of the same communication module comprises providing individual copies of the same communication module, said communication module being symmetric in operation. 9. The method of claim 8, wherein the step of providing individual copies of the same communication module, said communication module being symmetric in operation comprises providing communication modules that are modular. 10. The method of claim 8, wherein the step of providing individual copies of the same communication module, said communication module being symmetric in operation comprises providing communication modules that can determine the features present in another communication module operating on another computer. 11. The method of claim 5, wherein the step of streaming said video message from said sender computer to said receiver computer over the network independent of the operation of said server computer comprises performing said streaming operation using only communication modules present on said sender computer and said receiver computer. 12. The method of claim 5, further comprising the step: in response to an indication that said receiver computer is not online, optionally sending said video message from said sender computer to a host computer, for storage and later transmission to said receiver computer, independent of the operation of said server computer. 13. The method of claim 5, wherein said video message comprises video information. 14. The method of claim 13, wherein said video message further comprises information selected from audio information, a thumbnail image, a text identifying a subject, a message body text, a document attachment, a slide and meta-data. 15. A computer program recorded on a machine-readable medium, the computer program adapted to transmit and receive streaming video messages over a network independent of a server computer, the computer program when operating on a sender computer performing the steps of: requesting from a server computer the status of a receiver computer, said receiver computer being the intended recipient of a video message; receiving an indication of the online status of said receiver computer; and in response to an indication that said receiver computer is online, streaming said video message from said computer program of said sender computer to said computer program of said receiver computer independent of the operation of said server computer. 16. The computer program recorded on a machine-readable medium of claim 15, wherein said computer program is further capable, when operating on a receiver computer, of performing the step of: determining that a video message is being sent by a sender computer. 17. The computer program recorded on a machine-readable medium of claim 16, wherein said computer program is further capable, when operating on a receiver computer, of performing the step of: displaying to a viewer of said receiver computer information about said video message, said information comprising at least one of an identifier of the sender of said message, a subject of said message, and a title of said message. 18. The computer program recorded on a machine-readable medium of claim 15, wherein said computer program is further capable, when operating on a receiver computer, of performing the steps of: receiving said streaming video message; and displaying said streaming video message to a viewer of said receiver computer. 19. The computer program recorded on a machine-readable medium of claim 18, wherein the step of receiving said streaming video message comprises; receiving said streaming video message; presenting to a viewer of said receiver computer an opportunity to select one of: (i) accepting the streaming video message; and (ii) rejecting the streaming video message; accepting a response of said viewer; and processing said message in accordance with the response of said viewer. 20. The computer program recorded on a machine-readable medium of claim 19, wherein the step of accepting the streaming video message comprises at least one of; (i) accepting the streaming video message for immediate display; and (ii) accepting the streaming video message for storage in a machine-readable memory for display at a later time. 21. The computer program recorded on a machine-readable medium of claim 15, wherein said computer program is further capable, when operating on a receiver computer, of performing the step of: receiving said streaming video message is accomplished without the necessity of an action by a viewer of said computer. 22. The computer program recorded on a machine-readable medium of claim 21 , wherein displaying said streaming video message is accomplished without the necessity of an action by a viewer of said computer. 23. The computer program recorded on a machine-readable medium of claim 21 , wherein displaying said video message is performed by invoking the operation of a video display module. 24. The computer program recorded on a machine-readable medium of claim 15, further performing the step of: in response to an indication that said receiver computer is not online, optionally sending said video message from said communication module of said sender computer to a host computer, for storage and later transmission to said communication module of said receiver computer, independent of the operation of said server computer. 25. The computer program recorded on a machine-readable medium of claim 15, wherein said video message comprises video information. 26. The computer program recorded on a machine-readable medium of claim 25, wherein said video message further comprises information selected from audio information, a thumbnail image, a text identifying a subject, a message body text, a document attachment, a slide and meta-data. 27. A computer program recorded on a machine-readable medium, the computer program adapted to operate a server computer and to provide server functionality including database connectivity and communication protocols, the computer program when operating on a server computer performing the steps of: determining the online status of said receiver computer; and providing an indication of said status to said sender computer, such that in response to an indication that said receiver computer is online, streaming said video message from said computer program of said sender computer to said computer program of said receiver computer is accomplished independent of the operation of said server computer. 28. The computer program recorded on a machine-readable medium of claim 27, the computer program performing the further step of: accepting a request from a sender computer about the status of a receiver computer, said receiver computer being the intended recipient of a video message. 29. The computer program recorded on a machine-readable medium of claim 27, the computer program performing the further step of: providing communication protocols for exchanging information between interconnected computers. 30. The computer program recorded on a machine-readable medium of claim 27, the computer program performing the further step of: providing a database in which to store information, including information about sender and receiver computers authorized to communicate with said server computer.
PCT/US2000/021214 1999-08-03 2000-08-03 Instant video messenger WO2001010128A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU65156/00A AU6515600A (en) 1999-08-03 2000-08-03 Instant video messenger

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
US14702999P 1999-08-03 1999-08-03
US60/147,029 1999-08-03
US49758700A 2000-02-03 2000-02-03
US09/497,587 2000-02-03
US19606900P 2000-04-10 2000-04-10
US60/196,069 2000-04-10

Publications (1)

Publication Number Publication Date
WO2001010128A1 true WO2001010128A1 (en) 2001-02-08

Family

ID=27386504

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2000/021214 WO2001010128A1 (en) 1999-08-03 2000-08-03 Instant video messenger

Country Status (2)

Country Link
AU (1) AU6515600A (en)
WO (1) WO2001010128A1 (en)

Cited By (46)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001050742A1 (en) * 1999-12-30 2001-07-12 America Online, Inc. Method and system for selecting a television channel
WO2001069885A2 (en) * 2000-03-10 2001-09-20 Absolutefuture, Inc. Method and system for coordinating the sending of information
WO2002059802A1 (en) * 2001-01-25 2002-08-01 Gts Pacific Pty Ltd Non-recorded audio/video stream transmission using electronic mail
US6754904B1 (en) 1999-12-30 2004-06-22 America Online, Inc. Informing network users of television programming viewed by other network users
EP1556956A2 (en) * 2002-06-26 2005-07-27 Yahoo Inc. System and method for communicating images between intercommunicating users
EP1625476A2 (en) * 2003-05-16 2006-02-15 Picasa, Inc. Networked chat and media sharing systems and methods
WO2006022984A1 (en) * 2004-08-20 2006-03-02 Sony Computer Entertainment America Inc. System and method for effectively exchanging photo data in an instant messaging environment
WO2006105468A1 (en) * 2005-03-31 2006-10-05 Pando Networks, Inc. Method and apparatus for offline cooperative file distribution using cache nodes
WO2006116881A1 (en) * 2005-05-05 2006-11-09 Cameron, Allan Method and system for providing telephony greetings
EP1734716A1 (en) * 2005-06-17 2006-12-20 Chao-Hung Wu System for real-time transmitting and receiving of audio/video and handwriting information
WO2007008567A1 (en) * 2005-07-08 2007-01-18 Matsushita Electric Industrial Co., Ltd. Secure peer to peer messaging service
WO2007082386A1 (en) * 2006-01-20 2007-07-26 Rec2Mail Inc. Customized web interface for video communication
EP1968266A1 (en) * 2007-03-09 2008-09-10 My Hollywood Ltd. Device, system, and method of electronic communication utilizing audiovisual clips
CN100433776C (en) * 2004-10-20 2008-11-12 大竑企业股份有限公司 Network image and sound file fax device and synchronous classification pigeonhole work method
EP1337113A3 (en) * 2002-02-14 2009-08-05 Panasonic Corporation Content distribution system
EP2271095A1 (en) * 2001-02-21 2011-01-05 United Video Properties, Inc. Systems and methods for interactive program guides with personal video recording features
US7953844B2 (en) 2005-01-31 2011-05-31 Sharp Laboratories Of America, Inc. Systems and methods for implementing an instant messaging remote control service
US7987492B2 (en) 2000-03-09 2011-07-26 Gad Liwerant Sharing a streaming video
US20120304062A1 (en) * 2011-05-23 2012-11-29 Speakertext, Inc. Referencing content via text captions
CN103746904A (en) * 2013-12-27 2014-04-23 广州华多网络科技有限公司 Information interaction method and device
US9015736B2 (en) 2005-12-29 2015-04-21 Rovi Guides, Inc. Systems and methods for episode tracking in an interactive media environment
US9021538B2 (en) 1998-07-14 2015-04-28 Rovi Guides, Inc. Client-server based interactive guide with server recording
US9071872B2 (en) 2003-01-30 2015-06-30 Rovi Guides, Inc. Interactive television systems with digital video recording and adjustable reminders
US9075861B2 (en) 2006-03-06 2015-07-07 Veveo, Inc. Methods and systems for segmenting relative user preferences into fine-grain and coarse-grain collections
US9125169B2 (en) 2011-12-23 2015-09-01 Rovi Guides, Inc. Methods and systems for performing actions based on location-based rules
US9154552B2 (en) 2007-09-06 2015-10-06 Microsoft Technology Licensing, Llc Method and apparatus for cooperative file distribution with receiver determined quality of services
US9166714B2 (en) 2009-09-11 2015-10-20 Veveo, Inc. Method of and system for presenting enriched video viewing analytics
US9191722B2 (en) 1997-07-21 2015-11-17 Rovi Guides, Inc. System and method for modifying advertisement responsive to EPG information
US9264656B2 (en) 2014-02-26 2016-02-16 Rovi Guides, Inc. Systems and methods for managing storage space
US9294799B2 (en) 2000-10-11 2016-03-22 Rovi Guides, Inc. Systems and methods for providing storage of data on servers in an on-demand media delivery system
US9319735B2 (en) 1995-06-07 2016-04-19 Rovi Guides, Inc. Electronic television program guide schedule system and method with data feed access
US9326025B2 (en) 2007-03-09 2016-04-26 Rovi Technologies Corporation Media content search results ranked by popularity
EP2923497A4 (en) * 2012-11-21 2016-05-18 H4 Eng Inc Automatic cameraman, automatic recording system and video recording network
US9426509B2 (en) 1998-08-21 2016-08-23 Rovi Guides, Inc. Client-server electronic program guide
US9641566B1 (en) 2016-09-20 2017-05-02 Opentest, Inc. Methods and systems for instantaneous asynchronous media sharing
US9646444B2 (en) 2000-06-27 2017-05-09 Mesa Digital, Llc Electronic wireless hand held multimedia device
US9736524B2 (en) 2011-01-06 2017-08-15 Veveo, Inc. Methods of and systems for content search based on environment sampling
US9749693B2 (en) 2006-03-24 2017-08-29 Rovi Guides, Inc. Interactive media guidance application with intelligent navigation and display features
USRE46644E1 (en) 2003-12-09 2017-12-19 Lg Electronics Inc. Reserved image transmission system and method
EP1689155B1 (en) * 2005-02-02 2018-05-02 Creative Technology Ltd. Method and system to process video effects
US10063934B2 (en) 2008-11-25 2018-08-28 Rovi Technologies Corporation Reducing unicast session duration with restart TV
US10129569B2 (en) 2000-10-26 2018-11-13 Front Row Technologies, Llc Wireless transmission of sports venue-based data including video to hand held devices
US10225584B2 (en) 1999-08-03 2019-03-05 Videoshare Llc Systems and methods for sharing video with advertisements over a network
US10631066B2 (en) 2009-09-23 2020-04-21 Rovi Guides, Inc. Systems and method for automatically detecting users within detection regions of media devices
US20220394004A1 (en) * 2007-06-28 2022-12-08 Voxer Ip Llc Telecommunication and multimedia management method and apparatus
US20230130946A1 (en) * 2007-06-28 2023-04-27 Voxer Ip Llc Real-time messaging method and apparatus

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0762763A2 (en) * 1995-09-01 1997-03-12 AT&T Corp. Video messaging arrangement
WO1998016045A1 (en) * 1996-10-06 1998-04-16 Mirabilis Ltd. Communications system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0762763A2 (en) * 1995-09-01 1997-03-12 AT&T Corp. Video messaging arrangement
WO1998016045A1 (en) * 1996-10-06 1998-04-16 Mirabilis Ltd. Communications system

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ICQ INC.: "Mirabilis Announces Special Version of its Popular ICQ To Support Microsoft NetMeeting 2.0", PRESS RELEASE ICQ INC., 28 April 1997 (1997-04-28), XP002150291, Retrieved from the Internet <URL:http://www.icq.com/press/press_release_netmeeting.html> [retrieved on 20001016] *
JOSÉ ALVEAR: "Guide to Streaming Multimedia", 9 April 1998, WILEY COMPUTER PUBLISHING, XP002150113 *
ZDNET: "honeyq v1.5", ZDNET DOWNLOADS INFO, 6 November 1997 (1997-11-06), XP002150290, Retrieved from the Internet <URL:http://www.zdnet.com/downloads/stories/info/0,,000LVZ,.html> [retrieved on 20001016] *

Cited By (81)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9319735B2 (en) 1995-06-07 2016-04-19 Rovi Guides, Inc. Electronic television program guide schedule system and method with data feed access
US9191722B2 (en) 1997-07-21 2015-11-17 Rovi Guides, Inc. System and method for modifying advertisement responsive to EPG information
US9021538B2 (en) 1998-07-14 2015-04-28 Rovi Guides, Inc. Client-server based interactive guide with server recording
US9055318B2 (en) 1998-07-14 2015-06-09 Rovi Guides, Inc. Client-server based interactive guide with server storage
US9055319B2 (en) 1998-07-14 2015-06-09 Rovi Guides, Inc. Interactive guide with recording
US9118948B2 (en) 1998-07-14 2015-08-25 Rovi Guides, Inc. Client-server based interactive guide with server recording
US9226006B2 (en) 1998-07-14 2015-12-29 Rovi Guides, Inc. Client-server based interactive guide with server recording
US10075746B2 (en) 1998-07-14 2018-09-11 Rovi Guides, Inc. Client-server based interactive television guide with server recording
US9232254B2 (en) 1998-07-14 2016-01-05 Rovi Guides, Inc. Client-server based interactive television guide with server recording
US9154843B2 (en) 1998-07-14 2015-10-06 Rovi Guides, Inc. Client-server based interactive guide with server recording
US9426509B2 (en) 1998-08-21 2016-08-23 Rovi Guides, Inc. Client-server electronic program guide
US10362341B2 (en) 1999-08-03 2019-07-23 Videoshare, Llc Systems and methods for sharing video with advertisements over a network
US10225584B2 (en) 1999-08-03 2019-03-05 Videoshare Llc Systems and methods for sharing video with advertisements over a network
US6754904B1 (en) 1999-12-30 2004-06-22 America Online, Inc. Informing network users of television programming viewed by other network users
WO2001050742A1 (en) * 1999-12-30 2001-07-12 America Online, Inc. Method and system for selecting a television channel
US10277654B2 (en) 2000-03-09 2019-04-30 Videoshare, Llc Sharing a streaming video
US7987492B2 (en) 2000-03-09 2011-07-26 Gad Liwerant Sharing a streaming video
US10523729B2 (en) 2000-03-09 2019-12-31 Videoshare, Llc Sharing a streaming video
WO2001069885A2 (en) * 2000-03-10 2001-09-20 Absolutefuture, Inc. Method and system for coordinating the sending of information
WO2001069885A3 (en) * 2000-03-10 2002-06-20 Absolutefuture Inc Method and system for coordinating the sending of information
US9646444B2 (en) 2000-06-27 2017-05-09 Mesa Digital, Llc Electronic wireless hand held multimedia device
US9294799B2 (en) 2000-10-11 2016-03-22 Rovi Guides, Inc. Systems and methods for providing storage of data on servers in an on-demand media delivery system
US10129569B2 (en) 2000-10-26 2018-11-13 Front Row Technologies, Llc Wireless transmission of sports venue-based data including video to hand held devices
WO2002059802A1 (en) * 2001-01-25 2002-08-01 Gts Pacific Pty Ltd Non-recorded audio/video stream transmission using electronic mail
US9055322B2 (en) 2001-02-21 2015-06-09 Rovi Guides, Inc. Systems and methods for interactive program guides with personal video recording features
US9930374B2 (en) 2001-02-21 2018-03-27 Rovi Guides, Inc. Systems and methods for interactive program guides with personal video recording features
CN103873933A (en) * 2001-02-21 2014-06-18 联合视频制品公司 Systems and methods for interactive program guides with personal video recording features
EP2271095A1 (en) * 2001-02-21 2011-01-05 United Video Properties, Inc. Systems and methods for interactive program guides with personal video recording features
CN103873933B (en) * 2001-02-21 2018-01-16 乐威指南公司 The system and method for interactive program guides with personal video recording features
EP1337113A3 (en) * 2002-02-14 2009-08-05 Panasonic Corporation Content distribution system
EP1556956A4 (en) * 2002-06-26 2008-03-19 Yahoo Inc System and method for communicating images between intercommunicating users
EP1556956A2 (en) * 2002-06-26 2005-07-27 Yahoo Inc. System and method for communicating images between intercommunicating users
US7509377B2 (en) 2002-06-26 2009-03-24 Yahoo! Inc. System and method for communicating images between intercommunicating users
US9369741B2 (en) 2003-01-30 2016-06-14 Rovi Guides, Inc. Interactive television systems with digital video recording and adjustable reminders
US9071872B2 (en) 2003-01-30 2015-06-30 Rovi Guides, Inc. Interactive television systems with digital video recording and adjustable reminders
US7761507B2 (en) 2003-05-16 2010-07-20 Google, Inc. Networked chat and media sharing systems and methods
EP1625476A4 (en) * 2003-05-16 2008-02-27 Picasa Inc Networked chat and media sharing systems and methods
EP1625476A2 (en) * 2003-05-16 2006-02-15 Picasa, Inc. Networked chat and media sharing systems and methods
AU2004241581B2 (en) * 2003-05-16 2010-12-16 Google Llc Networked chat and media sharing systems and methods
USRE46644E1 (en) 2003-12-09 2017-12-19 Lg Electronics Inc. Reserved image transmission system and method
USRE46677E1 (en) 2003-12-09 2018-01-16 Lg Electronics Inc. Reserved image transmission system and method
WO2006022984A1 (en) * 2004-08-20 2006-03-02 Sony Computer Entertainment America Inc. System and method for effectively exchanging photo data in an instant messaging environment
CN100433776C (en) * 2004-10-20 2008-11-12 大竑企业股份有限公司 Network image and sound file fax device and synchronous classification pigeonhole work method
US7953844B2 (en) 2005-01-31 2011-05-31 Sharp Laboratories Of America, Inc. Systems and methods for implementing an instant messaging remote control service
EP1689155B1 (en) * 2005-02-02 2018-05-02 Creative Technology Ltd. Method and system to process video effects
WO2006105468A1 (en) * 2005-03-31 2006-10-05 Pando Networks, Inc. Method and apparatus for offline cooperative file distribution using cache nodes
WO2006116881A1 (en) * 2005-05-05 2006-11-09 Cameron, Allan Method and system for providing telephony greetings
EP1734716A1 (en) * 2005-06-17 2006-12-20 Chao-Hung Wu System for real-time transmitting and receiving of audio/video and handwriting information
WO2007008567A1 (en) * 2005-07-08 2007-01-18 Matsushita Electric Industrial Co., Ltd. Secure peer to peer messaging service
US9015736B2 (en) 2005-12-29 2015-04-21 Rovi Guides, Inc. Systems and methods for episode tracking in an interactive media environment
WO2007082386A1 (en) * 2006-01-20 2007-07-26 Rec2Mail Inc. Customized web interface for video communication
US9075861B2 (en) 2006-03-06 2015-07-07 Veveo, Inc. Methods and systems for segmenting relative user preferences into fine-grain and coarse-grain collections
US9092503B2 (en) 2006-03-06 2015-07-28 Veveo, Inc. Methods and systems for selecting and presenting content based on dynamically identifying microgenres associated with the content
US10984037B2 (en) 2006-03-06 2021-04-20 Veveo, Inc. Methods and systems for selecting and presenting content on a first system based on user preferences learned on a second system
US9128987B2 (en) 2006-03-06 2015-09-08 Veveo, Inc. Methods and systems for selecting and presenting content based on a comparison of preference signatures from multiple users
US9749693B2 (en) 2006-03-24 2017-08-29 Rovi Guides, Inc. Interactive media guidance application with intelligent navigation and display features
US9326025B2 (en) 2007-03-09 2016-04-26 Rovi Technologies Corporation Media content search results ranked by popularity
US10694256B2 (en) 2007-03-09 2020-06-23 Rovi Technologies Corporation Media content search results ranked by popularity
EP1968266A1 (en) * 2007-03-09 2008-09-10 My Hollywood Ltd. Device, system, and method of electronic communication utilizing audiovisual clips
US20230051915A1 (en) 2007-06-28 2023-02-16 Voxer Ip Llc Telecommunication and multimedia management method and apparatus
US11700219B2 (en) 2007-06-28 2023-07-11 Voxer Ip Llc Telecommunication and multimedia management method and apparatus
US11658929B2 (en) 2007-06-28 2023-05-23 Voxer Ip Llc Telecommunication and multimedia management method and apparatus
US11658927B2 (en) * 2007-06-28 2023-05-23 Voxer Ip Llc Telecommunication and multimedia management method and apparatus
US20230130946A1 (en) * 2007-06-28 2023-04-27 Voxer Ip Llc Real-time messaging method and apparatus
US20230065310A1 (en) * 2007-06-28 2023-03-02 Voxer Ip Llc Telecommunication and multimedia management method and apparatus
US20230047999A1 (en) * 2007-06-28 2023-02-16 Voxer Ip Llc Telecommunication and multimedia management method and apparatus
US20220394004A1 (en) * 2007-06-28 2022-12-08 Voxer Ip Llc Telecommunication and multimedia management method and apparatus
US11777883B2 (en) 2007-06-28 2023-10-03 Voxer Ip Llc Telecommunication and multimedia management method and apparatus
US9154552B2 (en) 2007-09-06 2015-10-06 Microsoft Technology Licensing, Llc Method and apparatus for cooperative file distribution with receiver determined quality of services
US10063934B2 (en) 2008-11-25 2018-08-28 Rovi Technologies Corporation Reducing unicast session duration with restart TV
US9166714B2 (en) 2009-09-11 2015-10-20 Veveo, Inc. Method of and system for presenting enriched video viewing analytics
US10631066B2 (en) 2009-09-23 2020-04-21 Rovi Guides, Inc. Systems and method for automatically detecting users within detection regions of media devices
US9736524B2 (en) 2011-01-06 2017-08-15 Veveo, Inc. Methods of and systems for content search based on environment sampling
US20120304062A1 (en) * 2011-05-23 2012-11-29 Speakertext, Inc. Referencing content via text captions
US9125169B2 (en) 2011-12-23 2015-09-01 Rovi Guides, Inc. Methods and systems for performing actions based on location-based rules
US10291725B2 (en) 2012-11-21 2019-05-14 H4 Engineering, Inc. Automatic cameraman, automatic recording system and automatic recording network
EP2923497A4 (en) * 2012-11-21 2016-05-18 H4 Eng Inc Automatic cameraman, automatic recording system and video recording network
CN103746904A (en) * 2013-12-27 2014-04-23 广州华多网络科技有限公司 Information interaction method and device
US9264656B2 (en) 2014-02-26 2016-02-16 Rovi Guides, Inc. Systems and methods for managing storage space
US10484737B2 (en) 2016-09-20 2019-11-19 Loom, Inc. Methods and systems for instantaneous asynchronous media sharing
US9641566B1 (en) 2016-09-20 2017-05-02 Opentest, Inc. Methods and systems for instantaneous asynchronous media sharing

Also Published As

Publication number Publication date
AU6515600A (en) 2001-02-19

Similar Documents

Publication Publication Date Title
WO2001010128A1 (en) Instant video messenger
US10523729B2 (en) Sharing a streaming video
US9374805B2 (en) System and method for combining memory resources for use on a personal network
US9166879B2 (en) System and method for enabling the establishment and use of a personal network
US8738730B2 (en) System and method for remotely controlling network resources
US20080163321A1 (en) Method and system for sharing video over a network
US20080147786A1 (en) Method and system for sharing video over a network
US20100169411A1 (en) System And Method For Improved Content Delivery

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AU CA CN IL JP

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP