WO2001093592A1 - Active layer for digital video - Google Patents

Active layer for digital video Download PDF

Info

Publication number
WO2001093592A1
WO2001093592A1 PCT/US2001/017793 US0117793W WO0193592A1 WO 2001093592 A1 WO2001093592 A1 WO 2001093592A1 US 0117793 W US0117793 W US 0117793W WO 0193592 A1 WO0193592 A1 WO 0193592A1
Authority
WO
WIPO (PCT)
Prior art keywords
video
information
digitized video
computer
mpeg
Prior art date
Application number
PCT/US2001/017793
Other languages
French (fr)
Inventor
Guy Fehn
Kristian Harding Hansen
Original Assignee
Activefilmlayer.Com
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Activefilmlayer.Com filed Critical Activefilmlayer.Com
Priority to AU2001265307A priority Critical patent/AU2001265307A1/en
Publication of WO2001093592A1 publication Critical patent/WO2001093592A1/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/472End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/4722End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting additional data associated with the content
    • H04N21/4725End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for requesting additional data associated with the content using interactive regions of the image, e.g. hot spots
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234318Processing of video elementary streams, e.g. splicing of video streams, manipulating MPEG-4 scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by decomposing into objects, e.g. MPEG-4 objects
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/235Processing of additional data, e.g. scrambling of additional data or processing content descriptors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/435Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/462Content or additional data management, e.g. creating a master electronic program guide from data received from the Internet and a Head-end, controlling the complexity of a video stream by scaling the resolution or bit-rate based on the client capabilities
    • H04N21/4622Retrieving content or additional data from different sources, e.g. from a broadcast channel and the Internet
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47End-user applications
    • H04N21/478Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4782Web browsing, e.g. WebTV
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/81Monomedia components thereof
    • H04N21/8126Monomedia components thereof involving additional data, e.g. news, sports, stocks, weather forecasts
    • H04N21/8133Monomedia components thereof involving additional data, e.g. news, sports, stocks, weather forecasts specifically related to the content, e.g. biography of the actors in a movie, detailed information about an article seen in a video program
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/854Content authoring
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/858Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85Assembly of content; Generation of multimedia applications
    • H04N21/858Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot
    • H04N21/8586Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot by using a URL
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/16Analogue secrecy systems; Analogue subscription systems
    • H04N7/173Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
    • H04N7/17309Transmission or handling of upstream communications
    • H04N7/17318Direct or substantially direct transmission and handling of requests

Definitions

  • the present invention relates to methods and systems for selecting and providing information on objects in a digital video.
  • the Internet has proved be an excellent medium for advertising goods and services. There are two main advantages. First, the cost per 100 exposures is the lowest of all media, including national television. Second, the interactive nature of the Internet encourages the empowerment of consumers because they have the ability sit in their own home, business, or school and gather information on advertized goods and services. Advertisers generally take advantage of this interactivity by offering multimedia delivery of information, including graphics, pictures, sound, video, and animation.
  • the interactivity of the Internet is largely limited to the use of graphical interfaces.
  • a Web site may offer a video that can be downloaded or streamed for viewing
  • DVDs movies offer interactivity through dropdown menus and menus accessed through remote controls, all through graphical interfaces. Use of these graphical interfaces may disrupt viewing of the video.
  • interactivity within the video presentation is not available.
  • Figure 1 is a schematic representation of active film layer media structure.
  • Figure 2 is a schematic representation of a preferred method for reading active film layer media.
  • Figure 3 is a schematic representation of designation of active objects within a video.
  • Figure 4 is a schematic representation of a system for using active film layer media.
  • Figure 5 is a flow chart representing operation of the object parser.
  • Figure 6 is a flow chart representing operation of a preferred active film layer player.
  • the present invention relates to methods and systems for selecting and providing information on objects in a digital video.
  • the present invention provides methods for providing information regarding an object in a video comprising: a) providing a digitized video display of at least one object, wherein the at least one object is selectable by a selection device; and b) providing information upon selection of the at least one object.
  • the digitized video further comprises an active film layer.
  • the present invention is not limited to video compressed by any particular compression format. Indeed, a variety of compression formats are contemplated, including, but not limited to, MPEG-2, DV, and M-JPEG.
  • the present invention is not limited to any particular selection device. Indeed, a variety of selection devices or input devices are contemplated, including, but not limited to, a computer mouse, a computer keyboard, a joystick, trackball, and a touchpad.
  • the present invention is not limited to providing any particular information. Indeed, the provision of a variety of information is contemplated, including, but not limited to, textual information, computer generated graphic information (e.g., bitmaps), and URLs or hypertext links.
  • the hypertext link is displayed after clicking on the at least one object with a computer mouse.
  • the hypertext link is to a manufacturer site or provider site of the object.
  • the methods of the present invention further comprise providing a public network for communication between computers, wherein the information in step (b) is provided via a hypertext link.
  • the present invention provides methods for providing information on objects pictured in digitized video comprising: a) providing: i) a digitized video of at least one moving object; ii) a selection device; and iii) a processor configured to allow selection of the at least one moving object; b) via the processor, implementing selection of the at least one moving object with the selection device to provide a selected object; and c) providing information on the selected object.
  • the present invention provides methods for providing information on objects in a digitized video comprising: a) providing: i) digitized video of at least one object, ii) a display unit, iii) a computer processor configured to display the digitized video and allow selection of the at least one object, and ii) a selection device; b) via the processor, displaying the digitized video of at least one object on the display unit; c) via the processor, implementing selection of the at least one object with the selection device to provide at least one selected object; and d) providing information on the at least one selected object.
  • the present invention provides systems for providing information on an object in a digitized video comprising: a digitized video of at least one object, a computer processor, a selection device, a display unit, and an active layer, wherein the processor is configured so that the digitized video of at least one object is displayed on the display unit and wherein the active layer implements selection of the at least one object with the selection device.
  • the present invention is not limited to video compressed by any particular compression format. Indeed, a variety of compression formats are contemplated, including, but not limited to, MPEG-2, DV, and M-JPEG. The present invention is not limited to any particular selection device.
  • the system further comprises a public network for communication between computers coupled to the processor.
  • the present invention is not limited to a particular source of digitized video. Indeed, a variety of sources of digitized video are contemplated, including, but not limited to, broadcast digital television, DVD (e.g., DVD-ROM or DVD-Video), and streamed digitized video provided from a server and viewed through a media player (e.g., QUICKTIME, WINDOWS MEDIA, and REALVIDEO)
  • a media player e.g., QUICKTIME, WINDOWS MEDIA, and REALVIDEO
  • the present invention provides systems for providing information on an object in a digitized video comprising: a digitized video encoding pictures of at least one object, a computer processor, and a selection device, wherein the processor is configured so that the at least one object is selectable by the selection device.
  • the present invention provides a computer readable medium comprising a plurality of instructions, which when executed by a computer cause the computer to perform the steps of: a) displaying a digitized video of at least one object; and b) implementing selection of the at least one object.
  • the digitized video supplied by the computer readable medium is compressed.
  • the computer readable medium of the present invention is not limited to digitized video compressed in any particular format. Indeed, a variety of compression formats are contemplated, including, but not limited to, MPEG-1, MPEG-2, DV, and M-JPEG.
  • the computer readable medium comprises an active layer.
  • the present invention is not limited to any particular computer readable medium. Indeed, a variety of media are contemplated, including, but not limited to, DVD, CD, magnetic tape, and hard disk drives.
  • the present invention provides a computer readable medium comprising a plurality of instructions, which when executed by a computer cause the computer to perform the steps of: a) displaying a digitized video of at least one object; b) implementing selection of the at least one object; and c)providing information on the at least one object following selection.
  • the present invention provides methods comprising: a) providing: i) a digitized video of at least one moving object; and ii) a selection device b) displaying the digitized video; and c) selecting the at least one object with the selection device.
  • the methods further comprise step d) displaying information on the at least one object
  • the present invention provides methods comprising providing: i) computer memory having stored therein a video file of a least one object; and ii) a selection device; b) displaying the video file; c) selecting the at least one object with the selection device; and d) providing information on the at least one object.
  • the video file comprised digitized video.
  • the digitized video is compressed with a MPEG- 1, MPEG-2, MPEG-4, or DV codec.
  • the video file further comprises computer generated graphics.
  • the present invention is not limited to a particular selection action. Indeed, a variety of selection actions are contemplated, including but not limited to, mouse over actions and mouse click actions.
  • a mouse click action opens a Web browser to a predetermined URL.
  • the Web browser is configured to provide source information (e.g. , referral information encoded in a cookie) to a server or other computer, in some embodiments, the selection of an object enters a user in a contest.
  • the present invention provides computer memory comprising a video editor, wherein the video editor comprises an active film layer.
  • the active film layer comprises object information and digitized video.
  • the present invention is not limited to any particular object information. Indeed, a variety of object information is contemplated including bitmaps and URLs.
  • the present invention provided methods comprising: a) providing: i) a computer memory having stored therein a video editor and an active film layer; ii) a video file; b) designating an image in the video file; c) writing object information on the object in the active film layer.
  • the video file is digitized video.
  • the present invention is not limited to any particular object information. Indeed, a variety of object information is contemplated including bitmaps and URLs.
  • the present invention provides computer readable medium comprising: a digitized video file of at least one moving object; and an object information file comprising data on the moving object.
  • the data on the moving object allows selection of the moving object.
  • the data on the moving object is object information.
  • the present invention provides methods comprising: a) providing: i) a digitized video file of at least one moving object; and ii) an object information file comprising data allowing selection of the at least one moving object; b) displaying the digitized video; and c) selecting the at least one moving object.
  • the methods further comprise further step: d) providing information on the moving object.
  • the present invention is not limited to providing any particular information. Indeed, the provision of a variety of information is contemplated, including, but not limited to, textual information, computer generated graphic information (e.g., bitmaps), digitized video files, and URLs or hypertext links.
  • Digitized video refers to video that is either converted to digital format from analog format or recorded in digital format. Digitized video can be uncompressed or compressed into any suitable format including, but not limited to, MPEG-1, MPEG-2, DV, M-JPEG or MOV. Digitized video can include computer generated graphics, but does consist entirely of computer generated graphics. For example, a digitized video image may comprise one or more graphic components, but a computer generated graphic by itself is not digitized video.
  • An example of digitized video comprising a graphic object include a QUICKTIME video including a Sprite track or layer.
  • Sprite tracks can be used to superimpose a graphical image (e.g., an image encoded by a bitmap such as a bouncing ball) during video playback.
  • digitized video can be delivered by a variety of methods, including playback from DVD, broadcast digital TV, and streaming over the Internet.
  • video display refers to a video that is actively running, streaming, or playing back on a display device.
  • codec refers to a device, either software or hardware, that translates video or audio between its uncompressed form and the compressed form (e.g., MPEG- 2) in which it is stored. Examples of codecs include, but are not limited to, CINEPAK, SORENSON VIDEO, ⁇ NDEO, and HEURIS codecs. "Symetric codecs" encodes and decodes video in approximately the same amount of time. Live broadcast and teleconferencing systems generally use symetric codecs in order to encode video in realtime as it is captured.
  • compression format refers to the format in which a video or audio file is compressed.
  • compression formats include, but are not limited to, MPEG-1, MPEG-2, MPEG-4, M-JPEG, DV, and MOV.
  • MPEG-2 refers to a particular compression format that is capable of producing a high data rate and full broadcast quality files.
  • MPEG-2 is full frame rate (e.g., 24-30 fps) and screen resolution (e.g., 720x480).
  • MPEG-2 is the current standard format for DVD-Video, broadcast digital TV, and satellite digital TV.
  • the term "active film layer” refers to a layer or track in a video timeline that comprises data relating to objects in a digitized video.
  • the data can serve to define the object (i.e., its presence and/or location on a display screen) and provide information on the object (e.g., a bitmap containing text or a URL of a Web site).
  • active object and “active video object” are used interchangeable and refer to an object in a video presentation (e.g., digitized video, movie, or TV) that is selectable by a selection device.
  • active object and “active video object” do not include computer generated graphic objects such as those encoded by bitmaps.
  • moving object and “moving active object” are used herein to refer that objects that move, change position, grow in size, or diminish in size from frame to frame in a video.
  • moving object and "moving active object” encompass objects whose movement is kinetic (e.g., a car driving down a road or a person walking) as well as objects whose movement from frame to frame is cause by the changing perspective of the recording instrument (e.g., a panning movie camera or television camera).
  • selection device refers to devices (e.g., data input devices) that can be used to select objects displayed on a display device.
  • Examples of selection devices include, but are not limited to, a computer mouse, a computer keyboard, a joystick, a trackball, a monitor touchpad, a computer touchpad, computer data entry pens, laser light, voice control and wireless versions thereof.
  • product information refers to any information relevant to a product, including, but not limited to, marketing information, source information, price, URL, purchase information, instructional information, identity information, etc.
  • the term "manufacturer site” refers to an electronically accessible site (e.g., a Web site) operated by the manufacturer of a product.
  • provider site refers to an electronically accessible site (e.g., a Web site) that sells or otherwise provides a product.
  • computer memory and “computer memory device” refer to any storage media readable by a computer processor.
  • Examples of computer memory include, but are not limited to, RAM, ROM, computer chips, digital video disc (DVDs), compact discs (CDs), hard disk drives (HDD), and magnetic tape.
  • computer readable medium refers to any device or system for storing and providing information (e.g., data and instructions) to a computer processor.
  • Examples of computer readable media include, but are not limited to, DVDs, CDs, hard disk drives, magnetic tape and servers for streaming media over networks.
  • video editor refers to computer hardware or software applications for adding or deleting information from a digitized video, and providing supplemental information to a digitized video file.
  • a non-limiting example of a video editor is the Adobe Premiere system.
  • streaming refers to the network delivery of media.
  • Truste streaming matches the bandwidth of the media signal to the viewer's connection, so that the media is seen in realtime.
  • specialized media servers and streaming protocols are used for true streaming.
  • RealTime Streaming Protocol (RTSP, REALNETWORKS) is a standard used to transmit true streaming media to one or more viewers simultaneously.
  • RTSP provides for viewers randomly accessing the stream, and uses RealTime Transfer Protocol (RTP, REALNETWORKS) as the transfer protocol.
  • RTP can be used to deliver live media to one or more viewers simultaneously.
  • HTTP streaming or “progressive download” refers to media that may be viewed over a network prior to being fully downloaded.
  • Examples of software for "streaming” media include, but are not limited to, QUICKTIME, NETSHOW, WINDOWS MEDIA, REALVIDEO, REALSYSTEM G2, and REALSYSTEM 8.
  • processor and “central processing unit” or “CPU” are used interchangeably and refers to a device that is able to read a program from a computer memory (e.g., ROM or other computer memory) and perform a set of steps according to the program.
  • client-server refers to a model of interaction in a distributed system in which a program at one site sends a request to a program at another site and waits for a response.
  • the requesting program is called the "client”
  • server the program which responds to the request.
  • client is a "Web browser” (or simply “browser") which runs on a computer of a user; the program which responds to browser requests by serving Web pages is commonly referred to as a "Web server.”
  • hyperlink refers to a navigational link from one document to another, or from one portion (or component) of a document to another.
  • a hyperlink is displayed as a highlighted word or phrase that can be selected by clicking on it using a mouse to jump to the associated document or documented portion.
  • hypertext system refers to a computer-based informational system in which documents (and possibly other types of data entities) are linked together via hyperlinks to form a user-navigable "web.”
  • Internet refers to a collection of interconnected (public and/or private) networks that are linked together by a set of standard protocols (such as TCP/IP, HTTP, and FTP) to form a global, distributed network. While this term is intended to refer to what is now commonly known as the Internet, it is also intended to encompass variations which may be made in the future, including changes and additions to existing standard protocols or integration with other media (e.g., television, radio, etc).
  • standard protocols such as TCP/IP, HTTP, and FTP
  • World Wide Web or “web” refer generally to both (i) a distributed collection of interlinked, user-viewable hypertext documents (commonly referred to as Web documents or Web pages) that are accessible via the Internet, and (ii) the client and server software components which provide user access to such documents using standardized Internet protocols.
  • Web documents typically referred to as Web documents or Web pages
  • client and server software components which provide user access to such documents using standardized Internet protocols.
  • HTTP HyperText Transfer Protocol
  • Web pages are encoded using HTML.
  • Web and “World Wide Web” are intended to encompass future markup languages and transport protocols which may be used in place of (or in addition to) HTML and HTTP.
  • the term "web site” refers to a computer system that serves informational content over a network using the standard protocols of the World Wide Web.
  • a Web site corresponds to a particular Internet domain name, such as "activefilmlayer.com” and includes the content associated with a particular organization.
  • the term is generally intended to encompass both (i) the hardware/software server components that serve the informational content over the network, and (ii) the "back end” hardware/software components, including any non-standard or specialized components, that interact with the server components to perform services for Web site users.
  • HTML HyperText Markup Language which is a standard coding convention and set of codes for attaching presentation and linking attributes to informational content within documents.
  • HTML codes referred to as "tags" are embedded within the info ⁇ national content of the document.
  • HTML tags can be used to create links to other Web documents (commonly referred to as "hyperlinks").
  • HTTP refers to HyperText Transport Protocol which is the standard World Wide Web client-server protocol used for the exchange of information (such as HTML documents, and client requests for such documents) between a browser and a Web server.
  • HTTP includes a number of different types of messages which can be sent from the client to the server to request different types of server actions. For example, a "GET" message, which has the format GET, causes the server to return the document or file located at the specified URL.
  • the term "URL” refers to Uniform Resource Locator which is a unique address which fully specifies the location of a file or other resource on the Internet.
  • the general format of a URL is protocol://machine address :port/path/filename.
  • the port specification is optional, and if none is entered by the user, the browser defaults to the standard port for whatever service is specified as the protocol. For example, if HTTP is specified as the protocol, the browser will use the HTTP default port of 80.
  • the term “cookies” refers to a technology that enables a Web server to retrieve information from a user's computer that reveals information about the user. The informational item stored on the user's computer (typically on the hard drive) is commonly referred to as a "cookie.” Many standard Web browsers support the use of cookies.
  • PUSH technology refers to an information dissemination technology used to send data to users over a network.
  • World Wide Web a "pull” technology
  • PUSH protocols send the informational content to the user computer automatically, typically based on information pre-specified by the user.
  • Analog video is rapidly being replaced by digitized video.
  • Digitized video is prepared by digitally encoding an analog video, such as a film, or by analog sampling through a digital video camera followed by encoding.
  • the digitized video is compressed into a suitable format (e.g., MPEG-1, MPEG-2, DV, M-JPEG, etc.). Compression allows a great savings in memory needed to store the digitized video and bandwidth needed to transmit the digitized video.
  • MPEG-2 is the most common compression format and is being used for both DVD (Digital Video Disc or Digital Versatile Disc) and digital TV (DTV) applications.
  • DVD is a next generation optical disc storage technology. While a standard CD holds 680 megabytes of information, a single sided, dual layer DVD holds 8.5 gigabytes of information and double sided, dual layer DVD can hold up to 17 gigabytes of information.
  • the popularity of the DVD format is rapidly increasing. In 1997, there were only 347,000 DVD-video players shipped in the United States and only 900 DVD- video titles available. However, as of March 2000, there were 6.2 million DVD players shipped and 6,300 titles available in the United States.
  • Digitized video provides a quality picture and easy editing.
  • providers are not using digitized video to provide enhanced information content.
  • This is in contrast to the development many media available on the Internet.
  • pictures and text presented on Internet Web pages are commonly selectable (e.g., through the hypertext protocol) so that links to other Web sites and product or other information can be provided.
  • Digitized video is not currently amenable to such uses without the use of icons and/or graphical interfaces (See, e.g., U.S. Pat. Nos. 5,963,203 and 5,978,855, each of which is incorporated herein by reference).
  • the present invention allows the direct selection of a moving object in a running or streaming video without the use of icons. This allows the viewer to select objects while watching the running or streaming video without referring to a graphical interface or icon.
  • the present invention relates to methods and systems for selecting and providing information on objects in a video.
  • the present invention contemplates the integration of an active film layer with digitized video.
  • the active film layer allows a viewer to select objects within the video presentation, and then provides information about the objects.
  • the present invention allows production of a video of a car being driven down the road with an active film layer that allows a viewer of the video to select the car.
  • information is displayed. Examples of such information include, but are not limited to a bitmap image of the car's manufacturer and URL of the manufacturers Web site.
  • the URL is a hypertext link to the manufacturers Web site.
  • the video may be digitized and/or compressed in a variety of formats. Examples of these formats are described in the following sections: A) Digitization and Compression; B) Digital Video Disc; C) Digital Television; D) Digital Video; and E) Internet formats.
  • the active layer of the present invention is compatible with a wide variety of digitization and compression formats. Conversion of analog video to digital video creates files of enormous size. In order to make file size more manageable, a variety of compression formats have been developed.
  • the present invention is compatible with any of the various formats. Therefore, the present invention is not limited to particular systems and methods for digitizing and compressing video. Indeed, the present invention contemplates that a variety of methods and systems may be used to digitize and/or compress the video.
  • Analog video may be digitized by any method known in the art. For example, analog video (e.g., from VHS, Hi8, BetaCamSP, Dl, studio master tapes, or other studio/broadcast equipment) is converted to digitized video by a computer configured to digitize analog video signals.
  • the computer comprises a video capture card (e.g., from Media 100, Avid, or Pinnacle) which converts the analog video signal into a digital file and a RAID or fast/wide/Ultra SCSI card.
  • the video is captured at full screen resolution (e.g., 640x480 or 720x486) from a master tape. Audio is captured along with the video (e.g., at 44 liHz, 16 bit, stereo).
  • the digital video file is compressed utilizing a codec corresponding to the desired compression format.
  • codecs include, but are not limited to, those provided by REALNETWORKS (e.g., REALVIDEO G2 with SVT), CINEPAK, SORENSON VIDEO, MEDIA 100, AVID, TRUEVISION, EIDOS, INDEO, and HEURIS.
  • the present invention contemplates the use of a variety of compression formats, including MPEG-1, MPEG-2, MPEG-4, QuickTime, and DV.
  • the video is compressed in the MPEG-2 format.
  • MPEG-2 is an international standard for encoding digital video information, and is formally designated ISO/IEC 13818.
  • MPEG-2 files are system independent, unlike system specific formats such as AVI and QUICKTIME. Therefore, it is contemplated that many different hardware and software players are able to play MPEG-2 files.
  • MPEG-2 is capable of providing broadcast quality video.
  • MPEG-2 is the specified standard for broadcast digital TV, satellite digital TV, DVD, cable networks, and video games.
  • MPEG-2 also supports CD quality audio, storing two channels of 16-bit audio sampled at 32 kHz, 44.1 kHz, or 48 kHz.
  • the compressed digitized video can be delivered by a variety of means.
  • the digitized video is uploaded to a client via HTTP or FTP from a Web server and played with a suitable player.
  • a progressive download or video streaming is used to deliver the digitized video.
  • the digitized video may be provided via digital TV broadcast, digital satellite TV, CD, CD-R (recordable CD-ROM), DVD-video, and DVD-R (recordable DVD-ROM). These formats are described in more detail below.
  • the active layer of the present invention is compatible with digitized video stored on various DVD formats. Therefore, the present invention contemplates the delivery of digitized video by DVD application formats including, but not limited to, DVD-Video, DVD-Video Recording, and DVD-Stream Recording, and DVD physical formats including, but not limited to, DVD-ROM, DVD-R, DVD-RAM, DVD-RW, and DVD+RW.
  • DVD video is encoded from digital master tapes to MPEG-2 format.
  • Audio may be encoded in a variety of formats, including, but not limited to, MPEG-2 audio, Linear PCM, DOLBY DIGITAL (i.e., AC-3 coding), Sony Dynamic Digital Sound, and DTS Digital Surround.
  • Digitized video stored on DVD may be displayed by a variety of systems.
  • DVD players are designed specifically for DVD-Video and include MPEG-2 decoders for decoding the digital video files so that they may be displayed.
  • the DVD player is incorporated in a system that is capable of accessing the Internet.
  • a preferred DVD player that includes an integrated Web Browser is the NEO IDVD (www.neo.com).
  • the DVD player may be configured with an Internet ready television system (e.g., Web TV) or a computer connected to the Internet.
  • the computer operating system or playback software should support regional codes and be licensed to descramble copyprotected movies.
  • the computer preferably further comprises either a software decoder of video card capable of decoding MPEG-2 video and Dolby Digital or MPEG-2 audio.
  • Microsoft Windows 98 and Windows 2000 include DirectShow, which provides standardized support for DVD-Video and MPEG-2 playback.
  • Suppliers of video decoder cards include, but are not limited to, Creative Technology (PC-DVD Encore Dxr2), Digital Connection (3Dfusion), IBM, Ravisent (Hardware Cinemaster), and Phillips Electronics (PCDV632).
  • the active layer of the present invention is also compatible with digital television (DTV) formats.
  • DTV transmits pictures and sounds as an on/off digital stream rather than as varying analog signals. Broadcasters in the top ten U.S. TV markets began digital broadcasts in 1998.
  • DTV uses MPEG-2 compression, which allows one high- resolution, wide screen program or 4, standard resolution video programs to be multiplexed on a single channel.
  • MPEG-2 compression allows one high- resolution, wide screen program or 4
  • standard resolution video programs to be multiplexed on a single channel.
  • Different DTV systems exists.
  • An ATSC system is currently utilized in the United States. Europe and Japan are currently moving towards the use of a DVB-T system.
  • DTV programming may be provided and received by a variety of systems. Broadcast, cable, satellite and DVD may be used to supply the signal (e.g., over-the-air DTV broadcasts may be received with a UHF antenna; satellite services [e.g., EchoStar and DirecTV] also supply DTV signals). Reception and display of DTV programming uses a decoder.
  • the decoder may be integral to the display device (e.g., HDTV or other TV containing an integral decoder), or be provided as set-top box (e.g., digital cable boxes).
  • the DTV system includes Internet access capability (e.g., Web TV).
  • the active layer of the present invention is compatible with digital video (DV) and other digital video systems such as Digital ⁇ (Sony), Digital- S (JVC) and DVCPRO50 (Panasonic).
  • DV is available from a variety of companies (e.g., Panasonic, Sony, JVC, and Sharp) and uses a 6.35 metal evaporate tape to record digital video.
  • the DV system samples video at the same rate as D-l, D-5, and Digital Betacam video.
  • the sampled video is compressed with DV codec utilizing a Discreet Cosine Transform (DCT).
  • DCT Discreet Cosine Transform
  • the compressed DV video may be converted to other compression formats such as M-JPEG and MPEG-2.
  • the active layer of the present invention is also compatible with media for delivering video files over the Internet.
  • the Internet which started in the late 1960s, is a vast computer network consisting of many smaller networks that span the entire globe.
  • the computers or networks of computers connected within the Internet known as "hosts", allow public access to databases featuring information in nearly every field of expertise and are supported by entities ranging from universities and government to many commercial organizations.
  • a server is a system running on an Internet host for making available files or documents contained within that host. Such files are typically stored on magnetic storage devices, such as tape drives or fixed disks, local to the host.
  • An Internet server may distribute information to any computer that requests the files on a host.
  • the computer making such a request is known as the "client”, which may be an Internet-connected workstation, bulletin board system or home personal computer (PC).
  • TCP/IP Transmission Control Protocol/Internet Protocol
  • IP Internet Protocol
  • the World-Wide Web is a method of accessing information on the Internet which allows a user to navigate the Internet resources intuitively, without IP addresses or other technical knowledge.
  • the Web dispenses with command-line utilities which typically require a user to transmit sets of commands to communicate with an Internet server. Instead, the Web is made up of hundreds of thousands of interconnected "pages", or documents, which can be displayed on a computer monitor.
  • the Web pages are provided by hosts running special servers. Software which runs these Web servers is relatively simple and is available on a wide range of computer platforms including PC's. Equally available is a form of client software, known as a Web "browser", which is used to display Web pages as well as traditional non- Web files on the client system.
  • HTTP Hypertext Transfer Protocol
  • HTML pages are made up of standard text as well as formatting codes which indicate how the page should be displayed. The Web client, a browser, reads these codes in order to display the page.
  • Each Web page may contain pictures and sounds in addition to text. Hidden behind certain text, pictures or sounds are connections, known as “hypertext links" ("links"), to other pages within the same server or even on other computers within the Internet. For example, links may be visually displayed as words or phrases that may be underlined or displayed in a second color. Each link is directed to a web page by using a special name called a URL (Uniform Resource Locator). URLs enable a Web browser to go directly to any file held on any Web server. A user may also specify a known URL by writing it directly into the command line on a Web page to jump to another Web page.
  • URL Uniform Resource Locator
  • the URL naming system consists of three parts: the transfer format, the host name of the machine that holds the file, and the path to the file.
  • An example of a URL may be: http://www.college.univ.edu/Adir/Bdir/Cdir/page.html, where "http” represents the transfer protocol; a colon and two forward slashes (://) are used to separate the transfer format from the host name; "www.college.univ.edu” is the host name in which "www” denotes that the file being requested is a Web page; "/Adir/Bdir/Cdir” is a set of directory names in a tree structure, or a path, on the host machine; and "page.html” is the file name with an indication that the file is written in HTML.
  • a number of formats are available for streaming video files over the Internet. These formats include, but are not limited to, REALVIDEO (REALNETWORKS, utilizing M-PEG2 compression), REALSYSTEM 8, QUICKTIME, WINDOWS MEDIA, and VIVOACTIVE. It is also contemplated that increasing bandwidth will enable streaming of DVD-Video and other large video files over the Internet.
  • Video files can be placed in Web pages, for example, in two ways: by use of an anchor tag ( ⁇ a href>) or an embed tag ( ⁇ embed>).
  • the anchor tag is used in the same manner as placing any other link within an HTML document.
  • the present invention provides methods and systems for providing information to users on objects in a video.
  • the methods and systems allow selection of objects displayed in a video that is derived from a digitally encoded or digitized video file as described above.
  • information is provided.
  • the information may be related to the object (e.g., product information about the object, info ⁇ nation about the manufacturer or provider of the object, or information about an actor in the video) or be completely independent of the nature of the object.
  • the information may be provided immediately or at a later time (e.g., e- mailed or placed in a bookmark file, favorite file, or some other file).
  • the methods and systems for object selection and information display are discussed in more detail below. This description is directed to certain preferred embodiments and is not intended to limit the scope of the invention.
  • the present invention provides video media 100 comprising an Active Film Layer (AFL) archive 105 and, optionally, other content 110 (e.g., CD audio, DOLBY DIGITAL sound, etc.).
  • the video media 100 may be provided by any computer readable memory suitable for storing digital video files (e.g., hard disk drives, CD, DVD, magnetic tape, etc.).
  • the AFL archive 105 comprises a data file with at least two components, object information 115 and digital video stream 120.
  • the object information 115 and digital video stream 120 are contained in a single file. It is contemplated that the single file format has several advantages, including ease of encryption and prevention of removal of the object information 115. Therefore, in preferred embodiments, the AFL archive 105 is encrypted to prevent hacking or other manipulation of the AFL content.
  • the AFL archive 200 is read in a two step procedure.
  • the object information 205 is read by an object parser 210. It is contemplated that the object parser 210 reads the object information from the AFL archive 200 and initializes the AFL at startup, before the video (e.g., a movie 215 or television program) starts playing.
  • the player 220 reads the compressed movie 215 from the AFL archive and streams the movie frame by frame.
  • the AFL detects any selection events (e.g., selection by a mouse over action or mouse click) and initiates the appropriate action (e.g., display of text containing information on the object or opening a Web browser to a site related or unrelated to the object).
  • the object information file is "flat" textfile. It is contemplated that the object information file has a simple structure and syntax, and can be edited with text editor software, as well as generated by special programs.
  • an object is an active part of the display on a display device (e.g., cathode ray tube, LCD screen), and typically is an item displayed in the video.
  • the object information file contains information on a plurality of objects displayed in the video. Referring to Figure 3, in some embodiments it is contemplated that an object in a video display 300 can be described by three functions: it is born, it moves/changes in size, and it dies. In the first frame 305, object A 310 is born.
  • object B 320 is born and object A 310 moves and changes in size.
  • object C 330 is born, object A 310 remains stationary, and object B 320 moves.
  • object B 320 dies, object A 310 remains stationary, and object C 330 moves.
  • object A 310 remains stationary, and object C 330 dies.
  • sixth fame 345 object A 310 dies.
  • the object information file includes information relating to each of the three functions.
  • the three functions are designated by keywords: "init”, “key”, and “kill”.
  • the "init” and “kill” functions both include an implicit "key” function.
  • the object information file contains templates (e.g., classes relating to the properties of the objects) for the different objects that appear in the video, as well as information about where and when the objects appear (e.g., keyframes with polygons on screen that refer to a class).
  • templates e.g., classes relating to the properties of the objects
  • information about where and when the objects appear e.g., keyframes with polygons on screen that refer to a class.
  • a non-limiting example of this type of file is as follows:
  • the first part of the file conatins the class information (i.e., the properties of the objects that are to appear during video playback).
  • a hash (#) at the start of the line denotes a comment in the file (i.e., the entire line is ignored when the file is parsed during the initialization of the AFL video).
  • Each class has a unique name that the objects will refer to (e.g., "Class Nokia").
  • Each class has its properties described between the parentheses ⁇ . . . ⁇ .
  • the class in the example has two properties: 1) the action to be executed when the user designates or selects (e.g., holds the mouse pointer or cursor over) an object of the class; and 2) the action to be executed when the user executes another selection action (e.g., clicks the mouse button) on an object of the class.
  • an image map e.g., corporate logo
  • a selection action e.g., a mouse click
  • a Web browser is started and the corporate web page is opened.
  • the second part of the file contains information about the time and space of the active objects (i.e., timecodes and coordinates).
  • the keyframes are interpolated to create fluid movement of the objects.
  • This example uses a simple geometric shape (i.e., a rectangle) to activate an area on the screen.
  • the rectangle is represented by the coordinates of its upper left and lower right corners.
  • more complex shapes e.g., circles, ellipses, triangles, squares, pentagons, rhombuses, hexagons, octagons, or more complex shapes and combinations thereof
  • the present invention is not intended to be limited to a particular mechanism of action.
  • the active object may be designated by a variety of polygon shapes or combination of individual polygon shapes.
  • the polygon may include 10, 100, 1000 or more sides, and in some cases approximate a tracing of the object.
  • the shape can also be a freehand or abstract shape.
  • the polygon is invisible to the viewer of the video. The designation of the polygon shapes in the file correlates the selection device with the object in the video.
  • the digital video stream 120 may be uncompressed or preferably compressed.
  • Software and hardware i.e., codecs
  • Preferred compression formats include, but are not limited to, MPEG-1, MPEG- 2, MPEG-4, DV, MOV, and M-JPEG.
  • AFL Active Film layers
  • the timeline of typical video editing software contains a number of Video and Audio layers in addition to Transition and Effect layers. Transition layers determine which video layer is the actual video output, while effect layers are used for superimposing graphical objects like text and logos on top of video images.
  • an AFL is added to the video timeline, making it possible to include additional data layers in the video output. In preferred embodiments, these layers are invisible to the viewer upon playback.
  • the AFLs are readable by the software or hardware playing back or streaming the video.
  • an AFL layer designated to that object is created.
  • the AFL is created from a menu in the same way that video and audio layers are created.
  • a key frame is created in the AFL.
  • a shape e.g., a polygonal shape
  • Tins shape is associated with the video image until it is removed from the AFL.
  • additional keyframes are created in the AFL so that the shapes follow the object movement in the video layer.
  • the polygon shape is edited in each keyframe to compensate for movement in the video layer.
  • blending techniques available in vector-based drawing programs e.g., Adobe Illustrator, Corel Draw, and Macromedia Flash
  • an action is added to the object.
  • Action objects are created in a separate window and are imported data objects (e.g., bitmap, vector, or audio files). Once imported, the action objects are assigned actions (e.g., a web link [URL]).
  • the AFL object is completed by association with an action. In preferred embodiments, this is done by selecting the desired action from a dropdown menu in the monitor window. This process provides a single, complete AFL active object. The process may be repeated to provide a variety of additional AFL objects in new AFL layers. It is contemplated that there is no theoretical limit to the number of AFLs that can be created for a video file, however, an increasing number of AFLs increases the requirements for the playback system.
  • the video file e.g., movie
  • the video file is then finished, with the desired video and audio formats being selected along with the desired level of compression, and the finished file rendered to disk or other suitable storage media.
  • a video containing an AFL layer After a video containing an AFL layer is produced, it may be played or provided through a variety of systems.
  • Preferred systems for playing AFL media comprise a selection device and a display device.
  • the selection device may be any type of selector or data input unit known in the art and its associated software, if any, including, but not limited to, a computer mouse, a keyboard, a joystick, a trackball, a touchscreen, a touchpad, a data input pen, and combinations and wireless versions thereof.
  • Preferred display devices include any devices for displaying video known in the art, including but not limited to, cathode ray tubes and LCD screens.
  • the selection device is in communication with a central processor so that actions performed with the selection device (e.g., mouse over or mouse click) are relayed to the processor to cause a particular action (e.g., display of a logo or opening a Web browser to a Web page).
  • actions performed with the selection device e.g., mouse over or mouse click
  • a particular action e.g., display of a logo or opening a Web browser to a Web page.
  • the system comprises a processor or processors configured to display AFL media.
  • the system also comprises a computer memory device for storing the AFL media so that it can be played.
  • a variety of computer memory devices find use with the present invention, including hard disk drives, magnetic tape, DVD-ROM, DVD-R, DVD-RAM, DVD-RW, DVD+RW, and CDs.
  • the computer memory device may be integral to the processor (i.e., form a part of the same machine) or be remote from the processor (e.g. , at a television station or transmitter in the case of broadcast digital television or a Web server for streaming video).
  • the system comprises a computer (e.g., a PC or any other device containing a central processing unit) configured to display AFL media.
  • the computer comprises at least one central processing unit (processor), a random access memory (RAM), a read only memory (ROM), a clock, and a computer memory device.
  • the computer comprises a display device, a selection device, a codec, and a communications port.
  • the processor can be connected to each of these elements by a shared data bus or dedicated connections.
  • the communications port can connect the computer to other computers in a network such as the Internet.
  • the ROM and/or computer memory device are operable to store one or more instructions (discussed further below), which the processor is operable to retrieve, interpret, and execute.
  • ROM and/or computer memory device can store AFL media (e.g., stored permanently or temporarily thereon) and a player for the AFL media.
  • the ROM and/or computer memory device store processes to accomplish, among other things, initialization of the AFL player (described below), decompression and playing of the video through the display device, and selection of objects in the video by the selection device.
  • the AFL media is provided in a computer memory device selected from DVD-ROM, DVD-R, DVD-RAM, DVD-RW, DVD+RW, and CDs.
  • the computer is in communication through its communications port with other computers via a computer network (e.g., Internet, intranet, or LAN).
  • a computer network e.g., Internet, intranet, or LAN.
  • the processor is configured so that data, including AFL media, cookies, and other client identification information can be sent between computers.
  • the processor is configured so that selection of an object imaged on a video (e.g., an object that has been "activated" through an AFL) initiates communication with other computers or servers via a Web browser or other software or hardware.
  • the computer is a Web TV. It is contemplated that the AFL media can be displayed on the Web TV by a variety of methods, including but not limited to, download of the AFL media from a server via the Internet (e.g., streaming video containing an AFL), reception of satellite or broadcast DTV on Web TVs equipped with a codec (e.g., an integral codec or set-top codec), and connection to a DVD player via video input connectors.
  • the computer is a DVD player with Internet capability (e.g., NEO IDVD, www.neo.com). It is contemplated that this system can be used to play AFL media stored on DVD-ROM (e.g., DVD-video), or AFL media downloaded or streamed from a server or other source over the Internet.
  • a computer 400 is provided.
  • the present invention is not limited to any particular computer 400. Indeed, a variety of computers 400 are contemplated, including, but not limited to, personal computers, Web TVs, TVs equipped to receive DTV, DVD players, and any other device containing a processor.
  • the computer further comprises a web browser 405 (e.g., NetScape or Internet Explorer), computer memory including cookies 410, and computer memory comprising an active film layer file 415.
  • a web browser 405 e.g., NetScape or Internet Explorer
  • computer memory including cookies 410
  • computer memory comprising an active film layer file 415.
  • the present invention is not limited to any particular media for providing the active film layer file 415.
  • active film layer media including, but not limited to, DVD-ROM (e.g., DVD-video), DVD-R+W, CD-ROM, hard disk, streaming from an internet web server or other source, and broadcast or satellite DTV.
  • the computer 400 is in communication with a display device 420 so that the video in the active film layer file 415 can be displayed to a viewer.
  • the computer 400 is also in communication with a selection device 425 so that objects (preferably moving objects) in the video encoded in the active layer file can be selected.
  • the computer 400 is in communication (e.g., for example, through the Internet) with a Web site 430.
  • the Web site 430 comprises a Web server 435, computer memory 440 comprising HTML documents (e.g., product pages, shopping cart pages, and product order pages), computer memory containing various databases, such as a shopping cart database 445, a customer database 450, and a referral database 455, and computer programs 460 (e.g., cookie processing software 465, shopping cart software 470, and customer database software 475).
  • selection of an active object in the video displayed from the active film layer file causes the computer 400 to open the web browser 405 to a URL specified in the active film layer file 415.
  • the Web browser 405 then accesses the Web site 430 identified by the URL.
  • the user can then examine, for instance, product web pages, place orders in the product order page, and examine the total order in the shopping cart page.
  • information about the user is provided to the web server 435 from the computer memory including cookies 410 and processed by the cookie processing software 465.
  • information about the customer e.g., name, electronic mail address, purchase preferences, etc.
  • the information is used to develop electronic mail lists via the customer database software 475 for future marketing.
  • information concerning the source of the referral to the Web site e.g., that the referral was through active film layer media
  • the system preferably comprises a ROM and/or other computer memory device storing processes for displaying AFL media.
  • the software for displaying the video from the AFL media is divided into two components, the object parser and the player.
  • the object parser preprocesses the object information before the video (e.g., a movie) starts playing.
  • the object parser reads all classes and generates templates for the objects.
  • the object parser then reads the timecode/polygon information and creates an object map. This map is then used for reference during playback of the video so that the presence, coordinates and properties of active objects may be determined.
  • step 500 an AFL archive is opened.
  • step 505 lines within the archive are read and classes are generated in random access memory.
  • step 510 a query is performed as to whether more classes are designated. If yes, step 505 is repeated and the additional class is generated in random access memory. This subroutine is repeated until all classes are read. If the answer to the query step 510 is no, step 515 is performed and the next line is read and a timechart created 515.
  • step 520 a query is performed as to whether more keyframes are present. If yes, the subroutine is repeated.
  • step 525 the information is provided to the player for use during playback.
  • the player also comprises two modules: the video streamer (e.g., codec) and the AFL.
  • the video streamer reads the video file and shows the video frame by frame, as well as playing the sound and any additional content.
  • the AFL reads the object map generated by the preprocessor to capture any interaction by the user (e.g., mouse over or mouse click) and initiate the appropriate action. Operation of the player is illustrated in Figure 6.
  • step 600 the video file is opened.
  • step 605 a frame the video file is read to memory.
  • a query is performed in step 610 to determine whether the end of the file has been reached. If yes, step 615 is executed and the player quits.
  • step 620 is executed and the video streamer gets a video frame and displays it in step 625.
  • step 630 the AFL subroutine is initiated to get the timecode and object information.
  • a query is performed in step 635 to determine whether any active objects are present in the frame (i.e., displayed on screen). If no, step 640 is performed to advance one frame and the subroutine is repeated. If yes, a second query is performed in step 645 to determine whether a mouse pointer is inside the object. If no, step 640 is performed to advance the video one frame and the subroutine repeated. If yes, step 650 is executed and the mouse over action performed. In step 655, another query is performed as to whether the mouse button is clicked.
  • step 640 is performed to advance one frame and the subroutine is repeated. If yes, an update is sent to a central server in step 660. Next, in step 665, the mouse click action is executed. Step 640 is then performed to advance one frame and the routine is repeated.
  • the present invention is not limited by the number of active objects within a video frame or frames nor by the device used to select the active object or action initiated upon selection of the object. D. Uses of AFL Media
  • AFL media finds use both on its own and as a part of a variety of methods. Any and all uses of the AFL media and systems are contemplated, and a number of illustrative examples are described below.
  • an AFL system may be used to select active objects imaged in a video stream.
  • the AFL system is configured to cause an action to be performed (e.g. , information related or unrelated to the active object is provided).
  • information relating to the active object is provided from the object information. It is contemplated that different types of information may be provided for active objects.
  • text identifying the active object is provided. Such text can include information such as the manufacturer of the object, suppliers of the object, history of the object, the model number of the object, etc.
  • a bitmap image relating to the object is displayed on the display device.
  • the bitmap image can be the logo of a supplier or manufacturer of the object, and can also include information on the object such as the model number, etc.
  • an additional selection action e.g., a mouse click
  • AFL media and systems are used in a business method to generate product placement revenue.
  • a manufacturer or supplier of a product displayed in a video (e.g., a movie or television) program pays a product identification fee to have AFL layers created so that product (i.e., object) can be made active in a digital video format.
  • product identification fees can reduce the cost (by recovering costs through fees) of movies and other programming distributed on digital media, including, but not limited to, DVDs, laser discs, and network TV, cable TV, and pay-per-view programming distributed by over the air broadcast, satellite broadcast, and cable.
  • AFL layers are created so that information on various items used by an actor (e.g. , cars, cellular phones, clothing, weapons, cigarettes, make up, hair products, and branded food such as chips, soda, or alcoholic beverages) are provided.
  • an actor e.g. , cars, cellular phones, clothing, weapons, cigarettes, make up, hair products, and branded food such as chips, soda, or alcoholic beverages
  • information on the actor(s), director(s), and others can be provided.
  • the information provided is a URL for a Web site.
  • an action by the user e.g., a mouse click
  • the Web browser window is displayed on a split screen or in a corner of the display device so that viewing of the video (e.g., movie or television) is not disrupted.
  • selection of an object whose object information comprises a URL causes the URL to be saved in a favorites list in a Web browser or other application so that the user can access the Web page addressed by the URL after the completion of the video (e.g., movie or television program), or during an intermission or commercial.
  • selection causes a pause in the video.
  • the Web browser is already opened and may preferably be minimalized. Selection of an object comprising a URL in its object information then either maximizes the Browser window and causes access of the desired Web page or stores the URL in a favorites file as described above.
  • the present invention also contemplates the integration of one click shopping for the display item (e.g., clicking on the object results in its purchase).
  • the Web browser configured with the AFL system comprises a cookie module which stores an identifier code for the AFL system.
  • a cookie is a file that stores information that can be utilized by a Web site.
  • the specification for cookies is defined by Netscape corporation and can be found at http://www.netscape.com/newsref/std/cookie_spec.html.
  • the cookie allows identification of visits to Web sites (e.g. , product supplier or producer Web sites) as being generated by links (e.g., URLs) provided by AFL media.
  • links e.g., URLs
  • Those skilled in the art will realize that other methods exist for determining whether a visit to a particular Web site is initiated from AFL media. Such methods, include, but are not limited to, encoding origination information (e.g., an AFL identifier) directly in the URL provided by the object information.
  • the AFL media finds use for both increasing the information content of video media such as movies and TV programming and allowing interactivity with such programming.
  • selection of an actor provides fan club information, information on other movies the actor is in, or biographical information.
  • an interactive video e.g., a movie
  • the information provided is an alternative scene from the viewpoint of the active object or zooming in on the active object.
  • selection of active object leads to a different plot being played out.
  • AFL media is used to enhance educational programming.
  • an interactive childrens program teaching math, spelling, or science is provided where a question is posed and various answers or objects displayed on the screen. A child watching the program then attempts to select the correct answer from among several choices, each of which is an active object.
  • the processor is configured to tabulate, display, and store the results so that they can be accessed by the user at any time or compared over time to monitor improvement. In other embodiments, a wrong answers initiates a tutorial video or graphic.
  • AFL media comprising a video on gardening is provided.
  • active objects in the video include, but are not limited to, plants, tools, and site information.
  • Selection of the plants, tools, or sites provides information on the plants, sources from which the plants are available, URLs of Web sites providing information on the plants, model numbers for tools, suppliers of tools, URLs of Web sites selling the tools, soil data, sun exposure data, and watering data.
  • AFL media is used to enhance game show programming.
  • the viewing audience participates in quiz show type programming by answering questions as they are posed.
  • the answers are provided as active objects and the user selects the correct answer from among several choices.
  • the answers are communicated via the Internet to a Web site configured to receive the results.
  • the viewer has previously created an account with the Web site into which the data is directed.
  • correctly answering all or a subset of the questions makes the user eligible for prizes, discounts, or rebates on products.
  • the AFL media is used to create games within digitized videos (e.g., TV programs, movies, or commercials). For example, in some embodiments it is contemplated that a collection of objects in a TV program or commercial are made active. The viewer then searches for and selects the objects during the program or commercial, and the results of the selection are communicated to a Web site. In some preferred embodiments, the viewer has previously created an account with the Web site into which the data is directed. Selection of all or a subset of the active objects then makes the viewer eligible for a prize, prizes, or discounts or rebates on products.
  • digitized videos e.g., TV programs, movies, or commercials.

Abstract

The present invention provides methods, systems, and media for enhancing the information content of video. The methods, systems, and media provided herein allow a viewer to select a moving object in a video display. Selection of the object can result in the display of information related or unrelated to the object. Examples of information include textual information, computer generated graphic information, other video files, hypertext links, and URLs.

Description

ACTIVE LAYER FOR DIGITAL VIDEO
FIELD OF THE INVENTION
The present invention relates to methods and systems for selecting and providing information on objects in a digital video.
BACKGROUND OF THE INVENTION
The Internet has revolutionized the way that information is delivered and business is done. In June of 1999, Nielsen/NetRatings reported that there were a total of 63.4 million active Internet users in the United States, and 105.4 million total Internet users with Internet access. The average user spent 7 hours, 38 minutes on-line that month. Furthermore, user year-to-year growth rate is expected be in the range of 15% to 25% percent. Worldwide, it expected that there be greater than 250 million residential users, and greater than 200 million corporate users by the year 2005.
The Internet has proved be an excellent medium for advertising goods and services. There are two main advantages. First, the cost per 100 exposures is the lowest of all media, including national television. Second, the interactive nature of the Internet encourages the empowerment of consumers because they have the ability sit in their own home, business, or school and gather information on advertized goods and services. Advertisers generally take advantage of this interactivity by offering multimedia delivery of information, including graphics, pictures, sound, video, and animation.
However, the interactivity of the Internet, as well as other digital media such as DVDs and CDs, is largely limited to the use of graphical interfaces. For example, while a Web site may offer a video that can be downloaded or streamed for viewing, there is currently no means for enhancing the information content of the video by providing interactivity within the video presentation. DVDs movies offer interactivity through dropdown menus and menus accessed through remote controls, all through graphical interfaces. Use of these graphical interfaces may disrupt viewing of the video. Furthermore, like streamed video over the Internet, interactivity within the video presentation is not available.
Accordingly, what is needed in the art are methods and systems for increasing the information content of video and providing interactivity within a video presentation. DESCRIPTION OF THE FIGURES
The following figures form part of the present specification and are included to further demonstrate certain aspects and embodiments of the present invention. The invention may be better understood by reference to one or more of these figures in combination with the detailed description of specific embodiments presented herein.
Figure 1 is a schematic representation of active film layer media structure.
Figure 2 is a schematic representation of a preferred method for reading active film layer media.
Figure 3 is a schematic representation of designation of active objects within a video.
Figure 4 is a schematic representation of a system for using active film layer media.
Figure 5 is a flow chart representing operation of the object parser.
Figure 6 is a flow chart representing operation of a preferred active film layer player.
SUMMARY OF THE INVENTION
The present invention relates to methods and systems for selecting and providing information on objects in a digital video. In some embodiments, the present invention provides methods for providing information regarding an object in a video comprising: a) providing a digitized video display of at least one object, wherein the at least one object is selectable by a selection device; and b) providing information upon selection of the at least one object. In some particularly preferred embodiments, the digitized video further comprises an active film layer.
The present invention is not limited to video compressed by any particular compression format. Indeed, a variety of compression formats are contemplated, including, but not limited to, MPEG-2, DV, and M-JPEG. The present invention is not limited to any particular selection device. Indeed, a variety of selection devices or input devices are contemplated, including, but not limited to, a computer mouse, a computer keyboard, a joystick, trackball, and a touchpad. The present invention is not limited to providing any particular information. Indeed, the provision of a variety of information is contemplated, including, but not limited to, textual information, computer generated graphic information (e.g., bitmaps), and URLs or hypertext links. In some embodiments, where the information is a hypertext link, the hypertext link is displayed after clicking on the at least one object with a computer mouse. In further embodiments, the hypertext link is to a manufacturer site or provider site of the object. In some embodiments, the methods of the present invention further comprise providing a public network for communication between computers, wherein the information in step (b) is provided via a hypertext link.
In other embodiments, the present invention provides methods for providing information on objects pictured in digitized video comprising: a) providing: i) a digitized video of at least one moving object; ii) a selection device; and iii) a processor configured to allow selection of the at least one moving object; b) via the processor, implementing selection of the at least one moving object with the selection device to provide a selected object; and c) providing information on the selected object.
In still further embodiments, the present invention provides methods for providing information on objects in a digitized video comprising: a) providing: i) digitized video of at least one object, ii) a display unit, iii) a computer processor configured to display the digitized video and allow selection of the at least one object, and ii) a selection device; b) via the processor, displaying the digitized video of at least one object on the display unit; c) via the processor, implementing selection of the at least one object with the selection device to provide at least one selected object; and d) providing information on the at least one selected object.
In other embodiments, the present invention provides systems for providing information on an object in a digitized video comprising: a digitized video of at least one object, a computer processor, a selection device, a display unit, and an active layer, wherein the processor is configured so that the digitized video of at least one object is displayed on the display unit and wherein the active layer implements selection of the at least one object with the selection device. The present invention is not limited to video compressed by any particular compression format. Indeed, a variety of compression formats are contemplated, including, but not limited to, MPEG-2, DV, and M-JPEG. The present invention is not limited to any particular selection device. Indeed, a variety of selection devices or input devices are contemplated, including, but not limited to, a computer mouse, a computer keyboard, a joystick, trackball, and a touchpad. The present invention is not limited to providing any particular information. Indeed, the provision of a variety of information is contemplated, including, but not limited to, textual information, computer generated graphic information (e.g., bitmaps), and URLs or hypertext links, the present invention is not limited to any particular display device. Indeed, a variety of display devices are contemplated, including, but not limited to, cathode ray tubes and liquid crystal display devices. In still further embodiments, the system further comprises a public network for communication between computers coupled to the processor.
The present invention is not limited to a particular source of digitized video. Indeed, a variety of sources of digitized video are contemplated, including, but not limited to, broadcast digital television, DVD (e.g., DVD-ROM or DVD-Video), and streamed digitized video provided from a server and viewed through a media player (e.g., QUICKTIME, WINDOWS MEDIA, and REALVIDEO)
In still further embodiments, the present invention provides systems for providing information on an object in a digitized video comprising: a digitized video encoding pictures of at least one object, a computer processor, and a selection device, wherein the processor is configured so that the at least one object is selectable by the selection device.
In other embodiments, the present invention provides a computer readable medium comprising a plurality of instructions, which when executed by a computer cause the computer to perform the steps of: a) displaying a digitized video of at least one object; and b) implementing selection of the at least one object. In some embodiments, the digitized video supplied by the computer readable medium is compressed. The computer readable medium of the present invention is not limited to digitized video compressed in any particular format. Indeed, a variety of compression formats are contemplated, including, but not limited to, MPEG-1, MPEG-2, DV, and M-JPEG. in still further embodiments, the computer readable medium comprises an active layer. The present invention is not limited to any particular computer readable medium. Indeed, a variety of media are contemplated, including, but not limited to, DVD, CD, magnetic tape, and hard disk drives.
In some embodiments, the present invention provides a computer readable medium comprising a plurality of instructions, which when executed by a computer cause the computer to perform the steps of: a) displaying a digitized video of at least one object; b) implementing selection of the at least one object; and c)providing information on the at least one object following selection.
In still further embodiments, the present invention provides methods comprising: a) providing: i) a digitized video of at least one moving object; and ii) a selection device b) displaying the digitized video; and c) selecting the at least one object with the selection device. In some embodiments, the methods further comprise step d) displaying information on the at least one object, in other embodiments, the present invention provides methods comprising providing: i) computer memory having stored therein a video file of a least one object; and ii) a selection device; b) displaying the video file; c) selecting the at least one object with the selection device; and d) providing information on the at least one object. In some embodiments, the video file comprised digitized video. In other embodiments, the digitized video is compressed with a MPEG- 1, MPEG-2, MPEG-4, or DV codec. In still other embodiments, the video file further comprises computer generated graphics. The present invention is not limited to a particular selection action. Indeed, a variety of selection actions are contemplated, including but not limited to, mouse over actions and mouse click actions. In some preferred embodiments, a mouse click action opens a Web browser to a predetermined URL. In still other embodiments, the Web browser is configured to provide source information (e.g. , referral information encoded in a cookie) to a server or other computer, in some embodiments, the selection of an object enters a user in a contest.
In still further embodiments, the present invention provides computer memory comprising a video editor, wherein the video editor comprises an active film layer. In some embodiments, the active film layer comprises object information and digitized video. The present invention is not limited to any particular object information. Indeed, a variety of object information is contemplated including bitmaps and URLs.
In other embodiments, the present invention provided methods comprising: a) providing: i) a computer memory having stored therein a video editor and an active film layer; ii) a video file; b) designating an image in the video file; c) writing object information on the object in the active film layer. In some embodiments, the video file is digitized video. The present invention is not limited to any particular object information. Indeed, a variety of object information is contemplated including bitmaps and URLs. In still further embodiments, the present invention provides computer readable medium comprising: a digitized video file of at least one moving object; and an object information file comprising data on the moving object. In some embodiments, the data on the moving object allows selection of the moving object. In other embodiments, the data on the moving object is object information.
In other embodiments, the present invention provides methods comprising: a) providing: i) a digitized video file of at least one moving object; and ii) an object information file comprising data allowing selection of the at least one moving object; b) displaying the digitized video; and c) selecting the at least one moving object. In some embodiments, the methods further comprise further step: d) providing information on the moving object. The present invention is not limited to providing any particular information. Indeed, the provision of a variety of information is contemplated, including, but not limited to, textual information, computer generated graphic information (e.g., bitmaps), digitized video files, and URLs or hypertext links.
DEFINITIONS
To facilitate understanding of the invention, a number of terms and phrases are defined below:
As used herein, the term "digitized video" refers to video that is either converted to digital format from analog format or recorded in digital format. Digitized video can be uncompressed or compressed into any suitable format including, but not limited to, MPEG-1, MPEG-2, DV, M-JPEG or MOV. Digitized video can include computer generated graphics, but does consist entirely of computer generated graphics. For example, a digitized video image may comprise one or more graphic components, but a computer generated graphic by itself is not digitized video. An example of digitized video comprising a graphic object include a QUICKTIME video including a Sprite track or layer. As is known in the art, Sprite tracks can be used to superimpose a graphical image (e.g., an image encoded by a bitmap such as a bouncing ball) during video playback. Furthermore, digitized video can be delivered by a variety of methods, including playback from DVD, broadcast digital TV, and streaming over the Internet.
As used herein, the term "video display" refers to a video that is actively running, streaming, or playing back on a display device. As used herein, the term "codec" refers to a device, either software or hardware, that translates video or audio between its uncompressed form and the compressed form (e.g., MPEG- 2) in which it is stored. Examples of codecs include, but are not limited to, CINEPAK, SORENSON VIDEO, ΓNDEO, and HEURIS codecs. "Symetric codecs" encodes and decodes video in approximately the same amount of time. Live broadcast and teleconferencing systems generally use symetric codecs in order to encode video in realtime as it is captured.
As used herein, the term "compression format" refers to the format in which a video or audio file is compressed. Examples of compression formats include, but are not limited to, MPEG-1, MPEG-2, MPEG-4, M-JPEG, DV, and MOV.
As used herein, the term "MPEG-2" refers to a particular compression format that is capable of producing a high data rate and full broadcast quality files. MPEG-2 is full frame rate (e.g., 24-30 fps) and screen resolution (e.g., 720x480). MPEG-2 is the current standard format for DVD-Video, broadcast digital TV, and satellite digital TV.
As used herein, the term "active film layer" refers to a layer or track in a video timeline that comprises data relating to objects in a digitized video. Among other things, the data can serve to define the object (i.e., its presence and/or location on a display screen) and provide information on the object (e.g., a bitmap containing text or a URL of a Web site).
As used herein, the terms "active object" and "active video object" are used interchangeable and refer to an object in a video presentation (e.g., digitized video, movie, or TV) that is selectable by a selection device. The terms "active object" and "active video object" do not include computer generated graphic objects such as those encoded by bitmaps. The terms "moving object" and "moving active object" are used herein to refer that objects that move, change position, grow in size, or diminish in size from frame to frame in a video. It is intended that the term "moving object" and "moving active object" encompass objects whose movement is kinetic (e.g., a car driving down a road or a person walking) as well as objects whose movement from frame to frame is cause by the changing perspective of the recording instrument (e.g., a panning movie camera or television camera).
As used herein, the term "selection device" refers to devices (e.g., data input devices) that can be used to select objects displayed on a display device. Example of selection devices include, but are not limited to, a computer mouse, a computer keyboard, a joystick, a trackball, a monitor touchpad, a computer touchpad, computer data entry pens, laser light, voice control and wireless versions thereof.
As used herein, the term "product information" refers to any information relevant to a product, including, but not limited to, marketing information, source information, price, URL, purchase information, instructional information, identity information, etc.
As used herein, the term "manufacturer site" refers to an electronically accessible site (e.g., a Web site) operated by the manufacturer of a product.
As used herein, the term "provider site" refers to an electronically accessible site (e.g., a Web site) that sells or otherwise provides a product.
As used herein, the terms "computer memory" and "computer memory device" refer to any storage media readable by a computer processor. Examples of computer memory include, but are not limited to, RAM, ROM, computer chips, digital video disc (DVDs), compact discs (CDs), hard disk drives (HDD), and magnetic tape.
As used herein, the term computer readable medium refers to any device or system for storing and providing information (e.g., data and instructions) to a computer processor. Examples of computer readable media include, but are not limited to, DVDs, CDs, hard disk drives, magnetic tape and servers for streaming media over networks.
As used herein, the term "video editor" refers to computer hardware or software applications for adding or deleting information from a digitized video, and providing supplemental information to a digitized video file. A non-limiting example of a video editor is the Adobe Premiere system.
As used herein, the term "streaming" refers to the network delivery of media. "True streaming" matches the bandwidth of the media signal to the viewer's connection, so that the media is seen in realtime. As is known in the art, specialized media servers and streaming protocols are used for true streaming. RealTime Streaming Protocol (RTSP, REALNETWORKS) is a standard used to transmit true streaming media to one or more viewers simultaneously. RTSP provides for viewers randomly accessing the stream, and uses RealTime Transfer Protocol (RTP, REALNETWORKS) as the transfer protocol. RTP can be used to deliver live media to one or more viewers simultaneously. "HTTP streaming" or "progressive download" refers to media that may be viewed over a network prior to being fully downloaded. Examples of software for "streaming" media include, but are not limited to, QUICKTIME, NETSHOW, WINDOWS MEDIA, REALVIDEO, REALSYSTEM G2, and REALSYSTEM 8. As used herein, the terms "processor" and "central processing unit" or "CPU" are used interchangeably and refers to a device that is able to read a program from a computer memory (e.g., ROM or other computer memory) and perform a set of steps according to the program.
As used herein, the term "client-server" refers to a model of interaction in a distributed system in which a program at one site sends a request to a program at another site and waits for a response. The requesting program is called the "client," and the program which responds to the request is called the "server." In the context of the World Wide Web (discussed below), the client is a "Web browser" (or simply "browser") which runs on a computer of a user; the program which responds to browser requests by serving Web pages is commonly referred to as a "Web server."
As used herein, the term "hyperlink" refers to a navigational link from one document to another, or from one portion (or component) of a document to another. Typically, a hyperlink is displayed as a highlighted word or phrase that can be selected by clicking on it using a mouse to jump to the associated document or documented portion.
As used herein, the term "hypertext system" refers to a computer-based informational system in which documents (and possibly other types of data entities) are linked together via hyperlinks to form a user-navigable "web."
As used herein, the term "Internet" refers to a collection of interconnected (public and/or private) networks that are linked together by a set of standard protocols (such as TCP/IP, HTTP, and FTP) to form a global, distributed network. While this term is intended to refer to what is now commonly known as the Internet, it is also intended to encompass variations which may be made in the future, including changes and additions to existing standard protocols or integration with other media (e.g., television, radio, etc).
As used herein, the terms "World Wide Web" or "web" refer generally to both (i) a distributed collection of interlinked, user-viewable hypertext documents (commonly referred to as Web documents or Web pages) that are accessible via the Internet, and (ii) the client and server software components which provide user access to such documents using standardized Internet protocols. Currently, the primary standard protocol for allowing applications to locate and acquire Web documents is HTTP, and the Web pages are encoded using HTML. However, the terms "Web" and "World Wide Web" are intended to encompass future markup languages and transport protocols which may be used in place of (or in addition to) HTML and HTTP.
As used herein, the term "web site" refers to a computer system that serves informational content over a network using the standard protocols of the World Wide Web. Typically, a Web site corresponds to a particular Internet domain name, such as "activefilmlayer.com" and includes the content associated with a particular organization. As used herein, the term is generally intended to encompass both (i) the hardware/software server components that serve the informational content over the network, and (ii) the "back end" hardware/software components, including any non-standard or specialized components, that interact with the server components to perform services for Web site users.
As used herein, the term "HTML" refers to HyperText Markup Language which is a standard coding convention and set of codes for attaching presentation and linking attributes to informational content within documents. During a document authoring stage, the HTML codes (referred to as "tags") are embedded within the infoπnational content of the document. When the Web document (or HTML document) is subsequently transferred from a Web server to a browser, the codes are interpreted by the browser and used to parse and display the document. Additionally, in specifying how the Web browser is to display the document, HTML tags can be used to create links to other Web documents (commonly referred to as "hyperlinks").
As used herein, the term "HTTP" refers to HyperText Transport Protocol which is the standard World Wide Web client-server protocol used for the exchange of information (such as HTML documents, and client requests for such documents) between a browser and a Web server. HTTP includes a number of different types of messages which can be sent from the client to the server to request different types of server actions. For example, a "GET" message, which has the format GET, causes the server to return the document or file located at the specified URL.
As used herein, the term "URL" refers to Uniform Resource Locator which is a unique address which fully specifies the location of a file or other resource on the Internet. The general format of a URL is protocol://machine address :port/path/filename. The port specification is optional, and if none is entered by the user, the browser defaults to the standard port for whatever service is specified as the protocol. For example, if HTTP is specified as the protocol, the browser will use the HTTP default port of 80. As used herein, the term "cookies" refers to a technology that enables a Web server to retrieve information from a user's computer that reveals information about the user. The informational item stored on the user's computer (typically on the hard drive) is commonly referred to as a "cookie." Many standard Web browsers support the use of cookies.
As used herein, the term "PUSH technology" refers to an information dissemination technology used to send data to users over a network. In contrast to the World Wide Web (a "pull" technology), in which the client browser must request a Web page before it is sent, PUSH protocols send the informational content to the user computer automatically, typically based on information pre-specified by the user.
I. GENERAL DESCRIPTION OF THE INVENTION
Analog video is rapidly being replaced by digitized video. Digitized video is prepared by digitally encoding an analog video, such as a film, or by analog sampling through a digital video camera followed by encoding. In most cases, the digitized video is compressed into a suitable format (e.g., MPEG-1, MPEG-2, DV, M-JPEG, etc.). Compression allows a great savings in memory needed to store the digitized video and bandwidth needed to transmit the digitized video. MPEG-2 is the most common compression format and is being used for both DVD (Digital Video Disc or Digital Versatile Disc) and digital TV (DTV) applications.
DVD is a next generation optical disc storage technology. While a standard CD holds 680 megabytes of information, a single sided, dual layer DVD holds 8.5 gigabytes of information and double sided, dual layer DVD can hold up to 17 gigabytes of information. The popularity of the DVD format is rapidly increasing. In 1997, there were only 347,000 DVD-video players shipped in the United States and only 900 DVD- video titles available. However, as of March 2000, there were 6.2 million DVD players shipped and 6,300 titles available in the United States.
Digital TV is also rapidly becoming an everyday reality in many households. On November 1, 1998, 45 broadcasters in the top 10 markets began transmission. Furthermore, the FCC is requiring that all television broadcasts be digital by 2006.
Digitized video provides a quality picture and easy editing. However, providers are not using digitized video to provide enhanced information content. This is in contrast to the development many media available on the Internet. For example, pictures and text presented on Internet Web pages are commonly selectable (e.g., through the hypertext protocol) so that links to other Web sites and product or other information can be provided. Digitized video is not currently amenable to such uses without the use of icons and/or graphical interfaces (See, e.g., U.S. Pat. Nos. 5,963,203 and 5,978,855, each of which is incorporated herein by reference). In contrast, the present invention allows the direct selection of a moving object in a running or streaming video without the use of icons. This allows the viewer to select objects while watching the running or streaming video without referring to a graphical interface or icon.
Accordingly, the present invention relates to methods and systems for selecting and providing information on objects in a video. The present invention contemplates the integration of an active film layer with digitized video. The active film layer allows a viewer to select objects within the video presentation, and then provides information about the objects. For example, the present invention allows production of a video of a car being driven down the road with an active film layer that allows a viewer of the video to select the car. Upon selection of the car, information is displayed. Examples of such information include, but are not limited to a bitmap image of the car's manufacturer and URL of the manufacturers Web site. In preferred embodiments, the URL is a hypertext link to the manufacturers Web site. The video may be digitized and/or compressed in a variety of formats. Examples of these formats are described in the following sections: A) Digitization and Compression; B) Digital Video Disc; C) Digital Television; D) Digital Video; and E) Internet formats.
A) Digitization and Compression
The active layer of the present invention is compatible with a wide variety of digitization and compression formats. Conversion of analog video to digital video creates files of enormous size. In order to make file size more manageable, a variety of compression formats have been developed. The present invention is compatible with any of the various formats. Therefore, the present invention is not limited to particular systems and methods for digitizing and compressing video. Indeed, the present invention contemplates that a variety of methods and systems may be used to digitize and/or compress the video. Analog video may be digitized by any method known in the art. For example, analog video (e.g., from VHS, Hi8, BetaCamSP, Dl, studio master tapes, or other studio/broadcast equipment) is converted to digitized video by a computer configured to digitize analog video signals. Typically, the computer comprises a video capture card (e.g., from Media 100, Avid, or Pinnacle) which converts the analog video signal into a digital file and a RAID or fast/wide/Ultra SCSI card. In preferred embodiments, the video is captured at full screen resolution (e.g., 640x480 or 720x486) from a master tape. Audio is captured along with the video (e.g., at 44 liHz, 16 bit, stereo).
After capture, the digital video file is compressed utilizing a codec corresponding to the desired compression format. Examples of suitable codecs include, but are not limited to, those provided by REALNETWORKS (e.g., REALVIDEO G2 with SVT), CINEPAK, SORENSON VIDEO, MEDIA 100, AVID, TRUEVISION, EIDOS, INDEO, and HEURIS. The present invention contemplates the use of a variety of compression formats, including MPEG-1, MPEG-2, MPEG-4, QuickTime, and DV.
In particularly preferred embodiments, the video is compressed in the MPEG-2 format. MPEG-2 is an international standard for encoding digital video information, and is formally designated ISO/IEC 13818. MPEG-2 files are system independent, unlike system specific formats such as AVI and QUICKTIME. Therefore, it is contemplated that many different hardware and software players are able to play MPEG-2 files. At the full screen resolution of 720x480 pixels for NTSC, MPEG-2 is capable of providing broadcast quality video. MPEG-2 is the specified standard for broadcast digital TV, satellite digital TV, DVD, cable networks, and video games. MPEG-2 also supports CD quality audio, storing two channels of 16-bit audio sampled at 32 kHz, 44.1 kHz, or 48 kHz.
The compressed digitized video can be delivered by a variety of means. In some embodiments, the digitized video is uploaded to a client via HTTP or FTP from a Web server and played with a suitable player. In preferred embodiments, a progressive download or video streaming is used to deliver the digitized video. In other embodiments, the digitized video may be provided via digital TV broadcast, digital satellite TV, CD, CD-R (recordable CD-ROM), DVD-video, and DVD-R (recordable DVD-ROM). These formats are described in more detail below. B) Digital Video Disc
The active layer of the present invention is compatible with digitized video stored on various DVD formats. Therefore, the present invention contemplates the delivery of digitized video by DVD application formats including, but not limited to, DVD-Video, DVD-Video Recording, and DVD-Stream Recording, and DVD physical formats including, but not limited to, DVD-ROM, DVD-R, DVD-RAM, DVD-RW, and DVD+RW. In preferred embodiments, DVD video is encoded from digital master tapes to MPEG-2 format. Audio may be encoded in a variety of formats, including, but not limited to, MPEG-2 audio, Linear PCM, DOLBY DIGITAL (i.e., AC-3 coding), Sony Dynamic Digital Sound, and DTS Digital Surround.
Digitized video stored on DVD may be displayed by a variety of systems. DVD players are designed specifically for DVD-Video and include MPEG-2 decoders for decoding the digital video files so that they may be displayed. In preferred embodiments, the DVD player is incorporated in a system that is capable of accessing the Internet. A preferred DVD player that includes an integrated Web Browser is the NEO IDVD (www.neo.com). Alternatively, the DVD player may be configured with an Internet ready television system (e.g., Web TV) or a computer connected to the Internet. For DVD movies to be displayed on a computer, the computer operating system or playback software should support regional codes and be licensed to descramble copyprotected movies. Furthermore, if the computer has TV video out, it should support macrovision in order to play copy protected movies. The computer preferably further comprises either a software decoder of video card capable of decoding MPEG-2 video and Dolby Digital or MPEG-2 audio. Microsoft Windows 98 and Windows 2000 include DirectShow, which provides standardized support for DVD-Video and MPEG-2 playback. Suppliers of video decoder cards include, but are not limited to, Creative Technology (PC-DVD Encore Dxr2), Digital Connection (3Dfusion), IBM, Ravisent (Hardware Cinemaster), and Phillips Electronics (PCDV632). C) Digital Television
The active layer of the present invention is also compatible with digital television (DTV) formats. DTV transmits pictures and sounds as an on/off digital stream rather than as varying analog signals. Broadcasters in the top ten U.S. TV markets began digital broadcasts in 1998. DTV uses MPEG-2 compression, which allows one high- resolution, wide screen program or 4, standard resolution video programs to be multiplexed on a single channel. Different DTV systems exists. An ATSC system is currently utilized in the United States. Europe and Japan are currently moving towards the use of a DVB-T system.
Regardless of the transmission system, DTV programming may be provided and received by a variety of systems. Broadcast, cable, satellite and DVD may be used to supply the signal (e.g., over-the-air DTV broadcasts may be received with a UHF antenna; satellite services [e.g., EchoStar and DirecTV] also supply DTV signals). Reception and display of DTV programming uses a decoder. The decoder may be integral to the display device (e.g., HDTV or other TV containing an integral decoder), or be provided as set-top box (e.g., digital cable boxes). In preferred embodiments, the DTV system includes Internet access capability (e.g., Web TV).
D) Digital Video
It is also contemplated that the active layer of the present invention is compatible with digital video (DV) and other digital video systems such as Digitalδ (Sony), Digital- S (JVC) and DVCPRO50 (Panasonic). DV is available from a variety of companies (e.g., Panasonic, Sony, JVC, and Sharp) and uses a 6.35 metal evaporate tape to record digital video. The DV system samples video at the same rate as D-l, D-5, and Digital Betacam video. The sampled video is compressed with DV codec utilizing a Discreet Cosine Transform (DCT). The compressed DV video may be converted to other compression formats such as M-JPEG and MPEG-2.
E) Internet formats
The active layer of the present invention is also compatible with media for delivering video files over the Internet. The Internet, which started in the late 1960s, is a vast computer network consisting of many smaller networks that span the entire globe. The computers or networks of computers connected within the Internet, known as "hosts", allow public access to databases featuring information in nearly every field of expertise and are supported by entities ranging from universities and government to many commercial organizations.
The information on the Internet is made available to the public through "servers." A server is a system running on an Internet host for making available files or documents contained within that host. Such files are typically stored on magnetic storage devices, such as tape drives or fixed disks, local to the host. An Internet server may distribute information to any computer that requests the files on a host. The computer making such a request is known as the "client", which may be an Internet-connected workstation, bulletin board system or home personal computer (PC).
TCP/IP (Transmission Control Protocol/Internet Protocol) is one networking protocol that permits full use of the Internet. All computers on a TCP/IP network use unique ID codes. Therefore, each computer or host on the Internet is identified by a unique number code, known as the IP (Internet Protocol) number or address, and corresponding network and computer names. In the past, an Internet user gained access to its resources only by identifying the host computer and a path through directories within the host's storage to locate a requested file. Although various navigating tools have helped users to search resources on the Internet without knowing specific host addresses, these tools still require a substantial technical knowledge of the Internet.
The World-Wide Web (Web) is a method of accessing information on the Internet which allows a user to navigate the Internet resources intuitively, without IP addresses or other technical knowledge. The Web dispenses with command-line utilities which typically require a user to transmit sets of commands to communicate with an Internet server. Instead, the Web is made up of hundreds of thousands of interconnected "pages", or documents, which can be displayed on a computer monitor. The Web pages are provided by hosts running special servers. Software which runs these Web servers is relatively simple and is available on a wide range of computer platforms including PC's. Equally available is a form of client software, known as a Web "browser", which is used to display Web pages as well as traditional non- Web files on the client system. Today, the Internet hosts that provide Web servers are increasing at a rate of more than 300 per month, en route to becoming the preferred method of Internet commumcation. Created in 1991, the Web is based on the concept of "hypertext" and a transfer method known as "HTTP" (Hypertext Transfer Protocol). HTTP is designed to run primarily over TCP/IP and uses the standard Internet setup, where a server issues the data and a client displays or processes it. One format for information transfer is to create documents using Hypertext Markup Language (HTML). HTML pages are made up of standard text as well as formatting codes which indicate how the page should be displayed. The Web client, a browser, reads these codes in order to display the page.
Each Web page may contain pictures and sounds in addition to text. Hidden behind certain text, pictures or sounds are connections, known as "hypertext links" ("links"), to other pages within the same server or even on other computers within the Internet. For example, links may be visually displayed as words or phrases that may be underlined or displayed in a second color. Each link is directed to a web page by using a special name called a URL (Uniform Resource Locator). URLs enable a Web browser to go directly to any file held on any Web server. A user may also specify a known URL by writing it directly into the command line on a Web page to jump to another Web page.
The URL naming system consists of three parts: the transfer format, the host name of the machine that holds the file, and the path to the file. An example of a URL may be: http://www.college.univ.edu/Adir/Bdir/Cdir/page.html, where "http" represents the transfer protocol; a colon and two forward slashes (://) are used to separate the transfer format from the host name; "www.college.univ.edu" is the host name in which "www" denotes that the file being requested is a Web page; "/Adir/Bdir/Cdir" is a set of directory names in a tree structure, or a path, on the host machine; and "page.html" is the file name with an indication that the file is written in HTML.
A number of formats are available for streaming video files over the Internet. These formats include, but are not limited to, REALVIDEO (REALNETWORKS, utilizing M-PEG2 compression), REALSYSTEM 8, QUICKTIME, WINDOWS MEDIA, and VIVOACTIVE. It is also contemplated that increasing bandwidth will enable streaming of DVD-Video and other large video files over the Internet.
Video files can be placed in Web pages, for example, in two ways: by use of an anchor tag (<a href>) or an embed tag (<embed>). The anchor tag is used in the same manner as placing any other link within an HTML document. A non-limiting example of the use of an anchor tag is: <a lιref="video/activefilmlayer.avi">matrix</a>. This method allows the user to save the file to a desktop, launch a helper application, or load a new browser page. Streaming video uses an embed tag and requires and an appropriate plug- in for viewing the video. A non-limiting example of an HTML for an embedded video file is: <embed src+"/activefilmlayer/demo/96/32/stuff/matrix.mov" height+108 width+33% controller+false autoplay=true playeveryframe=false pluginspage="fehn.html" loop=palindrome>.
II. DETAILED DESCRIPTION OF THE INVENTION
The present invention provides methods and systems for providing information to users on objects in a video. In particular, the methods and systems allow selection of objects displayed in a video that is derived from a digitally encoded or digitized video file as described above. Upon selection of the object, information is provided. The information may be related to the object (e.g., product information about the object, infoπnation about the manufacturer or provider of the object, or information about an actor in the video) or be completely independent of the nature of the object. Furthermore, the information may be provided immediately or at a later time (e.g., e- mailed or placed in a bookmark file, favorite file, or some other file). The methods and systems for object selection and information display are discussed in more detail below. This description is directed to certain preferred embodiments and is not intended to limit the scope of the invention.
A. Structure of Active Film Layer Media
Referring to Figure 1, the present invention provides video media 100 comprising an Active Film Layer (AFL) archive 105 and, optionally, other content 110 (e.g., CD audio, DOLBY DIGITAL sound, etc.). The video media 100 may be provided by any computer readable memory suitable for storing digital video files (e.g., hard disk drives, CD, DVD, magnetic tape, etc.). The AFL archive 105 comprises a data file with at least two components, object information 115 and digital video stream 120. Preferably, the object information 115 and digital video stream 120 are contained in a single file. It is contemplated that the single file format has several advantages, including ease of encryption and prevention of removal of the object information 115. Therefore, in preferred embodiments, the AFL archive 105 is encrypted to prevent hacking or other manipulation of the AFL content.
In some embodiments of the present invention, as shown Figure 2, the AFL archive 200 is read in a two step procedure. First, the object information 205 is read by an object parser 210. It is contemplated that the object parser 210 reads the object information from the AFL archive 200 and initializes the AFL at startup, before the video (e.g., a movie 215 or television program) starts playing. Second, the player 220 reads the compressed movie 215 from the AFL archive and streams the movie frame by frame. The AFL detects any selection events (e.g., selection by a mouse over action or mouse click) and initiates the appropriate action (e.g., display of text containing information on the object or opening a Web browser to a site related or unrelated to the object).
1. The Object Information File
In some embodiments of the present invention, the object information file is "flat" textfile. It is contemplated that the object information file has a simple structure and syntax, and can be edited with text editor software, as well as generated by special programs. In preferred embodiments, an object is an active part of the display on a display device (e.g., cathode ray tube, LCD screen), and typically is an item displayed in the video. In preferred embodiments, the object information file contains information on a plurality of objects displayed in the video. Referring to Figure 3, in some embodiments it is contemplated that an object in a video display 300 can be described by three functions: it is born, it moves/changes in size, and it dies. In the first frame 305, object A 310 is born. In the second frame 315, object B 320 is born and object A 310 moves and changes in size. In the third frame 325, object C 330 is born, object A 310 remains stationary, and object B 320 moves. In the fourth frame 335, object B 320 dies, object A 310 remains stationary, and object C 330 moves. In the fifth frame 340, object A 310 remains stationary, and object C 330 dies. In the sixth fame 345, object A 310 dies.
The object information file includes information relating to each of the three functions. In preferred embodiments, the three functions are designated by keywords: "init", "key", and "kill". In preferred embodiments, the "init" and "kill" functions both include an implicit "key" function. In preferred embodiments, the object information file contains templates (e.g., classes relating to the properties of the objects) for the different objects that appear in the video, as well as information about where and when the objects appear (e.g., keyframes with polygons on screen that refer to a class). A non-limiting example of this type of file is as follows:
#File begin
#Classes, templates for the objects
Class Nokia { mouseoveraction = Display__logo(nokia.bmp); mouseclickaction = open_URL(http://www.nokia.com);
};
# . . . more classes
# . . .
# . . .
#Frames / Polygons
01:14:56:23 init (Nokia (x200, y345, xx230, yy432)):
01:15:02:03 key (Nokia (x220, y365, xx240, yy444));
01:15:04:24 kill (Nokia (x200, y345, xx230, yy432));
#more keyframes/polygons/objects
# . . .
# . . .
#File end
The first part of the file conatins the class information (i.e., the properties of the objects that are to appear during video playback). A hash (#) at the start of the line denotes a comment in the file (i.e., the entire line is ignored when the file is parsed during the initialization of the AFL video). Each class has a unique name that the objects will refer to (e.g., "Class Nokia"). Each class has its properties described between the parentheses { . . . }. The class in the example has two properties: 1) the action to be executed when the user designates or selects (e.g., holds the mouse pointer or cursor over) an object of the class; and 2) the action to be executed when the user executes another selection action (e.g., clicks the mouse button) on an object of the class. In this example, an image map (e.g., corporate logo) is superimposed on the video during playback. In the event of a selection action (e.g., a mouse click), a Web browser is started and the corporate web page is opened.
The second part of the file contains information about the time and space of the active objects (i.e., timecodes and coordinates). When the file is parsed, the keyframes are interpolated to create fluid movement of the objects. This example uses a simple geometric shape (i.e., a rectangle) to activate an area on the screen. The rectangle is represented by the coordinates of its upper left and lower right corners. Of course, more complex shapes (e.g., circles, ellipses, triangles, squares, pentagons, rhombuses, hexagons, octagons, or more complex shapes and combinations thereof) are within the scope of the present invention. The present invention is not intended to be limited to a particular mechanism of action. However, it is contemplated that the active object may be designated by a variety of polygon shapes or combination of individual polygon shapes. Indeed, the polygon may include 10, 100, 1000 or more sides, and in some cases approximate a tracing of the object. The shape can also be a freehand or abstract shape. Furthermore, it is contemplated that the polygon is invisible to the viewer of the video. The designation of the polygon shapes in the file correlates the selection device with the object in the video.
2. The Video File
Referring back to Figure 1, the digital video stream 120 may be uncompressed or preferably compressed. Software and hardware (i.e., codecs) for compression and decompression of the digital video stream 120 are well known in the art as described above. Preferred compression formats include, but are not limited to, MPEG-1, MPEG- 2, MPEG-4, DV, MOV, and M-JPEG.
B. Creating Active Film Layers
Active Film layers (AFL) can be created, for example, using standard video editing software. The timeline of typical video editing software (e.g., Adobe Premiere) contains a number of Video and Audio layers in addition to Transition and Effect layers. Transition layers determine which video layer is the actual video output, while effect layers are used for superimposing graphical objects like text and logos on top of video images. In preferred embodiments of the present invention, an AFL is added to the video timeline, making it possible to include additional data layers in the video output. In preferred embodiments, these layers are invisible to the viewer upon playback. In further preferred embodiments, the AFLs are readable by the software or hardware playing back or streaming the video.
In some embodiments of the present invention, it is contemplated that, to make part of the video image active, an AFL layer designated to that object is created. In some embodiments, the AFL is created from a menu in the same way that video and audio layers are created. In some embodiments, in the first frame where interactivity starts, a key frame is created in the AFL. In the monitor window, a shape (e.g., a polygonal shape) is drawn using basic drawing tools. Tins shape is associated with the video image until it is removed from the AFL. In preferred embodiments, additional keyframes are created in the AFL so that the shapes follow the object movement in the video layer. In preferred embodiments, the polygon shape is edited in each keyframe to compensate for movement in the video layer. In other preferred embodiments, blending techniques available in vector-based drawing programs (e.g., Adobe Illustrator, Corel Draw, and Macromedia Flash) are used to fill in the changes between keyframes.
In further preferred embodiments, an action is added to the object. Action objects are created in a separate window and are imported data objects (e.g., bitmap, vector, or audio files). Once imported, the action objects are assigned actions (e.g., a web link [URL]). In other preferred embodiments, the AFL object is completed by association with an action. In preferred embodiments, this is done by selecting the desired action from a dropdown menu in the monitor window. This process provides a single, complete AFL active object. The process may be repeated to provide a variety of additional AFL objects in new AFL layers. It is contemplated that there is no theoretical limit to the number of AFLs that can be created for a video file, however, an increasing number of AFLs increases the requirements for the playback system. The video file (e.g., movie) is then finished, with the desired video and audio formats being selected along with the desired level of compression, and the finished file rendered to disk or other suitable storage media.
C. Initializing and Playing AFL Media
After a video containing an AFL layer is produced, it may be played or provided through a variety of systems. Preferred systems for playing AFL media comprise a selection device and a display device. The selection device may be any type of selector or data input unit known in the art and its associated software, if any, including, but not limited to, a computer mouse, a keyboard, a joystick, a trackball, a touchscreen, a touchpad, a data input pen, and combinations and wireless versions thereof. Preferred display devices include any devices for displaying video known in the art, including but not limited to, cathode ray tubes and LCD screens. In preferred embodiments, the selection device is in communication with a central processor so that actions performed with the selection device (e.g., mouse over or mouse click) are relayed to the processor to cause a particular action (e.g., display of a logo or opening a Web browser to a Web page).
In further preferred embodiments, the system comprises a processor or processors configured to display AFL media. In some embodiments, the system also comprises a computer memory device for storing the AFL media so that it can be played. A variety of computer memory devices find use with the present invention, including hard disk drives, magnetic tape, DVD-ROM, DVD-R, DVD-RAM, DVD-RW, DVD+RW, and CDs. The computer memory device may be integral to the processor (i.e., form a part of the same machine) or be remote from the processor (e.g. , at a television station or transmitter in the case of broadcast digital television or a Web server for streaming video).
Accordingly, in some preferred embodiments, the system comprises a computer (e.g., a PC or any other device containing a central processing unit) configured to display AFL media. Preferably, the computer comprises at least one central processing unit (processor), a random access memory (RAM), a read only memory (ROM), a clock, and a computer memory device. In further preferred embodiments, the computer comprises a display device, a selection device, a codec, and a communications port. The processor can be connected to each of these elements by a shared data bus or dedicated connections. The communications port can connect the computer to other computers in a network such as the Internet. The ROM and/or computer memory device are operable to store one or more instructions (discussed further below), which the processor is operable to retrieve, interpret, and execute. For example, ROM and/or computer memory device can store AFL media (e.g., stored permanently or temporarily thereon) and a player for the AFL media. Furthermore, the ROM and/or computer memory device store processes to accomplish, among other things, initialization of the AFL player (described below), decompression and playing of the video through the display device, and selection of objects in the video by the selection device. In particularly preferred systems, the AFL media is provided in a computer memory device selected from DVD-ROM, DVD-R, DVD-RAM, DVD-RW, DVD+RW, and CDs.
In further preferred embodiments, the computer is in communication through its communications port with other computers via a computer network (e.g., Internet, intranet, or LAN). Preferably, the processor is configured so that data, including AFL media, cookies, and other client identification information can be sent between computers. Furthermore, it is contemplated that the processor is configured so that selection of an object imaged on a video (e.g., an object that has been "activated" through an AFL) initiates communication with other computers or servers via a Web browser or other software or hardware.
In some preferred embodiments, the computer is a Web TV. It is contemplated that the AFL media can be displayed on the Web TV by a variety of methods, including but not limited to, download of the AFL media from a server via the Internet (e.g., streaming video containing an AFL), reception of satellite or broadcast DTV on Web TVs equipped with a codec (e.g., an integral codec or set-top codec), and connection to a DVD player via video input connectors. In other preferred embodiments, the computer is a DVD player with Internet capability (e.g., NEO IDVD, www.neo.com). It is contemplated that this system can be used to play AFL media stored on DVD-ROM (e.g., DVD-video), or AFL media downloaded or streamed from a server or other source over the Internet.
A schematic overview of an exemplary system is provided in Figure 4. In this exemplary system, a computer 400 is provided. As described above, the present invention is not limited to any particular computer 400. Indeed, a variety of computers 400 are contemplated, including, but not limited to, personal computers, Web TVs, TVs equipped to receive DTV, DVD players, and any other device containing a processor. In preferred embodiments, the computer further comprises a web browser 405 (e.g., NetScape or Internet Explorer), computer memory including cookies 410, and computer memory comprising an active film layer file 415. As described above, the present invention is not limited to any particular media for providing the active film layer file 415. Indeed, a variety of active film layer media are contemplated, including, but not limited to, DVD-ROM (e.g., DVD-video), DVD-R+W, CD-ROM, hard disk, streaming from an internet web server or other source, and broadcast or satellite DTV. In preferred embodiments, the computer 400, is in communication with a display device 420 so that the video in the active film layer file 415 can be displayed to a viewer. In further preferred embodiments, the computer 400 is also in communication with a selection device 425 so that objects (preferably moving objects) in the video encoded in the active layer file can be selected.
In some embodiments, the computer 400 is in communication (e.g., for example, through the Internet) with a Web site 430. In some preferred embodiments, the Web site 430 comprises a Web server 435, computer memory 440 comprising HTML documents (e.g., product pages, shopping cart pages, and product order pages), computer memory containing various databases, such as a shopping cart database 445, a customer database 450, and a referral database 455, and computer programs 460 (e.g., cookie processing software 465, shopping cart software 470, and customer database software 475). In some embodiments, selection of an active object in the video displayed from the active film layer file causes the computer 400 to open the web browser 405 to a URL specified in the active film layer file 415. The Web browser 405 then accesses the Web site 430 identified by the URL. The user can then examine, for instance, product web pages, place orders in the product order page, and examine the total order in the shopping cart page. In some embodiments, information about the user is provided to the web server 435 from the computer memory including cookies 410 and processed by the cookie processing software 465. In other embodiments, information about the customer (e.g., name, electronic mail address, purchase preferences, etc.) is stored in the customer data base 450. In further embodiments, the information is used to develop electronic mail lists via the customer database software 475 for future marketing. In other embodiments, information concerning the source of the referral to the Web site (e.g., that the referral was through active film layer media) is stored in the referral database 455.
As indicated above, the system preferably comprises a ROM and/or other computer memory device storing processes for displaying AFL media. It is contemplated that in some embodiments, the software for displaying the video from the AFL media is divided into two components, the object parser and the player. In some embodiments of the present invention, it is contemplated that the object parser preprocesses the object information before the video (e.g., a movie) starts playing. The object parser reads all classes and generates templates for the objects. The object parser then reads the timecode/polygon information and creates an object map. This map is then used for reference during playback of the video so that the presence, coordinates and properties of active objects may be determined. The operation of the object parser is illustrated in Figure 5. In step 500, an AFL archive is opened. In step 505, lines within the archive are read and classes are generated in random access memory. In step 510, a query is performed as to whether more classes are designated. If yes, step 505 is repeated and the additional class is generated in random access memory. This subroutine is repeated until all classes are read. If the answer to the query step 510 is no, step 515 is performed and the next line is read and a timechart created 515. In step 520, a query is performed as to whether more keyframes are present. If yes, the subroutine is repeated. In step 525, the information is provided to the player for use during playback.
In some preferred embodiments, the player also comprises two modules: the video streamer (e.g., codec) and the AFL. The video streamer reads the video file and shows the video frame by frame, as well as playing the sound and any additional content. The AFL reads the object map generated by the preprocessor to capture any interaction by the user (e.g., mouse over or mouse click) and initiate the appropriate action. Operation of the player is illustrated in Figure 6. In step 600, the video file is opened. In step 605, a frame the video file is read to memory. A query is performed in step 610 to determine whether the end of the file has been reached. If yes, step 615 is executed and the player quits. If no, step 620 is executed and the video streamer gets a video frame and displays it in step 625. In step 630, the AFL subroutine is initiated to get the timecode and object information. A query is performed in step 635 to determine whether any active objects are present in the frame (i.e., displayed on screen). If no, step 640 is performed to advance one frame and the subroutine is repeated. If yes, a second query is performed in step 645 to determine whether a mouse pointer is inside the object. If no, step 640 is performed to advance the video one frame and the subroutine repeated. If yes, step 650 is executed and the mouse over action performed. In step 655, another query is performed as to whether the mouse button is clicked. If no, step 640 is performed to advance one frame and the subroutine is repeated. If yes, an update is sent to a central server in step 660. Next, in step 665, the mouse click action is executed. Step 640 is then performed to advance one frame and the routine is repeated. The present invention is not limited by the number of active objects within a video frame or frames nor by the device used to select the active object or action initiated upon selection of the object. D. Uses of AFL Media
It is contemplated that AFL media finds use both on its own and as a part of a variety of methods. Any and all uses of the AFL media and systems are contemplated, and a number of illustrative examples are described below.
As described above, an AFL system may be used to select active objects imaged in a video stream. Upon selection of an active object, the AFL system is configured to cause an action to be performed (e.g. , information related or unrelated to the active object is provided). In preferred embodiments, information relating to the active object is provided from the object information. It is contemplated that different types of information may be provided for active objects. In some embodiments, text identifying the active object is provided. Such text can include information such as the manufacturer of the object, suppliers of the object, history of the object, the model number of the object, etc. In other embodiments, a bitmap image relating to the object is displayed on the display device. The bitmap image can be the logo of a supplier or manufacturer of the object, and can also include information on the object such as the model number, etc. In some embodiments, an additional selection action (e.g., a mouse click) can initiate printing of the information by a printing device in communication with the system.
Accordingly, it is contemplated that AFL media and systems are used in a business method to generate product placement revenue. In preferred embodiments, a manufacturer or supplier of a product displayed in a video (e.g., a movie or television) program pays a product identification fee to have AFL layers created so that product (i.e., object) can be made active in a digital video format. It is contemplated that these product identification fees can reduce the cost (by recovering costs through fees) of movies and other programming distributed on digital media, including, but not limited to, DVDs, laser discs, and network TV, cable TV, and pay-per-view programming distributed by over the air broadcast, satellite broadcast, and cable. For example, in a digitized movie, AFL layers are created so that information on various items used by an actor (e.g. , cars, cellular phones, clothing, weapons, cigarettes, make up, hair products, and branded food such as chips, soda, or alcoholic beverages) are provided. Alternatively, information on the actor(s), director(s), and others can be provided.
In some preferred embodiments, it is contemplated that the information provided is a URL for a Web site. In further embodiments, an action by the user (e.g., a mouse click) on the active object causes a Web browser to open and access the Web page addressed by the URL. In preferred embodiments, the Web browser window is displayed on a split screen or in a corner of the display device so that viewing of the video (e.g., movie or television) is not disrupted. In other embodiments, selection of an object whose object information comprises a URL causes the URL to be saved in a favorites list in a Web browser or other application so that the user can access the Web page addressed by the URL after the completion of the video (e.g., movie or television program), or during an intermission or commercial. In still further embodiments, selection causes a pause in the video. In other embodiments, the Web browser is already opened and may preferably be minimalized. Selection of an object comprising a URL in its object information then either maximizes the Browser window and causes access of the desired Web page or stores the URL in a favorites file as described above. The present invention also contemplates the integration of one click shopping for the display item (e.g., clicking on the object results in its purchase).
In some preferred embodiments, The Web browser configured with the AFL system comprises a cookie module which stores an identifier code for the AFL system. A cookie is a file that stores information that can be utilized by a Web site. The specification for cookies is defined by Netscape corporation and can be found at http://www.netscape.com/newsref/std/cookie_spec.html. In some embodiments, it is contemplated that the cookie allows identification of visits to Web sites (e.g. , product supplier or producer Web sites) as being generated by links (e.g., URLs) provided by AFL media. Those skilled in the art will realize that other methods exist for determining whether a visit to a particular Web site is initiated from AFL media. Such methods, include, but are not limited to, encoding origination information (e.g., an AFL identifier) directly in the URL provided by the object information.
It is also contemplated that the AFL media finds use for both increasing the information content of video media such as movies and TV programming and allowing interactivity with such programming. In some preferred embodiments of the present invention, selection of an actor provides fan club information, information on other movies the actor is in, or biographical information. In other preferred embodiments, it is contemplated that an interactive video (e.g., a movie) is provided, wherein a user selects an active object (e.g., an actor) and the information provided is an alternative scene from the viewpoint of the active object or zooming in on the active object. In other embodiments, selection of active object leads to a different plot being played out. In other embodiments of the present invention, AFL media is used to enhance educational programming. For example, in some embodiments of the present invention, an interactive childrens program teaching math, spelling, or science is provided where a question is posed and various answers or objects displayed on the screen. A child watching the program then attempts to select the correct answer from among several choices, each of which is an active object. In preferred embodiments, the processor is configured to tabulate, display, and store the results so that they can be accessed by the user at any time or compared over time to monitor improvement. In other embodiments, a wrong answers initiates a tutorial video or graphic. As another non-limiting example, AFL media comprising a video on gardening is provided. In preferred embodiments, active objects in the video include, but are not limited to, plants, tools, and site information. Selection of the plants, tools, or sites provides information on the plants, sources from which the plants are available, URLs of Web sites providing information on the plants, model numbers for tools, suppliers of tools, URLs of Web sites selling the tools, soil data, sun exposure data, and watering data.
In other embodiments of the present invention, AFL media is used to enhance game show programming. For example, in some embodiments of the present invention, the viewing audience participates in quiz show type programming by answering questions as they are posed. In some embodiments, the answers are provided as active objects and the user selects the correct answer from among several choices. In some embodiments, the answers are communicated via the Internet to a Web site configured to receive the results. In some preferred embodiments, the viewer has previously created an account with the Web site into which the data is directed. In further embodiments, correctly answering all or a subset of the questions makes the user eligible for prizes, discounts, or rebates on products.
In other embodiments, the AFL media is used to create games within digitized videos (e.g., TV programs, movies, or commercials). For example, in some embodiments it is contemplated that a collection of objects in a TV program or commercial are made active. The viewer then searches for and selects the objects during the program or commercial, and the results of the selection are communicated to a Web site. In some preferred embodiments, the viewer has previously created an account with the Web site into which the data is directed. Selection of all or a subset of the active objects then makes the viewer eligible for a prize, prizes, or discounts or rebates on products.
All publications and patents mentioned in the above specification are herein incorporated by reference. Various modifications and variations of the described method and system of the invention will be apparent to those skilled in the art without departing from the scope and spirit of the invention. Although the invention has been described in connection with specific preferred embodiments, it should be understood that the invention as claimed should not be unduly limited to such specific embodiments. Indeed, various modifications of the described modes for carrying out the invention which are obvious to those skilled in relevant fields are intended to be within the scope of the following claims.

Claims

CLAIMSWe claim:
1. A system for providing information on an object in a digitized video comprising: a digitized video of at least one object, a computer processor, a selection device, a display unit, and an active layer, wherein said processor is configured so that said digitized video of at least one object is displayed on said display unit and wherein said active layer implements selection of said at least one object with said selection device.
2. The system of Claim 1, wherein said digitized video is encoded by a compression format selected from the group consisting of MPEG- 1, MPEG-2, DV, and M-JPEG.
3. The system of Claim 1, wherein said display unit is selected from the group consisting of a cathode ray tube and a liquid crystal diode tube.
4. The system of Claim 1, wherein said selection device is selected from the group consisting of a computer mouse, a computer keyboard, a joystick, trackball, and a touchpad.
5. The system of Claim 1, further comprising a public network for communication between computers coupled to said processor.
6. The system of Claim 1, wherein said digitized video is supplied by broadcast digital television.
7. The system of Claim 1, wherein said digitized video is supplied by DVD.
8. The system of Claim 1, wherein said digitized video is supplied by a media player selected from QuickTime, Windows Media, and RealVideo.
9. A system for providing information on an object in a digitized video comprising: a digitized video encoding pictures of at least one object, a computer processor, and a selection device, wherein said processor is configured so that said at least one object is selectable by said selection device.
10. The system of Claim 9, wherein said digitized video is encoded by a compression format selected from the group consisting of MPEG- 1, MPEG-2, DV, and M-JPEG.
11. The system of Claim 9, wherein said selection device is selected from the group consisting of a computer mouse, a computer keyboard, a joystick, trackball, and a touchpad.
12. The system of Claim 9, further comprising a public network for communication between computers coupled to said processor.
13. A computer readable medium comprising a plurality of instructions, which when executed by a computer cause said computer to perform the steps of: a) displaying a digitized video of at least one object; and b) implementing selection of said at least one object.
14. The computer readable medium of Claim 13, wherein said digitized video is encoded by an compression format selected from the group consisting of MPEG- 1, MPEG-2, DV, and M-JPEG.
15. The computer readable medium of Claim 14, wherein said digitized video further comprises an active layer.
16. The computer readable medium of Claim 14, wherein said medium is selected from the group consisting of DVD, CD, magnetic tape, and hard disk drives.
17. A computer readable medium comprising a plurality of instructions, which when executed by a computer cause said computer to perform the steps of: a) displaying a digitized video of at least one object; b) implementing selection of said at least one object; and c) providing information on said at least one object following selection.
18. The computer readable medium of Claim 17, wherein said digitized video is encoded by an compression format selected from the group consisting of MPEG- 1, MPEG-2, DV, and M-JPEG.
19. The computer readable medium of Claim 18, wherein said digitized video further comprises an active layer.
20. The computer readable medium of Claim 17, wherein said medium is selected from the group consisting of DVD, CD, magnetic tape, and hard disk drives.
21. A method comprising: a) providing: i) a digitized video of at least one moving object; and ii) a selection device; b) displaying said digitized video; and c) selecting said at least one object with said selection device.
22. The method of Claim 21, further comprising step d) displaying information on said at least one object.
23. Computer readable medium comprising: a digitized video file of at least one moving object; and an object information file comprising data on said moving object.
24. The computer readable medium of Claim 23, wherein said video file is compressed in a format selected from the group consisting of MPEG- 1, MPEG-2, M- JPEG, and DV.
25. The computer readable medium of Claim 23, wherein said data on said moving object allows selection of said moving object.
26. The computer readable medium of Claim 23, wherein said data on said moving object is object information.
27. The computer readable medium of Claim 23, wherein said medium is DVD.
28. A method for providing information on objects pictured in digitized video comprising: a) providing: i) a digitized video of at least one moving object; ii) a selection device; and iii) a processor configured to allow selection of said at least one moving object; b) via said processor, implementing selection of said at least one moving object with said selection device to provide a selected object; and c) providing information on said selected object.
29. The method of Claim 28, wherein said digitized video is encoded by an compression format selected from the group consisting of MPEG-2, DV, and M-JPEG.
30. The method of Claim 29, wherein said digitized video further comprises an active layer.
31. The method of Claim 28, wherein said information is product information.
32. The method of Claim 28, wherein said information is a hypertext link.
33. The method of Claim 32, wherein said hypertext linlc is displayed after clicking on said at least one object with a computer mouse.
34. The method of Claim 28, further comprising providing a public network for communication between computers and wherein said information in step (c) is provided via a hypertext link to computer servers in said public network.
PCT/US2001/017793 2000-05-31 2001-05-31 Active layer for digital video WO2001093592A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2001265307A AU2001265307A1 (en) 2000-05-31 2001-05-31 Active layer for digital video

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US58333500A 2000-05-31 2000-05-31
US58445600A 2000-05-31 2000-05-31
US09/584,456 2000-05-31
US09/583,335 2000-05-31

Publications (1)

Publication Number Publication Date
WO2001093592A1 true WO2001093592A1 (en) 2001-12-06

Family

ID=27078783

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2001/017793 WO2001093592A1 (en) 2000-05-31 2001-05-31 Active layer for digital video

Country Status (2)

Country Link
AU (1) AU2001265307A1 (en)
WO (1) WO2001093592A1 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008125794A1 (en) * 2007-04-14 2008-10-23 Carl Livesey Product information display and purchasing
WO2009027110A1 (en) * 2007-08-28 2009-03-05 Sony Ericsson Mobile Communications Ab Methods, devices, and computer program products for providing unobtrusive video advertising content
WO2010047632A1 (en) * 2008-10-24 2010-04-29 Brixgate Communication Ab Advertising control system and method for motion media content
US9071730B2 (en) 2007-04-14 2015-06-30 Viap Limited Product information display and purchasing

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0596823A2 (en) * 1992-11-02 1994-05-11 International Business Machines Corporation Method and system for accessing associated data sets in a multimedia environment in a data processing system
WO1997041690A1 (en) * 1996-05-02 1997-11-06 Award Software International, Inc. Tv with url reception for internet access
US5708845A (en) * 1995-09-29 1998-01-13 Wistendahl; Douglass A. System for mapping hot spots in media content for interactive digital media program
WO1998057499A1 (en) * 1997-06-09 1998-12-17 Sharp Kabushiki Kaisha System for the transmission of audio, video and auxiliary data
EP0982947A2 (en) * 1998-08-24 2000-03-01 Sharp Kabushiki Kaisha Audio video encoding system with enhanced functionality

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0596823A2 (en) * 1992-11-02 1994-05-11 International Business Machines Corporation Method and system for accessing associated data sets in a multimedia environment in a data processing system
US5708845A (en) * 1995-09-29 1998-01-13 Wistendahl; Douglass A. System for mapping hot spots in media content for interactive digital media program
WO1997041690A1 (en) * 1996-05-02 1997-11-06 Award Software International, Inc. Tv with url reception for internet access
WO1998057499A1 (en) * 1997-06-09 1998-12-17 Sharp Kabushiki Kaisha System for the transmission of audio, video and auxiliary data
EP0982947A2 (en) * 1998-08-24 2000-03-01 Sharp Kabushiki Kaisha Audio video encoding system with enhanced functionality

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008125794A1 (en) * 2007-04-14 2008-10-23 Carl Livesey Product information display and purchasing
JP2010524110A (en) * 2007-04-14 2010-07-15 リヴゼイ カール Display and purchase of product information
US9071730B2 (en) 2007-04-14 2015-06-30 Viap Limited Product information display and purchasing
US9973818B2 (en) 2007-04-14 2018-05-15 Viap Limited Product information display and purchasing
WO2009027110A1 (en) * 2007-08-28 2009-03-05 Sony Ericsson Mobile Communications Ab Methods, devices, and computer program products for providing unobtrusive video advertising content
US7987478B2 (en) 2007-08-28 2011-07-26 Sony Ericsson Mobile Communications Ab Methods, devices, and computer program products for providing unobtrusive video advertising content
WO2010047632A1 (en) * 2008-10-24 2010-04-29 Brixgate Communication Ab Advertising control system and method for motion media content

Also Published As

Publication number Publication date
AU2001265307A1 (en) 2001-12-11

Similar Documents

Publication Publication Date Title
EP0902928B1 (en) System for using media content in interactive digital media program
US6496981B1 (en) System for converting media content for interactive TV use
US9955222B2 (en) Method and system for content delivery
US9445133B2 (en) DVD conversion for on demand
US8479246B2 (en) System and method for interactive video content programming
Mauthe et al. Professional content management systems: handling digital media assets
EP1547376B1 (en) Dynamic interactive content system
CN101523911B (en) Method and apparatus for downloading ancillary program data to dvr
CN1240216C (en) Video playback device with real-time on-line viewer feedback capability and method of operation
CN102696035B (en) For optionally limiting system, method and the program application of playing of changing places of copy-protected digital media content
US20100043046A1 (en) Internet video receiver
US20030145338A1 (en) System and process for incorporating, retrieving and displaying an enhanced flash movie
Pea et al. Video workflow in the learning sciences: Prospects of emerging technologies for augmenting work practices
JP2004517532A (en) Embedding object-based product information in audiovisual programs that is reusable for non-intrusive and viewer-centric use
US20090249427A1 (en) System, method and computer program product for interacting with unaltered media
US20030231861A1 (en) System and method for playing content information using an interactive disc player
WO2004073309A1 (en) Stream output device and information providing device
KR20110056476A (en) Multimedia distribution and playback systems and methods using enhanced metadata structures
MXPA05014045A (en) Method and apparatus for processing a video signal, method for playback of a recorded video signal and method of providing an advertising service.
JP2000069442A (en) Moving picture system
US7853617B2 (en) Method and system to enable dynamic modification of metadata in content
US9282370B2 (en) Open API digital video recorder and method of making and using same
WO2001093592A1 (en) Active layer for digital video
JP2005173781A (en) Dynamic change method of link information to provided content, user terminal, and intermediary server
CN101516024B (en) Information providing device,stream output device and method

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A1

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG US UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A1

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP