WO2006023770A2 - Methods and apparatus for generating signatures - Google Patents

Methods and apparatus for generating signatures Download PDF

Info

Publication number
WO2006023770A2
WO2006023770A2 PCT/US2005/029623 US2005029623W WO2006023770A2 WO 2006023770 A2 WO2006023770 A2 WO 2006023770A2 US 2005029623 W US2005029623 W US 2005029623W WO 2006023770 A2 WO2006023770 A2 WO 2006023770A2
Authority
WO
WIPO (PCT)
Prior art keywords
media samples
signature
frame
descriptor
media
Prior art date
Application number
PCT/US2005/029623
Other languages
French (fr)
Other versions
WO2006023770A3 (en
Inventor
Venugopal Srinivasan
Original Assignee
Nielsen Media Research, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Family has litigation
First worldwide family litigation filed litigation Critical https://patents.darts-ip.com/?family=35968228&utm_source=google_patent&utm_medium=platform_link&utm_campaign=public_patent_search&patent=WO2006023770(A2) "Global patent litigation dataset” by Darts-ip is licensed under a Creative Commons Attribution 4.0 International License.
Application filed by Nielsen Media Research, Inc. filed Critical Nielsen Media Research, Inc.
Priority to MX2007002071A priority Critical patent/MX2007002071A/en
Publication of WO2006023770A2 publication Critical patent/WO2006023770A2/en
Priority to US11/676,452 priority patent/US7783889B2/en
Publication of WO2006023770A3 publication Critical patent/WO2006023770A3/en
Priority to US12/822,777 priority patent/US8489884B2/en

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/16Analogue secrecy systems; Analogue subscription systems
    • H04N7/173Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3247Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving digital signatures
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44213Monitoring of end-user related data
    • H04N21/44222Analytics of user selections, e.g. selection of programs or purchase activity
    • H04N21/44224Monitoring of user activity on external systems, e.g. Internet browsing
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H04N21/44236Monitoring of piracy processes or activities
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/45Management operations performed by the client for facilitating the reception of or the interaction with the content or administrating data related to the end-user or to the client device itself, e.g. learning user preferences for recommending movies, resolving scheduling conflicts
    • H04N21/466Learning process for intelligent management, e.g. learning user preferences for recommending movies
    • H04N21/4667Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/835Generation of protective data, e.g. certificates
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/84Generation or processing of descriptive data, e.g. content descriptors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2209/00Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
    • H04L2209/60Digital content management, e.g. content distribution
    • H04L2209/601Broadcast encryption
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2209/00Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
    • H04L2209/80Wireless

Definitions

  • the present disclosure relates generally to media monitoring and, more particularly, to methods and apparatus for generating signatures for use in identifying media information.
  • Identifying media information and more specifically audio streams (e.g., audio information) using signature-matching techniques is well known.
  • Known signature-matching techniques are often used in television and radio audience metering applications and are implemented using several known methods for generating and matching signatures.
  • signatures are generated at monitoring sites (e.g., monitored households) and reference sites. Monitoring sites typically include locations such as, for example, households where the media consumption of audience members is monitored.
  • monitoring sites typically include locations such as, for example, households where the media consumption of audience members is monitored.
  • monitored signatures may be generated based on audio streams associated with a selected channel, radio station, etc. The monitored signatures may then be sent to a central data collection facility for analysis.
  • reference signatures are generated based on known programs that are provided within a broadcast region.
  • the reference signatures may be stored at the reference site and/or a central data collection facility and compared with monitored signatures generated at monitoring sites.
  • a monitored signature may be found to match with a reference signature and the known program corresponding to the matching reference signature may be identified as the program that was presented at the monitoring site.
  • FIGS. IA and IB illustrate example audio stream identification systems for generating digital spectral signatures and identifying audio streams.
  • FIG. 3 is a time-domain representation of an example reference audio stream and a plurality of audio sample frames acquired from the example reference audio stream.
  • FIG. 4 is a flow diagram of an example method for generating digital spectral signatures based on spectral decompositions.
  • FIG. 5 is a flow diagram of an example method for generating descriptors associated with the example method of FIG. 4.
  • FIG. 6 is a flow diagram of another example method for generating digital spectral signatures based on spectral decompositions.
  • FIG. 7 is a flow diagram of an example method for generating descriptors associated with the example method of FIG. 6.
  • FIG. 8 is a flow diagram of an example method for comparing the digital spectral signatures generated using the example methods of FIGS. 4-7.
  • FIG. 9 is a block diagram of an example signature generation system for generating digital spectral signatures based on audio streams.
  • FIG. 10 is a block diagram of another example signature generation system for generating digital spectral signatures based on audio streams.
  • FIG. 11 is a block diagram of an example signature comparison system for comparing digital spectral signatures.
  • FIG. 12 is a block diagram of an example processor system that may be used to implement the methods and apparatus described herein.
  • the methods and apparatus described herein generally relate to generating digital spectral signatures, which may be used to identify media information.
  • the disclosed methods and apparatus are described with respect to generating digital spectral signatures based on audio streams (e.g., audio information).
  • the methods and apparatus described herein may also be used to generate digital spectral signatures based on any other type of media information such as, for example, video information, web pages, still images, computer data, etc.
  • the media information may be associated with broadcast information (e.g., television information, radio information, etc.), information reproduced from any storage medium (e.g., compact discs (CD), digital versatile discs (DVD), etc.), or any other information that is associated with an audio stream, a video stream, or any other media information for which the digital spectral signatures are generated.
  • broadcast information e.g., television information, radio information, etc.
  • information reproduced from any storage medium e.g., compact discs (CD), digital versatile discs (DVD), etc.
  • any storage medium e.g., compact discs (CD), digital versatile discs (DVD), etc.
  • the audio streams are identified based on digital spectral signatures that include monitored digital signatures generated at a monitoring site (e.g., a monitored household) and reference digital signatures generated and/or stored at a reference site and/or a central data collection facility.
  • the methods and apparatus described herein identify media information including audio streams based on digital spectral signatures.
  • the digital spectral signatures may be formed using digital descriptors that are generated based on the spectral components of an audio stream and that may be analyzed using frequency transforms and/or wavelet transforms.
  • Intraframe operations may include, for example, comparison operations, determining percentage differences between values, etc. that are performed on two or more values (e.g., spectral power values) that are uniquely associated with or derived from a single frame.
  • a digital spectral signature may be generated by obtaining a frame of media samples, determining spectral power values by performing a spectral transform (e.g., a FFT, a wavelet transform, etc.) on the frame of media samples, and performing an intraframe operation (e.g., a comparison) based on two or more spectral power values that are uniquely associated with the frame of media samples.
  • a spectral transform e.g., a FFT, a wavelet transform, etc.
  • Frequency components of an audio signal are typically generated by transforming the audio signal data (e.g., an audio stream) from the time domain to the frequency domain using, for example, a Fourier Transform.
  • the Fourier Transform can be used to analyze the frequency components in an audio stream and identify the spectral power of each frequency component. The spectral powers may then be used to generate digital spectral signatures.
  • Digital spectral signatures may also be generated based on wavelet transforms which transform audio data from the time domain to the wavelet domain.
  • wavelet transforms may be used to decompose blocks or frames of data (e.g., time domain audio samples) into multiple sub-bands, thereby allowing data sets to be analyzed at various scales and/or resolutions. By separating data into multiple sub-bands, a wavelet transform may be used to analyze each time interval of data at a desired scale or resolution.
  • Monitored signatures may be generated at a monitoring site based on audio streams associated with media information (e.g., a monitored audio stream) that is consumed by an audience.
  • a monitored signature may be generated based on the audio track of a television program presented at a monitoring site.
  • the monitored signature may _ theti"lle ammurffdated 1 to axemM-data collection facility for comparison to one or more reference signatures.
  • Reference signatures are generated at a reference site and/or a central data collection facility based on audio streams associated with known media information.
  • the known media information may include media that is broadcast within a region, media that is reproduced within a household, media that is received via the internet, etc.
  • Each reference signature is stored in a memory with media identification information such as, for example, a song title, a movie title, etc.
  • the monitored signature is compared with one or more reference signatures until a match is found. This match information may then be used to identify the media information (e.g., monitored audio stream) from which the monitored signature was generated.
  • a look-up table or a database may be referenced to retrieve a media title, a program identity, an episode number, etc. that corresponds to the media information from which the monitored signature was generated.
  • more reference signatures are generated at a reference site and/or a central data collection facility for a given audio stream than monitored signatures generated for a given audio stream at a monitoring site.
  • the reference signatures generated for an audio stream i.e., a reference audio stream
  • overlap in time More specifically, the starting reference time or timestamp of each reference signature is shifted by a relatively small amount of time from the starting reference time or timestamp of a previous reference signature.
  • a monitored signature generated at a monitoring site and having a substantially arbitrary reference time may be aligned and/or matched with at least one of the reference signatures generated for a reference audio stream.
  • monitored signatures may be generated by processor systems and/or , , , ... hardware ⁇ devices' havihg relatively 1 li'ss computing power and/or memory than processor systems and/or hardware devices used to generate reference signatures.
  • FIGS. IA and IB illustrate example audio stream identification systems 100 and 150 for generating digital spectral signatures and identifying audio streams.
  • the example audio stream identification systems 100 and 150 may be implemented as a television broadcast information identification system and a radio broadcast information identification system, respectively.
  • the example audio stream identification system 100 includes a monitoring site 102 (e.g., a monitored household), a reference site 104, and a central data collection facility 106.
  • Monitoring television broadcast information involves generating monitored signatures at the monitoring site 102 based on the audio data of television broadcast information and communicating the monitored signatures to the central data collection facility 106 via a network 108.
  • Reference signatures may be generated at the reference site 104 and may also be communicated to the central data collection facility 106 via the network 108.
  • the audio content represented by a monitored signature that is generated at the monitoring site 102 may be identified at the central data collection facility 106 by comparing the monitored signature to one or more reference signatures until a match is found.
  • monitored signatures may be communicated from the monitoring site 102 to the reference site 104 and compared one or more reference signatures at the reference site 104.
  • the reference signatures may be communicated to the monitoring site 102 and compared with the monitored signatures at the monitoring site 102.
  • the monitoring site 102 may be, for example, a household for which the media consumption of an audience is monitored.
  • the monitoring site 102 may include a plurality of media delivery devices 110, a plurality of media presentation devices 112, and a signature generator 114 that is used to generate monitored signatures associated with media presented at the monitoring site 102.
  • [0Ui% 1 •Tnl'pfiyaWty" dFmeiatfdel ⁇ very devices 110 may include, for example, set top box tuners (e.g., cable tuners, satellite tuners, etc.), DVD players, CD players, radios, etc. Some or all of the media delivery devices 110 such as, for example, set top box tuners may be communicatively coupled to one or more broadcast information reception devices 116, which may include a cable, a satellite dish, an antenna, and/or any other suitable device for receiving broadcast information.
  • the media delivery devices 110 may be configured to reproduce media information (e.g., audio information, video information, web pages, still images, etc.) based on, for example, broadcast information and/or stored information.
  • Broadcast information may be obtained from the broadcast information reception devices 116 and stored information may be obtained from any information storage medium (e.g., a DVD, a CD, a tape, etc.).
  • the media delivery devices 110 are communicatively coupled to the media presentation devices 112 and configurable to communicate media information to the media presentation devices 112 for presentation.
  • the media presentation devices 112 may include televisions having a display device and/or a set of speakers by which audience members consume, for example, broadcast television information, music, movies, etc.
  • the signature generator 114 may be used to generate monitored digital signatures based on audio information as described in greater detail below.
  • the signature generator 114 may be configured to generate monitored signatures based on monitored audio streams that are reproduced by the media delivery devices 110 and/or presented by the media presentation devices 112.
  • the signature generator 114 may be communicatively coupled to the media delivery devices 110 and/or the media presentation devices 112 via an audio monitoring interface 118. In this manner, the signature generator 114 may obtain audio streams associated with media information that is reproduced by the media delivery devices 110 and/or presented by the media presentation devices 112.
  • the signature generator 114 may be communicatively coupled to microphones (not shown) that are placed in proximity to the media presentation devices 112 _ , _ o llifcttoliWsWyimlfe ⁇ e ⁇ gHfeitare generator 114 may also be communicatively coupled to the central data collection facility 106 via the network 108.
  • the network 108 may be used to communicate signatures (e.g., digital spectral signatures), control information, and/or configuration information between the monitoring site 102, the reference site 104, and the central data collection facility 106.
  • signatures e.g., digital spectral signatures
  • control information e.g., configuration information
  • configuration information e.g., configuration information
  • Any wired or wireless communication system such as, for example, a broadband cable network, a DSL network, a cellular telephone network, a satellite network, and/or any other communication network may be used to implement the network 108.
  • the reference site 104 may include a plurality of broadcast information tuners 120, a reference signature generator 122, a transmitter 124, a database or memory 126, and broadcast information reception devices 128.
  • the reference signature generator 122 and the transmitter 124 may be communicatively coupled to the memory 126 to store reference signatures therein and/or to retrieve stored reference signatures therefrom.
  • the broadcast information tuners 120 may be communicatively coupled to the broadcast information reception devices 128, which may include a cable, an antenna, a satellite dish, and/or any other suitable device for receiving broadcast information. Each of the broadcast information tuners 120 may be configured to tune to a particular broadcast channel. In general, the number of tuners at the reference site 104 is equal to the number of channels available in a particular broadcast region. In this manner, reference signatures may be generated for all of the media information transmitted over all of the channels in a broadcast region. The audio portion of the tuned media information may be communicated from the broadcast information tuners 120 to the reference signature generator 122.
  • the reference signature generator 122 may be configured to obtain the audio portion of all of the media information that is available in a particular broadcast region. The reference signature generator 122 may then generate a plurality of reference signatures (as described in greater detail below) based on the audio information and store the reference sign ⁇ fc ⁇ in ⁇ ne ⁇ €n ⁇ bry ⁇ y'. M ' t'n ⁇ l ⁇ gh one reference signature generator is shown in FIG. 1 , a plurality of reference signature generators may be used in the reference site 104. For example, each of the plurality of signature generators may be communicatively coupled to a respective one of the broadcast information tuners 120.
  • the transmitter 124 may be communicatively coupled to the memory 126 and configured to retrieve signatures therefrom and communicate the reference signatures to the central data collection facility 106 via the network 108.
  • the central data collection facility 106 may be configured to compare monitored signatures received from the monitoring site 102 to reference signatures received from the reference site 104.
  • the central data collection facility 106 may be configured to identify monitored audio streams by matching monitored signatures to reference signatures and using the matching information to retrieve television program identification information (e.g., program title, broadcast time, broadcast channel, etc.) from a database.
  • the central data collection facility 106 includes a receiver 130, a signature analyzer 132, and a memory 134, all of which are communicatively coupled as shown.
  • the receiver 130 may be configured to receive monitored signatures and reference signatures via the network 108.
  • the receiver 130 is communicatively coupled to the memory 134 and configured to store the monitored signatures and the reference signatures therein.
  • the signature analyzer 132 may be used to compare reference signatures to monitored signatures.
  • the signature analyzer 132 is communicatively coupled to the memory 134 and configured to retrieve the monitored signatures and the reference signatures from the same.
  • the signature analyzer 132 may be configured to retrieve reference signatures and monitored signatures from the memory 134 and compare the monitored signatures to the reference signatures until a match is found.
  • the memory 134 may be implemented using any machine accessible information storage medium such as, for example, one or more hard drives, one or more optical storage devices, etc. [0Ui% l! 132 is located at the central data collection facility 106 in FIG. IA, the signature analyzer 132 may instead be located at the reference site 104.
  • the monitored signatures may be communicated from the monitoring site 102 to the reference site 104 via the network 108.
  • the memory 134 may be located at the monitoring site 102 and reference signatures may be added periodically to the memory 134 via the network 108 by transmitter 124.
  • the signature analyzer 132 is shown as a separate device from the signature generators 114 and 122, the signature analyzer 132 may be integral with the reference signature generator 122 and/or the signature generator 114.
  • FIG. 1 depicts a single monitoring site (i.e., the monitoring site 102) and a single reference site (i.e., the reference site 104), multiple such sites may be coupled via the network 108 to the central data collection facility 106.
  • the audio stream identification system 150 of FIG. IB may be configured to monitor and identify audio streams associated with radio broadcast information.
  • the audio stream identification system 150 is used to monitor the content that is broadcast by a plurality of radio stations in a particular broadcast region.
  • the audio stream identification system 150 may be used to monitor music, songs, etc. that are broadcast within a broadcast region and the number of times that they are broadcast. This type of media tracking may be used to determine royalty payments, proper use of copyrights, etc. associated with each audio composition.
  • the audio stream identification system 150 includes a monitoring site 152, a central data collection facility 154, and the network 108.
  • the monitoring site 152 is configured to receive all radio broadcast information that is available in a particular broadcast region and generate monitored signatures based on the radio broadcast information.
  • the monitoring site 152 includes the plurality of broadcast information tuners 120, the transmitter 124, the memory 126, and the broadcast information above in connection with FIG. IA.
  • the monitoring site 152 includes a signature generator 156.
  • the broadcast information reception devices 128 are configured to receive radio broadcast information and the broadcast information tuners 120 are configured to tune to the radio broadcast stations.
  • the number of broadcast information tuners 120 at the monitoring site 152 may be equal to the number of radio broadcasting stations in a particular broadcast region.
  • the signature generator 156 is configured to receive the tuned to audio information from each of the broadcast information tuners 120 and generate monitored signatures for the same. Although one signature generator is shown (i.e., the signature generator 156), the monitoring site 152 may include multiple signature generators, each of which may be communicatively coupled to one of the broadcast information tuners 120. The signature generator 156 may store the monitored signatures in the memory 126. The transmitter 124 may retrieve the monitored signatures from the memory 126 and communicate them to the central data collection facility 154 via the network 108.
  • the central data collection facility 154 is configured to receive monitored signatures from the monitoring site 152, generate reference signatures based on reference audio streams, and compare the monitored signatures to the reference signatures.
  • the central data collection facility 154 includes the receiver 130, the signature analyzer 132, and the memory 134, all of which are described in greater detail above in connection with FIG. IA.
  • the central data collection facility 154 includes a reference signature generator 158.
  • the reference signature generator 158 is configured to generate reference signatures based on reference audio streams.
  • the reference audio streams may be stored on any type of machine accessible medium such as, for example, a CD, a DVD, a digital audio tape (DAT), etc.
  • artists and/or record producing companies send their audio data collection facility 154 to be added to a reference library.
  • the reference signature generator 158 may read the audio data from the machine accessible medium and generate a plurality of reference signatures based on each audio work (e.g., the reference audio stream 302 of FIG. 3).
  • the reference signature generator 158 may then store the reference signatures in the memory 134 for subsequent retrieval by the signature analyzer 132.
  • Identification information (e.g., song title, artist name, track number, etc.) associated with each reference audio stream may be stored in a database and may be indexed based on the reference signatures.
  • the central data collection facility 154 includes a database of reference signatures and identification information corresponding to all known and available song titles.
  • the receiver 130 is configured to receive monitored signatures from the network 108 and store the monitored signatures in the memory 134.
  • the monitored signatures and the reference signatures are retrieved from the memory 134 by the signature analyzer 132 for use in identifying the monitored audio streams broadcast within a broadcast region.
  • the signature analyzer 132 may identify the monitored audio streams by first matching a monitored signature to a reference signature. The match information and/or the matching reference signature is then used to retrieve identification information (e.g., a song title, a song track, an artist, etc.) from a database stored in the memory 134.
  • each monitoring site may be communicatively coupled to the network 108 and configured to generate monitored signatures.
  • each monitoring site may be located in a respective broadcast region and configured to monitor the content of the broadcast stations within a respective broadcast region.
  • FIG. 2 is a time-domain representation 200 of an example monitored audio stream 202 and a plurality of audio sample frames 204, 206, 208, and 210 acquired from the monitored audio stream 202.
  • a monitored digital spectral signature is generated at a mo'ti'i ⁇ ori'hg skfeW.rft ⁇ e nroniBrmg :1 site 102 of FIG. IA or the monitoring site 152 of FIG. IB) based on audio samples acquired from the example monitored audio stream 202.
  • the time-domain representation 200 illustrates the time relationship between the example monitored audio stream 202 and the audio sample frames 204, 206, 208, and 210, which are used to generate monitored signatures.
  • an N-bit monitored signature S x (I) is formed using one or more M-bit descriptors B x (to), B x (t 0 + ⁇ ), B x (to+2), B x (to+3).
  • a 32-bit monitored signature S x (t) includes four 8-bit descriptors B x (to), B x (to+ ⁇ ), B x (to+2), B x (to+3), each of which is generated based on a corresponding one of the audio sample frames 204, 206, 208, and 210. More specifically, each of the descriptors is generated using one or more intraframe operations (e.g., comparison operations) based on two or more spectral components that are uniquely associated with a single audio sample frame.
  • intraframe operations e.g., comparison operations
  • the four descriptors may be generated based on spectral decompositions (e.g., frequency decompositions or wavelet decompositions) of the audio sample frames 204, 206, 208, and 210 as described in detail below in connection with FIGS. 4-7.
  • the spectral decompositions are used to extract features that are uniquely characteristic of the example monitored audio stream 202.
  • a reference signature and a monitored signature that are generated based on the same audio stream using the same signature generation method (e.g., the same spectral decomposition-based method) will include similar features, and, thus can be used to reliably identify the monitored audio stream 202 using a matching algorithm.
  • a monitored signature may be generated by sampling the example monitored audio stream 202 to generate the audio sample frames 204, 206, 208, and 210, generating the descriptors B x (to), B x (to+ ⁇ ), B x (to+2), B x (to+3) based on spectral decompositions of the audio sample frames 204, 206, 208, and 210, and concatenating the descriptors.
  • the audio sample frames 204, 206, 208, and 210 are generated by sampling the example monitored audio stream 202 during four time intervals at a sampling For ⁇ anfple ⁇ a 11 S 1 Mi )H ⁇ freqtiMc'y/f of 6000 Hz will generate 6000 samples of audio data for each of the audio sample frames 204, 206, 208, and 210 (assuming the sample frames are collected over one second intervals).
  • any other suitable sampling frequency / ⁇ may instead be selected. As shown in FIG.
  • the duration of each of the audio sample frames 204, 206, 208, and 210 is one second (e.g., 0 to t 0 , to to t o +l, to+1 to t o +2, and t o +2 to t o +3).
  • the duration may instead be set to any other length of time.
  • the times within the monitored audio stream 202 during which monitored signatures are generated are substantially similar or identical to the times within a reference audio stream during which corresponding reference signatures are generated.
  • the features extracted from a monitored audio stream e.g., the example monitored audio stream 202
  • a corresponding reference audio stream e.g., the example reference audio stream 302 of FIG. 3
  • the audio sample frames 204, 206, 208, and 210 are shown in FIG. 2 as occurring consecutively in time, the audio sample frames 204, 206, 208, and 210 may occur at any time and in any sequence within the example monitored audio stream 202.
  • the monitored signatures may be generated relative to a reference time that is used during the signature comparison process to align a monitored signature with a reference signature. More specifically, during the generation of a monitored signature, the example monitored audio stream 202 is sampled starting at a time indicated by a reference time to, which may be selected relative to a time stamp embedded within the example audio stream 202, a system startup time, a daily recurring time (e.g., midnight), and/or any other reference time that may be indicative of the time at which a signature is generated.
  • a signature matching system e.g., the signature analyzer 132 of FIGS. IA and IB uses the reference time to to retrieve onlr' ⁇ l- to substantially the same time within reference audio streams as indicated by the reference time to.
  • one or more monitored signatures may be generated using the example monitored audio stream 202 so that multiple signatures are matched to identify the example monitored audio stream 202.
  • one or more monitored signatures generated using the example monitored audio stream 202 are substantially similar or identical to one or more reference signatures of a reference audio stream (e.g., the example reference audio stream 302 of FIG. 3) that does not correspond to the example monitored audio stream 202.
  • a reference audio stream e.g., the example reference audio stream 302 of FIG. 3
  • more than one monitored signature is generated for the monitored audio stream 202.
  • the signatures may be generated at multiple times throughout the example monitored audio stream 202.
  • a comparison algorithm may be configured to match two or more monitored signatures with corresponding reference signatures to accurately identify the example monitored audio stream 202.
  • FIG. 3 is a time-domain representation 300 of an example reference audio stream 302 and a plurality of audio sample frames 304, 306, 308, and 310 that may be acquired from the example reference audio stream 302.
  • the time-domain representation 300 shows two one second time intervals (e.g., 0 to to and to to t o +l) and the plurality of audio sample frames 304, 306, 308, and 310 that are collected during the time intervals and that are subsequently used to generate a plurality of reference signatures that are staggered in time (i.e., time shifted relative to one another).
  • the example reference audio stream 302 is sampled at a sampling frequency / ⁇ to collect the audio sample frames 304, 306, 308, and 310, which are used to generate M-bit descriptors B ⁇ n ⁇ t).
  • One or more of the descriptors B Rn ⁇ i) may then be concatenated to form an N-bit reference signature Sn n (Jt).
  • the number of descriptors in a reference signature is equal to the number of descriptors in a monitored signature.
  • each of the M-bit descriptors B R JJ) includes the same number of bits as ⁇ eMfpbr x ( ⁇ associated with the example monitored audio stream 202 (FIG. 2) and, thus, each N-bit reference signature S Rn (I) includes the same number of bits as a corresponding N-bit monitored signature S x ( ⁇ .
  • Multiple reference signatures are generated for the example reference audio stream 302 in a manner that causes the reference signatures to be time-shifted relative to each other and to overlap in time with one another. More specifically, the start time at which a first audio sample frame (e.g., the audio sample frame 304) of a first reference signature is
  • sample segments are associated with a number of equal segments ("sample segments") into which each time interval (e.g., to to t o +l) is divided. For example, if the number of sample segments Ts in a time interval of one second is set equal to thirty, the collection of a
  • each sample segment will include 200 samples.
  • Each audio sample frame is used to generate a single signature. More specifically, an audio sample frame is collected using a predetermined number of sample segments (e.g., the sample segments 312a-312e), which are concatenated to form the audio sample frame (e.g., the audio sample frame 304). To achieve overlap among consecutively generated signatures, each signature is formed using an audio sample frame that partially overlaps with an audio sample frame used to form the previously generated signature. More specifically, two audio sample frames that overlap include a common set of sample segments. For example, the audio sample frames 304 and 308 include a common set of sample segments comprising the sample segments 312b-312e.
  • the audio sample frame 308 may be formed by extracting a common plurality of media samples or the common set of sample segments 312b-312e from the audio sample frame 304 and appending a recently acquired sample common set of sample segments 312b-312e.
  • two audio sample frames that overlap also contain sample segments that are exclusive to one or the other of the audio sample frames (i.e., audio sample frames that overlap also contain sample segments that occur in one or the other of the audio sample frames but do not occur in both frames).
  • each reference signature is formed by four reference descriptors B Rn (t 0 + — ),
  • the audio sample frames 304 and 306 (generated at
  • the audio sample frames 308 and 310 form a first reference signature. Additionally, the audio sample frames 308 and 310
  • each of the reference T s is a reference to each of the reference T s.
  • descriptors is generated using one or more operations (e.g., comparison operations) based on two or more spectral components that are both associated with the same audio sample frame. These operations are referred to as intraframe operations because they are performed using are not dependent on sample data collected over other audio frames.
  • the sample segments 312a-312e are collected and are used to form the first audio sample frame 304, which is subsequently used to determine a descriptor. This descriptor is then used to form part of a first reference signature. Then, a new sample segment 312f is collected and a second audio sample frame 308 is formed using sample segments 312b-312f. The second audio sample frame 308 is then used to determine a reference descriptor which is used to form part of a second reference signature.
  • the first and second audio sample frames 304 and 308 include a common set of sample segments 312b-312e and each of the first and second audio sample frames additionally include a sample segment not included in the other (i.e., the first audio sample frame 304 includes sample segment 312a and the second audio sample frame 308 includes sample segment 312f).
  • the first and second reference signatures are generated using data collected at points in the audio stream that are staggered or shifted in time by
  • first and second signatures are shifted can be adjusted by changing the value of Ts thereby permitting the resolution of the signatures to vary as desired. Specifically, if a set of signatures that represents an audio stream with a greater resolution is desired, Ts can be increased accordingly. Likewise, if less resolution is desired, Ts can be decreased. As will be appreciated by one having ordinary skill in the art, the value of Ts may affect the quantity of signatures that can be generated for a given audio stream. For example, if the number of sample segments used to form a signature remains constant, a larger value of Ts will result in forming more audio sample frames than a smaller value of Ts. Therefore, the amount of memory available to store the signature data may be a factor in determining the desired value of T 5 .
  • a signature generator e.g., the signature generator 114 of FIG. IA
  • may be configured to monitor the audio emitted by a specific television e.g., one of the presentation devices 112 of FIG. IA located in the home of a specific panelist. Signatures are generated for the audio emitted by the television in the manner described above.
  • the time at which the audio corresponding to each signature was emitted is recorded and stored as a reference time to with the corresponding monitored signature in a memory device.
  • a timing device e.g., the timing device 903 of FIGS. 9 and 10 located at the monitoring site may be used to trigger the collection of the audio data for the subsequent generation of monitored signatures and to provide the corresponding data collection times (e.g., reference times to, timestamps, etc.) to the memory for storage with the corresponding signature.
  • a reference site e.g., the reference site 104 of FIG. IA located within the same broadcast region as a monitoring site (e.g., the monitoring site 102 of FIG. IA) is configured to generate reference signatures corresponding to the audio broadcast on all television broadcast channels at all times of the day.
  • a timing device e.g., the timing device 903 of FIGS. 9 and 10) located at the reference site may be used to trigger the collection of
  • the audio data at a set of equally spaced intervals corresponding to the time period — for the
  • each reference signature is stored in a memory device located at the reference site along with timestamp data indicating the time at which the underlying audio data was collected.
  • the timing devices e.g., the timing device 903 of FIG. 9 located at the monitoring site and the reference site are synchronized such that the timestamps can be used to align the ( reference* signMilrel ⁇ thfhV M ⁇ tdred signatures to facilitate the signature matching process.
  • each of the timing devices 903 located in each such monitoring site may be synchronized with the timing device 903 located in the single reference site.
  • the timing devices 903 at the monitoring site and the reference site do not have to be synchronized.
  • the value of T 5 may be adjusted.
  • the value of T s is generally selected to incrementally time shift a reference signature from a previous reference signature so that a monitored signature generated at an arbitrary reference time is highly likely to align with one of the staggered or time-shifted reference signatures. More specifically, increasing the value of T s causes the number of sample segments (e.g., the sample segments 312a-312f) to increase and the offset or time shift from one reference signature to the next reference signature to decrease. This, in turn, increases the likelihood that the times at which reference signatures are generated for a given audio stream correspond to substantially similar or identical reference times at which monitored signatures are generated for the same audio stream.
  • T s the likelihood of a match between a set of reference signatures and a set of monitored signatures corresponding to the same audio program.
  • a monitored signature generated for data collected at a time T need only be compared to each reference signature associated with the same timestamp T instead of all reference signatures generated during the same twenty-four hour period. This reduction in comparisons reduces the processing time required to find a match.
  • each monitored signature generated at the monitoring site at a time T need only be compared to all reference signatures generated from data collected within a window of the time spanning from T - E to T + E.
  • a set of monitored signatures are generated for a monitored audio stream and then compared to a database of reference signatures associated with a set of reference audio streams that, ideally, represent the universe of currently available audio streams.
  • reference signatures corresponding to reference audio streams may be stored in a database that is stored in, for example, the memory 134.
  • the matching information and/or the reference signature may be used to retrieve identification information (e.g., song title, song track, artist, etc.) from the database.
  • identification information is then used to identify the monitored audio stream.
  • reference times to or timestamps associated with each monitored signature may be used to identify the time (of day) at which the monitored audio streams were broadcast.
  • FIG. 4 is a flow diagram of an example method for generating digital spectral signatures based on spectral decompositions.
  • the example method of FIG. 4 may be used to generate digital spectral signatures (e.g., reference signatures and/or monitored signatures) based on frequency decomposition methods using a sliding Fast Fourier transform (FFT).
  • FFT Fast Fourier transform
  • an FFT may be used to convert a time domain signal (e.g., the example audio streams 202 and 302 of FIGS. 2 and 3) into a frequency domain representation of the same signal which may then be used to analyze the frequency components of the converted signal.
  • a sliding FFT provides advantages over a conventional non-sliding FFT for generating the digital spectral sigMliir ⁇ / ' llT ⁇ lift ⁇ i-Sb ⁇ nt ⁇ yit ⁇ - ⁇ li-sliding FFT, a sliding FFT can be used to incrementally compute an FFT.
  • one example approach to processing the audio streams 202 and 302 involves generating FFT data for each audio sample frame independent of any data associated with previous audio sample frames.
  • a sliding FFT involves generating FFT data for an audio sample frame by updating the FFT data generated in connection with a previous audio sample frame.
  • the digital spectral signatures can be generated using audio sample frames of any arbitrary size (i.e., any number of samples) that are acquired using any sampling
  • the example method involves obtaining an audio stream (block 402) (e.g., the example monitored audio stream 202 of FIG. 2 or the example reference audio stream 302 of FIG. 3).
  • a reference time to described above in connection with FIGS. 2 and 3 is determined (block 404) to indicate the time within an audio stream at which a signature is generated.
  • An initial audio sample set is then obtained (block 406).
  • the audio samples may be obtained by sampling an analog audio stream at a sampling frequency /j and performing an analog-to-digital conversion.
  • the audio samples may be obtained by extracting or acquiring samples from a digital audio stream at a sampling
  • the initial audio sample set may be a complete audio sample frame (e.g., one of the audio sample frames 204, 206, 208, and 210 of FIG. 2 or 304, 306, 308, and 310 of FIG. 3) or a portion thereof.
  • An initial FFT operation is performed on the initial audio sample set to establish an initial frequency spectrum (block 408JV L Tlleiny!n ! i8' ⁇ f ⁇ erf ⁇ rffliri ⁇ % FFT is well known in the art and, thus, is not discussed in detail herein.
  • a next set of audio samples is obtained (block 410).
  • the sliding FFT may then be used to update the initial frequency spectrum (generated at block 408) based on two most recently collected samples
  • Equation 1 may be used to update the frequency spectrum of an audio sample frame having a
  • v ⁇ _ 2 and v N _ are inserted into the audio sample frame to replace the two earliest collected
  • most recently collected audio samples e.g., the most recently collected audio samples v ⁇ _ 2
  • 5/ 1 may be set equal to a value as close as possible to one.
  • the first stability factor sf ⁇ may be set equal to 0.99995.
  • the second stability factor sfr may be set equal to where the value/? is equal to the number of sample shifts required to process an audio sample frame using the sliding FFT. For example, a two-sample shift is required to update an audio sample frame based on the two
  • the value p may be set equal to 3000. 11
  • a complete audio sample frame (e.g., one of the audio sample frames 204, 206, 208, and 210 of FIG. 2 or one of the audio sample frames 304, 306, 308, and 310 of FIG. 3) has been obtained (block 414).
  • a complete audio sample frame is obtained when a plurality of most recently collected Ns samples is obtained. For example, if an audio sample frame includes 6000 samples, a complete audio sample frame is obtained after 6000 new samples are obtained.
  • a complete audio sample frame is obtained when a most recently collected sample segment (e.g., one of the sample segments 312a-312f of FIG. 3) is obtained and a current audio sample frame is formed as described in greater detail above in connection with FIG. 3. If it is determined at block 414 that a complete audio sample frame has not been obtained, control is passed back to block 410. However, if it is determined at block 414 that a complete audio sample frame has been obtained, a descriptor is generated (block 416). An example method for generating descriptors based on frequency components is described in greater detail below in connection with FIG. 5.
  • a descriptor set includes a predetermined number of descriptors that are used to form a signature. For example, if a 32-bit signature is formed by 8-bit descriptors, then a descriptor set includes four descriptors. If it is determined at block 418 that a complete descriptor set has not been obtained, control is passed back to block 410. However, if it is determined at block 418, that a complete descriptor set has been obtained, a digital spectral signature is generated by concatenating the descriptors of the descriptor set (block 420).
  • An M-bit descriptor is generated by selecting M pairs of frequency components ⁇ that are uniquely associated with an audio sample frame and determining each bit of the descriptor based on intraframe comparisons of the spectral powers Pn, of the frequency components fi b -
  • the frequency components ⁇ and the spectral powers Pib are indexed by a frequency component index / and a bit index b, where
  • the example method initially selects a first pair of frequency componentsyoo and / 10 (block 502). Although consecutive frequency components are selected (i.e., yu b and/ ⁇ b> ./ 2b and fa,, etc.) in the example method, the frequency components may be selected from any location in the frequency spectrum of an audio sample frame (e.g., one of the audio sample frames 204, 206, 208, and 210 of FIG. 2 or one of the audio sample frames 304, 306, 308, and 310 of FIG. 3). However, the frequency component indexes / used to select pairs of frequency components for generating a monitored signature are the same frequency component indexes / used to select pairs of frequency components for generating a corresponding reference signature.
  • the spectral powers Poo and P ⁇ o corresponding to the selected frequency components are determined (block 504).
  • the spectral power for each frequency component can be obtained based on the results of the sliding FFT performed at block 412 of FIG. 4.
  • a descriptor bit is determined based on the frequency components ⁇ oo and/j 0 by comparing the first spectral power P 00 with the second spectral power P ⁇ o (block 506). If the first spectral power is greater than the second spectral power (i.e., P 00 > Pio), the descriptor 'Inste f the!' first spectral power is less than or equal to the second spectral power (i.e., the descriptor bit is set equal to zero.
  • FIG. 6 is a flow diagram of another example method for generating digital spectral signatures based on spectral decompositions.
  • the example method of FIG. 6 may be used to generate digital spectral signatures (e.g., reference signatures and monitored signatures) based on wavelet decompositions of audio sample frames (e.g., the audio sample frames 204, 206, 208, and 210 of FIG. 2 or 304, 306, 308, and 310 of FIG. 3) using wavelet transforms.
  • wavelet transforms may be employed to analyze data using different scales and/or resolutions by separating blocks or frames of data (e.g., the audio sample frames 204, 206, 208, 210, 304, 306, 308, and 310) into multiple sub-bands.
  • the example method obtains an audio stream (block 602) (e.g., the example monitored audio stream 202 of FIG. 2 or the example reference audio stream 302 of FIG. 3).
  • a reference time t 0 described above in connection with FIGS. 2 and 3 is determined (block 604) to indicate the time within an audio stream at which a signature is generated.
  • An audio sample frame is then obtained (block 606).
  • the audio sample frame may be obtained by sampling an analog audio stream at a sampling frequency ⁇ and performing an analog-to- digital conversion.
  • the audio sample frame may be obtained by extracting or acquiring samples from a digital audio stream at a sampling Based on the Nyquist Theorem, aliasing is avoided by sampling the audio samples at frequencies ranging
  • the descriptor After the descriptor is generated, it is determined if a complete descriptor set has been obtained (block 610). If a complete descriptor set has not been obtained, a next audio sample frame is obtained (block 612) and control is passed back to block 608. However, if a complete descriptor set has been obtained, a digital spectral signature is generated (block 614) by concatenating the descriptors of the descriptor set. After the digital spectral signature is generated, it is determined if another signature is to be generated (block 616). If another signature is to be generated, control is passed back to block 604. Otherwise, the example method is stopped.
  • FIG. 7 is a flow diagram of an example method for generating descriptors associated with the example method of FIG. 6.
  • the example method of FIG. 7 may be used to implement block 608 of FIG. 6.
  • An M-bit descriptor is generated by performing an M-level wavelet decomposition on an audio sample frame (e.g., one of the audio sample frames 204, 206, 208, and 210 of FIG. 2 or one of the audio sample frames 304, 306, 308, and 310 of FIG. 3). For each level wavelet decomposition, the energy of the audio signal for each sub-band is determined and descriptors are generated based on comparisons of the sub-band energies.
  • the M-level wavelet decomposition is implemented as an intraframe operation that is performed on spectral energies that are uniquely associated with an audio sample frame. Additionally, the M-level wavelet decomposition may be implemented using any wavelet transform.
  • the example method of FIG. 7 is described in terms of the well-known Daubechies wavelet transform. [0(Hf if l! 4iMi : a ⁇ ly ' tn'e ' ef kn ⁇ 'p i 'method performs a first-level wavelet decomposition (block 702) by applying the Daubechies wavelet transform to an audio sample frame. The first application of the Daubechies wavelet transform results in a low-frequency sub-band block of filtered values LQ and a high-frequency sub-band block of filtered values H 0 , each of which
  • N includes — -filtered values.
  • the Daubechies coefficients CQ, C ⁇ , C 2 , and c 3 are used to generate an N s x N s transformation matrix in which the coefficients are arranged as shown below.
  • the coefficients are ordered in the transformation matrix, as shown above, using two dominant patterns.
  • the odd rows include the first pattern, which is an ordering of the coefficients that functions as a smoothing filter (e.g., similar to a moving filter).
  • the even rows include the second pattern, which is an ordering of the coefficients that functions to bring out the details of data (e.g., the audio sample frame).
  • the transformation matrix is first applied to the entire audio sample frame (e.g., all of the Ns samples) to generate filtered values that include low-frequency sub-band filtered values alternated with high-frequency sub-band filtered values. The values are de-interleaved to generate the two sub-band blocks
  • N LQ and H 0 each of which includes — — samples.
  • the high-frequency sub-band block H 0 includes filtered values that are associated with 4
  • sub-band frequencies ranging from - 1 - to .
  • the low-frequency sub-band block LQ to generate two additional sub-band blocks L 1 and H 1 .
  • the number of filtered values in each sub-band block is halved.
  • a descriptor is generated based on a high-frequency sub-band block (e.g., H 0 , Hi, H 2 , etc.). Further details related to the implementation of wavelet transforms are well known in the art and are not described herein.
  • the high-frequency sub-band block Ho is parsed by separating the filtered values into a first half and a second half (block 704).
  • a first energy value E 0 is determined by squaring and summing the filtered values of the first half of the high-frequency sub-band block H 0 and a second energy value Ei is also determined (block 706) by squaring and summing the filtered values of the second half of the high-frequency sub-band block H 0 .
  • a descriptor bit is determined by comparing the first energy value E 0 with the second energy value Ei (block 708). For example, if the first energy value Eo is greater than the second energy value Ei (i.e., Eo > Ei), the first descriptor bit is set equal to one. If the first energy value Eo is less than or equal to the second energy value Ei (i.e., Eo ⁇ Ei), the first descriptor bit is set equal to zero. It is then determined if another descriptor bit is to be determined (block 710). If another descriptor bit is to be determined, a next-level wavelet decomposition is performed (block 712). For example, as described above, if a second-level
  • the example method of FIG. 7 may be stopped.
  • FIG. 8 is a flow diagram of an example method for comparing the digital spectral signatures (e.g., monitored signatures and reference signatures) generated using the example methods of FIGS. 4-7.
  • the example method may be used to compare a monitored signature with a reference signature, both of which are generated based on a sliding FFT or a wavelet transform.
  • the example method of FIG. 8 may be used to match a monitored signature with a reference signature by comparing the monitored signature with a plurality of reference signatures.
  • Identification information e.g., channel, program title, episode number, etc.
  • associated with a matching reference signature may then be retrieved from, for example, a database and used to generate media ratings information.
  • the comparisons may be performed by comparing any number of bits from a monitored signature with the same number of bits from a reference signature such as, for example, a bit-by-bit comparison, a byte-by-byte comparison, a word-by-word comparison, etc. Due to the large number of reference signatures available for comparison, a Hamming distance may be used for the comparisons to eliminate mismatches rapidly, thereby significantly decreasing the time required to compare a monitored signature with the reference signatures.
  • a Hamming distance between two values may be identified by determining how many bits, numbers, characters, etc. need to be changed to make the two values equal. For example, a first binary value of 0110 and a second binary value of 0101 have a Hamming distance of two because bit location zero and bit location one need to be changed to make the first binary value equal to the second binary value.
  • the example method involves first obtaining a monitored signature (block 802).
  • a reference time J 0 for the monitored signature is then obtained (block 804).
  • a first reference signature corresponding to indicated by the reference time t 0 is obtained from, for example, the memory 134 of FIGS. IA and IB (block 806).
  • the first descriptor of the monitored signature and the first descriptor of the reference signature are then obtained (block 808).
  • the first descriptor of the monitored signature is compared to the first descriptor of the reference signature to determine if the descriptors match (block 810). If a match is detected, it is determined if all of the descriptors of the monitored signature and the reference signature have been compared (block 812). If it is determined at block 812 that all of the descriptors have not been compared, the next descriptors are obtained from the monitored signature and the reference signature (block 816) and control is passed back to block 810. If it is determined at block 812 that all of the descriptors have been compared, the monitored audio stream is identified based on the matching reference signature (block 814). Alternatively, the example method may be implemented so that multiple monitored signatures of a single audio stream need to be matched to multiple signatures of a reference audio stream prior to identifying the monitored audio stream.
  • the example method of FIG. 8 is described as comparing one reference signature at a time, the example method can be adapted to compare multiple reference signatures with one monitored signature in parallel (i.e., at the same time).
  • the operation of block 806 may be configured to obtain a plurality of reference signatures at one tinfe* reference audio stream.
  • the operation of block 808 may be configured to obtain the first descriptor of each of the plurality of reference signatures retrieved at block 806.
  • the descriptors of each of the reference signatures may be compared with the each descriptor of the monitored signature until a match is found or until the plurality of reference signatures obtained at block 806 is eliminated, after which time another plurality of reference signatures may be obtained at block 820.
  • the Hamming distance may significantly reduce the time required to match all of the available reference signatures. For example, after the first descriptor of each of a plurality of reference signatures are compared to the first descriptor of a monitored signature, the first descriptor of each of the reference signatures is associated with a Hamming distance. Only reference signatures having first descriptors associated with a Hamming distance less than a predetermined Hamming distance threshold are further compared with the monitored signature based on the next descriptor of each of the reference signatures and the monitored signature. Reference signatures having descriptors associated with a Hamming distance greater than a predetermined threshold are discarded.
  • the number of reference signatures to be compared based on the next descriptors is reduced from the number of reference signatures compared to the monitored signature based on the first descriptor. In this manner, with each iteration of the comparison process, the number of reference signatures that remain to be compared in subsequent iterations of the signature comparison process quickly diminishes until it is determined that all of the descriptors of a single reference signature are associated with a Hamming distance below the predetermined Hamming distance threshold.
  • more than one monitored signature may need to be matched with respective reference signatures of the possible matching reference audio streams. It will be relatively unlikely that all on the monitored audio stream will match all of the reference signatures of more than one reference audio stream, and, thus erroneously matching more than one reference audio stream to the monitored audio stream can be prevented.
  • the example methods described above in connection with FIGS. 4-8 may be implemented by hardware, software, and/or any combination thereof. More specifically, the example methods may be executed in hardware defined by the block diagrams of FIGS. 9-11. The example methods may also be implemented by software executed on a processor system such as, for example, the processor system 1210 of FIG. 12.
  • FIG. 9 is a block diagram of an example signature generation system 900 for generating digital spectral signatures.
  • the example signature generation system 900 may be used to generate monitored signatures and/or reference signatures based on a sliding FFT as described above in connection with the example methods of FIGS. 4 and 5.
  • the example signature generation system 900 may be used to implement the signature generators 114 and 122 of FIG. IA or the signature generators 156 and 158 of FIG. IB.
  • the example signature generation system 900 may be used to implement the example methods of FIGS. 4 and 5.
  • the example signature generation system 900 includes a sample generator 902, a timing device 903, a reference time generator 904, a sliding FFT module 906, a frequency identifier 908, a spectral power value identifier 910, a comparator 912, a descriptor generator 914, a concatenator 916, and a data communication interface 918, all of which may be communicatively coupled as shown.
  • the example signature generation system 900 may be configured to obtain an example audio stream 920, acquire a plurality of audio samples from the example audio stream 920, and generate digital spectral signatures based on the audio samples.
  • the sample generator 902 may be configured to obtain the example audio stream 920, which may be any analog or digital audio stream. If the example audio stream 920 is an analog audio stream, the sample generator 902 may be implemented using an analog-to- digital converter. If the example audio stream 920 is a digital audio stream, the sample generator 902 may be implemented using a digital signal processor. Additionally, the sample generator 902 may be configured to acquire and/or extract audio samples at any desired sampling frequency ⁇ and notify the reference time generator 904 when an audio sample acquisition process begins. The sample generator 902 communicates samples to the sliding FFT module 906.
  • the sample generator 902 may also be configured to notify the frequency identifier 908 when an audio sample frame (e.g., one of the audio sample frames 204, 206, 208, and 210 of FIG. 2 or 304, 306, 308, and 310 of FIG. 3) or a sample segment (e.g., one of the sample segments 312a-f of FIG. 3) has been generated.
  • an audio sample frame e.g., one of the audio sample frames 204, 206, 208, and 210 of FIG. 2 or 304, 306, 308, and 310 of FIG. 3
  • a sample segment e.g., one of the sample segments 312a-f of FIG.
  • the timing device 903 may be configured to generate time data and/or timestamp information and may be implemented by a clock, a timer, a counter, and/or any other suitable device.
  • the timing device 903 may be communicatively coupled to the reference time generator 904 and may be configured to communicate time data and/or timestamps to the reference time generator 904.
  • the timing device 903 may also be communicatively coupled to the sample generator 902 and may assert a start signal or interrupt to instruct the sample generator 902 to begin collecting or acquiring audio sample data.
  • the timing device 903 may be implemented by a real-time clock having a 24-hour period that tracks time at a resolution of milliseconds. In this case, the timing device 903 may be configured to reset to zero at midnight and track time in milliseconds with respect to midnight.
  • the reference time generator 904 may initialize a reference time t 0 when a notification is received from the sample generator 902. As described above in connection with FIGS. 2 and 3, the reference time to may be used to indicate the time within an audio stream at which a signature is generated.
  • the reference time generator 904 may be i fe £ ⁇ bt ⁇ mg£Wateer ⁇ anMiimeK ⁇ l£tM/or a timestamp value from the timing device 903 when notified of the beginning of a sample acquisition process by the sample generator 902. The reference time generator 904 may then store the timestamp value as the reference time to.
  • the sliding FFT module 906 may be configured to perform a sliding FFT using the audio samples obtained from the sample generator 902. As described above in connection with FIG. 4, a sliding FFT may update frequency spectrum data each time two
  • samples e.g., the two most recently acquired samples v ⁇ _ 2 and v N _, ) are obtained from the
  • the frequency identifier 908 may be configured to identify one or more frequency pairs from frequency spectrum data in response to a notification from the sample generator 902 that a new audio sample frame or a new sample segment has been generated. For example, if the example signature generation system 900 is configured to generate monitored signatures, the frequency identifier 908 identifies frequency pairs from the frequency spectrum data in response to a new audio sample frame notification. Alternatively, if the example signature generation system 900 is configured to generate reference signatures, an audio sample frame of data is formed with each new sample segment as described above in connection with FIG. 3. Therefore, the frequency identifier 908 identifies frequency pairs from the frequency spectrum data in response to a new sample segment notification. The frequency identifier 908 may then be configured to communicate indexes identifying the frequency components of the frequency pairs to the spectral power value identifier 910.
  • the spectral power value identifier 910 may be configured to obtain the indexes associated with the frequency components of the frequency pairs from the frequency identifier 908. The spectral power value identifier 910 may then determine or identify the spectral power of each frequency component of the frequency pairs by retrieving the spectral power value for each frequency component from the frequency spectrum data generated by _ ,
  • the comparator 912 and the descriptor generator 914 may work cooperatively to generate M-bit descriptors.
  • the comparator 912 may obtain the spectral power values and compare the spectral power values for each frequency pair.
  • the descriptor generator 914 may be configured to obtain comparison results from the comparator 912 and generate the descriptor bits of an M-bit descriptor based on the comparison results.
  • the concatenator 916 may obtain descriptor values from the descriptor generator 914. When a complete set of descriptors is obtained, the concatenator 916 may concatenate the descriptors 916 to form a digital spectral signature.
  • the data communication interface 918 may obtain the digital spectral signatures from the concatenator 916 and the reference time to corresponding to the digital spectral signature and communicate the same to a memory and/or a reference site. For example, if the example signature generation system 900 is configured to generate monitored signatures at the monitoring site 102 (FIG. 1 A), the monitored signatures may be communicated to the central data collection facility (FIG. IA) via the network 108 (FIG. IA). Alternatively, if the example signature generation system 900 is configured to generate reference signatures, the reference signatures may be communicated to the central data collection facility 154 (FIG. IB) and/or stored in the memory 134 (FIG. IB).
  • FIG. 10 is a block diagram of another example signature generation system 1000 for generating digital signatures based on audio streams.
  • the example signature generation system 1000 may be used to generate monitored signatures and/or reference signatures based on wavelet transforms as described above in connection with the example methods of FIGS. 6 and 7.
  • the example signature generation system 1000 may be used to implement the signature generators 114 and 122 of FIG. IA and generate alternatively, the example signature generation system 1000 may be used to implement the signature generators 156 and 158 of FIG. IB.
  • the example signature generation system 1000 may be used to implement the example methods of FIGS. 6 and 7.
  • the example signature generation system 1000 includes the sample generator 902, the timing device 903, the reference time generator 904, the comparator 912, the descriptor generator 914, the concatenator 916, and the data communication interface 918 of the example signature generation system 900 described above in connection with FIG. 9. Additionally, the example signature generation system 1000 includes a wavelet transform module 1002, a sub-band block identifier 1004, and an energy value generator 1006, all of which may be communicatively coupled as shown.
  • the wavelet transform module 1002 may be configured to apply wavelet transforms to audio samples obtained from the sample generator 902. For example, the wavelet transform module 1002 may obtain an audio sample frame (e.g., one of the audio sample frames 204, 206, 208, and 210 of FIG. 2 or 304, 306, 308, and 310 of FIG. 3) from the sample generator 902 and perform an M-level wavelet decomposition on the audio samples to generate filtered data values using, for example, the Daubechies wavelet transform as described in connection with FIGS. 6 and 7. The filtered data values may then be communicated to the sub-band block identifier 1004.
  • an audio sample frame e.g., one of the audio sample frames 204, 206, 208, and 210 of FIG. 2 or 304, 306, 308, and 310 of FIG.
  • the filtered data values may then be communicated to the sub-band block identifier 1004.
  • the sub-band block identifier 1004 may be configured to obtain the filtered data values from the wavelet transform module 1002 and generate a low-frequency sub-band block L x and a high-frequency sub-band block H x . As described in greater detail above in connection with FIG. 7, the sub-band blocks L x and H x may be identified by de-interleaving the filtered data values. The low-frequency sub-band block may then be communicated to the wavelet transform module 1002 to perform another wavelet decomposition and the high- frequency sub-band filtered block may be communicated to the energy value generator 1006. [0OiOd] ⁇ 1006 may be configured to generate energy values E x based on the high-frequency sub-band block.
  • the energy value generator 1006 may be configured to parse or separate the high-frequency sub-band block into a first half of filtered data values and a second half of filtered data values as described in greater detail above in connection with FIG. 7. The energy value generator 1006 may then generate a first energy value E 0 by squaring and summing the first half of filtered data values. A second energy value Ei may be generated by squaring and summing the second half of filtered data values.
  • the comparator 912 and the descriptor generator 914 may be configured to generate descriptors based on energy values. For example, the comparator 912 may obtain energy values from the energy value generator 1006 and compare a first energy to a second energy value. The descriptor generator 914 may obtain comparison results from the comparator 912 and generate the bits of an M-bit descriptor based on the comparison results.
  • the concatenator 916 may obtain descriptors from the descriptor generator 914 and generate digital spectral signatures by concatenating a plurality of descriptors as described above in connection with FIG. 9.
  • the data communication interface 918 may then store or transmit signatures obtained from the concatenator 916 with corresponding reference times obtained from the reference time generator 904.
  • FIG. 11 is a block diagram of an example signature comparison system 1100 for comparing digital spectral signatures.
  • the example signature comparison system 1100 may be used to compare monitored signatures with reference signatures.
  • the example signature comparison system 1100 may be used to implement the signature analyzer 132 of FIG. 1 to compare monitored signatures with reference signatures.
  • the example signature comparison system 1100 may be used to implement the example method of FIG. 8.
  • the example signature comparison system 1100 includes a monitored signature receiver 1102, a reference signature receiver 1104, a comparator 1106, a Hamming distance media identification look-up table interface 11 12, all of which may be communicatively coupled as shown.
  • the monitored signature receiver 1102 may be configured to obtain monitored signatures via the network 106 (FIG. 1) and communicate the monitored signatures to the comparator 1106.
  • the reference signature receiver 1104 may be configured to obtain reference signatures from the memory 134 (FIGS. IA and IB) and communicate the reference signatures to the comparator 1106.
  • the comparator 1106 and the Hamming distance filter 1108 may be configured to compare reference signatures to monitored signatures using Hamming distances.
  • the comparator 1106 may be configured to compare descriptors of monitored signatures with descriptors from a plurality of reference signatures and to generate Hamming distance values for each comparison.
  • the Hamming distance filter 1108 may then obtain the Hamming distance values from the comparator 1106 and filter out non-matching reference signatures based on the Hamming distance values as described above in connection with FIG. 8.
  • the media identifier 1110 may obtain the matching reference signature and in cooperation with the media identification look-up table interface 1112 may identify the media information associated with an unidentified audio stream (e.g., the example monitored audio stream 202 of FIG. 2).
  • the media identification look-up table interface 1112 may be communicatively coupled to a media identification look-up table or a database that is used to cross-reference media identification information (e.g., movie title, show title, song title, artist name, episode number, etc.) based on reference signatures. In this manner, the media identifier 1110 may retrieve media identification information from the media identification database based on the matching reference signatures.
  • the processor system 1210 includes a processor 1212 that is coupled to an interconnection bus or network 1214.
  • the processor 1212 includes a register set or register space 1216, which is depicted in FIG. 12 as being entirely on-chip, but which could alternatively be located entirely or partially off-chip and directly coupled to the processor 1212 via dedicated electrical connections and/or via the interconnection network or bus 1214.
  • the processor 1212 may be any suitable processor, processing unit or microprocessor.
  • the system 1210 may be a multi -processor system and, thus, may include one or more additional processors that are identical or similar to the processor 1212 and that are communicatively coupled to the interconnection bus or network 1214.
  • the processor 1212 of FIG. 12 is coupled to a chipset 1218, which includes a memory controller 1220 and an input/output (I/O) controller 1222.
  • a chipset typically provides I/O and memory management functions as well as a plurality of general purpose and/or special purpose registers, timers, etc. that are accessible or used by one or more processors coupled to the chipset.
  • the memory controller 1220 performs functions that enable the processor 1212 (or processors if there are multiple processors) to access a system memory 1224 and a mass storage memory 1225.
  • the system memory 1224 may include any desired type of volatile and/or non ⁇ volatile memory such as, for example, static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, read-only memory (ROM), etc.
  • the mass storage memory 1225 may include any desired type of mass storage device including hard disk drives, optical drives, tape storage devices, etc.
  • the I/O controller 1222 performs functions that enable the processor 1212 to communicate with peripheral input/output (I/O) devices 1226 and 1228 via an I/O bus 1230.
  • the I/O devices 1226 and 1228 may be any desired type of I/O device such as, for example, a kefb'lardVa ' fey la/or ffitoir; a mouse, etc. While the memory controller 1220 and the I/O controller 1222 are depicted in FIG. 12 as separate functional blocks within the chipset 1218, the functions performed by these blocks may be integrated within a single semiconductor circuit or may be implemented using two or more separate integrated circuits.
  • the methods described herein may be implemented using instructions stored on a computer readable medium that are executed by the processor 1212.
  • the computer readable medium may include any desired combination of solid state, magnetic and/or optical media implemented using any desired combination of mass storage devices (e.g., disk drive), removable storage devices (e.g., floppy disks, memory cards or sticks, etc.) and/or integrated memory devices (e.g., random access memory, flash memory, etc.).

Abstract

Methods, apparatus, and articles of manufacture for media monitoring are disclosed. In particular, the example methods, apparatus, and articles of manufacture generate digital spectral signatures for use in identifying media information. Initially, a frame of media samples is obtained. A first frequency component having a first spectral power and a second frequency component having a second spectral power are identified by performing a spectral transform operation on the frame of media samples. A descriptor of the first frame of media samples is determined based on a comparison of the first spectral power and the second spectral power. A first signature is then generated based on the descriptor.

Description

L.;: "S '<}A&ti$ώ jti i ςkAτυs FOR GENERATING SIGNATURES
FIELD OF THE DISCLOSURE
[0001] The present disclosure relates generally to media monitoring and, more particularly, to methods and apparatus for generating signatures for use in identifying media information.
BACKGROUND
[0002] Identifying media information and more specifically audio streams (e.g., audio information) using signature-matching techniques is well known. Known signature-matching techniques are often used in television and radio audience metering applications and are implemented using several known methods for generating and matching signatures. For example, in television audience metering applications, signatures are generated at monitoring sites (e.g., monitored households) and reference sites. Monitoring sites typically include locations such as, for example, households where the media consumption of audience members is monitored. For example, at a monitoring site, monitored signatures may be generated based on audio streams associated with a selected channel, radio station, etc. The monitored signatures may then be sent to a central data collection facility for analysis. At a reference site, signatures, typically referred to as reference signatures, are generated based on known programs that are provided within a broadcast region. The reference signatures may be stored at the reference site and/or a central data collection facility and compared with monitored signatures generated at monitoring sites. A monitored signature may be found to match with a reference signature and the known program corresponding to the matching reference signature may be identified as the program that was presented at the monitoring site.
BRIEF DESCRIPTION OF THE DRAWINGS
[0003] FIGS. IA and IB illustrate example audio stream identification systems for generating digital spectral signatures and identifying audio streams. [OdtC- j" '1F rI a:itfmiFdi-fflMn representation of an example monitored audio stream and a plurality of audio sample frames acquired from the monitored audio stream.
[0005] FIG. 3 is a time-domain representation of an example reference audio stream and a plurality of audio sample frames acquired from the example reference audio stream.
[0006] FIG. 4 is a flow diagram of an example method for generating digital spectral signatures based on spectral decompositions.
[0007] FIG. 5 is a flow diagram of an example method for generating descriptors associated with the example method of FIG. 4.
[0008] FIG. 6 is a flow diagram of another example method for generating digital spectral signatures based on spectral decompositions.
[0009] FIG. 7 is a flow diagram of an example method for generating descriptors associated with the example method of FIG. 6.
[0010] FIG. 8 is a flow diagram of an example method for comparing the digital spectral signatures generated using the example methods of FIGS. 4-7.
[0011] FIG. 9 is a block diagram of an example signature generation system for generating digital spectral signatures based on audio streams.
[0012] FIG. 10 is a block diagram of another example signature generation system for generating digital spectral signatures based on audio streams.
[0013] FIG. 11 is a block diagram of an example signature comparison system for comparing digital spectral signatures.
[0014] FIG. 12 is a block diagram of an example processor system that may be used to implement the methods and apparatus described herein.
DETAILED DESCRIPTION
[0015] Although the following discloses example systems including, among other components, software executed on hardware, it should be noted that such systems are merely illustrative and should not be considered as limiting. For example, it is contemplated that any or all of these hardware and software components could be embodied exclusively in
Figure imgf000004_0001
combination of hardware and software. Accordingly, while the following describes example systems, persons of ordinary skill in the art will readily appreciate that the examples provided are not the only way to implement such systems.
[0016] The methods and apparatus described herein generally relate to generating digital spectral signatures, which may be used to identify media information. In particular, the disclosed methods and apparatus are described with respect to generating digital spectral signatures based on audio streams (e.g., audio information). However, the methods and apparatus described herein may also be used to generate digital spectral signatures based on any other type of media information such as, for example, video information, web pages, still images, computer data, etc. Further, the media information may be associated with broadcast information (e.g., television information, radio information, etc.), information reproduced from any storage medium (e.g., compact discs (CD), digital versatile discs (DVD), etc.), or any other information that is associated with an audio stream, a video stream, or any other media information for which the digital spectral signatures are generated. In one particular example, the audio streams are identified based on digital spectral signatures that include monitored digital signatures generated at a monitoring site (e.g., a monitored household) and reference digital signatures generated and/or stored at a reference site and/or a central data collection facility.
[0017] As described in detail below, the methods and apparatus described herein identify media information including audio streams based on digital spectral signatures. The digital spectral signatures may be formed using digital descriptors that are generated based on the spectral components of an audio stream and that may be analyzed using frequency transforms and/or wavelet transforms.
[0018] Unlike known methods in the prior art that use interframe operations (e.g., operations based on sample data within different data sample frames) to generate digital
Figure imgf000005_0001
described herein may be implemented using intraframe operations (e.g., operations based on sample data within a single frame). Intraframe operations may include, for example, comparison operations, determining percentage differences between values, etc. that are performed on two or more values (e.g., spectral power values) that are uniquely associated with or derived from a single frame. For example, using the methods and apparatus described herein, a digital spectral signature may be generated by obtaining a frame of media samples, determining spectral power values by performing a spectral transform (e.g., a FFT, a wavelet transform, etc.) on the frame of media samples, and performing an intraframe operation (e.g., a comparison) based on two or more spectral power values that are uniquely associated with the frame of media samples.
[0019] Frequency components of an audio signal are typically generated by transforming the audio signal data (e.g., an audio stream) from the time domain to the frequency domain using, for example, a Fourier Transform. The Fourier Transform can be used to analyze the frequency components in an audio stream and identify the spectral power of each frequency component. The spectral powers may then be used to generate digital spectral signatures.
[0020] Digital spectral signatures may also be generated based on wavelet transforms which transform audio data from the time domain to the wavelet domain. In general, wavelet transforms may be used to decompose blocks or frames of data (e.g., time domain audio samples) into multiple sub-bands, thereby allowing data sets to be analyzed at various scales and/or resolutions. By separating data into multiple sub-bands, a wavelet transform may be used to analyze each time interval of data at a desired scale or resolution.
[0021] Monitored signatures may be generated at a monitoring site based on audio streams associated with media information (e.g., a monitored audio stream) that is consumed by an audience. For example, a monitored signature may be generated based on the audio track of a television program presented at a monitoring site. The monitored signature may _ theti"lle ammurffdated1 to axemM-data collection facility for comparison to one or more reference signatures.
[0022] Reference signatures are generated at a reference site and/or a central data collection facility based on audio streams associated with known media information. The known media information may include media that is broadcast within a region, media that is reproduced within a household, media that is received via the internet, etc. Each reference signature is stored in a memory with media identification information such as, for example, a song title, a movie title, etc. When a monitored signature is received at the central data collection facility, the monitored signature is compared with one or more reference signatures until a match is found. This match information may then be used to identify the media information (e.g., monitored audio stream) from which the monitored signature was generated. For example, a look-up table or a database may be referenced to retrieve a media title, a program identity, an episode number, etc. that corresponds to the media information from which the monitored signature was generated.
[0023] As described below in connection with FIGS. 2 and 3, more reference signatures are generated at a reference site and/or a central data collection facility for a given audio stream than monitored signatures generated for a given audio stream at a monitoring site. In particular, the reference signatures generated for an audio stream (i.e., a reference audio stream) overlap in time. More specifically, the starting reference time or timestamp of each reference signature is shifted by a relatively small amount of time from the starting reference time or timestamp of a previous reference signature. In this manner, a monitored signature generated at a monitoring site and having a substantially arbitrary reference time may be aligned and/or matched with at least one of the reference signatures generated for a reference audio stream. As a result of the relatively fewer number of monitored signatures generated at the monitoring site, monitored signatures may be generated by processor systems and/or ,, , ... hardware^ devices' havihg relatively1 li'ss computing power and/or memory than processor systems and/or hardware devices used to generate reference signatures.
[0024] FIGS. IA and IB illustrate example audio stream identification systems 100 and 150 for generating digital spectral signatures and identifying audio streams. The example audio stream identification systems 100 and 150 may be implemented as a television broadcast information identification system and a radio broadcast information identification system, respectively. The example audio stream identification system 100 includes a monitoring site 102 (e.g., a monitored household), a reference site 104, and a central data collection facility 106.
[0025] Monitoring television broadcast information involves generating monitored signatures at the monitoring site 102 based on the audio data of television broadcast information and communicating the monitored signatures to the central data collection facility 106 via a network 108. Reference signatures may be generated at the reference site 104 and may also be communicated to the central data collection facility 106 via the network 108. The audio content represented by a monitored signature that is generated at the monitoring site 102 may be identified at the central data collection facility 106 by comparing the monitored signature to one or more reference signatures until a match is found. Alternatively, monitored signatures may be communicated from the monitoring site 102 to the reference site 104 and compared one or more reference signatures at the reference site 104. In another example, the reference signatures may be communicated to the monitoring site 102 and compared with the monitored signatures at the monitoring site 102.
[0026] The monitoring site 102 may be, for example, a household for which the media consumption of an audience is monitored. In general, the monitoring site 102 may include a plurality of media delivery devices 110, a plurality of media presentation devices 112, and a signature generator 114 that is used to generate monitored signatures associated with media presented at the monitoring site 102. ,
[0Ui% 1 •Tnl'pfiyaWty" dFmeiatfdelϊvery devices 110 may include, for example, set top box tuners (e.g., cable tuners, satellite tuners, etc.), DVD players, CD players, radios, etc. Some or all of the media delivery devices 110 such as, for example, set top box tuners may be communicatively coupled to one or more broadcast information reception devices 116, which may include a cable, a satellite dish, an antenna, and/or any other suitable device for receiving broadcast information. The media delivery devices 110 may be configured to reproduce media information (e.g., audio information, video information, web pages, still images, etc.) based on, for example, broadcast information and/or stored information. Broadcast information may be obtained from the broadcast information reception devices 116 and stored information may be obtained from any information storage medium (e.g., a DVD, a CD, a tape, etc.). The media delivery devices 110 are communicatively coupled to the media presentation devices 112 and configurable to communicate media information to the media presentation devices 112 for presentation. The media presentation devices 112 may include televisions having a display device and/or a set of speakers by which audience members consume, for example, broadcast television information, music, movies, etc.
[0028] The signature generator 114 may be used to generate monitored digital signatures based on audio information as described in greater detail below. In particular, at the monitoring site 102, the signature generator 114 may be configured to generate monitored signatures based on monitored audio streams that are reproduced by the media delivery devices 110 and/or presented by the media presentation devices 112. The signature generator 114 may be communicatively coupled to the media delivery devices 110 and/or the media presentation devices 112 via an audio monitoring interface 118. In this manner, the signature generator 114 may obtain audio streams associated with media information that is reproduced by the media delivery devices 110 and/or presented by the media presentation devices 112. Additionally or alternatively, the signature generator 114 may be communicatively coupled to microphones (not shown) that are placed in proximity to the media presentation devices 112 _, _ o llifcttoliWsWyimlfe^^e^gHfeitare generator 114 may also be communicatively coupled to the central data collection facility 106 via the network 108.
[0029] The network 108 may be used to communicate signatures (e.g., digital spectral signatures), control information, and/or configuration information between the monitoring site 102, the reference site 104, and the central data collection facility 106. Any wired or wireless communication system such as, for example, a broadband cable network, a DSL network, a cellular telephone network, a satellite network, and/or any other communication network may be used to implement the network 108.
[0030] As shown in FIG. 1 A, the reference site 104 may include a plurality of broadcast information tuners 120, a reference signature generator 122, a transmitter 124, a database or memory 126, and broadcast information reception devices 128. The reference signature generator 122 and the transmitter 124 may be communicatively coupled to the memory 126 to store reference signatures therein and/or to retrieve stored reference signatures therefrom.
[0031] The broadcast information tuners 120 may be communicatively coupled to the broadcast information reception devices 128, which may include a cable, an antenna, a satellite dish, and/or any other suitable device for receiving broadcast information. Each of the broadcast information tuners 120 may be configured to tune to a particular broadcast channel. In general, the number of tuners at the reference site 104 is equal to the number of channels available in a particular broadcast region. In this manner, reference signatures may be generated for all of the media information transmitted over all of the channels in a broadcast region. The audio portion of the tuned media information may be communicated from the broadcast information tuners 120 to the reference signature generator 122.
[0032] The reference signature generator 122 may be configured to obtain the audio portion of all of the media information that is available in a particular broadcast region. The reference signature generator 122 may then generate a plurality of reference signatures (as described in greater detail below) based on the audio information and store the reference signέfcά in^ne^^€n^bry^βy'. M't'nβlαgh one reference signature generator is shown in FIG. 1 , a plurality of reference signature generators may be used in the reference site 104. For example, each of the plurality of signature generators may be communicatively coupled to a respective one of the broadcast information tuners 120.
[0033] The transmitter 124 may be communicatively coupled to the memory 126 and configured to retrieve signatures therefrom and communicate the reference signatures to the central data collection facility 106 via the network 108.
[0034] The central data collection facility 106 may be configured to compare monitored signatures received from the monitoring site 102 to reference signatures received from the reference site 104. In addition, the central data collection facility 106 may be configured to identify monitored audio streams by matching monitored signatures to reference signatures and using the matching information to retrieve television program identification information (e.g., program title, broadcast time, broadcast channel, etc.) from a database. The central data collection facility 106 includes a receiver 130, a signature analyzer 132, and a memory 134, all of which are communicatively coupled as shown.
[0035] The receiver 130 may be configured to receive monitored signatures and reference signatures via the network 108. The receiver 130 is communicatively coupled to the memory 134 and configured to store the monitored signatures and the reference signatures therein.
[0036] The signature analyzer 132 may be used to compare reference signatures to monitored signatures. The signature analyzer 132 is communicatively coupled to the memory 134 and configured to retrieve the monitored signatures and the reference signatures from the same. The signature analyzer 132 may be configured to retrieve reference signatures and monitored signatures from the memory 134 and compare the monitored signatures to the reference signatures until a match is found. The memory 134 may be implemented using any machine accessible information storage medium such as, for example, one or more hard drives, one or more optical storage devices, etc. [0Ui% l!
Figure imgf000011_0001
132 is located at the central data collection facility 106 in FIG. IA, the signature analyzer 132 may instead be located at the reference site 104. In such a configuration, the monitored signatures may be communicated from the monitoring site 102 to the reference site 104 via the network 108. Alternatively, the memory 134 may be located at the monitoring site 102 and reference signatures may be added periodically to the memory 134 via the network 108 by transmitter 124. Additionally, although the signature analyzer 132 is shown as a separate device from the signature generators 114 and 122, the signature analyzer 132 may be integral with the reference signature generator 122 and/or the signature generator 114. Still further, although FIG. 1 depicts a single monitoring site (i.e., the monitoring site 102) and a single reference site (i.e., the reference site 104), multiple such sites may be coupled via the network 108 to the central data collection facility 106.
[0038] The audio stream identification system 150 of FIG. IB may be configured to monitor and identify audio streams associated with radio broadcast information. In general, the audio stream identification system 150 is used to monitor the content that is broadcast by a plurality of radio stations in a particular broadcast region. Unlike the audio stream identification system 100 used to monitor television content consumed by an audience, the audio stream identification system 150 may be used to monitor music, songs, etc. that are broadcast within a broadcast region and the number of times that they are broadcast. This type of media tracking may be used to determine royalty payments, proper use of copyrights, etc. associated with each audio composition. The audio stream identification system 150 includes a monitoring site 152, a central data collection facility 154, and the network 108.
[0039] The monitoring site 152 is configured to receive all radio broadcast information that is available in a particular broadcast region and generate monitored signatures based on the radio broadcast information. The monitoring site 152 includes the plurality of broadcast information tuners 120, the transmitter 124, the memory 126, and the broadcast information
Figure imgf000012_0001
above in connection with FIG. IA. In addition, the monitoring site 152 includes a signature generator 156. When used in the audio stream identification system 150, the broadcast information reception devices 128 are configured to receive radio broadcast information and the broadcast information tuners 120 are configured to tune to the radio broadcast stations. The number of broadcast information tuners 120 at the monitoring site 152 may be equal to the number of radio broadcasting stations in a particular broadcast region.
[0040] The signature generator 156 is configured to receive the tuned to audio information from each of the broadcast information tuners 120 and generate monitored signatures for the same. Although one signature generator is shown (i.e., the signature generator 156), the monitoring site 152 may include multiple signature generators, each of which may be communicatively coupled to one of the broadcast information tuners 120. The signature generator 156 may store the monitored signatures in the memory 126. The transmitter 124 may retrieve the monitored signatures from the memory 126 and communicate them to the central data collection facility 154 via the network 108.
[0041] The central data collection facility 154 is configured to receive monitored signatures from the monitoring site 152, generate reference signatures based on reference audio streams, and compare the monitored signatures to the reference signatures. The central data collection facility 154 includes the receiver 130, the signature analyzer 132, and the memory 134, all of which are described in greater detail above in connection with FIG. IA. In addition, the central data collection facility 154 includes a reference signature generator 158.
[0042] The reference signature generator 158 is configured to generate reference signatures based on reference audio streams. The reference audio streams may be stored on any type of machine accessible medium such as, for example, a CD, a DVD, a digital audio tape (DAT), etc. In general, artists and/or record producing companies send their audio
Figure imgf000013_0001
data collection facility 154 to be added to a reference library. The reference signature generator 158 may read the audio data from the machine accessible medium and generate a plurality of reference signatures based on each audio work (e.g., the reference audio stream 302 of FIG. 3). The reference signature generator 158 may then store the reference signatures in the memory 134 for subsequent retrieval by the signature analyzer 132. Identification information (e.g., song title, artist name, track number, etc.) associated with each reference audio stream may be stored in a database and may be indexed based on the reference signatures. In this manner, the central data collection facility 154 includes a database of reference signatures and identification information corresponding to all known and available song titles.
[0043] The receiver 130 is configured to receive monitored signatures from the network 108 and store the monitored signatures in the memory 134. The monitored signatures and the reference signatures are retrieved from the memory 134 by the signature analyzer 132 for use in identifying the monitored audio streams broadcast within a broadcast region. The signature analyzer 132 may identify the monitored audio streams by first matching a monitored signature to a reference signature. The match information and/or the matching reference signature is then used to retrieve identification information (e.g., a song title, a song track, an artist, etc.) from a database stored in the memory 134.
[0044] Although one monitoring site (e.g., the monitoring site 152) is shown in FIG. IB, multiple monitoring sites may be communicatively coupled to the network 108 and configured to generate monitored signatures. In particular, each monitoring site may be located in a respective broadcast region and configured to monitor the content of the broadcast stations within a respective broadcast region.
[0045] FIG. 2 is a time-domain representation 200 of an example monitored audio stream 202 and a plurality of audio sample frames 204, 206, 208, and 210 acquired from the monitored audio stream 202. A monitored digital spectral signature is generated at a mo'ti'iϊori'hg skfeW.rftϊe nroniBrmg:1site 102 of FIG. IA or the monitoring site 152 of FIG. IB) based on audio samples acquired from the example monitored audio stream 202. The time-domain representation 200 illustrates the time relationship between the example monitored audio stream 202 and the audio sample frames 204, 206, 208, and 210, which are used to generate monitored signatures.
[0046] In one example, an N-bit monitored signature Sx(I) is formed using one or more M-bit descriptors Bx(to), Bx(t0+\), Bx(to+2), Bx(to+3). For example, a 32-bit monitored signature Sx(t) includes four 8-bit descriptors Bx(to), Bx(to+\), Bx(to+2), Bx(to+3), each of which is generated based on a corresponding one of the audio sample frames 204, 206, 208, and 210. More specifically, each of the descriptors is generated using one or more intraframe operations (e.g., comparison operations) based on two or more spectral components that are uniquely associated with a single audio sample frame.
[0047] The four descriptors may be generated based on spectral decompositions (e.g., frequency decompositions or wavelet decompositions) of the audio sample frames 204, 206, 208, and 210 as described in detail below in connection with FIGS. 4-7. The spectral decompositions are used to extract features that are uniquely characteristic of the example monitored audio stream 202. In this manner, a reference signature and a monitored signature that are generated based on the same audio stream using the same signature generation method (e.g., the same spectral decomposition-based method) will include similar features, and, thus can be used to reliably identify the monitored audio stream 202 using a matching algorithm. A monitored signature may be generated by sampling the example monitored audio stream 202 to generate the audio sample frames 204, 206, 208, and 210, generating the descriptors Bx(to), Bx(to+\), Bx(to+2), Bx(to+3) based on spectral decompositions of the audio sample frames 204, 206, 208, and 210, and concatenating the descriptors.
[0048] The audio sample frames 204, 206, 208, and 210 are generated by sampling the example monitored audio stream 202 during four time intervals at a sampling
Figure imgf000014_0001
For^anfple^a11 S1Mi )H^ freqtiMc'y/f of 6000 Hz will generate 6000 samples of audio data for each of the audio sample frames 204, 206, 208, and 210 (assuming the sample frames are collected over one second intervals). However, any other suitable sampling frequency /÷ may instead be selected. As shown in FIG. 2, the duration of each of the audio sample frames 204, 206, 208, and 210 is one second (e.g., 0 to t0, to to to+l, to+1 to to+2, and to+2 to to+3). However, the duration may instead be set to any other length of time. The times within the monitored audio stream 202 during which monitored signatures are generated are substantially similar or identical to the times within a reference audio stream during which corresponding reference signatures are generated. By acquiring audio sample frames for monitored signatures and reference signatures at substantially the same times, the features extracted from a monitored audio stream (e.g., the example monitored audio stream 202) and a corresponding reference audio stream (e.g., the example reference audio stream 302 of FIG. 3) are substantially similar or identical. Although, the audio sample frames 204, 206, 208, and 210 are shown in FIG. 2 as occurring consecutively in time, the audio sample frames 204, 206, 208, and 210 may occur at any time and in any sequence within the example monitored audio stream 202.
[0049] To ensure that the monitored signature is compared with a reference signature that is generated at substantially the same time within respective audio streams, the monitored signatures may be generated relative to a reference time that is used during the signature comparison process to align a monitored signature with a reference signature. More specifically, during the generation of a monitored signature, the example monitored audio stream 202 is sampled starting at a time indicated by a reference time to, which may be selected relative to a time stamp embedded within the example audio stream 202, a system startup time, a daily recurring time (e.g., midnight), and/or any other reference time that may be indicative of the time at which a signature is generated. A signature matching system (e.g., the signature analyzer 132 of FIGS. IA and IB) uses the reference time to to retrieve onlr'όl-
Figure imgf000016_0001
to substantially the same time within reference audio streams as indicated by the reference time to.
[0050] Additionally, one or more monitored signatures may be generated using the example monitored audio stream 202 so that multiple signatures are matched to identify the example monitored audio stream 202. For example, it is possible that one or more monitored signatures generated using the example monitored audio stream 202 are substantially similar or identical to one or more reference signatures of a reference audio stream (e.g., the example reference audio stream 302 of FIG. 3) that does not correspond to the example monitored audio stream 202. In this case, to decrease the possibility of erroneously identifying the wrong reference audio stream, more than one monitored signature is generated for the monitored audio stream 202. More specifically, the signatures may be generated at multiple times throughout the example monitored audio stream 202. In addition, a comparison algorithm may be configured to match two or more monitored signatures with corresponding reference signatures to accurately identify the example monitored audio stream 202.
[0051] FIG. 3 is a time-domain representation 300 of an example reference audio stream 302 and a plurality of audio sample frames 304, 306, 308, and 310 that may be acquired from the example reference audio stream 302. The time-domain representation 300 shows two one second time intervals (e.g., 0 to to and to to to+l) and the plurality of audio sample frames 304, 306, 308, and 310 that are collected during the time intervals and that are subsequently used to generate a plurality of reference signatures that are staggered in time (i.e., time shifted relative to one another). The example reference audio stream 302 is sampled at a sampling frequency /÷ to collect the audio sample frames 304, 306, 308, and 310, which are used to generate M-bit descriptors Bϋn{t). One or more of the descriptors BRn{i) may then be concatenated to form an N-bit reference signature Snn(Jt). Typically, the number of descriptors in a reference signature is equal to the number of descriptors in a monitored signature. Additionally, each of the M-bit descriptors BRJJ) includes the same number of bits as
Figure imgf000017_0001
ΘeMfpbr x(ή associated with the example monitored audio stream 202 (FIG. 2) and, thus, each N-bit reference signature SRn(I) includes the same number of bits as a corresponding N-bit monitored signature Sx(ή.
[0052] Multiple reference signatures are generated for the example reference audio stream 302 in a manner that causes the reference signatures to be time-shifted relative to each other and to overlap in time with one another. More specifically, the start time at which a first audio sample frame (e.g., the audio sample frame 304) of a first reference signature is
collected is offset, or shifted, by an amount of time from the start time at which a first
Figure imgf000017_0002
audio sample frame (e.g., the audio sample frame 308) of a second reference signature is collected. The value Ts is associated with a number of equal segments ("sample segments") into which each time interval (e.g., to to to+l) is divided. For example, if the number of sample segments Ts in a time interval of one second is set equal to thirty, the collection of a
new data set will begin every th of a second. In addition, if the sampling frequency ^ is
set equal to 6000 Hz, each sample segment will include 200 samples.
[0053] Each audio sample frame is used to generate a single signature. More specifically, an audio sample frame is collected using a predetermined number of sample segments (e.g., the sample segments 312a-312e), which are concatenated to form the audio sample frame (e.g., the audio sample frame 304). To achieve overlap among consecutively generated signatures, each signature is formed using an audio sample frame that partially overlaps with an audio sample frame used to form the previously generated signature. More specifically, two audio sample frames that overlap include a common set of sample segments. For example, the audio sample frames 304 and 308 include a common set of sample segments comprising the sample segments 312b-312e. The audio sample frame 308 may be formed by extracting a common plurality of media samples or the common set of sample segments 312b-312e from the audio sample frame 304 and appending a recently acquired sample common set of sample segments 312b-312e. In addition, two audio sample frames that overlap also contain sample segments that are exclusive to one or the other of the audio sample frames (i.e., audio sample frames that overlap also contain sample segments that occur in one or the other of the audio sample frames but do not occur in both frames).
[0054] To further illustrate the generation of reference signatures that are time shifted and k overlapped, each reference signature is formed by four reference descriptors BRn(t0+ — ),
k k k
BRn(to+ — +1), BRn(to+ — +2), Bϋn{tϋ+ — +3). The four reference descriptors are separated
^s ^s * s
by one-second time intervals and are shifted by — seconds with respect to the reference time
Ts
to, where 0 < k < Ts . For example, the audio sample frames 304 and 306 (generated at
to+ — and to+ — +1), respectively, in combination with two other audio sample frames ^s Ts
generated at tQ+ — +2 and /0+ — +3 (not shown), are used to generate four descriptors that Ts Ts
form a first reference signature. Additionally, the audio sample frames 308 and 310
(generated at (to+ — ) and (tø+ — +1)), respectively, are used with two other audio sample Ts Ts
frames generated at (to+ — +2) and (J0+ — +3) (not shown) to generate four descriptors that
Ts Ts
form a second reference signature that is time shifted relative to the first reference signature
by — seconds. As described below in connection with FIG. 5, each of the reference Ts
descriptors is generated using one or more operations (e.g., comparison operations) based on two or more spectral components that are both associated with the same audio sample frame. These operations are referred to as intraframe operations because they are performed using
Figure imgf000019_0001
are not dependent on sample data collected over other audio frames.
[0055] During an example sample acquisition process, the sample segments 312a-312e are collected and are used to form the first audio sample frame 304, which is subsequently used to determine a descriptor. This descriptor is then used to form part of a first reference signature. Then, a new sample segment 312f is collected and a second audio sample frame 308 is formed using sample segments 312b-312f. The second audio sample frame 308 is then used to determine a reference descriptor which is used to form part of a second reference signature. Thus, the first and second audio sample frames 304 and 308 include a common set of sample segments 312b-312e and each of the first and second audio sample frames additionally include a sample segment not included in the other (i.e., the first audio sample frame 304 includes sample segment 312a and the second audio sample frame 308 includes sample segment 312f). In this manner, the first and second reference signatures are generated using data collected at points in the audio stream that are staggered or shifted in time by
— seconds and are generated using data that overlaps. In addition, the amount by which the
first and second signatures are shifted can be adjusted by changing the value of Ts thereby permitting the resolution of the signatures to vary as desired. Specifically, if a set of signatures that represents an audio stream with a greater resolution is desired, Ts can be increased accordingly. Likewise, if less resolution is desired, Ts can be decreased. As will be appreciated by one having ordinary skill in the art, the value of Ts may affect the quantity of signatures that can be generated for a given audio stream. For example, if the number of sample segments used to form a signature remains constant, a larger value of Ts will result in forming more audio sample frames than a smaller value of Ts. Therefore, the amount of memory available to store the signature data may be a factor in determining the desired value of T5. [00^]- « ■ ■'f&fά&MMbi aβo e rror er to identify the title of a song or the title of a program associated with a particular audio stream, a set of monitored signatures generated for a monitored audio stream are compared to a database of reference signatures associated with a plurality of reference audio streams. In one example system, a signature generator (e.g., the signature generator 114 of FIG. IA) may be configured to monitor the audio emitted by a specific television (e.g., one of the presentation devices 112 of FIG. IA) located in the home of a specific panelist. Signatures are generated for the audio emitted by the television in the manner described above. In addition, the time at which the audio corresponding to each signature was emitted is recorded and stored as a reference time to with the corresponding monitored signature in a memory device. A timing device (e.g., the timing device 903 of FIGS. 9 and 10) located at the monitoring site may be used to trigger the collection of the audio data for the subsequent generation of monitored signatures and to provide the corresponding data collection times (e.g., reference times to, timestamps, etc.) to the memory for storage with the corresponding signature. To enable the subsequent identification of the emitted audio, a reference site (e.g., the reference site 104 of FIG. IA) located within the same broadcast region as a monitoring site (e.g., the monitoring site 102 of FIG. IA) is configured to generate reference signatures corresponding to the audio broadcast on all television broadcast channels at all times of the day. A timing device (e.g., the timing device 903 of FIGS. 9 and 10) located at the reference site may be used to trigger the collection of
the audio data at a set of equally spaced intervals corresponding to the time period — for the
subsequent generation of the reference signatures and to provide a corresponding set of data collection times to a reference memory (e.g., the memory 126 of FIG. IA) for storage. Thus, each reference signature is stored in a memory device located at the reference site along with timestamp data indicating the time at which the underlying audio data was collected. In one example, the timing devices (e.g., the timing device 903 of FIG. 9) located at the monitoring site and the reference site are synchronized such that the timestamps can be used to align the ( reference* signMilrel ϊthfhV Mϊtdred signatures to facilitate the signature matching process. Likewise, because a plurality of monitoring sites are likely to be located in the same broadcast region as a single reference site, in the same example, each of the timing devices 903 located in each such monitoring site may be synchronized with the timing device 903 located in the single reference site. However, due to the staggered arrangement of the reference signatures described above in connection with FIG. 3, the timing devices 903 at the monitoring site and the reference site do not have to be synchronized.
[0057] To compensate for offsets between the timing devices located at the monitoring sites and the reference site, the value of T5 may be adjusted. The value of Ts is generally selected to incrementally time shift a reference signature from a previous reference signature so that a monitored signature generated at an arbitrary reference time is highly likely to align with one of the staggered or time-shifted reference signatures. More specifically, increasing the value of Ts causes the number of sample segments (e.g., the sample segments 312a-312f) to increase and the offset or time shift from one reference signature to the next reference signature to decrease. This, in turn, increases the likelihood that the times at which reference signatures are generated for a given audio stream correspond to substantially similar or identical reference times at which monitored signatures are generated for the same audio stream. Signatures generated at the same times for the same program are expected to be identical or at least similar enough to cause a match to be detected. Thus, increasing the value of Ts increases the likelihood of a match between a set of reference signatures and a set of monitored signatures corresponding to the same audio program. Additionally, assuming the timing devices located at the reference site and the monitoring site are synchronized with sufficient precision, a monitored signature generated for data collected at a time T need only be compared to each reference signature associated with the same timestamp T instead of all reference signatures generated during the same twenty-four hour period. This reduction in comparisons reduces the processing time required to find a match. Similarly, assuming there is
Figure imgf000022_0001
devices located at a monitoring site and a reference site, each monitored signature generated at the monitoring site at a time T need only be compared to all reference signatures generated from data collected within a window of the time spanning from T - E to T + E.
[0058] In another example system (e.g., the example audio identification system 150 of FIG. IB), a set of monitored signatures are generated for a monitored audio stream and then compared to a database of reference signatures associated with a set of reference audio streams that, ideally, represent the universe of currently available audio streams. For example, as described above, in connection with FIG. IB, reference signatures corresponding to reference audio streams may be stored in a database that is stored in, for example, the memory 134. For each reference signature that is matched to a monitored signature, the matching information and/or the reference signature may be used to retrieve identification information (e.g., song title, song track, artist, etc.) from the database. The identification information is then used to identify the monitored audio stream. In one example, reference times to or timestamps associated with each monitored signature may be used to identify the time (of day) at which the monitored audio streams were broadcast.
[0059] FIG. 4 is a flow diagram of an example method for generating digital spectral signatures based on spectral decompositions. In particular, the example method of FIG. 4 may be used to generate digital spectral signatures (e.g., reference signatures and/or monitored signatures) based on frequency decomposition methods using a sliding Fast Fourier transform (FFT). As is known by one having ordinary skill in the art, an FFT may be used to convert a time domain signal (e.g., the example audio streams 202 and 302 of FIGS. 2 and 3) into a frequency domain representation of the same signal which may then be used to analyze the frequency components of the converted signal.
[0060] As will be appreciated by one having ordinary skill in the art, a sliding FFT provides advantages over a conventional non-sliding FFT for generating the digital spectral sigMliirά/'llTΪlift^i-Sb^^ntϊyitϊ^ϊ-Ωόli-sliding FFT, a sliding FFT can be used to incrementally compute an FFT. For example, one example approach to processing the audio streams 202 and 302 involves generating FFT data for each audio sample frame independent of any data associated with previous audio sample frames. In contrast, a sliding FFT involves generating FFT data for an audio sample frame by updating the FFT data generated in connection with a previous audio sample frame. Updating the previous frame's FFT data is less computationally expensive than generating FFT data anew for each frame causing the sliding FFT technique to be more efficient than the non-sliding conventional FFT approach. Additionally, the number of samples forming each audio sample frame (e.g., the audio sample frames 204, 206, 208, and 210 of FIG. 2 or 304, 306, 308, and 310 of FIG. 3) need not be a power of two, as is required of the non-sliding FFT approach. Thus, when using a sliding FFT, the digital spectral signatures can be generated using audio sample frames of any arbitrary size (i.e., any number of samples) that are acquired using any sampling
Figure imgf000023_0001
[0061] Now turning in detail to the example method of FIG. 4, initially the example method involves obtaining an audio stream (block 402) (e.g., the example monitored audio stream 202 of FIG. 2 or the example reference audio stream 302 of FIG. 3). A reference time to described above in connection with FIGS. 2 and 3 is determined (block 404) to indicate the time within an audio stream at which a signature is generated. An initial audio sample set is then obtained (block 406). The audio samples may be obtained by sampling an analog audio stream at a sampling frequency /j and performing an analog-to-digital conversion. Alternatively, the audio samples may be obtained by extracting or acquiring samples from a digital audio stream at a sampling
Figure imgf000023_0002
The initial audio sample set may be a complete audio sample frame (e.g., one of the audio sample frames 204, 206, 208, and 210 of FIG. 2 or 304, 306, 308, and 310 of FIG. 3) or a portion thereof. An initial FFT operation is performed on the initial audio sample set to establish an initial frequency spectrum (block 408JVLTlleiny!n!i8'^f^erfόrffliri^% FFT is well known in the art and, thus, is not discussed in detail herein.
[0062] After the initial frequency spectrum is determined (block 408), a next set of audio samples is obtained (block 410). The sliding FFT may then be used to update the initial frequency spectrum (generated at block 408) based on two most recently collected samples
vw _2 and vN _, according to Equation 1 below.
r .i / r rh r rl / r .iv ( ι2nJ(2)λ ( ( UmI(N s -2) fi, μjx exptø,μ]) = αoμjxexptø>oμj)x exp — + vWj _2 x exp -^
( (HnJ(N3 -I)X] ( ( /2/_J(2)V| ( ( i HπJ
+ \ v» , x exp ^^ - vn xexp — - v, xexp —
L "'-' \ 6000 )) { ° \ Ns JJ [ ' \ N,
Equation 1
Equation 1 may be used to update the frequency spectrum of an audio sample frame having a
sample quantity Ns. The spectral amplitude <zo[J] and phase value φ0 [J] form the existing
frequency spectrum αo|/]x expføo[/])> which includes the frequencies indexed by the
frequency index J. When the two most recently collected audio samples v^ _2 and vN _, are
obtained, the existing frequency spectrum αo[J]x expføo[J]) may be updated to determine a
new frequency spectrum α, [J] x expfø [J]). The two most recently collected audio samples
vΛ _2 and vN _, are inserted into the audio sample frame to replace the two earliest collected
samples v0 and v, .
[0063] As shown in Equation 1, the updated frequency spectrum O1[JJx expfø, [J]) is
determined using one or more multiplication operations, addition operations, and subtraction
operations based on complex exponents, the two earliest collected samples v0 and v, , and the
two most recently collected samples v^ _2 and v^ _, . Initially, the existing frequency
spectrum αo[J]x expføo[J]) is multiplied by a first complex exponential value if-M,.;:.J --'' (UP 11 :::|1 -'■ ll;"" s "' - exp . The product of the multiplication is added to a product determined by
multiplying the first most recently collected audio sample vN 2 by a second complex
exponential value . The result is then added to a product determined by
Figure imgf000025_0001
multiplying the second most recently collected audio sample v^ _, by a third complex
exponential value . The first earliest collected audio sample v0 is then
Figure imgf000025_0002
and subtracted from the
Figure imgf000025_0003
previous addition result. The second earliest collected audio sample v, is then multiplied by
( '2 Λ a fourth complex exponential value exp and subtracted from the previous
Figure imgf000025_0004
subtraction result.
[0064] It is well known in the art that instabilities such as, for example, oscillation or data overflow can be substantially minimized when implementing a sliding FFT by multiplying
most recently collected audio samples (e.g., the most recently collected audio samples v^ _2
and vNi _, ) by a first stability factor 5/1 and earliest collected audio samples (e.g., the earliest
collected audio samples v0 and V1 ) by a second stability factor sfy. The first stability factor
5/1 may be set equal to a value as close as possible to one. In the case of an audio sample frame having 6000 samples, the first stability factor sf\ may be set equal to 0.99995. The second stability factor sfr may be set equal to
Figure imgf000025_0005
where the value/? is equal to the number of sample shifts required to process an audio sample frame using the sliding FFT. For example, a two-sample shift is required to update an audio sample frame based on the two
most recently collected audio samples vN _2 and vN _, . In the case of the audio sample
frame having 6000 samples, the value p may be set equal to 3000. 11
[Oό'όW IS • λltferl!th 'sϊidiffg FFT ϊs' determined or calculated at block 412, it is determined if a complete audio sample frame (e.g., one of the audio sample frames 204, 206, 208, and 210 of FIG. 2 or one of the audio sample frames 304, 306, 308, and 310 of FIG. 3) has been obtained (block 414). At the monitoring sites 102 (FIG. IA) and 152 (FIG. IB), a complete audio sample frame is obtained when a plurality of most recently collected Ns samples is obtained. For example, if an audio sample frame includes 6000 samples, a complete audio sample frame is obtained after 6000 new samples are obtained. At the reference site 104 (FIG. IA) or the central data collection facility 154 (FIG. IB), a complete audio sample frame is obtained when a most recently collected sample segment (e.g., one of the sample segments 312a-312f of FIG. 3) is obtained and a current audio sample frame is formed as described in greater detail above in connection with FIG. 3. If it is determined at block 414 that a complete audio sample frame has not been obtained, control is passed back to block 410. However, if it is determined at block 414 that a complete audio sample frame has been obtained, a descriptor is generated (block 416). An example method for generating descriptors based on frequency components is described in greater detail below in connection with FIG. 5.
[0066] It is then determined if a complete descriptor set has been obtained (block 418). A descriptor set includes a predetermined number of descriptors that are used to form a signature. For example, if a 32-bit signature is formed by 8-bit descriptors, then a descriptor set includes four descriptors. If it is determined at block 418 that a complete descriptor set has not been obtained, control is passed back to block 410. However, if it is determined at block 418, that a complete descriptor set has been obtained, a digital spectral signature is generated by concatenating the descriptors of the descriptor set (block 420). After the digital spectral signature is generated (block 420), it is determined if another signature is to be generated (block 422). If another signature is to be generated, control is passed back to block 404. [OUiif lf ΦMr^'i i'fl'olrdla ffeBf an example method for generating descriptors associated with the example method of FIG. 4. In particular, the example method of FIG. 5 may be used to implement block 416 of FIG. 4. An M-bit descriptor is generated by selecting M pairs of frequency components^ that are uniquely associated with an audio sample frame and determining each bit of the descriptor based on intraframe comparisons of the spectral powers Pn, of the frequency components fib- The frequency components^ and the spectral powers Pib are indexed by a frequency component index / and a bit index b, where
0 < / < /indexmax and O ≤ Zx M .
[0068] The example method initially selects a first pair of frequency componentsyoo and /10 (block 502). Although consecutive frequency components are selected (i.e., yub and/ϊb>./2b and fa,, etc.) in the example method, the frequency components may be selected from any location in the frequency spectrum of an audio sample frame (e.g., one of the audio sample frames 204, 206, 208, and 210 of FIG. 2 or one of the audio sample frames 304, 306, 308, and 310 of FIG. 3). However, the frequency component indexes / used to select pairs of frequency components for generating a monitored signature are the same frequency component indexes / used to select pairs of frequency components for generating a corresponding reference signature.
[0069] After the first pair of frequency components T00 and /^0 is selected, the spectral powers Poo and P\o corresponding to the selected frequency components are determined (block 504). One of ordinary skill in the art will readily appreciate that the spectral power for each frequency component can be obtained based on the results of the sliding FFT performed at block 412 of FIG. 4.
[0070] A descriptor bit is determined based on the frequency components^oo and/j0 by comparing the first spectral power P00 with the second spectral power P\o (block 506). If the first spectral power is greater than the second spectral power (i.e., P00 > Pio), the descriptor
Figure imgf000028_0001
'Inste f the!' first spectral power is less than or equal to the second spectral power (i.e.,
Figure imgf000028_0002
the descriptor bit is set equal to zero.
[0071] It is then determined if another descriptor bit is to be determined (block 508). If another descriptor bit is to be determined, another pair of frequency components is selected (e.g., f2\ and β\) (block 510) and control is passed back to block 504. If, instead, another descriptor bit is not to be determined, the example method of FIG. 5 may be stopped.
[0072] FIG. 6 is a flow diagram of another example method for generating digital spectral signatures based on spectral decompositions. In particular, the example method of FIG. 6 may be used to generate digital spectral signatures (e.g., reference signatures and monitored signatures) based on wavelet decompositions of audio sample frames (e.g., the audio sample frames 204, 206, 208, and 210 of FIG. 2 or 304, 306, 308, and 310 of FIG. 3) using wavelet transforms. As described above, wavelet transforms may be employed to analyze data using different scales and/or resolutions by separating blocks or frames of data (e.g., the audio sample frames 204, 206, 208, 210, 304, 306, 308, and 310) into multiple sub-bands.
[0073] Initially, the example method obtains an audio stream (block 602) (e.g., the example monitored audio stream 202 of FIG. 2 or the example reference audio stream 302 of FIG. 3). A reference time t0 described above in connection with FIGS. 2 and 3 is determined (block 604) to indicate the time within an audio stream at which a signature is generated. An audio sample frame is then obtained (block 606). The audio sample frame may be obtained by sampling an analog audio stream at a sampling frequency^ and performing an analog-to- digital conversion. Alternatively, the audio sample frame may be obtained by extracting or acquiring samples from a digital audio stream at a sampling
Figure imgf000028_0003
Based on the Nyquist Theorem, aliasing is avoided by sampling the audio samples at frequencies ranging
/ from zero to — *- . 2
Figure imgf000029_0001
on wavelet decomposition performed using a wavelet transform. An example method for generating descriptors based on one or more wavelet decompositions is described in greater detail below in connection with FIG. 7.
[0075] After the descriptor is generated, it is determined if a complete descriptor set has been obtained (block 610). If a complete descriptor set has not been obtained, a next audio sample frame is obtained (block 612) and control is passed back to block 608. However, if a complete descriptor set has been obtained, a digital spectral signature is generated (block 614) by concatenating the descriptors of the descriptor set. After the digital spectral signature is generated, it is determined if another signature is to be generated (block 616). If another signature is to be generated, control is passed back to block 604. Otherwise, the example method is stopped.
[0076] FIG. 7 is a flow diagram of an example method for generating descriptors associated with the example method of FIG. 6. In particular, the example method of FIG. 7 may be used to implement block 608 of FIG. 6. An M-bit descriptor is generated by performing an M-level wavelet decomposition on an audio sample frame (e.g., one of the audio sample frames 204, 206, 208, and 210 of FIG. 2 or one of the audio sample frames 304, 306, 308, and 310 of FIG. 3). For each level wavelet decomposition, the energy of the audio signal for each sub-band is determined and descriptors are generated based on comparisons of the sub-band energies. For each descriptor, the M-level wavelet decomposition is implemented as an intraframe operation that is performed on spectral energies that are uniquely associated with an audio sample frame. Additionally, the M-level wavelet decomposition may be implemented using any wavelet transform. For purposes of clarity, the example method of FIG. 7 is described in terms of the well-known Daubechies wavelet transform. [0(Hf if l! 4iMi:aϊly ' tn'e'ef knϊ'p i 'method performs a first-level wavelet decomposition (block 702) by applying the Daubechies wavelet transform to an audio sample frame. The first application of the Daubechies wavelet transform results in a low-frequency sub-band block of filtered values LQ and a high-frequency sub-band block of filtered values H0, each of which
N includes — -filtered values. 2
[0078] Turning in greater detail to the Daubechies wavelet transform implementation, the Daubechies coefficients CQ, C\, C2, and c3 are used to generate an Ns x Ns transformation matrix in which the coefficients are arranged as shown below.
Figure imgf000030_0001
[0079] The coefficients are ordered in the transformation matrix, as shown above, using two dominant patterns. The odd rows include the first pattern, which is an ordering of the coefficients that functions as a smoothing filter (e.g., similar to a moving filter). The even rows include the second pattern, which is an ordering of the coefficients that functions to bring out the details of data (e.g., the audio sample frame). The transformation matrix is first applied to the entire audio sample frame (e.g., all of the Ns samples) to generate filtered values that include low-frequency sub-band filtered values alternated with high-frequency sub-band filtered values. The values are de-interleaved to generate the two sub-band blocks
N LQ and H0, each of which includes — — samples. The low-frequency sub-band block L0
includes filtered values that are associated with sub-band frequencies ranging from zero to f !!" IL' W / 1U1 -.» ••'' i ι= •■»
— *- . The high-frequency sub-band block H0 includes filtered values that are associated with 4
sub-band frequencies ranging from -1- to .
N N [0080] An — — x — — transformation matrix of the Daubechies coefficients is then applied
to the low-frequency sub-band block LQ to generate two additional sub-band blocks L1 and H1. For each transformation, the number of filtered values in each sub-band block is halved. Additionally, for each transformation, a descriptor is generated based on a high-frequency sub-band block (e.g., H0, Hi, H2, etc.). Further details related to the implementation of wavelet transforms are well known in the art and are not described herein.
[0081] After the first-level wavelet transform is applied, the high-frequency sub-band block Ho is parsed by separating the filtered values into a first half and a second half (block 704). Next, at a block 706, a first energy value E0 is determined by squaring and summing the filtered values of the first half of the high-frequency sub-band block H0 and a second energy value Ei is also determined (block 706) by squaring and summing the filtered values of the second half of the high-frequency sub-band block H0.
[0082] A descriptor bit is determined by comparing the first energy value E0 with the second energy value Ei (block 708). For example, if the first energy value Eo is greater than the second energy value Ei (i.e., Eo > Ei), the first descriptor bit is set equal to one. If the first energy value Eo is less than or equal to the second energy value Ei (i.e., Eo≤Ei), the first descriptor bit is set equal to zero. It is then determined if another descriptor bit is to be determined (block 710). If another descriptor bit is to be determined, a next-level wavelet decomposition is performed (block 712). For example, as described above, if a second-level
TV N wavelet decomposition is performed, an — — x — — transformation matrix is applied to the
filtered values of the low-frequency sub-band block Z0 to determine filtered sub-band blocks U
Figure imgf000032_0001
710 that another descriptor bit is not to be determined, the example method of FIG. 7 may be stopped.
[0083] FIG. 8 is a flow diagram of an example method for comparing the digital spectral signatures (e.g., monitored signatures and reference signatures) generated using the example methods of FIGS. 4-7. In particular, the example method may be used to compare a monitored signature with a reference signature, both of which are generated based on a sliding FFT or a wavelet transform. In general, the example method of FIG. 8 may be used to match a monitored signature with a reference signature by comparing the monitored signature with a plurality of reference signatures. Identification information (e.g., channel, program title, episode number, etc.) associated with a matching reference signature may then be retrieved from, for example, a database and used to generate media ratings information. The comparisons may be performed by comparing any number of bits from a monitored signature with the same number of bits from a reference signature such as, for example, a bit-by-bit comparison, a byte-by-byte comparison, a word-by-word comparison, etc. Due to the large number of reference signatures available for comparison, a Hamming distance may be used for the comparisons to eliminate mismatches rapidly, thereby significantly decreasing the time required to compare a monitored signature with the reference signatures.
[0084] As is known to one of ordinary skill in the art, a Hamming distance between two values may be identified by determining how many bits, numbers, characters, etc. need to be changed to make the two values equal. For example, a first binary value of 0110 and a second binary value of 0101 have a Hamming distance of two because bit location zero and bit location one need to be changed to make the first binary value equal to the second binary value.
[0085] Now turning in detail to the example method of FIG. 8, the example method involves first obtaining a monitored signature (block 802). A reference time J0 for the monitored signature is then obtained (block 804). A first reference signature corresponding to
Figure imgf000033_0001
indicated by the reference time t0 is obtained from, for example, the memory 134 of FIGS. IA and IB (block 806). The first descriptor of the monitored signature and the first descriptor of the reference signature are then obtained (block 808).
[0086] The first descriptor of the monitored signature is compared to the first descriptor of the reference signature to determine if the descriptors match (block 810). If a match is detected, it is determined if all of the descriptors of the monitored signature and the reference signature have been compared (block 812). If it is determined at block 812 that all of the descriptors have not been compared, the next descriptors are obtained from the monitored signature and the reference signature (block 816) and control is passed back to block 810. If it is determined at block 812 that all of the descriptors have been compared, the monitored audio stream is identified based on the matching reference signature (block 814). Alternatively, the example method may be implemented so that multiple monitored signatures of a single audio stream need to be matched to multiple signatures of a reference audio stream prior to identifying the monitored audio stream.
[0087] If it is determined at block 810 that the descriptors do not match, then it is determined if all of the reference signatures has been compared (block 818). If all of the reference signatures have not been compared, the next reference signature is obtained (block 820) and control is passed to block 808. However, if it is determined at block 818 that all of the reference signatures have been compared, the media, channel, radio station, etc. associated with the monitored audio stream may be unidentifiable and the example method is stopped. A flag may be set to indicate that the monitored audio stream is unidentifiable.
[0088] Although, the example method of FIG. 8 is described as comparing one reference signature at a time, the example method can be adapted to compare multiple reference signatures with one monitored signature in parallel (i.e., at the same time). For example, the operation of block 806 may be configured to obtain a plurality of reference signatures at one tinfe*
Figure imgf000034_0001
reference audio stream. The operation of block 808 may be configured to obtain the first descriptor of each of the plurality of reference signatures retrieved at block 806. The descriptors of each of the reference signatures may be compared with the each descriptor of the monitored signature until a match is found or until the plurality of reference signatures obtained at block 806 is eliminated, after which time another plurality of reference signatures may be obtained at block 820.
[0089] One of ordinary skill in the art can readily appreciate that applying the Hamming distance to the comparison process may significantly reduce the time required to match all of the available reference signatures. For example, after the first descriptor of each of a plurality of reference signatures are compared to the first descriptor of a monitored signature, the first descriptor of each of the reference signatures is associated with a Hamming distance. Only reference signatures having first descriptors associated with a Hamming distance less than a predetermined Hamming distance threshold are further compared with the monitored signature based on the next descriptor of each of the reference signatures and the monitored signature. Reference signatures having descriptors associated with a Hamming distance greater than a predetermined threshold are discarded. The number of reference signatures to be compared based on the next descriptors is reduced from the number of reference signatures compared to the monitored signature based on the first descriptor. In this manner, with each iteration of the comparison process, the number of reference signatures that remain to be compared in subsequent iterations of the signature comparison process quickly diminishes until it is determined that all of the descriptors of a single reference signature are associated with a Hamming distance below the predetermined Hamming distance threshold.
[0090] In instances where all of the descriptors of more than one reference signature are associated with a Hamming distance below the predetermined Hamming distance threshold, more than one monitored signature may need to be matched with respective reference signatures of the possible matching reference audio streams. It will be relatively unlikely that all
Figure imgf000035_0001
on the monitored audio stream will match all of the reference signatures of more than one reference audio stream, and, thus erroneously matching more than one reference audio stream to the monitored audio stream can be prevented.
[0091] The example methods described above in connection with FIGS. 4-8 may be implemented by hardware, software, and/or any combination thereof. More specifically, the example methods may be executed in hardware defined by the block diagrams of FIGS. 9-11. The example methods may also be implemented by software executed on a processor system such as, for example, the processor system 1210 of FIG. 12.
[0092] FIG. 9 is a block diagram of an example signature generation system 900 for generating digital spectral signatures. In particular, the example signature generation system 900 may be used to generate monitored signatures and/or reference signatures based on a sliding FFT as described above in connection with the example methods of FIGS. 4 and 5. For example, the example signature generation system 900 may be used to implement the signature generators 114 and 122 of FIG. IA or the signature generators 156 and 158 of FIG. IB. Additionally, the example signature generation system 900 may be used to implement the example methods of FIGS. 4 and 5.
[0093] As shown in FIG. 9, the example signature generation system 900 includes a sample generator 902, a timing device 903, a reference time generator 904, a sliding FFT module 906, a frequency identifier 908, a spectral power value identifier 910, a comparator 912, a descriptor generator 914, a concatenator 916, and a data communication interface 918, all of which may be communicatively coupled as shown. The example signature generation system 900 may be configured to obtain an example audio stream 920, acquire a plurality of audio samples from the example audio stream 920, and generate digital spectral signatures based on the audio samples. _ , lf ■'ttέiynpii gen'eiitlόrl|9b2'may be configured to obtain the example audio stream 920, which may be any analog or digital audio stream. If the example audio stream 920 is an analog audio stream, the sample generator 902 may be implemented using an analog-to- digital converter. If the example audio stream 920 is a digital audio stream, the sample generator 902 may be implemented using a digital signal processor. Additionally, the sample generator 902 may be configured to acquire and/or extract audio samples at any desired sampling frequency^ and notify the reference time generator 904 when an audio sample acquisition process begins. The sample generator 902 communicates samples to the sliding FFT module 906. The sample generator 902 may also be configured to notify the frequency identifier 908 when an audio sample frame (e.g., one of the audio sample frames 204, 206, 208, and 210 of FIG. 2 or 304, 306, 308, and 310 of FIG. 3) or a sample segment (e.g., one of the sample segments 312a-f of FIG. 3) has been generated.
[0095] The timing device 903 may be configured to generate time data and/or timestamp information and may be implemented by a clock, a timer, a counter, and/or any other suitable device. The timing device 903 may be communicatively coupled to the reference time generator 904 and may be configured to communicate time data and/or timestamps to the reference time generator 904. The timing device 903 may also be communicatively coupled to the sample generator 902 and may assert a start signal or interrupt to instruct the sample generator 902 to begin collecting or acquiring audio sample data. In one example, the timing device 903 may be implemented by a real-time clock having a 24-hour period that tracks time at a resolution of milliseconds. In this case, the timing device 903 may be configured to reset to zero at midnight and track time in milliseconds with respect to midnight.
[0096] The reference time generator 904 may initialize a reference time t0 when a notification is received from the sample generator 902. As described above in connection with FIGS. 2 and 3, the reference time to may be used to indicate the time within an audio stream at which a signature is generated. In particular, the reference time generator 904 may be i fe £δbtømg£Wateerøά\anMiimeKάl£tM/or a timestamp value from the timing device 903 when notified of the beginning of a sample acquisition process by the sample generator 902. The reference time generator 904 may then store the timestamp value as the reference time to.
[0097] The sliding FFT module 906 may be configured to perform a sliding FFT using the audio samples obtained from the sample generator 902. As described above in connection with FIG. 4, a sliding FFT may update frequency spectrum data each time two
samples (e.g., the two most recently acquired samples v^ _2 and vN _, ) are obtained from the
sample generator 902.
[0098] The frequency identifier 908 may be configured to identify one or more frequency pairs from frequency spectrum data in response to a notification from the sample generator 902 that a new audio sample frame or a new sample segment has been generated. For example, if the example signature generation system 900 is configured to generate monitored signatures, the frequency identifier 908 identifies frequency pairs from the frequency spectrum data in response to a new audio sample frame notification. Alternatively, if the example signature generation system 900 is configured to generate reference signatures, an audio sample frame of data is formed with each new sample segment as described above in connection with FIG. 3. Therefore, the frequency identifier 908 identifies frequency pairs from the frequency spectrum data in response to a new sample segment notification. The frequency identifier 908 may then be configured to communicate indexes identifying the frequency components of the frequency pairs to the spectral power value identifier 910.
[0099] The spectral power value identifier 910 may be configured to obtain the indexes associated with the frequency components of the frequency pairs from the frequency identifier 908. The spectral power value identifier 910 may then determine or identify the spectral power of each frequency component of the frequency pairs by retrieving the spectral power value for each frequency component from the frequency spectrum data generated by _,
ΛeNiJdiJg^H^!!ϊadl/9δ6.1:iΩectra1 power values may then be communicated to the comparator 912.
[00100] As described above in connection with FIG. 5, the comparator 912 and the descriptor generator 914 may work cooperatively to generate M-bit descriptors. The comparator 912 may obtain the spectral power values and compare the spectral power values for each frequency pair. The descriptor generator 914 may be configured to obtain comparison results from the comparator 912 and generate the descriptor bits of an M-bit descriptor based on the comparison results.
[00101] The concatenator 916 may obtain descriptor values from the descriptor generator 914. When a complete set of descriptors is obtained, the concatenator 916 may concatenate the descriptors 916 to form a digital spectral signature. The data communication interface 918 may obtain the digital spectral signatures from the concatenator 916 and the reference time to corresponding to the digital spectral signature and communicate the same to a memory and/or a reference site. For example, if the example signature generation system 900 is configured to generate monitored signatures at the monitoring site 102 (FIG. 1 A), the monitored signatures may be communicated to the central data collection facility (FIG. IA) via the network 108 (FIG. IA). Alternatively, if the example signature generation system 900 is configured to generate reference signatures, the reference signatures may be communicated to the central data collection facility 154 (FIG. IB) and/or stored in the memory 134 (FIG. IB).
[00102] FIG. 10 is a block diagram of another example signature generation system 1000 for generating digital signatures based on audio streams. In particular, the example signature generation system 1000 may be used to generate monitored signatures and/or reference signatures based on wavelet transforms as described above in connection with the example methods of FIGS. 6 and 7. For example, the example signature generation system 1000 may be used to implement the signature generators 114 and 122 of FIG. IA and generate
Figure imgf000039_0001
alternatively, the example signature generation system 1000 may be used to implement the signature generators 156 and 158 of FIG. IB. In addition, the example signature generation system 1000 may be used to implement the example methods of FIGS. 6 and 7.
[00103] The example signature generation system 1000 includes the sample generator 902, the timing device 903, the reference time generator 904, the comparator 912, the descriptor generator 914, the concatenator 916, and the data communication interface 918 of the example signature generation system 900 described above in connection with FIG. 9. Additionally, the example signature generation system 1000 includes a wavelet transform module 1002, a sub-band block identifier 1004, and an energy value generator 1006, all of which may be communicatively coupled as shown.
[00104] The wavelet transform module 1002 may be configured to apply wavelet transforms to audio samples obtained from the sample generator 902. For example, the wavelet transform module 1002 may obtain an audio sample frame (e.g., one of the audio sample frames 204, 206, 208, and 210 of FIG. 2 or 304, 306, 308, and 310 of FIG. 3) from the sample generator 902 and perform an M-level wavelet decomposition on the audio samples to generate filtered data values using, for example, the Daubechies wavelet transform as described in connection with FIGS. 6 and 7. The filtered data values may then be communicated to the sub-band block identifier 1004.
[00105] The sub-band block identifier 1004 may be configured to obtain the filtered data values from the wavelet transform module 1002 and generate a low-frequency sub-band block Lx and a high-frequency sub-band block Hx. As described in greater detail above in connection with FIG. 7, the sub-band blocks Lx and Hx may be identified by de-interleaving the filtered data values. The low-frequency sub-band block may then be communicated to the wavelet transform module 1002 to perform another wavelet decomposition and the high- frequency sub-band filtered block may be communicated to the energy value generator 1006. [0OiOd] ϊ
Figure imgf000040_0001
1006 may be configured to generate energy values Ex based on the high-frequency sub-band block. The energy value generator 1006 may be configured to parse or separate the high-frequency sub-band block into a first half of filtered data values and a second half of filtered data values as described in greater detail above in connection with FIG. 7. The energy value generator 1006 may then generate a first energy value E0 by squaring and summing the first half of filtered data values. A second energy value Ei may be generated by squaring and summing the second half of filtered data values.
[00107] The comparator 912 and the descriptor generator 914 may be configured to generate descriptors based on energy values. For example, the comparator 912 may obtain energy values from the energy value generator 1006 and compare a first energy to a second energy value. The descriptor generator 914 may obtain comparison results from the comparator 912 and generate the bits of an M-bit descriptor based on the comparison results.
[00108] The concatenator 916 may obtain descriptors from the descriptor generator 914 and generate digital spectral signatures by concatenating a plurality of descriptors as described above in connection with FIG. 9. The data communication interface 918 may then store or transmit signatures obtained from the concatenator 916 with corresponding reference times obtained from the reference time generator 904.
[00109] FIG. 11 is a block diagram of an example signature comparison system 1100 for comparing digital spectral signatures. In particular, the example signature comparison system 1100 may be used to compare monitored signatures with reference signatures. For example, the example signature comparison system 1100 may be used to implement the signature analyzer 132 of FIG. 1 to compare monitored signatures with reference signatures. Additionally, the example signature comparison system 1100 may be used to implement the example method of FIG. 8.
[00110] The example signature comparison system 1100 includes a monitored signature receiver 1102, a reference signature receiver 1104, a comparator 1106, a Hamming distance
Figure imgf000041_0001
media identification look-up table interface 11 12, all of which may be communicatively coupled as shown.
[00111] The monitored signature receiver 1102 may be configured to obtain monitored signatures via the network 106 (FIG. 1) and communicate the monitored signatures to the comparator 1106. The reference signature receiver 1104 may be configured to obtain reference signatures from the memory 134 (FIGS. IA and IB) and communicate the reference signatures to the comparator 1106.
[00112] The comparator 1106 and the Hamming distance filter 1108 may be configured to compare reference signatures to monitored signatures using Hamming distances. In particular, the comparator 1106 may be configured to compare descriptors of monitored signatures with descriptors from a plurality of reference signatures and to generate Hamming distance values for each comparison. The Hamming distance filter 1108 may then obtain the Hamming distance values from the comparator 1106 and filter out non-matching reference signatures based on the Hamming distance values as described above in connection with FIG. 8.
[00113] After a matching reference signature is found, the media identifier 1110 may obtain the matching reference signature and in cooperation with the media identification look-up table interface 1112 may identify the media information associated with an unidentified audio stream (e.g., the example monitored audio stream 202 of FIG. 2). For example, the media identification look-up table interface 1112 may be communicatively coupled to a media identification look-up table or a database that is used to cross-reference media identification information (e.g., movie title, show title, song title, artist name, episode number, etc.) based on reference signatures. In this manner, the media identifier 1110 may retrieve media identification information from the media identification database based on the matching reference signatures. ^
[0(JH'i'4] 8 ■ lFl<!yr';illiisk'b c¥( ϊ aτn of an example processor system 1210 that may be used to implement the apparatus and methods described herein. As shown in FIG. 12, the processor system 1210 includes a processor 1212 that is coupled to an interconnection bus or network 1214. The processor 1212 includes a register set or register space 1216, which is depicted in FIG. 12 as being entirely on-chip, but which could alternatively be located entirely or partially off-chip and directly coupled to the processor 1212 via dedicated electrical connections and/or via the interconnection network or bus 1214. The processor 1212 may be any suitable processor, processing unit or microprocessor. Although not shown in FIG. 12, the system 1210 may be a multi -processor system and, thus, may include one or more additional processors that are identical or similar to the processor 1212 and that are communicatively coupled to the interconnection bus or network 1214. [00115] The processor 1212 of FIG. 12 is coupled to a chipset 1218, which includes a memory controller 1220 and an input/output (I/O) controller 1222. As is well known, a chipset typically provides I/O and memory management functions as well as a plurality of general purpose and/or special purpose registers, timers, etc. that are accessible or used by one or more processors coupled to the chipset. The memory controller 1220 performs functions that enable the processor 1212 (or processors if there are multiple processors) to access a system memory 1224 and a mass storage memory 1225.
[00116] The system memory 1224 may include any desired type of volatile and/or non¬ volatile memory such as, for example, static random access memory (SRAM), dynamic random access memory (DRAM), flash memory, read-only memory (ROM), etc. The mass storage memory 1225 may include any desired type of mass storage device including hard disk drives, optical drives, tape storage devices, etc.
[00117] The I/O controller 1222 performs functions that enable the processor 1212 to communicate with peripheral input/output (I/O) devices 1226 and 1228 via an I/O bus 1230. The I/O devices 1226 and 1228 may be any desired type of I/O device such as, for example, a kefb'lardVa' fey la/or ffitoir; a mouse, etc. While the memory controller 1220 and the I/O controller 1222 are depicted in FIG. 12 as separate functional blocks within the chipset 1218, the functions performed by these blocks may be integrated within a single semiconductor circuit or may be implemented using two or more separate integrated circuits.
[00118] The methods described herein may be implemented using instructions stored on a computer readable medium that are executed by the processor 1212. The computer readable medium may include any desired combination of solid state, magnetic and/or optical media implemented using any desired combination of mass storage devices (e.g., disk drive), removable storage devices (e.g., floppy disks, memory cards or sticks, etc.) and/or integrated memory devices (e.g., random access memory, flash memory, etc.).
[00119] Although certain methods, apparatus, and articles of manufacture have been described herein, the scope of coverage of this patent is not limited thereto. To the contrary, this patent covers all methods, apparatus, and articles of manufacture fairly falling within the scope of the appended claims either literally or under the doctrine of equivalents.

Claims

1. A method for generating signatures, comprising: obtaining a first frame of media samples; identifying a first frequency component having a first spectral power and a second frequency component having a second spectral power by performing a spectral transform operation on the first frame of media samples; determining a first descriptor of the first frame of media samples based on a comparison of the first spectral power and the second spectral power; and generating a first signature based on the first descriptor.
2. A method as defined in claim 1 , further comprising identifying media information based on the first signature.
3. A method as defined in claim 2, wherein a Hamming distance is used to identify the media information based on the first signature.
4. A method as defined in claim 2, wherein the media information is associated with at least one of audio information and video information.
5. A method as defined in claim 1, wherein the first descriptor is associated with only the first frame of media samples.
6. A method as defined in claim 1 , further comprising: obtaining a first plurality of media samples; and identifying a second frame of media samples by extracting a common plurality of media samples from the first frame of media samples and appending the first plurality of media samples to the common plurality of media samples. P CTI ' /
Figure imgf000045_0001
Es »"έ:dih claim 6, fiirther comprising: identifying a third spectral power associated with a third frequency component and a fourth spectral power associated with a fourth frequency component, wherein the third frequency component and the fourth frequency component are associated with performing the spectral transform operation on the second frame of media samples; determining a second descriptor based on a comparison of the third spectral power and the fourth spectral power; and generating a second signature based on the second descriptor.
8. A method as defined in claim 7, wherein the second descriptor is of the second frame of media samples.
9. A method as defined in claim 1 , wherein the spectral transform operation is a sliding Fast Fourier Transform.
10. A method as defined in claim 1, further comprising determining a reference time value associated with the first frame of media samples.
11. A method as defined in claim 1 , wherein the first signature is at least one of a reference signature and a monitored signature.
12. An apparatus for generating signatures, comprising: a processor system including a memory; and instructions stored in the memory that enable the processor system to: obtain a first frame of media samples; identify a first frequency component having a first spectral power and a second frequency component having a second spectral power by performing a spectral transform operation on the first frame of media samples; determine a first descriptor of the first frame of media samples based on a comparison of the first spectral power and the second spectral power; and generate a first signature based on the first descriptor. P L" IIB/
Figure imgf000046_0001
in claim 12, wherein the instructions stored in the memory enable the processor system to identify media information based on the first signature.
14. An apparatus as defined in claim 13, wherein a Hamming distance is used to identify the media information based on the first signature.
15. An apparatus as defined in claim 13, wherein the media information is associated with at least one of audio information and video information.
16. An apparatus as defined in claim 12, wherein the first descriptor is associated with only the first frame of media samples.
17. An apparatus as defined in claim 12, wherein the instructions stored in the memory enable the processor system to: obtain a first plurality of media samples; and identify a second frame of media samples by extracting a common plurality of media samples from the first frame of media samples and appending the first plurality of media samples to the common plurality of media samples.
18. An apparatus as defined in claim 17, wherein the instructions stored in the memory enable the processor system to: identify a third spectral power associated with a third frequency component and a fourth spectral power associated with a fourth frequency component, wherein the third frequency component and the fourth frequency component are associated with performing the spectral transform operation on the second frame of media samples; determine a second descriptor based on a comparison of the third spectral power and the fourth spectral power; and generate a second signature based on the second descriptor.
19. An apparatus as defined in claim 18, wherein the second descriptor is of the second frame of media samples. iP lC 2Ib:' ' y
Figure imgf000047_0001
in claim 12, wherein the spectral transform operation is a sliding Fast Fourier Transform.
21. An apparatus as defined in claim 12, wherein the instructions stored in the memory enable the processor system to determine a reference time value associated with the first frame of media samples.
22. An apparatus as defined in claim 12, wherein the first signature is at least one of a reference signature and a monitored signature.
23. A machine accessible medium having instructions stored thereon that, when executed, cause a machine to: obtain a first frame of media samples; identify a first frequency component having a first spectral power and a second frequency component having a second spectral power by performing a spectral transform operation on the first frame of media samples; determine a first descriptor of the first frame of media samples based on a comparison of the first spectral power and the second spectral power; and generate a first signature based on the first descriptor.
24. A machine accessible medium as defined in claim 23 having instructions stored thereon that, when executed, cause the machine to identify media information based on the first signature.
25. A machine accessible medium as defined in claim 24, wherein a Hamming distance is used to identify the media information based on the first signature.
26. A machine accessible medium as defined in claim 24, wherein the media information is associated with at least one of audio information and video information.
27. A machine accessible medium as defined in claim 23, wherein the first descriptor is associated with only the first frame of media samples. P C 2S/
Figure imgf000048_0001
as defined in claim 23 having instructions stored thereon that, when executed, cause the machine to: obtain a first plurality of media samples; and identify a second frame of media samples by extracting a common plurality of media samples from the first frame of media samples and appending the first plurality of media samples to the common plurality of media samples.
29. A machine accessible medium as defined in claim 28 having instructions stored thereon that, when executed, cause the machine to: identify a third spectral power associated with a third frequency component and a fourth spectral power of a fourth frequency component, wherein the third frequency component and the fourth frequency component are associated with performing the spectral transform operation on the second frame of media samples; determine a second descriptor based on a comparison of the third spectral power and the fourth spectral power; and generate a second signature based on the second descriptor.
30. A machine accessible medium as defined in claim 23, wherein the second descriptor is of the second frame of media samples.
31. A machine accessible medium as defined in claim 23, wherein the spectral transform operation is a sliding Fast Fourier Transform.
32. A machine accessible medium as defined in claim 23, wherein the instructions stored in the memory enable the processor system to determine a reference time value associated with the first frame of media samples.
33. A machine accessible medium as defined in claim 23, wherein the first signature is at least one of a reference signature and a monitored signature. IP U3H/
Figure imgf000049_0001
ϊorl aMng signatures, comprising: identifying a first plurality of filtered values by performing a wavelet transform on a first frame of media samples; determining a first energy value based on a first portion of the first plurality of filtered values; determining a second energy value based on a second portion of the first plurality of filtered values; determining a first descriptor of the first frame of media samples based on a comparison of the first energy value and the second energy value; and generating a first signature based on the first descriptor.
35. A method as defined in claim 34, wherein the first plurality of filtered values is associated with a high-frequency sub-band of the first frame of media samples.
36. A method as defined in claim 34, wherein the first descriptor is associated with only the first frame of media samples.
37. A method as defined in claim 34, further comprising: obtaining a first plurality of media samples; and identifying a second frame of audio samples by extracting a common plurality of media samples from the first frame of media samples and appending the first plurality of media samples to the common plurality of media samples.
μ 1[.Ii3S8," U^ o-d Ss fiMStfn claim 37, further comprising: identifying a second plurality of filtered values by performing the wavelet transform on the second frame of media samples; determining a third energy value based on a first portion of the second plurality of frequency values; determining a fourth energy value based on a second portion of the second plurality of frequency values; determining a second descriptor based on a comparison of the third energy value and the fourth energy value; and generating a second signature based on the second descriptor.
39. A method as defined in claim 38, wherein the second descriptor is of the second frame of media samples.
40. A method as defined in claim 34, wherein the wavelet transform is based on a Daubechies wavelet transform.
41. A method as defined in claim 34, further comprising determining a reference time value associated with the first frame of media samples.
42. A method as defined in claim 34, further comprising identifying media information based on the first signature.
43. A method as defined in claim 42, wherein a Hamming distance is used to identify the media information based on the first signature.
44. A method as defined in claim 42, wherein the media is associated with at least one of audio information and video information.
45. A method as defined in claim 34, wherein the first signature is one of a reference signature and a monitored signature. P L- 46/
Figure imgf000051_0001
signatures, comprising: a processor system including a memory; and instructions stored in the memory that enable the processor system to: identify a first plurality of filtered values by performing a wavelet transform on a first frame of media samples; determine a first energy value based on a first portion of the first plurality of filtered values; determine a second energy value based on a second portion of the first plurality of filtered values; determine a first descriptor of the first frame of media samples based on a comparison of the first energy value and the second energy value; and generate a first signature based on the first descriptor.
47. An apparatus as defined in claim 46, wherein the first plurality of filtered values is associated with a high-frequency sub-band of the first frame of media samples.
48. An apparatus as defined in claim 46, wherein the first descriptor is associated with only the first frame of media samples.
49. An apparatus as defined in claim 46, wherein the instructions stored in the memory enable the processor system to: obtain a first plurality of media samples; and identify a second frame of audio samples by extracting a common plurality of media samples from the first frame of media samples and appending the first plurality of media samples to the common plurality of media samples. ]!■■■ L, ζjβs
Figure imgf000052_0001
in claim 49, wherein the instructions stored in the memory enable the processor system to: identity a second plurality of filtered values by performing the wavelet transform on the second frame of media samples; determine a third energy value based on a first portion of the second plurality of frequency values; determine a fourth energy value based on a second portion of the second plurality of frequency values; determine a second descriptor based on a comparison of the third energy value and the fourth energy value; and generate a second signature based on the second descriptor.
51. An apparatus as defined in claim 50, wherein the second descriptor is of the second frame of media samples.
52. An apparatus as defined in claim 46, wherein the wavelet transform is based on a Daubechies wavelet transform.
53. An apparatus as defined in claim 46, wherein the instructions stored in the memory enable the processor system to determine a reference time value associated with the first frame of media samples.
54. An apparatus as defined in claim 46, wherein the instructions stored in the memory enable the processor system to identify media information based on the first signature.
55. An apparatus as defined in claim 54, wherein a Hamming distance is used to identify the media information based on the first signature.
56. An apparatus as defined in claim 54, wherein the media is associated with at least one of audio information and video information.
57. An apparatus as defined in claim 46, wherein the first signature is one of a reference signature and a monitored signature. P' L-sϊ/
Figure imgf000053_0001
having instructions stored thereon that, when executed, cause a machine to: identify a first plurality of filtered values by performing a wavelet transform on a first frame of media samples; determine a first energy value based on a first portion of the first plurality of filtered values; determine a second energy value based on a second portion of the first plurality of filtered values; determine a first descriptor of the first frame of media samples based on a comparison of the first energy value and the second energy value; and generate a first signature based on the first descriptor.
59. A machine accessible medium as defined in claim 58, wherein the first plurality of filtered values is associated with a high-frequency sub-band of the first frame of media samples.
60. A machine accessible medium as defined in claim 58, wherein the first descriptor is associated with only the first frame of media samples.
61. A machine accessible medium as defined in claim 58 having instructions stored thereon that, when executed, cause the machine to: obtain a first plurality of media samples; and identify a second frame of audio samples by extracting a common plurality of media samples from the first frame of media samples and appending the first plurality of media samples to the common plurality of media samples.
Figure imgf000054_0001
as defined in claim 61 having instructions stored thereon that, when executed, cause the machine to: identify a second plurality of filtered values by performing the wavelet transform on the second frame of media samples; determine a third energy value based on a first portion of the second plurality of frequency values; determine a fourth energy value based on a second portion of the second plurality of frequency values; determine a second descriptor based on a comparison of the third energy value and the fourth energy value; and generate a second signature based on the second descriptor.
63. A machine accessible medium as defined in claim 62, wherein the second descriptor is of the second frame of media samples.
64. A machine accessible medium as defined in claim 58, wherein the wavelet transform is based on a Daubechies wavelet transform.
65. A machine accessible medium as defined in claim 58 having instructions stored thereon that, when executed, cause the machine to determine a reference time value associated with the first frame of media samples.
66. A machine accessible medium as defined in claim 58 having instructions stored thereon that, when executed, cause the machine to identify media information based on the first signature.
67. A machine accessible medium as defined in claim 66, wherein a Hamming distance is used to identify the media information based on the first signature.
68. A machine accessible medium as defined in claim 66, wherein the media is associated with at least one of audio information and video information. P i:φ/
Figure imgf000055_0001
defined in claim 58, wherein the first signature is one of a reference signature and a monitored signature.
70. A method for generating signatures, comprising: obtaining a first plurality of time domain media samples; determining frequency spectrum information based on the first plurality of time domain media samples; obtaining a second plurality of time domain media samples; updating the frequency spectrum information based on the second plurality of time domain media samples; identifying a first frequency component and a second frequency component based on the frequency spectrum information; determining a first spectral power value associated with the first frequency component and a second spectral power value associated with the second frequency component; determining a descriptor bit value based on a comparison of the first spectral power value and the second spectral power value; and generating a signature based on the first descriptor bit value.
71. A method as defined in claim 70, wherein a Hamming distance is used to identify media information based on the signature.
72. A method as defined in claim 71, wherein the media information is associated with at least one of audio information and video information.
73. A method as defined in claim 70, further comprising determining a reference time value associated with the first plurality of time domain media samples.
74. A method as defined in claim 70, wherein the signature is at least one of a reference signature and a monitored signature. " '" 75. Ah apparatus for generating signatures, compπsing: a processor system including a memory; and instructions stored in the memory that enable the processor system to: obtain a first plurality of time domain media samples; determine frequency spectrum information based on the first plurality of time domain media samples; obtain a second plurality of time domain media samples; update the frequency spectrum information based on the second plurality of time domain media samples; identify a first frequency component and a second frequency component based on the frequency spectrum information; determine a first spectral power value associated with the first frequency component and a second spectral power value associated with the second frequency component; determine a descriptor bit value based on a comparison of the first spectral power value and the second spectral power value; and generate a signature based on the first descriptor bit value.
76. An apparatus as defined in claim 75, wherein a Hamming distance is used to identify media information based on the signature.
77. An apparatus as defined in claim 76, wherein the media information is associated with at least one of audio information and video information.
78. An apparatus as defined in claim 75, wherein the instructions stored in the memory enable the processor system to determine a reference time value associated with the first plurality of time domain media samples.
79. An apparatus as defined in claim 75, wherein the signature is at least one of a reference signature and a monitored signature. IP IL §ό/
Figure imgf000057_0001
stored thereon that, when executed, cause a machine to: obtain a first plurality of time domain media samples; determine frequency spectrum information based on the first plurality of time domain media samples; obtain a second plurality of time domain media samples; update the frequency spectrum information based on the second plurality of time domain media samples; identify a first frequency component and a second frequency component based on the frequency spectrum information; determine a first spectral power value associated with the first frequency component and a second spectral power value associated with the second frequency component; determine a descriptor bit value based on a comparison of the first spectral power value and the second spectral power value; and generate a signature based on the first descriptor bit value.
81. A machine accessible medium as defined in claim 80, wherein a Hamming distance is used to identify media information based on the signature.
82. A machine accessible medium as defined in claim 81 , wherein the media information is associated with at least one of audio information and video information.
83. A machine accessible medium as defined in claim 80 having instructions stored thereon that, when executed, cause a machine to determine a reference time value associated with the first plurality of time domain media samples.
84. A machine accessible medium as defined in claim 80, wherein the signature is at least one of a reference signature and a monitored signature. IP C$/ ILi y pkaffi :ig'eW§rating signatures, comprising: a sliding Fast Fourier Transform module for generating frequency spectrum information associated with a plurality of media samples; a frequency identifier configured to identify a plurality of frequency components in the frequency spectrum information; a spectral power value identifier configured to identify a plurality of spectral power values based on the plurality of frequency components; a descriptor generator configure to generate a plurality of descriptors based on comparisons of the plurality of spectral power values; and a concatenator configured to generate a plurality of digital spectral signatures by concatenating two or more of the plurality of descriptors.
86. An apparatus as defined in claim 85, wherein the frequency identifier is configured to identify at least one of the plurality of frequency components in response to at least one of a new media sample frame notification and a new sample segment notification.
87. An apparatus as defined in claim 85, wherein at least some of the plurality of media samples form a plurality of media sample frames.
88. An apparatus as defined in claim 87, wherein the descriptor generator is configured to generate at least one of the plurality of descriptors based on at least one of the plurality of media sample frames.
89. An apparatus as defined in claim 87, wherein a first one of the plurality of media sample frames includes at least some of the plurality of media samples that form a portion of a second one of the plurality of media sample frames.
90. An apparatus as defined in claim 85, wherein each of the plurality of digital spectral signatures is at least one of a monitored signature and a reference signature. P 9J/ VfotyføM geae ύng signatures, comprising: a wavelet transform module for generating a plurality of filtered values associated with a plurality of media samples; a sub-band block identifier for identifying a plurality of low-frequency sub-band blocks and a plurality of high-frequency sub-band blocks based on the plurality of media samples; an energy value generator for generating a plurality of energy values based on the plurality of sub-band blocks; a descriptor generator for generating a plurality of descriptors based on comparisons of the plurality of energy values; and a concatenator for generating a plurality of digital spectral signatures by concatenating two or more of the plurality of descriptors.
92. An apparatus as defined in claim 91, wherein the wavelet transform module performs a Daubechies wavelet transform based on the plurality of media samples.
93. An apparatus as defined in claim 91, wherein the plurality of media samples form a plurality of media sample frames.
94. An apparatus as defined in claim 93, wherein the descriptor generator generates at least one of the plurality of descriptors based on at least one of the plurality of media sample frames.
95. An apparatus as defined in claim 93, wherein a first set of the plurality of media sample frames includes at least some of the plurality of media samples that form a portion of a second set of the plurality of media sample frames.
96. An apparatus as defined in claim 91 , wherein the plurality of digital spectral signatures are at least one of a monitored signature and a reference signature. 11 ' 97. An apparatus as denned in claim 91 , wherein the low- frequency sub-band block and the high-frequency sub-band block are identified by de-interleaving at least some of the plurality of filtered values.
98. An apparatus as defined in claim 91 , wherein the wavelet transform module is configurable to perform a multiple-level wavelet decomposition based on the filtered values.
99. An apparatus as defined in claim 98, wherein the plurality of descriptors are generated based on the multiple-level wavelet decomposition.
100. A method for generating signatures, comprising: obtaining a first frame of media samples; identifying a first frequency having a first spectral power by performing a spectral transform operation on the first frame of media samples; identifying a second frequency having a second spectral power based on the spectral transform operation; determining a first descriptor of the first frame of media samples based on a comparison of the first spectral power and the second spectral power; and generating a first signature based on the first descriptor.
101. A method as defined in claim 100, wherein the first descriptor is associated with only the first frame of media samples.
102. A method as defined in claim 100, further comprising: obtaining a first plurality of media samples; and identifying a second frame of media samples by extracting a common plurality of media samples from the first frame of media samples and appending the first plurality of media samples to the common plurality of media samples. ^ L 1 bϊ. lf JIMetldd ES »§αin claim 102, further comprising: identifying a third spectral power associated with a third frequency and a fourth spectral power associated with a fourth frequency, wherein the third frequency and the fourth frequency are associated with performing the spectral transform operation on the second frame of media samples; determining a second descriptor based on a comparison of the third spectral power and the fourth spectral power; and generating a second signature based on the second descriptor.
104. A method as defined in claim 103, wherein the second descriptor is of the second frame of media samples.
105. A method as defined in claim 100, wherein the spectral transform operation is a sliding Fast Fourier Transform.
106. An apparatus for generating signatures, comprising: a processor system including a memory; and instructions stored in the memory that enable the processor system to: obtain a first frame of media samples; identify a first frequency having a first spectral power by performing a spectral transform operation on the first frame of media samples; identify a second frequency having a second spectral power based on the spectral transform operation; determine a first descriptor of the first frame of media samples based on a comparison of the first spectral power and the second spectral power; and generate a first signature based on the first descriptor. IP- C uM<ip$aϊaBsii"ΑEM£d in claim 106, wherein the first descriptor is associated with only the first frame of media samples.
108. An apparatus as defined in claim 106, wherein the instructions stored in the memory enable the processor system to: obtain a first plurality of media samples; and identify a second frame of media samples by extracting a common plurality of media samples from the first frame of media samples and appending the first plurality of media samples to the common plurality of media samples.
109. An apparatus as defined in claim 108, wherein the instructions stored in the memory enable the processor system to: identify a third spectral power associated with a third frequency and a fourth spectral power associated with a fourth frequency, wherein the third frequency and the fourth frequency are associated with performing the spectral transform operation on the second frame of media samples; determine a second descriptor based on a comparison of the third spectral power and the fourth spectral power; and generate a second signature based on the second descriptor.
110. An apparatus as defined in claim 109, wherein the second descriptor is of the second frame of media samples.
111. An apparatus as defined in claim 106, wherein the spectral transform operation is a sliding Fast Fourier Transform. IH* "--1% 2". ^"''^''niacnϊnd^cC^ffile'niedium having instructions stored thereon that, when executed, cause a machine to: obtain a first frame of media samples; identify a first frequency having a first spectral power by performing a spectral transform operation on the first frame of media samples; identify a second frequency having a second spectral power based on the spectral transform operation; determine a first descriptor of the first frame of media samples based on a comparison of the first spectral power and the second spectral power; and generate a first signature based on the first descriptor.
113. A machine accessible medium as defined in claim 112, wherein the first descriptor is associated with only the first frame of media samples.
114. A machine accessible medium as defined in claim 112 having instructions stored thereon that, when executed, cause the machine to: obtain a first plurality of media samples; and identify a second frame of media samples by extracting a common plurality of media samples from the first frame of media samples and appending the first plurality of media samples to the common plurality of media samples.
1 ,
P Lr1 |5;J ^βp ^ff jsi aeάixaa as defined in claim 114 having instructions
stored thereon that, when executed, cause the machine to: identify a third spectral power associated with a third frequency and a fourth spectral power associated with a fourth frequency, wherein the third frequency and the fourth frequency are associated with performing the spectral transform operation on the second frame of media samples; determine a second descriptor based on a comparison of the third spectral power and the fourth spectral power; and generate a second signature based on the second descriptor.
116. A machine accessible medium as defined in claim 115, wherein the second descriptor is of the second frame of media samples.
117. A machine accessible medium as defined in claim 112, wherein the spectral transform operation is a sliding Fast Fourier Transform.
PCT/US2005/029623 2004-08-18 2005-08-18 Methods and apparatus for generating signatures WO2006023770A2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
MX2007002071A MX2007002071A (en) 2004-08-18 2005-08-18 Methods and apparatus for generating signatures.
US11/676,452 US7783889B2 (en) 2004-08-18 2007-02-19 Methods and apparatus for generating signatures
US12/822,777 US8489884B2 (en) 2004-08-18 2010-06-24 Methods and apparatus for generating signatures

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US60302404P 2004-08-18 2004-08-18
US60/603,024 2004-08-18

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US11/676,452 Continuation US7783889B2 (en) 2004-08-18 2007-02-19 Methods and apparatus for generating signatures

Publications (2)

Publication Number Publication Date
WO2006023770A2 true WO2006023770A2 (en) 2006-03-02
WO2006023770A3 WO2006023770A3 (en) 2008-04-10

Family

ID=35968228

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2005/029623 WO2006023770A2 (en) 2004-08-18 2005-08-18 Methods and apparatus for generating signatures

Country Status (3)

Country Link
US (2) US7783889B2 (en)
MX (1) MX2007002071A (en)
WO (1) WO2006023770A2 (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008008911A2 (en) 2006-07-12 2008-01-17 Arbitron Inc. Methods and systems for compliance confirmation and incentives
US8060372B2 (en) * 2007-02-20 2011-11-15 The Nielsen Company (Us), Llc Methods and appratus for characterizing media
US8458737B2 (en) 2007-05-02 2013-06-04 The Nielsen Company (Us), Llc Methods and apparatus for generating signatures
US8457951B2 (en) 2008-01-29 2013-06-04 The Nielsen Company (Us), Llc Methods and apparatus for performing variable black length watermarking of media
US8600531B2 (en) 2008-03-05 2013-12-03 The Nielsen Company (Us), Llc Methods and apparatus for generating signatures
AU2012211498B2 (en) * 2007-02-20 2014-09-11 The Nielsen Company (Us), Llc Methods and apparatus for characterizing media
AU2013203321B2 (en) * 2007-02-20 2015-01-15 The Nielsen Company (Us), Llc Methods and apparatus for characterizing media
US9460730B2 (en) 2007-11-12 2016-10-04 The Nielsen Company (Us), Llc Methods and apparatus to perform audio watermarking and watermark detection and extraction

Families Citing this family (91)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8959016B2 (en) 2002-09-27 2015-02-17 The Nielsen Company (Us), Llc Activating functions in processing devices using start codes embedded in audio
US9711153B2 (en) 2002-09-27 2017-07-18 The Nielsen Company (Us), Llc Activating functions in processing devices using encoded audio and detecting audio signatures
EP2456104A1 (en) 2003-02-10 2012-05-23 Nielsen Media Research, Inc. Methods and apparatus to adaptively gather audience measurement data
MX2007002071A (en) * 2004-08-18 2007-04-24 Nielsen Media Res Inc Methods and apparatus for generating signatures.
US11234029B2 (en) 2017-08-17 2022-01-25 The Nielsen Company (Us), Llc Methods and apparatus to generate reference signatures from streaming media
US7583684B2 (en) * 2005-08-24 2009-09-01 The Boeing Corporation Automatic commandable SSID switching
US20070288277A1 (en) 2005-12-20 2007-12-13 Neuhauser Alan R Methods and systems for gathering research data for media from multiple sources
EP2011002B1 (en) 2006-03-27 2016-06-22 Nielsen Media Research, Inc. Methods and systems to meter media content presented on a wireless communication device
MX2007015979A (en) 2006-03-31 2009-04-07 Nielsen Media Res Inc Methods, systems, and apparatus for multi-purpose metering.
US8090161B2 (en) * 2006-06-19 2012-01-03 Christiane Kaplan Systems and method for signature verification
US8588377B2 (en) * 2007-03-02 2013-11-19 Cisco Technology, Inc. Method and system for grouping voice messages
US7912894B2 (en) * 2007-05-15 2011-03-22 Adams Phillip M Computerized, copy-detection and discrimination apparatus and method
US8229227B2 (en) * 2007-06-18 2012-07-24 Zeitera, Llc Methods and apparatus for providing a scalable identification of digital video sequences
US8099757B2 (en) 2007-10-15 2012-01-17 Time Warner Cable Inc. Methods and apparatus for revenue-optimized delivery of content in a network
US8503991B2 (en) * 2008-04-03 2013-08-06 The Nielsen Company (Us), Llc Methods and apparatus to monitor mobile devices
US20100205628A1 (en) 2009-02-12 2010-08-12 Davis Bruce L Media processing methods and arrangements
US9986279B2 (en) 2008-11-26 2018-05-29 Free Stream Media Corp. Discovery, access control, and communication with networked services
US10880340B2 (en) 2008-11-26 2020-12-29 Free Stream Media Corp. Relevancy improvement through targeting of information based on data gathered from a networked device associated with a security sandbox of a client device
US10567823B2 (en) 2008-11-26 2020-02-18 Free Stream Media Corp. Relevant advertisement generation based on a user operating a client device communicatively coupled with a networked media device
US10631068B2 (en) 2008-11-26 2020-04-21 Free Stream Media Corp. Content exposure attribution based on renderings of related content across multiple devices
US10419541B2 (en) 2008-11-26 2019-09-17 Free Stream Media Corp. Remotely control devices over a network without authentication or registration
US9519772B2 (en) 2008-11-26 2016-12-13 Free Stream Media Corp. Relevancy improvement through targeting of information based on data gathered from a networked device associated with a security sandbox of a client device
US10334324B2 (en) 2008-11-26 2019-06-25 Free Stream Media Corp. Relevant advertisement generation based on a user operating a client device communicatively coupled with a networked media device
US9961388B2 (en) 2008-11-26 2018-05-01 David Harrison Exposure of public internet protocol addresses in an advertising exchange server to improve relevancy of advertisements
US9154942B2 (en) 2008-11-26 2015-10-06 Free Stream Media Corp. Zero configuration communication between a browser and a networked media device
US10977693B2 (en) 2008-11-26 2021-04-13 Free Stream Media Corp. Association of content identifier of audio-visual data with additional data through capture infrastructure
US8180891B1 (en) 2008-11-26 2012-05-15 Free Stream Media Corp. Discovery, access control, and communication with networked services from within a security sandbox
EP2406903A4 (en) * 2009-03-11 2013-01-16 Ravosh Samari Digital signatures
US8687839B2 (en) 2009-05-21 2014-04-01 Digimarc Corporation Robust signatures derived from local nonlinear filters
US10949458B2 (en) 2009-05-29 2021-03-16 Inscape Data, Inc. System and method for improving work load management in ACR television monitoring system
US9449090B2 (en) 2009-05-29 2016-09-20 Vizio Inscape Technologies, Llc Systems and methods for addressing a media database using distance associative hashing
US10116972B2 (en) 2009-05-29 2018-10-30 Inscape Data, Inc. Methods for identifying video segments and displaying option to view from an alternative source and/or on an alternative device
US10375451B2 (en) 2009-05-29 2019-08-06 Inscape Data, Inc. Detection of common media segments
US8595781B2 (en) 2009-05-29 2013-11-26 Cognitive Media Networks, Inc. Methods for identifying video segments and displaying contextual targeted content on a connected television
US9055309B2 (en) 2009-05-29 2015-06-09 Cognitive Networks, Inc. Systems and methods for identifying video segments for displaying contextually relevant content
US8813124B2 (en) 2009-07-15 2014-08-19 Time Warner Cable Enterprises Llc Methods and apparatus for targeted secondary content insertion
US20110041154A1 (en) * 2009-08-14 2011-02-17 All Media Guide, Llc Content Recognition and Synchronization on a Television or Consumer Electronics Device
US8677400B2 (en) 2009-09-30 2014-03-18 United Video Properties, Inc. Systems and methods for identifying audio content using an interactive media guidance application
US8161071B2 (en) 2009-09-30 2012-04-17 United Video Properties, Inc. Systems and methods for audio asset storage and management
US8245249B2 (en) 2009-10-09 2012-08-14 The Nielson Company (Us), Llc Methods and apparatus to adjust signature matching results for audience measurement
US9218530B2 (en) 2010-11-04 2015-12-22 Digimarc Corporation Smartphone-based methods and systems
US9197736B2 (en) * 2009-12-31 2015-11-24 Digimarc Corporation Intuitive computing methods and systems
US8175617B2 (en) 2009-10-28 2012-05-08 Digimarc Corporation Sensor-based mobile search, related methods and systems
US8121618B2 (en) 2009-10-28 2012-02-21 Digimarc Corporation Intuitive computing methods and systems
US8682145B2 (en) 2009-12-04 2014-03-25 Tivo Inc. Recording system based on multimedia content fingerprints
US8886531B2 (en) * 2010-01-13 2014-11-11 Rovi Technologies Corporation Apparatus and method for generating an audio fingerprint and using a two-stage query
US20110258211A1 (en) * 2010-04-18 2011-10-20 Kalisky Ofer System and method for synchronous matching of media samples with broadcast media streams
US10192138B2 (en) 2010-05-27 2019-01-29 Inscape Data, Inc. Systems and methods for reducing data density in large datasets
US9838753B2 (en) 2013-12-23 2017-12-05 Inscape Data, Inc. Monitoring individual viewing of television events using tracking pixels and cookies
US8863165B2 (en) * 2010-11-01 2014-10-14 Gracenote, Inc. Method and system for presenting additional content at a media system
US9484046B2 (en) 2010-11-04 2016-11-01 Digimarc Corporation Smartphone-based methods and systems
EP2692142A1 (en) * 2011-03-30 2014-02-05 Thomson Licensing Method for image playback verification
US8667283B2 (en) * 2011-05-09 2014-03-04 Verayo, Inc. Soft message signing
US9374183B2 (en) 2011-08-30 2016-06-21 Iheartmedia Management Services, Inc. Broadcast source identification based on matching via bit count
US8484170B2 (en) * 2011-09-19 2013-07-09 International Business Machines Corporation Scalable deduplication system with small blocks
US9223893B2 (en) 2011-10-14 2015-12-29 Digimarc Corporation Updating social graph data using physical objects identified from images captured by smartphone
US9402099B2 (en) 2011-10-14 2016-07-26 Digimarc Corporation Arrangements employing content identification and/or distribution identification data
AU2013203410B2 (en) * 2012-03-26 2015-09-24 The Nielsen Company (Us), Llc Media monitoring using multiple types of signatures
US8768003B2 (en) 2012-03-26 2014-07-01 The Nielsen Company (Us), Llc Media monitoring using multiple types of signatures
US9854280B2 (en) * 2012-07-10 2017-12-26 Time Warner Cable Enterprises Llc Apparatus and methods for selective enforcement of secondary content viewing
US9282366B2 (en) 2012-08-13 2016-03-08 The Nielsen Company (Us), Llc Methods and apparatus to communicate audience measurement information
US9226011B2 (en) * 2012-09-11 2015-12-29 Comcast Cable Communications, Llc Synchronizing program presentation
US9106953B2 (en) 2012-11-28 2015-08-11 The Nielsen Company (Us), Llc Media monitoring based on predictive signature caching
US9179185B2 (en) * 2013-03-14 2015-11-03 The Nielsen Company (Us), Llc Methods and apparatus to determine a number of people in an area
US9354778B2 (en) 2013-12-06 2016-05-31 Digimarc Corporation Smartphone-based methods and systems
US9955192B2 (en) 2013-12-23 2018-04-24 Inscape Data, Inc. Monitoring individual viewing of television events using tracking pixels and cookies
US9311639B2 (en) 2014-02-11 2016-04-12 Digimarc Corporation Methods, apparatus and arrangements for device to device communication
US9420349B2 (en) 2014-02-19 2016-08-16 Ensequence, Inc. Methods and systems for monitoring a media stream and selecting an action
US9699499B2 (en) 2014-04-30 2017-07-04 The Nielsen Company (Us), Llc Methods and apparatus to measure exposure to streaming media
US9497505B2 (en) 2014-09-30 2016-11-15 The Nielsen Company (Us), Llc Systems and methods to verify and/or correct media lineup information
US9704507B2 (en) 2014-10-31 2017-07-11 Ensequence, Inc. Methods and systems for decreasing latency of content recognition
US9747906B2 (en) 2014-11-14 2017-08-29 The Nielson Company (Us), Llc Determining media device activation based on frequency response analysis
MX2017007165A (en) 2014-12-01 2017-11-17 Inscape Data Inc System and method for continuous media segment identification.
CA2973740C (en) 2015-01-30 2021-06-08 Inscape Data, Inc. Methods for identifying video segments and displaying option to view from an alternative source and/or on an alternative device
US9680583B2 (en) 2015-03-30 2017-06-13 The Nielsen Company (Us), Llc Methods and apparatus to report reference media data to multiple data collection facilities
WO2016168556A1 (en) 2015-04-17 2016-10-20 Vizio Inscape Technologies, Llc Systems and methods for reducing data density in large datasets
MX2018000567A (en) 2015-07-16 2018-04-24 Inscape Data Inc Detection of common media segments.
BR112018000801A2 (en) 2015-07-16 2018-09-04 Inscape Data Inc system, and method
US10080062B2 (en) 2015-07-16 2018-09-18 Inscape Data, Inc. Optimizing media fingerprint retention to improve system resource utilization
MX2018000568A (en) 2015-07-16 2018-04-24 Inscape Data Inc Prediction of future views of video segments to optimize system resource utilization.
US9900636B2 (en) * 2015-08-14 2018-02-20 The Nielsen Company (Us), Llc Reducing signature matching uncertainty in media monitoring systems
US10586023B2 (en) 2016-04-21 2020-03-10 Time Warner Cable Enterprises Llc Methods and apparatus for secondary content management and fraud prevention
US10740728B1 (en) * 2016-06-28 2020-08-11 Innovate Care Llc Computer-based access security and verification
KR20190134664A (en) 2017-04-06 2019-12-04 인스케이프 데이터, 인코포레이티드 System and method for using media viewing data to improve device map accuracy
US10236005B2 (en) 2017-06-08 2019-03-19 The Nielsen Company (Us), Llc Methods and apparatus for audio signature generation and matching
US11252460B2 (en) 2020-03-27 2022-02-15 The Nielsen Company (Us), Llc Signature matching with meter data aggregation for media identification
US11088772B1 (en) 2020-05-29 2021-08-10 The Nielsen Company (Us), Llc Methods and apparatus to reduce false positive signature matches due to similar media segments in different reference media assets
US11736765B2 (en) 2020-05-29 2023-08-22 The Nielsen Company (Us), Llc Methods and apparatus to credit media segments shared among multiple media assets
US11523175B2 (en) 2021-03-30 2022-12-06 The Nielsen Company (Us), Llc Methods and apparatus to validate reference media assets in media identification system
US11894915B2 (en) 2021-05-17 2024-02-06 The Nielsen Company (Us), Llc Methods and apparatus to credit media based on presentation rate
US11689764B2 (en) 2021-11-30 2023-06-27 The Nielsen Company (Us), Llc Methods and apparatus for loading and roll-off of reference media assets

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6226387B1 (en) * 1996-08-30 2001-05-01 Regents Of The University Of Minnesota Method and apparatus for scene-based video watermarking
US6714683B1 (en) * 2000-08-24 2004-03-30 Digimarc Corporation Wavelet based feature modulation watermarks and related applications
US6839673B1 (en) * 1999-03-29 2005-01-04 Markany Inc. Digital watermarking method and apparatus for audio data
US7006555B1 (en) * 1998-07-16 2006-02-28 Nielsen Media Research, Inc. Spectral audio encoding

Family Cites Families (85)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4230990C1 (en) 1979-03-16 2002-04-09 John G Lert Jr Broadcast program identification method and system
US4450531A (en) 1982-09-10 1984-05-22 Ensco, Inc. Broadcast signal recognition system and method
US4697209A (en) * 1984-04-26 1987-09-29 A. C. Nielsen Company Methods and apparatus for automatically identifying programs viewed or recorded
US4677466A (en) * 1985-07-29 1987-06-30 A. C. Nielsen Company Broadcast program identification method and apparatus
US4739398A (en) * 1986-05-02 1988-04-19 Control Data Corporation Method, apparatus and system for recognizing broadcast segments
GB8630118D0 (en) 1986-12-17 1987-01-28 British Telecomm Speaker identification
US4843562A (en) 1987-06-24 1989-06-27 Broadcast Data Systems Limited Partnership Broadcast information classification system and method
US4931871A (en) 1988-06-14 1990-06-05 Kramer Robert A Method of and system for identification and verification of broadcasted program segments
US4945412A (en) 1988-06-14 1990-07-31 Kramer Robert A Method of and system for identification and verification of broadcasting television and radio program segments
US5019899A (en) * 1988-11-01 1991-05-28 Control Data Corporation Electronic data encoding and recognition system
JPH03173291A (en) 1989-11-15 1991-07-26 Control Data Corp <Cdc> Method for generating digital sign from video signal or tv transmitting frame
US5436653A (en) 1992-04-30 1995-07-25 The Arbitron Company Method and system for recognition of broadcast segments
US5721788A (en) * 1992-07-31 1998-02-24 Corbis Corporation Method and system for digital image signatures
US5437050A (en) 1992-11-09 1995-07-25 Lamb; Robert G. Method and apparatus for recognizing broadcast information using multi-frequency magnitude detection
US5630203A (en) 1993-01-12 1997-05-13 Weinblatt; Lee S. Technique for surveying a radio or a television audience
JP3173291B2 (en) 1994-07-22 2001-06-04 市光工業株式会社 Rearview mirror device for vehicles
US5485518A (en) 1993-09-30 1996-01-16 Yellowstone Environmental Science, Inc. Electronic media program recognition and choice
US5481294A (en) 1993-10-27 1996-01-02 A. C. Nielsen Company Audience measurement system utilizing ancillary codes and passive signatures
US6983051B1 (en) 1993-11-18 2006-01-03 Digimarc Corporation Methods for audio watermarking and decoding
US5748763A (en) * 1993-11-18 1998-05-05 Digimarc Corporation Image steganography system featuring perceptually adaptive and globally scalable signal embedding
ATE237197T1 (en) 1993-11-18 2003-04-15 Digimarc Corp IDENTIFICATION/CREDITION CODING METHOD AND APPARATUS
US5768426A (en) 1993-11-18 1998-06-16 Digimarc Corporation Graphics processing system employing embedded code signals
US5822436A (en) * 1996-04-25 1998-10-13 Digimarc Corporation Photographic products and methods employing embedded information
US6681029B1 (en) * 1993-11-18 2004-01-20 Digimarc Corporation Decoding steganographic messages embedded in media signals
US5581658A (en) 1993-12-14 1996-12-03 Infobase Systems, Inc. Adaptive system for broadcast program identification and reporting
US5450490A (en) * 1994-03-31 1995-09-12 The Arbitron Company Apparatus and methods for including codes in audio signals and decoding
AU709873B2 (en) 1994-03-31 1999-09-09 Arbitron Inc. Apparatus and methods for including codes in audio signals and decoding
US6873734B1 (en) * 1994-09-21 2005-03-29 Ricoh Company Ltd Method and apparatus for compression using reversible wavelet transforms and an embedded codestream
US6560349B1 (en) 1994-10-21 2003-05-06 Digimarc Corporation Audio monitoring using steganographic information
US7107451B2 (en) * 1996-07-02 2006-09-12 Wistaria Trading, Inc. Optimization methods for the insertion, protection, and detection of digital watermarks in digital data
US5918223A (en) 1996-07-22 1999-06-29 Muscle Fish Method and article of manufacture for content-based analysis, storage, retrieval, and segmentation of audio information
US6061793A (en) * 1996-08-30 2000-05-09 Regents Of The University Of Minnesota Method and apparatus for embedding data, including watermarks, in human perceptible sounds
US6002443A (en) * 1996-11-01 1999-12-14 Iggulden; Jerry Method and apparatus for automatically identifying and selectively altering segments of a television broadcast signal in real-time
US5909518A (en) 1996-11-27 1999-06-01 Teralogic, Inc. System and method for performing wavelet-like and inverse wavelet-like transformations of digital data
US5875122A (en) 1996-12-17 1999-02-23 Intel Corporation Integrated systolic architecture for decomposition and reconstruction of signals using wavelet transforms
US6675383B1 (en) * 1997-01-22 2004-01-06 Nielsen Media Research, Inc. Source detection apparatus and method for audience measurement
JP3938238B2 (en) 1997-02-04 2007-06-27 沖電気工業株式会社 Fast Fourier transform processor
CA2196930C (en) 1997-02-06 2005-06-21 Nael Hirzalla Video sequence recognition
JP3951071B2 (en) 1997-05-02 2007-08-01 ソニー株式会社 Arithmetic apparatus and arithmetic method
US5940135A (en) 1997-05-19 1999-08-17 Aris Technologies, Inc. Apparatus and method for encoding and decoding information in analog signals
ES2190578T3 (en) 1997-06-23 2003-08-01 Liechti Ag METHOD FOR THE COMPRESSION OF ENVIRONMENTAL NOISE RECORDINGS, METHOD FOR DETECTION OF THE SAME PROGRAM ELEMENTS, DEVICE AND COMPUTER PROGRAM FOR APPLICATION.
CA2269651A1 (en) * 1998-05-12 1999-11-12 Lucent Technologies, Inc. Transform domain image watermarking method and system
IL124595A (en) * 1998-05-21 2009-09-01 Yossef Tsuria Anti-piracy system
US6556689B1 (en) * 1998-05-29 2003-04-29 University Of Delaware Watermarking methods for digital images and videos
US6272176B1 (en) * 1998-07-16 2001-08-07 Nielsen Media Research, Inc. Broadcast encoding system and method
US6253182B1 (en) * 1998-11-24 2001-06-26 Microsoft Corporation Method and apparatus for speech synthesis with efficient spectral smoothing
US6366937B1 (en) 1999-03-11 2002-04-02 Hitachi America Ltd. System and method for performing a fast fourier transform using a matrix-vector multiply instruction
JP2000286836A (en) * 1999-03-30 2000-10-13 Fujitsu Ltd Certification device and recording medium
US6477269B1 (en) * 1999-04-20 2002-11-05 Microsoft Corporation Method and system for searching for images based on color and shape of a selected image
US7284255B1 (en) 1999-06-18 2007-10-16 Steven G. Apel Audience survey system, and system and methods for compressing and correlating audio signals
US6469749B1 (en) * 1999-10-13 2002-10-22 Koninklijke Philips Electronics N.V. Automatic signature-based spotting, learning and extracting of commercials and other video content
US6971010B1 (en) * 1999-10-15 2005-11-29 Koninklijke Philips Electronics N.V. Image and video authentication system
DE60045787D1 (en) * 1999-12-24 2011-05-12 Ibm Method and system for identifying identical digital data
US6799274B1 (en) * 2000-03-30 2004-09-28 Western Digital Ventures, Inc. Device comprising encryption circuitry enabled by comparing an operating spectral signature to an initial spectral signature
GB0029855D0 (en) * 2000-04-05 2001-01-24 Sony Uk Ltd Embedding data in material and removing the embedded data
JP4813001B2 (en) * 2000-04-08 2011-11-09 オラクル・アメリカ・インコーポレイテッド Media resynchronization during streaming
US6879652B1 (en) * 2000-07-14 2005-04-12 Nielsen Media Research, Inc. Method for encoding an input signal
US6574594B2 (en) 2000-11-03 2003-06-03 International Business Machines Corporation System for monitoring broadcast audio content
US6604072B2 (en) 2000-11-03 2003-08-05 International Business Machines Corporation Feature-based audio content identification
US7562012B1 (en) 2000-11-03 2009-07-14 Audible Magic Corporation Method and apparatus for creating a unique audio signature
AU2002232817A1 (en) * 2000-12-21 2002-07-01 Digimarc Corporation Methods, apparatus and programs for generating and utilizing content signatures
JP2002202787A (en) * 2000-12-28 2002-07-19 Sony Corp Contents data, recording medium, data recording method and apparatus, data transmission method and apparatus, data output method and apparatus, data reproducing method and apparatus, data receiving method and apparatus
WO2002065782A1 (en) 2001-02-12 2002-08-22 Koninklijke Philips Electronics N.V. Generating and matching hashes of multimedia content
US7711123B2 (en) * 2001-04-13 2010-05-04 Dolby Laboratories Licensing Corporation Segmenting audio signals into auditory events
ATE334465T1 (en) * 2001-05-08 2006-08-15 Koninkl Philips Electronics Nv WATERMARK
US6714594B2 (en) * 2001-05-14 2004-03-30 Koninklijke Philips Electronics N.V. Video content detection method and system leveraging data-compression constructs
US7000115B2 (en) * 2001-06-19 2006-02-14 International Business Machines Corporation Method and apparatus for uniquely and authoritatively identifying tangible objects
US8572640B2 (en) 2001-06-29 2013-10-29 Arbitron Inc. Media data use measurement with remote decoding/pattern matching
US6968337B2 (en) * 2001-07-10 2005-11-22 Audible Magic Corporation Method and apparatus for identifying an unknown work
AU2002346116A1 (en) * 2001-07-20 2003-03-03 Gracenote, Inc. Automatic identification of sound recordings
US7269338B2 (en) * 2001-12-11 2007-09-11 Koninklijke Philips Electronics N.V. Apparatus and method for synchronizing presentation from bit streams based on their content
US20030131350A1 (en) 2002-01-08 2003-07-10 Peiffer John C. Method and apparatus for identifying a digital audio signal
US7366909B2 (en) * 2002-04-29 2008-04-29 The Boeing Company Dynamic wavelet feature-based watermark
GB0211488D0 (en) * 2002-05-18 2002-06-26 Univ Aston Information embedding method
US7519819B2 (en) * 2002-05-29 2009-04-14 Digimarc Corporatino Layered security in digital watermarking
KR100479338B1 (en) * 2002-10-22 2005-03-31 한국전자통신연구원 Apparatus for verifying an online signature using of transform technique and method teherefor
US7483835B2 (en) * 2002-12-23 2009-01-27 Arbitron, Inc. AD detection using ID code and extracted signature
EP1579386A1 (en) * 2002-12-31 2005-09-28 France Telecom Method and device for detection of points of interest in a source digital image, corresponding computer programme and data support
EP1645136B1 (en) 2003-06-20 2017-07-05 Nielsen Media Research, Inc. Signature-based program identification apparatus and methods for use with digital broadcast systems
WO2005041109A2 (en) * 2003-10-17 2005-05-06 Nielsen Media Research, Inc. Methods and apparatus for identifiying audio/video content using temporal signal characteristics
US7120562B1 (en) * 2003-12-17 2006-10-10 L-3 Integrated Systems Company Signal source identification utilizing wavelet-based signal processing and associated method
US8943325B2 (en) * 2004-02-06 2015-01-27 Signoptic Technologies Savoie Technolac Use of a digital signature obtained from at least one structural characteristic of a material element to protect the direct reading of sensitive data and method for reading this protected data
US7221902B2 (en) * 2004-04-07 2007-05-22 Nokia Corporation Mobile station and interface adapted for feature extraction from an input media sample
FR2870376B1 (en) * 2004-05-11 2006-09-22 Yann Boutant METHOD FOR RECOGNIZING FIBROUS MEDIA, AND APPLICATIONS OF SUCH A METHOD IN THE COMPUTER FIELD, IN PARTICULAR
MX2007002071A (en) * 2004-08-18 2007-04-24 Nielsen Media Res Inc Methods and apparatus for generating signatures.

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6226387B1 (en) * 1996-08-30 2001-05-01 Regents Of The University Of Minnesota Method and apparatus for scene-based video watermarking
US7006555B1 (en) * 1998-07-16 2006-02-28 Nielsen Media Research, Inc. Spectral audio encoding
US6839673B1 (en) * 1999-03-29 2005-01-04 Markany Inc. Digital watermarking method and apparatus for audio data
US6714683B1 (en) * 2000-08-24 2004-03-30 Digimarc Corporation Wavelet based feature modulation watermarks and related applications

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008008915A2 (en) 2006-07-12 2008-01-17 Arbitron Inc. Methods and systems for compliance confirmation and incentives
WO2008008911A2 (en) 2006-07-12 2008-01-17 Arbitron Inc. Methods and systems for compliance confirmation and incentives
AU2012211498B2 (en) * 2007-02-20 2014-09-11 The Nielsen Company (Us), Llc Methods and apparatus for characterizing media
US8060372B2 (en) * 2007-02-20 2011-11-15 The Nielsen Company (Us), Llc Methods and appratus for characterizing media
US20120071995A1 (en) * 2007-02-20 2012-03-22 Alexander Topchy Methods and appratus for characterizing media
US8364491B2 (en) * 2007-02-20 2013-01-29 The Nielsen Company (Us), Llc Methods and apparatus for characterizing media
US8457972B2 (en) 2007-02-20 2013-06-04 The Nielsen Company (Us), Llc Methods and apparatus for characterizing media
AU2013203321B2 (en) * 2007-02-20 2015-01-15 The Nielsen Company (Us), Llc Methods and apparatus for characterizing media
US8458737B2 (en) 2007-05-02 2013-06-04 The Nielsen Company (Us), Llc Methods and apparatus for generating signatures
US9136965B2 (en) 2007-05-02 2015-09-15 The Nielsen Company (Us), Llc Methods and apparatus for generating signatures
US9460730B2 (en) 2007-11-12 2016-10-04 The Nielsen Company (Us), Llc Methods and apparatus to perform audio watermarking and watermark detection and extraction
US9972332B2 (en) 2007-11-12 2018-05-15 The Nielsen Company (Us), Llc Methods and apparatus to perform audio watermarking and watermark detection and extraction
US10580421B2 (en) 2007-11-12 2020-03-03 The Nielsen Company (Us), Llc Methods and apparatus to perform audio watermarking and watermark detection and extraction
US10964333B2 (en) 2007-11-12 2021-03-30 The Nielsen Company (Us), Llc Methods and apparatus to perform audio watermarking and watermark detection and extraction
US11562752B2 (en) 2007-11-12 2023-01-24 The Nielsen Company (Us), Llc Methods and apparatus to perform audio watermarking and watermark detection and extraction
US8457951B2 (en) 2008-01-29 2013-06-04 The Nielsen Company (Us), Llc Methods and apparatus for performing variable black length watermarking of media
US9947327B2 (en) 2008-01-29 2018-04-17 The Nielsen Company (Us), Llc Methods and apparatus for performing variable block length watermarking of media
US10741190B2 (en) 2008-01-29 2020-08-11 The Nielsen Company (Us), Llc Methods and apparatus for performing variable block length watermarking of media
US11557304B2 (en) 2008-01-29 2023-01-17 The Nielsen Company (Us), Llc Methods and apparatus for performing variable block length watermarking of media
US8600531B2 (en) 2008-03-05 2013-12-03 The Nielsen Company (Us), Llc Methods and apparatus for generating signatures
US9326044B2 (en) 2008-03-05 2016-04-26 The Nielsen Company (Us), Llc Methods and apparatus for generating signatures

Also Published As

Publication number Publication date
MX2007002071A (en) 2007-04-24
WO2006023770A3 (en) 2008-04-10
US20100262642A1 (en) 2010-10-14
US8489884B2 (en) 2013-07-16
US20070274537A1 (en) 2007-11-29
US7783889B2 (en) 2010-08-24

Similar Documents

Publication Publication Date Title
US8489884B2 (en) Methods and apparatus for generating signatures
EP2156583B1 (en) Methods and apparatus for generating signatures
EP2263335B1 (en) Methods and apparatus for generating signatures
AU2008218716B2 (en) Methods and apparatus for characterizing media
US6574594B2 (en) System for monitoring broadcast audio content
US20140114456A1 (en) Methods and Systems for Clock Correction and/or Synchronization for Audio Media Measurement Systems
US10757456B2 (en) Methods and systems for determining a latency between a source and an alternative feed of the source
AU2012211498B2 (en) Methods and apparatus for characterizing media
AU2013203321B2 (en) Methods and apparatus for characterizing media

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KM KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NG NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SM SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU LV MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

DPEN Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed from 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
WWE Wipo information: entry into national phase

Ref document number: MX/a/2007/002071

Country of ref document: MX

Ref document number: 11676452

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase
WWP Wipo information: published in national office

Ref document number: 11676452

Country of ref document: US