WO2003032136A1

WO2003032136A1 - Method and system for identifying and verifying content of multimedia documents

Info

Publication number: WO2003032136A1
Application number: PCT/FR2002/003471
Authority: WO
Inventors: Hassane Essafi; Marc Pic
Original assignee: Commissariat A L'energie Atomique
Priority date: 2001-10-12
Filing date: 2002-10-11
Publication date: 2003-04-17
Also published as: EP1435032A1; FR2831006B1; FR2831006A1

Abstract

The invention concerns a method identifying and verifying content of multimedia documents accessible in a distributed system with a plurality of input points comprising: a) a step of recording multimedia documents as identified works, said recording step comprising for each multimedia document concerned extraction of a fingerprint including an ordered cascade sequence of signatures resulting from an analysis and multicriteria breakdown of the multimedia document concerned, and b) a step of verifying authorized use or not as recorded work, of a given multimedia document accessible to the public, said verification step comprising successive comparisons by means of the cascade signatures of the recorded multimedia documents with corresponding signatures of the given multimedia document, the following comparison being carried out only if the signatures previously compared have revealed similarities, each comparison of cascade signatures being carried out only for the signatures of a group of recorded multimedia documents whereof the previously compared signatures have revealed similarities with the signatures of the given multimedia document, the final result of the last comparison enabling to establish a report containing the list of multimedia documents registered having revealed similarities with the given multimedia document input.

Description

Method and system for identifying and verifying the content of multimedia documents

The present invention relates to a method and a system for identifying and verifying the content of multimedia documents, which can be applied in particular to the day before the proper use of a work, and to the certification of conformity of the content of multimedia documents with respect to multimedia reference documents. Telematic networks such as the Internet offer authors the advantage of making it possible to rapidly disseminate and publicize their works.

In return, by the ease of access, downloading and dissemination of information via the global network, the heritage of these same authors is threatened by malicious people who seek to distribute illegal copies or to counterfeit a work. People of good faith also sometimes put themselves in illegal situations through ignorance of copyright by disseminating content for which they have not paid corresponding rights. Counterfeiting and the transfer of illegal documents are more and more a currency of exchange (pedophilia, deposit of questionable documents in a private place (portal), distribution / sale of copies of works, ...). This practice is amplified with the growing number of network users and the feeling of impunity provided by the supposed anonymity of electronic transactions. The images and sounds are copied and distributed on the websites of “Monsieur Tout-le-Monde”. You can easily find photos of artists or works of art, MP3 music, jingles or iconographic elements that have been cheerfully looted and possibly transformed without the agreement of their authors. It is now common to find, for example, on the network whole films copied from private DVDs or even captured with a video camera in a cinema ... Private exchange formats (point-to-point) or audiences are increasing. The authorities who oversee the clean-up of the Internet or the proper use of the works under their charge are disarmed because of the large volume of data available on the Internet and by its rapid increase (twice every year).

The invention aims in particular to allow the detection, the identification of the violation of the copyright or the rights of authors of works present on Internet or on other supports of information (CD, hard disks, etc.). The invention also makes it possible to identify the transit on the Internet of works of a private nature. Another object of the invention is to make it possible to certify the content of a document and thus to improve the exchange and exploitation of the information available on telematic networks.

In general, the invention aims to allow faster and more reliable, even in the case of managing a large number of documents, the identification and verification of the content of multimedia documents.

These aims are achieved by a method of identifying and verifying the content of multimedia documents accessible in a distributed system with multiple entry points, characterized in that it comprises: a / a step of recording multimedia documents in as identified works, this recording step comprising the extraction of a fingerprint from each multimedia document considered and the storage of this fingerprint in a database independent of the database in which is possibly archived the multimedia document, the fingerprint of the multimedia document considered comprising an ordered sequence of cascading signatures resulting from a multi-criteria analysis and breakdown of the multimedia document considered, and b / a step of checking the authorized or unauthorized use in as a recorded work, of a given multimedia document accessible to the public, this verification step comprising successive comparisons using the cascading signatures of the multimedia documents recorded with corresponding signatures of the given multimedia document, the signature of the multimedia document given corresponding to an analysis criterion considered for a given comparison being calculated just before carrying out this comparison and the following comparison being carried out only if the signatures compared previously revealed similarities, each comparison of cascaded signatures not being done that on the signatures of a group of recorded multimedia documents whose signatures previously compared revealed similarities with the signatures of the given multimedia document, the final result of the last comparison allowing the establishment of a report containing the list of multimedia documents recorded having revealed similarities with the multimedia document given as input.

The subject of the invention is also a method of managing client databases containing a set of client multimedia documents, characterized in that it comprises: a / a step of recording multimedia documents as identified works, this recording step comprising extracting a fingerprint from each multimedia document considered and storing this fingerprint in a database independent of the database in which is possibly archived the multimedia document, the fingerprint of the multimedia document considered comprising an ordered sequence of cascading signatures resulting from a multi-criteria analysis and breakdown of the multimedia document considered, and b / a step of verification and certification of the conformity of the content client multimedia documents compared to the multimedia documents recorded as identified works, the verification and conformity certification step comprising: bl / the prior extraction of a fingerprint from each client multimedia document, the fingerprint comprising an ordered sequence of cascading signatures resulting from a multi-criteria analysis and decomposition of the multimedia document under consideration; b2 / successive comparisons using the cascaded signatures of the multimedia documents recorded with corresponding signatures of the fingerprints of each of the client multimedia documents, each comparison of cascaded signatures being carried out only on the signatures of a group of recorded multimedia documents whose signatures previously compared revealed similarities with the signatures of the multimedia document of the client considered, the final result of the last comparison allowing the establishment of a report making it possible to establish a certificate of conformity of content or Content non-compliance based on the degree of similarity observed between client multimedia documents and pre-recorded multimedia documents. In all cases, according to one aspect of the invention, the ordered sequence of cascading signatures comprises a first signature constituting a hook signature based on a rapid comparison criterion. According to another aspect of the invention, the ordered sequence of cascading signatures comprises signatures representing global characteristics of a recorded multimedia document and signatures representing local characteristics of the recorded multimedia document considered.

According to a particular characteristic, a signature of the ordered sequence of cascading signatures constituting the fingerprint of a recorded multimedia document considered itself constitutes a cascading signature applied to an elementary medium of the recorded multimedia document or to a homogeneous component d '' elementary media of the recorded multimedia document.

The method for identifying and verifying the content of multimedia documents may also comprise a step of monitoring on a network such as an Internet / Intranet network to highlight multimedia documents accessible to the public to be verified which have content corresponding to at least one criterion used to define the fingerprints of multimedia documents already recorded as identified works, and identify an address for each of the multimedia documents to be checked highlighted. The invention also relates to a system for identifying and verifying the content of multimedia documents accessible in a distributed system with multiple entry points, characterized in that it comprises around an interconnection and connection platform. inter-communication, a segmentation module for dissecting the content of a multimedia document, a fingerprint generator for generating a fingerprint of a multimedia document, the fingerprint of the multimedia document comprising an ordered sequence of cascading signatures resulting from a multi-criteria analysis and breakdown of the multimedia document considered, a notifying agent, a database referral fingerprint, a content hunt manager, a content hunt supervisor and a reference directory.

This system for identifying and verifying the content of multimedia documents can also include at least one of the following elements: watch agent, subscriber manager, certifying agent.

Other characteristics and advantages of the invention will emerge from the following description of particular embodiments, given by way of examples, with reference to the appended drawings, in which: - Figure 1 is a block diagram of an example of system for identifying and verifying the content of multimedia documents according to the invention,

FIG. 2 is a flowchart showing the main stages in the preparation of a fingerprint of a multimedia work to be protected, in accordance with the invention,

FIG. 3 is a flow diagram showing an example of a method for identifying and verifying the content of multimedia documents according to the invention, applied to a watch on a telematic network,

FIG. 4 is a block diagram showing the relationships between several functional units cooperating with an identification and verification system for the content of multimedia documents in accordance with the invention,

FIG. 5 is a flowchart showing a process for verifying documents (proofs) before their insertion into a database of reliable documents or a database of uncertain documents as a function of the result of the verification operation, and

- Figure 6 shows the general flowchart of an identification and verification process according to the invention implementing document notification and certification processes. The method for identifying and verifying the content of multimedia documents according to the invention essentially involves a first step of recording multimedia documents as identified works and a second step of verifying, for a given multimedia document, the conformity or not, partial or total, of the content of this document with prerecorded multimedia documents, in order to then be able to deduce from this verification, for example the introduction or not of modifications or non-conforming uses of the content of reference multimedia documents. The management method according to the invention also makes it possible to manage sites or subscriber databases by ensuring the verification and certification of conformity of their content.

The invention makes it possible in particular to verify whether a multimedia document accessible to the system, on any medium or on a network such as the Internet, corresponds to an authorized use of a recorded work and to certify, if necessary, the content of this document in relation to recorded works. The method according to the invention is thus particularly useful for solving copyright problems. The method authorizes a watch consisting of an exploration of sites on a network using a meta-search engine associated with a dictionary of concepts. It is thus possible to search for documents whose content falls under at least one of the concepts of the dictionary.

According to an important aspect of the invention, for the recording of a work to be protected or for the verification of multimedia documents, a digital fingerprint, or synthetic signature, is extracted from each multimedia document, which identifies each document and replaces to him for all subsequent treatments. For multimedia documents to be recorded, it suffices to store fingerprints in databases. reference data without the need to store the entire content of the documents, provided that the fingerprint of each multimedia document considered is stored in a reference database independent of the database in which is possibly archived the multimedia document.

More specifically, the fingerprint of the multimedia document considered comprises an ordered sequence of cascading signatures resulting from a multi-criteria analysis and breakdown of the multimedia document considered. Protection against unauthorized exploitation of the recorded works entrusted to the system is ensured by comparing the fingerprint of each of the recorded works of the system with those extracted from documents stored on any media (CD, disc, DVD, ...) or in any site on the Internet. The system begins by dissecting the content of the documents or works from the site or medium in question.

The work can be a textual, pictorial, musical, elementary document, or a composite document made up of several elementary documents. The content of the works can be stored on a site or on a CD, disc, DVD, etc. The creation of the fingerprint or synthetic digital signature is carried out in several stages: For each work in the medium:

1) Reading the content of the work and analyzing the structure of the document,

2) If the content is composite, the work is dissected: extraction of the components of the content,

3) For each component: extraction and fabrication of its digital signature,

4) Composition of a synthetic signature gathering all the information. Figure 2 summarizes this process of developing a fingerprint or synthetic signature of a work.

The first step 101 consists in reading the document in question and in analyzing the structure of its content, the document in question being able to be supplied by its author to constitute a work to be protected recorded or possibly resulting from a hunt on a network or a medium in the as part of a watch operation, or can be provided by a customer for the purpose of certification, for example, or to control its use.

The second step 102 consists in segmenting or dissecting the multimedia document considered, if it turns out that it is composite, in order to extract the different homogeneous components or elementary documents.

The third step 103 consists in isolating each elementary document in order to proceed to step 104 in extracting and manufacturing a digital signature for each elementary document.

Step 105 consists of the creation of a fingerprint constituting a synthetic signature grouping together all of the information obtained during step 104 of manufacturing a digital signature for each elementary document. Between step 104 and step 105, each elementary document may, if necessary, in turn be broken down into homogeneous components giving rise to the establishment of a digital signature at the level of each component. The process of developing a fingerprint of a multimedia document in the form of an ordered sequence of cascading signatures resulting from the multi-criteria analysis and decomposition of the multimedia document will be presented in more detail in the following. the description, in relation to examples of implementation of each of the aforementioned stages of developing a fingerprint. We will first describe examples of reading the content of a document presented on a website.

The system receives the standard address of a page, URL, as an entry point. The calculation of the page's fingerprint is carried out in several stages:

1) Local copy: The system entry is a URL which is analyzed:

• Non-executable document: the document is loaded on the local machine • Executable document (PHP, CGI): a copy of the execution is generated in a local document (stored in the form of a file or in the form of a computer object in the C ++ / JAVA sense)

2) Identification and invocation of the document structure analysis agent: for example by analyzing by means of the extension or “word magic” or a MIME type to determine the nature of the document and invoke the adequate indexing agent: 1. if it is a single-media document (image, video, audio, plain text), send the document to the indexing agent as a block of the corresponding type with its URL • if the document is of type Shockwave (swf, dcr) we send the document to the SWF analyzer with its URL

• if the document is of HTML type, we send the document to the HTML parser with its URL

• if the document is of the VRML type, the document is sent to the VRML analyzer with its URL

• if the document is a java applet: we send the document to the java executor with its URL

• if the document is a file associated with an activeX: we send the document to the activeX executor with its URL • if the document is a file associated with an external plug-in element: the document is sent to the executor of the plug-in element (plug-in) with its URL.

The dissection of a SWF type document is carried out in several stages: a) decompression of the document b) extraction of tags (identifying images, videos, animations, etc.) c) production of image text blocks etc. correspondents. d) Production of relationships between these blocks

The dissection of an HTML or VRML document uses the same principles as the SWF analyzer but with extended HTML or VRML constraints. Dissecting the result of a Java / ActiveX / Plug-In executor begins by capturing a non-event execution in a document to be dissected 1) The indexer begins by creating a list (empty at the start) of the blocks containing a pointer to a local copy, a URL pointing to the original data, the type of block (still image, moving image, etc.) and a unique identifier. It receives data from the various departments. It analyzes for each object the object's subtype from a list of transformation rules and production rules which it applies to produce one or more blocks which will be added to the list. Example of transformation rule: animated GIF rule: if (type is animated GIF image) then apply:

1. extraction of each image,

2. image backup,

3. adding a block for each image The stage of dissection or segmentation of a composite multimedia work consists in deconstructing the content of the composite document:

In the case of a dissection of a video work, the content of the video document is analyzed to produce the summary containing, in addition to the text and audio information of the video, the images representative of the sequences of the video. The result is an XML document containing the URLs of the elements extracted from the video (the URL of the original document, the URL of the page of the images representative of the video, the URL of the text, ...). In the case of dissection of a site or an HTML page, the content of the page is analyzed to identify and extract its various constituents (flash, film, image, text, audio, ...). Each of these components is dissected in turn. The end result is an XML document gathering the structure of the site as well as the URLs of the pages storing the information extracted from the site / page.

The invention also takes into account the dissection of a medium with dynamic structure (that is to say including not only static data, but also parts of executable code: such as javascript code) or a medium interactive (CD-ROM, DVD, Flash). Such types of documents are more and more often present in multimedia content and require more complex analytical work than the reading / analysis mechanisms previously presented.

The problem concerning these media lies in the fact that one cannot completely dissect the content of these media into elementary media by a simple reading of their content because portions of this content are only generated at runtime and sometimes only dependent on user interaction. These two cases can be illustrated by two examples: First example: code producing the inclusion of media: a javascript code within an HTML document can calculate by concatenation the URL address of a link on this page, thereby deciding to include a media whose description is not directly written to the HTML file. The description is indirect since it is only produced when the code is executed.

Second example: interactive code: the production of a URL address for the content of an HTML page results from a concatenation as in the first example but one of the terms of this concatenation depends on a choice of the user . In this second case, not only is the description indirect, but it will not take on the same meaning depending on the choices made by the user.

In the case of documents having these characteristics, an approximation is made of the behavior of the document program so as to best characterize dynamic content and interactive media. Several diagrams can be used to produce such an approximation, depending on the goals sought in the analysis. We can use operational semantics, denotational, axiomatic, ...). The approximation of these semantics and then the induction of properties by approximation make it possible to transform a document describing a potential for media inclusion into a set of potential documents including exactly one specific medium. The Static Analysis which will be described below constitutes a particular method given by way of example.

The abstract interpretation applied in the form of a static analysis is conducted in the form of intervals of values / sets of values that the variables of the program can take at each stage of this program: If (a == 2) {b = "http://www.audio"} else {b = "http://www.video"} b will be described by the set of values E (b) = {{ b = "http://www.audio", b = "http://www.video"}

A variable c whose choice is left to the user by means of a graduated cursor between -0.5 and +0.5 will be described by the range of values: E (c) = [-0.5; + 0.5]

The operations performed on these variables will be approximated by the possible consequences for the interval / set of values of these actions.

Take for example the following operation: d = concatenation ^, ". Html").

This operation will produce from the set:

E (b) = {{b = "http://www.audio", b = "http://www.video"} the set:

E (d) = {b = "http: //www.audio.html", b = "http: //www.video.html"}

Different techniques can be implemented to improve the convergence of these methods towards the production of stable results (what are called "fixed points" of the approximate semantics), for example widening of the interval of variables, shrinking, cutting out intervals or sets into several subsets. To optimize the application of these methods, an evolutionary strategy is used comparing test solutions to which these strategies are applied alternately according to an encoder scheme in a "genetic code" called "abstraction code". Each of the solutions is processed in parallel and the convergence of these solutions is compared to the others. The best solutions (critical threshold or elitism, but other selection criteria can be applied) are preserved and undergo the action of mutation operators and bridging operators who mix the abstraction codes to converge as quickly as possible possible to fixed points.

This evolutionary optimization scheme may be reduced to its simplest expression (direct comparative test of each of the methods of accelerating convergence). The sets of potentials represented by these fixed points represent the media potentially inserted in the multimedia document. We will consider that the multimedia document contains all of these potentials in the abstract. The multimedia document to be indexed is then a nebula of potentialities, and each of these potentialities is indexed with the document. The referencing system ensures that this status of "potentialities" is noted in the meta-index so as to distinguish "safe" inclusions from "potential" inclusions and to characterize these to the best of a percentage (100% for safe inclusions, and if not lower in proportion to the relationship between the interval and the documents that can be included).

The static analysis of dynamic or interactive code therefore aims to extract "potential" links / media and identify content despite the possibility of dynamic generation (on the fly) of this content or these links depending on the user.

The goal of this task is to extract a mathematical characterization representing the work independently of the conditions of capture or digitization of the work (lighting, position, ...). This in order to be able to identify the presence of all or part of the work in various contexts (inlay, rotation, sound effects, ...). For this do it is developed a cascading signature (nested signature). This cascading signature represents both global characteristics (colors, shapes, textures) and local characteristics (particular detail). The cascading signature is composed of a sequence of elementary signatures associated with the methods used in the production of elementary signatures. Among the methods used to produce cascading signatures, we can cite:

A) Colorimetric quantification of the image and of the homogeneous zones (from the color point of view): the result is a set of vectors representing the dominant colors of the image and its various constituents. The method is based on the analysis of the histogram (vector of M dimension reflecting the distribution of colors in the image). The algorithm is as follows:

1. calculation of the colors of each band of the image (HSV / RGB)

2. normalization: division of the value of each sample by the sum of the values of all the samples in the image. The vector produced makes the histogram invariant to the different geometric operations of the image (change of scale, rotation, ...)

3. quantification of the histogram: production of a vector of reduced size and less sensitive to the shooting conditions. The elements of this vector are the parameters of a series of Gaussian distributions approximating the normalized histogram.

Two methods are used: a. the first is based on the Fisher algorithm (distribution of the histogram in N classes), each class corresponding to homogeneous areas of the image, b. the second is based on an iterative calculation. At each iteration the parameters of the Gaussian corresponding to the maximum of the Gaussian are estimated. Pixels whose values are covered by this Gaussian are narrow with the iteration index, the pixel values are set to zero. The histogram is calculated again and iterates until all the points of the image are reset to zero.

4. Calculation and quantification of the histogram of each of the areas of the image.

B) Characterization of the areas of interest (points of support and the areas or patterns constituting the components). A possible characterization method characterizes the neighborhoods of the areas of interest as well as the partition obtained from the points of interest.

C) Characterization of the positions of the pixels belonging to the same entity (objects, shape, ...). This characterization depends on the complexity of the shape of the components of the image. Simple forms such as straight, circle, ... are described by the inherent equations. The method used to extract the circles and ellipses is based on the detection of the outline and the spocke filter.

The complex shapes are described by a series of affine invariants which are determined from the position of the support points. The method used is as follows:

1. calculation of support points and contours of image components, 2. sorting the support points to keep only those positioned on the contours.

3. regrouping of support points and calculation for each group of the affine function approximating the curve passing through the support points of the group.

D) Characterization of the visual aspect of the image and its components: one can use a method based on the decomposition in wavelets.

In the same way as for a visual work, the fingerprint of a sound work is carried out in several stages: first of all the work is broken down into several homogeneous components (homogeneous zone: same speaker, note, same rhythm , ...). Then comes the characterization of each of these homogeneous components and finally the structuring of the work.

The fingerprint describes the content of a multimedia document. In the case of a recorded work managed by the system, the fingerprints are stored in a database called Base d'Empreintes de References (BER). The model of an imprint is advantageously as follows:

1. composition of the work:

1. list of elementary works constituting the document or the work: a poster can consist of several photos. In general, a multimedia document is made up of several works, each with its own operating conditions.

2. factual information: operating conditions, authors, date, place, ...

2. signature in cascade of each of the elementary works: 1. the sequence of elementary signatures and the methods used for the production of each of them

2. the spatial relationships between the components of the elementary work 3. methods used in the extraction of these components

4. the signature in cascade of each component of the elementary work. 3. The hanging signature serving as the entry point for the imprint. It is this which is used in the first phase of pairing a document with the BER database, it allows you to confirm if a document does not contain a work from the database or to report an inference with the database. In this case, the system refines the procedure for matching the elements of the document signature only with the fingerprints of the works that have caused interference. The method of identifying and verifying the content of multimedia documents according to the invention applies to multimedia documents accessible in a distributed system with multiple entry points.

The method can ensure a watch intended to highlight multimedia documents accessible to the public which are likely to constitute a misuse of the prerecorded works and present a content meeting at least one criterion having served to define the fingerprints of multimedia documents. already registered as identified works. As indicated above, a process for protecting a work takes place in two phases: the deposit phase (registration) and the watch phase for proper use. To effectively protect the works of authors that are scattered around the world and to facilitate the filing task and increase the efficiency of the protection, a content tracking system according to the invention is a distributed system with several entry points which can be distributed around the world. The author can register his work in one of the entry points of the system and, automatically, the protection process is triggered to ensure that the work is exploited under legal conditions. The system will then make it possible to detect unauthorized uses of the work or parts thereof.

In the deposit phase, the work is registered in one of the entry points of the system. This system analyzes the work to extract a fingerprint that characterizes its content. As already indicated above, the fingerprint is the composition of the signature characterizing the signal or the physical information of the work and of contextual information such as the name of the author, the date of creation, the type of the work, ... as well as the methods used to generate the signature.

The fingerprint is used, during the monitoring phase, to identify and track the misuse of the work (presence of the work in another document, presence of the work in an unauthorized site, transformation and distortion of the 'artwork...). It is only the fingerprint that must be stored in one of the databases of the content tracking system, the work itself does not need to be saved in the system. The extraction of the fingerprint can be performed locally, but this fingerprint can then be stored either locally or in a remote location. We will now describe with reference to Figure 1 the modules forming part of a minimum embodiment of a system for identifying and verifying the content of multimedia documents, such as a system integrated for example in the aforementioned content tracking system . The system for identifying and verifying the content of multimedia documents essentially comprises, around an interconnection and intercommunication platform 10, a segmentation module 11 for dissecting the content of a multimedia document, a generator fingerprint 12 to create a fingerprint of a multimedia document, a notifying agent 13, a reference fingerprint base 14, a content tracking manager 15, a content tracking supervisor 16 and a reference directory 17.

The content tracking manager is used by the administrator to define the configuration of the content tracking system. It is distributed over the set of compute nodes participating in the configuration. One and only one instance is activated on each of the calculation nodes.

The activated modules as well as the content tracking manager 15 are recorded in the reference directory 17. This directory 17 allows a module to retrieve the reference of another module to which it wishes to request a service from a generic name.

The creation of platform 10 is based on standard protocols (CORBA, UDP / IP, TCP / IP, RTP / RTSP, HTTP, XML / SOAP) but adapted to the needs of the application. For example, communications between agents (modules) located in the same machine or on machines connected to the same local network use protocols (CORBA, UDP / IP, TCP / IP). However, communications between modules located in machines connected via the Internet to two separate networks can use the HTTP / XML / SOAP (Simple Object Access Protocol) protocol. The latter mode of communication has the advantage of launching the execution of a remote task using the Remote Call Protocol (RPC). The invocation of the task as well as the transmission of the arguments are described by an XML page (extension format of the HTML). Sending the page to the node (server) hosting the task causes the task to run. The result is sent to the sender as an XML page. The advantage of this invocation (execution) mode is that it is only based on the HTTP protocol and is therefore less restrictive to implement.

FIG. 4 shows the modules or agents of the system according to the invention which, once a documentary reference base 14 containing the fingerprints of the pre-recorded reference multimedia documents has been created, participate in the monitoring process the proper use of these pre-recorded multimedia reference documents.

A notifying agent compares the fingerprint of the input documents and issues a compliance report for these documents compared to the pre-recorded works.

The role of an explorer or watchman 21 is to identify sites likely to contain pre-recorded works. It is made up of a metasearch engine coupled to a dictionary of concepts 31. The metasearch engine explores the Internet for sites containing suspicious documents (the content of which corresponds to at least one of the concepts in dictionary 31). The fingerprint of each of these documents is transmitted to the notifier 13 who compares the fingerprints of these documents with those of the reference base 14 and issues either a certificate of compliance (acquittal) or a report of non-compliance. The explorer 21 enriches this report with information concerning the sites holding these documents and also the accomplice sites (sites serving as relays in locating the document). This is to provide the operator responsible for detecting fraud with all the information necessary to locate the target. Only the non-compliance report is transmitted to a human operator and it does not contain any documents. The documents are not kept; they are only used to calculate fingerprints

A subscriber manager 22 certifies the content of documents from a subscriber base 32. It analyzes the content of the site passed as an argument and compares the fingerprints of these documents with those who have acquired the right to exploit them legally,

A content certifying agent 23 certifies the content of a site, file, CD or any other medium. He works in collaboration with the notifier 13 in order to clean up the content of a site. Each time a document (called Proof) is inserted into the site, its content is analyzed and a compliance report is issued. This module is intended to be coupled to a content delivery system. It cooperates with a reliable database 33 and an uncertain database 34. With reference to FIG. 5, a process of inserting proofs (multimedia documents to be analyzed and verified) will be described in a reliable database 33 or a database of uncertain documents 34 associated with the certifying agent 23.

This phase is technically similar to the phase of depositing works, but diverges in the use made of the analyzed data. The pre-recorded works represent the documents that we wish to protect, those with which we will carry out similarity comparisons, for example to discover possible counterfeits or pirated copies. Proofs are documents that we want to know if they are themselves counterfeits or pirated copies. We calculate their signature in the same way as for works already recorded (with the four phases of reading / analysis, decomposition, atomic signatures, composition of signatures), but we insert these signatures into another base: the footprint database of trials. This base can contain the imprint of a single document (verification of content) or a very large number (comparison base to base). The fingerprint base is then transmitted to the notifier 23. The latter compares the test fingerprint base with the reference fingerprint base 32 and returns a report in which appears, for each fingerprint, an indication of its conformity with respect to the reference fingerprints. The tests whose control has proved positive (fingerprint similar to at least one or part of a reference fingerprint) are placed in an Uncertain Document Database (BDI) 34. The others are inserted in the Document Database Reliable (BDF) 33. The insertion of BDI 34 documents into the BDF 33 database is not automatic and requires human intervention.

The BDF 33 database can contain a copy of the original documents of the tests, as well as the related information allowing them to be found and possibly serving as proof (http address, etc.). This copy will serve, for example, as a certified rerun basis for the application concerning information reruns. It will be cleaned of its questionable elements by the system at the end of the process and can therefore be used for example as a proxy or server. On the flow diagram of FIG. 5, we see tests to be inserted introduced at the input (step 201) of the fingerprint generator 12, a step 204 of calculating the fingerprints of the tests introduced at the input, a step 210 of comparison carried out within of the notifier 13 to compare the fingerprints of the tests calculated in step 204 with the reference fingerprints contained in the reference fingerprint database 14, and a sorting step 220 to insert the fingerprints of the tests either in the base of reliable documents 33, ie in the base of uncertain documents 34 as a function of the result of the conformity test of step 210. The process of notification, certification and management of subscribers or customers is illustrated in FIG. 6 where we see a step 301 of entering multimedia documents to be recorded, a step 304 of calculating fingerprints of the documents to be recorded performed within the fingerprint generator 12, to deliver fingerprints stored in the base 14 of reference fingerprints.

Documents to be verified can be introduced by a watch agent 21 (step 321) by a subscriber manager 22 (step 322) or by a certifier 23 (step 323). These documents to be verified are subject to a fingerprint calculation in step 341, these fingerprints being applied in a step 342 to the notifier 13 to be subjected to comparisons with the fingerprints of the fingerprint database. reference 14.

In step 343, a first comparison is carried out between the first signatures of the fingerprints to be compared, these first signatures constituting hook signatures preferably based on a rapid comparison criterion.

At the end of step 343, either an acknowledgment occurs, the document to be checked being considered as valid and not affecting a pre-recorded reference document, or, in the event of interference between the signatures of hanging from compared documents, moving to a stage

344 in which a selection is made of the footprints of the reference base 14 which caused the collision, then the passage to a step

345 where a new comparison is made of a lower level elementary signature of the cascading signature constituting the fingerprint of a document to be checked and of elementary signatures of the same level of the cascading signatures constituting the fingerprints of the documents parameters selected in previous step 344. The process is repeated between steps 345 and 344 as long as there is interference and until all elementary signatures or reference documents are exhausted. Acknowledgments are possible at each step 344. If at the last comparison of step 345, there was no acknowledgment, a non-compliance report is issued at step 366. Thus, after comparing the fingerprints of documents to be checked with those of the reference fingerprint database, a certificate of conformity or a report of non-conformity is issued. Once the comparison of fingerprints received as inputs with those of the reference base 14 has been made to the desired level of precision, the system will produce a decision as to the validity, invalidity or doubt concerning the documents to be checked. This decision can take three distinct forms depending on the application.

In the case of an application for hunting illegal or pirated content controlled by the watch agent 21, this will be a notification phase. In the case of a content validation application controlled by the certifying agent 23 (for example for content redistributors), this will be a certification phase. In the case of a subscriber management application controlled by the subscriber manager 22, this will be a reporting procedure. In the context of tracking down illegal content and therefore a notification procedure, the production of a positive result (recognition of a document from the proof base in the basis of works 14) during the comparison phase causes a mechanism to verify this result. This mechanism for eliminating false alarms takes place in two phases. The first phase consists in refining the comparison by using higher terms of the signature (the choice of these terms is determined in proportion to the available computing time and to the size of the two bases to be compared according to the linear law). Comparing these higher terms is more costly in terms of time. calculation and therefore requires to be made only on a subset of elements from each of the bases: the couples (works-proofs) for which a positive result has been produced. The set of result pairs are then sorted in decreasing order of maximum similarity at the highest precision, then by decreasing precision. The second phase of the process consists in comparing the pairs (works-proofs) of this list by their common components and in calculating the most suspect proofs in order of decreasing similarity to produce a list of the first N (N being adjusted by the operator ). This second phase can be reduced to a simple cutting of the list of couples sorted in descending order to keep only the N first elements (N being set by the operator).

The list obtained, with the fingerprint information associated with each of these elements, constitutes the output of the system in this case.

Within the framework of a certification procedure, the production of a positive result during the comparison phase can lead to a validation phase identical to that described above, but it is not compulsory. If this is not the case, an ordered list of suspect couples (works-proofs) is drawn up on the basis of their decreasing similarity rates. This list or the list produced by refinement is used to generate the deletion of the corresponding files from the proof base and issuance of warning messages or of a report containing this list to the operator. Once the doubtful files have been deleted, the proof base is declared certified.

Within the framework of a procedure calling upon the subscriber manager 22, the aim is to ensure the conformity of the content of the subscribed sites with the subscription contract (document specifying the works for which the subscriber has acquired the right operating). The manager of subscribers scans subscriber sites one by one for each site. For each site visited, it analyzes its content (in collaboration with the notifier 13). For each non-conforming document which is not mentioned in the subscription contract, a reporting procedure may be initiated.

Figure 3 shows the flowchart of an example verification process applied to a suspicious document identified during a content hunt or provided from a particular medium, this suspect document to be compared to pre-recorded documents . In this case, the reference fingerprints of the various pre-recorded documents have been calculated beforehand and stored in a reference fingerprint database (step 152).

The suspect document to be verified is itself the subject of a calculation of a first high level signature (hooked signature) in step 151.

A first comparison is then made of the signature for hanging the suspect document and the signatures for hanging the reference fingerprints of the reference base 152 (step 153).

If, following this first comparison of hanging signatures, the suspect document is considered to be close to certain pre-recorded reference documents (step 154), these pre-recorded reference documents are retained for a new comparison, this new comparison between signatures of a lower level than the hook signature previously used. At this lower level, which can correspond, for example, to the development of elementary signatures from the points of interest method, the corresponding signature of the suspect document is first developed and this signature is compared with the corresponding signatures in the same way. pre level stored in the reference database, prerecorded documents preserved at the end of step 153.

If following the comparison in step 155, the suspect document is still considered to be close to certain pre-recorded reference documents (step 156), these reference documents are retained for a new comparison occurring between signatures still of lower level which can correspond for example to the elaboration of elementary signatures following a segmentation method to extract different components from the document, here again, the corresponding signature of the suspect document is elaborated for each component (step 157 ) and these signatures are compared with the corresponding signatures of the same level pre-stored in the reference database, pre-recorded documents preserved at the end of step 155. If following the comparison of the step 158, the suspect document is considered to constitute, for example, a forgery, taking into account the similarities detected, for example, mple a report explaining the history of decisions and giving the way to retrieve the addresses and allow manual observation.

Claims

1. A method for identifying and verifying the content of multimedia documents accessible in a distributed system with multiple entry points, characterized in that it comprises: a / a step of recording multimedia documents as identified works , this recording step comprising the extraction of a fingerprint from each multimedia document considered and the storage of this fingerprint in a database independent of the database in which the multimedia document is possibly archived, the fingerprint digital of the multimedia document considered comprising an ordered sequence of cascading signatures resulting from a multi-criteria analysis and breakdown of the multimedia document considered, and b / a step of verifying the use authorized or not as a recorded work, of a given multimedia document accessible to the public, this verification step including suc comparisons discontinuous using the cascaded signatures of the multimedia documents saved with corresponding signatures of the given multimedia document, the signature of the given multimedia document corresponding to an analysis criterion considered for a given comparison being calculated just before carrying out this comparison and the following comparison being carried out only if the signatures compared previously revealed similarities, each comparison of cascading signatures being carried out only on the signatures of a group of recorded multimedia documents whose signatures previously compared revealed similarities with the signatures of the given multimedia document, the result final of the last comparison allowing the establishment of a report containing the list of recorded multimedia documents having revealed similarities with the multimedia document given as input.

2. Method according to claim 1, characterized in that the ordered sequence of cascading signatures comprises a first signature constituting an attachment signature based on a rapid comparison criterion.

3. Method according to claim 2, characterized in that the ordered sequence of cascading signatures comprises signatures representing global characteristics of a recorded multimedia document and signatures representing local characteristics of the recorded multimedia document considered.

4. Method according to any one of claims 1 to 3, characterized in that a signature of the ordered sequence of cascading signatures constituting the fingerprint of a recorded multimedia document considered itself constitutes a cascading signature applied to an elementary medium of the recorded multimedia document or to a homogeneous component of an elementary medium of the recorded multimedia document.

5. Method according to any one of claims 1 to 4, characterized in that it further comprises a monitoring step on a network such as an Internet / Intranet network for highlighting multimedia documents accessible to the public to be verified. which present content meeting at least one criterion used to define the fingerprints of multimedia documents already recorded as identified works, and identify an address for each of the multimedia documents to be checked highlighted.

6. System for identifying and verifying the content of multimedia documents accessible in a distributed system with multiple entry points, characterized in that it comprises around an interconnection and intercommunication platform (10) , a segmentation module (11) for dissecting the content of a multimedia document, a fingerprint generator (12) for generating a fingerprint of a multimedia document, the fingerprint of the multimedia document comprising an ordered sequence of cascading signatures resulting from a multi-criteria analysis and decomposition of the multimedia document considered, a notifying agent (13), a reference fingerprint base (14), a content tracking manager (15), a supervisor (16) content tracking and a reference directory (17).

7. System according to claim 6, characterized in that it further comprises a subscriber manager (22).

8. System according to claim 6 or claim 7, characterized in that it further comprises a certifying agent (23).

9. System according to any one of claims 6 to 8, characterized in that it further comprises a watchdog agent (21).

10. A method of managing client databases containing a set of client multimedia documents, characterized in that it comprises: a / a step of recording multimedia documents as identified works, this step of recording comprising extracting a fingerprint from each multimedia document considered and storing this fingerprint in a database independent of the database in which the multimedia document is possibly archived, the fingerprint of the multimedia document considered comprising an ordered sequence of cascading signatures resulting from a multi-criteria analysis and breakdown of the multimedia document under consideration, and b / a step of verifying and certifying the conformity of the content of the client's multimedia documents with respect to the multimedia documents recorded as identified works, l step of verification and certification of conformity comprising: bl / the prior extraction of a fingerprint from each client multimedia document, the fingerprint comprising an ordered sequence of cascading signatures resulting from a multi-criteria analysis and breakdown of the multimedia document considered; b2 / successive comparisons using the cascaded signatures of the multimedia documents recorded with corresponding signatures of the fingerprints of each of the client multimedia documents, each comparison of cascaded signatures being carried out only on the signatures of a group of recorded multimedia documents whose signatures previously compared revealed similarities with the signatures of the multimedia document of the client considered, the final result of the last comparison allowing the establishment of a report making it possible to establish a certificate of conformity of content or Content non-compliance based on the degree of similarity observed between client multimedia documents and pre-recorded multimedia documents.

11. Method according to claim 10, characterized in that the ordered sequence of cascading signatures comprises a first signature constituting an attachment signature based on a rapid comparison criterion.

12. Method according to claim 10 or claim 11, characterized in that the ordered sequence of cascading signatures comprises signatures representing global characteristics of a recorded multimedia document and signatures representing local characteristics of the recorded multimedia document considered.

13. Method according to any one of claims 10 to 12, characterized in that a signature of the ordered sequence of cascading signatures constituting the fingerprint of a recorded multimedia document considered itself constitutes a cascading signature applied to an elementary medium of the recorded multimedia document or to a homogeneous component of an elementary medium of the recorded multimedia document.