CA2470299A1 - Systems, methods, and software for classifying documents - Google Patents
Systems, methods, and software for classifying documents Download PDFInfo
- Publication number
- CA2470299A1 CA2470299A1 CA002470299A CA2470299A CA2470299A1 CA 2470299 A1 CA2470299 A1 CA 2470299A1 CA 002470299 A CA002470299 A CA 002470299A CA 2470299 A CA2470299 A CA 2470299A CA 2470299 A1 CA2470299 A1 CA 2470299A1
- Authority
- CA
- Canada
- Prior art keywords
- target
- class
- input text
- cndot
- text
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/35—Clustering; Classification
- G06F16/353—Clustering; Classification into predefined classes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10—TECHNICAL SUBJECTS COVERED BY FORMER USPC
- Y10S—TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y10S707/00—Data processing: database and file management or data structures
- Y10S707/99941—Database schema or data structure
- Y10S707/99942—Manipulating data structure, e.g. compression, compaction, compilation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Databases & Information Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Devices For Executing Special Programs (AREA)
Abstract
To reduce cost and improve accuracy, the inventors devised systems, methods, and software to aid classification of text, such as headnotes and other documents, to target classes in a target classification system. For example, one system computes composite scores based on: similarity of input text to text assigned to each of the target classes; similarity of non-target classes assigned to the input text and target classes; probability of a target class given a set of one or more non-target classes assigned to the input text;
and/or probability of the input text given text assigned to the target classes. The exemplary system then evaluates the compposite scores using class-specific decision criteria, such as thresholds, ultimately assigning or recommending assignment of the input text to one or more of the target classes. The exemplary system is particularly suitable for classification systems having thousands of classes.
and/or probability of the input text given text assigned to the target classes. The exemplary system then evaluates the compposite scores using class-specific decision criteria, such as thresholds, ultimately assigning or recommending assignment of the input text to one or more of the target classes. The exemplary system is particularly suitable for classification systems having thousands of classes.
Claims (28)
1. A computerized system for classifying input text to target classification system having two or more target classes, the system comprising:
.cndot. means for determining for each of the target classes at least first and second scores based on the input text and the target class;
.cndot. means for determining for each of the target classes a corresponding composite score based on the first score scaled by a first class-specific weight for the target class and the second score scaled by a second class-specific weight for the target class; and .cndot. means for determining for each of the target classes whether to classify or recommend classification of the input text to the target class based on the corresponding composite score and a class-specific decision threshold for the target class.
.cndot. means for determining for each of the target classes at least first and second scores based on the input text and the target class;
.cndot. means for determining for each of the target classes a corresponding composite score based on the first score scaled by a first class-specific weight for the target class and the second score scaled by a second class-specific weight for the target class; and .cndot. means for determining for each of the target classes whether to classify or recommend classification of the input text to the target class based on the corresponding composite score and a class-specific decision threshold for the target class.
2. A computer-implemented method of classifying input text to a target classification system having two or more target classes, the method comprising:
for each target class:
.cndot. providing at least first and second class-specific weights and a class-specific decision threshold;
.cndot. using at least first and second classification methods to determine respective first and second scores based on the input text and the target class;
.cndot. determining a composite score based on the first score scaled by the first class-specific weight for the class and the second score scaled by the second class-specific weight for the target class; and .cndot. classifying or recommending classification of the input text to the target class based on the composite score and the class-specific decision threshold.
for each target class:
.cndot. providing at least first and second class-specific weights and a class-specific decision threshold;
.cndot. using at least first and second classification methods to determine respective first and second scores based on the input text and the target class;
.cndot. determining a composite score based on the first score scaled by the first class-specific weight for the class and the second score scaled by the second class-specific weight for the target class; and .cndot. classifying or recommending classification of the input text to the target class based on the composite score and the class-specific decision threshold.
3. The method of claim 2, wherein at least one of the first and second scores is based on a set of one or more noun-words pairs associated with the input text and a.
set of one or more noun-word pairs associated with the target class, with at least one noun-word pair in each set including a noun and a non-adjacent word.
set of one or more noun-word pairs associated with the target class, with at least one noun-word pair in each set including a noun and a non-adjacent word.
4. The method of claim 2, wherein providing each first and second class-specific weight and class-specific decision threshold comprises searching for a combination of first and second class-specific weights and class-specific decision threshold that yield a predetermined level of precision at a predetermined level of recall based on text classified to the target classification system.
5.The method of claim 2, wherein a non-target classification system includes two or more non-target classes, and at least one of the first and second scores is based on one or more of the non-target classes that are associated with the input text and one or more of the non-target classes that are associated with the target class.
6. The method of claim 5:
.cndot. wherein the input text is a headnote for a legal document; and .cndot. wherein the target classification system and the non-target classification system are legal classification systems
.cndot. wherein the input text is a headnote for a legal document; and .cndot. wherein the target classification system and the non-target classification system are legal classification systems
7. The method of claim 2, wherein the target classification system includes more than 1000 target classes.
8. The method of claim 2, further comprising:
.cndot. displaying a graphical user interface including first and second regions, with the first region displaying or identifying at least a portion of the input text and the second region displaying information regarding the target classification system and at least one target class for which the input text was recommended for classification; and ~ displaying a selectable feature on the graphical user interface, wherein selecting the feature initiates classification of the input text to the one target class.
.cndot. displaying a graphical user interface including first and second regions, with the first region displaying or identifying at least a portion of the input text and the second region displaying information regarding the target classification system and at least one target class for which the input text was recommended for classification; and ~ displaying a selectable feature on the graphical user interface, wherein selecting the feature initiates classification of the input text to the one target class.
9. A machine-readable medium comprising instructions for implementing the method of claim 2.
10. A computer-implemented method of classifying input text to a target classification system having, two or more target classes, the method comprising.
for each target class:
~ determining first and second scores based on the input text and the target class;
~ determining a composite score based on the first score scaled by a first class specific weight for the target class and the second score scaled by a second class-specific weight for the target class; and ~ determining whether to identify the input text for classification to the target class based on the composite score and a class-specific decision threshold for the target class.
for each target class:
~ determining first and second scores based on the input text and the target class;
~ determining a composite score based on the first score scaled by a first class specific weight for the target class and the second score scaled by a second class-specific weight for the target class; and ~ determining whether to identify the input text for classification to the target class based on the composite score and a class-specific decision threshold for the target class.
11. The method of claim 10, wherein at least one of the first and second scores is based on a set of one or more noun-words pairs associated with the input text and a set of one or more noun-word pairs associated with the target class, with at least one noun-word pair in each set including a noun and a non-adjacent word.
12. The method of claim 10, wherein determining the .first and second scores comprises determining any two of:
.cndot. a score based on similarity of at least one or more portions of the input text to text associated with the target class;
.cndot. a score based on similarity of a set of one or more non-target classes associated with the input text and a set of one or more non-target classes associated with the target class;
.cndot. a score based on probability of the target class given a set of one or more non-target classes associated with the input text; and .cndot. a score based on probability of the target class given at least a portion of the input text.
.cndot. a score based on similarity of at least one or more portions of the input text to text associated with the target class;
.cndot. a score based on similarity of a set of one or more non-target classes associated with the input text and a set of one or more non-target classes associated with the target class;
.cndot. a score based on probability of the target class given a set of one or more non-target classes associated with the input text; and .cndot. a score based on probability of the target class given at least a portion of the input text.
13. The method of claim 12, wherein each target class is a document and the text associated with the target class comprises text of the document or text of another document associated with the target class.
14. The method of claim 10:
.cndot. wherein determining the first and second scares for each target class comprises:
o determining the first score based on similarity of at least one or more portions of the input text to text associated with the target class; and o determining the second score based on similarity of a set of one or more non-target classes associated with the input text and a set of one or more non-target classes associated with the target class;
wherein the method further comprises detern~ining for each target class:
o a third score based on probability of the target class given a sct ofone or more non-target classes associated with the input text; and o a fourth score based on probability of the target class given at least a portion of the input text; and .cndot.wherein the composite score is further based on the third score scaled by a third class-specific weight for the target class and the fourth score scaled by a fourth class-specific weight for the target class.
.cndot. wherein determining the first and second scares for each target class comprises:
o determining the first score based on similarity of at least one or more portions of the input text to text associated with the target class; and o determining the second score based on similarity of a set of one or more non-target classes associated with the input text and a set of one or more non-target classes associated with the target class;
wherein the method further comprises detern~ining for each target class:
o a third score based on probability of the target class given a sct ofone or more non-target classes associated with the input text; and o a fourth score based on probability of the target class given at least a portion of the input text; and .cndot.wherein the composite score is further based on the third score scaled by a third class-specific weight for the target class and the fourth score scaled by a fourth class-specific weight for the target class.
15. The method of claim 10:
.cndot. wherein. the input text is associated with first meta-data and each target class is associated with second meta-data; and .cndot. wherein at least one of the first and second scores is based on the first meta-data and the second meta-data.
.cndot. wherein. the input text is associated with first meta-data and each target class is associated with second meta-data; and .cndot. wherein at least one of the first and second scores is based on the first meta-data and the second meta-data.
16. The method of claim 15, wherein the first meta-data comprises a first set of non-target classes that are associated with the input text and the second meta-data comprises a second set of non-target classes that are associated with the target class,
17. A machine-readable medium comprising instructions for performing the method of claim 9.
18. A computer-implemented method of classifying input text according to a target classification system having two or more target classes, the method comprising:
.cndot.for each target class, determining a composite score based on a first score scaled by a first class-specific weight for the target class and a second score scaled by a second class-specific weight for the target class, with the first and second scores based on an input text and text associated with the target class;
and cndot.for each target class, classifying or recommending classification of the input text to the target class based on the composite score and a class-specific decision threshold for the target class.
.cndot.for each target class, determining a composite score based on a first score scaled by a first class-specific weight for the target class and a second score scaled by a second class-specific weight for the target class, with the first and second scores based on an input text and text associated with the target class;
and cndot.for each target class, classifying or recommending classification of the input text to the target class based on the composite score and a class-specific decision threshold for the target class.
19. The method of claim 18, wherein the first and second scores are selected from the group consisting of:
.cndot. a score based on similarity of at least one or more portions of the input text to text associated with the target class;
.cndot. a score based on similarity of a set of one or more non-target classes associated with the input text and a set of one or more non-target classes associated with the target class;
.cndot. a score based on probability of the target class given a set of one or more non-target classes associated with the input text; and .cndot. a score based on probability of the target class given at least a portion of the input text.
.cndot. a score based on similarity of at least one or more portions of the input text to text associated with the target class;
.cndot. a score based on similarity of a set of one or more non-target classes associated with the input text and a set of one or more non-target classes associated with the target class;
.cndot. a score based on probability of the target class given a set of one or more non-target classes associated with the input text; and .cndot. a score based on probability of the target class given at least a portion of the input text.
20. The method of claim 18, further comprising:
updating the class-specific threshold for one of the target classes based on acceptance or rejection of recommended classifications of the input text.
updating the class-specific threshold for one of the target classes based on acceptance or rejection of recommended classifications of the input text.
21. A computer-implemented method of classifying text to one or more target classes in a target classification system, the method comprising:
.cndot.identifying one or more noun-word pairs in a portion of text.
.cndot.identifying one or more noun-word pairs in a portion of text.
22. The method of claim 21, wherein identifying one or more noun-word pairs in the portion of text, comprises:
.cndot. identifying a first noun in the portion of text; and .cndot. identifying one or more words within a predetermined numbers of words of the first noun.
.cndot. identifying a first noun in the portion of text; and .cndot. identifying one or more words within a predetermined numbers of words of the first noun.
23. The method of claim 21, wherein, identifying one or more words within a predetermined number of words of the first noun comprises excluding a set of one or more stop words.
24. The method of claim 21, wherein the portion of text is a paragraph.
25. The method of claim 21, further comprising:
determining one or more scores based on frequencies of one or more of the identified noun-word pairs in the portion of text and one or more noun-word pairs in text associated with one of the target classes.
determining one or more scores based on frequencies of one or more of the identified noun-word pairs in the portion of text and one or more noun-word pairs in text associated with one of the target classes.
26. The method of claim 25, wherein the one or more scores include:
.cndot. at least one score based on similarity of at least one or more portions of the input text to text associated with the target class;
.cndot. at least one score based on similarity of a set of one or more non-target classes associated with the input text and a set of one or more non-target classes associated with the target class;
.cndot. at least one score based on probability of the target class given a set of one or more non-target classes associated with the input text; and .cndot. at least one score based on probability of the target class even at least a portion of the input text
.cndot. at least one score based on similarity of at least one or more portions of the input text to text associated with the target class;
.cndot. at least one score based on similarity of a set of one or more non-target classes associated with the input text and a set of one or more non-target classes associated with the target class;
.cndot. at least one score based on probability of the target class given a set of one or more non-target classes associated with the input text; and .cndot. at least one score based on probability of the target class even at least a portion of the input text
27. The method of claim 25, wherein determining one or more scores based on one or mare identified noun-word pairs and one or more noun-word pairs in other text associated with one of the target classes, comprises:
.cndot. determining a respective weight for each identified noun-word pair, with the respective weight based on a product of a term frequency of the identified word-noun pair in the text and an inverse document frequency of the noun-word pairs in the other text associated with one of the target classes.
.cndot. determining a respective weight for each identified noun-word pair, with the respective weight based on a product of a term frequency of the identified word-noun pair in the text and an inverse document frequency of the noun-word pairs in the other text associated with one of the target classes.
28. A computer-implemented method of classifying input text to one or more target classes in a target classification system, the method comprising:
.cndot. identifying a first set of noun-word pairs in the input text, with the first set including at least one noun-word pair formed from a noun and non-adjacent word in the input text;
.cndot. identifying two or more second sets of noun-word pairs, with each second set including at least one noun word pair formed from a noun and non-adjacent word in text associated with a respective one of the target classes;
.cndot. determining a set of scores based on the first and second sets of noun-word pairs; and .cndot. classifying or recommending classification of the input text to one or more of the target classes based on the set of scores
.cndot. identifying a first set of noun-word pairs in the input text, with the first set including at least one noun-word pair formed from a noun and non-adjacent word in the input text;
.cndot. identifying two or more second sets of noun-word pairs, with each second set including at least one noun word pair formed from a noun and non-adjacent word in text associated with a respective one of the target classes;
.cndot. determining a set of scores based on the first and second sets of noun-word pairs; and .cndot. classifying or recommending classification of the input text to one or more of the target classes based on the set of scores
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA2737943A CA2737943C (en) | 2001-11-02 | 2002-11-01 | Systems, methods, and software for classifying documents |
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US33686201P | 2001-11-02 | 2001-11-02 | |
| US60/336,862 | 2001-11-02 | ||
| PCT/US2002/035177 WO2003040875A2 (en) | 2001-11-02 | 2002-11-01 | Systems, methods, and software for classifying documents |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA2737943A Division CA2737943C (en) | 2001-11-02 | 2002-11-01 | Systems, methods, and software for classifying documents |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA2470299A1 true CA2470299A1 (en) | 2003-05-15 |
| CA2470299C CA2470299C (en) | 2011-04-26 |
Family
ID=23317997
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA2737943A Expired - Lifetime CA2737943C (en) | 2001-11-02 | 2002-11-01 | Systems, methods, and software for classifying documents |
| CA2470299A Expired - Lifetime CA2470299C (en) | 2001-11-02 | 2002-11-01 | Systems, methods, and software for classifying documents |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA2737943A Expired - Lifetime CA2737943C (en) | 2001-11-02 | 2002-11-01 | Systems, methods, and software for classifying documents |
Country Status (12)
| Country | Link |
|---|---|
| US (3) | US7062498B2 (en) |
| EP (2) | EP1464013B1 (en) |
| JP (3) | JP4342944B2 (en) |
| CN (1) | CN1701324B (en) |
| AT (1) | ATE421730T1 (en) |
| AU (2) | AU2002350112B8 (en) |
| CA (2) | CA2737943C (en) |
| DE (1) | DE60231005D1 (en) |
| DK (1) | DK1464013T3 (en) |
| ES (1) | ES2321075T3 (en) |
| NZ (1) | NZ533105A (en) |
| WO (1) | WO2003040875A2 (en) |
Families Citing this family (103)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6154757A (en) * | 1997-01-29 | 2000-11-28 | Krause; Philip R. | Electronic text reading environment enhancement method and apparatus |
| WO2002082224A2 (en) * | 2001-04-04 | 2002-10-17 | West Publishing Company | System, method, and software for identifying historically related legal opinions |
| US7062498B2 (en) * | 2001-11-02 | 2006-06-13 | Thomson Legal Regulatory Global Ag | Systems, methods, and software for classifying text from judicial opinions and other documents |
| US7139755B2 (en) | 2001-11-06 | 2006-11-21 | Thomson Scientific Inc. | Method and apparatus for providing comprehensive search results in response to user queries entered over a computer network |
| US7356461B1 (en) * | 2002-01-14 | 2008-04-08 | Nstein Technologies Inc. | Text categorization method and apparatus |
| US7188107B2 (en) * | 2002-03-06 | 2007-03-06 | Infoglide Software Corporation | System and method for classification of documents |
| US8201085B2 (en) * | 2007-06-21 | 2012-06-12 | Thomson Reuters Global Resources | Method and system for validating references |
| JP2006512693A (en) * | 2002-12-30 | 2006-04-13 | トムソン コーポレイション | A knowledge management system for law firms. |
| US20040133574A1 (en) | 2003-01-07 | 2004-07-08 | Science Applications International Corporaton | Vector space method for secure information sharing |
| US7089241B1 (en) * | 2003-01-24 | 2006-08-08 | America Online, Inc. | Classifier tuning based on data similarities |
| US7725544B2 (en) | 2003-01-24 | 2010-05-25 | Aol Inc. | Group based spam classification |
| US20040193596A1 (en) * | 2003-02-21 | 2004-09-30 | Rudy Defelice | Multiparameter indexing and searching for documents |
| US7590695B2 (en) | 2003-05-09 | 2009-09-15 | Aol Llc | Managing electronic messages |
| US7218783B2 (en) * | 2003-06-13 | 2007-05-15 | Microsoft Corporation | Digital ink annotation process and system for recognizing, anchoring and reflowing digital ink annotations |
| US7739602B2 (en) | 2003-06-24 | 2010-06-15 | Aol Inc. | System and method for community centric resource sharing based on a publishing subscription model |
| US7051077B2 (en) * | 2003-06-30 | 2006-05-23 | Mx Logic, Inc. | Fuzzy logic voting method and system for classifying e-mail using inputs from multiple spam classifiers |
| US8473532B1 (en) * | 2003-08-12 | 2013-06-25 | Louisiana Tech University Research Foundation | Method and apparatus for automatic organization for computer files |
| US20050097120A1 (en) * | 2003-10-31 | 2005-05-05 | Fuji Xerox Co., Ltd. | Systems and methods for organizing data |
| US7676739B2 (en) * | 2003-11-26 | 2010-03-09 | International Business Machines Corporation | Methods and apparatus for knowledge base assisted annotation |
| NZ571889A (en) * | 2003-12-31 | 2010-02-26 | Thomson Global Resources | Systems, methods, software and interfaces for integration of case law with legal briefs, litigation documents, and/or other litigation-support documents |
| WO2005066849A2 (en) * | 2003-12-31 | 2005-07-21 | Thomson Global Resources | Systems, methods, interfaces and software for extending search results beyond initial query-defined boundaries |
| US7647321B2 (en) * | 2004-04-26 | 2010-01-12 | Google Inc. | System and method for filtering electronic messages using business heuristics |
| US7680890B1 (en) | 2004-06-22 | 2010-03-16 | Wei Lin | Fuzzy logic voting method and system for classifying e-mail using inputs from multiple spam classifiers |
| US7953814B1 (en) | 2005-02-28 | 2011-05-31 | Mcafee, Inc. | Stopping and remediating outbound messaging abuse |
| US8484295B2 (en) | 2004-12-21 | 2013-07-09 | Mcafee, Inc. | Subscriber reputation filtering method for analyzing subscriber activity and detecting account misuse |
| KR20070061887A (en) * | 2004-09-21 | 2007-06-14 | 코닌클리케 필립스 일렉트로닉스 엔.브이. | Method of providing compliance information |
| US9015472B1 (en) | 2005-03-10 | 2015-04-21 | Mcafee, Inc. | Marking electronic messages to indicate human origination |
| US9160755B2 (en) | 2004-12-21 | 2015-10-13 | Mcafee, Inc. | Trusted communication network |
| US8738708B2 (en) * | 2004-12-21 | 2014-05-27 | Mcafee, Inc. | Bounce management in a trusted communication network |
| CN101151631A (en) * | 2005-01-28 | 2008-03-26 | 汤姆森环球资源公司 | Systems, methods, software for integration of case law, legal briefs, and litigation documents into law firm workflow |
| US7499591B2 (en) * | 2005-03-25 | 2009-03-03 | Hewlett-Packard Development Company, L.P. | Document classifiers and methods for document classification |
| EP1941402A1 (en) * | 2005-10-04 | 2008-07-09 | Thomson Global Resources | Systems, methods, and software for identifying relevant legal documents |
| US9177050B2 (en) | 2005-10-04 | 2015-11-03 | Thomson Reuters Global Resources | Systems, methods, and interfaces for extending legal search results |
| US20070078889A1 (en) * | 2005-10-04 | 2007-04-05 | Hoskinson Ronald A | Method and system for automated knowledge extraction and organization |
| US7917519B2 (en) * | 2005-10-26 | 2011-03-29 | Sizatola, Llc | Categorized document bases |
| US7529748B2 (en) * | 2005-11-15 | 2009-05-05 | Ji-Rong Wen | Information classification paradigm |
| CN100419753C (en) * | 2005-12-19 | 2008-09-17 | 株式会社理光 | Method and device for digital data central searching target file according to classified information |
| US8726144B2 (en) * | 2005-12-23 | 2014-05-13 | Xerox Corporation | Interactive learning-based document annotation |
| US7333965B2 (en) * | 2006-02-23 | 2008-02-19 | Microsoft Corporation | Classifying text in a code editor using multiple classifiers |
| KR100717401B1 (en) * | 2006-03-02 | 2007-05-11 | 삼성전자주식회사 | Method and apparatus for normalizing voice feature vector by backward cumulative histogram |
| US7735010B2 (en) * | 2006-04-05 | 2010-06-08 | Lexisnexis, A Division Of Reed Elsevier Inc. | Citation network viewer and method |
| US8392417B2 (en) * | 2006-05-23 | 2013-03-05 | David P. Gold | System and method for organizing, processing and presenting information |
| JP2008070958A (en) * | 2006-09-12 | 2008-03-27 | Sony Corp | Information processing device and method, and program |
| JP4910582B2 (en) * | 2006-09-12 | 2012-04-04 | ソニー株式会社 | Information processing apparatus and method, and program |
| US20080071803A1 (en) * | 2006-09-15 | 2008-03-20 | Boucher Michael L | Methods and systems for real-time citation generation |
| US7844899B2 (en) * | 2007-01-24 | 2010-11-30 | Dakota Legal Software, Inc. | Citation processing system with multiple rule set engine |
| US20080235258A1 (en) * | 2007-03-23 | 2008-09-25 | Hyen Vui Chung | Method and Apparatus for Processing Extensible Markup Language Security Messages Using Delta Parsing Technology |
| US9323827B2 (en) * | 2007-07-20 | 2016-04-26 | Google Inc. | Identifying key terms related to similar passages |
| DE102007034505A1 (en) * | 2007-07-24 | 2009-01-29 | Hella Kgaa Hueck & Co. | Method and device for traffic sign recognition |
| CN100583101C (en) * | 2008-06-12 | 2010-01-20 | 昆明理工大学 | Text categorization feature selection and weight computation method based on field knowledge |
| US10354229B2 (en) | 2008-08-04 | 2019-07-16 | Mcafee, Llc | Method and system for centralized contact management |
| US8352857B2 (en) * | 2008-10-27 | 2013-01-08 | Xerox Corporation | Methods and apparatuses for intra-document reference identification and resolution |
| WO2010141477A2 (en) | 2009-06-01 | 2010-12-09 | West Services, Inc. | Improved systems, methods, and interfaces for extending legal search results |
| WO2010141799A2 (en) * | 2009-06-05 | 2010-12-09 | West Services Inc. | Feature engineering and user behavior analysis |
| US8572084B2 (en) * | 2009-07-28 | 2013-10-29 | Fti Consulting, Inc. | System and method for displaying relationships between electronically stored information to provide classification suggestions via nearest neighbor |
| EP2471009A1 (en) | 2009-08-24 | 2012-07-04 | FTI Technology LLC | Generating a reference set for use during document review |
| US10146864B2 (en) * | 2010-02-19 | 2018-12-04 | The Bureau Of National Affairs, Inc. | Systems and methods for validation of cited authority |
| EP2583204A4 (en) | 2010-06-15 | 2014-03-12 | Thomson Reuters Scient Inc | System and method for citation processing, presentation and transport for validating references |
| US8195458B2 (en) * | 2010-08-17 | 2012-06-05 | Xerox Corporation | Open class noun classification |
| CN102033949B (en) * | 2010-12-23 | 2012-02-29 | 南京财经大学 | Correction-based K nearest neighbor text classification method |
| US9122666B2 (en) | 2011-07-07 | 2015-09-01 | Lexisnexis, A Division Of Reed Elsevier Inc. | Systems and methods for creating an annotation from a document |
| US9305082B2 (en) | 2011-09-30 | 2016-04-05 | Thomson Reuters Global Resources | Systems, methods, and interfaces for analyzing conceptually-related portions of text |
| WO2013123182A1 (en) * | 2012-02-17 | 2013-08-22 | The Trustees Of Columbia University In The City Of New York | Computer-implemented systems and methods of performing contract review |
| US9058308B2 (en) | 2012-03-07 | 2015-06-16 | Infosys Limited | System and method for identifying text in legal documents for preparation of headnotes |
| US9201876B1 (en) * | 2012-05-29 | 2015-12-01 | Google Inc. | Contextual weighting of words in a word grouping |
| US8955127B1 (en) * | 2012-07-24 | 2015-02-10 | Symantec Corporation | Systems and methods for detecting illegitimate messages on social networking platforms |
| CN103577462B (en) * | 2012-08-02 | 2018-10-16 | 北京百度网讯科技有限公司 | A kind of Document Classification Method and device |
| JP5526209B2 (en) | 2012-10-09 | 2014-06-18 | 株式会社Ｕｂｉｃ | Forensic system, forensic method, and forensic program |
| JP5823943B2 (en) * | 2012-10-10 | 2015-11-25 | 株式会社Ｕｂｉｃ | Forensic system, forensic method, and forensic program |
| US9083729B1 (en) | 2013-01-15 | 2015-07-14 | Symantec Corporation | Systems and methods for determining that uniform resource locators are malicious |
| US9189540B2 (en) * | 2013-04-05 | 2015-11-17 | Hewlett-Packard Development Company, L.P. | Mobile web-based platform for providing a contextual alignment view of a corpus of documents |
| US20150026104A1 (en) * | 2013-07-17 | 2015-01-22 | Christopher Tambos | System and method for email classification |
| JP2015060581A (en) * | 2013-09-20 | 2015-03-30 | 株式会社東芝 | Keyword extraction device, method and program |
| CN103500158A (en) * | 2013-10-08 | 2014-01-08 | 北京百度网讯科技有限公司 | Method and device for annotating electronic document |
| WO2015063784A1 (en) * | 2013-10-31 | 2015-05-07 | Hewlett-Packard Development Company, L.P. | Classifying document using patterns |
| US10255646B2 (en) | 2014-08-14 | 2019-04-09 | Thomson Reuters Global Resources (Trgr) | System and method for implementation and operation of strategic linkages |
| US20160048510A1 (en) * | 2014-08-14 | 2016-02-18 | Thomson Reuters Global Resources (Trgr) | System and method for integration and operation of analytics with strategic linkages |
| US10572877B2 (en) * | 2014-10-14 | 2020-02-25 | Jpmorgan Chase Bank, N.A. | Identifying potentially risky transactions |
| US9652627B2 (en) * | 2014-10-22 | 2017-05-16 | International Business Machines Corporation | Probabilistic surfacing of potentially sensitive identifiers |
| US20160162576A1 (en) * | 2014-12-05 | 2016-06-09 | Lightning Source Inc. | Automated content classification/filtering |
| US20160314184A1 (en) * | 2015-04-27 | 2016-10-27 | Google Inc. | Classifying documents by cluster |
| JP5887455B2 (en) * | 2015-09-08 | 2016-03-16 | 株式会社Ｕｂｉｃ | Forensic system, forensic method, and forensic program |
| US9852337B1 (en) * | 2015-09-30 | 2017-12-26 | Open Text Corporation | Method and system for assessing similarity of documents |
| US11176145B2 (en) * | 2015-10-17 | 2021-11-16 | Ebay Inc. | Generating personalized user recommendations using word vectors |
| CN106874291A (en) * | 2015-12-11 | 2017-06-20 | 北京国双科技有限公司 | The processing method and processing device of text classification |
| WO2017167836A1 (en) * | 2016-03-31 | 2017-10-05 | Bitdefender Ipr Management Ltd | System and methods for automatic device detection |
| US11347777B2 (en) * | 2016-05-12 | 2022-05-31 | International Business Machines Corporation | Identifying key words within a plurality of documents |
| AU2017274558B2 (en) | 2016-06-02 | 2021-11-11 | Nuix North America Inc. | Analyzing clusters of coded documents |
| CA3023079A1 (en) | 2016-06-16 | 2017-12-21 | Thomson Reuters Global Resources Unlimited Company | Scenario analytics system |
| US10146758B1 (en) * | 2016-09-30 | 2018-12-04 | Amazon Technologies, Inc. | Distributed moderation and dynamic display of content annotations |
| US10325409B2 (en) * | 2017-06-16 | 2019-06-18 | Microsoft Technology Licensing, Llc | Object holographic augmentation |
| CN107657284A (en) * | 2017-10-11 | 2018-02-02 | 宁波爱信诺航天信息有限公司 | A kind of trade name sorting technique and system based on Semantic Similarity extension |
| CN110390094B (en) * | 2018-04-20 | 2023-05-23 | 伊姆西Ip控股有限责任公司 | Method, electronic device and computer program product for classifying documents |
| US11087088B2 (en) * | 2018-09-25 | 2021-08-10 | Accenture Global Solutions Limited | Automated and optimal encoding of text data features for machine learning models |
| US11862305B1 (en) | 2019-06-05 | 2024-01-02 | Ciitizen, Llc | Systems and methods for analyzing patient health records |
| US11424012B1 (en) * | 2019-06-05 | 2022-08-23 | Ciitizen, Llc | Sectionalizing clinical documents |
| US11170271B2 (en) * | 2019-06-26 | 2021-11-09 | Dallas Limetree, LLC | Method and system for classifying content using scoring for identifying psychological factors employed by consumers to take action |
| US11636117B2 (en) | 2019-06-26 | 2023-04-25 | Dallas Limetree, LLC | Content selection using psychological factor vectors |
| CN110377742A (en) * | 2019-07-23 | 2019-10-25 | 腾讯科技（深圳）有限公司 | Text classification evaluating method, device, readable storage medium storing program for executing and computer equipment |
| WO2022015798A1 (en) * | 2020-07-14 | 2022-01-20 | Thomson Reuters Enterprise Centre Gmbh | Systems and methods for the automatic categorization of text |
| US11775592B2 (en) * | 2020-08-07 | 2023-10-03 | SECURITI, Inc. | System and method for association of data elements within a document |
| US11941497B2 (en) * | 2020-09-30 | 2024-03-26 | Alteryx, Inc. | System and method of operationalizing automated feature engineering |
| US11782957B2 (en) * | 2021-04-08 | 2023-10-10 | Grail, Llc | Systems and methods for automated classification of a document |
Family Cites Families (34)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US583120A (en) * | 1897-05-25 | Soldeeing machine | ||
| US5054093A (en) * | 1985-09-12 | 1991-10-01 | Cooper Leon N | Parallel, multi-unit, adaptive, nonlinear pattern class separator and identifier |
| US5157783A (en) | 1988-02-26 | 1992-10-20 | Wang Laboratories, Inc. | Data base system which maintains project query list, desktop list and status of multiple ongoing research projects |
| US4961152A (en) * | 1988-06-10 | 1990-10-02 | Bolt Beranek And Newman Inc. | Adaptive computing system |
| US5488725A (en) | 1991-10-08 | 1996-01-30 | West Publishing Company | System of document representation retrieval by successive iterated probability sampling |
| US5265065A (en) | 1991-10-08 | 1993-11-23 | West Publishing Company | Method and apparatus for information retrieval from a database by replacing domain specific stemmed phases in a natural language to create a search query |
| US5383120A (en) * | 1992-03-02 | 1995-01-17 | General Electric Company | Method for tagging collocations in text |
| US5438629A (en) * | 1992-06-19 | 1995-08-01 | United Parcel Service Of America, Inc. | Method and apparatus for input classification using non-spherical neurons |
| US5497317A (en) | 1993-12-28 | 1996-03-05 | Thomson Trading Services, Inc. | Device and method for improving the speed and reliability of security trade settlements |
| US5434932A (en) | 1994-07-28 | 1995-07-18 | West Publishing Company | Line alignment apparatus and process |
| WO1996034344A1 (en) * | 1995-04-27 | 1996-10-31 | Northrop Grumman Corporation | Adaptive filtering neural network classifier |
| US5778397A (en) * | 1995-06-28 | 1998-07-07 | Xerox Corporation | Automatic method of generating feature probabilities for automatic extracting summarization |
| US5918240A (en) * | 1995-06-28 | 1999-06-29 | Xerox Corporation | Automatic method of extracting summarization using feature probabilities |
| DE19526264A1 (en) * | 1995-07-19 | 1997-04-10 | Daimler Benz Ag | Process for creating descriptors for the classification of texts |
| US5644720A (en) | 1995-07-31 | 1997-07-01 | West Publishing Company | Interprocess communications interface for managing transaction requests |
| JP3040945B2 (en) | 1995-11-29 | 2000-05-15 | 松下電器産業株式会社 | Document search device |
| US6539352B1 (en) * | 1996-11-22 | 2003-03-25 | Manish Sharma | Subword-based speaker verification with multiple-classifier score fusion weight and threshold adaptation |
| JPH1185797A (en) * | 1997-09-01 | 1999-03-30 | Canon Inc | Automatic document classification device, learning device, classification device, automatic document classification method, learning method, classification method and storage medium |
| US6052657A (en) | 1997-09-09 | 2000-04-18 | Dragon Systems, Inc. | Text segmentation and identification of topic using language models |
| JP3571231B2 (en) * | 1998-10-02 | 2004-09-29 | 日本電信電話株式会社 | Automatic information classification method and apparatus, and recording medium recording automatic information classification program |
| AU1122100A (en) * | 1998-10-30 | 2000-05-22 | Justsystem Pittsburgh Research Center, Inc. | Method for content-based filtering of messages by analyzing term characteristicswithin a message |
| JP2000222431A (en) * | 1999-02-03 | 2000-08-11 | Mitsubishi Electric Corp | Document classifying device |
| EP1212699A4 (en) | 1999-05-05 | 2006-01-11 | West Publishing Co | Document-classification system, method and software |
| JP2001034622A (en) * | 1999-07-19 | 2001-02-09 | Nippon Telegr & Teleph Corp <Ntt> | Document sorting method and its device, and recording medium recording document sorting program |
| CA2381460A1 (en) * | 1999-08-06 | 2001-02-15 | James S. Wiltshire, Jr. | System and method for classifying legal concepts using legal topic scheme |
| SG89289A1 (en) * | 1999-08-14 | 2002-06-18 | Kent Ridge Digital Labs | Classification by aggregating emerging patterns |
| US6651058B1 (en) * | 1999-11-15 | 2003-11-18 | International Business Machines Corporation | System and method of automatic discovery of terms in a document that are relevant to a given target topic |
| US7565403B2 (en) * | 2000-03-16 | 2009-07-21 | Microsoft Corporation | Use of a bulk-email filter within a system for classifying messages for urgency or importance |
| US20020099730A1 (en) * | 2000-05-12 | 2002-07-25 | Applied Psychology Research Limited | Automatic text classification system |
| US6751600B1 (en) * | 2000-05-30 | 2004-06-15 | Commerce One Operations, Inc. | Method for automatic categorization of items |
| US6782377B2 (en) * | 2001-03-30 | 2004-08-24 | International Business Machines Corporation | Method for building classifier models for event classes via phased rule induction |
| US7295965B2 (en) * | 2001-06-29 | 2007-11-13 | Honeywell International Inc. | Method and apparatus for determining a measure of similarity between natural language sentences |
| US20030130993A1 (en) * | 2001-08-08 | 2003-07-10 | Quiver, Inc. | Document categorization engine |
| US7062498B2 (en) * | 2001-11-02 | 2006-06-13 | Thomson Legal Regulatory Global Ag | Systems, methods, and software for classifying text from judicial opinions and other documents |
-
2001
- 2001-12-21 US US10/027,914 patent/US7062498B2/en not_active Expired - Lifetime
-
2002
- 2002-11-01 AT AT02786640T patent/ATE421730T1/en not_active IP Right Cessation
- 2002-11-01 CA CA2737943A patent/CA2737943C/en not_active Expired - Lifetime
- 2002-11-01 AU AU2002350112A patent/AU2002350112B8/en not_active Expired
- 2002-11-01 CN CN028266501A patent/CN1701324B/en not_active Expired - Fee Related
- 2002-11-01 ES ES02786640T patent/ES2321075T3/en not_active Expired - Lifetime
- 2002-11-01 NZ NZ533105A patent/NZ533105A/en not_active IP Right Cessation
- 2002-11-01 JP JP2003542441A patent/JP4342944B2/en not_active Expired - Fee Related
- 2002-11-01 EP EP02786640A patent/EP1464013B1/en not_active Expired - Lifetime
- 2002-11-01 DK DK02786640T patent/DK1464013T3/en active
- 2002-11-01 DE DE60231005T patent/DE60231005D1/en not_active Expired - Lifetime
- 2002-11-01 CA CA2470299A patent/CA2470299C/en not_active Expired - Lifetime
- 2002-11-01 EP EP08017291A patent/EP2012240A1/en not_active Ceased
- 2002-11-01 WO PCT/US2002/035177 patent/WO2003040875A2/en active Application Filing
-
2005
- 2005-08-30 US US11/215,715 patent/US7580939B2/en not_active Expired - Lifetime
-
2009
- 2009-04-20 JP JP2009102510A patent/JP5392904B2/en not_active Expired - Fee Related
- 2009-07-23 AU AU2009202974A patent/AU2009202974B2/en not_active Expired
- 2009-08-21 US US12/545,642 patent/US20100114911A1/en not_active Abandoned
-
2013
- 2013-06-21 JP JP2013130658A patent/JP2013178851A/en not_active Withdrawn
Also Published As
| Publication number | Publication date |
|---|---|
| EP1464013B1 (en) | 2009-01-21 |
| ES2321075T3 (en) | 2009-06-02 |
| US7062498B2 (en) | 2006-06-13 |
| DK1464013T3 (en) | 2009-05-18 |
| WO2003040875A2 (en) | 2003-05-15 |
| EP1464013A2 (en) | 2004-10-06 |
| CN1701324A (en) | 2005-11-23 |
| JP2005508542A (en) | 2005-03-31 |
| CA2737943A1 (en) | 2003-05-15 |
| JP2009163771A (en) | 2009-07-23 |
| CA2470299C (en) | 2011-04-26 |
| JP5392904B2 (en) | 2014-01-22 |
| JP4342944B2 (en) | 2009-10-14 |
| ATE421730T1 (en) | 2009-02-15 |
| US20100114911A1 (en) | 2010-05-06 |
| CA2737943C (en) | 2013-07-02 |
| AU2009202974B2 (en) | 2012-07-19 |
| DE60231005D1 (en) | 2009-03-12 |
| EP2012240A1 (en) | 2009-01-07 |
| JP2013178851A (en) | 2013-09-09 |
| AU2002350112B8 (en) | 2009-04-30 |
| CN1701324B (en) | 2011-11-02 |
| US20030101181A1 (en) | 2003-05-29 |
| NZ533105A (en) | 2006-09-29 |
| US7580939B2 (en) | 2009-08-25 |
| AU2002350112A1 (en) | 2003-05-19 |
| AU2002350112B2 (en) | 2009-04-23 |
| US20060010145A1 (en) | 2006-01-12 |
| AU2009202974A1 (en) | 2009-08-13 |
| WO2003040875A3 (en) | 2003-08-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA2470299A1 (en) | Systems, methods, and software for classifying documents | |
| US7899816B2 (en) | System and method for the triage and classification of documents | |
| KR101219366B1 (en) | Classification of ambiguous geographic references | |
| US8706475B2 (en) | Method and apparatus for detecting a table of contents and reference determination | |
| Robertson et al. | The TREC 2002 Filtering Track Report. | |
| US8078601B1 (en) | Determining unambiguous geographic references | |
| EP2287750B1 (en) | Methods and apparatus to classify text communications | |
| EP1524610A3 (en) | Systems and methods for performing electronic information retrieval | |
| EP0996927A1 (en) | Text classification system and method | |
| US7587381B1 (en) | Method for extracting a compact representation of the topical content of an electronic text | |
| AU2005203239A1 (en) | Phrase-based indexing in an information retrieval system | |
| CN106649849A (en) | Text information base building method and device and searching method, device and system | |
| Mishra et al. | Text document clustering on the basis of inter passage approach by using K-means | |
| US20080082483A1 (en) | Method and apparatus for normalizing protein name using ontology mapping | |
| JP3333998B2 (en) | Automatic classifying apparatus and method | |
| JPH10254883A (en) | Automatic document sorting method | |
| KR100435442B1 (en) | Method And System For Summarizing Document | |
| JP2002007433A (en) | Information sorter, information sorting method, computer readable recording medium recorded with information sorting program and information sorting program | |
| Khaleel et al. | An automatic text classification system based on genetic algorithm | |
| KR101158750B1 (en) | Text classification device and classification method thereof | |
| CN107092679B (en) | Feature word vector obtaining method and text classification method and device | |
| JP4479745B2 (en) | Document similarity correction method, program, and computer | |
| CN114461624A (en) | Unbalanced data processing method for traffic accident analysis | |
| Tran et al. | Learning based approaches for vietnamese question classification using keywords extraction from the web | |
| Hirsch et al. | Evolving rules for document classification |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request | ||
| MKEX | Expiry |
Effective date: 20221101 |