WO2016094206A1

WO2016094206A1 - Method and apparatus for processing information

Info

Publication number: WO2016094206A1
Application number: PCT/US2015/063881
Authority: WO
Inventors: Arden A. Ash; Krystle SWAVING; Victor Wang
Original assignee: Thomson Licensing
Priority date: 2014-12-11
Filing date: 2015-12-04
Publication date: 2016-06-16

Abstract

The present principles generally relate to an apparatus and a method for processing information. In one exemplary embodiment, an informational element is received from a user. The system provides a supplemental informational element for the input informational element if the input informational element matches a user-selected category of input informational elements. The provided supplemental informational element is selected from one of a plurality of user-selected categories of supplemental informational elements. In one embodiment, the input informational element is processed by facial recognition.

Description

METHOD AND APPARATUS FOR PROCESSING INFORMATION

BACKGROUND OF THE INVENTION

Field of the Invention

The present principles generally relate to an apparatus and a method for processing information. In one exemplary embodiment, an informational element is received from a user. The system provides a supplemental informational element for the input informational element if the input informational element matches a user- selected category of input informational elements. The provided supplemental informational element is selected from one of a plurality of user-selected categories of supplemental informational elements. In one embodiment, the input informational element is processed via facial recognition.

Background Information

It is well known that internet websites such as, e.g., Yahoo.com,

Huffingtonpost.com, Wikipedia.com, and etc., have certain elements on their webpages that when selected by a user, the user will be directed to the internet links associated with these elements using hyperlinks. A hyperlink in computing is a reference to data that a user can directly follow either by clicking or by hovering (e.g., by moving a cursor using a mouse), or that is followed automatically. A software system for viewing and creating hypertext is a hypertext system, and to create a hyperlink is to hyperlink (or simply to link). A user following hyperlinks is said to navigate or browse the hypertext. In addition, different streaming media sites (e.g., Hulu, Netflix, M-GO, and etc.) currently provide various user interfaces for users to search media assets for view and/or purchase. The users may search e.g., different movies and/or shows available on these websites by typing in a query string related to movie or show titles, using keyboards on their e.g., PCs, laptops, cellphones, and various other user devices.

It is well known that Google provides a search engine to allow a user to enter a text string to search content on the internet. Google also provides another separate website (images.google.com) which allows a user to search the internet content related to an image by uploading the image or entering a URL corresponding to the image.

SUMMARY OF THE INVENTION

The present inventor recognizes that the existing systems and methods, however, do not provide a unified website, service or system which would allow users to select dynamically which categories of input informational elements to be processed and which categories of output supplemental informational elements to be provided for the selected input informational elements. Therefore, the existing systems and methods do not provide an easy, informative and efficient user interface for users to obtain and navigate information. Hence, there is a need to improve the existing systems and methods for processing input information to provide output supplemental information. In accordance with an aspect of the present principles, an apparatus is presented, comprising:

a user input device configured to receive an informational element; and a processor configured to supplement said received informational element with at least one supplemental informational element selected from a plurality of supplemental informational elements, wherein said at least one supplemental informational element is selected from the plurality of supplemental informational elements in response to at least one of a first plurality of user-selected category identifiers capable of identifying a plurality of user-selected supplemental informational elements, and wherein a category of said received informational element matches one of a second plurality of user- selected category identifiers capable of identifying user-selected categories of informational elements.

In another exemplary embodiment, an apparatus is presented, comprising: a user input device configured receiving an informational element from a user input device; and

a processor configured to determine if the received informational element is an image; if the received informational element is an image, then the processor further processes the image via image recognition to provide processed image information; and the processor further provides at least one supplemental informational element in response to the recognized image if the processed image information belongs to the one of a user-selected category of informational elements.

In another exemplary embodiment, a method is presented comprising:

receiving an informational element via a user input device; and

supplementing, by a processor, said received informational element with at least one supplemental informational element selected from a plurality of supplemental informational elements, wherein said at least one supplemental informational element is selected from the plurality of supplemental informational elements in response to at least one of a first plurality of user-selected category identifiers capable of identifying a plurality of user-selected supplemental informational elements, and wherein a category of said received informational element matches one of a second plurality of user- selected category identifiers capable of identifying user- selected categories of informational elements.

In another exemplary embodiment, a method is presented comprising:

receiving an informational element from a user input device;

determining if the received informational element is an image;

if the received informational element is an image, then processing the image via image recognition to provide processed image information;

determining if the received informational element belongs to one of a user- selected category of informational elements based on the processed image information; and

providing at least one supplemental informational element in response to the recognized image if the processed image information belongs to the one of a user- selected category of informational elements.

In another exemplary embodiment, a computer program product stored in a non-transitory computer-readable storage media is presented, comprising computer- executable instructions for:

receiving an informational element via a user input device; and

receiving (140) an informational element from a user input device (280);

determining (145) if the received informational element is an image;

if the received informational element is an image, then processing (150) the image via image recognition to provide processed image information;

determining (160) if the received informational element belongs to one of a user-selected category of informational elements based on the processed image information; and

providing (170) at least one supplemental informational element in response to the recognized image if the processed image information belongs to the one of a user- selected category of informational elements.

DETAILED DESCRIPTION OF THE DRAWINGS

The above-mentioned and other features and advantages of this invention, and the manner of attaining them, will become more apparent and the invention will be better understood by reference to the following description of embodiments of the invention taken in conjunction with the accompanying drawings, wherein:

FIG. 1A and FIG. IB show exemplary processes according to the present principles;

FIG. 2 shows an example system according to the present principles;

FIG. 3A to FIG. 3C show exemplary user input device screens and their functions according to the present principles;

FIG. 4 shows an exemplary data structure according to the present principles; and

FIG. 5 shows another exemplary system according to the present principles. The examples set out herein illustrate exemplary embodiments of the present principles. Such examples are not to be construed as limiting the scope of the invention in any manner.

DETAILED DESCRIPTION

FIG. 2 shows an exemplary system according to the present principles. Fig. 2 illustrates, e.g., a system capable of receiving and processing a user input. According to the present principles, a user input may comprise one or more of the following input informational elements such as e.g.,: 1) an image, 2) a text string, 3) an internet link, 4) video content, and 5) audio content, and etc. An input informational element may be provided by a user and received by one of a plurality of exemplary end user devices 260-1 to 260-n. According to another aspect of the present principles, the input informational element may be contained in an electronic file.

As used herein, an electronic file is an electronic content or document which may be in various formats and/or in compliance with various standards. An electronic file can be processed by a computer, and may comprise e.g., one or more of: 1) an electronic text document, 2) a web page, 3) a HTML file, 4) an e-mail, 5) a PDF file, 6) a JavaScript file, 7) a Microsoft Word document, and etc. Also as used herein, an image is synonymous with a photo, picture, graphical design, drawing, and etc. and is typically generated in the present day by an electronic camera or the like, obtained from various websites of the internet, from different drawing or graphics software, scanned into the electronic form by a scanner or the like from a paper photography, and etc.

Various user devices 260-1 to 260-n in FIG. 2 may communicate with an exemplary server 205 over a communication network 250 such as the internet, a wide area network (WAN), and/or a local area network (LAN). Server 205 may communicate with user devices 260-1 to 260-n in order to provide further processing and/or provide relevant data not available on the local user devices 260-1 to 260-n. As an example, server 205 may be a computer having a processor 210 such as, e.g., an Intel processor, running an appropriate operating system such as, e.g., Windows 2008 R2, Windows Server 2012, Linux operating system, and etc. Devices 260-1 to 260-n may access different web pages, services or databases provided by server 205 using, e.g., HTTP protocol. A well-known web server software application which may be run by server 205 to provide web pages is Apache HTTP Server software available from http://www.apache.org.

In addition, server 205 may also provide media content services similar to, e.g., Amazon.com, Netflix, or M-GO. Server 205 may use a streaming protocol such as e.g., Apple HTTP Live Streaming (HLS) protocol, Adobe Real-Time Messaging Protocol (RTMP), Microsoft Silverlight Smooth Streaming Transport Protocol, and etc., to transmit various media assets such as, e.g., video programs, audio programs, movies, TV shows, software, games, electronic books, electronic magazines, electronic articles, and etc., to an end-user device 260-1 for purchase and/or view.

In addition, a server administrator may interact with and configure server 205 to run different applications using user I/O devices 215 (e.g., a keyboard and/or a display) as well known in the art. Furthermore, various webpages, data, media assets and their associated metadata may be stored in a database 225 and accessed by processor 210 as needed. In particular, database 225 may store an exemplary data structure 401 shown in FIG. 4, to be described in detail later. Database 225 may reside in appropriate storage media, such as, e.g., one or more hard drives and/or other suitable memory devices, as well known in the art.

Server 205 is connected to network 250 through a communication input device 220 for communicating with other servers or web sites (not shown) and to one or more user devices 260-1 to 260-n, as shown in FIG. 2. In addition, one skilled in the art would readily appreciate that other server components, such as, e.g., ROM, RAM, power supply, cooling fans, etc., may also be needed, but are not shown in FIG. 2 to simplify the drawing. User devices 260-1 to 260-n shown in FIG. 2 may comprise one or more of, e.g., a PC, a laptop, a tablet, a cellphone, and etc. One of such devices may be, e.g., a Microsoft Windows 7 or Windows 8 computer, an Android phone (e.g., Samsung S3, S4, or S5), an Apple IOS phone (e.g., IPhone 5S or 5C), or an Apple IPad. For example, a detailed block diagram of an exemplary device according to the present principles is illustrated in block 260-1 of FIG. 2 as Device 1.

Exemplary user device 260-1 in FIG. 2 comprises a processor 265 for processing various data and for controlling various functions and components of the device 260-1. In additional, device 260-1 also comprises user I/O devices 280 which may include, e.g., a touch and/or a physical keyboard for inputting user data, and/or a display and/or a speaker for outputting visual and/or audio user data and feedback. Device 260-1 also comprises a memory 285 for processing and storing different files and information as necessary, including webpages, user interface information, and/or user profiles shown in FIG. 3A to FIG. 3C and relevant information in an exemplary data structure 401 of a database 400 shown in FIG. 4 and to be described later. Device 260-1 also comprises a communication interface 270 for connecting and communicating to/from server 205 and/or other devices, via, e.g., network 250 using e.g., a connection through a cable network, a FIOS network, a Wi-Fi network, and/or a cellphone network (e.g., 3G, 4G, LTE), and etc.

FIG. 3A to FIG. 3C illustrate exemplary user interface screens 300, 310 and 312, and their functions according to the present principles. These user interface screens and functions may be controlled and/or provided by e.g., processor 265 in device 260-1 of FIG. 4 and/or processor 210 in web server 205 remotely. FIG. 3A to FIG. 3C will be described in detailed below in connection with the exemplary processes shown in FIG. 1A and FIG. IB. FIG. 1A and FIG. IB are respective flow charts of exemplary processes 100 and 190 according to the present principles. Processes 100 and 190 may be implemented as computer executable instructions which may be executed by, e.g., a processor 265 in device 260-1 in FIG. 2 and/or a processor 210 in server 205 of FIG 2. For example, the computer program products having the computer-executable instructions may be stored in non-transitory computer-readable storage media of the respective device 260-1 and/or server 205. The exemplary control programs shown in FIG. 1A and FIG. IB, when executed, facilitate processing and displaying of user interfaces screens shown, for example, in FIG. 3A to FIG. 3C, and controlling of their respective functions and interactions with a user. One skilled in the art can readily recognize that the exemplary processes shown in FIG. 1A and FIG. IB may also be implemented using a combination of hardware and software (e.g., a firmware implementation), and/or executed using logic arrays or ASIC. At step 110 of FIG. 1, in an exemplary embodiment, an informational element is received from a user as a user input from one of a user device 260- 1 to 260-n shown in FIG. 2, using e.g., one or more of user Input/Output (I/O) devices 280 shown in device 260-1. An example of a user input is also illustrated in a user input section 301 shown in FIG. 3A.

FIG 3A is an exemplary display screen 300 of a user device 260-1 according to the present principles. For example, a user may first enter a HTTP or web address "www.mgo.com" 302 at an address bar 304 of a web browser 306 to provide user input informational elements and respective selections of input informational element and output supplemental informational element categories (e.g., as shown in elements 320, 322, 340, 342) according to the exemplary process 100 shown in FIG. 1A and to be described in more detail below.

User input section 301 of FIG. 3 A allows a user to provide, and user device 260-1 to receive, one or more of user input informational elements. As shown in FIG. 3A, a user has entered a text search query with a first informational element of "Breakfast at Tiffany's" 311 at a text search query field 313 of FIG. 3 A. In addition, the user also has entered and uploaded another informational element, a picture of Audrey Hepburn 315 in the area "upload search content" 316 for the search query. One skilled in the art may readily appreciate that the method of entering a search text query 311, as well as entering and uploading an input informational element such as a picture 315, are well known in the art. Also, the format of the exemplary user input section 301 shown in FIG.3 is for illustration purpose only and other possible formats may be used instead without departing from the teachings of the present principles. In addition, to execute a search or a query, a user may either enter or push a return key on a keyboard, or highlight and select the search icon 317 shown in FIG. 3A.

At step 120 of FIG. 1A, a supplemental informational element is provided by e.g., user device 260-1, to supplement the exemplary user input informational elements 315 and 311 shown in input section 301 of FIG. 3A. As used herein, a supplemental informational element is an output informational element which provides supplemental or additional information for the user-input informational element. An example of supplemental information may be a result of a search from a user input to be described in more detail below

FIG 3B also shows an exemplary display screen 310 of a user device 260-1 according to the present principles. As shown in FIG. 3B, display screen 310 contains an output section 308 which may have one or more exemplary output supplement informational elements, provided in response to a user input as shown, e.g., in section 301 of FIG. 3 A.

Also at step 102 of FIG. 1A, at least one supplemental informational element is selected from a plurality of supplemental informational elements. The at least one supplemental informational element is selected from the plurality of supplemental informational elements in response to at least one of a first plurality of user-selected category identifiers capable of identifying a plurality of user- selected supplemental informational elements. This user selection is also illustrated in both FIG. 3A and 3B. For example, a user may select one or more category identifiers 332-1 to 332-n shown in "Output Categories" section 322 of FIG. 3 A and FIG 3B. As illustrated, the plurality of user selectable identifiers 332-1 to 332-n respectively identify different selectable categories which may include, for example, 1) type of people, 2) type of objects, 3) type of places, 4) type of media assets, 5) type of subjects, 6) type of interests, and etc.

In the example of FIG. 3A and FIG. 3B, a user has selected output categories of "Things to Buy" 332-4 and "Places to Visit" 332-5, by e.g., highlighting them with one of user selection devices 280 (such as a mouse) of device 260-1 in FIG. 2. According to the present principles, user-selected categories of output supplemental informational elements will be provided in the output section 321 of FIG. 3B, in response to the user-selected categories of output to be described below.

According to another exemplary embodiment, as part of the output supplemental informational element category selections, a user may also select an additional layer of category identifiers for the output supplemental informational elements to be provided. This is illustrated in section 342 of both FIG. 3A and FIG. 3B. As illustrated, a user may select the additional category identifiers 352-1 to 352-n corresponding to one of a: 1) graphical image, 2) picture, 3) text string, 4) internet link, 5) video content, 6) audio content, and etc.

Note that 3A and 3B illustrate an example of selecting the additional identifiers 352-1 to 352-n as a second layer of selection in addition to the primary selections made in section 322 as described before. However, in an alternative embodiment not shown, all the category identifier selections in both section 342 and section 322 may be combined with into one layer and shown in selection section instead.

In addition, at step 120 of FIG. 1A, according to the present principles, the output supplemental informational elements shown in output section 308 of FIG. IB will be provided in response to the received input informational element matching one of a plurality of user-selected category identifiers identifying user-selected categories of input informational elements, as shown in e.g., sections 320 and 340 of FIG 3A and 3B. That is, for example, FIG. 3A and FIG 3B show that a user has selected an informational element identifier "Movies" 330-1 in section 320. In addition, the user has also selected a second layer of informational element identifiers of

"picture/image" 350-1 and "text string" 350-2 in section 340. The process of the selections of in sections 320 and 340 is similar to that which has already been described above for the selection of output category identifiers in sections 322 and 342 of FIG. 3A and FIG. 3B and is not repeated here.

According to the present principles, a user input containing a plurality of input informational elements will be processed according to a plurality of user selected category identifiers. For example, as shown in FIG. 3 A, when a user selects identifiers "Movies" 330-1, "picture/image" 350-1 and "text string" 350-2, only picture and text string informational elements matching a category entry of Movies in a database shown in FIG. 4 will be processed (as to be described later). The other categories of informational elements will be ignored. FIG. 4 shows an exemplary data structure 401 of an exemplary relational database 400 (e.g., also shown as element 225 and/or element 285 of FIG. 2) according to the present principles. Data structure 401 may comprise two columns of database tuples or records 410 and 420, each column corresponds respectively to input informational elements and output supplemental informational elements. Under each column, the records are organized by grouping the elements by the first layer of category identifiers such as those identifiers shown in section 320 or section 322 of FIG. 3A and FIG. 3B, and then grouped by the second layer of category identifiers such as those shown in section 340 or section 342 of FIG. 3 A and FIG. 3B. One skilled in the art can readily recognize that other suitable data structure formats and implementations, or the types of databases, may be utilized as well known in the art.

A plurality of relationship links 430-1 to 430-n identify the linkage and relationship between the input and out elements in the relational database 400 so that one of more of the related supplemental informational elements may be provided in response to one or more of input informational elements, according to the processes of FIG. 1A and FIG. IB.

For example, in an embodiment as shown in FIG. 4, for an input informational element "Breakfast at Tiffany's" 450-1 in column 410, its relational links 430-1 to 430-5 point to a plurality of output supplemental informational elements in any of the output categories in column 420 which may have some relationship or connection with the element "Breakfast at Tiffany's" 450-1. The linkages between the input and the output elements in database 400 may be predefined, provided by a user or a server, and etc., as to be described in connection with FIG. 5 below.

In addition, as described previously in connection with FIG. 1A, when a user selects identifiers of "Movies" 330-1, "picture" 350-1 and "text string" 350-2, the user's entered informational elements of Audrey Hepburn's picture 315 and the text string "Breakfast at Tiffany's" will be searched against the entries in column 410 the data structure 401 in FIG. 4. In this example, the record of a text string "Breakfast at Tiffany's" 450-1 and the record of the picture of Audrey Hepburn 450-2 will be found in column 410 of data structure 401. Once the user-selected input informational elements are identified in the first column 410 of the data structure 401, the relationship arrows 430-1 to 430-n will locate the potential supplemental

informational elements to be provided according to the present principles.

As noted before, in one exemplary embodiment according to the present principles, only one or more of user-selected categories of output supplemental informational elements will be provided to the user. For example, as described before in connection with the process 100 of FIG. 1A and shown in user output section 308 of FIG. 3B, since the user selected only the output category identifier of "Things to Buy" 332-5, "Places to Visit" 332-6, "picture" 352-1 and "internet link" 352-3, only those categories of supplemental informational elements in column 420 of the relational data base 400 will be provided to the user. That is, e.g., only elements 460- 2, 460-3 and 460-6 from column 420 of FIG. 4 will be provided to the user. Other supplemental informational elements are ignored and not provided to the user according to the present principles, even though they may relate to an input informational element and provide additional information to the input informational element.

The resulting output is shown in FIG. 3B where the output supplemental informational elements 308-1 is a picture showing a Tiffany neckless to buy, and element 308-2 is an internet link of Tiffany.com, where the Tiffany neckless may be purchased. Additionally, element 308-3 is also an internet link showing the website of 5th Avenue which is a place to visit in connection with the input informational element of the movie "Breakfast at Tiffany's," since Tiffany & Co. is located on the 5th Avenue in NYC. Therefore, these exemplary supplemental elements 308-1 to 308-3 correspond to the user selected categories of output supplemental informational elements as shown in FIG. 3 A and 3B.

Also as noted before, an informational element may be contained in one of: 1) an electronic text document, 2) a web page, 3) a HTML file, 4) an e-mail, 5) a PDF file, 6) a JavaScript file, 7) a Microsoft Word document, and etc. An example of when an input informational element is contained in a file is shown in FIG. 3C, where the user input section 301 contains a user input file 316 which may contain, e.g., text informational elements "Audrey Hepburn" 353, "Roman Holidays" 354 and

"Charade" 355; and a graphical image informational element 356. In another exemplary embodiment, selections of input and output categories of elements may be part of a user profile which the user may have already preselected or can be changed dynamically. This exemplary user profile is shown e.g., as an area 390 surround by a dashed box in FIGs 3A to 3C.

In addition, according to the present principles, information in an exemplary database 400 shown in FIG. 4 may be populated in various ways as shown in FIG. 5. In one exemplary embodiment, default sets of input and output informational elements shown in database 400 of FIG. 4 may be pre -populated. This can either be provided by a server 205 of FIG. 2, as illustrated by element 510 of FIG. 5, and/or locally from a local database of a device 260-1, as illustrated by element 520 of FIG. 5. In one exemplary embodiment, server 205 may populate the default sets of information via a web crawler server as shown by element 530 of FIG. 5, or a user device may also directly implement and use a web crawler as shown by element 540 of FIG. 5.

Modern web searches may be done by search engines called "Web Spiders" or "Web Crawlers" and are used for data grabbing and gathering. This technology is well known in the art (see, e.g., http://en.wikipedia.org/wikiAVeb_crawler). Such a web- crawler typically needs some information such as, e.g., a start page, a depth of search tree and wanted details (e.g., years, links, and etc.). The web crawling of the categories for both input informational elements and output supplemental informational elements according to the principles of the present invention may be provided by, e.g., web server 205 or a device 260-1 as noted before. For example, information on input and output elements corresponding to movies and actor categories could be grabbed from e.g., the following databases:

http://en.wikipedia.org/wiki/Lists_of_actors

Apple iTunes and iTunes store

- www.imbd.com (international movie data base)

local contact/address lists

audio and/or video playlists of a user device FIG. 5 also illustrates other exemplary methods of populating a local database of information to be used by e.g., device 260-1 according to the present principles. Besides having predetermined default sets of information either provided by a web server remotely (e.g., 510, 530 of FIG. 5), or locally in a user device 260-n (e.g., 520, 540 of FIG. 5), a user can also select user-defined sets of informational elements or keywords for one or more corresponding user-defined categories as shown in element 550 of FIG. 5. As shown in element 550, one exemplary way of doing this is for a user to enter the relevant information in an exemplary data structure 401 shown e.g., in FIG. 4, manually via e.g., a keyboard and/or other user input device.

Furthermore, as shown in element 590 of FIG. 5, an exemplary embodiment of the present invention may be that an application provider of the present invention may allow or provide different personalized sets of information from different users of the same application to be able to be exchanged and/or downloaded to the local device 260-1. For example, each personalized set of information may have different emphasis on different categories of topics, and thus would tend to have more informational elements and supplemental informational elements for the emphasized categories. In one exemplary embodiment, one or more of the above described exemplary methods of gathering the sets of information may be combined to form combined information sets to be used by an end-user device, as shown in element 505 of FIG. 5.

In another exemplary embodiment according to the present principles, a process 190 is shown in FIG. IB. At step 140 of FIG. IB, a device such as a user device 260-1 shown in FIG. 2 receives an informational element input by a user from a user interface. At step 145, a determination is made as to whether the received informational element is an image. At step 150, if the received informational element is an image as determined by step 145, then the informational element is processed by image recognition to provide processed image information. If on the other hand, the received informational element is not an image, then the process can proceed as described before at step 120 of FIG. 1A. One skilled in the art can readily appreciate that image recognition processing, including facial recognition processing, of an image, is well known in the art and can be done locally in a user device or in a remote server. As an example,

moodstocks.com provides an image recognition software platform for iPhones, Android and other devices so these devices may incorporate image recognition capabilities to recognize user input images. Also, Applied Recognition, a company that develops facial recognition technology, has launched FotoTiger for Facebook. It is a free Android app downloadable from Google Play Store that uses facial recognition to identify individuals on a photo, e.g., taken by phone camera. The app automates tagging of Facebook friends on the user's smart phone or tablet devices, as it indexes photos according to the recognized faces of people in them. Once tagged, people are automatically recognized and tagged in future photos. A user can share photos with any app or post directly to Facebook, Flickr or other social networks. Applied Recognition claims that the tagging and face recognition of the app are done on the phone locally with no server-side processing necessary. In addition, other image recognition apps for Apple or Android devices are also available for download such as, e.g., Camfind from Camfindapp.com, Face Recognition-FastAccess app from Sensible Vision, Inc., Face Recognition SDK Demo from Vinisoft, and etc. One skilled in the art may readily appreciate that one or more of these exemplary image recognition apps or other software may be used to provide the imagerecognition functionalities in accordance with the present principles.

At step 160, a determination is made to see if the informational element belongs to one of a user- selected category of informational elements based on the processed image information. At step 170, at least one supplemental informational element is provided in response to the processed image information if the processed image information belongs to the one of a user-selected category of informational elements. As described at step 120 of FIG. 1A, in one exemplary embodiment, the user may also select one or more of categories of supplemental informational elements to be provided. At step 180 of FIG. IB, in another exemplary embodiment, the device 260-1 may search one or more websites based on processed image information such as, e.g., the name of the person with the recognized face to derive the at least one

supplemental informational element. In one exemplary embodiment, the searching is done on one or more social networking websites such as Facebook, Twitter, Flickr, etc.

The present principles thus provide an easy, informative and efficient system and method for users to obtain and navigate information. For example, if a user takes a picture of the Eiffel Tower, a person can indicate in their user profile that they want any "tourist" category associated supplemental information such as travel related picture, address, historical information to be provided, instead of just a picture of the Eiffel Tower. In addition, EXIF (Exchange Image file Format) information and other types of metadata from photos or other input informational elements may assist with helping to narrow down specifics about things being recognized, as EXIF information can contain among other things, GPS information. As is well known in the art, EXIF is a standard that specifies the formats for images, sound, and ancillary tags used by digital cameras (including smartphones), scanners and other systems handling image and sound files recorded by digital cameras.

As noted before, the present principles may use a profile associated with a user to show the relevant categories of supplemental informational elements. In one exemplary embodiment, when a user takes a picture of a person, the person is recognized based on a facial recognition technique as described before. Based on the profile for a first user, for example, the address and GPS location corresponding the address of the recognized person may be provided as output supplemental

informational elements. In another embodiment, based on the profile of a second user, a social network service such as Facebook may be searched to see which people both the recognized person and the second user have in common. The biographical information of all the "shared" people is may then be provided as supplemental informational elements.

The disclosed principles may be used to produce an application or service where a user may collect all of her or his recognized things using photos, videos, and etc., where such items can be categorized into different lists which can be consulted later. For instance, a person may visit a city and take pictures of different people and items. The principles disclosed above can be used to create a summary of the trip listing people seen and items that were either visited as tourist locations and/or that can be purchased, by e.g., taking a picture of a coat/hat/car.

The foregoing has provided by way of exemplary embodiments and non- limiting examples a description of the method and systems contemplated by the inventor. It is clear that various modifications and adaptations may become apparent to those skilled in the art in view of the description. However, such various modifications and adaptations fall within the scope of the teachings of the various embodiments described above.

The embodiments described herein may be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method), the implementation of features discussed above may also be implemented in other forms (for example, an apparatus or program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The methods may be implemented in, for example, an apparatus such as, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, cell phones, portable/personal digital assistants ("PDAs"), and other devices that facilitate communication of information between end-users.

Reference to "one embodiment" or "an embodiment" or "an exemplary embodiment" or "one implementation" or "an implementation" of the present principles, as well as other variations thereof, mean that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment of the present principles. Thus, the appearances of the phrase "in one embodiment" or "an exemplary embodiment" or "in an embodiment" or "in one implementation" or "in an implementation", as well any other variations, appearing in various places throughout the specification are not necessarily all referring to the same embodiment.

Additionally, this application or its claims may refer to "determining" various pieces of information. Determining the information may include one or more of, for example, estimating the information, calculating the information, predicting the information, or retrieving the information from memory.

Further, this application or its claims may refer to "accessing" various pieces of information. Accessing the information may include one or more of, for example, receiving the information, retrieving the information (for example, from memory), storing the information, processing the information, transmitting the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information.

Additionally, this application or its claims may refer to "receiving" various pieces of information. Receiving is, as with "accessing", intended to be a broad term. Receiving the information may include one or more of, for example, accessing the information, or retrieving the information (for example, from memory). Further, "receiving" is typically involved, in one way or another, during operations such as, for example, storing the information, processing the information, transmitting the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information.

As will be evident to one of skill in the art, implementations may produce a variety of signals formatted to carry information that may be, for example, stored or transmitted. The information may include, for example, instructions for performing a method, or data produced by one of the described embodiments. For example, a signal may be formatted to carry the bitstream of a described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries may be, for example, analog or digital information. The signal may be transmitted over a variety of different wired and/or wireless links, as is known. The signal may be stored on a processor-readable medium.

While several embodiments have been described and illustrated herein, those of ordinary skill in the art will readily envision a variety of other means and/or structures for performing the functions and/or obtaining the results and/or one or more of the advantages described herein, and each of such variations and/or modifications is deemed to be within the scope of the present embodiments. More generally, those skilled in the art will readily appreciate that all parameters, dimensions, materials, and configurations described herein are meant to be exemplary and that the actual parameters, dimensions, materials, and/or configurations will depend upon the specific application or applications for which the teachings herein is/are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific embodiments described herein. It is, therefore, to be understood that the foregoing embodiments are presented by way of example only and that, within the scope of the appended claims and equivalents thereof, the embodiments disclosed may be practiced otherwise than as specifically described and claimed. The present embodiments are directed to each individual feature, system, article, material and/or method described herein. In addition, any combination of two or more such features, systems, articles, materials and/or methods, if such features, systems, articles, materials and/or methods are not mutually inconsistent, is included within the scope of the present embodiment.

Claims

1. A method comprising:

receiving (101) an informational element via a user input device (280); and supplementing (120), by a processor (265), said received informational element with at least one supplemental informational element selected from a plurality of supplemental informational elements, wherein said at least one supplemental informational element is selected from the plurality of supplemental informational elements in response to at least one of a first plurality of user-selected category identifiers capable of identifying a plurality of user-selected supplemental informational elements, and wherein a category of said received informational element matches one of a second plurality of user-selected category identifiers capable of identifying user- selected categories of informational elements.

2. The method of claim 1 wherein the first plurality of user-selected category identifiers and the second plurality of user-selected category identifiers identifying categories comprising: 1) type of people, 2) type of objects, 3) type of places, 4) type of media assets, 5) type of subjects, and 6) type of interests.

3. The method of claim 1 wherein the first plurality of user-selected category identifiers and the second plurality of user-selected category identifiers identifying categories comprising one or more of: 1) an image, 2) a text string, 3) an internet link, 4) video content, and 5) audio content.

4. The method of claim 2 wherein the first plurality of user-selected category identifiers and the second plurality of user-selected category identifiers identifying categories further comprising one or more of: 1) an image, 2) a text string, 3) an internet link, 4) video content, and 5) audio content.

5. The method of claim 1 wherein the first plurality of user- selected category identifies and the second plurality of user- selected identifiers are selected in a user profile of a user.

6. The method of claim 2 wherein the type of media assets comprises: 1) video content, 2) audio content, 3) movie, 4) TV show, 5) software, 6) game, 7) electronic book 8) electronic magazine, and 9) electronic article.

7. The method of claim 1 wherein the informational element is contained in one of: 1) an electronic text document, 2) a web page, 3) a HTML file, 4) an e-mail, 5) a PDF file, 6) a JavaScript file, and 7) a Microsoft Word document.

8. The method of claim 3 wherein the image further comprises metadata information, and the method further comprising steps of:

processing the metadata information; and

providing the at least one supplemental informational element based on the processed metadata information.

9. The method of claim 8 wherein the metadata information is EXIF information of the picture.

10. The method of claim 1 wherein the received informational element is an image, the method further comprising steps of:

processing (150) the image via image recognition to produce processed image information; and

providing (170) at least one supplemental informational element in response to the processed image information.

11. The method of claim 10 further comprising a step of searching (180) one or more websites based on the processed image information to derive the at least one supplemental informational element, wherein the one or more websites comprising one or more social networking websites.

12. A method comprising:

receiving (140) an informational element from a user input device (280); determining (145) if the received informational element is an image; if the received informational element is an image, then processing (150) the image via image recognition to provide processed image information;

determining (160) if the received informational element belongs to one of a user- selected category of informational elements based on the processed image information; and providing (170) at least one supplemental informational element in response to the recognized image if the processed image information belongs to the one of a user-selected category of informational elements.

13. The method of claim 12 wherein the at least one supplemental informational element is provided from one of a plurality of user- selected categories of supplemental informational elements.

14. The method of claim 13 further comprising a step of searching (180) one or more websites based on the processed image information to derive the at least one supplemental informational element.

15. The method of claim 14 wherein the one or more websites comprising one or more social networking websites.

16. An apparatus comprising:

a user input device (280) configured to receive an informational element; and a processor (265) configured to supplement said received informational element with at least one supplemental informational element selected from a plurality of supplemental informational elements, wherein said at least one supplemental informational element is selected from the plurality of supplemental informational elements in response to at least one of a first plurality of user-selected category identifiers capable of identifying a plurality of user-selected supplemental informational elements, and wherein a category of said received informational element matches one of a second plurality of user-selected category identifiers capable of identifying user- selected categories of informational elements.

17. The apparatus of claim 16 wherein the first plurality of user-selected category identifiers and the second plurality of user-selected category identifiers identifying categories comprising: 1) type of people, 2) type of objects, 3) type of places, 4) type of media assets, 5) type of subjects, and 6) type of interests.

18. The apparatus of claim 16 wherein the first plurality of user-selected category identifiers and the second plurality of user- selected category identifiers identifying categories comprising one or more of: 1) an image, 2) a text string, 3) an internet link, 4) video content, and 5) audio content.

19. The apparatus of claim 17 wherein the first plurality of user-selected category identifiers and the second plurality of user- selected category identifiers identifying categories further comprising one or more of: 1) an image, 2) a text string, 3) an internet link, 4) video content, and 5) audio content.

20. The apparatus of claim 16 wherein the first plurality of user-selected category identifies and the second plurality of user-selected identifiers are selected in a user profile of a user.

21. The apparatus of claim 17 wherein the type of media assets comprises: 1) video content, 2) audio content, 3) movie, 4) TV show, 5) software, 6) game, 7) electronic book 8) electronic magazine, and 9) electronic article.

22. The apparatus of claim 16 wherein the received informational element is contained in one of: 1) an electronic text document, 2) a web page, 3) a HTML file, 4) an e-mail, 5) a PDF file, 6) a JavaScript file, and 7) a Microsoft Word document.

23. The apparatus of claim 19 wherein the image further comprises metadata information, and the processor further processes the metadata information, and provides the at least one supplemental informational element based on the processed metadata information.

24. The apparatus of claim 23 wherein the metadata information is EXIF information of the picture.

25. The apparatus of claim 24 wherein the received informational element is an image, and the processor further processes the image via image recognition to produce processed image information; and the processor provides at least one supplemental informational element in response to the processed image information.

26. The apparatus of claim 25 wherein the processor further searches one or more websites based on the processed image information to derive the at least one supplemental informational element, wherein the one or more websites comprising one or more social networking websites.

27. An apparatus comprising:

a user input device (280) configured receiving an informational element from a user input device; and

a processor (265) configured to determine if the received informational element is an image; if the received informational element is an image, then the processor further processes the image via image recognition to provide processed image information; and the processor further provides at least one supplemental informational element in response to the recognized image if the processed image information belongs to the one of a user-selected category of informational elements.

28. The apparatus of claim 27 wherein the at least one supplemental informational element is provided from one of a plurality of user-selected categories of supplemental informational elements.

29. The apparatus of claim 28 wherein the processor further searches one or more websites based on the processed image information to derive the at least one supplemental informational element.

30. The apparatus of claim 29 wherein the one or more websites comprising one or more social networking websites.

31. A computer program product stored in non-transitory computer-readable storage media comprising computer-executable instructions for:

receiving (110) an informational element; and

supplementing (120) said received informational element with at least one supplemental informational element selected from a plurality of supplemental informational elements, wherein said at least one supplemental informational element is selected from the plurality of supplemental informational elements in response to at least one of a first plurality of user-selected category identifiers capable of identifying a plurality of user-selected supplemental informational elements, and wherein a category of said received informational element matches one of a second plurality of user-selected category identifiers capable identifying user-selected categories of informational elements.

32. computer program product stored in non-transitory computer-readable storage media comprising computer-executable instructions for:

receiving (140) an informational element from a user input device;

determining (145) if the received informational element is an image;

33. A user interface apparatus, comprising:

a display device; and a processor configured to receive at least one input informational element, wherein when a match is identified between said at least one input informational element and an identifier included in a user selected first category of a first plurality of categories, and, in addition, a user selected second category of a second plurality of categories includes at least one output supplemental informational element that is associated with said identifier, then said processor applies said at least output supplemental informational element to said display device to display said at least output supplemental informational element on said display device.