WO2007080570A1

WO2007080570A1 - Image insertion for text messaging

Info

Publication number: WO2007080570A1
Application number: PCT/IL2007/000005
Authority: WO
Inventors: Eyal Gura
Original assignee: Picscout (Israel) Ltd.
Priority date: 2006-01-08
Filing date: 2007-01-03
Publication date: 2007-07-19
Also published as: IL173011A; IL173011A0

Abstract

An apparatus for the enhancement of text messages passing over a network includes a text analyzer, a search unit and a media inserter. The text analyzer analyzes at least one of a text of the passing messages, and a context of the passing messages. The search unit searches for a media item in accordance with the analysis. The media inserter inserts media from the search into the analyzed messages, thereby enhancing the passing messages for sending on to the recipient.

Description

IMAGE INSERTION FOR TEXT MESSAGING

FIELD AND BACKGROUND OF THE INVENTION

The present invention relates to a method and apparatus for image insertion into text messaging, such as a cellular telephone SMS₅ and, more particularly, but not exclusively to such image insertion that relies upon an automatic analysis of the text.

Today cellular text messaging via the short messaging service SMS is popular, especially among the young. Most cellular phones additionally provide a multi-media messaging service, MMS, which allows photographs, video clips etc to be added to the text message. However MMS is rarely used since it is often felt to be awkward or because there just are not suitable media files on the telephone that the user would want to attach to his messages. Non-use of the MMS service translates into loss of revenue opportunities for the cellular network operators. ^" Text message users are well known for using emoticons, short combinations of punctuation marks that indicate emotions in a message. An example is the colon, dash, close parenthesis emoticon :-) indicating happiness. Users of equivalent messaging services over the Internet, IM or Internet messaging, are used to these symbols being automatically translated into corresponding images, thus "©". With SMS messaging over cellular phones, the ability to supply the corresponding images depends on the local telephone client and is awkward and sporadic.

An additional forum which can benefit from personalization of messages is blogging. Currently users wanting to add visual content to a blog must find and insert the content manually, which takes them outside the blog posting workflow. There is thus a widely recognized need for, and it would be highly advantageous to have, a text messaging service devoid of the above limitations. SUMMARY OF THE INVENTION

According to one aspect of the present invention there is provided apparatus for enhancement of text messages passing over a network, comprising: a text analyzer for analyzing at least one of text of said passing messages, and a context of said message, a search unit for searching for a media file in accordance with said analysis, and a media inserter for inserting media from said search into said analyzed messages, thereby to enhance said passing messages for sending on to said recipient.

According to a second aspect of the present invention there is provided a method for enhancement of text messages passing over a network, comprising: analyzing at least one of text of said passing messages, and a context of said message, searching a database of media files in accordance with a result of said analysis, and inserting media files retrieved by said search into said analyzed messages, thereby to enhance said passing messages for sending on to said recipient.

According to a third aspect of the present invention there is provided a method of enhancing text messages passing over a network comprising: analyzing respective text messages, using said analysis to identify a suitable media file from a database of media files, and inserting said media file into said text message to provide a media-enhanced text message for sending on to a recipient.

According to a fourth aspect of the present invention there is provided a user client for a message enhancement engine for enhancing text messages being sent over a network, the user client comprising: a text message interception unit for interception of text messages prior to sending; a model for indicating which messages are to be enhanced; and a message director, associated with said text message interception unit and said model for sending respective text messages to said message enhancement engine if they correspond to said model.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The materials, methods, and examples provided herein are illustrative only and not intended to be limiting.

Implementation of the method and system of the present invention involves performing or completing certain selected tasks or steps manually, automatically, or a combination thereof. Moreover, according to actual instrumentation and equipment of preferred embodiments of the method and system of the present invention, several selected steps could be implemented by hardware or by software on any operating system of any firmware or a combination thereof. For example, as hardware, selected steps of the invention could be implemented as a chip or a circuit. As software, selected steps of the invention could be implemented as a plurality of software instructions being executed by a computer using any suitable operating system. In any case, selected steps of the method and system of the invention could be described as being performed by a data processor, such as a computing platform for executing a plurality of instructions.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is herein described, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of the preferred embodiments of the present invention only, and are presented in order to provide what is believed to be the most useful and readily understood description of the principles and conceptual aspects of the invention. In this regard, no attempt is made to show structural details of the invention in more detail than is necessary for a fundamental understanding of the invention, the description taken with the drawings making apparent to those skilled in the art how the several forms of the invention may be embodied in practice.

In the drawings: Fig. Ia is a simplified diagram illustrating a text message being enhanced according to a first preferred embodiment of the present invention;

Fig. Ib is a simplified block diagram illustrating an exemplary system in which the text enhancement apparatus receives text message inputs from multiple input devices/applications and outputs an appropriately formatted enhanced message;

Fig. 2 is a simplified diagram illustrating an apparatus for enhancing of a text message according to a preferred embodiment of the present invention;

Fig. 3 is a flow chart showing the operation of a preferred embodiment of the present invention. Fig. 4 is another simplified diagram showing an application server according to a preferred embodiment of present invention.

Fig. 5 is a simplified diagram showing components of and around the application server of Fig. 4,

Fig. 6 is a diagram illustrating message flow in a successful message enhancement scenario according to a preferred embodiment of the present invention.

Fig. 7 illustrates message flow in a first unsuccessful message enhancement scenario according to a preferred embodiment of the present invention.

Fig. 8 illustrates message flow in a second unsuccessful message enhancement scenario according to a preferred embodiment of the present invention; Fig. 9 is a simplified block diagram of an apparatus for enhancement of text messages passing over a network, according to a preferred embodiment of the present invention.

Fig. 10 is a simplified flow chart of a method for enhancement of text messages passing over a network, according to a preferred embodiment of the present invention.

Fig. 11 is a simplified flow chart of a method for enhancing text messages passing over a network, according to a preferred embodiment of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present embodiments comprise an apparatus and a method for enhancement of a text message as it passes over a network, typically a cellular network. The message is enhanced by the addition of media files (also denoted media items) such as image, sound and video files. Enhancement is based on understanding the text or context of the originating message. In some embodiments the enhancement is first returned to the sender for approval. Other embodiments teach a user-configured model for defining which messages are enhanced and which not. The principles and operation of an apparatus and method according to the present invention may be better understood with reference to the drawings and accompanying description.

Before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of the components set forth in the following description or illustrated in the drawings. The invention is capable of other embodiments or of being practiced or carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting. Reference is now made to Fig. Ia, which shows a text message that is enhanced by the apparatus of a preferred embodiment of the present invention.

A text message 10 is sent by the user of a first cellular telephone 12. Over the network the message is enhanced by the addition of images 16 and 18 to form enhanced message 20. The message as sent is an SMS message. However the message as received is an MMS message, since it includes multimedia attachments. The enhancement is preferably carried out over the cellular network as will be explained below.

It should however be borne in mind that the same enhancement may be carried out using other networks, such as the Internet, and/or applications that support text messaging as well as messages containing visual or audio content. The simple text message is personalized by automatically including content reflecting the text meaning, based on an analysis of the message text, context, and user preferences. Examples of such text messages include:

1. Application-to-mobile telephone messages (i.e. from the operator website to the user mobile telephone)

2. Application-to-application messages (i.e. instant messaging) 3. Blog posts and Blog post titles (the image may be automatically selected to suit the content that is described in the blog or in its title).

4. Email messages

5. Web site posts

Examples of media files (items) which may be inserted into the message include image files, video files, and sound files. In one embodiment, a text message, is analyzed, and a related advertisement is inserted into the message. Advertisements may thus be automatically inserted into blogs or web sites. Fig. Ib illustrates an exemplary system in which the text enhancement apparatus 100 receives text message inputs from multiple input devices/applications, 110.1-110.5, and outputs the enhanced message formatted for the particular device/application on which the enhanced message will be displayed, such as cellular telephone 120.1 or IM application 120.5. Reference is now made to Fig. 2, which is a simplified diagram showing an apparatus for enhancement of text messages passing over a cellular network according to a first preferred embodiment of the present invention. The apparatus is preferably located in the cellular network and is associated with the servers that provide SMS and MMS features to the network, namely the Short Message Service Center (SMSC) 20 and the Multimedia Service Center (MMSC) 22. A message interpreter 24 receives the SMS message and analyzes it for suitable enhancements. The analysis may involve analyzing the text of the message being sent for keywords or key phrases, based on which images may be selected. Thus, keywords indicating sentiment may be used to select images illustrating the corresponding emotion. Alternatively the context of the message, say the identity of the recipient may be matched with pre-set lists of the sender. Thus certain images may be used for groups of users, say business contacts, and other images may be used for personal contacts. Yet again other images or sets of images may be used for designated close personal contacts. Included in analyzing the context may be the location of the sender or recipient. For example, the location may be used to include images that include greetings in the local language, or the location may be used to include characters of local appearance, including locally recognized costumes or local racial appearance. A particular preferred embodiment uses both keywords from text and the message context. The context indicates a set or collection of images to be used and the keywords point to individual images within the set or collection.

The message inserter thus includes both a text analyzer 26 which finds sentiments within the text of the message, and a context analyzer 28, which determines the context of the message.

The text analyzer is preferably able to analyze text, including abbreviations often used in text messaging, such as "gr8" for "great". Such text analyzers are known in the art. Principally the feature is intended to allow machine analysis of the text, and may include text recognition and natural language processing (NLP) algorithms as appropriate. The algorithms used are optimized to understand context from very short messages and are preferably self-learning.

If desired it is possible to provide a feature that enhances the text as sent by rewriting the abbreviations in full. A young user may wish to use such a feature when communicating with an older user not familiar with the abbreviations.

An image database 30 stores images for insertion into the message. The image database is preferably a large database in which the images are selected and categorized in ways that are appropriate for use with the message interpreter. That is to say images are chosen for their suitability with text messages, and are indexed for usability with the message interpreter.

The database is connected to an image inserter 32 which takes the images selected at the database and inserts them into the text messages. The result is a multimedia message which is an enhanced version of the original text message. The multimedia message is then sent on to the recipient. The database may include a set of stock images or videos, for example of the kind provided by Gettyimages.com and clipart.com, or stock animations such as provided by animationfactory.com.

According to the indexing used in the image database 30, the text of the original message is correlated with images in the image database. The context may be determined by comparing the recipient against user lists of his recipients. The user may designate particular recipients as business associates, others as customers, others as personal Mends, yet others as close personal friends etc. Alternatively the user may designate individual recipients as special.

A message grabber 34 may be supplied to identify and grab messages originating from subscribed senders only. Thus the service becomes an added-value service for which it is possible to charge. The message grabber may be located on the network or may be a client installed at the sender's telephone.

In one embodiment a preview functionality is provided. The preview functionality bounces the enhanced message back to the sender for approval prior to sending on to the recipient, allowing the user to approve or disapprove of the selected images. If the enhanced message is not approved, the user may select a "shuffle" function, and return the message for enhancement with different media files from those currently proposed. A learning feature can be added to allow the network to learn the preferences of the user. Such a learning feature may include a feedback mechanism that allows images to be ranked according to user approval. Higher ranked images are then more often inserted. The ranking may be based on individual users or an overall ranking may be made based on all users, or users having similar characteristics, say male users under the age of 25.

Alternatively or additionally, a user feedback mechanism may be provided for allowing a user to directly indicate preferred images for insertion. A user client at the sender may include the message grabber. The operational procedure with such a client is as illustrated in Fig. 3.

The message grabber grabs the SMS before leaving the handset and sends it to the server (step 1). The server analyzes the text or content or both of the message

(step 2) and uses the analysis result to get a suitable media file for inserting into the text (step 3). The enhancement is inserted (step 4) and is then sent as a preview MMS for approval by the sender (steps 5-6).

Upon approval the MMS is sent to the recipient (step 7). If not approved, the text is resubmitted for enhancement.

In an embodiment the approval stage may be left out and the message is sent straight to the recipient. In one preferred embodiment the sender has the option of selecting the preview stage by pressing a "compose" button before sending the SMS.

If the compose button is used then the text is transmitted to the server and the result is sent as a preview. Upon approval the sender presses "send", and the message is sent as MMS.

In another embodiment, upon pressing send the user is queried by the message grabber whether the message should be enhanced. The user then indicates whether the enhancement process should be performed or if the message should be sent to the recipient as an unenhanced text message.

The application is preferably activated automatically upon SMS submission.

The application may require an initial installation procedure to install and configure the user client. The installation procedure may ask the user for preferences of his future MMSs (i.e. his location, whether he prefers art images or 3d images or motion pictures or cartoons, whether different preferences should be used for different recipients etc.

In one embodiment, the images are provided in a format that makes it difficult to reuse them in other applications, say for copyright purposes.

In another embodiment the user client may work together with handset features such as the phone book, calendar and others.

The user can be given the option to set the application to be his default SMS sending system. In one preferred embodiment the user is provided with a personalized web site to manage his accounts, set up new preferences for new recipients etc.

The user may be set up to adjust his account both directly from his telephone or from the website as convenient.

The following features may be provided via the website/direct updating: • The user can set his preferences at any given time for images/video/music he likes to be added to his text.

• The user can tag files on the site and index them with his preferences of text and language.

• The user can rank files. The website is preferably connected to the server on which the service is provided and images may be ranked according to their actual usage as well. Thus if the user did not like the image and asked for the message to be recomposed, then the image will get a negative rank. The more often the image is approved of the higher the ranking.

A typical overall flow is as follows:

1. The user downloads the application, or obtains it using his handset from the carrier.

2. The application is configured.

3. The application is launched.

4. The user writes an SMS.

5. The user types "compose". 6. The application sends the SMS text to the server.

7. The server matches appropriate images or other media files to the text and sends them back to the user.

8. The application presents a preview of the enhanced message that is going to be sent. 9. The user may press "compose" again and ask for other images.

10. When the enhanced message is approved, the user hits "send" and sends the MMS.

11. The MMS is displayed to the receiver, preferably with a link to download the application for his handset The present embodiments provide an application whose purpose is to send a text message enhanced with MMS media, typically but not limited to pictures which are sent to friends.

Referring now to Fig. 4, the sender produces regular SMS text for sending.

The message is sent in the normal way to the SMSC 20 but is then diverted to an application server 40, which searches for pictures based on words in the message text.

The application server 40 enhances the original text messages with the selected pictures and sends this upgraded MMS message to the destination on behalf of the sender.

The application server communicates with the SMSC 20 and the MMSC 22 of the network operator. In addition, as explained the application server is connected to a large image database 30 (also denoted a content "bank") of pictures or other media files. The application server can be integrated into networks based on Global System for Mobile Communications (GSM), Code-Division Multiple Access (CDMA), Time Division Multiple Access (TDMA) or Integrated Digital Enhanced Network (iDEN) networks since it is signaling independent, and in fact can be work on Voice over IP (VoIP) and any other networks that can send text messages with media files.

Reference is now made to Fig. 5 which illustrates the high level architecture of an exemplary system. The application server comprises: 1. Image inserter Engine 50 2. SMSC Connector 52

3. MMSC Connector 54

4. On-Line-Billing (Pre-Paid) Connector 56

5. Topology and Monitoring agent 58

6. Call Detail Record (CDR) Collection unit 7. CDR Statistics and Reporting unit 62

In addition, the following WEB tools are preferably provided:

1. Pictures Administration & Provisioning Interface

2. Users Administration & Provisioning Interface.

Database 30 preferably stores: • MMS Pictures

• Row CDRs in files

• Statistics & Reporting results

The following information is stored in the database:

• Users Table and their Sending Model • Key words and references to the pictures storage

• Temporary Text message to appear on a user screen while the MMS is in the course of being downloaded.

Preferably, information such as the temporary text messages are maintained in the application cache. The database schema is itself used as backup. Such an architecture improves availability, in other words reduces the database access delay during run-time. The image inserter engine 50 (also denoted the "enhancement engine", the "application engine" or the "engine") is the main business logic of the application. It is responsible for some or all of the following tasks:

• Accepting incoming SMS messages from originators via the SMSC component.

• Deciding if this message should be converted to MMS.

• Decoding the Text Messages,

• Finding the appropriate key words,

• Extracting the Picture(s) from the database, • Composing the outgoing MMS message from the Original Text and the enhancement picture(s)

• Initiating the MMS Connector to send the MMS message to the recipient,

• Keeping the original SMS message in the database for backup until MMS Client Delivery approval is received, • Maintaining the MMS Delivery Report from the MMSC through the MMSC Connector,

• Converting the MMSC Delivery Error to an appropriate SMS response, if required,

• Cleaning up of messages in the storage, • Generating of CDRs for Billing & Statistics

• Handling of On-Line-Billing (Pre-Paid) queries to the On-Line-Billing Connector

The SMSC connector 52 is responsible for interfacing with the SMSC to carry out the following tasks: 1. Connecting on initialization and continuously maintaining connections to the SMSC in order to receive Incoming Packets and send outgoing messages,

2. Accepting incoming messages from originators, and

3. Sending outgoing messages to recipients in case of MMSC Failure or other application decisions. The MMSC connector 54 is responsible for interfacing with the MMSC to carry out the following tasks: 1. Sending outgoing MMS messages to recipient, and 2. Requesting a delivery report from the MMSC and handling the response, timeout and failures. A Picture Administration & Provisioning Interface is preferably provided for

• Adding and deleting pictures, and • Adding Key words to the files.

Picture Administration & Provisioning is done through a web client who prepares the data for quick searching by the application.

The Users Administration & Provisioning Interface is preferably provided to add or delete a new user, and managing the sender model, to be described hereinbelow.

The application program interface (API) is exposed as necessary through the following:

• Exposing of the Extensible Markup Language (XML) interface for the operator to manage customer relationship management (CRM) commands. • Exposing the web client for the end-user.

CDR Collector 62 preferably collects transaction records written by the server application component, and inserts them for concentrated storage for future data warehousing and for transfer it to Off-Line billing operator mediation.

The On-Line Billing Connector (Pre-Paid) 56 provides access to the operators online billing system. The connector is responsible for generating charging requests to the online billing system for authorized actions.

The CDR Statistics & Reporting unit 62 operates a scheduled process which summarizes the collected CDRs to statistics on daily and monthly basis, and sends the report to the appropriate customers. The billing system allows for the system to be used with digital asset management to ensure that sources of the media material are rewarded for their work.

Artists and managers of stock photograph libraries and the like gain a new source of income. Likewise MMS, currently an underutilized resource, becomes of more interest and a greater source of revenue to the providers. The Topology & Monitoring manager 58 preferably manages the application initiation process by proving processes with information on other existing processes and notifying in case of process failure and restart. The manager 58 sends Simple Network Management Protocol (SNMP) traps in case of a failure in interconnect internal components and about failure in connection to other home domains.

The manager exposes the SNMP interface for getting dynamic data onto the application servers.

A logging unit is a server-side facility that provides logging capabilities to server components.

Server components may launch operations and receive events. It is noted that the ability to monitor these tasks is contributive to quality management of the entire platform.

System Architecture

The architecture of the system of Fig. 5 is now considered in greater detail. The following components are stateless and can be scalable without take down of the system:

1. SMSC Connector 52

2. MMSC Connector 54

3. On-Line-Billing (Pre-Paid) Connector 56.

Each one of these components can run multiple instances and load-sharing and can run on multiple severs. This approach supports redundancy and scalability since failure of one server does not interfere with the performance of other transactions.

The application engine preferably works using an N+l model, meaning it is divided into N+l instances. All the registered users are divided between the N instances of each engine, so that each engine takes one N-th part of the users. The consequence is that inter-routing to the engine depends on the Sender.

The topology unit 58 ensures that there is always a standby instance ready as backup in case one of the active message enhancement Engine instances fails.

Each Server runs the following processes: 1. Topology and Monitoring agent 2. CDR Collection.

In addition a CDR Statistic and reporting process runs. The process is run in Active-standby mode for redundancy. In addition, the WEB tools referred to above are supported, namely:

1. Pictures Administration & Provisioning Interface

2. Users Administration & Provisioning Interface

The components communicate among themselves using internal transactions. Most of the Transactions are non-synchronic to allow high traffic.

The system includes a database (DB) and storage. The following information is stored in the DB:

1. Users Table and their Sending Model

2. Key words and references to the pictures storage 3. Temporary Text message until MMS Delivery

The following items are stored in the database:

1. MMS Pictures or other media items

2. CDRs

The following components use the database: 1. Engine 50

2. MMSC Connector 54

3. Pictures Administration & Provisioning Interface

4. Users Administration & Provisioning Interface

The database information is for backup only. During real-time operation the components update the DB but read the content from their cache.

Preferably, the Users Table and the User Sending Models, together with the keywords and references to the pictures storage, are distributed between the Message Enhancement Engine instances to reduce DB access.

The temporary text message sits both in the DB and in the Message Enhancement Engine until MMS delivery to reduce DB access.

Application Rules & Flow I) Rule Engine

The following rules determine whether a given SMS message should be converted to an MMS message:

1) The Sender is registered to the Application 2) The destination fits the Sender Model The following are examples of sender models:

• All SMS Messages should be converted to MMS Messages.

• Only SMS Messages within the same carrier should be converted to MMS Messages

• Only SMS Message for a recipient who belongs to a list of destinations is converted to MMS Messages. The sender predefines his own list of destination.

3) There is at least one picture associated with the text Errors can occur after the message was sent to the destination as MMS Client, for example the end-user destination may not retrieve the message within a configurable time period.

II) Application Flow Reference is now made to Figure 6, which is a flow chart of a positive scenario, in which an incoming SMS is received, the application converts the SMS to an MMS message, and the MMSC succeeds in delivering the MMS to the recipient. Reference is also made to Figs. 7 and 8, which illustrate failures in the flow procedure. Fig 7 shows a procedure in which no conversion is carried out of the incoming SMS into an MMS. The result is that the outgoing message to the recipient is the original incoming SMS Message. Fig. 8 shows a procedure in which the message is successfully generated but is then rejected by the recipient so the original SMS is sent instead.

Refering again to Fig. 6, the SMSC Connector receives the incoming SMS message, acknowledges the SMSC for positive delivery and sends the incoming text message to one of the application Engines. Choosing the appropriate Engine instance is based on the MSISDN of the Sender.

The message always contains data about the following:

• The Sender • The destination

• The text body

The application engine verifies that: • The Sender is a subscribed user

• The destination is in accordance with the Sender Model. Otherwise the message is sent back to the SMSC connector.

The application engine decodes the text body and searches for key words. The key words are organized in a hash table for quick search, with references to the pictures. In case no picture is found, the message is sent back to the SMSC connector

52. The number of pictures to associate with the message is taken from the Sender

Model.

The text message is stored in the database for backup. If Pre-Paid users are supported, the application Engine queries the Pre-Paid

Operator for authorization.

The application composes the MMS Messages from the Text and the associated pictures. Then the message is sent to the MMSC connector.

The message insertion engine chooses one of the MMSC connectors, for example using Round Robin type selection, and sends the chosen connector the MMS Message body, Original Sender, destination and DB Message Index.

In the successful enhanced message delivery scenario of Fig. 6, the flow is as follows:

The MMSC Connector and sends the MMS message to the chosen MMSC that it has configured. The MMS messaging is either by MM7 Submission or the MMS Vendor API. Each MMSC Connector can send to all of the operators' MMSCs.

The Original Sender is inserted as the Sender and as the user who is charged for the service. Now, the Destination can be a non-MMS Client, say a user with an older mobile telephone. The server therefore requests an acknowledgement if the message arrives within a certain period. In order to provide such acknowledgement the MMSC Connector requests a Delivery Report as part of the Submission.

In the event of a failure in the submission, the MMSC Connector either retransmits the MMS Message to another MMSC or sends a failure indication to the Insertion Engine, as discussed in greater detail below. In the event of successful submission, the MMSC Connector updates the database with the Message ID returned from the MMSC. The connector would also take responsibility for timing matters associated with the delivery report in the event that the MMSC does not support such a timer.

From MMSC to MMSC connector The MMSC itself preferably verifies the submission request. There may be several causes of delivery failure, for example: the destination cannot be resolved, the application is not allowed to send, the message overrides the allowed size, and other reasons. In these cases the MMSC responds with an indication of submission response failure. The MMSC tries to deliver the message. It sends the end-user an

MMl_Noification Request. In any case it sends an MM7 Delivery Report request to the application with the following exemplary return codes in case of failure: End-User rejected the request, Time-Out, etc., or reports successful message delivery. The Message ID indicates the message. Since there is a Load Balancer in front of the image inserter Engine, each MMSC Connector can receive the MM7 Delivery Report Request. The MMSC Connector acknowledges its response to the MMSC using the MM7 Delivery Report.

From MMSC connector to Image Inserter Engine The MMSC connector sends the Delivery Report Response to the appropriate image inserter Engine or instance thereof. Choosing the Engine comprises finding the instance that is responsible for the given sender.

In the event that the MMSC operates the Expiration Delivery Timer and the timer indeed expires prior to an acknowledgement being received, the MMSC Connector sends a Cancel request to the MMSC and sends a Failed Delivery Report to the image inserter engine with the appropriate return code.

Image Inserter Engine - Delivery Report

The Engine itself verifies the return code of the Delivery Report. In the event of an error, the engine fetches the originating message from its cache memory, and sends it to the destination as an unenhanced SMS message, as discussed below. The image inserter Engine generates a computer data record CDR with the information of the Sender, Destination, number of pictures and size, and the return code. The CDR may subsequently be used to determine billing.

In any case — failure or success — once delivery of either the enhanced or unenhanced message is made, the original message is cleaned from the cache and from the database.

From Image Insertion Engine to SMSC Connector

The text message may in certain cases be returned to the SMSC connector for sending on to the recipient as an unenhanced message. Such cases include the following:

• The Sender is not subscribed to the message enhancement service

• The destination does not conform to the Sender Model

• There are no pictures that the analyzer has associated with the text • The Pre-Paid Server has rejected the request, typically due to insufficient balance

• The message could not be delivered to the destination as an MMS message. Typically because the recipient is not sufficiently powerful to receive MMS. In any of the above cases the Image Inserter Engine sends the text message back to the SMSC connector to be sent on to the recipient as an unenhanced message. The Image Inserter Engine typically chooses between the available SMSC connectors using the Round Robin protocol.

From SMSC Connector to SMSC

The SMSC Connector uses the round robin protocol to choose an available connection to send the SMS message. Preferably, each SMSC connector is connected to all the SMSCs of the given Operators so that the traffic is distributed equally.

The Sender of the message is retained in the send field as the Original Sender, thus the message is delivered and received as if the sender has sent a simple text message.

In the following the path taken by the outgoing SMS message is considered. SMSC connector General

The SMSC connector interfaces between the SMSC and the image insertion engine. The connector is initialized and then connects continuously to the SMSC in order to receive incoming packets and send outgoing messages.

The connector accepts incoming SMS messages from originators so that they can be passed to the engine for enhancement. The connector sends outgoing Text messages from the engine to the recipient in case no enhanced MMS message was produced, as discussed above.

Communication between SMSC connector and Message Enhancement Engine

Two internal messages are supported as follows: • The SMSC connector sends the incoming text messages as received

• The engine sends outgoing messages to the recipient in case of MMSC failure or for any other reason that the application is not available.

Multiple SMSC Connector Instances

There may be several instances of the SMSC Connector in one message enhancement system, either for redundancy, load sharing, or both.

Different SMSC Connectors may run on different machines or on the same machine.

Connections to SMSC

The SMSC connector may connect to one SMSC or to different SMSCs, depending on whether it is provided for redundancy or for load sharing (on the SMSC resource). Different instances of the SMSC Connector may connect to different SMSCs or to the same SMSCs, or a combination of both. In the event of an outgoing SMS message, the SMSC chooses one of the connections using the Round Robin protocol as explained above, and sends the message over the selected connection.

Protocols Supported

The following protocol implementations are preferably supported (in descending order):

• SMPP v3.3, v3.4

• UCP v3.1 -v4.0 • CIMD2 - Computer Interface to Message Distribution

Supported Character Sets

One configurable character set is supported per installation. The default character set is GSM-7. Within the system, messages are processed in UTF-8 character sets. Whenever a message is sent to or received from an SMSC, it is transcoded from/to UTF-8 to/from the local character set.

Keep- Alive Mechanisms The SMSC Connector makes use of the keep-alive mechanisms provided within the SMSC communication protocols, polling the SMSC periodically in order to verify it is still connected and responding. The keep-alive mechanism is configured separately for each SMSC. If the keep-alive mechanism fails, then the SMSC Connector resets the connection to that SMSC. The keep-alive mechanism is also used to stop firewalls from disconnecting the connection when idle.

• In CIMD-2, the built-in keep-alive function is used

• In SMPP, the Enquire Link function is used

• In UCP, the Enquire Message Operation (UCP55) function is used

Packet Retransmissions The SMSC Connector has a mechanism intended to recover from errors while sending messages to the SMSC. Three types of "sending errors" are recognized:

• No acknowledgement received from the SMSC

• TCP socket error on connection • Reject received from the SMSC

The retransmission is carried out on another connection selected via Round Robin.

MMSC connector General

The MMSC connector component is now considered in greater detail. The MMSC connector preferably interfaces with the MMSC in order to send outgoing MMS messages, that is the enhanced SMS messages, to the recipient.

The MMSC connector, as discussed above, requests a Delivery Report in every MM7 Submit message and takes care that the delivery report expires within a predetermined period. The main reason for the expiry is for Non-MMS supporting

Clients at the recipient, and as discussed, the main fallback comprises sending the original SMS message.

Communication between MMSC connector and Image Insertion Engine

Again, two internal messages are supported:

• The message enhancement Engine sends MMS outgoing messages to the recipient

• The MMSC connector sends Delivery Report responses (Success or Failure) to the message enhancement Engine

Multiple MMSC Connector Instances

As with the SMSC connector, there may be several instances of the MMSC Connector in one message enhancement system, either for redundancy, load sharing, or both.

Different MMSC Connectors may run on different machines or on the same machine, again as with the SMSC connectors. Connections to MMSC

The MMSC connector may connect to one MMSC or to different MMSCs, either for redundancy or for load sharing over the MMSC resource. Different instances of the MMSC Connector may connect to different MMSCs or to the same

MMSCs, or a combination of both, depending on whether the purpose is for redundancy or for load sharing.

Protocol Supported Two approaches can be taken:

• Support MM7 (SOAP/XML)

• Support the appropriate SDK API of the MMSC Vendor. Current MMSC vendors include Nokia, OpenWave, LogicaCMG, and Comverse. Two MM7 messages are preferably supported. The first MM7 message is an

MM7_Submit Request and Response, which comprises a request from the MMSC Connector to the MMSC, and a response from the MMSC to the MMSC Connector. The second MM7 message is an MM7_Delivery_Report Request and Response, which comprises a request from the MMSC to the MMSC Connector, and a response from the MMSC Connector to the MMSC.

Return Code, Time Out & Errors

Several return codes & errors are catered for. In most of them, the message enhancement system sends the original SMS text message to the destination. A failed MM7_Submit can be signaled by a response with the appropriate

Return Code, called Status code in the Protocol. As the MMSC cannot authorize the Value-added Service Provider (VASP), the MMSC cannot resolve any of the recipient addresses, thus the message exceeds the maximum allowed message size etc.

If the MMSC is working in a Pre-Provisioning configuration or is using a device detection mechanism to observe whether the end-user is an MMSC Client or not, the MM7_Submit Response from the MMSC may return with an error indicating that the end-user is not an MMSC Client. In the event that the MMSC is not aware of the end-user device capabilities and there is no other failure, the MMSC preferably returns an OK in the MM7_Submit Response. In the response the Message_ID is returned, so as to associate it later with the MMl_DeliveryReport Request. A requirement is set to ensure delivery of the message within a certain period.

In some MMSCs there is support for an ExpiryDate. A string of XML using the dateTime or duration datatypes may specify the point in time after which MMSC does not attempt to delivers the message. In this case the MMSC is responsible for timekeeping and there is no need for the MMSC Connector to take this time. In the event that the MMSC does not support the ExpiryDate feature, the

MMSC Connector takes responsibility for counting from when a positive MM7_Submit Response is received from the MMSC.

In the event that the message cannot be delivered:

• If the timer is taken from the MMSC, the MMSC preferably returns a negative MM7_Deliver Request to the MMSC Connector with the appropriate Status Code and with the appropriate Message lD.

• If the timer is taken from the MMSC Connector, the MMSC preferably sends a Cancel Request to the MMSC with the appropriate Message_ID. The MM7 now tries to deliver the message to the end-user. The destination may reject the message. That is, the user receives an Indication that an MMS message is waiting for him and he himself chooses to reject and not to retrieve it, an option with MMS. In such a case the MMSC sends a negative MM7_Delver Request to the MMSC Connector with the appropriate Status Code and with the appropriate MessageJOD.

In the event that the end-user retrieves the message and sends MMl_Acknowledgement to the MMSC then the MMSC sends a positive MM7_Deliver Request to the MMSC Connector with the appropriate Message_ID.

Packet Retransmissions

The MMSC connector has a mechanism intended to recover from errors while sending messages to the MMSC. Two types of "sending errors" are recognized: • No acknowledgement received from the MMSC

• Reject received from the MMSC

Provisioning in the message enhancement server Sender Model

The user preferably subscribes to the application and decides on a message enhancement model which may be one of the following alternatives:

• AU SMS Messages are converted to MMS Messages.

• Only SMS Messages to users of the same carrier are converted to MMS Messages

• Only SMS Messages to a preset list of destinations are converted to MMS Messages. The user is able to update the list.

The user should decide on the size of the MMS message or on a preferred number of pictures to associate with the MMS message. There are two preferred ways to configure the user for the system. The XML interface may be exposed to allow the operator to manage CRM commands. This is suitable for cases where the operator builds the user interface or the user requests the operator by phone to join the application. Alternately, the web client may be exposed for the end-user. The client is re-directed from the Portal Operator. A De-Registration command from the application is also supported.

Pictures Maintenance

The message enhancement server and product preferably provides a tool to work with the database. The tool allows picture banks or other media sources to insert a new picture into the database or to add key words for searching for the pictures. The tool also allows for maintenance of the current pictures.

Billing

The message enhancement engine preferably supports two types of Billing: 1) Off-line Billing - Two CDRs are generated for billing purposes. The first is when an outgoing MMS message is submitted, and the message enhancement Engine tries to send the MMS message. The second is when the outgoing MMS message is delivered, and the destination receives the MMS message.

2) On-Line Billing - For Pre-Paid users, the message enhancement engine queries the Operator Pre-Paid server about whether the sender who is the charging user has permission or the necessary balance to send an MMS message. In case no account is allowed, the service is not allowed to the sender and the message is sent back to the SMSC for delivery as an unenhanced text message.

In the event that the Message is not delivered, a Refund request is sent to the Operator Pre-Paid server.

Statistic & Reporting

The message enhancement system preferably collects the CDRs and maintains system and usage statistics.

The statistics are provided for different audiences, such as marketing managers, operator system engineer, and picture or media bank engineers.

A summary of the computer data records, CDRs, that may be generated by the system is as follows:

• Incoming SMS Text is Delivered - The SMSC routes Incoming SMS message to the message enhancement Application • Outgoing MMS message is submitted - The message enhancement

Engine is trying to send an MMS message

• The outgoing MMS message is delivered - The destination has received the MMS message

• The outgoing SMS message is submitted - The message enhancement Application sends the SMS message to the destination with the appropriate reason why the message could not be delivered as an MMS message.

• User Provisioning: Tracking of active users (performed through the Users Administration & Provisioning Interface) - New registration, changing the Sender model etc. • Pictures maintenance - Tracking of action at the image database through the Pictures Administration & Provisioning Interface, including setting a new picture, changing the key words, etc.

Network Monitoring

The following items are monitored through the message enhancement Product by the Topology & Monitoring manager. An SNMP trap is sent to the Operator operations and maintenance (O&M). Failure or initialization of a message enhancement component - for every process that starts running or fails the Operator O&M is informed.

Failure and initialization of the SMSC connection is likewise monitored.

Database

As discussed at length hereinabove, a database and storage are provided within the message enhancement server system. The following information is stored in the DB:

• Users Table including each user's Sending Model — this information is kept also in each message enhancement Engine cache or the cache associated with each instance of the engine. • Key words and references to the pictures storage — this information is kept also in each message enhancement Engine cache or the cache associated with each instance of the engine.

• Temporary Text message until MMS Delivery - this information is kept also in the cache of the engine of the current user. The following items are stored in the database storage:

• The MMS Pictures or other media items

• Row CDRs in files

• The Statistic & Reporting results

• The following components use the database: • The message enhancement engine

• The Pictures Administration & Provisioning Interface

• The Users Administration & Provisioning Interface The Database tables are supported only for backup - In real time the message enhancement Engine reads from its cache.

The following transactions are supported:

• Update Users data - The Users Administration & Provisioning Interface updates the DB Users table. The DB Process updates the message enhancement Engine cache with the updated Users table.

• Update Pictures data - The Pictures Administration & Provisioning Interface updates the DB Pictures and key words table. The DB Process updates the message enhancement engine cache with the updated pictures table.

• Storage of the SMS message - The Prime message enhancement Engine, meaning the engine or the instance serving the present user, stores the SMS message for backup.

• Update SMS message - The Prime message enhancement Engine updates the SMS message for backup. An example of an update is providing the MMS Message ID.

• Clean SMS message — The Prime message enhancement Engine cleans the SMS message from the DB.

Architecture for System Reliability

The cellular environment is one of those environments where high reliability is mandated. The following discusses issues of the architecture that provide for high reliability.

Multi Instance

The Message Enhancement system can run more than one instance or server process, of most system components. These instances have already been referred to above. A multi-instance configuration increases the level of system survivability and the service level. For some components, multiple instances significantly improve system performance.

Multi Sever Message enhancement components are able to run on more than one server. Such a feature is crucial for high availability of the system configuration, and prevents a physical server from becoming a single point-of-failure.

Load Sharing

Load sharing is a very basic mechanism of load distribution between system resources, utilized internally by the message enhancement system, where the system uses a sharing algorithm, usually the round robin algorithm, to allocate tasks to various components and servers.

Load Balancing

Load balancing is a more comprehensive mechanism of system load distribution. This is achieved by a physical Load Balancer.

Redundancy

A robust hardware system design should support hardware hot-swapping or alternative path for all hardware and software components. The redundancy principle means that each system element (hardware or software) is duplicated.

Full Redundancy Implications

To meet high availability requirements, all system elements are preferably redundant, with no single points-of-failure in the system. Such a principle is reflected in local and shared storage, all server hardware elements (i.e. CPU, Power Supply, Fans etc.), local network devices (which can be implemented by multi-pathing with a standby Ethernet card) or trunk (dual Ethernet card sharing) solutions. A full high availability solution also requires redundant switches, routers, and firewalls.

Scalability

The message enhancement system offers gradual scalability, allowing operators to start with a relatively small system, and scale upwards to support millions of users, as all components and processes can run multiple instances. Resulting from built-in load balancing between components, scalability is near linear. When traffic grows, scaling the system upwards means adding servers and configuring the system to utilize them. There is no downtime involved in such upgrades, as the same mechanism that manages faulty components and ensures high availability is also responsible for the addition of new components into a live system.

Alternative Solutions for Message Routing

There are several ways in which the SMSC may route the Message to the message enhancement application. These include: Specific Short Code, SMSC with enhanced routing capabilities, and Concentrator / Aggregator.

Specific Short Code

Specific short code is a simple solution - The sender, or perhaps his user client, adds the Short Code Prefix before the MSISDN of the destination. The Short Code is preferably provided for the service by the Operator. Although the specific short code makes the routing very simple it is generally regarded as an unfriendly solution.

SMSC with enhanced routing capabilities

An alternative solution is to require the SMSC to route the SMS Message to the message enhancement server whenevever an SMS message is submitted from the list of subscribed Users. This Users List preferably resides in the SMSC or lightweight directory access protocol (LDAP). If the Sender exists in this List, the SMSC preferably delivers the message to the application with all the information - Sender, Destination and message Body.

Concentrator / Aggregator

In many systems, mainly in the USA network, there are separate MO (Mobile Originated) SMSCs and MT (Mobile Terminated) SMSCs. The MO SMSC sends its incoming traffic to an Aggregator which carries out the routing. The message enhancement application is preferably connected to the Aggregator using Short Message Peer to Peer (SMPP) or even HTTP access. Instead of connecting to many SMSCs the application is connected to a small number of Aggregators. The requirement is that the Aggregator routes the SMS Message to the message enhancement server every time an SMS message is submitted from anyone belonging to the Users list. This Users List preferably resides in the Aggregator or LDAP. If the Sender exists in the list, the Aggregator should deliver the message to the application with all the information - Sender, Destination and message Body.

Reference is now made to Fig. 9, which is a simplified block diagram of an apparatus for enhancement of text messages passing over a network, according to a preferred embodiment of the present invention. Apparatus 900 includes a text analyzer 910, a search unit 920, and a media inserter 930. The text analyzer 910 analyzes at least one of a text of the passing messages, and a context of the passing messages. The search unit 920 searches for a media item in accordance with the analysis. Search unit 920 preferably searches for the media item(s) in database 925, which contains media items for insertion, such as images, video clips, music clips, etc... The media inserter 930 inserts media from the search into the analyzed messages, thereby enhancing the passing messages for sending on to the recipient.

Reference is now made to Fig. 10, which is a simplified flow chart of a method for enhancement of text messages passing over a network, according to a preferred embodiment of the present invention. In step 1010, the text and/or context of the passing messages is analyzed. In step 1020, a database of media files is searched in accordance with a result of the analysis. In step 1030, media files retrieved by the search are inserted into the analyzed messages, thereby to enhance the passing messages for sending on to the recipient.

Reference is now made to Fig. 11, which is a simplified flow chart of a method for enhancing text messages passing over a network, according to a preferred embodiment of the present invention. In step 1110 respective text messages are analyzed. In step 1120, the analysis is used to identify a suitable media file from a database of media files. In step 1130, the media file is inserted into the text message to provide a media-enhanced text message for sending on to a recipient.

In a further preferred embodiment, a user client for a message enhancement engine for enhancing text messages being sent over a network includes: a text message interception unit for interception of text messages prior to sending, a model for indicating which messages are to be enhanced, and a message director, associated with the text message interception unit and the model for sending respective text messages to the message enhancement engine if they correspond to the model. The user client may be installed in a cellular telephone device.

It is expected that during the life of this patent many relevant devices and systems will be developed and the scope of the terms herein, is intended to include all such new technologies a priori.

It is appreciated that certain features of the invention, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the invention, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable subcombination.

Although the invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims. All publications, patents, and patent applications mentioned in this specification are herein incorporated in their entirety by reference into the specification, to the same extent as if each individual publication, patent or patent application was specifically and individually indicated to be incorporated herein by reference. In addition, citation or identification of any reference in this application shall not be construed as an admission that such reference is available as prior art to the present invention.

Claims

WHAT IS CLAIMED IS:

1. Apparatus for enhancement of text messages passing over a network, comprising: a text analyzer for analyzing at least one of a text of said passing messages, and a context of said passing messages, a search unit for searching for a media item in accordance with said analysis, and a media inserter for inserting media from said search into said analyzed messages, thereby to enhance said passing messages for sending on to said recipient.

2. Apparatus according to claim 1, wherein said network comprises a cellular network.

3. Apparatus according to claim 1, wherein said network comprises the Internet.

4. Apparatus according to claim 1, wherein said message comprises an

SMS.

5. Apparatus according to claim 1, wherein said message comprises one of a group consisting of: a blog post, an instant messaging text message, an email, and a web-site post.

6. Apparatus according to claim 1, wherein said analyzing said text comprises identifying key words or phrases in said text, said key words or phrases being correlated with images in an image database, thereby to allow insertion of said correlated images.

7. Apparatus according to claim 1, wherein said analyzing said context comprises identification of said recipient against a predetermined list.

8. Apparatus according to claim 1, further comprising using a message grabber configured to identify and grab messages originating from subscribed senders only, for carrying out said enhancing.

9. Apparatus according to claim 1, further comprising a preview functionality for bouncing the enhanced message back to the respective sender for approval prior to sending on to said recipient.

10. Apparatus according to claim 7, wherein said analyzing said text comprises identifying key words or phrases in said text, said key words or phrases being correlated with images in an image database in accordance with said context, thereby to allow insertion of said correlated images in accordance with said context.

11. Apparatus according to claim 10, wherein said image database comprises images grouped into categories, and wherein said context determines a category in which to search and said key words or phrases identify which images within said determined category.

12. Apparatus according to claim 1, further comprising a user feedback mechanism for allowing a user to indicate preferred images for insertion.

13. Apparatus according to claim 1 further comprising a feedback mechanism for allowing images to be ranked according to user approval, such that higher ranked images are more often inserted.

14. Apparatus according to claim 8, wherein said message grabber is located at an SMSC to grab any text messages originating with a user subscribed with said apparatus.

15. Apparatus according to claim 8, wherein said message grabber is located at an aggregator connected downstream of a plurality of SMSCs, to grab any text messages routed via said plurality of SMSCs originating with a user subscribed with said apparatus.

16. Apparatus according to claim 8, wherein said message grabber is located at a sender-end mobile telephony device, and comprises a user client configured to add a short code to direct respective text messages to said apparatus.

17. Apparatus according to claim 8, wherein said message grabber is further operable to query a user whether a message should be enhanced.

18. Apparatus according to claim 1, wherein said media file comprises any one of the group consisting of an image file, a video file, a sound file, and an advertisement.

19. Method for enhancement of text messages passing over a network, comprising: analyzing at least one of text of said passing messages, and a context of said messages, searching a database of media files in accordance with a result of said analysis, and inserting media files retrieved by said search into respective analyzed messages, thereby to enhance said passing messages for sending on to said recipient.

20. A method of enhancing text messages passing over a network comprising: analyzing respective text messages, using said analysis to identify a suitable media file from a database of media files, and inserting said media file into said text message to provide a media-enhanced text message for sending on to a recipient.

21. A user client for a message enhancement engine for enhancing text messages being sent over a network, the user client comprising: a text message interception unit for interception of text messages prior to sending; a model for indicating which messages are to be enhanced; and a message director, associated with said text message interception unit and said model, for sending respective text messages to said message enhancement engine if they correspond to said model.