WO2000067460A1

WO2000067460A1 - Method and system for fraud detection in telecommunications

Info

Publication number: WO2000067460A1
Application number: PCT/GB2000/001676
Authority: WO
Inventors: Katherine Butchart; Derek Dempsey
Original assignee: Nortel Networks Limited
Priority date: 1999-05-04
Filing date: 2000-04-28
Publication date: 2000-11-09
Also published as: IL146314A0; GB9910268D0; CA2373017A1; EP1179260A1; AU4588400A

Abstract

A method and apparatus for profiling a flow of event data packets. The method comprises the steps of: receiving data defining sub-periods which partition a base time period, creating a profile of recent behaviour for each sub-period, and allocating each event data packet to one of the sub-periods according to a time indication associated with the event data packet. The method and apparatus may be used in anomaly detection within data streams and, in particular, account fraud detection where the event data relates to account usage.

Description

METHOD AND SYSTEM FOR FRAUD DETECTION IN TELECOMMUNICATIONS

FIELD OF THE INVENTION

The present invention relates to a method and apparatus for performing pattern recognition within event streams, and a system incorporating the same.

BACKGROUND TO THE INVENTION

In recent years there has been a rapid increase in the number of commercially operated telecommunications networks in general and in particular wireless telecommunication networks. Associated with this proliferation of networks is a rise in fraudulent use of such networks the fraud typically taking the form of gaining illicit access to the network, and then using the network in such a way that the fraudulent user hopes subsequently to avoid paying for the resources used. This may for example involve misuse of a third party's account on the network so that the perpetrated fraud becomes apparent only when the third party is charged for resources which he did not use.

Since fraudulent use of a single account can cost a network operator a large sum of money within a short space of time it is important that the operator be able to identify and deal with the most costly forms of fraud at the earliest possible time.

One of the steps employed in, but not limited to use in, such fraud detection systems is pattern recognition from event streams.

Pattern recognition for event streams can be achieved by building up profiles of the behaviour of an entity and performing pattern recognition over these profiles. In order for an entity to be profiled in this way, the entity must be able to have events associated with it. Examples of entities are: a single subscriber in a telephone network, a user accessing a data network, a switch in a telephone network or a server in a data network. The events to be associated with the user must be able to be represented in an Event Data Packet (EDP). The profiles of entities behaviour are compared with known patterns of unacceptable behaviour to determine if the system should alert the end user to the entities behaviour pattern.

The flow of Event Data Packets 1 10 of information through a profiling pattern recognition system is shown in Figure 1 . The Recent profile 130 represents the typical usage for the entity over a recent period of time, approximately the last week. The Historical profile 140 represents the typical use for the entity over a preceding and longer time period, for example approximately the last six weeks. The EDPs are all accumulated into Polls of information. A Poll 120 is a set of EDPs received over a particular time period (e.g. 4 hours). The Poll information is then used to update the values in the Recent profile, and the Recent profile is then used to update the values in the Historical profile. The solid arrow between the EDPs and the Poll indicates that the information in each Poll is directly created from the EDPs. The dotted arrow between the Poll and the Recent indicates that the Poll information is used only to update the

Recent behaviour, as is true for the Recent to Historical.

In an example where the EDPs are Call Detail Records (CDRs) and the profiles represent voice telephony usage is given the profiles may consist of number of calls made and the duration of national and international calls. Table 1 shows an example of Recent and Historical profiles for such an example.

Table 1 : Voice telephony recent and historic profile example If subsequent CDRs create a Poll of:

• calls 3,

national 500,

international 100.

Then after polling and once all updates to Recent and Historic profiles have completed the Recent and Historic profiles may be as shown in Table 2.

The new recent profile is derived from the previous recent profile plus a proportion of the difference between the new and old recent profiles

The new historic profile is derived from the previous historic profile plus a proportion of the difference between the new and old historic profiles, but the proportions typically differ from that of the recent profile case in that a higher proportion of the old historic profile is taken.

Table 2: Voice telephony recent and historic profile example after update

It can be seen that the Recent profile has moved towards the newly added Poll profile and the Historic toward the previous Recent profile. These profiles provide a view of the entity's behaviour and how it changes over time. The profiles of behaviour can then be used for pattern recognition to identify which entity's behaviour reflects patterns to which the user of system wishes to be alerted. There are however the following limitations to the method described above:

The Recent and Historic profiles are built up from a series of Poll profiles. In order for the Recent and Historic profiles to maintain their integrity all Poll profiles must cover the same amount of time, for example a 4 hour period.

The period of time the Polls must all cover must not be too small, otherwise natural variations in behaviour will appear to be anomalous. A typical recommended minimum is two hours.

These two limitations, taken in consideration, mean that this method cannot be used for real time data feeds.

It is also incumbent upon the user to ensure that the data given to the product is split into appropriately sized chunks. This can be a burden to the user if, for example, hardware downtime means it is necessary to feed a backlog of data into the system.

The profiles generated only represent the active periods for the user, this means that a user who is active in only one two hour period a week could have a similar profile to a user who is active in twenty of the two hour periods in a week.

The nature of the data in the profile - as an average of activity in all X minute periods where the user had actually been active - where X is the duration of the Poll, is not intuitive to many end users of the system.

In order for pattern recognition to occur effectively, the known patterns have to be represented in the same time period that the systems polls over. This can increase training times for the account fraud detection system which analyses the Poll, Recent profile, and Historical profile information in order to identify anomalies.

OBJECT OF THE INVENTION

The invention seeks to provide an improved method and apparatus for behavioural pattern recognition for event streams in general and for event streams in an account fraud detection systems in particular. SUMMARY OF THE INVENTION

According to a first aspect of the present invention there is provided a method of profiling a flow of event data packets comprising the steps of: receiving data defining a plurality of sub-periods which partition a base time period; creating a profile of recent behaviour for each of said sub- periods; allocating each Event Data Packet to one of said sub-periods according to a time indication associated with said Event Data Packet.

The method may also comprise the steps of: creating a profile of historical behaviour for each of said sub-periods; at the end of said Base Time Period updating each of said Historical profiles responsive to the previous value of said Historical profile and a corresponding Recent profile, and resetting each said Recent profile.

The method may also comprise the steps of: calculating an Event density for at least one of said Recent profiles.

In a preferred embodiment, the said step of calculating an Event density comprises the steps of: identifying a current time; identifying a Recent profile within which said current time falls; dividing a number of events recorded in said Recent profile by a time duration determined by a difference between said current time and a start time of sub-period associated with said Recent profile.

Said Event Data may correspond to time intervals of differing length.

The method may be used to capture a representation of inactivity within said flow.

The method may also be used to permit trend analysis for an initial sub- period during said sub-period.

According to a further aspect of the present invention there is provided a method of performing anomaly detection on a stream of Event Data Packets and comprising the steps of: receiving data defining a plurality of sub-periods which partition a base time period; creating a Recent profile for each of said sub-periods; allocating each Event Data Packet to a sub- period according a time indication in said Event Data Packet. According to a further aspect of the present invention there is provided a method of account fraud detection comprising the steps of: receiving data defining a plurality of sub-periods which partition a base time period; creating a Recent profile for each of said sub-periods; receiving a series of Event Data Packets relating to account use; allocating each Event

Data Packet to a sub-period according a time indication in said Event Data Packet.

In a preferred embodiment account use relates to telecommunications network use.

In a preferred embodiment said Event Data Packets are call detail records.

According to a further aspect of the present invention there is provided a method of network intrusion detection comprising the steps of: receiving data defining a plurality of sub-periods which partition a base time period; creating a Recent profile for each of said sub-periods; receiving a series of Event Data Packets relating to account use; allocating each said Event Data Packet to a sub-period according to a time indication in said Event Data Packet.

In a preferred embodiment said Event Data Packets relate to network audit log data.

In a preferred embodiment said Event Data Packets relate to IP packet data.

According to a further aspect of the present invention there is provided a system for profiling a flow of event data packets comprising: apparatus arranged to receive and store data defining a plurality of sub-periods which partition a base time period; apparatus arranged to create and store a Recent profile for each of said sub-periods; allocating each Event Data Packet to one of said sub-periods according to a time indication associated with said Event Data Packet.

The system may be arranged to receive a plurality of flows and to perform processing on each flow independently of each other.

According to a further aspect of the present invention there is provided a system for performing anomaly detection on a stream of Event Data Packets and comprising: apparatus arranged to receive and store data defining a plurality of sub-periods which partition a base time period; apparatus arranged to create a profile of recent behaviour for each of said sub-periods; apparatus arranged to allocate each Event Data Packet to a sub-period according a time indication in said Event Data Packet.

According to a further aspect of the present invention there is provided a system for account fraud detection comprising: apparatus arranged to receive and store data defining a plurality of sub-periods which partition a base time period; apparatus arranged to create a profile of recent behaviour for each of said sub-periods; apparatus arranged to allocate each Event Data Packet to a sub-period according a time indication in said Event Data Packet.

According to a further aspect of the present invention there is provided a system for network intrusion detection comprising: apparatus arranged to receive and store data defining a plurality of sub-periods which partition a base time period; apparatus arranged to create a profile of recent behaviour for each of said sub-periods; apparatus arranged to allocate each Event Data Packet to a sub-period according a time indication in said Event Data Packet.

The invention also provides for a system for the purposes profiling a flow of event data packets which comprises one or more instances of apparatus embodying the present invention, together with other additional apparatus.

According to a further aspect of the present invention there is provided software on a machine readable medium arranged for profiling a flow of event data packets by: receiving data defining a plurality of sub-periods which partition a base time period; creating a Recent profile for each of said sub-periods; allocating each Event Data Packet to one of said sub- periods according to a time indication associated with said Event Data Packet.

The preferred features may be combined as appropriate, as would be apparent to a skilled person, and may be combined with any of the aspects of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

In order to show how the invention may be carried into effect, embodiments of the invention are now described below by way of example only and with reference to the accompanying figures in which:

Figure 1 shows a block diagram of information flow in a behavioural pattern recognition system in accordance with the prior art;

Figure 2 shows a block diagram of information flow in a behavioural pattern recognition system in accordance with the present invention.

DETAILED DESCRIPTION OF INVENTION

The method proposed here is illustrated in Figure 2. The EDPs 210 (in this example taking the form of Call Detail Records (CDRs)) again feed into a Poll 220 of information and the Poll information is used to update the values in the Recent profiles 230a-f In this case each entity has associated with it multiple Recent Profiles (six are shown but more or fewer may be used), where each Recent profile represents a period of time within a week (though a larger or shorter base period could be used), for example Saturday and Sunday between midnight and 8am. The Recent Profiles together cover the whole of a week period. Each Recent Profile has a related Historic Profile 240a-f which covers the same time period.

Recent Profiles are filled until they contain all the data for the time period they cover. Once filled the values are used to update the corresponding Historic profile, and then the Recent profile values are reset to zero, and filled with the next CDRs in the time covered by the profile.

For example, a customer of voice telephony may have the Recent profiles of behaviour illustrated in Table 3 and corresponding Historic profiles illustrated in Table 4.

Table 3: Voice telephony recent profiles example

Table 4: Voice telephony historic profiles example

A collection of Event Data (CDRs) is then presented to the system. The CDRs cover 7am on a Monday through to 1 pm on the same Monday. The previous collection of data presented to the system had contained a CDR for 5am on the same Monday.

The CDR at 7am is added to Recent Profile 1. When this profile is 'complete' the historic profile is updated. When the next time period is entered its recent profile values are reset to zero and new values accumulated.

The Recent and Historical profiles after the data has been processed areas illustrated in Tables 5 and 6 respectively.

Table 5: Voice telephony recent profiles after processing

Table 6: Voice telephony historic profiles after processing

The only Recent profiles changed are those that cover the same time period as the CDRs in the poll namely periods 1 and 2. The only Historic profile changed is in period 1 , the values in the Recent profile having been used to update the Historic profile. After updating the Historic profile, the Recent profile is then reset to zero before new CDR information is added to it.

Historic profiles are only updated once the Recent profile has been filled with all the information for that time period. This means that the size of the Poll has no influence over the Historic profiles, and the Recent profiles can contain details for any sub-period of the time period they cover, or the whole time period. The profiles of behaviour are converted into Event Densities before pattern recognition is performed on them. Event Densities are produced by dividing the event data value by the number of seconds in the period during which those events occurred. For example, Table 6 shows an example set of Historic profile values and the corresponding event densities values where the period covered 14400 seconds (4 hours).

Table 7: Voice telephony historic profiles after processing

Event densities for historic profiles provide an average of behaviour over the whole time period. This means that dividing by the number of seconds in the time period gives the normal amount of behaviour in any one second. These are generally small values.

Recent profiles however may or may not contain values for the whole the time period they cover. Freqυently the Recent profile that is being analysed is not yet complete. For example, if ten minutes of event data require analysing for the time period 9.15am to 9.25am then a recent profile that covers the time period 8am to 6pm will be updated, but the time period for this profile is not yet complete. As the period is incomplete the number of seconds to divide by is calculated as follows. The complete time period is divided into blocks of time, for example 30 minutes. A usage period consists of x of these blocks of time. The event data in the current incomplete Recent profile is divided by the number of seconds in the blocks covered so far. So event data covering up to 9.25 am has covered three 30 minute blocks so far and the values are divided by 5400 seconds (90 minutes). Conversion into densities enables pattern recognition to be performed over event data that covers just a portion of the total time period. This method has the advantages that:

• the polls of event data can be of any size whilst still allowing the profiles produced by the system to maintain their integrity;

• polls of data for very small time periods can be handled easily;

• the preceding two advantages have the consequence that the system is suitable for both real time feeds and bulk batch feeds of poll data;

• there is consequently no burden on the end user to divide up the event data into fixed sized chunks; and

• the profiles represent accurately the behaviour of the user, including a representative of inactivity by the user, and a representation of the time of use.

This method may be used in several application areas. These include telephony fraud detection using call detail records (CDRs), anomaly detection on data streams, network intrusion detection using audit log data or IP packet data. The method also provides a means of comparison between recent behaviour and past behaviour for event streams that has potentially wide application for the rapid detection of behavioural changes.

Any range or device value given herein may be extended or altered without losing the effect sought, as will be apparent to the skilled person for an understanding of the teachings herein.

Claims

1. A method of profiling a flow of event data packets comprising the steps of:

receiving data defining a plurality of sub-periods which partition a base time period;

creating a profile of recent behaviour for each of said sub- periods;

allocating each Event Data Packet received to one of said sub- periods according to a time indication associated with said Event Data Packet.

2. A method according to claim 1 comprising the steps of:

creating a profile of historical behaviour for each of said sub- periods;

at the end of said Base Time Period updating each of said Historical profiles responsive to the previous value of said Historical profile and a corresponding Recent profile, and resetting each said Recent profile.

3. A method according to any one of claims 1 - 2 additionally comprising the step of:

calculating an Event density for at least one of said Recent profiles.

4. A method according to claim 3 wherein said step of calculating an Event density comprises the steps of:

identifying a current time;

identifying a Recent profile within which said current time falls;

dividing a number of events recorded in said Recent profile by a time duration determined by a difference between said current time and a start time of sub-period associated with said Recent profile.

5. A method according to any one of claims 1 - 4, wherein said Event Data may correspond to time intervals of differing length.

6. A method according to any one of claims 1 - 5, whereby to capture a representation of inactivity within said flow.

7. A method according to any one of claims 1 - 6, whereby to permit trend analysis for an initial sub-period during said sub-period.

8. A method of performing anomaly detection on a stream of Event Data Packets and comprising the steps of:

creating a Recent profile for each of said sub-periods;

allocating each Event Data Packet to a sub-period according to a time indication in said Event Data Packet.

9. A method of account fraud detection comprising the steps of:

creating a Recent profile for each of said sub-periods;

receiving a series of Event Data Packets relating to account use;

10. A method of account fraud detection according to claim 9, wherein said account use relates to telecommunications network use.

11. A method of account fraud detection according to any one of claims 9 - 10, wherein said Event Data Packets are call detail records.

12. A method of network intrusion detection comprising the steps of: receiving data defining a plurality of sub-periods which partition a base time period;

creating a Recent profile for each of said sub-periods;

receiving a series of Event Data Packets relating to account use;

allocating each said Event Data Packet to a sub-period according to a time indication in said Event Data Packet.

13. A method of network intrusion detection according to claim 12, wherein said Event Data Packets relate to network audit log data.

14. A method of network intrusion detection according to claim 12, wherein said Event Data Packets relate to IP packet data.

15. A system for profiling a flow of event data packet polls comprising:

apparatus arranged to receive and store data defining a plurality of sub-periods which partition a base time period;

apparatus arranged to create and store a Recent profile for each of said sub-periods;

allocating each Event Data Packet in said Poll to one of said sub-periods according to a time indication associated with said Event Data Packet.

16. A system according to claim 15 arranged to receive a plurality of flows and to perform process each flow independently of each other.

17. A system for performing anomaly detection on a stream of Event Data Packets and comprising:

apparatus arranged to create a Recent profile for each of said sub-periods; apparatus arranged to allocate each Event Data Packet to a sub-period according to a time indication in said Event Data Packet.

18. A system for account fraud detection comprising:

apparatus arranged to create a profile of recent behaviour for each of said sub-periods;

apparatus arranged to allocate each Event Data Packet to a sub-period according a time indication in said Event Data Packet.

19. A system for of network intrusion detection comprising:

20. Software on a machine readable medium arranged for profiling a flow of event data packet polls by:

creating a profile of recent behaviour for each of said sub- periods;

allocating each Event Data Packet inset Poll to one of said sub- periods according to a time indication associated with said Event Data Packet.