Systems and methods for performing scoring optimization

Does the selected weighting factor (3 satisfy threshold criteria?

Does the selected weighting factor (3 provide an optimized result vis-a-vis a previous weighting, i.e., a weighting that was previously determined to be the optimized weighting?

Is the value of P better?

Yes

Return to step 560 (Fig. 5)

1

SYSTEMS AND METHODS FOR
PERFORMING SCORING OPTIMIZATION

BACKGROUND OF THE INVENTION

5

There are a wide variety of situations where determinations are made at different points through the course of a process and/or where determinations are made based on different data. Such determinations may vary by the particular type of data that is used in the determination. Alternatively, such determinations may involve data that is secured at a different time, i.e., updated data may be used (instead of older data that was used in a prior determination). A determination might be expressed in terms of a score, i.e., some other quantitative representation.

As can be appreciated, such different determinations made 15 over a period of time or made based on different data may vary in the result such determinations yield. For example, a credit card issuer may be conducting a campaign to secure new credit card customers. The campaign might typically involve determining individuals that should be mailed credit card 20 offers. In determining such individuals, the credit card issuer generates a credit risk score for each individual. The credit risk score may be based on data secured from a credit bureau or other data that is assessable by the credit card issuer. At this point in the process, the credit risk score might be character- 25 ized as a "front end" risk score. In other words, at actual mailing selection time, the credit card issuer has to select names for offers from the whole credit eligible universe.

Individuals who receive the offer (through mailings, e-mailings, or any other suitable medium) have the opportu- 30 nity to review and accept the offer. Accordingly, at some later time, the credit card issuer will receive responses from some of those individuals.

Once a response is received from an individual (a respondent), the credit card issuer then determines whether the 35 credit card issuer will indeed issue a credit card to the respondent. In other words, at credit approval/decline time, the business has to make the booking decision among all of respondent applicants. This decision involves determination of a further risk score, i.e., a "back-end" risk. The back-end 40 risk score will thus be determined at a later time, than the front end risk-score, and might also involve different parameters. As a result, it is very likely the back-end risk score is different from the one based on the random sample of the whole eligible credit universe, i.e., different from the front end risk 45 score.

In such situation, the credit card issuer, as between the front-end risk score and the back-end risk score, has two different universes and two different goals. Using known techniques, it is very difficult to provide satisfactory results 50 for one goal while it is developed against another goal. Historically business uses two different scores, one for the front end determination and one for the back end determination. However, that approach sometimes causes problems since the credit card issuer or other business makes the selection deci- 55 sion to mail an offer based on one score, and later the business decides to decline a responder of the offer based on second score. Such action is unfortunately sometimes necessary, from a business perspective, but is not beneficial to the business from a public relations perspective. 60

The above and other problems are present in known processes.

BRIEF SUMMARY OF THE INVENTION

65

The invention provides systems and methods relating to generating a unified determination based on subdetermina

2

tions, and in particular, generating a unified score based on respective scores. For example, the invention provides a method for generating a unified determination based on subdeterminations, the method including generating a first subdetermination based on first criteria; generating a second subdetermination based on second criteria; and generating a unified determination based on the first subdetermination and the second subdetermination. The generation of the unified determination includes (a) assigning, using iterative processing, an assigned weighting respectively to the first determination and second determination; (b) determining if the assigned weighting satisfies at least one constraint; (c) comparing the assigned weighting to an optimized weighting, which was previously determined, to determine if the assigned weighting is improved over the optimized weighting; and (d) if the assigned weighting is improved, then assigning the assigned weighting to be the optimized weighting.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention can be more fully understood by reading the following detailed description together with the accompanying drawings, in which like reference indicators are used to designate like elements, and in which:

FIG. 1 is a flowchart showing a process of building a unified score for both front end mail selection and back end credit decision in accordance with one embodiment of the invention;

FIG. 2 is a flow chart showing further details of the "generate combination score using optimization process" step of FIG. 1 in accordance with one embodiment of the invention;

FIG. 3 is a block diagram showing a score processing system in accordance with one embodiment of the invention;

FIG. 4 is a block diagram showing the combination score processor of FIG. 3 in further detail in accordance with one embodiment of the invention;

FIG. 5 is a flowchart showing a process of building a unified determination based on subdeterminations in accordance with one embodiment of the invention; and

FIG. 6 is a flow chart showing further details of the "generate the unified determination (based on the first subdetermination and the second subdetermination) using an optimization process" step of FIG. 5 in accordance with one embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, aspects of the systems and methods in accordance with various embodiments of the invention will be described. As used herein, any term in the singular may be interpreted to be in the plural, and alternatively, any term in the plural may be interpreted to be in the singular.

The systems and methods of the invention are directed to the above stated problems, as well as other problems, that are present in conventional techniques.

In introduction, the invention, in one embodiment, provides a system and method to perform risk based scoring. For example, the invention may be applied to the situation where risk scores are needed both in (1) selection of recipients for mailings; and (2) the decision whether to book applicants responding to such mailings, i.e., a risk score is needed that is representative of both the "front end" and "back end" risk assessment criteria.

As further background information in accordance with one embodiment of the invention, historically a business uses two different scores, one for the front end risk assessment and one 3

for the back end risk assessment. That approach causes problems in the situation that (1) a business makes a selection decision to mail an offer based on one score, and later, (2) the business decides to decline a responder of the offer based on second score. The invention provides a single score to over- 5 come this problem.

The invention provides a novel approach that generates a score that may be used for both front end and back end risk assessment. In a method of one embodiment of the invention, a front end initial score and a back end initial score are 10 developed using data obtained from a credit bureau, for example. The two initial scores are developed based on respective criteria relating to the front and back end risk assessments. Then, the two initial scores are combined into a single score. This combining is performed using an optimi- 15 zation process. The goal is that the single score resulting from the combination should satisfy applied constraints and outperform benchmark scores, so as to maximize the total performance measurement.

In accordance with one embodiment of the invention, the 20 optimization process (by which the initial front end score and the initial back end score are combined) may utilize Kolmogorov-Smirnov test (KS-test) processing. The KS-test is typically used to determine the magnitude that two data sets differ. In the invention, the KS-test may be utilized in iterative 25 processing to determine a best parameter to use in combining the two initial scores. In particular, the KS-test may be used to determine a weighting parameter based on respective objectives of the front end initial score and a back end initial score, and predetermined constraints. The predetermined con- 30 straints may be based on performance of benchmark scores and business requirements, for example. As noted below, other statistical methodologies may be used in lieu of the KS Test.

The invention as described herein may be applied to the 35 selection of recipients for mailings and the booking of the respondents to such mailings. Accordingly, the system and method of the invention may provide a key tool in acquisition mailing campaigns. In addition, the invention may be utilized in a wide variety of other situations to develop models—to 40 achieve satisfactory results for multiple goals at the same time.

Hereinafter, further aspects of the systems and methods of embodiments will be described in further detail.

Accordingly, in one embodiment, the invention addresses 45 the need to develop credit risk score for new account acquisition campaigns, and in particular in the situation where a business requires a single score for both front end processing (in selection of persons to send offers) and back end processing (in the credit approval or decline processing). 50

In this situation, at actual mailing selection time, a business has to select names, to which offers will be sent, from the whole credit eligible universe. At this stage of the process, a business may use the criteria of whether a prospect will go bad on any trade with a financial institution as objective 55 function for the risk score, i.e., the front-end risk score is predicting the possibility of whether the particular individual will have at least one bad trade with any financial institution in next several months, i.e., such as 12 months, for example.

However, at credit approval or decline time, the particular 60 business has to make the booking decision among all of responder applicants. This responder population has to become the development population for the back-end risk score and very likely it is very different from one based on the random sample of whole eligible credit universe. Further- 65 more for this back-end risk score, what the known processing predicts, for example, is whether or not an approved account

4

with the offering institution will be defaulted in the future, i.e., if the institution indeed decides to approve and book the account.

Accordingly, it shouldbe appreciated that the business, i.e., the offering institution, clearly has two different universes and two different objective functions. Using known techniques, it is very difficult to provide satisfactory results for one goal while it is developed against another goal, as is noted above. In particular, there is no easy statistical methodology to achieve both goals at same time. As should be appreciated, if the back end score provides different indicators, e.g. such as the credit worthiness of the individual, as compared to the front end score, the back end score may indicate to not extend the credit card offer. Such is problematic in disappointing the individual, who had been offered the card, and thus bad from a public relations perspective.

In order to solve the above conflict between two scores, the invention provides a unified score to represent both the traditional front end score and the traditional back end. The invention provides the unified score in what is believed to be a very novel approach. In a first step, each score is generated based on its own population and objective function. In accordance with one embodiment of the invention, each score is generated with the same set of bureau variables and bureau data.

In a second phase of the process, an optimization process is used to combine the two scores into a final product, i.e., a unified score. One goal is that the final score should outperform the individual benchmark scores on its own population, but at the same time maximize the total performance measurement.

In accordance with one embodiment of the invention, FIG. 1 is a flowchart showing a process of building a unified score for both front end mail selection and back end credit decision. As shown, the process starts in step 100 and passes to steps 112 and 114. That is, steps 112 and 114 are performed in parallel.

In step 112, a first objective is identified. In this example, the first objective is to predict whether the particular individual is a bad risk based on credit bureau data, i.e., so as to determine whether the particular individual is a good candidate to forward an offer. The first objective might be thought of as determining the merits of the individual and their credit risk vis-a-vis the individuals existing creditors. Accordingly, from step 112, the process passes to step 122. in step 122, the front end risk score, score_l is determined. In other words, score_l assesses available candidates in the credit eligible universe to determine which candidates should be mailed, or otherwise forwarded offers. After step 122, the process passes to step 130.

In parallel to steps 112 and 122, in this embodiment, the process of FIG. 1 includes step 114. In step 114, an objective is determined to predict an internal bad determination, i.e., from the perspective of the bank considering issuance of the credit offerto the individual. Accordingly, in step 124, what is herein characterized as the back end risk score (score_2) is determined. Score_2 is a score assessing respondents from the responder population, i.e., so as to determine whether the bank will extend a credit line to a responder.

Accordingly, as shown in FIG. 1, score_l is the score built to predict objective one and score_2 is the score built to predict objective two. In this embodiment, we can find the best parameter to combine the two scores by solving an optimization problem.

That is, after performing the steps 122 and 124, the process of FIG. 1 passes to step 130. In step 130, a combination score is generated using an optimization process as described 5

below. The optimization process results in a new combination score, which is output in step 160.

After step 160, the process of FIG. 1 passes to step 170. In step 170, the process ends.

FIG. 2 is a flow chart showing further details of the "gen- 5 erate combination score using optimization process step" of FIG. 1 in accordance with one embodiment of the invention. As shown in FIG. 2, the process starts in step 130, and passes to step both of steps 132 and 134.

In step 132, the subprocess of FIG. 2 inputs the front end 10 risk score, score_2. In parallel with step 132, in step 134, the process inputs the back end risk score, score_2. After each of steps 132 and 134, as shown in FIG. 2, the process passes to step 140.

In step 140, the optimization process is initiated. That is, in 15 this embodiment, the value of N is assigned 1,000, a counter "i" is assigned a value of 1, and a is assigned a value of zero. In other words, initial values are assigned to variables in order to initiate the iterative optimization process. The values of "i" and "N" control the progression through and the termination 20 of the iterative process. After step 140, the process passes to step 142.

As shown in FIG. 2, the variables are as follows:

(3 is a weighting factor or value that is generated for consideration in the optimization process. |3 might be generated 25 by a random number generator in some controlled manner;

a is a weighting factor or value that represents the best weighting factor achieved at a particular point in the progression of the optimization process, a might be initiated at zero (0), for example; 30

"i" is the progressing integer value, i.e., a counter, that controls the progression of generating the |3 values. For example, "i" might be 1, 2, 3, 4 . . . 1000;

N is the value to which "i" will progress, e.g., 1000;

CI is a benchmark score to determine if, in terms of Objec- 35 tive 1, (3 is a feasible value;

C2 is a benchmark score to determine if, in terms of Objective 2, (3 is a feasible value; and

KS is a Kolmogorov-Smirnov statistic.

Hereinafter, further general aspects of the systems and 40 methods of embodiments will be described in further detail. In this embodiment as described above, first, the process develops each score based on its own population and objective function, using the same set of bureau variables, or other suitable variables. Then, an optimization process is used to 45 combine these two scores into a final product, i.e., a unified score. The goal is the final score should outperform existing individual score on its own population at same time and maximizes the total performance measurement. In this embodiment, it is assumed that score_l is the score built to 50 predict objective one and score_2 is the score built to predict objective two. In this embodiment, the invention finds the best parameter to combine the two scores by solving the following optimization problem:

Maximize(iffi(SO_l+a*SO_2,objective one)+KS 55
(SCR_l+a*SCR_2,objectwe two))

Subject to:

KS(SCR_l+a*SCR_2,ob)ective one)>Cl; AND 60
KS(SCR_l+a*SCR_2,ob)ectivetxo)>C2

Here the benchmark scores CI and C2 are predetermined by the performances of benchmark scores and business requirements, or other parameters as may be desired. As a 65 further step, adverse action reason codes may be generated from the same pool of model attributes by using appropriate

6

weights. That is, an adverse action reason code may be needed to provide a reason why a responder (who was initially extended an offer) was declined, i.e., subsequent to responding to the offer. Accordingly, the optimization process (in conjunction with generating the unified value) may also be manipulated so as to provide some intelligence regarding why a respondent might be declined (i.e., adverse action reason codes).

With the approaches described above, the invention achieves the goal of obtaining a unified score by taking advantages out of both scores. In general, it is appreciated that the idea of the invention may be used to develop models to achieve satisfactory result for multiple goals at the same time. The invention might be used in a variety of business and/ or for other objectives.

Returning now to FIG. 2, after step 140 in which the optimization process is initiated, the process passes to step 142. In step 142, a (3 value is generated. The (3 value is generated in some suitable manner, i.e., such as using a random number generator. Various known techniques might be used such as a Monte Carlo approach and/or stratification of the (3 values. Accordingly, in some suitable manner, (3 is generated in step 142. As shown, (3 maybe constrained to be between 1 and 10.

As shown in step 142, (3 is generated so as to be used in the KS statistic, and specifically in the parameter:

score l+p*score 2.

After step 142, the process passes to step 144. Step 144 might be characterized as a determination of whether the current value of (3 is feasible. Such feasibility is determined vis-a-vis benchmark scores CI and C2. That is, in step 142, the process determines if the KS statistic based on (3 is satisfied vis-a-vis CI and C2, i.e., the process determines if:

iCS(score_l+p*score_2,Objective 1)>C1 AND
iCS(score l+p*score 2,Objective 2)>C2

If such two relationships are not satisfied, such is indicative that the current value of (3 (i.e., the current weighting of the first and second scores) is simply not feasible. Accordingly, the process of FIG. 2 passes from step 144 to step 152. In step 152, the value of "i" is incremented by "1", i.e., to count of one iteration. Then the process passes to step 154 to determine if another iteration should be performed, i.e., if the value of N has been attained by "i". If the value of N has not been attained, the process passes back to step 142 for another iteration.

Accordingly, in step 142 another (3 value is generated, e.g., using a random number generator. The process will then again proceed to step 144 to determine if the new value of (3 is feasible, i.e., to determine if the new value of (3 satisfies the criteria of step 144. Processing will then proceed as discussed above.

In step 144, if a particular value of (3 satisfies the benchmark criteria, the process passes to step 146. Step 146 might be characterized as presenting a challenger (3 that is compared with the existing champion a. That is, the total KS statistic of (score_l+(3*score_2,Objective 1) plus the KS statistic of (score_l+(3*score_2,Objective 2) is determined. This KS statistic is compared with the KS statistic of (score_l+ a*score_2,Objective 1) plus the KS statistic of (score_l+ a*score_2,Objective 2). In other words, the KS statistic based on (3 is compared with the KS statistic based on a.

In step 146, if the KS statistic based on (3 is "less" than the KS statistic based on a, i.e., the relationship of step 146 is not satisfied, than the existing a remains the best weighting