CA2694451A1 - Cluster and discriminant analysis for vehicles detection - Google Patents

Abstract

A method is provided herein for determining and recognizing types of vehicles passing a cheek point.
The method takes advantage of an EM algorithm which is up-loaded into a CPU
and which processes data of the vehicles which drive past a checkpoint, the data being representative of essential characteristics of vehicles to produce an output model of the traffic volumes of the various types of vehicles. This model enables the forecasting of future road maintenance costs and the, planning and designing of future road networks.

Description

C

Tide, CLUSTER AND DISCRIMINANT ANALYSIS FOR VEHICLES DETECTION
100011 This invention relates to a system and method for cluster analysis for vehicle iden0catiort and claims priority of application 61/154866, filed 02/24/2009, the entire content of which are incorporated herein by reference

(0002) It is very useful to build an automatic computer system to recognize the types of vehicles passing a checkpoint given some easy-to-get data about the vehicles, such as the distances between axles, the weights on each axle. Such a system has many applications, for example, in monitoring traffic volumes and identifies the type of vehicle, which will be helpful in budgeting road maintenance costs.

[0003] The simplest clustering technique is the K-means clustering- However, K-means clustering requires that the users supply with a number of clusters. X-means clustering may be an alternative method since it can detect the number of clusters with some simple criteria, but X-means would introduce more severe local mode problem.

BACKGROUND INFORMATION

[0004] The partitioning of large data sets into similar subsets (Cluster Analysis) is an important statistical technique used in many fields (data 'mining, machine learning, bioinfonrnatics, and pattern. recognition and image analysis). In traffic research, it is useful both to determine and to recognize the types of vehicles passing a checkpoint.
Traffic data collection systems would collect data (e. g.., vehicle length, distances between axles, weights on axles) and such data may be used to determine and recognize vehicle types in high volume traffic, monitoring traffic volumes of various types of vehicles forecasting future road maintenance costs and planning and design of future road networks..

[0005) ,The consequence of such determination and recognition of vehicle types in high volume traffic has many applications, C.&,, monitoring traffic volumes of various types of vehicles, forecasting future road maintenance costs and planning and design of future road networks.

DESCRIPTION OF THE INVENTION
AIMS OF THE INVENTION

(0006] A main aim of the present invention is to develop a better methodology for cluster analysis with application to the problem of vehicle detection and determination of its type as noted above.

[0007) Another aim of the present invention is to provide a new method to overcome potential. problems by merging similar clusters after running X-means clustering.
[0068] Another aim of the invention is to provide better methodology for cluster analysis with application to the vehicle detection problem.

STATEMENT OF INVENTION

(0009] One aspect of the present invention provides a method of determining and recognizing -the' types of vehicles passing a checkpoint by collecting vehicle data (e.g., vehicle length, distances between axles, weights on axles) and using that data to determine and recognize vehicle types, particularly in high volume traffic for monitoring traffic volumes of various types of vehicles, forecasting future road maintenance costs and planning and design of future road networks, the method comprising: uploading a computer program into a CPU, the computer program comprising an EM algorithm as particularly described in the specification herein, the algorithm including data representations, of essential characteristics vehicles as they drive past the checkpoint; entering such measured characteristics vehicles as they travel past the checkpoint into that CPU; and deriving an output from that CPU, and thereby determining and recognizing the types of vehicles passing the checkpoint, particularly in high volume traffic, for monitoring traffic volumes of various types of vehicles, forecasting future road maintenance costs and planning and design of future road networks.

[00101 Another aspect of the present invention provides an apparatus comprising the combination of, a CPU: and a computer program which has been uploaded into said CPU, the computer program comprising an EM algorithm as particularly described in the specification herein, the algorithm including data representations of essential characteristics of vehicles.

[00'11) It has been found according to aspects of the present invention, that there are correlations between different variables. This invention proposes to avoid the problem which arises by using the Euclidean distance, since data may be assigned to the wrong centrolds. The present invention seeks to overcome this problem by replacing the Euclidean distance with the Mahalanobis distance-(00121 The following description provides examples of methods of aspects of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS
In the accompanying drawings, [0013.) Fig 1 is .a graph showing some data points derived from traffic data collection systems which have collected data (e. g.., vehicle length, distances between axles, weights on axles, etc.), which data may be used to determine and recognize vehicle types in high volume traffic, but in which the cluster points are incorrectly clustered [0014] Fig 2 is a graph showing some data points derived from traffic data collection systems which have collected data (a. g.., vehicle length, distances between mules, weights on axles, etc.), which data may be used to determine and recognize Vehicle types in high volume traffic, but in which the cluster points are correctly clustered [0015] Fig 3 is a graph showing. some data points derived from traffic data collection systems which have collected data, (e. S.., vehicle length, distances between ,axles, weights on axles, etc.), which data may be used to deter-mine and recognize vehicle types in high volume traffic, where the X-means is used to cluster these points, using Euclidean distances, which is not correct.

[0016] one explanation of the results plotted in Figures I, 2 and 3 is because the X-means algorithin'does not permit returning back to re-cluster the data, since it runs a local K-means for each pair of "children". The K means is local in that the "children"
are fighting each other for the points in the "parent's"region and in no others. All the points from the other regions are ignored.

[0017] This problem of local mode can be overcome, according to broad aspects of the present invention, by merging two regions which are close to each other after the X-means algorithm is run. If the model after merging has a higher BIC score than the model before merging, these regions will be merged. Otherwise, the original model is kept.

ASSUMPTIONS
(00IS].In the method of determining and recognizing the types of vehicles passing a checkpoint by collecting vehicle data (e ,g., vehicle length, distances between axles, weights on axles) according to aspects of the present invention, by plotting graphs for all the variables (axle spacing, weights, front bumping spacing, and rear bumper spacing) in the data set, each variable forms a pattern which is similar to a "Student's" t. distribution. Therefore, it will be assumed that each variable in the data set comes from a "Student's" t-distribution Since all the variables for a given data point must' be considered, it will be assumed that each data point forms a multivariate "Student" distribution. In statistics, a "multivariate Student distribution" is a multivariate generalization of the Students t-distribution.

[40l9f The Present invention will be further described by reference to method steps to be carried out [ 00020] FINDING PARAMETER VALUES WITH EM ALGORITHM METHOD STEPS
[00021] The setup for the method steps is that given "N" data points in a V-dimensional space, it is desired to find a set of "K" "multivariate Student's t-distribution" that best represents the distribution of all the data points. Without being bound by theory, it is believed that the given data set is an N * V matrix, where N stands for the number of data points and V stands for the number of variables for each data point.

[OOO22] DEFT TION OF TERMS
N = number of data points.

V number of variables.
K number of dusters.

k = the mean for kill cluster, each a vector of length V.

E k:- the covariance matrices for kth duster, each of size V* V.
X11.= the nth data point, which is a vector with length V.

S

P(k ! xa:) - the probability that xn Comes from cluster k.

p(k) = : the probability that a data point chosen randomly comes from cluster k.
P(xn) = the probability of finding a data point at position xn = the value of log likelihood of the estimated parameter set.

(00023] For simplicity, it is assumed that k is a diagonal matrix, i.e., a matrix whose non diagonal entries are all 0, and where the diagonal entries are the variances for each variable.
[00024) Three statistical methods are used for the method steps, which are are carried out to find pararncter values with the EM algorithm, splitting clusters using Principle Component Analysis (PCA), and comparing models by Bayesian Information Criterion (BIC).

[00025] X is the key to tktis method. We it is not desired to be limited by any particular theory, it is believed that it is necessary to find the best values for the parameters by maximizing the value of I X. This method maximizes the posterior probability of the parameters if the specific priors are given.

[00026) The -steps are described as follows [00027] Step I

[00028] Set the=starting vahies forthe pks, Ek s, P (k). The method to obtain these values is by means of splitting clusters using PCA as follows:

[00029) The setup for this method is that, given some data points in one cluster (mother cluster),it is necessary to split these data points into two clusters (children clusters), using PCA.
PCA is mathestiatjcally deAned'as "an orthogonal linear transformation that transforms the data to anew coordinate system such that the greatest variance by any projection of the data comes to lie on the fast coordinate (called the first principal component), the second greatest 'variance on the second coordinate", and so on. The data set comprises an N * matrix. PCA
is .now performed an this data matrix. The standard deviations are now calculated of the principal components, namely, the square roots of the eigenvalues of the covariance matrix. The matrix is now calculated far variable loadings, namely, a matrix whose columns contain the eigenvectors.
In R, the is a built-in function called "preomp" which helps the calculations.

(00030] The terms used are defined as follows:
std = a vector contains the square roots of the eigenvalues of the covariance matrix.
Rotation = a matrix whose columns contain the eigenvectors Range: how far the data is to be split; the.value of range is usually between 1.5 and K = the mean for the mother cluster, each a vector of length V

E = the covariance matrix for the mother cluster, each of size 'P(M) = the probability that a data point chosen randomly comes from the ritother cluster (0003 i] Since the first principal component is the most important component, two vectors are created with length V from the first principal component. Two vectors are created since it is desired to to split the data into two clusters. The first element in. the first vector is the value of range (plus range) , and the other elements are all zero. The first element in the second vector is the value of -range (minus range) . and the other elements are all zero. The first vector is V 1, and the second vector is V 2. Consider V 1 , V2, and std to be matrices with one column. After the splitting is done, two meads are provided for two children clusters. The mean for the first children cluster is placid the mean for the second cluldren cluster }-2.The calculation for gland is as follows:

Pl li+ rotation W% (V 1 * Std) p2= 1,E+ rotation %* %o (V2 std) [000321 Here, k*k and "%.*%" different operations.
(00033]For example, a d a+d a d g j a*j +d*h+g*1 b*e -b*e but beh%*% k b*j +e*k+h*1 c f c*f c f i I c+j +f*k+l+l [0034] The covariance matrices for two children clusters would be the same as the mother cluster, and the probability that a data point chosen randomly comes from the children clusters would be half ofP(m).

[000351 In stmunary, for the first children cluster, mean = l , covariance matrix = E, and probability that a data point chosen randomly comes from this cluster = (I
/2)*P{m); for the second children cluster, mean 112, covariance matrix = E, and probability that a data point chosen randomly comes from this cluster = (1/2)*P(m).

[00030] There is one limitation about PCA. If PCA is performed on a given data matrix, PCA
requires 'the number of data points'to be larger than the number of variables.
If the number of data points is smaller than the number of variables, PCA will do nothing on the data matrix, and splitting will not happen [000361 Step 2-[0003.71 Given the values for the k's, F..k s, and P (k), and the data, of P
(xn 1 k , Ek) can be calculated. we assume all the variables in the data set form a multivariate Student's distribution, P (xn - k, Mk) is the multivariate Student's density, that is, rj td_'~
P (xul Pe. fir) {dr r (doh (detE) z I t (s) * (xa ^ Ne)F . E-A . (xn - pk}

where df w the degree of freedom, p T the number of variables, and detE = the determinant for [00037] One important thing about P (xn l k , Ek) is that the values of P (xn 1 Ak , FY often be very small as to underflow to zero. Thereforc,it is necessary to work with the logarithm ofP (xn 1 k Lk) instead of P(zn 1 k , Ek), that is 10$ P (X.1 uk= F-k) logr(df+p) - logr (d2) - log (rc)gz - log(df)Pz - log(det'E)z -iogl+\_) (xõ_1i*)Ts E r(,xx-Pk)J

[00038]After the value'of P (xn Ilk , Ek) is obtained, it becomes possible to calculate the value of P (xc) splitting P (xn) into its contribution from each of the K
multivariate Student's t-distributions, that is, P (xii) P (xn I Pk, Ek) P(k) k [00039] One problem may rise for P(xn), where it becomes necessary to calculate the sum of quantities. Some of these quantities may be so small that they underflow to zero. According to an aspect of the present invention, it has been found that one possible way to fix this problem is construct the quantities from their Logarithms. That is, store P (xn 1 k , F.k) P(i) in log P (xn l k , Zk), and let m, - max log (P (xn 10k , )-k)P(i)0..., log P(k))). Then the logarithm of the sum is computed as follows:

IagP (x,1) a log(m ,,) + log(l exp(log (P(; l1 L. Ef)F(t)) - mraa)) 1000401 Using the values of P ,(xn I k , Zk) and P (xn), the value of P(kl xn) and 3.:
P(klx,) = P(XRII~~=EK)P(k) P (xõ) A = tog (fl P(xõ )) _ X logP (x') 100041 ] Since the values of log P (x, ' I l k Ek) and log P (xn) can be computed in order to overcome the problem of underflow, it is possible to write P(k 1 xn) in terms of log P
(xn l k , Ek) and I. P (xn), p(klxx) = exp (IogP(xnl Lk.Er) + iogP(k) - logP(x,,)) [00042]By calculating P (ac I xn) for all values of k's and xõ's, it is now possible to obtain all.of the value's P (k I x,1}'s, and it now becomes possible to write P (k I xa)'s as a probability matrix of size N * K. Each row = one data point, and each column =
one cluster. Each element in the matrix = the value of P (k l xn), that is, the probability that a given data point comes from a specific cluster k. In the language of the EM
algorithm, this is called an expectation step or an E-step.

[00043] Step 3:

[00] Using P (k I Qs from step 2, the values of maximum likelihood estimates for k's is, Ek s and P(k)'s and for all values of k, can be calculated, that is , the values of k's , Zk's and Pik)'s that maximize the log likelihood function X. The maximum likelihood estimate for P(k) is easy to obtain-P(k) p (klx=,) 1000451 The process to calculate the maximum likelihood estimates for id's , rk's is very complex. For a given cluster k (k = t, 2, 3 ...K , ), the log likelihood function needed to maximize is as follows-A log (P (x l k=Ek)) * P(klx.) [00046j'11 is now necessary to find the values of pk and F..k'sfor k =1, 2, 3 ...K that maximize the above function.

[00047) Most of the programming languages have the build-in functions to calculate the values of the parameters that maximize a given function. For example, in R, we can use a build-in fimction called "DIM" can be used to calculate the maximum likelihood estimates for -k'S 'S. the language of the EM steps is called a maximization step or M-step.
[00045] Step 4;

[000491 Using the maxirnuri likelihood estimates for ttk's , F..k's and P(k}'s as the new pi's, F.k's and F(k}'s, repeat Step 2 and Step 3 until the value of X no longer changes.
(00050] After the clustering process, the final values for gl s , E, s and P(k)'s have been obtained for all values of k. A probability matrix whose entries, are the final values of F (k I Qs have also been obtained Given a data point,the corresponding row in the probability matrix for this data point can be found Then, it, is possible to determine which cluster most likely comes from by looking at the values of P
(k I ys from this row. The column index which produces the largest value of P (k I xn) is the cluster where it below.

[00051) COMPARING MODELS BY BAYESIAN INFORMATION CRITERION (BIC) [00052] Suppose the parameters have been estimated for models with different number of components, the "best" . model is selected according to the Bayesian Information Criterion (BIC). The BIC score is defined as), - (1/2) * v' log{n), where X is the value of log likelihood function using the estimated parameters, v is the number of independent parameters of the model, and n is the number of observation. The selected model is that with the highest BIC score.

[00053]For example, if there are two BIC s corresponding to a new model and an old model,they are named BIC new and B1Cw. BJCõ (1/2) * V,,* log{N), and BICw = X (.112)" Vow * log(N). The new model is accepted if SICõ BICew, That is, (1/2) * VneW* log(N) > 7lm - (1/2) * Võ4 * log(N), which is the same as X, -1b, (1/2) * log{N) * (vnew` V.W) [00054]' Since BIC is an approximation, it is not 100% accurate. A variable is added to control the BIC.. A variable, a, is therefore introduced, such that the model is selected if I.. - Xw > * (1/2) * log (N) * (vdEw V. In theory, the value of a is 1, but by changing the value of a, the model selected can be controlled For example, if a is set to be relatively small, then there is a high probability that the new model will be selected.
If a is set to be relatively large, then there is a high probability that the old model will be selected.

[00055]. Standardize the Data Set [00056) One problem with the data set is that each variable may have different shape in terms of the student t-distribution. 'T'herefore, each variable must be 'standardized order to let it follow a standard student t-distribution. The steps for such standardization are as follows:

[00057] Consider all.data points coming from one cluster. Set initial value of 1 to be the mean of the data points, and set the initial diagonal entries of E to be the variances of each variable, E0005$1 Find' the maximum likelihood estimates for 1,6 and Em, Create a vector with the diagcmal entries of E. and call it varm [000591 For a given data point xn, standardize it by using {xõ . õI/ varm E00060] METHOD STEPS

[000611 Using these statistical methods explained above, the method steps according to aspects ' of the present invention is now constructed.. If the vehicles have a different number of clusters, they can not be in the- same cluster. Hence, the data set can be classified into groups according to the number 'Of the -axles. Each group can therefore be partitioned into small groups by grouping vehicles with the saute axle pattern {s, d, t, or q) together.
Then the method steps are run inside each small group 13.

1) Standardize said data in setsi 2) When said data is standardized in sets, start with k 1;
3) Set the initial value of p , to be the mean of the data set.

4) Set the initial diagonal entries of Zk to be the variances of each variable.
S) Set P(K) =I .

6) Run clustering with the EM algorithm in this cluster.

7) Obtain the new values for 11k , Zk= P(k) and the probability matrix P (k I
x,)

8) Define the BIC for this model as B=lCold= - (112) = V,,, = log{N)

9) Set k_p,*v . k and Repeat the following steps until k~,e k a) Set k~, = Ic and a new variable called trace =1.
b) Repeat the following steps until trace me _ka..
(i) Split the cluster at position trace into two clusters using PCA
(ii) Select data points to perform PCA., from the data points that are most likely come from cluster trace by checking the values in the probability matrix P (k I x ) (iii) Run clustering with the EM algorithm for this new model, (iv) Obtain pk's , E:k's and P(k}'s and the probability matrix P (k I
xõ )'s, and for the new model, 1'4 (v) ,Define the BIC for this new model as BICnew; -X... - (112) =
Vnew, log(N) (vi) If > a' (112) = (N) ' (Vnew- Vw), then replace the old model with the new model obtained in step (iii).

(vii) Set K-+ 1 (uiii)lf' " - X,,, is not > a= (4) * (N) = (vnew Vw), then keep the ot'iginal model.

(ix) Trace = trace + I
11) Finally report the final model;

thereby determining and recognizing the types of vehicles passing the checkpoint to determine and recognize vehicle types in high volume traffic for monitoring traffic volumes of various types of vehicles, forecasting future road maintenance costs and planning and design of future road networks, wherein, in said steps, N - number of data points.
V - number of variables.

K - number ofdu sters.

ilk = the meat, for kill cluster, each a vector of length V, fa k:- the covariance matrices for kth duster, each of size V ` V.
xn.= the nth data point, which is a vector with length V.

P(k l xn:) - the probability that xn comes from cluster k.

p(k) -: the probability that a data point chosen randomly comes from cluster k.
P(xn) - the probability of finding a data point at position xn X - the value of log likelihood of the estimated parameter set.
PCA Principal Component Analysis BIC = Bayesian Information Criterion C00062]The clustering result is obtained by checking the values ,in the final probability matrix.

[00063] Using the final model, any data points may be clustered , For example, if some data points are. given, they can be assigned to their corresponding clusters.
Using Step 2 to 8 as defined above with the EM algorithm method step a probability matrix can be obtained whose entries arc the values of P (k l xn) for all values of cluster k's and x.'sw From these values it is possible to determine which cluster that each data point most likely comes from by checking the values in the probability matrix.

[000641 If there is a vehicle that seldom appears, and it is desired to cluster it into a single cluster once it appears, this can be accomplished by adding a new cluster to the final model. The value of la for the new cluster is the same as the variable values for this vehicle. The covariance matrix E is a diagonal matrix. The diagonal entries of E is set to be very small numbers, namely the variances for each variable are small numbers. The values of P(k) for this cluster are, set to be a small number since this vehicle is very rare to appear.

Claims (11)

The invention claimed is:

1. A method for determining and recognizing types of vehicles passing a check point, which comprises:
up-loading an EM algorithm into a CPU;
collecting vehicle data as vehicles drive past a check point;
entering said data into said CPU said data being representative of essential characteristics of vehicles;
processing said data by said EM algorithm to produce an output model of the traffic volumes of the various types of vehicles; and utilizing said output model to forecast future road maintenance costs and/or to plan and design future road networks, wherein said EM algorithm is specially adapted to carry out the following steps:
1) standardize said data in sets;
2) when said data is standardized in sets, start with k 1;
3) set the initial value of pk to be the mean of the data set;
4) set the initial diagonal entries of a to be the variances of each variable;
5) set P(K)=-1;
6) run clustering with the EM algorithm in this cluster;
7) obtain the new values for µk, .SIGMA.k, (Pk) and the probability matrix P (k 1 xn);
8) define the BIC for this model as BICold=¨(1/2).cndot.Vold.cndot.log(N);
9) set k_prev=.k; and 10) repeat the following steps until k_prev=k;
a) set k_prev=k and a new variable called trace=1;
b) repeat the following steps until trace=k_prev;
(i) split the cluster at position trace into two clusters using PCA;
(ii) select data points to perform PCA from the data points that are most likely to come from cluster trace by checking the values in the probability marix P (k 1 xn);
(iii) run clustering with the EM algorithm for this new model;
(iv) obtain µk's, .SIGMA.k's and (Pk)'s and the probability matrix P (k 1 xn)'s, and for the new model;

(v) define the BIC for this new model as BICnew¨.lambda.new¨(1/2)*Vnew*log(N);
(vi) if .lambda.new¨.lambda.old>a.cndot.(1/2).cndot.(N).cndot.(vnew¨Vold), then replace the old model with the new model obtained in step (iii);
(vii) set K=+1;
(viii) if .lambda.new¨.lambda.old is not >a.cndot. (1/2).cndot.
(N).cndot.(vnew¨Vold), then keep the original model;
(ix) trace=trace+1; and 11) finally report the final model;
thereby determining and recognizing the types of vehicles passing the checkpoint to determine and recognize vehicle types in high volume traffic for monitoring traffic volumes of various types of vehicles, forecasting future road maintenance costs and planning and design of future road networks; wherein in said steps:
N=number of data points;
V=number of variables;
K=number of clusters;
µk=the mean for kill cluster, each a vector of length V;
f ° k:=the covariance matrices for kth cluster, each of size V*V;
xn.=the nth data point, which is a vector with length V;
P(k ! xn:)=the probability that xn comes from cluster k;
p(k)=: the probability that a data point chosen randomly comes from cluster k;
P(xn)=the probability of finding a data point at position xn;
.lambda.=the value of log likelihood of the estimated parameter set;
PCA=Principal Component Analysis; and BIC=Bayesian Information Criterion.

2. The method of claim 1, wherein said vehicle data comprises length of said vehicle, distance between axles of said vehicle, and weights on said axles of said vehicle.

3. The method of claim 1, including the additional step of obtaining the clustering results by checking the values in the final probability matrix.

4. The method of claim 1 including the additional step of using the final model, to cluster many data points where, if some data points are given, they can be assigned to their corresponding clusters, and obtaining a probability matrix whose entries are the values of P (k 1 x n) for all values of cluster k's and x n's, and from these values determining which cluster that each data point most likely comes from by checking the values in the probability matrix.

5. The method of claim 1, for a vehicle that seldom appears, and it is desired to cluster it into a single cluster once it appears, by adding a new cluster to the final model, where the value of µ for the new cluster is the same as the variable values for this vehicle, where the covariance matrix .SIGMA. is a diagonal matrix, and setting diagonal entries of .SIGMA. to be very small numbers, where the variances for each variable are small numbers so that the values of P(k) for this cluster are set to be a small number since this vehicle is very rare to appear.

6. A method of determining and recognizing the types of vehicles passing a checkpoint which comprises the steps of:
uploading a computer program into a CPU, said computer program comprising an EM
algorithm said EM algorithm including data representations of essential characteristics of vehicles collecting vehicle data as said vehicles drive past a checkpoint to determine and recognize vehicle types for monitoring traffic volumes of various types of vehicles, entering said data into said CPU; and deriving an output from said CPU, thereby monitoring traffic volumes of various types of vehicles for forecasting future road maintenance costs and planning and design of future road networks, wherein the EM algorithm algorithm is specially adapted to carry out the following steps:
1) standardize said data in sets;
2) when said data is standardized in sets, start with k 1;
3) set the initial value of µk to be the mean of the data set;
4) set the initial diagonal entries of .SIGMA.k to be the variances of each variable;

5) set P(K)=1;
6) run clustering with the EM algorithm in this cluster;
7) obtain the new values for µk, .SIGMA.k, (Pk) and the probability matrix P (k 1 xn);
8) define the BIC for this model as BICold=¨(1/2).cndot.Vold.cndot.log (N);
9) set k_prev=.k; and 10) repeat the following steps until k_prev=k;
a) set k_prev=k and a new variable called trace=1;
b) repeat the following steps until trace=k_prev;
(i) split the cluster at position trace into two clusters using PCA;
(ii) select data points to perform PCA from the data points that are most likely to come from cluster trace by checking the values in the probability marix P (k 1 xn);
(iii) run clustering with the EM algorithm for this new model;
(iv) obtain µk's, .SIGMA.k's and (Pk)'s and the probability matrix P (k 1 xn)'s, and for the new model;
(v) define the BIC for this new model as BICnew=-.lambda.new¨(1/2)*Vnew*log(N);
(vi) if .lambda.new¨.lambda.old>a.cndot.(1/2).cndot.(N).cndot.(vnew¨Vold), then replace the old model with the new model obtained in step (iii);
(vii) set K=+1;
(viii) if .lambda.new¨.lambda.old is not >a.cndot.(1/2).cndot.(N).cndot.(vnew¨Vold), then keep the original model;
(ix) trace=trace+1; and 11) finally report the final model;
thereby determining and recognizing the types of vehicles passing the checkpoint to determine and recognize vehicle types in high volume traffic for monitoring traffic volumes of various types of vehicles, forecasting future road maintenance costs and planning and design of future road networks;
wherein in said steps:
N=number of data points;
V=number of variables;
K=number of clusters;
µk=the mean for kill cluster, each a vector of length V;

f ° k:=the covariance matrices for kth cluster, each of size V*V;
xn.=the nth data point, which is a vector with length V;
P(k ! xn:)=the probability that xn comes from cluster k;
p(k)=: the probability that a data point chosen randomly comes from cluster k;
P(xn)=the probability of finding a data point at position xn;
.lambda.=the value of log likelihood of the estimated parameter set;
PCA=Principal Component Analysis; and BIC=Bayesian Information Criterion.

7. An apparatus comprising the combination of:
a CPU; and a computer program which has been uploaded into said CPU, said computer program comprising an EM algorithm, said EM algorithm including data representations of essential characteristics of vehicles, wherein the EM algorithm is specially adapted to carry out the following steps:
1) standardize said data in sets;
2) when said data is standardized in sets, start with k 1;
3) set the initial value of µk to be the mean of the data set;
4) set the initial diagonal entries of .SIGMA.k to be the variances of each variable;
5) set P(K)=1;
6) run clustering with the EM algorithm in this cluster;
7) obtain the new values for µk, .SIGMA.k, (Pk) and the probability matrix P (k 1 xn);
8) define the BIC for this model as BICold=¨(1/2).cndot.Vold-log(N);
9) set k_prev=.k; and 10) repeat the following steps until k_prev=k;
a) set k_prev=k and a new variable called trace=1;
b) repeat the following steps until trace=k_wev;
(i) split the cluster at position trace into two clusters using PCA;
(ii) select data points to perform PCA from the data points that are most likely to come from cluster trace by checking the values in the probability marix P (k 1 xn);
(iii) run clustering with the EM algorithm for this new model;

(iv) obtain µk's, .SIGMA.k's and (Pk)'s and the probability matrix P (k 1 xn)'s, and for the new model;
(v) define the BIC for this new model as BICnew=¨.lambda.new¨(1/2)*Vnew*log(N);
(vi) if .lambda.new¨.lambda.old>a.cndot.(1/2).cndot.(N).cndot.(vnew¨Vold), then replace the old model with the new model obtained in step (iii);
(vii) set K=+1;
(viii) if .lambda.new¨.lambda.old is not >a.cndot.(1/2).cndot.(N).cndot.(vnew¨Vold), then keep the original model;
(ix) trace=trace+1; and 11) finally report the final model;
thereby determining and recognizing the types of vehicles passing the checkpoint to determine and recognize vehicle types in high volume traffic for monitoring traffic volumes of various types of vehicles, forecasting future road maintenance costs and planning and design of future road networks; wherein in said steps:
N=number of data points;
V=number of variables;
K=number of clusters;
µk=the mean for kill cluster, each a vector of length V;
f ° k:=the covariance matrices for kth cluster, each of size V*V;
xn.=the nth data point, which is a vector with length V;
P(k ! xn:)=the probability that xn comes from cluster k;
p(k)=: the probability that a data point chosen randomly comes from cluster k;
P(xn)=the probability of finding a data point at position xn;
.lambda.=the value of log likelihood of the estimated parameter set;
PCA=Principal Component Analysis; and BIC=Bayesian Information Criterion.

8. The apparatus of claim 7 wherein said vehicle data comprises length of said vehicle, distance between axles of said vehicle and weights on said axles of said vehicle.

9. An apparatus for determining and recognizing types of vehicles passing a check point, which comprises:
a CPU;
an EM algorithm uploaded into said CPU;
structure operatively associated with said CPU for collecting vehicle data as vehicles drive past said check point;
means, operatively associated with said CPU for entering said data into said CPU said data being representative of essential characteristics of vehicles;
means for processing said data by said EM algorithm to produce an output model of the traffic volumes of the various types of vehicles; and means for utilizing said output model to forecast future road maintenance costs and/or to plan and design future road networks, wherein the EM algorithm is specially adapted to carry out the following steps:
1) standardize said data in sets;
2) when said data is standardized in sets, start with k 1;
3) set the initial value of µk to be the mean of the data set;
4) set the initial diagonal entries of .SIGMA.k a to be the variances of each variable;
5) set P(K)=1;
6) run clustering with the EM algorithm in this cluster;
7) obtain the new values for µk, .SIGMA.k, (Pk) and the probability matrix P (k 1 xn);
8) define the BIC for this model as BICold=-(1/2).cndot.Vold.cndot.log (N);
9) set k_prev=.cndot.k; and

10) repeat the following steps until k_prev=k;
a) set k_prev=k and a new variable called trace=1;
b) repeat the following steps until trace=k_prev;
(i) split the cluster at position trace into two clusters using PCA;
(ii) select data points to perform PCA from the data points that are most likely to come from cluster trace by checking the values in the probability marix P (k 1 xn);
(iii) run clustering with the EM algorithm for this new model;
(iv) obtain µk's, .SIGMA.k's and (Pk)'s and the probability matrix P (k 1 xn)'s, and for the new model;

(v) define the BIC for this new model as BICnew=¨.lambda.new¨(1/2).cndot.Vnew.cndot.log(N);
(vi) if .lambda.new¨.lambda.old>a.cndot.(1/2).cndot.(N).cndot.(vnew--Vold), then replace the old model with the new model obtained in step (iii);
(vii) set K+1;
(viii) if .lambda.new¨.lambda.old is not >a.cndot.(/2).cndot.(N).cndot.(vnew¨Vold), then keep the original model;
(ix) trace trace+1; and

11) finally report the final model;
thereby determining and recognizing the types of vehicles passing the checkpoint to determine and recognize vehicle types in high volume traffic for monitoring traffic volumes of various types of vehicles, forecasting future road maintenance costs and planning and design of future road networks; wherein in said steps:
N=number of data points;
V=number of variables;
K=number of clusters;
µk=the mean for kill cluster, each a vector of length V;
f ° k:=the covariance matrices for kth cluster, each of size V*V;
xn. =the nth data point, which is a vector with length V;
P(k ! xn:)=the probability that xn comes from cluster k;
p(k)=: the probability that a data point chosen randomly comes from cluster k;
P(xn)=the probability of finding a data point at position xn;
.lambda.=the value of log likelihood of the estimated parameter set;
PCA=Principal Component Analysis; and BIC=Bayesian Information Criterion.