I am by no means an actuarial and wouldn't want to be one. We do use one on a semi-regular basis as we adjust our risk models for investment evaluation. They are absolutely worth every single penny they cost.
The one we use also has an actual personality which is rare in that profession. My interest and curiosity in the FICO as a risk assessment model was sparked by conversations over adult beverages with the actuarial we use on a contract basis.
As we first talked about it, I was struck by the absolute fact the model is totally devoid of any attempt at addressing human behavior. Hence, my point about it being flawed because the data it relies upon is not reliable and easily gamed.
Some of the other things our actuarial mentioned about the FICO model is it fails to look at how the borrower evaluates and adjusts to risk. One big area is in the down payment.
Sure, intuitively, everyone knows the more a borrower puts down the less likely they are to default. But, the models FICO built for all lenders never took that into account even in a subjective manner.
In other words, a more accurate model would adjust the FICO based on the proposed down payment. Put 20% down and your score goes up x points, for example.
Another big area is as I mentioned before, since the FICO became all important in the financial lives of borrowers, some figured out ways to alter the data to provide an inaccurate picture of their financial history. That is what I meant by "gaming the system".
It also fails to address any macro economic issues like the health of the local economy.
The list of short comings is long and no doubt some will be addressed over time. But, FICO suffers the ultimate same failing as any other statistical gating model. It cannot overcome chaotic events and behavior.
Granted, many of the things it does not address could not be addressed easily or quickly, like economic conditions. But, that does not negate the problems with the model.