Machine Learning: We're doing it wrong

I mentioned that this book started in response to a CRO's question. "We've done all the ML stuff and it hasn't moved the needle. What are we doing wrong?" I answered, "You're just building a fancier 1960s score. You have not integrated with cash flows, yield, and pricing."

But how do you do that? In fact, we have to revisit the ML scores. 99% are being built wrong, because the lending industry kept the original framework of building rank-order scores. Everything in our world requires probabilities. We can do this.

Panel data methods are the answer, and they can be implemented with neural nets and stochastic gradient boosted *regression* trees (SGBRT). For neural nets, the trick is to provide the APC lifecycle and environment as inputs, without coefficients, feeding all the way to the final forecast node. Parallel to these you build your NN however you have been most successful so far.

For SGBRT, the R is essential. Regression tree packages like LightGBM allow the specification of a fixed input, just like logistic regression. Provide the APC lifecycle and environment as the fixed input, and presto-changeo, your ML is now an ML *cash flow* model.

Honestly, I think this is awesomely cool. Why waste time on an ML rank order score when your ML model can predict the monthly probabilities of default and prepayment. You immediately can use your one ML cash flow model for IFRS 9 / CECL, stress testing, yield forecasting, and pricing optimization -- not to mention origination scoring as usual. The very best part of all, this approach solves the "overfitting" problem that I described in my previous post. When we build ML models this way, the out-of-time degradation in Gini is no worse than a logistic regression. We don't need to rebuild these models for years. The quarterly refresh cycle for so many ML models just highlights how broken they are. That is not necessary when using ML.

The academic article is here: https://lnkd.in/gXVmxFw6

The concepts are here: https://lnkd.in/gjPPDWYU

Build great models!

Joseph Breeden
Posted on LinkedIn