The Evolution of Regression Modeling – Part 1 Review

Regression Modeling

Regression Modeling can be quite confusing if it’s your first time at the rodeo. The history of Regression Modeling is fascinating and its applications continue to grow. We had the pleasure of listening to a webinar put on by Salford Systems titled The Evolution of Regression Modeling. It was the first of a four part series. The first webinar was so fantastic that we think everyone should attend. For those of you that missed the first webinar or who won’t be able to attend the rest of the series, we’ll be sharing our takeaways with all of you. Let’s take a look at the highlights from the first webinar.

The World Is Not Linear

Regression techniques began by fitting a straight line to a set of data. The earliest form of regression was the method of Ordinary Least Squares (OLS), which is used for estimating the unknown parameters in a linear regression model. The next phase of the Regression Modeling evolution happened in 2008 when Jerome Friedman introduced Generalized Path Seeker (GPS). GPS proved to be more accurate than OLS.

Although Jerome Friedman’s advancements improved the accuracy of regression models, they still didn’t fit the data perfectly. The fact of the matter is that most linear regressions don’t fit the data to its optimum, because it’s a linear regression! Mikhail Golovnya, Senior Scientist at Salford Systems, said it best when he said “A plane is a plane, a circle is a circle. There’s nothing you can do to make a plane a circle”. In other words, stop trying to fit things into things that don’t fit. Linear regression techniques continue to evolve and improve but are limited to spaces where the data is linear.

Non-Linear Regression Is The Future

Data driven, localized, non-linear regression more accurately fits the data set by building locally linear models nicely connected to each other at the boundary points called “knots”. Non-linear regression has a lower MSE and more accurately fits the curve of the data. In other words, non-linear regression more naturally fits the data and consequently, can deliver more accurate results.

If you’re going to take one thing from this webinar, understand that the world is not linear, so we shouldn’t try to fit our data into linear models. Businesses have historically used linear models because they were “do-able” with available technology and people understood how to build them. The good news is there are far better methods and tools available now.

We hope you get the opportunity to listen to the rest of the webinar series. If you did listen to the first webinar, what other takeaways did you have?