Ayres stressed the importance of organizations performing regressions and randomized tests on their data to uncover business trends and winning techniques. Although these methods have been around for centuries, they’re increasingly essential to gain insight from the “monstrously large” databases of customer and business information that large organizations maintain today. “The speed (of analysis) and scale of the impacts (that this data) can have is what is revolutionary,” Ayres said.

He offered several examples of businesses that are using this type of “super crunching” to differentiate their businesses, including Epagogix, a company that estimates the box office receipts of movies before they’ve even hit the theaters, based on the performance of movies with similar themes, actors and other traits.

Regression analysis is vital for uncovering the relationships between variables that often may otherwise not be easily uncovered in large data sets. But, Ayres said, analysts need to also focus on the precision of the predictions garnered through their analysis. Often the size of the sample on which a prediction or score are made will determine its precision – or the likelihood that it is correct.

He stressed the importance of doing randomized trials of the predictions: “Any organization that doesn’t use regression and randomization is screwing up.” Randomized tests of multiple theories – whether investing in stocks or choosing the most effective subject line for direct mail campaigns -- make it possible to optimize results in real time and applies the learnings ongoing.

To leave comments, please login or sign up to the Engage Network at the top of this page.