A case for data science

Deriving insight from data has become the mantra of many public and private companies. But how does a company harness the power and potential of its customer data and really make it the workhorse of its operation?

This was the challenge faced by a transportation provider. Several years ago, the company had implemented a model to predict revenue and passenger traffic, but only a few years later, the system started to show signs of underperforming. Its reliance on multiple technologies, some of which were proprietary, made the system slow, cumbersome and expensive to operate. And the system’s predictive model didn’t deliver useful results.

After considerable deliberation, the company decided to scratch its existing application as well as its model and started a search for a firm skilled in statistical analysis, model development and manipulation, data management and the use of computer technologies–skills that have been more broadly dubbed as data science. The company’s search ended with discussions with Milliman consultants.

The process

The Milliman team was asked to think of a new way to predict the company’s traffic and revenue using open source technology. While considerably less expensive to operate, the software is typically used for decision support rather than as a real-time predictive tool as the client intended to use it. Relying on its actuarial background, the team suggested a number of ways to optimise the software for everyday use.

The end product also had to be more reliable than its predecessor and easy for different levels of management to use. And because modelling results were needed at the start of every day, computation time couldn’t be more than a few hours.

These client requirements led the team to use a machine-learning model that relies on data pattern recognition to predict the company’s traffic flow and revenue. Like more traditional actuarial models, the one proposed for the client was challenged under testing in both a proof of concept and development, but it was designed so that it could be used without re-calibrating it periodically.

The third facet of the project called for the team to design and integrate a system to the specifications of the client’s current data warehouse using an open source technology as much as possible and without installing any new technology.

The result

Working from a mock-up drafted by the client, the team reproduced the dashboard to the client’s specifications, but it is now supported by newly developed software as well as the client’s data warehouse. The dashboard allows the client’s management team to quire different aspects of passenger usage to gain insight into traffic flows and revenue. Colour-coded symbols, which when clicked on, give managers a concise picture of a train’s revenue and traffic. Managers can also query the system based on selected features for both past usage and anticipated ridership, and are now able to make more informed decisions about pricing, the need for discounts or adjustments to marketing campaigns.

Because the model can adapt to new situations, deviations from the average error are confined to a much more narrow range. This means managers can have more confidence in the model’s predictive value and increases their ability to manage revenue.