Chapter 9. Reporting and Testing – Iterating on Analytic Systems
In previous chapters we have considered many components of an analytical application, from the input data set to the choice of algorithm and tuning parameters, and even illustrated a potential deployment strategy using a web server. In this process, we considered parameters such as scalability, interpretability, and flexibility in making our applications robust to both later refinements of an algorithm and changing requirements of scale. However, these sorts of details miss the most important element of this application: your business partners who hope to derive insight from the model and the continuing needs of the organization. What metrics should we gather on the performance of a model to make the case for its impact? How can we iterate on an initial model to optimize its use for a business application? How can these results be articulated to stakeholders? These sorts of questions are key in conveying the benefit of building...