Is this the age of Big OLAP?
There was a time when analytics was synonymous with #BI, and BI was synonymous with #OLAP — Online Analytical Processing. The term was meant to contrast with the more common Online Transaction Processing (OLTP), and involved the creation of multi-dimensional “cubes” rather than 2-dimensional tables. Each dimension is a different category with which to perform drill-down analyses on numerical data, known as measures. Now that you’ve had your BI 101 crash course, look around at numerous analytics products like Tableau, and you’ll see the paradigm of dimensions and measures is alive and well. OLAP never died, even if its underlying technologies morphed a bit. The justice of scale What has dogged OLAP, though, is its scalability. Most OLAP servers run on single, albeit beefy, servers, which limits the parallelism that can be achieved and therefore imposes de facto limits on data volumes. Customers who hit these scalability ceilings may contemplate using Big Data technologies, like Hadoop and Spark, but those tend not to employ the dimensional paradigm to which OLAP users are accustomed. What to do? Well, a few vendors have decided to take Hadoop and Spark, and leverage them as platforms on which big OLAP cubes can be run and built. The vendors, namely AtScale, Kyvos Insights and Arcadia Data, have looked at Big Data adoption patterns in some enterprises, and seen that momentum has stalled. Their approach has been to let people in those enterprises work in the OLAP environments they are comfortable with and, at the same time, make use of their #Hadoop clusters. AtScale’s CEO, David Mariani, was the person behind a couple of the biggest #Microsoft OLAP projects (at #Yahoo and #Klout). And while he was able to do big things with Microsoft’s OLAP platform, SQL Server Analysis Services (SSAS), he definitely hit up against scalability limits, including cube re-processing times of as much as a week. His motivation for creating a more resilient OLAP platform was pretty clear, so he formed #AtScale.