Key Elements of In-Memory Analysis

I finally found some time to review this white paper on in-memory analysis – a technology that allows for shorter seek time when querying data. The article was originally posted by BeyeNetwork, and it lends some pretty good insight into the current market. In addition, the paper mentions several vendors in the space who are fighting to achieve dominance in the near future. I think it’s certainly worthwhile to read through, despite being 45 pages.

Our product  — miVEDiX — is serious about its architecture, and the research from this study does a good job of highlighting the importance of back-end data architectures. It very clearly acknowledges that the success and scalability of  the BI tools “of the future” will have a very strong focus on the back-end, and not just simple visualization layers.  At least half of the work is devoted to the technical aspects of back-end architectures, query optimization, etc.  Author Wayne Eckerson states the issue plainly: “[Y]ou can’t discuss next-generation BI tools without examining their back-end data architectures.”

miVEDiX also aims to meet the needs of those that fall into different user groups. The research Eckerson draws from does a good job at breaking down who BI users are and I agree with their complicated, yet eventually simple, analysis.

Here’s a good breakdown:

“There are two types of self-service BI, they must support: one for information consumers and another for information producers. An information consumer uses information created by an information producer. Simple enough. But an information consumer isn’t equivalent to a casual user, and an information producer doesn’t equate to a power user. Let me explain.

An information consumer is anyone who consumes information. This can be a casual user or a power user. An information producer is anyone who produces information. This is typically a power user or IT professional but can occasionally be a casual user. For example, casual users consume information 80% of the time and produce it 20% of the time (or at least try to produce it until they give up and ask for help from a superuser or IT professional). The reverse is true for power users. They produce information 80% of the time and consume it 20% of the time. (Ideally, power users should do less producing and more consuming or analyzing.) IT professionals are almost entirely information producers”

Like the miVEDiX roadmap, this article highlights many functional areas. One that caught my eye was the power and importance of collaboration:

“Beyond publishing, casual users need BI tools that enable them to associate comments with an entire dashboard, individual charts or tables or even cells within a chart or table. Ideally, the comments form an ongoing discussion among team members about what’s happened and what to do about it. The discussion culminates when team members evaluate the effectiveness of the decision or action taken by the team.”

These comments refer to traditional web-based applications but the same should be true for mobile BI, especially those like miVEDiX, which skip the middle layer and go directly to the source. Collaboration will be an important element for us in the future if we intend to eliminate a user’s dependence on laptops.

Here’s another good quote:

“Next-generation BI tools offer strong administrative tools to track usage and optimize performance, queries and other facets of a BI environment. They show which tables and fields are used most, make suggestions for adding aggregate tables and indexes and modifying schema. They also provide query governors that prevent runaway queries from bogging down performance for everyone, among other things. These tools also interoperate with third-party security frameworks, like Lightweight Directory Access Protocol and Active Directory, to manage users and permissions. “

And…  in the spirit of trying to keep this short… a couple other things that caught my eye:

• A lot of content focused on next-generation BI architectures.  Several comments point in the direction of BI vendors adopting in-memory solutions for storing and processing, and also mid-tier intelligent caching on the BI Server (including a shout-out to our partner, arcplan).
• Other discussions on next-generation architectures include things like MPP, columnar storage, and adding more functions at the database level that used to occur at BI level.
• Lastly, the paper discusses two very important areas:  cloud and unstructured data (two items that are highly relevant to some of the world’s largest companies).