Why SlamData?

The Complex Data problem

Over the last decade the explosion of SaaS, IoT and Mobile apps has driven increasingly complex application data. As developers imagined ever more powerful ways to capture and store data for these new applications, the challenges of preparing data for analytics has grown increasingly difficult. Highly flexible data models like JSON, assisted by the popular adoption of JSON first NoSQL Databases has made creating and capturing complex data cheap and easy. The JSON data model is very popular if not the norm for most modern SaaS, IoT and Mobile apps. The attraction of JSON is the tremendous flexibility it allows developers when designing and implementing an application. It also allows them to store data in far more complex models than a traditional RDBMS would support and to quickly change and update schemas as needed to support rapidly changing applications. However all this flexibility comes at a price, and this becomes evident when users need to perform analytics on all this complex data.

The ETL and Analytics Challenge

Modern applications generate huge volumes of data, whether its user event data or IoT sensor data the sheer volume of data is daunting. Add to this the complex ways many of these applications capture and store this data and you have a serious challenge when it comes to analytics. All companies need to perform analysis on the data they collect in order to improve their product or service. While there are many tools available to transform data including traditional ETL (Extract, Load and Transform) that let data engineers transform and load data into a data warehouse, or more recently some of the new generation of data preparation tools designed for end users,  none of them can handle the challenges presented by highly complex JSON data. To solve this job companies revert to manually writing code in order to deal with the complex and changing nature of the data. This makes the process slow, tedious and costly. It also makes it inaccessible to non-engineers. Custom parsers that require frequent updates is just one example of the way companies address this issue today, but sadly end users, analysts and the majority of Data Scientists aren’t able to write custom parsers or any other kind of code to solve this data challenge.

Let’s look at an example of the kinds of data commonly found in today’s SaaS ecosystem. Following is an example of data for an application called Post Giraffe that sends email similar to MailChimp. Each JSON object represents a sent email so you can imagine the huge volumes of data generated with hundreds of millions of emails being sent.

Post Giraffe data sample

[ { “c45355fb-98c4-4a59-b408-f74f2a44bd2d”: {
“dateTime”: “2018-02-14T16:18:23.534z”,
“S”: {
“rId”: “1db1a6cf-5aee-4fc8-9b9f-9bd8f48658eb”,
“to”: [
[email protected]”,
[email protected]
],
“cc”: “”,
“bcc”: “”,
“subject”: “Congratulations”,
“body”: “…”
}
}
}
, { “c45355fb-98c4-4a59-b408-f74f2a44bd2d”: {
“dateTime”: 1530548859,
“S”: {
“rId”: “1db1a6cf-5aee-4fc8-9b9f-9bd8f48658eb”,
“to”: “[email protected]”,
“cc”: “”,
“bcc”: “”,
“subject”: “Welcome!”,
“body”: “…”
}
}
}
, {
“5573e6c6-0024-4cc8-a287-9cc39ad3f801”: {
“dateTime”: 1530548938,
“MAS”: {
“rId”: “1db1a6cf-5aee-4fc8-9b9f-9bd8f48658eb”,
“sId”: “c45355fb-98c4-4a59-b408-f74f2a44bd2d”
}
}
}
, {
“ef978ddf-80bd-40f5-9422-c9cda48acacf”: {
“dateTime”: 1530548953,
“MAS”: {
“rId”: “1db1a6cf-5aee-4fc8-9b9f-9bd8f48658eb”,
“sId”: “c45355fb-98c4-4a59-b408-f74f2a44bd2d”
},
“S”: {
“rId”: “1db1a6cf-5aee-4fc8-9b9f-9bd8f48658eb”,
“to”: “[email protected]”,
“cc”: “”,
“bcc”: “”,
“subject”: “Thank you for registering”,
“body”: “…”
}
}
}
]

This data presents a number of challenges for anyone needing to perform analytics. For example, datetime is stored as ISO standard text format, and as number of seconds since 1970. Next, the record ID and the type of record are stored as keys rather than values, the dateTime and the type of record are stored in the same object  which is the value of the ID key which changes from record to record. While complex, this type of data is increasingly common for today’s SaaS, mobile and IoT apps among others. Existing ETL and data prep tools cannot handle this kind of data and the only other alternative for making this data analytics ready is writing code such as Python or Scala which makes it all but inaccessible to anyone who can’t write code. Analysts, Data Scientists and business users will need to wait for Data engineers to deliver the data and then work thru many iterations.

A Unique Solution

SlamData VDW is delivering a new vision for working with this kind of complex data. Based on a novel but powerful underlying algebra call Multi-dimensional Relational Algebra (MRA) we are able to natively understand the complex and changing nature of the data and provide any user the mean to browse and build analytic ready tables. When discussing this challenge with hundreds of users across many companies two common themes emerged. First, existing tools did not work as data complexity increased. Some tools could handle very simple JSON data, but once the data became more complex they failed, and ultimately users had to revert to the data engineer and coding solution. Second, existing tools were far too complex to use and required users to have a deep understanding of the data and structure ahead of time, which defeats the purpose for most end users.

“Users want a tool that is easy to use, and can handle any data regardless of complexity.”

So we set out to build a solution that solves these two massive problems. The result is a powerful new tool that natively understands the most complex JSON you can throw at it, and provides a user experience that is as easy to use as a file browser. Now virtually any user can access complex JSON stored in S3, Azure, Wasabi or just about anywhere and turn this into insightful analytic ready tables easily. And they can edit and iterate on these tables quickly and easily as they discover what they need.

Summary

You may be asking yourself, why has SlamData succeeded where so many others have failed? It’s actually simple yet complex. We stopped assuming one size fits all, and approaches that rely on the same Relational Algebra will always fail. We realized that modern JSON data structures needed something more powerful, so that’s what we built.  

As the saying goes, “show me”. We have encountered lots of tools claiming they can handle complex JSON, but the reality is far different. We have tried dozens of ETL tools, Data prep tools and even some purpose built solutions like Amazon Athena, and while each can handle some aspects of complex JSON, as data complexity increased they either failed or required ongoing user interventions and updates to handle the data. In this regard SlamData is unique, we can seamlessly handle the data and simultaneously provide an inspiring user experience. The days of waiting on data integration teams to deliver curated data, and then repeating the cycle each time new data is needed are over. Now end users can curate data in minutes, and iterate just as quickly.

Why SlamData?

  • Advanced Mathematical foundation handles more complex data
  • Fast streaming data engine
  • Inspiring easy to learn and use interface
  • Removes the barrier between user needs and integration efforts
  • Simple to install and run
  • SlamData VDW succeeds where others fail

Send Us A Message

Get In Touch

1919 14th St Suite 700
Boulder, CO 80302

Our Investors

© 2018 SlamData, Inc.