DataOps: The Latest Buzzword

I saw a recent online presentation on DataOps. I’ll be honest; the title did catch my attention. What’s this cool new data thing?

It’s not new. In fact I’ve been working this way for almost 10 years now. And if you haven’t, you need to.

So what is DataOps?

It’s essentially the new word (taken from software development: DevOps) for managing your data and analytics projects in an agile fashion. It’s no secret that I am a big fan of working within an agile framework when it comes to all things data, analytics, and business intelligence.

But what does agile mean?

When I talk about agile methodology, I’m really talking about an iterative development process. Start with a limited scope, work closely with stakeholders during development, and deploy the first version. The first version is just that—the first step. The intention is to continually improve upon the end product, whether it’s a software application, data pipeline, data warehouse, or business intelligence application.

There are lots of different styles of agile development, including SCRUM, Kanban, and others. Some have very strict rules and time periods using sprints and daily stand up meetings, while others focus more on prioritization and simply getting things done. I will adjust the style according to the needs of the project. If there is a strict deadline, perhaps SCRUM is the way to go. If you have a long list of to-dos with no specific timeline, Kanban helps you prioritize the work and get more done by concentrating on a limited number of tasks at a time.

Why is it better?

The thing with data, especially within the last few years, is that if you wait too long to analyze it, it may no longer be relevant. Or perhaps the business couldn’t wait for the data and instead made a decision based on incomplete information, or even worse, gut feeling. Getting data faster and working closely with the data consumer benefits everyone. The developer spends less time going down a potentially incorrect path, and the data consumer has a better understanding of the end product.

Let’s get back to DataOps. It is a rebranding of a project management methodology that has been around for some time now. If you’re already operating this way—fantastic! If you are not and you need to sell the concept up the ladder, then by all means start spreading around the concept. “Hey, I’ve heard of this new thing called DataOps and I really think it would benefit the company.” Or perhaps something like, “Our data consumers are just not getting what they need fast enough. We really need a DataOps team.”

I haven’t decided if I’m personally going to propagate this new buzz word or not. I suppose it will be a case by case decision. Or maybe it will go the way of Decision Support Systems. Either way, the concept is solid and proven and you need to be doing it.