BI Must Be Agile
A lot of people are firm believers in following a strict waterfall model when starting any type of IT project. This methodology definitely works in many situations, but your data, analytics, and business intelligence projects need to follow an Agile methodology in order to be successful.
Traditionally a data warehouse project that follows the waterfall model could take at least six months to complete (or even more). The problem with these large and complex projects is that business users do not always have a good idea of what they want, or even need. This makes requirements gathering sessions long and drawn out and you end up with a lot of change orders that need to be managed along the way. Month seven flies by. Then month 13, and, well you get the idea.
You might ask, “do I even need a data warehouse?” My answer is yes, but I will save the details for another post.
There are quite a few Modern Business Intelligence tools out on the market today that do not require you to have a data warehouse in place in order to start exploring data.
Enter Agile methodology.
These modern tools give you the flexibility to access data from many different systems and in many different formats. You can do some quick data wrangling and get your initial results in front of your data consumer in days instead of months. And you get feedback faster. Perhaps this is not quite the data they need, or thought they needed. Now that they’ve had a chance to review, adjustments can be made earlier in the process. Now you’ve saved yourself, your stakeholders, and your developers, a lot of time.
BI projects are also not your typical start and end projects. As more and more data is analyzed, new questions and new insights emerge. Analytics will be ever evolving. Once you get used to the idea that you will not have one large project that starts and ends, it will be easier for you to run short, iterative projects.
It does not need to be perfect the first time around also. Limit the scope so that you can fail fast, but get your analysts what they need faster. Management will be happy with faster benefits realization and your analysts will be happy digging through the data sooner than they originally thought was possible.
