Four years ago we started with a data analytics platform to learn more about how our customers use our on-premise software and how it behaves out in the fields regarding function usage, exception rates and overall performance.
The talk is about the journey we had to take, coming from an existing web apps statistics tracking system to our current and still evolving Hadoop based ETL system. This includes the current technologies we use and the approcache on how we support reporting and dashboards.
This new Hadoop platform is used to collect, transform, enrich with data warehouse data, and analyze millions of log files every day. The generated insights help us to make data driven decisions for portfolio management, UX-Design, and overall software quality improvements with real business value.
You will hear about the Dos and Donts we learned, what we think are best practices, and the new challenges we have to deal with while data volume and management awareness is still emerging.