The importance data access governance is continuously growing due to new regulations, such as GDPR, and industry policies. In this talk, we will share learnings at Svenska Spel from implementing and supporting this growing demand. Our journey started a year ago implementing Atlas and Ranger in our Hortonworks Data Platform solution. From start we wanted to implement a process and solution that had:
Minimal impact on ETL-developers and analysts
Privacy and security by design
Low maintenance burden using automatization
To be able to manage our policies and metadata we developed and open sourced cobra-policytool. Cobra-policytool integrates into our deployment pipeline and simplifies the way we express our policies. Thanks to the tool we can use Git as our truth of source for our policies and metadata information, along with the source code. It is close to seamless for our ETL-developers and with trust deploy the right policies.
Throughout the course of our journey, we learned the importance of having company policies and technology walk in tandem. Single source of truth, open platforms, and automatization are key to success. Data governance and metadata management are complex areas requiring high attention and are nothing you can duct tape in the end or solve buying a new power tool. With the right approach, it can be rewarding and make new requirements easy to implement and audit.