Hadoop Storage in the Cloud Native Era

Thursday, May 23
11:00 AM - 11:40 AM
Marquis Salon 7

Hadoop was born much earlier than the Cloud Native era. But the question is still the same: what can it offer in the time of Kubernetes, containerization and hybrid clouds?

Apache Hadoop Ozone is a new subproject of Hadoop. It has a generic low-level binary layer, the Hadoop Distributed Data Storage (HDDS) and a S3 compatible Object Store implementation on top of it.

But the HDDS data storage layer is not just for the object store. It could be used for multiple purposes: to enhance the scalability the HDFS or provide block level access to the managed storage space. With this approach the same Hadoop Ozone cluster could provide hadoop file system based storage, object store space and block level storage.

Storage is still a hot topic with Kubernetes and in Cloud Native environments. Container Storage Interface specification is a vendor neutral standard to provide storage plugin for multiple container orchestration system.

Quadra provides block level access on top of the Hadoop Distributed Data Storage layer and it’s first class citizen of the containerized word. It implements the Container Storage Interface and can work as a Kubernetes dynamic volume provisioner.

In this talk we will demonstrate how the Hadoop Ozone storage could be used from containers. We will explain the basic storage type of Kubernetes clusters and show how Hadoop Ozone and Quadra could help to solve the storage problem in an industry standard way.

講演者

Nandakumar Vadivelu
Senior Software Engineer
Cloudera
Nandakumar Vadivelu is a Senior Software Engineer at Cloudera and an Apache Hadoop committer. He has been working on Hadoop for over 5 years and contributed to HDFS and Ozone. Before Cloudera, he worked at Ericsson, where he was part of the team which migrated telecom data warehouse form Oracle to Hadoop ecosystem.