Apache Hadoop YARN: state of the union

Wednesday, February 6
11:50 AM - 12:30 PM
Room 101/102

Apache Hadoop YARN is the modern distributed operating system for big data applications. It morphed the Hadoop compute layer to be a common resource management platform that can host a wide variety of applications. Many organizations leverage YARN in building their applications on top of Hadoop without themselves repeatedly worrying about resource management, isolation, multi-tenancy issues, etc.
In this talk, we’ll start with the current status of Apache Hadoop YARN—how it is used today in deployments large and small. We'll then move on to the exciting present and future of YARN—features that are further strengthening YARN as the first class resource management platform for data centers running enterprise Hadoop.
We’ll discuss the current status as well as the future promise of features and initiatives like: powerful container placement, global scheduling, support for machine learning and deep learning workloads through GPU and FPGA support, extreme scale with YARN federation, containerized apps on YARN, support for long-running services (alongside applications) natively without any changes, seamless application upgrades, powerful scheduling features like application priorities, intra-queue preemption across applications, and operational enhancements including insights through Timeline Service V2, a new web UI, and better queue management.

講演者

サンジェイ・ラディア
チーフアーキテクト
創立者
Hortonworks, Inc.
サンジェイは、Hortonworksの設立者、チーフアーキテクトであり、Apache Hadoop コミッタ、およびApache Hadoop PMCのメンバーでもあります。Hortonworks 共同設立に先立ち、サンジェイはYahooのコアHadoopのチーフアーキテクトとしてHadoopの設計に携わりました。Hadoopでは、HDFS、MapReduceスケジューラー、YARNの設計、高可用性、互換性など様々な分野に貢献しています。また、Sun Microsystems と INRIA ではシニアエンジニアの役職を果たし、分散システム、グリッド/ユーティリティ·コンピューティング·インフラストラクチャのためのソフトウェアを開発しました。サンジェイは、カナダの University of Waterloo にてコンピュータ科学の博士号を取得しています。