Deep Learning with DL4J on Apache Spark: Yeah it’s Cool, but are You Doing it the Right Way?

Deep Learning with DL4J on Apache Spark: Yeah it's Cool, but are You Doing it the Right Way?

Thursday, March 21
2:50 PM - 3:30 PM
Room 118-119

DeepLearning4J (DL4J) is a powerful Open Source distributed framework that brings Deep Learning to the JVM (it can serve as a DIY tool for Java, Scala, Clojure and Kotlin programmers). It can be used on distributed GPUs and CPUs. It is integrated with Hadoop and Apache Spark. ND4J is a Open Source, distributed and GPU-enabled library that brings the intuitive scientific computing tools of the Python community to the JVM. Training neural network models using DL4J, ND4J and Spark is a powerful combination, but the overall cluster configuration can present some unespected issues that can compromise performances and nullify the benefits of well written code and good model design. In this talk I will walk through some of those problems and will present some best practices to prevent them. The presented use cases will refer to DL4J and ND4J on different Spark deployment modes (standalone, YARN, Kubernetes). The reference programming language for any code example would be Scala, but no preliminary Scala knowledge is mandatory in order to better understanding the presented topics.

Presentation Video

講演者

Guglielmo Iozzia
Regional (EMEA) Associate Director
MSD Biotech
I am currently Regional (EMEA) Associate Director at MSD Biotech and was previously at Optum (UnitedHealth Group) and am based in Dublin, Ireland. My teams and I deal with projects in the PI (fraud, waste and abuse, claims processing) and the healthcare space. I worked previously at IBM Ireland, where I switched my career path from Test Automation to Analytics and Machine Learning. I am passionate about coding, Big Data, AI/ML/DL, test automation, Open Source, DevOps and cooking (homemade pizza is my speciality!). I share my tech thoughts via my blog (http://googlielmo.blogspot.ie/) and DZone (https://dzone.com/users/2532948/virtualramblas.html) where I am a Golden Member. During 2018 I have presented at several international conferences such as DataWorks Summit Berlin, Google I/O Extended, Predictive Analytics World for Industry 4.0 and many others. My first book "Hands-on Deep Learning with Apache Spark" (https://tinyurl.com/y7d98s64) is going to be released in December 2018.