Apache NiFi is an open source tool for automating and managing the flow of data between systems (Databases, Sensors, Data Lakes, Data Platforms). It is a real-time integrated data logistics and simple event processing platform that automates the movement of data between disparate data sources and systems, making data ingestion fast, easy and secure. It provides real-time control that makes it easy to manage the movement of data between any source and any destination. It is data source agnostic, supporting disparate and distributed sources of differing formats, schemas, protocols, speeds, and sizes such as machines, geolocation devices, click streams, files, social feeds, log files and videos and more. It is configurable plumbing for moving data around, similar to how FedEx, UPS or other courier delivery services move parcels around. And just like those services, Apache NiFi allows you to trace your data in real time, just like you could trace a delivery.
Apache NiFi has been evolving over years as a "Dataflow Orchestration Tool". A dataflow consists of different Processors, small modules specialized to execute certain tasks. Now NiFi contains more than 300 Processors. Join Koji Kawamura, Senior Software Engineer with Hortonworks as he leads a discussion describing the core data flow patterns based on common NiFi use-cases.