Apache Flume
An integrated part of CDH that is supported with Cloudera Enterprise, Flume lets you reliably stream log and event data into Apache Hadoop.
Flume Key Features
Efficiently Stream Data:
Easily collect, aggregate, and move streaming log or event data from multiple sources into Hadoop. As a critical part of building complete stream processing pipelines, Flume is designed to ingest this data as it is generated for near real-time analytics — making it ideal for sensor data aggregation or “Internet of Things” use cases.
Built for Hadoop Scale:
As streaming data grows, you can simply scale horizontally to handle the increased load. You can also extend to many data sources to efficiently gather logs from multiple systems or sensors, and connectors are available to stream data into multiple systems.
Always-on Reliability:
Protect against data loss and ensure that streaming data will continue to be delivered, even in the event of failure, with fault tolerance built into the core and tunable reliability to best fit your needs.
Common Use Cases
As the standard tool for streaming log and event data into Hadoop, Flume is a critical component for building end-to-end streaming workloads, with typical use cases including:
Fraud detection
Internet of Things applications
Aggregation of sensor and machine data
- Alerting/SIEM
Integrated across the platform
As an integrated part of Cloudera’s platform, Flume can easily work with other components, such as Apache Kafka and Spark Streaming, to build complete streaming workloads within a single platform. It also benefits from unified resource management (through YARN), simple deployment and administration (through Cloudera Manager), and shared compliance-ready security and governance (through Apache Sentry and Cloudera Navigator) — all critical for running in production.
Cloudera’s commitment to Flume
Cloudera, the original developer of Flume, is actively involved with the Flume community, with committers on-staff to continue to drive innovations. As a deeply integrated part of the platform, Cloudera has built in critical production-ready capabilities, especially around reliability and Apache Kafka integration, helping to solidify Flume’s place as an open standard for real-time streaming in Hadoop.
Cloudera’s engineering expertise, combined with support experience with large-scale production customers, means you get direct access and influence to the roadmap based on your needs and use cases.
Partnered with the ecosystem
Seamlessly integrate with the tools your business already uses by leveraging Cloudera’s 1,700+ partner ecosystem. With a robust partner certification program, we are continuously working to build out production-hardened integrations between Flume and the most popular third-party tools and platform components.
Expert support for Flume
Trained by its creators, Cloudera has Flume experts available across the globe to deliver world-class support 24/7. With more experience across more production customers, for more use cases, Cloudera is the leader in Flume support so you can focus on results.
