What you'll learn:
Learn designing an end-to-end Real-time Streaming pipeline for Big Data using latest technologies.
Understand the different components in Big Data streaming pipeline.
Use Kafka as the connecting tool between ETL components in the real-time streaming pipeline.
Use Apache Flink, Spark Streaming and Kafka Streams to perform different transformations and aggregations.
Use Druid and Pinot as OLAP technologies in the streaming pipeline.
Use Superset to visualize the real-time incoming data stream to explore and visualize the transformed data.
Hands-on Practicals helping you build all the components and forming a complete end-to-end pipeline.
Learn multiple technologies used in Real-time Streaming pipelines, and you can use the one that better suits your use-case.

Requirements
An exposure to Big Data world will help you better appreciate Real-time Streaming pipelines, but is completely optional.
Basic knowledge of Java and Scala will be helpful, but not mandatory

Description
Getting real-time insights from huge volumes of data is very important for a majority of companies today.

Big data Real-time streaming is used by some of the biggest companies in the world like e-commerce companies, Video streaming companies, Banks, Ride-hailing companies, etc.

Knowing about the concepts of realtime streaming and the various realtime streaming technologies will be a great addition to your skillset and will enable you to build some of the most cutting-edge solutions that exist today.

We have created this Hands-On Course so that you get a good understanding about how realtime streaming systems can be built

This course will ensure that you get a hands-on experience with Apache Kafka, Apache Flink, Spark Streaming, Kafka Streams, Apache Pinot, Apache Druid, and Apache Superset.

This course covers the following topics

An Introduction to Kafka with hands-on Kafka setup

Understanding basic transformations and aggregations which can be done in a real time system

Learn how transformations and aggregations can be done using Apache Flink with hands-on coding exercises

Learn how transformations and aggregations can be done using Spark streaming with hands-on coding exercises

Learn how Kafka streams can be used to perform transformations and aggregations with hands-on coding exercises

Ingest data into Apache Pinot which is an OLAP technology

Ingest data into Apache Druid which is also an OLAP technology

Using Apache Superset to create some insightful dashboards

If you are interested in learning how all these technologies can be connected together to build an end to end real-time streaming system, then this course is for you.

Who this course is for
Students who want to learn building real-time streaming pipelines from SCRATCH to its Live Project Implementation.
Students who want to learn latest technologies that are used in Big Data Engineering.
Developers who want to learn different well-known tools to build streaming pipelines.
Students who want to pursue and grow career in Data Engineering.