In this lab, students will build a streaming data pipeline using Amazon Kinesis Data Streams and AWS Glue to ingest and transform data in real-time. By leveraging Kinesis Data Streams for data ingestion and AWS Glue for data transformation, learners will develop a deeper understanding of processing streaming data at scale. This lab simulates a financial services company collecting and analyzing real-time stock market data to provide analytics dashboards to their clients. Participants will set up data streams, configure AWS Glue jobs for ETL processes, and validate data flow through the system. This will include critical tasks like setting up Kinesis producers, processing data with AWS Glue jobs, and ensuring transformed data is stored accurately in Amazon S3.
Global Financial Services Inc. requires the ability to process real-time stock market data to provide live analytics to their clients for better decision-making. They need a robust and scalable system to handle high-frequency data ingestion and transformation. The current challenge is setting up a seamless streaming process using Amazon Kinesis and AWS Glue to ensure timely and accurate data analysis.
