Kinesis Data stream+Kinesis Firehose – UI events

Introduction

We will explain how we set up a data streaming pipeline from Kinesis Data stream to Athena using various AWS services.

This solution will result in a presentation of the events written to the stream as records in a table.. 

Architecture Overview

The solution will involve the following components:

1. Events: Usage of aiOla app triggers events that are directed to a central stream.

2. Kinesis Data Stream: Real-time data streaming service provided by Amazon Web Services (AWS) that allows you to collect and process large volumes of data in real time..

3. Kinesis Firehose: Streams processed data from Lambda to an S3 bucket.

4. Kinesis Firehose Role: IAM role that grants necessary permissions to Kinesis Firehose.

5. S3 Bucket: Stores the streamed data in a structured format.

6. Glue Table: Catalogs the data in S3, making it queryable by Athena.

7. Athena: Allows SQL queries on data stored in the Glue table