Skip to content

VincentNjoku/real-time-data-pipeline

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 Cannot retrieve latest commit at this time.

History

2 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Real-Time Serverless Data Processing Pipeline with Tableau Visualization

Objective

This project demonstrates a serverless application to process and store IoT sensor data in real time, with data visualization using Tableau.

Features

  • Simulate IoT Sensor Data: Generate temperature and humidity data using Python.
  • Real-Time Data Processing: AWS Lambda processes and stores data in DynamoDB.
  • Data Export: Export DynamoDB data to CSV for Tableau integration.
  • Visualization: Tableau dashboards to analyze sensor trends and insights.

Technologies

  • AWS Lambda
  • AWS S3
  • DynamoDB
  • CloudWatch
  • Tableau
  • Python

Setup and Execution

Step 1: Simulate IoT Data

  1. Run iot_sensor_simulation.py to upload sensor data to S3.
  2. AWS Lambda processes the data and stores it in DynamoDB.

Step 2: Export Data for Tableau

  1. Run export_dynamodb_to_csv.py:
    python export_dynamodb_to_csv.py

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages