AI Trend Forecaster: End-to-End ETL Pipeline with Azure

In this project, Our team developed a scalable data pipeline using Azure tools to predict AI trends by analyzing stock performance, Reddit activity, and Google Trends data. By training an LSTM-based model and integrating real-time data streams, we forecasted market sentiment and visualized insights through Power BI dashboards. This work bridges the gap between public sentiment, market trends, and AI advancements, providing stakeholders with actionable predictions and strategic insights.

My Contribution

Designed and implemented a streamlined data pipeline to process, clean, and analyze data using Azure tools.
Built and trained an LSTM model with features such as early stopping and learning rate scheduling for time-series trend forecasting.

Final Presentation

Final Report

Final Presentation Slides

Pipeline Development

I learned how to create an end-to-end pipeline, starting with data collection and cleaning, and ending with real-time visualization in Power BI.

Azure Ecosystem Expertise

I gained hands-on experience with Azure tools like Data Factory, Event Hubs, and Databricks to process and analyze data efficiently.

Developing LSTM Models

I learned to design and train LSTM models for time-series forecasting, capturing complex patterns in stock and public sentiment data.