I am an Aspiring Data Engineer with a solid background in data analysis, automation, and business intelligence. My goal is to expand my expertise in data engineering, building on my skills in data pipelines, data modeling, and data automation. This portfolio showcases projects where I apply these concepts to solve real-world problems.
- SAP Automation with Python: Automating SAP data extraction using Python and combining datasets with Pandas, integrating results into Power BI.
- Batch File Integration: Leveraging Windows Task Scheduler to automate and streamline the process.
- Apache Airflow Automation (optional): Demonstrating orchestration and scheduling capabilities using Airflow.
This project demonstrates how to use Apache Spark to process large datasets in parallel, orchestrating Docker containers via Kubernetes to optimize performance. It showcases dynamic partitioning and cloud scalability, with an automated workflow using Apache Airflow as the orchestrator for the entire process.
- Data Marts vs. Data Warehouses: A project comparing performance, scalability, and use cases for Data Marts versus Data Warehouses, optimized for business intelligence tools like Power BI.
- Dashboard for Volume Comparison: A Power BI dashboard showcasing scenarios to decide between Data Marts or a Data Warehouse based on data volume and complexity.
I'm on a continuous journey to become a proficient Data Engineer, building on my industrial engineering and business intelligence background. I aim to deliver valuable insights and automation by leveraging my skills.
Feel free to explore the projects, and I’m always open to collaboration or discussions on data-related topics.
If you have any questions feel free to contact me at [email protected] or via LinkedIn.