Professional Overview
Data Engineer with 4+ years of experience working on both AWS and Azure cloud platforms across two organizations. Skilled in Azure Data Factory, Databricks, ADLS, AWS Glue, EMR, Redshift, PySpark, and SQL. Experienced in building scalable ETL pipelines, lakehouse architectures, and optimizing high-volume data processing workflows.
Experience
AI Expertise
Skills
Employment Type
Description of Responsibilities
- Built scalable Apache Airflow workflows for reliable data pipeline runs.
- Developed ETL pipelines using AWS Glue and PySpark for large datasets.
- Designed EventBridge and schedule-based workflows for task automation.
- Monitored pipelines with AWS CloudWatch and resolved production issues.
- Worked with teams to meet workflow, business, and data quality needs.
Key Achievements / KPIs
- Processed 200+ GB data daily across enterprise ETL pipelines
- Reduced pipeline runtime by 35% through Spark and SQL optimization
- Improved query performance by 40% using Redshift tuning techniques
- Automated workflows, reducing manual effort and operational delays
AI Expertise
Skills
Employment Type
Description of Responsibilities
- Built Bronze, Silver, Gold lakehouse layers using Delta Lake on Amazon S3.
- Developed PySpark ETL pipelines in AWS Glue and EMR with tuning.
- Managed end-to-end workflows using Apache Airflow orchestration.
- Integrated curated data with Amazon Redshift for BI and reporting.
- Optimized schemas and Parquet/Delta storage for better performance.
Key Achievements / KPIs
- Built scalable AWS lakehouse for structured and semi-structured data.
- Implemented Bronze, Silver, Gold layers using Delta Lake on Amazon S3.
- Enabled ACID transactions and versioned data management with Delta Lake.
- Used Apache Airflow to orchestrate complex batch ETL workflows.
AI Expertise
Skills
Employment Type
Description of Responsibilities
- Built ETL pipelines using AWS Glue, PySpark, and Apache Spark.
- Ingested data from APIs, files, and databases into Amazon S3.
- Applied PySpark transformations and cleansing for quality datasets.
- Modeled and loaded data into Amazon Redshift for analytics reporting.
- Improved pipeline speed using partitioning, tuning, and indexing.
Key Achievements / KPIs
- Built scalable AWS pipelines for customer transaction data processing.
- Used Amazon S3 as a centralized data lake for large-scale ingestion.
- Developed ETL workflows using AWS Glue and PySpark transformations.
- Modeled curated datasets in Amazon Redshift for BI and analytics.
Education
Field of Study: Electronics and Communication Engineering
Areas of Expertise
Data Science & Analytics
Extract business insights from raw data using statistical modelling, predictive analytics, A/B testing, and interactive dashboards.
