ETL Data Engineer
- Ohm Systems, Inc.
12 Hours Ago
NA
W2
Chicago-IL
10-15 Years
Required Skills: Apache Spark, Data Engineering, Databricks, Microsoft Azure, Extract, Transform, Load, SQL, Python
Job Description
We are looking for an experienced Senior Data Engineer with strong expertise in Apache Spark, data engineering, and production-grade ETL pipelines. The ideal candidate should have hands-on experience with Python, SQL, Microsoft Azure, Azure Data Factory, and Databricks. The candidate should be capable of developing scalable, reliable, and high-performance data solutions.
Key Responsibilities
- Design, develop, and maintain production-grade ETL data pipelines.
- Build and optimize data transformation and processing solutions using Python, SQL, and Apache Spark.
- Work with Azure cloud services, particularly Azure Data Factory and Databricks.
- Ensure data pipelines are scalable, reliable, and performance-efficient.
- Monitor, troubleshoot, and maintain data pipelines in production environments.
- Collaborate with cross-functional teams in an Agile environment.
- Communicate technical requirements, progress, and challenges effectively.
- Support infrastructure automation and deployment using Infrastructure as Code (IaC).
- Work with Power BI to support data analytics and reporting requirements.
Required Skills & Qualifications
- 10+ years of experience in data engineering and building production-grade ETL pipelines.
- Strong proficiency in Python and SQL for data transformation and scripting.
- Hands-on experience with Apache Spark.
- Strong understanding of data engineering principles, scalability, reliability, and performance optimization.
- Experience with Microsoft Azure cloud services, especially Azure Data Factory (ADF) and Databricks.
- Familiarity with Power BI.
- Exposure to Infrastructure as Code (IaC).
- Strong communication and collaboration skills.
- Ability to work effectively in Agile environments.
