Azure Data Engineer
- Apetan Consulting LLC
2 Hours Ago
NA
C2C
Phoenix-AZ
5-7 Years
Required Skills: Databricks, Delta Lake, Python
Job Description
Seeking a Specialist with 5 to 7 years of experience in Azure Databricks Python Delta Lake and Azure Container Apps to design and implement advanced data engineering solutions
Work within the Azure Databricks skill cluster family focusing on data migration ingestion and processing Utilize Databricks Python Delta Lake and Azure Container Apps ACA to build scalable reliable and efficient data pipelines Collaborate with source data application teams and product owners to integrate data solutions supporting analytics and business intelligence Apply best practices in cloud engineering data modeling and performance tuning to optimize data workflows Ensure data governance privacy and security standards are upheld in all data engineering activities Leverage expertise in Azure native cloud services to manage data storage transformation and consumption layers Utilize AI assisted developer tools to enhance development efficiency and quality Implement and improve CICD pipelines using GitHub and GitHub Actions Support monitoring ing and performance tuning of data platforms Mentor junior team members and contribute to knowledge sharing within the team
Roles and Responsibilities
Design develop and implement batch and streaming data pipelines using Azure Databricks Python Delta Lake and Azure Container Apps
Collaborate closely with cross functional teams including source data teams and product owners to deliver analytics solutions that enhance decision making
Lead development of coding standards best practices and security guidelines for data engineering projects
Participate actively in the full software development lifecycle including testing deployment and production support
Provide technical leadership and work independently as well as part of a DevOps team responsible for end-to-end product ownership
Perform cloud infrastructure design network security and administration related to data engineering solutions
Mentor others on technical and domain skills to create multifunctional teams
Implement and maintain CICD pipelines leveraging GitHub Actions for automation and continuous delivery
