Required Skills: Python, Java, C#, Azure, AWS, Google Cloud Platform, OLTP, OLAP systems Apache Spark

Job Description

Responsibilities

• Design, develop, and maintain scalable ETL pipelines using Azure Databricks
• Build resilient data ingestion processes from on-premise and cloud-based transactional databases and APIs
• Develop reusable and efficient workflows using Databricks Notebooks and Delta Live Tables
• Collaborate with technical leads and development teams for solution design and implementation
• Work with business stakeholders to gather requirements and support change management activities
• Develop batch and streaming data processing solutions using Apache Spark
• Collaborate with DBAs and business users to create high-quality data products
• Ensure data quality, validation, governance, and security best practices
• Maintain technical documentation for data engineering processes
• Support enterprise data architecture initiatives and modernization efforts

Required Skills

• 5+ years of excellent verbal and written communication skills
• 2+ years of experience writing clean and maintainable Databricks Notebook code
• Strong understanding of:
• Data Engineering Best Practices
• Data Warehouses & Data Lakes
• Delta Lake Architecture
• Hands-on experience with:
• Azure Databricks
• Spark / PySpark
• Databricks SQL
• Delta Live Tables
• Experience working with cloud platforms such as:
• Azure
• AWS
• Google Cloud Platform (GCP)
• Strong knowledge of:
• OLTP & OLAP systems
• Apache Spark
• Streaming technologies like Azure Service Bus
• Experience building scalable ETL and streaming solutions

Desired Skills

• Experience with object-oriented programming languages such as:
• Python
• Java
• C#
• Prior experience working with State or Government clients preferred