Databricks Migration Engineer
- VYZE INC
2 Days Ago
NA
C2C
Laurel-MD
14-15 Years
Required Skills: Azure Databricks, Azure Data Lake Storage, ADLS, Azure Data Factory, ADF, SQL Server, Oracle
Job Description
Client currently uses Informatica Power Center ETL workflows, mappings, sessions, and related integration processes to support enterprise reporting, analytics, operational data integration, financial reporting, and regulatory reporting. Client is undertaking a modernization initiative to migrate and transform existing ETL assets from Informatica Power Center to Microsoft Azure-based cloud data platforms, including Azure Databricks and Azure Synapse Analytics.
Required Skills:
Need senior candidates 14-15 years atleast. Remote is okay and needs utility experience, like someone from (Oil, Gas, and Water Industry)
Lead the assessment, design, migration, modernization, testing, and deployment of Informatica Power Center ETL workflows to Azure Databricks and Azure Synapse Analytics.
Strong experience with Informatica Power Center development and migration projects.
Hands-on experience with Azure Databricks.
Strong expertise in Azure Synapse Analytics.
Experience with Azure Data Lake Storage (ADLS).
Experience with Azure Data Factory (ADF) or similar orchestration tools.
Strong SQL development and query optimization skills.
Experience building ETL/ELT pipelines in cloud environments.
Experience with data integration involving:
Oracle
SQL Server
APIs
File-based integrations
Experience implementing CI/CD pipelines using Azure DevOps.
Knowledge of source control and release management processes.
Experience with monitoring, logging, and operational support frameworks.
Understanding of data validation, reconciliation, and data quality processes.
Experience with cloud security concepts including:
RBAC
Azure Key Vault
Managed Identities
Service Principals
Microsoft Entra ID (Azure AD)
Roles & Responsibilities
Assess and inventory existing Informatica Power Center workflows, mappings, sessions, schedules, and dependencies.
Analyze current ETL architecture and identify migration complexity, risks, and optimization opportunities.
Develop migration roadmaps, implementation strategies, and modernization plans.
Design Azure-based ETL/ELT solutions using Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage, and Azure Data Factory.
Convert and migrate Informatica Power Center workflows and mappings to Azure cloud-native solutions.
Develop Databricks notebooks, jobs, Synapse pipelines, scripts, and orchestration workflows.
Implement ETL best practices including logging, monitoring, error handling, audit tracking, restartability, and parameterization.
Optimize ETL performance through partitioning, parallel processing, query tuning, and cloud resource optimization.
Integrate Azure data platforms with Oracle, SQL Server, APIs, GIS systems, Power BI, and other enterprise applications.
Configure Azure security components including Azure Key Vault, RBAC, Managed Identities, Service Principals, and Microsoft Entra ID.
Build and maintain CI/CD pipelines using Azure DevOps and source control repositories.
Create technical designs, source-to-target mappings, architecture documentation, and operational runbooks.
Develop testing strategies and execute unit, integration, regression, performance, and reconciliation testing.
Validate data quality, transformation accuracy, data completeness, and end-to-end processing consistency.
Support production deployments, cutover activities, rollback planning, and post-production stabilization.
Configure monitoring dashboards, alerts, logging, and operational reporting solutions.
Provide technical training, knowledge transfer sessions, and operational support documentation.
Collaborate with business stakeholders, architects, developers, QA teams, and production support teams.
Provide project status updates, risk assessments, issue tracking, and migration progress reporting
