Responsibilities

• Lead the design, implementation, and optimization of enterprise Databricks solutions.
• Manage and support key Databricks capabilities including Unity Catalog, Delta Lake, Databricks SQL, Workflows, Jobs, MLflow, and Mosaic AI.
• Design and maintain scalable data ingestion, transformation, and orchestration pipelines.
• Optimize Spark workloads, SQL performance, cluster utilization, and overall platform efficiency.
• Establish and enforce governance, security, compliance, and data management standards.
• Support machine learning and Generative AI initiatives utilizing MLflow, Vector Search, Feature Store, and LLM capabilities.
• Serve as the escalation point for complex Databricks production issues and performance challenges.
• Develop integrations between Databricks, cloud platforms, orchestration tools, BI solutions, and enterprise applications.
• Implement monitoring, logging, alerting, and operational dashboards for platform visibility and reliability.
• Collaborate with Data Engineering, Data Science, Analytics, Infrastructure, and business stakeholders to deliver data and AI solutions.
• Provide technical leadership, mentoring, training, and platform best practices across the organization.

Required Skills

• Bachelor's degree in Computer Science, Information Technology, Business, or a related field, or equivalent experience.
• U.S. Citizenship required.
• 7+ years of experience in Data Engineering, Data Architecture, or related data platform roles.
• 4+ years of dedicated hands-on Databricks experience.
• Proven experience implementing Unity Catalog, including cross-workspace governance and fine-grained security controls.
• Deep expertise with Apache Spark performance tuning and optimization.
• Experience supporting production Databricks environments at enterprise scale.
• Hands-on experience building Generative AI, LLM, or machine learning solutions using Databricks technologies.
• Advanced proficiency in Python, Scala, and SQL.
• Experience deploying and managing Databricks infrastructure using Terraform or other Infrastructure-as-Code tools.
• Experience implementing secure Databricks networking architectures, including Private Link, VPC Peering, encryption, and access controls.
• Strong experience with CI/CD pipelines using GitHub Actions, Azure DevOps, GitLab, Databricks CLI, or Databricks Asset Bundles.
• Strong troubleshooting, architecture, communication, and stakeholder management skills.

Preferred Skills

• Databricks Certified Solutions Architect.
• Databricks Certified Data Engineer Professional.
• Databricks Certified Machine Learning Professional.
• AWS Certified Solutions Architect or Azure Solutions Architect Expert certification.
• Experience with Mosaic AI, Feature Store, Vector Search, and MLflow.
• Experience designing enterprise-scale data platforms, AI ecosystems, and modern Lakehouse architectures.
• Experience supporting highly regulated environments with strong governance and compliance requirements.