Required Skills: ETL, SSIS, Pyspark, sql, python, Databricks, data modeling

Job Description

• Core needs are around ETL, using SSIS immediately for the ingestion of source data into Postgres.
• Will need PySpark, experience in building PySpark notebooks, for migration from Postgres to Databricks/Snowflake.
• Can have experience in either Databricks or Snowflake, knowledge of medallion architecture--curating and modeling data through Bronze, Silver, Gold layers to semantic/reporting layer.
• Responsible for supporting some modeling efforts--not full-blown data modelers but should know entity-relationship, dimensional models.
• Ingestion of data, deduplication
• Another team within Optum is responsible for building the visualizations themselves.
• Do not quite have a data dictionary, in the process of analyzing and documenting all lineage. They know the data sources though.
• Considering Purview or Unity Catalog long-term.
• Needs all Senior Engineers, 6-8 years of experience, entirely hands-on and need to "hit the ground running"
• Healthcare experience is preferred to understand the data, data formats, compliance regulations, etc.

• Purview or Unity Catalog
• Healthcare experience