Snowflake Data Engineer
- Tekshapers Inc.
1 Days Ago
NA
C2C
California City-CA
5-15 Years
Required Skills: Python , scikit learn, PyTorch, TensorFlow, SnowSQL
Job Description
Snowflake Data Engineering –
o Design and implement enterprise-grade data pipelines using Snowflake, including ingestion and transformation
o Must be strong in both Core and Semantic aspects
o Develop complex SQL transformations, stored procedures and Dynamic tables inside Snowflake to enable near real-time and batch processing
o Implement Snowflake data sharing, data marketplace integrations
o Engineer Snowpipe and Kafka-to-Snowflake streaming ingestion pipelines also handling high throughput event data at scale
o Optimize Snowflake cluster performance – virtual warehouse sizing, query profiling, clustering keys
o Architecture, design aspects, performance tuning, time travel, warehouse concepts - scaling, clustering, micro-partitioning
o Experience with SnowSQL, Snowpipe
• Data Integration aspects –
o Design and maintain end-to-end ETL/ELT pipelines using Apache Airflow
o Experience in building reusable parameterized data ingestion pipelines/frameworks is beneficial.
o Thorough on data quality checks
• AI and Data Science –
o Integrate AI/LLMs with data pipelines via Python UDFs or API callouts – enabling text analytics, semantic search and GEN-AI augmented workflows
o Experience with Python based frameworks – scikit learn, PyTorch, TensorFlow
o Experience with NLP and text-mining techniques on unstructured data to identify actionable information
o Time-series forecasting, anomaly detection and propensity modeling
• Experience with Data Visualization aspects
• Hands-on experience with writing Complex queries using – Joins, Self Joins, Views, Materialized Views, Cursor also Recursive, use of GROUP BY, PARTITION BY functions / SQL Performance tuning
• Hands-on experience with ETL and Dimensional Data Modelling – Slowly Changing Dimensions (SCD – Type 1, 2, 3)
o Good understanding of concepts like schema types, table types - fact-dimension etc. like how to design a dimension vs fact, design considerations factored etc.
• Proficiency in Python scripting/programming – using Pandas, PyParsing, Airflow.
o Pandas, Tableau server modules, Numpy, Datetime, Apache Airflow related modules, APIs
o Data Pipeline automation
o Strong Python programming skills
• Actively participating in discussions with business to understand requirements, perform thorough impact analysis and provide suitable solutions.
