Senior Data Quality Engineer
- S Linx LLC
111 Days Ago
NA
NA
Remote
5-11 Years
Required Skills: Python, PySpark, Pandas
Job Description
ESSENTIAL DUTIES, RESPONSIBILITIES & OUTCOMES
• Support design, build and implement automated test framework and tools to automate the Data platform services and applications.
• Develop and implement testing practices and tools as part of the framework which ensure releases are defect-free and perform at or better than expected levels
• Learn the Worldwide Express business and ensure the Quality Assurance team understands business requirements at a detailed level
• Provide technical directions, mentor quality assurance team as well as the scrum teams on effective testing practices, including test driven development
• Use SQL queries and analyze log files to test various ETL and data pipelines
• Develop data quality testing plans and scripts. Perform data validity, accuracy and integrity test across different components of the Data platform.
• Automate the regression testing and ensure quality practices are built into the SDLC and DevOps practices
• Ensure that business requirements are fully met before each release; certify releases for quality and completeness of functionality
• Closely collaborate with other technology and business users to ensure business and technical requirements are understood and met
Minimum Qualifications:
• Bachelor’s Degree, Computer Science, Engineering or related is requirement
• 5+ years of work experience in QA, preferably in data science or relevant space
• 5+ yrs applied experience writing complex SQL / Spark SQL on large data sets to transform data, facilitate accurate and reliable data analytics across various functions.
• Strong coding abilities in Python, PySpark, Pandas or other similar tools for large scale data processing, data quality checks, validations and analysis.
• Advanced proficiency in SQL and familiarity with other relational data via Oracle, Postgres, and/or SQL Server
• Proficiency with industry recognized ETL/ELT testing methodologies, techniques, processes and standards
• Hands-on experience with Databricks in AWS environment
• Experience in automation of data pipelines, data services, cloud data warehouses, business intelligence, and machine learning platforms
• Experience in developing Databricks notebook to test data quality and business requirements
• Passionate and highly skilled in utilizing programming languages and analytics tools/technologies to validate products, machine learning models, data pipelines, and data deliverables
KNOWLEDGE OF
• Understanding of the industry, specific --business model, and unique characteristics of the Company.
• Detailed understanding of Quality Assurance tools, practices and success measures. Known to others for knowledge and leadership in this area.
• Agile development methodologies.
• Common business practices and software product design and development.
• Building collaborative relationships.
