Spark and Scala Engineer
- Gurus Infotech, Inc
11 Days Ago
NA
NA
O'Fallon-MO
8-12 Years
Required Skills: Spark development, Scala development, Python, Hadoop, SQL, Shell Scripts, Data modelling, data integration, data engineering
Job Description
Roles and Responsibilities:
· Core is Spark/Scala development opportunity.
· Strong People management, leadership, organizational skills. Outstanding communication (written and verbal) skills.
· Experience leveraging open-source tools, predictive analytics, machine learning, Advanced Statistics, and other data techniques to perform basic analyses.
· Demonstrated basic knowledge of statistical analytical techniques, coding, and data engineering.
· Experience developing and configuring dashboards is a plus.
· Demonstrated judgement when escalating issues to the project team.
· High proficiency in Python/Spark, Hadoop platforms & tools (Hive, Impala, Airflow, NiFi), SQL.
· Curiosity, creativity, and excitement for technology and innovation.
· Demonstrated quantitative and problem-solving abilities.
· Expert proficiency in using Python/Scala, Spark (tuning jobs), SQL, Hadoop platforms to build Big Data products & platforms.
· Comfortable in developing shell scripts for automation.
· Proficient in standard software development, such as version control, testing, and deployment.
· Experience with visualization tools like tableau, looker.
· At least 5 years leading collaborative work in complex engineering projects in an Agile setting e.g. Scrum.
· Extensive data warehousing/data lake development experience with strong data modelling and data integration experience.
· Good SQL and higher-level programming languages with solid knowledge of data mining, machine learning algorithms and tools.
· Strong hands-on experience in Analytics & Computer Science.
· Demonstrated basic knowledge of statistical analytical techniques, coding, and data engineering.
· Experience in building and deploying production-level data-driven applications and data processing workflows/pipelines and/or implementing machine learning systems at scale in Java, Scala, or
· Python and deliver analytics involving all phases like data ingestion, feature engineering, modelling, tuning, evaluating, monitoring, and presenting.
· At least 10 years of relevant hands-on experience as a Data Engineer in an individual contributor capacity.
· Able to lead the implementation of machine learning production systems.
· Demonstrated ability, through hands-on experience, to develop production machine learning pipelines.
