Ab Initio Data Engineer
- Micasa Global
1 Days Ago
NA
C2C
Berkeley Heights-NJ
8-12 Years
Required Skills: ETL, Ab Initio, Ab Initio GDE, Co>Op, EME, Conduct>It, parallel processing, Multi-file System, MFS, SQL, Unix Shek Scripting, Linux Scripting, Data warehousing, Kafka, COBOL copybooks, VSAM, flat files, Kubernetes, OpenShift, Hadoop ecosystems, Spark ecosystems, hybrid ETL architectures
Job Description
We are seeking a highly skilled resource to design and implement high-performance, event-driven data pipelines, ensuring low-latency data processing and high availability system for the large credit card processing system. The ideal candidate will work with the Ab Initio ecosystem (GDE, EME, Conduct>It) to build stateful services that ingest, filter, and transform data from sources like Kafka or message queues, pushing updates to dashboards or downstream databases in near-real-time.
Responsibilities
• Design, develop, and maintain complex ETL graphs using Ab Initio GDE (Graphical Development Environment)
• Architect end-to-end data integration solutions spanning multiple source/target systems (RDBMS, mainframe, flat files, message queues)
• Optimize graph performance — parallelism, partitioning (broadcast, hash, range), layout tuning, checkpointing
• Lead development using Ab Initio components: Co>Operating System, EME (Enterprise Meta>Environment), Conduct>It, Data Profiler, Continuous Flows
• Own metadata management and data lineage within EME
• Design and implement CDC (Change Data Capture) and near-real-time data pipelines
• Troubleshoot production job failures, perform root-cause analysis, and implement fixes with minimal downtime
• Define coding standards, review peer-developed graphs, and enforce best practices
• Mentor junior/mid-level Ab Initio developers
• Collaborate with data architects, DBAs, and business analysts to translate requirements into technical design
Required Qualifications
• Bachelor’s degree in Computer Science, Engineering, or related field
• 7+ years of ETL development experience, with 5+ years specifically in Ab Initio
• Strong hands-on expertise with Ab Initio GDE, Co>Op, EME, and Conduct>It
• Deep understanding of parallel processing concepts and Ab Initio partitioning/multi-file system (MFS)
• Strong SQL skills and experience with major RDBMS (DB2)
• Experience with Unix/Linux shell scripting
• Solid understanding of data warehousing concepts (dimensional modeling, SCDs, star/snowflake schemas)
• Experience with Kafka
• Strong debugging and performance-tuning skills for large-volume data processing
Preferred Qualifications
• Experience with mainframe data integration (COBOL copybooks, VSAM, flat files)
• Exposure to Kubernetes, OpenShift
• Familiarity with Hadoop/Spark ecosystems and hybrid ETL architectures
• Knowledge of API-based or streaming integration (Kafka)
• Prior experience as a technical lead or in a client-facing architecture role
Soft Skills / Leadership
• Strong analytical and problem-solving skills for complex data issues
• Ability to lead design discussions and drive technical consensus
• Effective mentorship and knowledge-sharing across the team
• Clear communication with both technical and business stakeholders
Behavioral Skills:
- Good Communication skills
- 5 days Work from Office at Berkley Heights, NJ
- Team Player
- Ability to work in a changing environment
- Strong problem solving and analytical skills
- Ability to work independently or within a team
- Manage day-to-day challenges and communicate developmental risks with the technical team
