Senior Data Disposition & Migration Intelligence Engineer
- Nityo Infotech
3 Days Ago
NA
W2, C2C
Remote
6-12 Years
Required Skills: Data migration frameworks, .NET, Java, Angular, Python, Cloud migration, Automation & scripting, Cloud migration tools.
Job Description
Key Skills Required: engineer
Must-have (prioritize candidates with depth in these areas):
Critical Amazon Bedrock — model selection, prompt design, Knowledge Bases, retrieval-augmented generation (RAG), guardrails, and production inference patterns
Critical AWS platform — CI/CD (CodePipeline / GitHub Actions on AWS), Lambda, Step Functions, OpenSearch (vector search), Neptune or graph storage, Secrets Manager, S3 artifact pipelines, IAM, and cost/scale design
Required SQL Server at scale — stored procedure dependency tracing, cross-database schema reconciliation, RLS / SESSION_CONTEXT
Required Dispositioning & triage — turning automated outputs (schema drift, source-only tables, static-analysis findings) into auditable, dependency-backed decisions
Required ID de-collision analysis — overlapping ID ranges, true collisions vs. co-mingled data, table-level resolution
Required ETL / batch consolidation — Informatica (or equivalent), migration script quality, tenant-scoped batch jobs
Required LLM output validation — guardian / evaluation patterns, golden-set regression, confidence thresholds
Required Code literacy (.NET, Java, Angular, Python); static analysis & SARIF; financial-services consolidation experience
Must-have (prioritize candidates with depth in these areas):
Critical Amazon Bedrock — model selection, prompt design, Knowledge Bases, retrieval-augmented generation (RAG), guardrails, and production inference patterns
Critical AWS platform — CI/CD (CodePipeline / GitHub Actions on AWS), Lambda, Step Functions, OpenSearch (vector search), Neptune or graph storage, Secrets Manager, S3 artifact pipelines, IAM, and cost/scale design
Required SQL Server at scale — stored procedure dependency tracing, cross-database schema reconciliation, RLS / SESSION_CONTEXT
Required Dispositioning & triage — turning automated outputs (schema drift, source-only tables, static-analysis findings) into auditable, dependency-backed decisions
Required ID de-collision analysis — overlapping ID ranges, true collisions vs. co-mingled data, table-level resolution
Required ETL / batch consolidation — Informatica (or equivalent), migration script quality, tenant-scoped batch jobs
Required LLM output validation — guardian / evaluation patterns, golden-set regression, confidence thresholds
Required Code literacy (.NET, Java, Angular, Python); static analysis & SARIF; financial-services consolidation experience
Role Summary:
Senior practitioner who combines deterministic data engineering with Bedrock-powered AI reasoning to accelerate a large-scale multi-tenant consolidation. You turn automated signals — schema drift, source-only tables, ID collisions, and static-analysis findings — into defensible disposition decisions grounded in dependency chains (stored procedures, application code, ETL, batch jobs). You also productionize the toolchain on AWS: RAG through CI/CD, knowledge capture, and a guardian pattern that validates every model output before it reaches reviewers or migration teams.
Senior practitioner who combines deterministic data engineering with Bedrock-powered AI reasoning to accelerate a large-scale multi-tenant consolidation. You turn automated signals — schema drift, source-only tables, ID collisions, and static-analysis findings — into defensible disposition decisions grounded in dependency chains (stored procedures, application code, ETL, batch jobs). You also productionize the toolchain on AWS: RAG through CI/CD, knowledge capture, and a guardian pattern that validates every model output before it reaches reviewers or migration teams.
Core Responsibilities:
1. Dispositioning Reasoning
· Adjudicate schema drift, source-only tables, and migration-priority findings using SP → app → ETL dependency evidence.
· Apply deterministic rules where the answer is deterministic; invoke Bedrock RAG where genuine judgment is required.
· Produce auditable dispositions with confidence levels and blast-radius assessment; feed validated decisions back into the knowledge base.
2. ID De-Collision Reasoning
· Investigate overlapping ID ranges; distinguish true collisions, co-mingled data, and namespace overlap without semantic conflict.
· Define table-level resolutions (re-keying, offset mapping, surrogate keys, RLS-scoped acceptance) with end-to-end traceability through SPs, apps, and ETL.
3. Data Migration & Batch Consolidation
· Review migration script quality (idempotency, ordering, rollback, tenant scoping).
· Consolidate Informatica / batch jobs for tenant-specific routing; align with comment-out vs. refactor vs. data-layer-only strategies.
· Support cutover sequencing, reconciliation, and post-migration validation.
4. Productionizing the Toolchain on AWS
· Integrate the Bedrock RAG disposition assistant into CI/CD (retrieval over dependency graphs, schema diffs, SME feedback, prior dispositions).
· Implement a guardian / evaluation pattern: grounding checks, structured-output validation, confidence thresholds, escalation for high-blast-radius decisions, golden-set regression.
· Operationalize SME knowledge capture, observability, and versioning for prompts, indexes, and disposition rules.
5. Toolchain Assessment
· Outside-in review of the code-graph generator (resolution accuracy, build vs. leverage, polyglot coverage).
· Recommend AWS scaling paths: Bedrock Knowledge Bases, OpenSearch vectors, Neptune, serverless batch, SARIF/report pipelines.
· Deliver a prioritized roadmap with measurable coverage improvement (EAC and disposition throughput).
1. Dispositioning Reasoning
· Adjudicate schema drift, source-only tables, and migration-priority findings using SP → app → ETL dependency evidence.
· Apply deterministic rules where the answer is deterministic; invoke Bedrock RAG where genuine judgment is required.
· Produce auditable dispositions with confidence levels and blast-radius assessment; feed validated decisions back into the knowledge base.
2. ID De-Collision Reasoning
· Investigate overlapping ID ranges; distinguish true collisions, co-mingled data, and namespace overlap without semantic conflict.
· Define table-level resolutions (re-keying, offset mapping, surrogate keys, RLS-scoped acceptance) with end-to-end traceability through SPs, apps, and ETL.
3. Data Migration & Batch Consolidation
· Review migration script quality (idempotency, ordering, rollback, tenant scoping).
· Consolidate Informatica / batch jobs for tenant-specific routing; align with comment-out vs. refactor vs. data-layer-only strategies.
· Support cutover sequencing, reconciliation, and post-migration validation.
4. Productionizing the Toolchain on AWS
· Integrate the Bedrock RAG disposition assistant into CI/CD (retrieval over dependency graphs, schema diffs, SME feedback, prior dispositions).
· Implement a guardian / evaluation pattern: grounding checks, structured-output validation, confidence thresholds, escalation for high-blast-radius decisions, golden-set regression.
· Operationalize SME knowledge capture, observability, and versioning for prompts, indexes, and disposition rules.
5. Toolchain Assessment
· Outside-in review of the code-graph generator (resolution accuracy, build vs. leverage, polyglot coverage).
· Recommend AWS scaling paths: Bedrock Knowledge Bases, OpenSearch vectors, Neptune, serverless batch, SARIF/report pipelines.
· Deliver a prioritized roadmap with measurable coverage improvement (EAC and disposition throughput).
Qualifications:
Required
· 8+ years in data engineering, platform migration, or enterprise modernization (financial services preferred).
· Production experience with Amazon Bedrock and core AWS services (not exploratory POCs).
· Large-scale SQL Server dependency analysis and schema reconciliation.
· Hands-on ETL/batch consolidation (Informatica or equivalent).
· CI/CD integration and LLM output validation in production workflows.
Required
· 8+ years in data engineering, platform migration, or enterprise modernization (financial services preferred).
· Production experience with Amazon Bedrock and core AWS services (not exploratory POCs).
· Large-scale SQL Server dependency analysis and schema reconciliation.
· Hands-on ETL/batch consolidation (Informatica or equivalent).
· CI/CD integration and LLM output validation in production workflows.
Preferred
· Multi-tenant consolidation, tenant discriminators, feature entitlements.
· Code graph / static analysis (Roslyn, OpenRewrite, CodeQL, custom analyzers).
· Guardian patterns for LLM outputs; Python/PowerShell report pipeline automation.
· Multi-tenant consolidation, tenant discriminators, feature entitlements.
· Code graph / static analysis (Roslyn, OpenRewrite, CodeQL, custom analyzers).
· Guardian patterns for LLM outputs; Python/PowerShell report pipeline automation.
Engagement & Success Metrics
Phase Focus
Assess Toolchain review; disposition backlog; ID collision inventory
Establish Disposition standards; Bedrock RAG + guardian pattern; CI/CD skeleton
Execute Domain dispositioning; migration/batch review; ID resolution
Operationalize Knowledge loops, metrics, program handoff
Phase Focus
Assess Toolchain review; disposition backlog; ID collision inventory
Establish Disposition standards; Bedrock RAG + guardian pattern; CI/CD skeleton
Execute Domain dispositioning; migration/batch review; ID resolution
Operationalize Knowledge loops, metrics, program handoff
Success: ≥90% SME acceptance on first review; guardian catches ≥95% of ungrounded model outputs; zero post-cutover ID collision defects in pilot domains; documented AWS toolchain roadmap with coverage targets.
