Rust Engineer
- Hirextra Inc.
18 Hours Ago
NA
W2
San Francisco-CA
2-5 Years
Required Skills: Rust, Python, PostgreSQL, Redis
Job Description
The Role
LiteLLM is migrating its core gateway from Python to Rust, and we're looking for the engineer to lead it. You'll work directly with the CTO to plan the migration, restructure the codebase, and drive a gradual, stability-first rollout — with the explicit goal of making LiteLLM the fastest AI gateway in the industry. This is a high visibility, high-ownership role: your architectural decisions will directly shape latency and throughput for enterprise customers running production AI workloads at scale.
What You'll Do
• Own the end-to-end migration of LiteLLM's gateway from Python to Rust
• Partner directly with the CTO to define the technical roadmap and migration strategy
• Restructure and modernize the codebase for performance, safety, and maintainability
• Lead a gradual, stability-first rollout across a system processing 1B+ requests/month
• Drive performance testing, latency tuning, and throughput optimization for enterprise-grade reliability
What We're Looking For
• 2–5 years of production engineering experience, including hands-on Rust
• Experience migrating a system to Rust, or building production Rust services from scratch
• Hands-on experience with performance workload testing, latency tuning, or throughput optimization
• Prior exposure to AI infrastructure or API gateways
• Familiarity with backend infrastructure such as PostgreSQL and Redis
• A degree in Computer Science, ECE, EECS, or Information Systems
• Strong collaboration skills — you'll be working closely with the CTO day to day
Why Join
• Own the core architecture: lead the migration that will shape the foundation of the industry-standard AI gateway
• High-impact visibility: work directly with the CTO on a product processing 1B+ requests/month
• Scale and performance: solve hard low-latency problems inside a profitable, fast-growing company
