Job Details

Software Engineer III (Lead IC – GenAI / RAG Systems)

San Bruno, CA - Hybrid (3 days onsite)

Type: Contract (6 months, high potential for extension)

Role Overview

We are looking for a highly skilled Software Engineer III (Lead Individual Contributor) to productionize high-impact AI prototypes into scalable, production-grade systems. This role sits at the intersection of engineering and product, focusing on improving system efficiency, reliability, and developer workflows through advanced AI solutions.

You will take ownership of transforming experimental tools into robust, production-ready architectures, while building strong evaluation frameworks and ensuring high-quality model outputs.

Key Responsibilities

• Architect and productionize AI-powered systems, transitioning prototype tools into scalable production environments
• Design and implement context retrieval systems (RAG) with optimized context construction strategies
• Build and own evaluation frameworks from scratch, focusing on precision, recall, cost, and model performance
• Develop and optimize embedding strategies (dense, sparse, hybrid) and re-ranking mechanisms
• Build data pipelines to maintain real-time semantic indexing as underlying data evolves
• Define and enforce system requirements for accuracy, determinism, and reliability
• Collaborate cross-functionally with engineering, product, security, and privacy teams for successful launches
• Lead system design discussions, break down complex problems, and drive architecture decisions
• Debug and analyze large-scale datasets using SQL to identify edge cases and improve model behavior
• Ensure engineering excellence through code reviews, testing (integration, performance, stress), and system monitoring
• Mentor engineers and contribute to technical roadmap and long-term system strategy

Required Skills & Experience

• 8+ years of software engineering experience with strong fundamentals in data structures and algorithms
• 3–5+ years of experience building context retrieval / RAG-based systems for LLM applications
• Strong experience with GenAI systems using pre-trained models (e.g., Gemini or equivalent), including fine-tuning via evaluations
• Deep understanding of:
• Evaluation metrics (precision, recall, cost trade-offs)
• Semantic search and vector space models
• Embedding strategies and ranking systems
• Proficiency in SQL for large-scale data analysis and debugging
• Experience designing and building backend systems using Python or Go
• Experience with frontend technologies such as Angular, TypeScript, or JavaScript
• Familiarity with cloud platforms (e.g., GCP) and distributed systems
• Experience with Docker, Kubernetes, or similar deployment frameworks

Preferred Qualifications

• Experience building end-to-end AI applications such as chatbots or context-aware systems over large datasets
• Strong background in system design and architecture for scalable AI systems
• Experience working in cross-functional environments bridging product and engineering
• Prior experience leading large-scale technical initiatives or acting as a technical SME

What You'll Bring

• Ability to independently own and drive complex systems from concept to production
• Strong problem-solving skills with a data-driven approach
• Experience balancing short-term delivery with long-term scalability
• Leadership mindset as a Lead IC, influencing architecture and engineering direction

Thanks,

Nandit