Job Details

Overview

Checksum is a high-growth startup revolutionizing software testing with AI-powered, self-healing QA automation. Checksum's platform eliminates the need for manual test maintenance by automatically generating and adapting tests based on real user behavior. Engineering teams can now release code faster without sacrificing quality—and our traction proves it: we've tripled ARR in the last 9 months. We are backed by Leap Global Partners and were formed at super{set}, whose co-founders have over $1B in exits to Microsoft and Salesforce.

We're looking for a builder who ships. You have hands on experience building with LLMs, shown through shipped features or serious prototypes. You have worked with prompts, function calling, tool use, structured outputs, evals and failure handling. You turn ambiguous problems into iterative plans, wire up tools and feedback loops, and measure what matters. You are comfortable moving between prompt design, Python code, and product tradeoffs. You enjoy pairing with engineers and PMs, mentoring teammates, and owning outcomes in a dynamic startup environment.

This is a hybrid role, 4x a week, in the San Francisco Office.

What You'll Do

• Design, build and operate AI agents that plan, act, observe and learn across browser, API and data tools
• Integrate models with tools such as Playwright, internal APIs, vector search, document stores and webhooks
• Create evaluation harnesses, golden datasets and automated regressions for reliability and safety
• Optimize latency, accuracy and cost using caching, routing, function calling, tool selection and streaming I/O
• Collaborate on product UX for agent feedback, interruptions, approvals and failure recovery
• Own the full lifecycle from prototype to production, including CI, experiments and on call rotations

What We're Looking For

• BSc in computer science or equivalent experience
• Ability to work independently, and solve problems based on clear goals
• Strong Python or TypeScript skills
• Experience building evals, datasets and automated tests for ML or LLM systems
• Experience building agents that control browsers with Playwright or Selenium
• Knowledge of CI/CD, containerization and cloud deployments
• Exposure to retrieval augmented generation, planning and memory, multi-agent orchestration and toolformer style designs
• Experience with Anthropic, OpenAI, Google and open source models such as Llama, plus model selection and routing
• Background in testing automation, QA engineering or developer tools

Benefits & Perks

• A strategic role in shaping the growth trajectory of a cutting-edge AI startup
• Competitive salary, equity, and benefits package
• Lunch every day you are in the office
• Monthly company happy hours

Details

• Seniority level: Entry level
• Employment type: Full-time
• Job function: Engineering and Information Technology
• Industries: Venture Capital and Private Equity Principals