Senior Engineer, AI and Agents
Location:
Remote / Hybrid (flexible)
About Falcon
Falcon Labs turns the post-transaction experience into an incremental revenue stream for online services and a rewarding moment for their customers. Our solution uses advanced technology and AI to deliver personalized, contextually relevant offers to consumers at their peak engagement moment, right after they complete a transaction.
The moment after a transaction is one of the most underutilized surfaces in commerce: intent is high, trust is established, and attention is real, yet for most brands it still ends with a static confirmation screen. We operate across major e-commerce ecosystems (Shopify, Wix, and a growing set of technology partnerships), earn a take rate on advertiser conversions, and run on real-time decisioning, Snowflake analytics, and an expanding push into agentic automation across the network. We are a team of ad-tech veterans from the mobile advertising world (ironSource/Unity).
The position
Full time. Senior individual contributor, reporting to the VP R&D and working closely with our Director of AI Enablement.
About the role
Agents already do real work across our company, and we keep adding more. We need a senior engineer who can build agents, run them reliably, and tell from the evidence whether a change made them better or worse. This is a senior engineering role first. Agents are where you start, and you pick up backend and platform work when the team needs it.
What you’ll do
Build and run the platform our agents live on: isolated runtimes, access control, cost limits, and the deployment and CI around them.
Instrument every agent with tracing, so any run can be replayed step by step, from prompts and tool calls to intermediate state and the final result.
Read trajectories to find where an agent went wrong, whether in the prompt, the context, a tool, the model or the data, and fix it.
Build evals that gate changes: golden datasets, step-level and end-to-end checks, LLM-as-judge graders calibrated against human review, and regression suites that run in CI.
Turn production failures and user feedback into new eval cases, and track quality, latency and cost per agent over time.
Work with our Director of AI Enablement and the teams who use the agents to turn manual work into automation people trust.
Later on
Customer-facing AI, where the bar for reliability, latency and cost is higher. Backend projects when the team needs another senior engineer.
Requirements
Must have:
LLM agents you built and ran in production: tool use, multi-step workflows, context management, cost and latency budgets.
Hands-on agent evaluation, including trajectory and task-success evals, LLM-as-judge and regression testing. You can explain how you know an eval measures the right thing.
Tracing and observability for agents, with OpenTelemetry or tools like Langfuse, LangSmith, Braintrust or Arize. You have debugged a failure from a trace.
Strong backend engineering in TypeScript or Python, and hands-on AWS: containers, IAM, secrets, infrastructure as code.
Comfort with data. You can write the SQL that shows whether an agent actually helped.
Nice to have:
Go, Terraform, MCP servers, LLM gateways and model routing, agent sandboxing, comparing models on accuracy, cost and latency, adtech or marketplace experience.
How we work with AI
AI is part of how we build every day. Coding agents write most of our code, and agents already handle real parts of our engineering and operations work. When a task comes in, we first ask how to automate it, where an agent fits in the loop, and how to get it to production through agents. We’re looking for people who already work this way. In this role, that is the job itself.
How to Apply
Send your CV and a short cover note to: