Senior Engineer, Data and ML Platform

The position

Full time. Senior individual contributor, reporting to the VP R&D and working day to day with our ML lead.

About the role

Our ranking model is only as good as the data under it. You own what it learns from: the raw sources, the pipelines, the tables it trains on and the monitoring around them.

This is a senior engineering role first. Data and ML are where you spend your first months. After that you work wherever the platform needs a strong engineer, including backend services and infrastructure.

What you’ll do

• Build and run batch pipelines on Spark and Databricks, from raw data to feature tables to training tables, including enrichment and backfills for late-arriving data.

• Bring in new data sources, from internal databases to external APIs, partner feeds and web data, and match them to our own entities.

• Use LLMs as a step in batch pipelines, for example to turn text into embeddings and structured attributes.

• Keep training and serving consistent. Features computed in training have to match what the inference service computes in production, which means reading its Java code.

• Watch data quality with scheduled checks and Grafana alerts that fire before bad data reaches the model. When something breaks, you find the cause.

Later on

• Wider data and analytics work, and backend and infrastructure projects as priorities shift.

Requirements

Must have

• Spark in production: complex transformations, window functions, large joins, partitioning, and tuning for shuffles and skew. Databricks experience (jobs, Delta tables) is a big plus.

• Snowflake in production, including SQL, loading and unloading, and cost.

• Experience with large data volumes, for example hundreds of millions of rows a month. You know how to partition, load incrementally and backfill at that scale without blowing up run time or cost.

• Strong Python. Clean, tested code, Spark UDFs including pandas UDFs, pandas and PyArrow.

• Experience building data quality checks, drift and anomaly detection, and alerts people act on.

• You deploy your own pipelines with GitHub Actions, Docker, AWS (S3, IAM) and proper secrets handling.

• Solid general backend engineering. You are comfortable in a service codebase, not only in notebooks and SQL.

Nice to have

• Scala for Spark, reading Java, Kafka or Spark Structured Streaming, Airflow or Databricks Workflows, dbt, Terraform, MLflow and feature stores, time-based splits and data leakage, adtech or e-commerce data.

How we work with AI

AI is part of how we build every day. Coding agents write most of our code, and agents already handle real parts of our engineering and operations work. When a task comes in, we first ask how to automate it, where an agent fits in the loop, and how to get it to production through agents. We’re looking for people who already work this way. In data work, that means agents on the warehouse and the pipelines, with you checking row counts, distributions and edge cases before anyone relies on the result.