The job, in the employer's words
Role Summary
You'll design and run AI data projects and build the automation behind them, combining deterministic tooling with LLM agents to deliver solutions in quality management, project management, and data analytics.
Key Responsibilities
Build hybrid automation: Pair deterministic components (scripts, validation rules, trackers, scheduled jobs) with stochastic ones (LLM agents that score, classify, check, report, etc.). Decide which steps must be rules and which can be models, and design the hand-offs between them.
Set up and run agents in Claude and in our native enterprise agentic environment: task specs, connections to data and tools, schedules, monitoring and failure handling.
Quality management: Design QA workflows, guidelines, rubrics and scorecards; calibrate automated checks against human reviewers; turn review findings into error categories you can measure and fix.
Project management: Run data collection and annotation projects end to end (scoping, vendors, workflows, tracking, delivery) and build the tooling that lets coordinators run the day.
Data analytics: Collect, clean, and analyse production and quality data (throughput, rework, error rates, contributor performance) and turn it into reporting that teams and clients act on.
Verify everything: Test automation before release and build continuous auditing parallel workflows to ensure quality.
Document so others can run it: short specs, tickets and runbooks that let coordinators and engineers operate and extend what you build.
Required Qualifications
Intermediate JavaScript and Python: enough to build and maintain automation in production, not just prototypes.
Hands-on agent building in Claude (projects, skills, connectors, scheduled tasks), and the ability to do the same in our native enterprise agentic environment (connecting different APIs with our native infrastructure).
LLM evaluation experience: prompt and rubric design, calibration against human judgement, and the skepticism to catch a confident wrong answer.
A data analytics background, with experience in the data collection or annotation industry.
Experience managing projects with external vendors or contributors and multiple stakeholders.
Comfort making calls on incomplete information, and knowing how to make and own your decisions.
Requirements mentioned
- Intermediate JavaScript and Python: enough to build and maintain automation in production, not just prototypes.
- Hands-on agent building in Claude (projects, skills, connectors, scheduled tasks), and the ability to do the same in our native enterprise agentic environment (connecting different APIs with our native infrastructure).
- LLM evaluation experience: prompt and rubric design, calibration against human judgement, and the skepticism to catch a confident wrong answer.
- A data analytics background, with experience in the data collection or annotation industry.
- Experience managing projects with external vendors or contributors and multiple stakeholders.
- Comfort making calls on incomplete information, and knowing how to make and own your decisions.
No Mondays shows this listing as the employer published it. Check the details on the employer's site before you apply.