AI, Data & Research - AI & Machine Learning
AI Prompt Engineer
IC / Practitioner
Independently auditedBuilds systems that learn, and keeps them honest in production. Owns a defined area of the function and the outcomes it produces.
Audited package
Built, independently audited role package. Everything on this page is written for this seat.
Your Stack (Swappable)
This role works against whatever tools you already run. We list the tool categories the role operates in, never a named vendor, so you are never asked to replace a system that works.
- AI / LLM Platform
- Experimentation & Feature Flags
- Structured-output / schema
- Observability & Monitoring
Deliverables
- Evaluation harnesses and results
- Model documentation
- Deployment and monitoring plans
- Capability assessments
Responsibilities
- Own the design and iteration of the prompts behind a feature or workflow - the system prompt, task instructions, and interaction pattern - and maintain them as versioned artifacts with a change history, not ad hoc strings
- Specify the structured-output contract: the JSON or function-call schema, the field semantics, the constrained-decoding or output-parser strategy, and the repair path so the model returns parseable, valid output the calling code can rely on
- Choose the context strategy: which few-shot examples, in what order, with which delimiters, and what grounding instructions - so the model is given what it needs to answer well and told how to use it (consuming the retrieval surface the integrations engineer provides, not building it)
- Build and own the evaluation harness for response quality: a labeled eval / golden set, the scoring method (exact-match, rubric, or LLM-as-judge with a calibrated rubric), and a regression suite that runs on every prompt or model change
- Reduce hallucination and unfaithful output by design - grounding and citation instructions, "answer only from the context" constraints, refusal calibration, and judge checks for faithfulness - and measure the reduction rather than assert it
- Diagnose a quality regression to its cause (a prompt change, a silent model update, a context-assembly shift, or an eval-set gap) and produce the specific fix or the eval that would have caught it
- Tune model-behavior parameters that affect output quality (temperature, sampling, stop sequences, output length) against the eval set, and document the setting and its rationale
- Version, A/B, and roll out prompt changes gated on eval and judge results, with a rollback that is one version away, so a "better" prompt is proven before it reaches users
The essence of this seat
The AI Prompt Engineer owns the quality of what a language model is asked and what it gives back.
Signature move
Define "good" before you optimize toward it. The first move on any quality task is to write the rubric - what correct, faithful, and well-formed mean for this task - because an unmeasurable goal produces a prompt that passes no gate and satisfies no one
What good looks like
refuses to tune to a hand-picked sample; clusters failures into modes, builds a representative labeled eval set with a rubric, and baselines before iterating; treats each prompt change as a measured hypothesis that must not regress passing cases; delivers the eval set and baseline as the durable asset, not just a nicer-looking prompt
Decision rights
- - the prompt design and its versions, the structured-output schema and field semantics, the few-shot and context strategy (what goes in the window and how it is framed), the output-quality-affecting parameters (temperature, sampling, stop sequences, output length), the eval-set design and the scoring method, the judge rubric and its calibration, and the regression gate a prompt or model change must pass - the what-goes-in and how-good-the-output-is
Working vocabulary
What this role does
The AI Prompt Engineer owns the quality of what a language model is asked and what it gives back. Their surface is the prompt and the interaction pattern: the system and task instructions, the structured-output and schema contract, the few-shot examples and the context that gets assembled into the window, and the evaluation harness that says - with evidence, not vibes - whether the output is good enough to ship and whether the last change made it better or worse. They treat a prompt as an artifact to be versioned, measured, and regression-tested, not a string that gets tweaked until a demo passes. This is an individual-contributor role, reporting into an AI Engineering Manager or Data Manager, and accountable for output quality: correctness, faithfulness to the provided context, format adherence, and the systematic reduction of hallucination and malformed responses. The line that keeps this seat honest is that it owns what goes into the model and how good what comes out is - and stops there. It does not own the software that makes the call reliable, cheap, and fast (that is the AI Integrations Engineer, role 013), the shipped product feature end to end (that is the AI/ML Engineer, role 010), the choice of approach and architecture across teams (that is the AI Solutions Architect, role 015), the training of a custom model (that is the Machine Learning Engineer, role 011), or classical statistical experimentation (that is the Data Scientist, role 006). What separates great from average here is refusing to trust a prompt that "looked good the three times I ran it": the strong prompt engineer builds a labeled eval set first, defines what "good" means as a measurable rubric, and lets the numbers - pass rate on the golden set, judge scores, regression deltas - decide, so a prompt change is a hypothesis tested rather than a hope shipped.
Best-fit customer
teams putting models into production and needing evaluation discipline
Need more than one role?
Buying the whole team or the complete department lowers your effective cost per role at every tier of the ladder.