Applied AI Engineer

About the Client

Our client is an innovative software company focused on transforming complex, high-volume data into clear, actionable insights. Its platform empowers organizations to make informed decisions through real-time analytics, forecasting, and operational intelligence.

The company develops cloud-native solutions for large-scale data processing, streaming, analytics, and decision support, handling over one billion data updates per day from over 1000+ data sources daily.

The Role

Responsibilities

Own the end-to-end assessment framework for AI-generated responses, retrieval results, and contextual outputs, ensuring consistent quality and reliability.
Design benchmarks, automated test suites, and monitoring processes to identify hallucinations, factual inaccuracies, retrieval failures, and performance regressions.
Build and maintain scalable evaluation pipelines that continuously validate prompt, model, and data changes before release.
Lead prompt engineering and RAG optimisation initiatives, using evaluation outcomes to improve answer quality and system performance.
Collaborate with technical and non-technical stakeholders to translate domain expertise into repeatable testing criteria and measurable quality standards.
Provide actionable insights that influence engineering and product decisions, promoting AI reliability as a key product requirement.

Must-haves

Bachelor's degree in Computer Science, Data Science, Mathematics, or a related field.
5+ years of software engineering experience, including significant exposure to testing frameworks, quality systems, AI/ML tooling, or evaluation platforms.
Strong Python programming skills and experience developing production-grade evaluation, testing, or data-processing pipelines.
Deep understanding of LLM evaluation techniques, including benchmarking, hallucination detection, semantic similarity assessment, and RAG quality measurement.
Hands-on experience with evaluation frameworks such as RAGAS, LangSmith, PromptFlow, or similar tools.
Experience working with RAG architectures, vector databases, or data-intensive production environments.
Ability to design and implement quality frameworks in fast-changing, ambiguous environments while exercising sound judgement to separate meaningful findings from noise.
Strong stakeholder management, communication, and presentation skills, with the ability to work effectively with both technical and business audiences.
Experience operating in environments where data quality, governance, and accuracy are business critical.

Nice-to-haves

Familiarity with cloud platforms (AWS, Azure, or GCP).
Background in energy, utilities, commodities trading, or other data-intensive industries.

Benefits

Performance bonus of 10% to 15% of annual base salary
Flexible time-off policy with untracked vacation days

Selection Process

Interview Process & How to Prepare

Screening Interview (with Hiring Manager) Focus: culture fit, background, and your motivation. Be ready to walk through your CV and ask questions about the company.
Panel Interview (mixed US/RO panel, evening slot 17:00-18:00 start, ~2 hours)

Hour 1: Behavioral round covering conflict resolution and ownership of responsibilities. Prepare concrete examples from your own experience (STAR format works well).

Hour 2: Technical round focused on your expertise, project experience, and problem-solving approach. Expect discussions around technical challenges, key decisions, and your contributions.

Good to Know

Come prepared with real examples from your experience rather than hypothetical scenarios. During discussions, focus on explaining your thought process, key decisions, challenges encountered, trade-offs considered, and the impact of your work.

Applied AI Engineer

About the Client

The Role

Responsibilities

Must-haves

Nice-to-haves

Benefits

Selection Process

Good to Know

Explore further:

IT career perspectives

About EDUROM

For companies