GenAI Evaluation Product Owner

Job description

Who are we?

Joining NEXTON means joining a company that combines the mindset of a consulting firm, the creativity of a Factory, and the energy of a digital services company. We support major clients and digital-native companies such as SNCF, Orange, and BNP Paribas in their digital evolution and innovation strategies. Founded in 2011, NEXTON now brings together more than 450 experts in digital, design, Agile, product, and development.

NEXTON is hiring a full-time GenAI Evaluation Product Owner in Paris.

Your future work environment

We are hiring for a major banking client to help build its Group Agentic Stack. You will join a specialized team responsible for establishing continuous evaluation and approval of generative AI applications. You will play a key role and bring value to the product’s functional, methodological, and economic dimensions.

Your responsibilities

• Scope and manage automated evaluation mechanisms for GenAI solutions (golden datasets, technical and business metrics, regression tests, LLM-as-a-Judge approaches).
• Define the evaluation product vision, roadmap, backlog, use cases, and acceptance criteria in close coordination with IT teams.
• Help select evaluation components by balancing user needs with technical, security, operations, and cost constraints.
• Design outputs and key indicators: detailed views for engineering/Data Science and concise dashboards for business, risk, and approval bodies.
• Structure the evaluation platform’s service offering (services, access arrangements, service levels, responsibilities, and pricing model).
• Coordinate business, Data Science, IT, architecture, security, and compliance stakeholders around the product.
• Position the platform as an industrializable, traceable, essential quality gate before AI products enter production.

Profile

• Around 3–6 years’ experience as a technical Product Owner, with a significant role in evaluation, approval, and quality of AI/GenAI systems.
• Strong command of GenAI evaluation methods and frameworks (golden datasets, performance metrics, LLM-as-a-Judge, automated tests) and integration into CI/CD pipelines.
• Proficient with product-scoping tools, backlog management, roadmap development, and service-offering/pricing models.
• Experience assembling composite solutions (open-source components, external components, and internal solutions).
• Rigorous, strong synthesis skills, and comfortable coordinating a multidisciplinary ecosystem (Data Science, IT, business, security, compliance).

Why choose NEXTON?

We offer benefits and opportunities throughout the year to enrich your professional and personal experience:

• Active communities to support your career.
• Meetups to develop your skills.
• Training to help you grow and reinvent yourself.
• Sports and social events.
• Sustainable mobility allowance to support everyday travel and low-impact transport.
• A latest-generation phone to stay connected.

Ready to join us? Let’s meet! NEXTON.