资深应用AI科学家
查看雇主原标题
Staff Applied AI ScientistCulture Amp · Sydney
职位信息来自雇主公开的招聘页面。申请前请务必在雇主官网核实详情。
为什么值得关注？
发现指数 42/100，仅依据与该职位一起存储的证据计算。
- 新的雇主官方职位
分数构成
- 时效性 （随职位发布时间变化）+18
- 雇主官方来源+15
- 稀有职位+1
- 公司来源健康度+8
该职位未包含：已披露薪资、远程职位、提及签证担保、提及搬迁、未出现在监控的职位板上。
这些理由来自雇主自己的职位描述与我们核实过的来源检查结果。除了已存储的信号之外，我们不做任何推测。
职位描述
机器翻译我们坚信线下办公的力量，因此对于大多数岗位，我们要求 Campers 平均每周有 2 天在当地的 Culture Amp 办公室工作，以便共同解锁连接、节奏与文化。
加入我们，一起完成打造更美好工作世界的使命。
Culture Amp 是全球领先的员工体验平台，正在彻底改变 6,000 多家公司的 2,500 万名员工创造更美好工作世界的方式。Culture Amp 赋能各种规模和行业的公司，帮助他们转变员工敬业度、推动绩效管理并打造高绩效团队。依托人员科学和全球最全面的员工数据集，Canva、On、Asana、Dolby、McDonalds 和 Nasdaq 等最具创新力的公司每天都依赖 Culture Amp。
Culture Amp 由领先的风险投资基⾦⽀持，并在美国、英国、德国和澳⼤利亚设有办公室。Culture Amp 被 Forbes 评为全球顶尖私有云计算公司之一，并被 Fast Company 评为最具创新力公司之一。
如需了解更多信息，请访问 cultureamp.com。
你如何帮助打造更美好的工作世界
交付一个 AI 产品只是开始。更艰难的挑战——也是少数团队已经掌握的挑战——是持续的生产环境评估：实时诊断性能变化，解读其发生的“原因”，并大规模推动可衡量的质量改进。我们正在寻找一位具备深厚 AI Engineering 背景的 Staff 级 Applied AI Scientist，为我们的 Coach AI 系统解决这一问题，建立可观测性和评估框架，将早期生产版本转化为稳健、高性能的生产产品，并通过让工程组织中的其他团队也能做到这一点来实现可持续。
作为这个由优秀人才组成的团队的一员，
你将
• 负责端到端反馈闭环：建立严格的提示词工程、规模化评估和持续改进循环。你将构建由 LLM 驱动的分析工具，用于诊断性能变化、提供深入洞察，并自动给出提示词或系统级增强建议。
• 参与 Context engineering：设计和优化实际进入模型上下文的内容：检索、跨会话记忆、上下文组装与压缩，以及在长流程或多轮 agentic 流程中管理上下文预算。每个变更都要通过 eval 验证，而不是依赖观点或临时测试。
• 设计和运行 evals：在去标识化的生产 traces 上进行采样、LLM-as-a-judge 和标注系统（例如使用 Langfuse），以构建纵向评估监控和告警。
• 以 eval 驱动的 agentic orchestration：参与 agent 架构（规划、工具使用、路由、分解、验证/批判步骤），并让 eval 发现推动结构性变更——例如当某种失败模式出现时，增加自检步骤、更改工具选择或重新路由。
• 模型和供应商选择：根据质量、延迟和成本权衡做出并负责模型/路由决策，包括何时使用提示词、何时微调、何时更换模型。
• 在生产环境中创建并监控 guardrails 和 safety：鉴于 coaching 和人员数据的敏感性，将输入/输出 guardrails、PII 处理、内容安全和抗越狱能力作为系统的一部分进行设计。
• 赋能他人：通过可复用的框架、工具和文档，让产品和工程团队能够运行自己的评估。带头示范，然后交接。
• 紧密合作：与 AI Coach 团队、产品、数据科学和人员科学合作，使经过衡量的质量映射到真实的客户价值。
• 保持前沿：跟进最新的评估、可观测性和 LLMOps 研究及供应商产品。
你具备
• 构建并将生产级 agentic 系统转化为成果的经验，包括 context engineering、RAG、记忆、成本、模型选择和性能。
• 已证明的经验：分析生产环境中 AI 或数据产品的性能，并将其转化为维持和改进产品的变更。
• 在生产环境中实操 LLM 评估：LLM-as-judge、eval 数据集、human-in-the-loop 标注、按阈值评分。
• 具备 LLM 和 agentic 系统的 Observability 工具经验（traces、采样、提示词管理、生产监控，如 Langfuse 或同类工具）。
• 具备纵向测量经验：指标和基线、回归检测、随时间推移的质量跟踪。
• 日常以 AI 原生方式实践，能够熟练使用 agentic coding 工具（Claude Code、Cursor、Codex 或类似工具）处理多步骤任务，并清楚判断何时指挥 agent、何时自己编写代码。
• 出色的技术写作和沟通能力，并有将能力构建到系统中并教会他人掌握它的过往记录。
• 强信号：曾跨多个团队构建或扩展 eval 和可观测性实践；用 AI 演进现有企业代码库；生产级 agentic 系统（orchestration、RAG）；ML、CS、应用数学或相关领域的 postgraduate degree；在 eval、可观测性或 LLMOps 方面的公开写作、演讲或开源工作。
你是
• 受到在生产环境中有效扩展 AI 系统性能和采用度的激励，同时具备公开学习的谦逊和作为 self-starter 的韧性。
• 受到赋能他人的激励。你最大的胜利来自教导他人并将其构建到我们的系统中，这可能意味着你不会永远拥有你所构建的东西。
我们在 Culture Amp 的构建方式
在 Culture Amp，我们的工程师越来越多地在编排编写代码的 agents，而不仅仅是自己直接编写代码。我们引导、规划、构建和审查循环，让 AI 在常规工作中主动推进，使你能够把控架构、权衡和质量。我们正在投资一个共享的工具和标准“harness”，让 agents 能够安全地完成真实的产品工作，我们也都将这些能力视为我们交付方式的核心部分。
请注意：候选人必须在整个雇佣期间拥有在 Australia 合法工作的授权，该岗位设在我们 Melbourne 或 Sydney hub。
福利待遇
在 Culture Amp，员工是我们成功的核心。我们提供有竞争力的薪酬和旨在支持你工作与生活的整体奖励方案。其中包括：
• 通过我们的 Employee Share Option Program 获得股权，让你分享我们的长期成功
• 学习项目和 coaching，帮助你茁壮成长
• 每季度 refresh days、延长的年末假期以及每月津贴，支持你的身心健康和生活方式
• 从第一天起提供包容性的育儿假
• 一台 MacBook 和用于布置居家办公空间的预算，支持灵活性
• 每年五天社会影响日，用于回馈对你重要的事业
• 为你和家人提供医疗保险（仅限 US 和 UK）
我们的奖励旨在支持不同需求和人生阶段，并认识到对每个人来说最重要的事情可能各不相同。
研究表明，来自代表性不足背景的候选人如果不符合每一项要求，可能会犹豫是否申请，但你独特的经验很重要。如果你有兴趣加入我们，我们强烈鼓励你申请，帮助我们打造一个更多元、更有影响力的团队。
便利安排与数据隐私
如果你因残障在完成在线申请或参与面试流程时需要合理的便利安排或调整，请联系 accommodations@cultureamp.com，并说明你请求的便利安排或协助类型。请勿在此电子邮件中包含任何医疗或健康信息。Reasonable Accommodations 团队将及时回复你的电子邮件。
Culture Amp 将在你的申请流程完成之日起两年内（美国为四年）保留你的 CV 和个人信息。在此期间，Culture Amp 可能会就未来的工作机会与你联系。如需了解更多信息，请在此查看我们的隐私政策，或联系 privacy@cultureamp.com。
以上内容由机器翻译自动生成，可能存在错误；投递前请以雇主原文为准。
查看雇主原文
职位描述
We’re big believers in the power of IRL, so for most roles we ask Campers to work from their local Culture Amp office an average of 2 days a week to unlock connection, pace and culture together.
Join us on our mission to make a better world of work.
Culture Amp is the world’s leading employee experience platform, revolutionizing how 25 million employees across more than 6,000 companies create a better world of work. Culture Amp empowers companies of all sizes and industries to transform employee engagement, drive performance management, and develop high-performing teams. Powered by people science and the most comprehensive employee dataset in the world, the most innovative companies including Canva, On, Asana, Dolby, McDonalds and Nasdaq depend on Culture Amp every day.
Culture Amp is backed by leading venture capital funds and has offices in the US, UK, Germany and Australia. Culture Amp has been recognized as one of the world’s top private cloud companies by Forbes and most innovative companies by Fast Company.
For more information visit cultureamp.com .
How you can help make a better world of work
Shipping an AI product is only the beginning. The harder challenge, one few teams have mastered is continuous production evaluation: diagnosing performance shifts in real-time, decoding 'why' they occur, and driving measurable quality improvements at scale. We are looking for a Staff level Applied AI Scientist with a strong AI Engineering background to solve this problem for our Coach AI system, establishing the observability and evaluation frameworks that turn early production releases into robust, high-performance production products and then to make this sustainable by enabling the rest of our engineering org to do the same.
As part of this team of amazing humans,
You will
• Own the end-to-end feedback loop: establish a rigorous cycle of prompt engineering, evaluation at scale, and continuous improvement. You will build LLM-powered analysis tools that diagnose performance shifts, provide deep-dive insights, and automate recommendations for prompt or system-level enhancements.
• Contribute to Context engineering: design and optimise what actually enters the model's context: retrieval, memory across sessions, context assembly and compression, and managing context budget in long or multi-turn agentic flows. Validate each change against eval rather than opinion or adhoc testing.
• Design and run evals: sampling, LLM-as-a-judge, and labelling systems over de-identified production traces (for example, with Langfuse) to build longitudinal evaluation monitoring and alerting.
• Eval-driven agentic orchestration: contribute to the agent architecture (planning, tool use, routing, decomposition, verification/critique steps) and let eval findings drive structural changes — e.g. when a failure mode surfaces, add a self-check step, change tool selection, or re-route.
• Model and provider selection: make and own model/routing decisions against quality, latency and cost trade-offs, including when to prompt vs fine-tune vs swap models.
• Create and Monitor guardrails and safety in production: given sensitive coaching and people data, design input/output guardrails, PII handling, content-safety and jailbreak resistance as part of the system.
• Enable others: through reusable frameworks, tooling and documentation so product and engineering teams run their own evaluations. Lead from the front, then hand over.
• Partner closely: with the AI Coach team, product, data science and people science so measured quality maps to real customer value.
• Stay current: with the latest evaluation, observability and LLMOps research and provider offerings.
You have
• Experience building and turning production agentic systems, including context engineering, RAG, memory, cost, model selection and performance.
• Proven experience analysing the performance of AI or data products in production and turning it into changes that maintained and improved the product.
• Hands-on LLM evaluation in production: LLM-as-judge, eval datasets, human-in-the-loop labelling, scoring against thresholds.
• Experience with Observability tooling for LLM and agentic systems (traces, sampling, prompt management, production monitoring such as Langfuse or comparable).
• Experience with longitudinal measurement: metrics and baselines, regression detection, quality tracking over time.
• AI-native daily practice, comfortable using agentic coding tools (Claude Code, Cursor, Codex or similar) on multi-step tasks, with clear judgment on when to direct an agent versus write code yourself.
• Strong technical writing and communication, and a track record of building capability into systems and teaching others to own it.
• Strong signals: built or scaled an eval and observability practice across multiple teams; evolved existing enterprise codebases with AI; production agentic systems (orchestration, RAG); a postgraduate degree in ML, CS, Applied Maths or related; public writing, talks or open-source work in eval, observability or LLMOps.
You are
• Motivated by the effective scaling of AI system performance and adoption in production with the humility to learn in public and the resilience to be a self-starter.
• Motivated by enablement. Your biggest wins come from teaching others and building this into our systems, which can mean you do not own what you build forever.
The way we build at Culture Amp
At Culture Amp, our engineers are increasingly orchestrating agents that write code, rather than just writing it directly themselves. We guide, plan, build, and review loops where AI takes the initiative on routine work, allowing you to steer architecture, trade-offs, and quality. We're investing in a shared "harness" of tooling and standards so agents can do real product work safely, and we all embrace these capabilities as a core part of how we ship.
Please note: candidates must be legally authorised to work in the Australia for the duration of employment, the role is based out of our Melbourne or Sydney hubs.
福利待遇
At Culture Amp, our people are at the heart of our success. We offer competitive pay and a total rewards package designed to support you at work and in life. This includes:
• Equity through our Employee Share Option Program, so you can share in our long-term success
• Learning programs and coaching to help you thrive and grow
• Quarterly refresh days, an extended end-of-year break and a monthly allowance to support your wellbeing and lifestyle
• Inclusive parental leave from day one
• A MacBook and budget to set up your home workspace, enabling flexibility
• Five annual social impact days to to give back to causes that matter to you
• Medical insurance coverage for you and your family (Available for US & UK only)
Our rewards are designed to support different needs and life stages, recognising that what matters most can vary from person to person.
Research shows that candidates from underrepresented backgrounds may hesitate to apply if they don’t meet every requirement, but your unique experience matters. If you’re interested in joining us, we strongly encourage you to apply and help us build a more diverse and impactful team.
Accommodations & Data Privacy
If you require reasonable accommodations or adjustments due to a disability to complete the online application or to participate in the interview process, please contact accommodations@cultureamp.com and identify the type of accommodation or assistance you are requesting. Do not include any medical or health information in this email. The Reasonable Accommodations team will respond to your email promptly.
Culture Amp will retain your CV & personal information for a period of two years (four years for the US) from the date of your application process completion. Culture Amp may contact you in relation to future job opportunities during this time period. For further information please see our privacy policy here or contact privacy@cultureamp.com .