Pattern Hiring LLM Ops Engineer

LLM Ops Engineer

Company: Pattern
Location: Pune, IN
Job Type: Full-Time
Posted on: Posted Today
Job Requisition ID: R-101452

---

Role Overview:

Monitor, evaluate, and optimize AI/LLM workflows in production environments. Ensure reliable, efficient, and high-quality AI system performance by building out an LLM Ops platform that is self-serve for the engineering and data science departments.

---

Key Responsibilities:

• Collaborate with data scientists and software engineers to integrate an LLM Ops platform (Opik by CometML) for existing AI workflows.
• Identify valuable performance metrics (accuracy, quality, etc) for AI workflows and create ongoing sampling evaluation processes using the LLM Ops platform that alert when metrics drop below thresholds.
• Cross-team collaboration to create datasets and benchmarks for new AI workflows.
• Run experiments on datasets and optimize performance via model changes and prompt adjustments.
• Debug and troubleshoot AI workflow issues.
• Optimize inference costs and latency while maintaining accuracy and quality.
• Develop automations for LLM Ops platform integration to empower data scientists and software engineers to self-serve integration with the AI workflows they build.

---

Requirements:

• Strong Python programming skills.
• Experience with generative AI models and tools (OpenAI, Anthropic, Bedrock, etc).
• Knowledge of fundamental statistical concepts and tools in data science such as: heuristic and non-heuristic measurements in NLP (BLEU, WER, sentiment analysis, LLM-as-judge, etc.), standard deviation, sampling rate, and a high-level understanding of how modern AI models work (knowledge cutoffs, context windows, temperature, etc).
• Familiarity with AWS.
• Understanding of prompt engineering concepts.
• Strong people skills: you will be expected to frequently collaborate with other teams to help perfect their AI workflows.

---

Experience:

• 3-5 years of experience in LLM/AI Ops, MLOps, Data Science, or MLE.

---

Pattern is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees.

---

Follow Us:

@Instagram
@Youtube
@Telegram Channel
@LinkedIn

---

#AIJobs #LLMEngineer #MLOps #AIWorkflows #GenerativeAI #PatternJobs #PythonJobs #AIProgramming #DataScience #TechCareers