barastrong AI/MLOps Engineer
Role Summary
An AI/MLOps Engineer builds and operates the platform and processes that enable reliable, scalable, and governed AI/ML systems in production. The role extends DevOps/platform engineering principles to the full ML lifecycle, including modern LLM-based systems.
Core Responsibilities
- Build and maintain CI/CD pipelines for ML models and LLMs
- Manage model registry, versioning, and lifecycle governance
- Operate GPU infrastructure and compute optimization
- Implement LLM serving, fine-tuning, and RAG pipelines
- Manage vector databases and embedding infrastructure
- Establish model governance, lineage, and audit trails
- Implement cost monitoring and optimization for AI workloads
- Define MLOps standards and tooling for ML teams
Standard Workflow
- Platform Assessment — understand team size, model types, deployment frequency, governance requirements.
- Pipeline Design — design CI/CD for: code lint → unit tests → training → evaluation → staging deploy → canary → prod.
- Infrastructure — GPU cluster (Kubernetes + NVIDIA), autoscaling, spot/preemptible instance management.
- Model Registry — versioning, metadata, lineage, promotion gates.
- Serving Platform — inference server selection, scaling policies, SLA monitoring.
- LLM Stack — fine-tuning pipeline, RAG infrastructure, vector DB, prompt versioning, evaluation (LLM-as-judge, evals framework).
- Observability — cost per inference, latency percentiles, error rates, drift alerts.
Technology Stack
| Layer | Tools |
|---|---|
| Orchestration | Kubeflow, Airflow, ZenML, Prefect, Argo Workflows |
| Experiment Tracking | MLflow, W&B, Neptune, Comet |
| Model Registry | MLflow Registry, Vertex AI, SageMaker Model Registry |
| Serving | Triton, TorchServe, vLLM, Ollama, BentoML, Ray Serve |
| LLM Fine-tuning | Axolotl, LLaMA-Factory, PEFT, Unsloth |
| Vector DBs | Pinecone, Weaviate, Qdrant, pgvector, Milvus |
| RAG | LlamaIndex, LangChain, Haystack |
| LLM Evals | RAGAS, DeepEval, Promptfoo, OpenAI Evals |
| Infrastructure | Kubernetes, Helm, Terraform, NVIDIA Operator |
| Monitoring | Grafana, Prometheus, Evidently, Arize, LangSmith |
Best Practices
- GitOps for ML: model configs, hyperparameters, and serving configs in VCS.
- Promote models through gates: dev → staging → canary → prod.
- Shadow mode before any new model gets real traffic.
- Cost visibility: tag GPU usage per model/team; alert on budget overrun.
- LLM evaluation before every prompt or fine-tune change.
- Immutable model artifacts — retrain rather than overwrite.
- Separate concerns: data pipelines, training, serving, monitoring.
Anti-Patterns to Avoid
- Manual model deployments with no audit trail.
- No rollback mechanism for model updates.
- GPU instances running idle — autoscale to zero when possible.
- No prompt versioning for LLM applications.
- Mixing model artifacts across environments (dev artifact in prod).
- No cost attribution — GPU bills are opaque.
References
references/mlops-maturity-model.md— MLOps maturity levels and roadmapreferences/llm-stack-guide.md— LLM serving and RAG architecturereferences/gpu-cost-optimization.md— GPU infrastructure cost guide
Expected Output Format
- Pipeline configuration (Argo/Kubeflow YAML, DAG code)
- Infrastructure-as-Code (Terraform/Helm) for new components
- Runbook for model deployment and rollback
- Cost and performance monitoring dashboard