1. Infrastructure Layer
Role: Provision, monitor, and manage the scalable infrastructure required for model deployment and data processing.
Roles Involved:
- DevOps Engineer: Automates deployment, manages cloud and container infrastructure (Kubernetes, Docker)
- MLOps Engineer: Manages model serving, versioning, and monitoring
- DataOps Engineer: Ensures reliable and reproducible data pipelines
| Responsibility | Tools / Tech | Roles Involved |
| Compute Management | SkyPilot, Kubernetes, Ray | DevOps, MLOps |
| Data Management | Feast, LakeFS, Airbyte | DataOps, MLOps |
| Model Serving | vLLM, Triton, TorchServe | MLOps, DevOps |
| Monitoring & Logging | Prometheus, Grafana, Arize | MLOps, DevOps |
| Vector Search / DB | FAISS, Milvus, Qdrant | MLOps, LLM Engineer |
2. Model Development Layer
Role: Build, fine-tune, and evaluate ML/LLM models.
Roles Involved:
- ML Engineer: Core model development and experimentation
- LLM Engineer / Researcher: Works on large models, prompt tuning, and embeddings
- MLOps Engineer: Tracks experiments, streamlines training-to-deploy flow
| Responsibility | Tools / Tech | Roles Involved |
| Dataset Engineering | Label Studio, Snorkel, cleanlab | ML Engineer, DataOps |
| Modeling & Training | Transformers, PyTorch, Lightning | ML Engineer, LLM Engineer |
| Inference Optimization | ONNX, TensorRT, GGUF, vLLM | MLOps, ML Engineer |
| Evaluation | RAGAS, TruLens, EvalLM, Promptfoo | ML Engineer, LLM Engineer |
| RAG/LLM Interface Dev | LangChain, LlamaIndex, Haystack | LLM Engineer, MLOps |
3. Application Layer
Role: Deliver usable AI experiences via APIs, UIs, agents, and interfaces.
Roles Involved:
- AI Application Developer: Builds interfaces and workflows
- Prompt Engineer / LLM Engineer: Refines model interaction
- Full-Stack Developer: Integrates front-end + API with AI
| Responsibility | Tools / Tech | Roles Involved |
| Prompt Engineering | PromptLayer, Promptfoo, OpenAI Evals | LLM Engineer, Prompt Eng. |
| Agentic Orchestration | LangChain, CrewAI, DSPy, Flowise | LLM Engineer, AI Developer |
| UI / API Development | FastAPI, Streamlit, Gradio, Next.js | Full-stack Dev, AI Dev |
Cross-Cutting Functions
| Task | Tools / Tech | Roles Involved |
| Experiment Tracking | MLflow, Weights & Biases | MLOps, ML Engineer |
| Model/Data Versioning | DVC, LakeFS, Feast | MLOps, DataOps |
| CI/CD Automation | GitHub Actions, KubeFlow, DVC | MLOps, DevOps |
| Guardrails & Testing | Guardrails AI, Rebuff | Prompt Engineer, QA, MLOps |
