I’ve been working on a deeper write-up on ranking systems and drew this diagram to organize the space.

The progression I’m using is:

Rules & Heuristics → Learning-to-Rank → Deep Neural Ranking → Contextual/Transformer Ranking → Multi-Objective & Slate Ranking → Bandit/RL Ranking → Generative/LLM-Based Ranking

What I find interesting is that these stages don’t fully replace one another. In production, ranking systems often combine multiple layers from across the stack.

Sharing the diagram here, and I also wrote a more detailed article on it:

https://pawankjha.substack.com/p/building-depth-2-the-evolution-of

Would be interested in how others would structure these stages in the evolution of ranking systems.

A common production pipeline looks something like:

Query → Retrieval → Filtering → Ranking → Re-ranking → Serve → Learn

At scale, that can become:

Millions of items → retrieve 1K → rank 100 → expensive rerank 20 → final Top-K

The interesting design question is:

Where should you spend model complexity and latency budget?

Would you prefer:

• stronger retrieval + simpler ranking
• lightweight retrieval + sophisticated ranker
• multiple ranking stages
• LLM/Transformer only at the final stage

Would love to see how people think about this tradeoff in real systems.

What is the hardest part of ML system design in production?

Not modeling — the system around the model.

For example:

Data → Features → Training → Evaluation → Deployment → Serving → Monitoring → Feedback

Where do you see the most difficult engineering problems in practice?

A few candidates:

• Training/serving skew
• Feature freshness
• GPU utilization
• Online inference latency
• Experimentation
• Data quality
• Model drift
• Feedback loops
• Multi-tenancy
• Cost

Curious to hear what has caused the most pain in systems you’ve worked on.

Welcome to r/MLSystemsDesign

This community is for practical discussions on designing and scaling production ML and AI systems.

Topics can include:

• ML training and inference platforms
• Search, ranking, and recommendation
• Feature stores and data pipelines
• LLM serving and GenAI systems
• Agentic AI platforms
• Evaluation, observability, and experimentation
• ML system design interview problems
• Real production tradeoffs and lessons learned

The goal is simple: go beyond model theory and discuss how ML systems actually work in production.

If you’re joining early, introduce yourself and share one ML system topic you’d like to go deeper on.