AI in Production

Deploying a Multistage Multimodal Recommender System on Amazon Elastic Kubernetes Service

A summary of a Towards Data Science walkthrough of a four-stage multimodal recommender (Two-Tower retrieval, Bloom filter, DLRM ranking, reranking) deployed on Amazon EKS.

Deploying a Multistage Multimodal Recommender System on Amazon Elastic Kubernetes Service

A summary of Mustapha Momoh's Towards Data Science article on deploying a multistage, multimodal recommender system on Amazon Elastic Kubernetes Service (EKS), with Bloom filters, feature caching and contextual recommendations.

The Four Stages

The system serves recommendations in four stages: a Two-Tower model generates candidate items (retrieval); a Bloom filter hides items the user recently interacted with (filtering); a DLRM ranker scores the remaining items using user, item and context features (ranking); and a final stage orders and samples from the scores to produce the recommendations (reranking).

Multistage recommender pipelineRetrievalTwo-Tower modelgenerates candidatesFilteringBloom filter hidesrecent interactionsRankingDLRM ranker scoresthe remaining itemsRerankingorders and samplesfrom the scores
Figure. The four serving stages described in the article. In the article they run as a single Triton ensemble model.

Multimodal Features

The models combine tabular collaborative features with precomputed CLIP image embeddings and Sentence-BERT text embeddings. In retrieval, the pretrained embeddings feed the candidate tower alongside learned item features, and the dot product of the query and candidate tower outputs is the relevance score. In the DLRM ranker, they enter the dot-product interaction layer, whose outputs feed the top MLP.

The Infrastructure

  • Amazon EKS runs the containerized workflows and scales compute with demand.
  • Kubeflow Pipelines manages training and daily fine-tuning on Kubernetes.
  • NVIDIA Merlin handles GPU-accelerated feature engineering, preprocessing and model training.
  • Triton Inference Server hosts the multistage serving graph as a single ensemble model.
  • FAISS provides approximate nearest neighbor search for candidate retrieval.
  • Feast manages user and item features across training and serving.
  • Amazon ElastiCache for Valkey (Redis) backs the online feature store, stores per-user Bloom filters and holds popularity counts.
  • Amazon Athena with S3 and Glue backs the offline feature store.

Two Pipelines

The first pipeline builds preprocessing, trains the models from scratch, builds the approximate-nearest-neighbor index and deploys Triton and the models. The second runs daily fine-tuning, which updates the query tower and the ranker but does not regenerate the item embeddings.

Two training and deployment pipelinesPreprocessing(NVIDIA Merlin)Train the modelsfrom scratchBuild theANN indexDeploy Tritonand the modelsDaily fine-tuningUpdates the querytower and the rankerItem embeddingsare not regeneratedPipeline 1: build and deployPipeline 2: daily fine-tuning
Figure. The two Kubeflow pipelines described in the article: the full build and deployment, and the daily fine-tuning.

The Data

The training data comes from a modified AWS Retail Demo Store interaction generator, scaled to 300,000 users and a catalog of 2,465 items that have images and descriptions. The dataset contains 13 million interactions over 14 days.

The article credits in-memory feature caching with a latency win; this summary does not quote a figure because none is given in the material it is based on.

Sources

Share

Subscribe for daily recipes. No spam, just food.