Building the ML Stack of the Future

Ethan Steininger
3 min readFeb 1, 2023


If you’ve read my prediction on which companies will vertically integrate the AI stack, you may have your own guesses.

But the real question is, what do you do with that information?

In this article you’ll build an end-to-end application that uses the following technologies:

Quick Definitions

Model Hub: A repository of self-contained deep learning models pre-trained for a wide variety of applications. They are typically low friction ways to integrate AI into your code with just a python import.

Serverless GPU: On-demand access to powerful GPU resources. GPUs are important for ML workloads because they enable high volume linear algebra, critical for ML.

ACID-Compliant Database: An ACID (Atomicity, Consistency, Isolation, Durability) compliant database provides strong guarantees for data integrity and consistency.

Vector Engine: A vector search engine enables efficient and accurate search and retrieval of high-dimensional data. This is through vector representations of data, which capture the semantic relationships between terms. More here:

Application Search: This is where the business value is presented. All the technology under the hood is useless if it doesn’t generate value to the end-user.

What are we building?

We’ll be building an application that generates AI portraits of deceased celebrities. It stores them in a database and makes them searchable via vector search:

Don’t try to purchase a print, it’s not ready :)

Model Hub

from diffusers import StableDiffusionPipeline, DPMSolverMultistepScheduler

prompt = "Steve Jobs portrait in Surrealism style"

def create_image():
model_id = "stabilityai/stable-diffusion-2-1"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
pipe ="cuda")

image = pipe(prompt).images[0]"steve_jobs.png")
Generated using Stable Diffusion on HuggingFace

Serverless GPU

import modal

stub = modal.Stub(name="")

def create_image():

ACID-Compliant Database

import pymongo


mdb_url = ""
mdb_client = pymongo.MongoClient(mdb_url)['db']['collection']


You do need to store the file in a cloud store somewhere for reference, I’ll use S3:

import boto3

def save_to_s3(url, file_id):
r = requests.get(url, stream=True)


Vector Engine

We’ll use the change_streams method to copy every change from MongoDB to keep our Weaviate database up-to-date.

import weaviate

client = weaviate.Client(weaviate_url)

pipeline = [{'$match': {'operationType': 'insert'}}]

# listen on changes to MDB
with as stream:

# each change will insert into Weaviate
for insert_change in stream:
with client.batch as batch:
batch.add_data_object(encoded_file_from_hf, prompt)

Application Search Wrapped Up in a Single API

All of these technologies, wrapped up in a single Python library:

from mixpeek import Mixpeek

mix = Mixpeek(api_key="mixpeek_api_key")

mix.generate("image", "Steve Jobs in Surrealism style")"Jobs")

Congrats! You’ve successfully leveraged the ML stack of the future! Now that you’ve built this, you can start to ideate on how to do finance, education and healthcare use cases leveraging this advanced technology.

Questions/comments? Reach out to

Want LIVE guidance?

I’m hosting a LIVE course that walks you through engineering the above in a 6 part stream: