Building the ML Stack of the Future

If you’ve read my prediction on which companies will vertically integrate the AI stack, you may have your own guesses.

But the real question is, what do you do with that information?

In this article you’ll build an end-to-end application that uses the following technologies:

Quick Definitions

Model Hub: A repository of self-contained deep learning models pre-trained for a wide variety of applications. They are typically low friction ways to integrate AI into your code with just a python import.

Serverless GPU: On-demand access to powerful GPU resources. GPUs are important for ML workloads because they enable high volume linear algebra, critical for ML.

ACID-Compliant Database: An ACID (Atomicity, Consistency, Isolation, Durability) compliant database provides strong guarantees for data integrity and consistency.

Vector Engine: A vector search engine enables efficient and accurate search and retrieval of high-dimensional data. This is through vector representations of data, which capture the semantic relationships between terms. More here:

Application Search: This is where the business value is presented. All the technology under the hood is useless if it doesn’t generate value to the end-user.

What are we building?

We’ll be building an application that generates AI portraits of deceased celebrities. It stores them in a database and makes them searchable via vector search:

Don’t try to purchase a print, it’s not ready :)

from diffusers import StableDiffusionPipeline, DPMSolverMultistepScheduler

prompt = "Steve Jobs portrait in Surrealism style"

def create_image():
model_id = "stabilityai/stable-diffusion-2-1"
pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16)
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config)
pipe ="cuda")

image = pipe(prompt).images[0]"steve_jobs.png")
Generated using Stable Diffusion on HuggingFace

import modal

stub = modal.Stub(name="")

def create_image():

import pymongo


mdb_url = ""
mdb_client = pymongo.MongoClient(mdb_url)['db']['collection']


You do need to store the file in a cloud store somewhere for reference, I’ll use S3:

import boto3

def save_to_s3(url, file_id):
r = requests.get(url, stream=True)


We’ll use the change_streams method to copy every change from MongoDB to keep our Weaviate database up-to-date.

import weaviate

client = weaviate.Client(weaviate_url)

pipeline = [{'$match': {'operationType': 'insert'}}]

# listen on changes to MDB
with as stream:

# each change will insert into Weaviate
for insert_change in stream:
with client.batch as batch:
batch.add_data_object(encoded_file_from_hf, prompt)

All of these technologies, wrapped up in a single Python library:

from mixpeek import Mixpeek

mix = Mixpeek(api_key="mixpeek_api_key")

mix.generate("image", "Steve Jobs in Surrealism style")"Jobs")

Congrats! You’ve successfully leveraged the ML stack of the future! Now that you’ve built this, you can start to ideate on how to do finance, education and healthcare use cases leveraging this advanced technology.

Questions/comments? Reach out to

I’m hosting a LIVE course that walks you through engineering the above in a 6 part stream:



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store