Skip to content

Externalize embedding generation

Chris Zubak-Skees requested to merge improve-embedding-speed into main

This MR helps address meta#61 by using an external API hosted on RunPod which has access to GPUs to speed up generating embeddings. Companion MR to ots/mediawiki/semantic-search!3

Steps to Test

Define the following in .env.local with an OpenAI-compatible API and key:

EMBEDDING_MODEL=Snowflake/snowflake-arctic-embed-m-v1.5
EMBEDDING_API_BASE=
EMBEDDING_API_KEY=
make run

Use the API console to make an embedding request.

Merge request reports