Skip to content

Externalize embedding generation

Chris Zubak-Skees requested to merge improve-embedding-speed into main

This MR helps address meta#61 (closed) by using an external API hosted on RunPod which has access to GPUs to speed up generating embeddings. Companion MR to ots/mediawiki/semantic-search!3 (merged)

Steps to Test

Define the following in .env.local with an OpenAI-compatible API and key:

EMBEDDING_MODEL=Snowflake/snowflake-arctic-embed-m-v1.5
EMBEDDING_API_BASE=
EMBEDDING_API_KEY=
make run

Use the API console to make an embedding request.

Merge request reports

Loading