Skip to content

Upgrade embedding model to support longer contexts

Chris Zubak-Skees requested to merge upgrade-embedding-model into main

This MR enables an upgrade to the Snowflake Arctic Embed M v2.0 model released last week, which promises longer context lengths (up to 8,192 tokens), foreign language support and better performance.

To test it, update .env.local to:

EMBEDDING_MODEL=Snowflake/snowflake-arctic-embed-m-v2.0

On deployment, we need to update the setting used in production, tweak the cosine threshold in Torque's Semantic Search configuration (currently testing SEMANTIC_SEARCH_SIMILARITY = 0.87) and reindex the documents, so this shouldn't be merged until we can do that.

Once all that's done we can close meta#62

Edited by Chris Zubak-Skees

Merge request reports

Loading