Skip to content

Batch semantic search indexing

Chris Zubak-Skees requested to merge improve-semantic-search-indexing-3 into main

This MR addresses ots/llm/meta#61 by batching embedding requests during indexing for Semantic Search so that we can take advantage of RunPod GPUs. It is a companion MR to ots/llm/llm-api!62

Steps to Test

Follow the steps in ots/llm/llm-api!62.

In settings.py, make sure SEMANTIC_SEARCH_EMBEDDING_API_BASE_URL is pointed at http://localhost:8889 or equivalent.

pipenv install -e ../semantic-search/django-torque-semantic-search
pipenv run python manage.py shell
from torque import models
models.WikiConfig.objects.get(collection__name="DemoView", group="TorqueAdmin").rebuild_search_index()
Edited by Chris Zubak-Skees

Merge request reports