Smart Start can generate impossible filter sets
A reasonable guiding principle for the LLM-driven Smart Start is that it should not be able to generate a combination of filters that no human could generate by manually clicking around the interface. If the LLM can't be prevented from generating such a combination, some part of the system should recognize it happened and
For example, @kfogel entered a search string which generated the following combination of filters:
-
Key Words:
- India
- Waste
-
Primary Subject Area:
- Clean water supply
- Irrigation and water management
- Water access, sanitation, and hygiene
- Water conservation
- Water pollution
- Water resources
- Water utilities
However, if a human (including but not limited to @kfogel) went to Explore and manually selected the "India" and "Waste" keywords, the Primary Subject Area options available to them would be limited to these:
That's because the other subject areas, which the LLM decided were relevant to the query, don't intersect with Keywords=India,Waste
in the actual database — but the LLM doesn't know that.
In fact, for a real user to "back out" of the LLM response and get back to any usable responses, they would have to deselect all of the suggested Primary Subject Areas: seven clicks and queries before getting any results at all.
This issue is an expression of the problem.