-
Notifications
You must be signed in to change notification settings - Fork 150
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move embeddings & WikipediaSimilaritySearchTool from API to FW #166
Comments
We would need to extend the |
Ref: #166 Signed-off-by: Tomas Dvorak <toomas2d@gmail.com>
Ref: #166 Signed-off-by: Tomas Dvorak <toomas2d@gmail.com>
Ref: #166 Signed-off-by: Tomas Dvorak <toomas2d@gmail.com>
Such a tool can now be created by using the However the functionality is still blocked by #176 |
Embedding has been added and functionality has been refactored in the API. |
Is your feature request related to a problem? Please describe.
The WikipediaSimilaritySearchTool and the related embedding capabilities are very useful when working with smaller LLMs. For example with granite bee its really the only way to use the Wikipedia tool. Granite will eventually get a larger context window but the ability to chunk/embed/retrieve seems generally useful for tool developers, and provides some core building blocks for optimized tools.
Right now if I want to provide a granite bee + wikipedia example in the FW I cant use the WikipediaSimilaritySearchTool.
Describe the solution you'd like
Move embeddings & WikipediaSimilaritySearchTool from API to FW to make the functionality available at lower level in stack.
Describe alternatives you've considered
Summarization of wikipedia article using an llm post processing step.
Additional context
None.
The text was updated successfully, but these errors were encountered: