PrivateGPT: Chat to Your PDFs Offline

[sarutobiumon]

Plz consider integrating this code for offline chatting with files like PDF's.
https://github.com/imartinez/privateGPT
demo:
https://www.youtube.com/watch?v=xs9TnY2z8jE

[ccelik97]

+1. Once it supports GPU acceleration it'll be big. I already find Semantra really useful for more efficiently searching through documents via transformers so this may be the next logical step from there really.

[Trahloc]

Yeah just saw AItrepreneur's video on this and it looks awesome. Relying on the CPU is kinda lame but integration to Oob would be awesome.

[St33lMouse]

It needs GPU support, quantization support, and a gui. At that point, you could take an entire library of .epub books, ingest them all, and the AI would have access to your whole library as hard data. We're all going to be passing around epub libraries to each other soon.

Looking at the project, it's been a few been a few days since anything changed, so development over there is moving at a moderate pace. We won't be getting these things tomorrow, but they will come fairly soon.

The possibilities are astounding. New information does not have to be trained into the AI with a laborious process--you just drop a new document into the source folder, "ingest" it, and now the AI knows the document, like one of the characters in the Matrix sucking up the helicopter flight manual in 3 seconds.

Extensions can be written to allow the AI access to daily news so it can be up to date on current events. This should come pretty quickly as well. A few months, perhaps?

With news piped into your computer on a daily basis, the information and culture wars which dominate the public square will intensify dramatically. You do not want your computer's "mind" infested with false information--it is an extension of YOUR OWN mind. Efforts to poison the well, spin the news and modify history will be stepped up by the usual suspects. Big Tech AIs are untrustworthy and will become ever more so as time passes.

Genuinely trustworthy sources of information are hard to sort out, and a disturbingly large percentage of the population seem quite happy to lap up security state lies or whatever other garbage they find psychologically comforting. Still, there is a powerful hunger for integrity and honesty which will elevate honorable people and trustworthy sources to great status in the eyes of millions.

I wonder how far this particular advance can go? Can it deal with a huge library of information well? What does it do with conflicting information in different documents?

[shotintoeternity]

Hi, I'm extremely interested in this potential solution considering the ease of use of Oobagooba along with the need to supplement LLMs with confidential data on my network. Is there any simple way to integrate PrivateGPT or a similar solution into Oobagooba (https://github.com/oobabooga/text-generation-webui)? Thanks!

[ccelik97]

This question still being up like this makes me feel awkward about the whole "community" side of the things. If only I could read the minds of the developers behind these "I wish it was available as an extension" kind of projects lol.

The last words I've seen on such things for oobabooga text generation web UI are:

• The developer of marella/chatdocs (based on PrivateGPT with more features) stating that he's created the project in a way that it can be integrated with the other Python projects, and he's working on stabilizing the API. (here)
• @oobabooga (on r/oobaboogazz) expressing himself that he's interested in/open to such ideas.

[TomLucidor]

Khoj and GPT4All's UI both deserve some mention since they can juggle documents (websites and PDFs included) but they have yet to master Autonomous Agents for searching the web and crawling whole blogs. (sorry crossing idea into #3203)

[velyan]

I built Collate to chat with PDFs offline. It uses an embedded quantized LLM