LLM: RAG-GPT

Sumber: https://github.com/gpt-open/rag-gpt?tab=readme-ov-file#using-local-llms

Buat API di LLamaCloud, login mengggunakan email

https://cloud.llamaindex.ai/api-key

Sebagai user biasa, clone RAG-GPT

git clone https://github.com/open-kf/rag-gpt.git && cd rag-gpt
cp env_of_ollama .env

Edit .env

LLM_NAME="Ollama"
OLLAMA_MODEL_NAME="xxxx"
OLLAMA_BASE_URL="http://127.0.0.1:11434"
MIN_RELEVANCE_SCORE=0.4
BOT_TOPIC="xxxx"
URL_PREFIX="http://127.0.0.1:7000/"
USE_PREPROCESS_QUERY=1
USE_RERANKING=1
USE_DEBUG=0
USE_LLAMA_PARSE=0
LLAMA_CLOUD_API_KEY="xxxx"

menjadi misalnya

LLM_NAME="Ollama"
OLLAMA_MODEL_NAME="llama3"
OLLAMA_BASE_URL="http://127.0.0.1:11434"
MIN_RELEVANCE_SCORE=0.4
BOT_TOPIC="OpenIM"
URL_PREFIX="http://127.0.0.1:7000/"
USE_PREPROCESS_QUERY=1
USE_RERANKING=1
USE_DEBUG=0
USE_LLAMA_PARSE=0
LLAMA_CLOUD_API_KEY="llx-HYs1KFrtKm5eeLAJeJW4goCnAaeKZeHd6RmjKSprrogxYdwP!"

Buat dari source

sudo apt install python3.12-venv
python3 -m venv myenv
source myenv/bin/activate
pip install -r requirements.txt
python3 create_sqlite_db.py

sh start.sh

Browser

http://ipaddress:7000

Referensi

• https://github.com/gpt-open/rag-gpt?tab=readme-ov-file#using-local-llms