posted 29th March, 2023 - 11:50, GPT4ALL launched 1 hr ago
What if we use AI generated prompt and response to train another AI
- Exactly the idea behind GPT4ALL, they generated 1 million prompt-response pairs using the GPT-3.5-Turbo OpenAI
API between March 20, 2023 and March 26th,
2023, and used this to train a large language model.
Work on prompt response model -
1. using dataset - Laion from transformers (huggingface) - https://huggingface.co/datasets/laion/OIG
2. randomly selecting stackoverflow questions for coding knowledge
3. using dataset bigscience 's bloomz p3 from transformers (huggingface) - https://huggingface.co/bigscience/bloomz-p3
2. randomly selecting stackoverflow questions for coding knowledge
3. using dataset bigscience 's bloomz p3 from transformers (huggingface) - https://huggingface.co/bigscience/bloomz-p3
RESULTS -
bells and some drumrolls
1. Amazing response with accuracy since the trained data is very fine tuned, i.e. running LLaMa on a 8GB RAM laptop possible.
bells and some drumrolls
1. Amazing response with accuracy since the trained data is very fine tuned, i.e. running LLaMa on a 8GB RAM laptop possible.
2. Able to produce these models with about
four days work, $800 in GPU costs and $500 in OpenAI API spend.
Our released model, gpt4all-lora, can be trained in
about eight hours on a Lambda Labs DGX A100
8x 80GB for a total cost of $100. - words exactly from the original paper.
Yep it is that affordable, if someone understands the graphs please comment.
Technical report can be read here - https://s3.amazonaws.com/static.nomic.ai/gpt4all/2023_GPT4All_Technical_Report.pdf
Github repo - https://github.com/nomic-ai/gpt4all
Github repo - https://github.com/nomic-ai/gpt4all
