Open Source Language Model Named Dolly 2.0 Trained Similarly To ChatGPT

Databricks announced the release of the first open source instruction-tuned language model, called Dolly 2.0. This model is free to use, including for commercial purposes, because every part of the model is 100% open source. Open Source Instruction TrainingWhat makes ChatGPT able to follow directions is the training it receives using techniques outlined in the InstructGPT research paper. By using human evaluated question and answer training, OpenAI was able to train a better language model using one hundred times fewer parameters than the previous model, GPT-3. Open source is picking up momentum and it will be interesting to see where this industry is at within the next two years.