Latest News in Machine Learning - Analytics Vidhya Edition

A roundup of exciting developments in the world of machine learning, curated by FOSAI's new semi-automated news report! Let me know if you'd like to see the format of these new reports changed. I'll be experimenting with a few templates to see which stick the most.

Table of Contents

1. Variational Autoencoders for Anomaly Detection
2. AI and Image Generation Aesthetics
3. Text to Sound with Large Language Models
4. RLHF for High-Performance Decision-Making
5. Generative Models in Semi-Supervised Learning
6. Serverless Large Language Models with RunPod
7. ChatGPT Plugins for Educational Enhancement
8. Python in Excel for Advanced Analytics
9. Harnessing Zero-shot and Few-shot Prompting in LLMs

Variational Autoencoders for Anomaly Detection

Intro: Explore the practical applications of generative AI in anomaly detection using Variational Autoencoders (VAEs).
Read More

AI and Image Generation Aesthetics

Intro: Dive into the creative and technical aspects of AI-powered artistic expression, including Neural Style Transfer and GANs.
Read More

Text to Sound with Large Language Models

Intro: Discover how AI can transform a musician's voice command into melodious guitar sounds through ‘Musician's Intent Recognition’.
Read More

RLHF for High-Performance Decision-Making

Intro: Learn about RLHF, an emerging field blending Reinforcement Learning and human feedback for optimizing complex system performance.
Read More

Generative Models in Semi-Supervised Learning

Intro: Understand how leveraging generative models can maximize the utility of limited labeled data in semi-supervised learning scenarios.
Read More

Serverless Large Language Models with RunPod

Intro: Explore how serverless computing and Generative AI can work in harmony, especially for developers lacking local high-GPU resources.
Read More

ChatGPT Plugins for Educational Enhancement

Intro: ChatGPT Plugins are customizing the educational experience, allowing users to browse the web and access specialized knowledge.
Read More

Python in Excel for Advanced Analytics

Intro: Microsoft integrates Python into Excel, enhancing its capabilities in data analysis, machine learning, and predictive analytics.
Read More

Harnessing Zero-shot and Few-shot Prompting in LLMs

Intro: Uncover the potential of Large Language Models in tasks like question-answering, creative writing, and critical analysis.
Read More

That's the roundup for now. Stay tuned for more updates from this new semi-automated workflow.

I have returned!

I was right, it was a simple oversight on an email verification I missed. I have now regained access thanks to the admins. No funny business to be seen with the Ban of Blaed!

I know I said a lot of content is coming, and it is! But this week in particular is going to be rather busy for me. Please stay tuned and patient until we clear Thursday. I have a huge presentation to give for a full-stack Machine Learning application I have been developing for quite some time (since starting [email protected]!). It is crucial I take the time to polish some of the work needed to hit it home.

Let's see if everything I've learned up to this point has paid off! Keep in mind that I knew literally nothing about Machine Learning, Deep Learning, or AI prior to starting this project. All I have had to work with is a high school diploma and access to the internet. My study/source material has been everything I have shared with each and every one of you here so far (plus some of my own intuitions).

The point of me saying this is that the fact that if I can do it, so can you!

Excelsior,

Blaed

EDIT: Easter egg

Training Diffusion Models with Reinforcement Learning

Source

Written by: Kevin Black

Diffusion models have recently emerged as the de facto standard for generating complex, high-dimensional outputs. They are known for producing stunning AI art and hyper-realistic synthetic images, but have also been applied in drug design and continuous control. The core principle of diffusion models involves the iterative transformation of random noise into a sample, often guided by a maximum likelihood estimation approach.

However, many use cases aim not merely to replicate training data, but to achieve specific objectives. In this post, we discuss how reinforcement learning (RL) can train diffusion models to meet these unique goals. Specifically, we finetune Stable Diffusion for various objectives, incorporating feedback from a large vision-language model to enhance the model’s output quality. This demonstrates the potential for powerful AI models to enhance one another without human intervention.

A diagram illustrating the prompt-image alignment objective using LLaVA, a large vision-language model.

Denoising Diffusion Policy Optimization

In adapting diffusion to RL, we make a fundamental assumption: for a given sample (e.g. an image), a reward function can evaluate its quality. The goal is for the diffusion model to maximize this reward function. Traditional diffusion models lean on maximum likelihood estimation (MLE) to generate samples, but in the RL context, we use the reward-weighted regression (RWR) method, inspired by existing RL algorithms.

However, this method presents challenges. Our denoising diffusion policy optimization (DDPO) algorithm overcomes these by considering the entire denoising sequence. By viewing the diffusion process as a Markov decision process (MDP), we leverage advanced RL algorithms that focus on multi-step MDPs, using exact likelihood calculations for each denoising step.

We've applied policy gradient algorithms due to their past success in language model finetuning. This led to two DDPO variants: DDPOSF (using the REINFORCE policy gradient) and DDPOIS (following the proximal policy optimization (PPO) method).

Finetuning Stable Diffusion Using DDPO

We finetuned Stable Diffusion v1-4 using DDPOIS for the following reward functions:

• Compressibility: Ease of image compression using JPEG.
• Incompressibility: Difficulty of image compression using JPEG.
• Aesthetic Quality: Evaluated by the LAION aesthetic predictor.
• Prompt-Image Alignment: Uses LLaVA to describe the image, then matches this description to the prompt using BERTScore.

For finetuning, we provided prompts like “a(n) [animal]” for the first three tasks. For prompt-image alignment, we gave prompts like “a(n) [animal] [activity]”.

We also explored DDPO's application in prompt-image alignment, observing a trend toward a more cartoonish style.

There is a lot more they covered on their website!

Click here for the full report.

GPT-4 + Stable-Diffusion = ?: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models

Source

TL;DR: Text Prompt -> LLM -> Intermediate Representation (such as an image layout) -> Stable Diffusion -> Image.

Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, despite their impressive capabilities, diffusion models, such as Stable Diffusion, often struggle to accurately follow the prompts when spatial or common sense reasoning is required.

The following figure lists four scenarios in which Stable Diffusion falls short in generating images that accurately correspond to the given prompts, namely negation, numeracy, attribute assignment, spatial relationships. In contrast, our method, LLM-grounded Diffusion (LMD), delivers much better prompt understanding in text-to-image generation in those scenarios.

Figure 1: LLM-grounded Diffusion enhances the prompt understanding ability of text-to-image diffusion models.

One possible solution to address this issue is of course to gather a vast multi-modal dataset comprising intricate captions and train a large diffusion model with a large language encoder. This approach comes with significant costs: It is time-consuming and expensive to train both large language models (LLMs) and diffusion models.

Our Solution

To efficiently solve this problem with minimal cost (i.e., no training costs), we instead equip diffusion models with enhanced spatial and common sense reasoning by using off-the-shelf frozen LLMs in a novel two-stage generation process.

First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We invite readers to read the paper on arXiv for additional details.

Figure 2: LMD is a text-to-image generative model with a novel two-stage generation process: a text-to-layout generator with an LLM + in-context learning and a novel layout-guided stable diffusion. Both stages are training-free.

LMD’s Additional Capabilities

Additionally, LMD naturally allows dialog-based multi-round scene specification, enabling additional clarifications and subsequent modifications for each prompt. Furthermore, LMD is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

Figure 3: Incorporating an LLM for prompt understanding, our method is able to perform dialog-based scene specification and generation from prompts in a language (Chinese in the example above) that the underlying diffusion model does not support.

Given an LLM that supports multi-round dialog (e.g., GPT-3.5 or GPT-4), LMD allows the user to provide additional information or clarifications to the LLM by querying the LLM after the first layout generation in the dialog and generate images with the updated layout in the subsequent response from the LLM. For example, a user could request to add an object to the scene or change the existing objects in location or descriptions (the left half of Figure 3).

Furthermore, by giving an example of a non-English prompt with a layout and background description in English during in-context learning, LMD accepts inputs of non-English prompts and will generate layouts, with descriptions of boxes and the background in English for subsequent layout-to-image generation. As shown in the right half of Figure 3, this allows generation from prompts in a language that the underlying diffusion models do not support.

Visualizations

We validate the superiority of our design by comparing it with the base diffusion model (SD 2.1) that LMD uses under the hood. We invite readers to our work for more evaluation and comparisons.

Figure 4: LMD outperforms the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. LMD also enables counterfactual text-to-image generation that the base diffusion model is not able to generate (the last row).

For more details about LLM-grounded Diffusion (LMD), visit our website and read the paper on arXiv.

Click Here to Learn More

Disclaimer: This report was generated by ChatGPT from third-party sources and may be subject to change. Always refer to the original source for the most up-to-date information. This automated news report is still experimental but open to improvements! More news soon!

In this video, we'll learn how to build Large Language Model (LLM) + Retrieval Augmented Generation (RAG) pipelines using open-source models from Hugging Face deployed on AWS SageMaker. We use the MiniLM sentence transformer to power our semantic search component with Pinecone.

Hello everyone!

I am working on figuring out better workflows bringing back more consistent post schedules. In the meantime, I'd like to leave you with a new update from LocalAI & Continue.

Check these projects out! More info from the Continue & LocalAI teams below:

Continue

• https://continue.dev/
• https://github.com/continuedev/continue

The open-source autopilot for software development A VS Code extension that brings the power of ChatGPT to your IDE

LocalAI

• https://localai.io/basics/news/
• https://github.com/go-skynet/LocalAI

LocalAI is a drop-in replacement REST API that’s compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families that are compatible with the ggml format. Does not require GPU.

Combining the Power of Continue + LocalAI!

• https://github.com/go-skynet/LocalAI/blob/master/examples/continue/README.md

Note

From this release the llama backend supports only gguf files (see 943 ). LocalAI however still supports ggml files. We ship a version of llama.cpp before that change in a separate backend, named llama-stable to allow still loading ggml files. If you were specifying the llama backend manually to load ggml files from this release you should use llama-stable instead, or do not specify a backend at all (LocalAI will automatically handle this).

Continue

This document presents an example of integration with continuedev/continue.

For a live demonstration, please click on the link below:

• How it works (Video demonstration)

Integration Setup Walkthrough

1. As outlined in continue's documentation, install the Visual Studio Code extension from the marketplace and open it.

2. In this example, LocalAI will download the gpt4all model and set it up as "gpt-3.5-turbo". Refer to the docker-compose.yaml file for details.

   # Clone LocalAI
   git clone https://github.com/go-skynet/LocalAI
   
   cd LocalAI/examples/continue
   
   # Start with docker-compose
   docker-compose up --build -d
   

3. Type /config within Continue's VSCode extension, or edit the file located at ~/.continue/config.py on your system with the following configuration:

   from continuedev.src.continuedev.libs.llm.openai import OpenAI, OpenAIServerInfo
   
   config = ContinueConfig(
      ...
      models=Models(
           default=OpenAI(
              api_key="my-api-key",
              model="gpt-3.5-turbo",
              openai_server_info=OpenAIServerInfo(
                 api_base="http://localhost:8080",
                 model="gpt-3.5-turbo"
              )
           )
      ),
   )
   

This setup enables you to make queries directly to your model running in the Docker container. Note that the api_key does not need to be properly set up; it is included here as a placeholder.

If editing the configuration seems confusing, you may copy and paste the provided default config.py file over the existing one in ~/.continue/config.py after initializing the extension in the VSCode IDE.

Additional Resources

• Official Continue documentation
• Documentation page on using self-hosted models
• Official extension link

cross-posted from: https://lemmy.world/post/3879861

Beating GPT-4 on HumanEval with a Fine-Tuned CodeLlama-34B

Hello everyone! This post marks an exciting moment for [email protected] and everyone in the open-source large language model and AI community.

We appear to have a new contender on the block, a model apparently capable of surpassing OpenAI's state of the art ChatGPT-4 in coding evals (evaluations).

This is huge. Not too long ago I made an offhand comment on us catching up to GPT-4 within a year. I did not expect that prediction to end up being reality in half the time. Let's hope this isn't a one-off scenario and that we see a new wave of open-source models that begin to challenge OpenAI.

Buckle up, it's going to get interesting!

Here's some notes from the blog, which you should visit and read in its entirety:

• https://www.phind.com/blog/code-llama-beats-gpt4

Blog Post

We have fine-tuned CodeLlama-34B and CodeLlama-34B-Python on an internal Phind dataset that achieved 67.6% and 69.5% pass@1 on HumanEval, respectively. GPT-4 achieved 67% according to their official technical report in March. To ensure result validity, we applied OpenAI's decontamination methodology to our dataset.

The CodeLlama models released yesterday demonstrate impressive performance on HumanEval.

• CodeLlama-34B achieved 48.8% pass@1 on HumanEval
• CodeLlama-34B-Python achieved 53.7% pass@1 on HumanEval

We have fine-tuned both models on a proprietary dataset of ~80k high-quality programming problems and solutions. Instead of code completion examples, this dataset features instruction-answer pairs, setting it apart structurally from HumanEval. We trained the Phind models over two epochs, for a total of ~160k examples. LoRA was not used — both models underwent a native fine-tuning. We employed DeepSpeed ZeRO 3 and Flash Attention 2 to train these models in three hours using 32 A100-80GB GPUs, with a sequence length of 4096 tokens.

Furthermore, we applied OpenAI's decontamination methodology to our dataset to ensure valid results, and found no contaminated examples. 

The methodology is:

• For each evaluation example, we randomly sampled three substrings of 50 characters or used the entire example if it was fewer than 50 characters.
• A match was identified if any sampled substring was a substring of the processed training example.

For further insights on the decontamination methodology, please refer to Appendix C of OpenAI's technical report. Presented below are the pass@1 scores we achieved with our fine-tuned models:

• Phind-CodeLlama-34B-v1 achieved 67.6% pass@1 on HumanEval
• Phind-CodeLlama-34B-Python-v1 achieved 69.5% pass@1 on HumanEval

Download

We are releasing both models on Huggingface for verifiability and to bolster the open-source community. We welcome independent verification of results.

• https://huggingface.co/Phind/Phind-CodeLlama-34B-v1
• https://huggingface.co/Phind/Phind-CodeLlama-34B-Python-v1

If you get a chance to try either of these models out, let us know how it goes in the comments below!

If you found anything about this post interesting, consider subscribing to [email protected].

Cheers to the power of open-source! May we continue the fight for optimization, efficiency, and performance.

cross-posted from: https://lemmy.world/post/3879861

Beating GPT-4 on HumanEval with a Fine-Tuned CodeLlama-34B

Hello everyone! This post marks an exciting moment for [email protected] and everyone in the open-source large language model and AI community.

We appear to have a new contender on the block, a model apparently capable of surpassing OpenAI's state of the art ChatGPT-4 in coding evals (evaluations).

This is huge. Not too long ago I made an offhand comment on us catching up to GPT-4 within a year. I did not expect that prediction to end up being reality in half the time. Let's hope this isn't a one-off scenario and that we see a new wave of open-source models that begin to challenge OpenAI.

Buckle up, it's going to get interesting!

Here's some notes from the blog, which you should visit and read in its entirety:

• https://www.phind.com/blog/code-llama-beats-gpt4

Blog Post

We have fine-tuned CodeLlama-34B and CodeLlama-34B-Python on an internal Phind dataset that achieved 67.6% and 69.5% pass@1 on HumanEval, respectively. GPT-4 achieved 67% according to their official technical report in March. To ensure result validity, we applied OpenAI's decontamination methodology to our dataset.

The CodeLlama models released yesterday demonstrate impressive performance on HumanEval.

• CodeLlama-34B achieved 48.8% pass@1 on HumanEval
• CodeLlama-34B-Python achieved 53.7% pass@1 on HumanEval

We have fine-tuned both models on a proprietary dataset of ~80k high-quality programming problems and solutions. Instead of code completion examples, this dataset features instruction-answer pairs, setting it apart structurally from HumanEval. We trained the Phind models over two epochs, for a total of ~160k examples. LoRA was not used — both models underwent a native fine-tuning. We employed DeepSpeed ZeRO 3 and Flash Attention 2 to train these models in three hours using 32 A100-80GB GPUs, with a sequence length of 4096 tokens.

Furthermore, we applied OpenAI's decontamination methodology to our dataset to ensure valid results, and found no contaminated examples. 

The methodology is:

• For each evaluation example, we randomly sampled three substrings of 50 characters or used the entire example if it was fewer than 50 characters.
• A match was identified if any sampled substring was a substring of the processed training example.

For further insights on the decontamination methodology, please refer to Appendix C of OpenAI's technical report. Presented below are the pass@1 scores we achieved with our fine-tuned models:

• Phind-CodeLlama-34B-v1 achieved 67.6% pass@1 on HumanEval
• Phind-CodeLlama-34B-Python-v1 achieved 69.5% pass@1 on HumanEval

Download

We are releasing both models on Huggingface for verifiability and to bolster the open-source community. We welcome independent verification of results.

• https://huggingface.co/Phind/Phind-CodeLlama-34B-v1
• https://huggingface.co/Phind/Phind-CodeLlama-34B-Python-v1

If you get a chance to try either of these models out, let us know how it goes in the comments below!

If you found anything about this post interesting, consider subscribing to [email protected].

Cheers to the power of open-source! May we continue the fight for optimization, efficiency, and performance.

cross-posted from: https://lemmy.world/post/3879861

Beating GPT-4 on HumanEval with a Fine-Tuned CodeLlama-34B

Hello everyone! This post marks an exciting moment for [email protected] and everyone in the open-source large language model and AI community.

We appear to have a new contender on the block, a model apparently capable of surpassing OpenAI's state of the art ChatGPT-4 in coding evals (evaluations).

This is huge. Not too long ago I made an offhand comment on us catching up to GPT-4 within a year. I did not expect that prediction to end up being reality in half the time. Let's hope this isn't a one-off scenario and that we see a new wave of open-source models that begin to challenge OpenAI.

Buckle up, it's going to get interesting!

Here's some notes from the blog, which you should visit and read in its entirety:

• https://www.phind.com/blog/code-llama-beats-gpt4

Blog Post

We have fine-tuned CodeLlama-34B and CodeLlama-34B-Python on an internal Phind dataset that achieved 67.6% and 69.5% pass@1 on HumanEval, respectively. GPT-4 achieved 67% according to their official technical report in March. To ensure result validity, we applied OpenAI's decontamination methodology to our dataset.

The CodeLlama models released yesterday demonstrate impressive performance on HumanEval.

• CodeLlama-34B achieved 48.8% pass@1 on HumanEval
• CodeLlama-34B-Python achieved 53.7% pass@1 on HumanEval

We have fine-tuned both models on a proprietary dataset of ~80k high-quality programming problems and solutions. Instead of code completion examples, this dataset features instruction-answer pairs, setting it apart structurally from HumanEval. We trained the Phind models over two epochs, for a total of ~160k examples. LoRA was not used — both models underwent a native fine-tuning. We employed DeepSpeed ZeRO 3 and Flash Attention 2 to train these models in three hours using 32 A100-80GB GPUs, with a sequence length of 4096 tokens.

Furthermore, we applied OpenAI's decontamination methodology to our dataset to ensure valid results, and found no contaminated examples. 

The methodology is:

• For each evaluation example, we randomly sampled three substrings of 50 characters or used the entire example if it was fewer than 50 characters.
• A match was identified if any sampled substring was a substring of the processed training example.

For further insights on the decontamination methodology, please refer to Appendix C of OpenAI's technical report. Presented below are the pass@1 scores we achieved with our fine-tuned models:

• Phind-CodeLlama-34B-v1 achieved 67.6% pass@1 on HumanEval
• Phind-CodeLlama-34B-Python-v1 achieved 69.5% pass@1 on HumanEval

Download

We are releasing both models on Huggingface for verifiability and to bolster the open-source community. We welcome independent verification of results.

• https://huggingface.co/Phind/Phind-CodeLlama-34B-v1
• https://huggingface.co/Phind/Phind-CodeLlama-34B-Python-v1

If you get a chance to try either of these models out, let us know how it goes in the comments below!

If you found anything about this post interesting, consider subscribing to [email protected].

Cheers to the power of open-source! May we continue the fight for optimization, efficiency, and performance.

Introducing CodeLlama-34B - One of the First Open-Source Models to Beat OpenAI's ChatGPT-4!

Hello everyone! This post marks an exciting moment for [email protected] and everyone in the open-source large language model and AI community.

We appear to have a new contender on the block, a model apparently capable of surpassing OpenAI's state of the art ChatGPT-4 in coding evals (evaluations).

This is huge. Not too long ago I made an offhand comment on us catching up to GPT-4 within a year. I did not expect that prediction to end up being reality in half the time. Let's hope this isn't a one-off scenario and that we see a new wave of open-source models that begin to challenge OpenAI.

Buckle up, it's going to get interesting!

Here's some notes from the blog, which you should visit and read in its entirety:

• https://www.phind.com/blog/code-llama-beats-gpt4

Blog Post

We have fine-tuned CodeLlama-34B and CodeLlama-34B-Python on an internal Phind dataset that achieved 67.6% and 69.5% pass@1 on HumanEval, respectively. GPT-4 achieved 67% according to their official technical report in March. To ensure result validity, we applied OpenAI's decontamination methodology to our dataset.

The CodeLlama models released yesterday demonstrate impressive performance on HumanEval.

• CodeLlama-34B achieved 48.8% pass@1 on HumanEval
• CodeLlama-34B-Python achieved 53.7% pass@1 on HumanEval

We have fine-tuned both models on a proprietary dataset of ~80k high-quality programming problems and solutions. Instead of code completion examples, this dataset features instruction-answer pairs, setting it apart structurally from HumanEval. We trained the Phind models over two epochs, for a total of ~160k examples. LoRA was not used — both models underwent a native fine-tuning. We employed DeepSpeed ZeRO 3 and Flash Attention 2 to train these models in three hours using 32 A100-80GB GPUs, with a sequence length of 4096 tokens.

Furthermore, we applied OpenAI's decontamination methodology to our dataset to ensure valid results, and found no contaminated examples. 

The methodology is:

• For each evaluation example, we randomly sampled three substrings of 50 characters or used the entire example if it was fewer than 50 characters.
• A match was identified if any sampled substring was a substring of the processed training example.

For further insights on the decontamination methodology, please refer to Appendix C of OpenAI's technical report. Presented below are the pass@1 scores we achieved with our fine-tuned models:

• Phind-CodeLlama-34B-v1 achieved 67.6% pass@1 on HumanEval
• Phind-CodeLlama-34B-Python-v1 achieved 69.5% pass@1 on HumanEval

Download

We are releasing both models on Huggingface for verifiability and to bolster the open-source community. We welcome independent verification of results.

• https://huggingface.co/Phind/Phind-CodeLlama-34B-v1
• https://huggingface.co/Phind/Phind-CodeLlama-34B-Python-v1

If you get a chance to try either of these models out, let us know how it goes in the comments below!

If you found anything about this post interesting, consider subscribing to [email protected].

Cheers to the power of open-source! May we continue the fight for optimization, efficiency, and performance.

GPT-3.5 Turbo fine-tuning and API updates

Developers can now bring their own data to customize GPT-3.5 Turbo for their use cases.

• https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updates

Personally, I wish OpenAI opened more, but this is a step in the right direction. The costs to fine-tune continue to lower, with more and more access being given to this path of customization.

Here's the news:

Fine-tuning for GPT-3.5 Turbo is now available, with fine-tuning for GPT-4 coming this fall. This update gives developers the ability to customize models that perform better for their use cases and run these custom models at scale. Early tests have shown a fine-tuned version of GPT-3.5 Turbo can match, or even outperform, base GPT-4-level capabilities on certain narrow tasks. As with all our APIs, data sent in and out of the fine-tuning API is owned by the customer and is not used by OpenAI, or any other organization, to train other models.

Fine-tuning use cases

Since the release of GPT-3.5 Turbo, developers and businesses have asked for the ability to customize the model to create unique and differentiated experiences for their users. With this launch, developers can now run supervised fine-tuning to make this model perform better for their use cases.

In our private beta, fine-tuning customers have been able to meaningfully improve model performance across common use cases, such as:

Improved steerability: Fine-tuning allows businesses to make the model follow instructions better, such as making outputs terse or always responding in a given language. For instance, developers can use fine-tuning to ensure that the model always responds in German when prompted to use that language.

Reliable output formatting: Fine-tuning improves the model's ability to consistently format responses—a crucial aspect for applications demanding a specific response format, such as code completion or composing API calls. A developer can use fine-tuning to more reliably convert user prompts into high-quality JSON snippets that can be used with their own systems.

Custom tone: Fine-tuning is a great way to hone the qualitative feel of the model output, such as its tone, so it better fits the voice of businesses’ brands. A business with a recognizable brand voice can use fine-tuning for the model to be more consistent with their tone.

In addition to increased performance, fine-tuning also enables businesses to shorten their prompts while ensuring similar performance. Fine-tuning with GPT-3.5-Turbo can also handle 4k tokens—double our previous fine-tuned models. Early testers have reduced prompt size by up to 90% by fine-tuning instructions into the model itself, speeding up each API call and cutting costs.

Fine-tuning is most powerful when combined with other techniques such as prompt engineering, information retrieval, and function calling. Check out our fine-tuning guide to learn more. Support for fine-tuning with function calling and gpt-3.5-turbo-16k will be coming later this fall.

Fine-tuning steps

Step 1

Prepare your data

{ "messages": [ { "role": "system", "content": "You are an assistant that occasionally misspells words" }, { "role": "user", "content": "Tell me a story." }, { "role": "assistant", "content": "One day a student went to schoool." } ] }

Step 2

Upload files

curl -https://api.openai.com/v1/files \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -F "purpose=fine-tune" \ -F "file=@path_to_your_file"

Step 3

Create a fine-tuning job

curl https://api.openai.com/v1/fine_tuning/jobs \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "training_file": "TRAINING_FILE_ID", "model": "gpt-3.5-turbo-0613", }'

Once a model finishes the fine-tuning process, it is available to be used in production right away and has the same shared rate limits as the underlying model.

Step 4

Use a fine-tuned model

curl https://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{ "model": "ft:gpt-3.5-turbo:org_id", "messages": [ { "role": "system", "content": "You are an assistant that occasionally misspells words" }, { "role": "user", "content": "Hello! What is fine-tuning?" } ] }'

OpenAI will also be debuting a fine-tuning UI in the near future, which will give developers easier access to information about ongoing fine-tuning jobs, completed model snapshots, and more.

Safety

It is very important to us that the deployment of fine-tuning is safe. To preserve the default model's safety features through the fine-tuning process, fine-tuning training data is passed through our Moderation API and a GPT-4 powered moderation system to detect unsafe training data that conflict with our safety standards.

Pricing

Fine-tuning costs are broken down into two buckets: the initial training cost and usage cost:

• Training: $0.008 / 1K Tokens
• Usage input: $0.012 / 1K Tokens
• Usage output: $0.016 / 1K Tokens

For example, a gpt-3.5-turbo fine-tuning job with a training file of 100,000 tokens that is trained for 3 epochs would have an expected cost of $2.40.

Updated GPT-3 models

In July, we announced that the original GPT-3 base models (ada, babbage, curie, and davinci) would be turned off on January 4th, 2024. Today, we are making babbage-002 and davinci-002 available as replacements for these models, either as base or fine-tuned models. Customers can access those models by querying the Completions API.

These models can be fine-tuned with our new API endpoint /v1/fine_tuning/jobs. This new endpoint offers pagination and more extensibility to support the future evolution of the fine-tuning API. Transitioning from /v1/fine-tunes to the updated endpoint is straightforward and more details can be found in our new fine-tuning guide. This deprecates the old /v1/fine-tunes endpoint, which will be turned off on January 4th, 2024.

Pricing for base and fine-tuned GPT-3 models is as follows:

Read more on the updated documentation below.

• https://platform.openai.com/docs/guides/fine-tuning/common-use-cases

Dolma

Dolma is an open dataset of 3 trillion tokens from a diverse mix of web content, academic publications, code, books, and encyclopedic materials. It was created as a training corpus for OLMo, AI2 language model.

Designed as a foundational training corpus for the AI2 language model, OLMo, Dolma offers an expansive playground for developers, researchers, and innovators.

• https://github.com/allenai/dolma/tree/main

Usage

This repository contains tools for generating and inspecting Dolma. To get started, install the Dolma Python library from PyPI.

pip install dolma

The dolma CLI can be access using the dolma command. To see the available commands, use the --help flag.

dolma --help

At the moment, the CLI supports three commands: tag, dedupe, and mix.

For all commands, configurations can be specified from command line, or by passing a YAML or JSON file using the -c flag. For example:

dolma -c config.yaml dedupe --dedupe.name "test"

What Sets Dolma Apart?

Versatility: With its extensive and varied content, Dolma provides ample opportunities for experimentation and research across different AI domains.

Ease of Use: The Dolma Python library can be quickly installed from PyPI, allowing you to jump into your projects without delay.

Robust Tools: This repository equips you with tools for generating, tagging, deduplicating, and mixing the dataset, tailoring it to your specific needs.

Getting Started with Dolma

Utilize the dolma CLI to explore the available commands and configurations. From deduplication with dedupe to document mixing with mix, Dolma opens up numerous possibilities. Full usage instructions are available in the repository.

Contribute and Collaborate

Dolma isn't just a dataset; it's a community-driven initiative that welcomes contributions and ideas. Check out the development guide to see how you can get involved.

• https://github.com/allenai/dolma/tree/main

There are some questions about the fair usage and license of this dataset, but if those concerns are cleared this may be a dataset worth starring or looking into.