Arcee-Nova: a new high for open-source language models!

Introduction

Hi everybody, this is Julian from RC. I'm thrilled to introduce a new state-of-the-art model called Nova. Nova has been developed by RC, starting from Quen 2, and has been improved with merging and fine-tuning. As of the time of recording this video, it sits at the top of the Hugging Face LLM leaderboard.

Introduction to Nova

Nova was literally released yesterday, and here's the blog post announcing it. The post contains a few details on the model itself, and what's particularly interesting is its performance metrics.

Performance Metrics

The performance metrics are illustrated below with a screenshot from the Hugging Face leaderboard. Here's the live version for more up-to-date information. The top models are still the same: Quen 272b Instruct, another Quen model, and Mini and LLaMA 370b. RC Nova has been evaluated on the same benchmarks, such as Big Bench Hard, the Math benchmark, Multi-Step Reasoning, MLU, etc. Nova is currently the top model when averaged across these benchmarks.

Availability

You can find Nova on the Hugging Face website. There are different versions available, including vanilla and quantized models in various resolutions. For simplicity, I'll be working with one of the quantized versions, specifically the 3-bit version (Q3_KS flavor).

Running Nova on Your Machine

Given that Nova is a 72 billion parameter model, running the 8-bit version might be impractical for many. I chose to go with the 3-bit version for optimal performance on my Mac.

Steps to Install

Download and Merge Shards: Download the eight chunks of the model, as each chunk is over 4GB. Once downloaded, you can stitch them back together using the LLaMA GGF split utility with the merge option.
```
./llama-split-merge --input <path_to_first_shard> --output <path_to_full_model>
```
Note: The full-size model will be around 32GB even when quantized to 3 bits.
Install oLLaMA: Although I couldn't get the model to work with LLaMA CPP, you can use oLLaMA. Follow the instructions on their repository to install it.
```
git clone <olama_repo>
cd olama_repo
make install
```
Create Environment: Use a one-line config file to create the model environment.
```
model: Nova
```
Then, run the following command to create the environment:
```
oLLaMA create --config <config_file> <model_name>
```

Running the Model

You can list and run the model using oLLaMA. The model loads in about a minute, after which you are ready to start testing.

```shell
oLLaMA list
oLLaMA run <model_name>
```

Testing the Model

Here are some example prompts to showcase Nova's capabilities:

Model Information: You can ask Nova about itself, and it will provide some basic information.
```
What are you based on?
```
Response: "I am based on Quen 2, developed by RC."
Marketing Speech: You can also try creative writing tasks, like writing a marketing speech for a fictional AI platform called RC Cloud.
```
Write a marketing speech for a SaaS AI platform called RC Cloud.
```
Multi-Step Reasoning: One major strength of Nova is its multi-step reasoning capabilities. Here’s an example prompt:
```
Given this story: "Fol works at an art gallery. Three people: Patricia, Matthew, and Rebecca each have different skills..." Who should be creating and selling art?
```
Nova's response includes reasoning about the allocation of tasks based on the characters' traits, ensuring a detailed and accurate answer.

Future Work

Well, that's it for now with RC Nova. In upcoming videos, we'll delve deeper into the RC platform, where you can learn about model merging, continuous pre-training, and more. Self-service model creation is on the horizon, so stay tuned.

Until then, keep exploring and innovating!

Keywords

RC Nova
Quen 2
Hugging Face LLM leaderboard
Performance metrics
Quantized versions
oLLaMA
Multi-step reasoning

FAQ

Q1: What is RC Nova?

A1: RC Nova is a state-of-the-art language model developed by RC, based on Quen 2, and improved by merging and fine-tuning.

Q2: How does RC Nova perform on benchmarks?

A2: As of now, RC Nova sits at the top of the Hugging Face LLM leaderboard, excelling in benchmarks like Big Bench Hard, Multi-Step Reasoning, and MLU.

Q3: Where can I find RC Nova?

A3: You can find RC Nova on the Hugging Face website, available in both vanilla and various quantized versions.

Q4: What is the recommended version to run on a typical machine?

A4: For standard machines, it’s recommended to use the 3-bit quantized version (Q3_KS flavor) for optimal performance.

Q5: What is unique about RC Nova's multi-step reasoning capability?

A5: RC Nova has shown superior performance in multi-step reasoning benchmarks, often providing detailed explanations for its answers.

Q6: How do I install and run RC Nova?

A6: You can merge the downloaded shards using the LLaMA split utility and run the model using oLLaMA. Detailed steps for installation and environment setup are provided in the article.