Unleash Your Inner Composer: Generate Stunning Sheet Music with NotaGen (and NotaGen-X!)

Want to create beautiful, high-quality classical sheet music, but don't know where to start? NotaGen is here to revolutionize symbolic music generation. Inspired by the power of Large Language Models (LLMs), NotaGen empowers you to compose music like never before. Plus, discover the enhanced capabilities of NotaGen-X!

What is NotaGen?

NotaGen is a groundbreaking music generation model designed to produce exceptional classical sheet music. It leverages a three-stage training paradigm inspired by the success of large language models:

• Pre-training: Trained on a massive dataset of 1.6 million musical pieces.
• Fine-tuning: Refined on approximately 9,000 classical compositions, using prompts based on period, composer, and instrumentation.
• Reinforcement Learning: Optimized using CLaMP-DPO.

Supercharge Your Compositions with NotaGen-X

Inspired by Deepseek-R1, NotaGen-X takes music generation to the next level. Here's how it improves upon the original NotaGen:

• Classical-Style Post-Training: A dedicated stage focuses on classical music nuances, resulting in more authentic compositions.
• Improved Instrument Range: By removing key augmentation in the fine-tuning stage, NotaGen-X generates music with more realistic instrument usage.
• Iterative Refinement: NotaGen-X uses another round of post-training, fine-tuning and reinforcement learning.

Get Started with NotaGen Today

Ready to dive in? Here's how to set up your environment:

1. Create a Conda environment: conda create --name notagen python=3.10
2. Activate the environment: conda activate notagen
3. Install PyTorch: conda install pytorch==2.3.0 pytorch-cuda=11.8 -c pytorch -c nvidia
4. Install dependencies: pip install -r requirements.txt

Download Pre-trained Model Weights

• Pre-trained Weights: Ideal for general music generation.
• Fine-tuned Weights: Optimized for classical compositions.
• Reinforcement Learning Weights: Benefit from CLaMP-DPO optimization.

Remember: Pre-trained weights cannot be used for conditional generation based on period, composer, and instrumentation. You need the fine-tuned weights for that!

Explore the Interactive Demo

You can experience the magic of NotaGen-X firsthand!

• Online Gradio Demo: Input a "Period-Composer-Instrumentation" prompt and listen to the results.
• Local Gradio Demo: For local deployment, please see the gradio/README.md and require 8GB of GPU Memory to run the demo.
• ComfyUI: Thanks to @billwuhao's contribution, you can automatically convert generated .abc files to .xml, .mp3, and .png formats.

Fine-Tuning for Specific Styles

Want to train NotaGen on your own music style? While the provided example uses Schubert's lieder, the principle applies to any genre.

1. Data Preprocessing: Prepare your data and generate index files.
2. Configuration: Modify the DATA_TRAIN_INDEX_PATH and DATA_EVAL_INDEX_PATH in finetune/config.py.
3. Execution: Run CUDA_VISIBLE_DEVICES=0 python train-gen.py in the finetune/ directory.

Reinforcement Learning (CLaMP-DPO)

Optimize your fine-tuned model with CLaMP-DPO. This process involves:

1. Inference: Generate music using your fine-tuned model.
2. Feature Extraction: Extract features from both the generated and ground truth data using CLaMP 2.
3. Calculate Average CLaMP 2 Score (Optional) Check the quality of the current model.
4. Preference Data Generation: Construct preference data based on CLaMP 2 scores.
5. DPO Training: Train the model using Direct Preference Optimization.

Cite Us

If NotaGen or CLaMP-DPO contribute to your work, please cite our paper!

NotaGen opens a world of possibilities for music generation, and NotaGen-X takes it even further. Start creating your own masterpieces today! Start composing new music with NotaGen!