Unleash Your Inner Composer: Generate Stunning Sheet Music with NotaGen (and Dramatically Improve Your Music!)

Want to create beautiful classical sheet music effortlessly? NotaGen is a groundbreaking symbolic music generation model that leverages the power of Large Language Models (LLMs) to help you do just that. Prepare to be amazed by the quality and sophistication of the music you can generate with this innovative tool.

Why NotaGen is a Game-Changer for Music Creation

NotaGen isn't just another music generator. It's a sophisticated system trained using a unique three-stage process, resulting in music that's both creative and structurally sound. Here's what makes it special:

• Massive Pre-training: Trained on 1.6 million musical pieces, NotaGen has a deep understanding of musical structures and patterns.
• Classical Fine-Tuning: Fine-tuned on approximately 9,000 classical compositions, ensuring high-quality, stylistically appropriate output.
• Reinforcement Learning (CLaMP-DPO): Uses a novel Reinforcement Learning method (CLaMP-DPO) to refine its output, no human input needed!

Ready to hear what NotaGen can do? Check out the [demo page](link-to-demo-page-here - placeholder)!

Get Started with NotaGen: Easy Setup

Setting up NotaGen is straightforward. Follow these simple instructions to get started:

1. Create a Conda environment: conda create --name notagen python=3.10
2. Activate the environment: conda activate notagen
3. Install PyTorch: conda install pytorch==2.3.0 pytorch-cuda=11.8 -c pytorch -c nvidia
4. Install necessary packages: pip install accelerate optimum pip install -r requirements.txt

That's it! You're now ready to explore the world of AI-powered music generation.

The Power of NotaGen-X: Enhanced Performance and Musicality

Inspired by Deepseek-R1, NotaGen-X takes music generation to the next level. This optimized version incorporates key improvements for even better results:

• Post-Training Refinement: A post-training stage refines the model with a classical-style subset, improving stylistic accuracy.
• Improved Instrument Range: Key augmentation removed during fine-tuning, resulting in more reasonable instrument ranges.
• Iterative Optimization: Multiple rounds of post-training, fine-tuning, and reinforcement learning for continuous improvement.

NotaGen-X builds upon the foundation of NotaGen, delivering enhanced musicality and control for discerning users.

Dive into Demos: Experience NotaGen-X in Action

The best way to understand NotaGen-X is to experience it yourself. Here are several ways to interact with the model:

• Online Gradio Demo: Use the online Gradio demo on Huggingface Space. Input "Period-Composer-Instrumentation" prompts to generate music, preview audio/PDF scores, and download your creations.
• Local Gradio Demo: Deploy a local Gradio demo. Requires approximately 8GB of GPU memory. Refer to gradio/README.md for setup instructions.
• Online Colab Notebook: Thanks to @deeplearn-art, a Google Colab notebook is available. Access a public Gradio link to experiment with NotaGen-X.
• ComfyUI Node: Huge thanks to @billwuhao for creating a ComfyUI node for NotaGen! Automatically convert generated .abc files to .xml, .mp3, and .png formats.

Customizing NotaGen: Pre-training, Fine-tuning, and Reinforcement Learning

NotaGen offers flexibility in how you use it. Here's a brief overview of each stage:

Pre-training

Want to train NotaGen with your own data?

1. Preprocess your data and generate data index files, following the instructions in data/README.md.
2. Modify the parameters in pretrain/config.py.
3. Run the pre-training script: cd pretrain/ followed by accelerate launch --multi_gpu --mixed_precision fp16 train-gen.py

Fine-tuning

Fine-tune NotaGen on a specific musical style. For instance, you can use Schubert's lieder data (available here). Requires at least 24GB of GPU memory for NotaGen-large. You can also use NotaGen-small or NotaGen-medium and change the configuration of models in finetune/config.py.

1. Modify DATA_TRAIN_INDEX_PATH and DATA_EVAL_INDEX_PATH in finetune/config.py.
2. Download pre-trained NotaGen weights and update PRETRAINED_PATH in finetune/config.py.
3. Set EXP_TAG (e.g., "schubert") for model differentiation.
4. Execute the fine-tuning script: cd finetune/ followed by CUDA_VISIBLE_DEVICES=0 python train-gen.py

Reinforcement Learning (CLaMP-DPO)

Enhance your fine-tuned model using CLaMP-DPO.

CLaMP 2 Setup

1. Download CLaMP 2 model weights.
2. Download M3 Model Weights.
3. Modify input_dir and output_dir in clamp2/extract_clamp2.py.
4. Extract features: cd clamp2/ followed by python extract_clamp2.py

CLaMP-DPO Iteration

1. Inference:
   • Modify INFERENCE_WEIGHTS_PATH and NUM_SAMPLES in inference/config.py.
   • Run inference: cd inference/ followed by python inference.py
2. Extract Generated Data Features:
   • Modify input_dir and output_dir in clamp2/extract_clamp2.py.
   • Extract features: cd clamp2/ followed by python extract_clamp2.py
3. Statistics on Average CLaMP 2 Score (Optional):
   • Modify parameters in clamp2/statistics.py.
   • Run statistics script: cd clamp2/ followed by python statistics.py
4. Construct Preference Data:
   • Modify parameters in RL/data.py.
   • Run data construction script.
5. DPO Training:
   • Modify parameters in RL/config.py.
   • Run training script: cd RL/ followed by CUDA_VISIBLE_DEVICES=0 python train.py

Unlock Your Musical Potential with NotaGen today

NotaGen represents a significant leap forward in symbolic music generation. By combining the power of LLMs with innovative training techniques, it opens up new possibilities for composers, musicians, and anyone interested in exploring the world of AI-generated music. Start experimenting with NotaGen and NotaGen-X today and unleash your musical creativity.

Citing NotaGen

If you use NotaGen in your work, please cite our paper:

@misc { wang2025notagenadvancingmusicalitysymbolic,
 title = { NotaGen: Advancing Musicality in Symbolic Music Generation with Large Language Model Training Paradigms},
 author = { Yashan Wang and Shangda Wu and Jianhuai Hu and Xingjian Du and Yueqi Peng and Yongxin Huang and Shuai Fan and Xiaobing Li and Feng Yu and Maosong Sun},
 year = { 2025},
 eprint = { 2502.18008},
 archivePrefix = { arXiv},
 primaryClass = { cs.SD},
 url = { https://arxiv.org/abs/2502.18008},
}