Enhance Video Quality: Implement BiM-VFI for Bidirectional Frame Interpolation (CVPR 2025)

Are you struggling with non-uniform motions in your videos and seeking a solution to enhance frame interpolation? The Bidirectional Motion Field-Guided Frame Interpolation (BiM-VFI) model, presented at CVPR 2025, offers a powerful solution using PyTorch. This guide provides a comprehensive walkthrough on setting up, using, and training the BiM-VFI model to achieve smoother, high-quality video frame interpolation, even with complex movements.

What is BiM-VFI?

BiM-VFI is an advanced frame interpolation technique designed to handle videos with non-uniform motions. It uses bidirectional motion fields to predict intermediate frames, resulting in smoother and more realistic video sequences. This technique is crucial for applications requiring high-quality video enhancement, such as:

• Improving slow-motion videos.
• Enhancing video resolution.
• Creating seamless transitions between frames.

Setting Up Your Environment

Before diving into BiM-VFI, let's set up the necessary environment:

1. Create a Conda Environment: Isolate your project dependencies.

   conda create -n bimvfi python=3.11
   conda activate bimvfi
   

   Copy
2. Install Dependencies: Install the required PyTorch packages.

   pip install basicsr-fixed Ipython torchsummary wandb moviepy pyyaml imageio packaging tqdm opencv-python tensorboardx ptflops pyiqa lpips stlpips_pytorch dists_pytorch torch==2.4.1 torchvision==0.19.1
   conda install cupy -c conda-forge
   

   Copy
   These packages ensure compatibility and provide necessary functionalities.

Dataset Preparation

Download Datasets

To train and test BiM-VFI, you'll need appropriate datasets:

• [Vimeo90K](link to Vimeo90K)
• [SNU-FILM](link to SNU-FILM)
• [SNU-FILM-arb](link to SNU-FILM-arb)
• [X4K1000FPS](link to X4K1000FPS)

Organize Datasets

Organize your datasets correctly for evaluation:

Move `test-[easy, medium, hard, extreme].txt` and `test-arb-[medium, hard, extreme].txt` to `<PATH_TO_SNU_FILM>/eval_modes` directory for SNU-FILM and SNU-FILM-arb datasets.

Using the Pre-trained Model

For immediate results, use the pre-trained BiM-VFI model:

Download the pre-trained model [here](link to pretrained model).

Evaluating Performance

Evaluate BiM-VFI's performance using benchmark datasets:

1. Configure Evaluation: Edit cfgs/bimvfi_benchmark.yaml to specify the benchmark dataset.

   • name: Name of benchmark datasets. Options: [vimeo, vimeo_septuplet, snu_film, snu_film_arb, xtest]
   • args:
     • root_path: Path to the dataset.
     • split: Splits to evaluate ([test, val] for Vimeo; [easy, medium, hard, extreme] for SNU-FILM; [single, multiple] for XTEST).
     • pyr_lvl: Pyramid levels (3 for Vimeo, 5 for SNU-FILM, 7 for XTEST).
   • save_imgs: Set to True to save interpolated images. Note: This increases processing time.

2. Run Evaluation: Execute the main script.

   python main.py --cfg cfgs/bim_vfi_benchmark.yaml
   

   Copy

Training Your Own Model

Single GPU Training

Start training with a single GPU using:

python main.py --cfg cfgs/bim_vfi.yaml

Multi-GPU Training

Utilize multiple GPUs (e.g., GPUs 0, 1, 2, 3) for faster training:

CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --nproc_per_node 4 main.py --cfg cfgs/bim_vfi.yaml

Training with WandB

Integrate Weights & Biases (WandB) for experiment tracking:

1. Fill in the wandb.yaml configuration file.
2. Run the training script with the -w flag:

python main.py --cfg cfgs/bim_vfi.yaml -w

Running a Demo

Prepare Your Videos/Images

Organize your custom videos or images in a directory structure like this:

demo_root/
├── video1.mp4
├── video2.mp4
├── video3/
│   ├── img0.png
│   ├── img1.png
│   └── ...
└── ...

Configure and Run the Demo

1. Set the Root Path: Update the root_path in cfgs/bim_vfi_demo.yaml to point to your data directory.
2. Run the Demo: Start the demo script.

   python main.py --cfg cfgs/bim_vfi_demo.yaml
   

   Copy

License Information

The BiM-VFI source code and pre-trained models are available for research and educational purposes only. Commercial use requires explicit permission from Prof. Munchurl Kim ([email protected]).