Achieve Stunning Image Edits with ICEdit: In-Context Image Editing Made Easy

Want to make precise, instruction-based edits to your images? ICEdit offers a groundbreaking solution, achieving top-tier image manipulation with minimal data and parameters. This guide walks you through everything you need to know about using ICEdit, from installation to advanced inference techniques.

What is ICEdit?

ICEdit is an innovative approach to instruction-based image editing. It leverages in-context generation within a large-scale diffusion transformer, and it is designed for high precision and diverse editing results. Achieve state-of-the-art results, use minimal training data, and require fewer parameters than other methods.

Key Advantages of ICEdit

• Efficiency: Achieves state-of-the-art results using only 0.5% of the training data and 1% of parameters compared to previous methods.
• Precision: Executes multi-turn edits with high precision, ensuring your vision comes to life.
• Speed: Quickly edit images, taking approximately 9 seconds per image.

Getting Started with ICEdit

Here's how to set up ICEdit and start editing your images:

1. Installation

Follow these steps to install ICEdit in a Conda environment:

conda create -n icedit python=3.10
conda activate icedit
pip install -r requirements.txt
pip install -U huggingface_hub

2. Download Pretrained Weights

If you can connect to Huggingface, you don't need to download the weights. Otherwise, manually download the following pretrained weights:

• Flux.1-fill-dev
• ICEdit-MoE-LoRA

3. Inference in Bash (Without VLM Scaling)

Now, you can start editing images using the following command:

python scripts/inference.py --image assets/girl.png \
--instruction " Make her hair dark green and her clothes checked. " \
--seed 42

Important: ICEdit works best with images that have a width of 512 pixels. The model automatically resizes images to this width if necessary.

Tips for Optimal Results with ICEdit

• Experiment with Seeds: If the initial results aren't what you expected, try changing the --seed parameter.
• Memory Management: For systems with limited GPU memory (e.g., 24GB), use the --enable-model-cpu-offload parameter.
• Local Weights: If you've downloaded the pretrained weights locally, specify their paths using --flux-path and --lora-path.

Here's an example of how to specify local weights and enable CPU offloading:

python scripts/inference.py --image assets/girl.png \
--instruction " Make her hair dark green and her clothes checked. " \
--flux-path /path/to/flux.1-fill-dev \
--lora-path /path/to/ICEdit-MoE-LoRA \
--enable-model-cpu-offload

User-Friendly Editing with Gradio Demo

ICEdit offers a Gradio demo for a more intuitive editing experience. Launch the demo with this command:

python scripts/gradio_demo.py --port 7860

Similar to the inference script, you can use --enable-model-cpu-offload for low-memory systems and specify local weights with --flux-path and --lora-path.

Understanding ICEdit's Capabilities and Limitations

Keep the following points in mind when using ICEdit for instruction-based image editing:

• Style Transfer: The base model may sometimes alter the artistic style due to the dataset heavily involving style transfer.
• Image Realism: ICEdit is primarily trained on realistic images; editing non-realistic images (e.g., anime) might yield less successful results.
• Object Removal: Object removal might have lower success rates due to the quality of the OmniEdit removal dataset.

Future Enhancements

The Development team is working on scaling up the dataset and enhancing the model for even more powerful performance. Stay tuned for future updates!