In-Context Image Editing: Achieve SOTA Results With Minimal Data & Parameters

Want to edit images based on simple instructions? ICEdit delivers state-of-the-art results while using significantly less data and computing power. Learn how this innovative approach leverages in-context generation for precise and impressive image manipulations.

What is ICEdit and Why Should You Care?

ICEdit ("In-Context Edit") is a groundbreaking method for instruction-based image editing that utilizes a large-scale diffusion transformer. This means you can simply tell the system what changes you want to make, and it will modify the image accordingly. ICEdit achieves top-tier performance using only a fraction of the resources required by previous state-of-the-art techniques.

• Highly Efficient: Uses just 0.5% of the training data and 1% of the parameters compared to other leading methods.
• Precise Editing: Achieve a high degree of accuracy in multi-turn edits.
• Visually Stunning Results: Produce diverse and impressive single-turn edits.

Quick Start: Installation and Inference

Ready to jump in? Here's how to get ICEdit up and running.

Installation Steps:

1. Set up your Conda environment:

   conda create -n icedit python=3.10
   conda activate icedit
   pip install -r requirements.txt
   pip install -U huggingface_hub
   

   Copy
2. Download the pretrained weights: Get them from Hugging Face, or download them locally if you have connection issues. The files are named Flux.1-fill-dev and ICEdit-MoE-LoRA.

Running Inference:

Use the following command to edit an image with a specified instruction, replacing the image path and instruction as needed. Note that the image width must be 512 pixels.

python scripts/inference.py --image assets/girl.png \
--instruction " Make her hair dark green and her clothes checked. " \
--seed 42

• Remember to try different --seed values if you aren't getting the desired results.
• If you have limited GPU memory (24GB like NVIDIA RTX3090), add the --enable-model-cpu-offload parameter.
• If you downloaded the pretrained weights locally, specify their paths using --flux-path and --lora-path.

Improve Image Editing Results By Changing The Seed

The success of image editing can depend on the initial random seed. If your first attempt doesn't produce the desired outcome, simply try running the same command with a different --seed value. Experimentation can lead to significantly improved results.

Handle Artistic Style Changes and Realistic Images

ICEdit's base model may sometimes alter the artistic style of your image due to the style transfer focus in the training data. Also, the model is optimized for realistic images, so results may vary with anime or blurry pictures. Here's what to keep in mind:

• Style Transfer Notice: Be aware that the model might change the artistic style.
• Realistic Images Preferred: Editing works best with realistic images. Expect lower success rates for anime or blurry content.
• Object Removal Challenges: Object removal might be less effective due to limitations in the OmniEdit removal dataset.

Run ICEdit With Gradio for an Easier User Experience

For a more intuitive image editing experience, use the Gradio demo. Run this command:

python scripts/gradio_demo.py --port 7860

• Like the inference script, use --enable-model-cpu-offload for systems with 24GB GPU memory.
• Specify local weight paths with --flux-path and --lora-path if needed.

Then, open the link provided in your browser to start editing.

ICEdit vs. Commercial Models: A Performance Advantage

ICEdit holds its own against commercial giants like Gemini and GPT-4o. It offers comparable or even superior performance in preserving character identity and following instructions, with the added benefits of being open-source, cost-effective, and fast (approximately 9 seconds per image).

Key Advantages:

• Comparable Performance: Matches or exceeds commercial models in key areas.
• Open Source: Transparent and customizable.
• Lower Cost: Reduce expenses drastically.
• Faster Speed: Achieve results in seconds.

Future Enhancements: A Sneak Peek

The development team is committed to making ICEdit even better. Expect these improvements in the future:

• Enhanced Dataset: A more comprehensive training dataset for improved performance.
• Model Scaling: Scale-up for a more powerful model.
• Further Releases: Training code, more inference demos, and ComfyUI demo.

Cite ICEdit

If you find ICEdit useful in your research, please cite the following BibTeX entry:

@misc{zhang2025ICEdit,
 title={In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer},
 author={Zechuan Zhang and Ji Xie and Yu Lu and Zongxin Yang and Yi Yang},
 year={2025},
 eprint={2504.20690},
 archivePrefix={arXiv},
 primaryClass={cs.CV},
 url={https://arxiv.org/abs/2504.20690},
}