Achieve AI Image Editing with ICEdit: A Guide to In-Context Image Manipulation

Want to edit images using AI, but overwhelmed by complex tools and massive datasets? ICEdit, or In-Context Edit, is a novel and powerful approach to instructional image editing. It delivers state-of-the-art results while using significantly less training data and fewer parameters than previous methods. This means faster processing and lower costs!

What is In-Context Edit (ICEdit)?

ICEdit lets you edit images simply by providing instructions. It achieves impressive results by using "in-context generation," meaning it learns from the provided image and instructions without needing extensive retraining for every new task. Think of it as a smart AI that understands your needs and adapts quickly.

• Instruction-based editing: Simply tell ICEdit what you want to change.
• Efficient training: Requires only 0.5% of the data and 1% of the parameters compared to other state-of-the-art methods.
• High precision: Achieve accurate and visually appealing results.

Getting Started with ICEdit: Installation and Setup

Ready to try ICEdit? Here's how to get it up and running:

1. Create a Conda environment: This helps manage dependencies.

   conda create -n icedit python=3.10
   conda activate icedit
   

   Copy

2. Install required packages: Use the requirements.txt file.

   pip install -r requirements.txt
   pip install -U huggingface_hub
   

   Copy

3. Download pretrained weights: Only necessary if you cannot connect to Huggingface. The weights consist of Flux.1-fill-dev and ICEdit-MoE-LoRA.

Running Inference: Editing Your Images

Once installed, you can start editing images! Here's how to run inference from the command line:

python scripts/inference.py --image assets/girl.png \
--instruction " Make her hair dark green and her clothes checked. " \
--seed 42

Important considerations:

• Image size: ICEdit works best with images 512 pixels wide. It will automatically resize other images.

• GPU memory: Editing a 512x768 image requires 35GB of GPU memory. Use the --enable-model-cpu-offload parameter for systems with 24GB GPUs (like NVIDIA RTX3090).

  python scripts/inference.py --image assets/girl.png \
  --instruction " Make her hair dark green and her clothes checked. " \
  --enable-model-cpu-offload
  

  Copy

• Local weights: If you've downloaded the weights locally, specify their paths by using the --flux-path and --lora-path parameters.

  python scripts/inference.py --image assets/girl.png \
  --instruction " Make her hair dark green and her clothes checked. " \
  --flux-path /path/to/flux.1-fill-dev \
  --lora-path /path/to/ICEdit-MoE-LoRA
  

  Copy

• Troubleshooting: If the results aren't what you expected, try changing the --seed parameter to get different results.

The Gradio Demo: A User-Friendly Interface for In-Context Image Editing

For a more visual and interactive experience, use the Gradio demo:

python scripts/gradio_demo.py --port 7860

Access the demo in your browser using the link provided.

Like the inference script, you have some options when running the Gradio demo:

• --enable-model-cpu-offload to enable CPU offloading for systems with less GPU memory.
• --flux-path to the path of the Flux.1-fill-dev file.
• --lora-path to the path of the ICEdit-MoE-LoRA file.

ICEdit vs. Commercial Models: Why Choose ICEdit for Image Instruction Following?

ICEdit offers several advantages over commercial models like Gemini and GPT-4o:

• Comparable or superior performance: ICEdit excels at preserving character identity and following instructions accurately.
• Open-source: Greater transparency and community support.
• Lower cost: Reduce compute expenditure with a smaller model.
• Faster speed: It takes only about 9 seconds to process an image.

Limitations to Keep in Mind

While ICEdit is powerful, be aware of its limitations:

• Style transfer: The base model sometimes changes the artistic style of the image unexpectedly.
• Realistic images: ICEdit is primarily trained on realistic images, which degrades performance on anime or blurry images.
• Object removal: Object removal may be less accurate due to dataset quality.

The developers are actively working on improving these limitations. An improved training dataset to scale model performance is forthcoming.

Contribute and Stay Updated

This is an actively developed project. Future plans include:

• Enhanced datasets for improved performance.
• Scale-up of the model for even greater capabilities.

Keep up with the latest news and updates on the project page! Try ICEdit today and experience the future of AI-powered image editing.