SGEdit

SGEdit is an innovative image editing tool that combines large language models (LLM) with text-to-image generative models to enable highly precise and flexible image editing based on scene graphs. Developed by researchers at the City University of Hong Kong and Microsoft GenAI, SGEdit allows users to make complex adjustments—such as adding, removing, replacing, and modifying objects—while preserving image quality and consistency. This tool uses a scene graph to represent objects and their relationships, offering an intuitive, structured way to navigate and edit image elements.

SGEdit consists of a two-step process: first, it parses an image’s scene graph to capture objects, relationships, and fine-grained attributes. Then, using a diffusion model fine-tuned with the scene graph annotations, it executes targeted edits directed by an LLM editing controller. This unique integration enables detailed and visually coherent edits, outperforming traditional methods in both precision and aesthetic coherence.

Key Features:

• Scene Graph-Based Editing: Leverages scene graphs to provide an intuitive, structured interface for object-level image editing.

• Precise Object-Level Edits: Easily add, remove, replace, or adjust objects without disrupting overall image quality.

• LLM and Generative Model Integration: Combines LLMs with text-to-image models for edits guided by detailed text descriptions and object relationships.

• High Consistency: Ensures that modifications blend seamlessly with the original image, preserving visual integrity and aesthetics.

• Intuitive User Interface: Allows modifications via scene graph nodes and edges, making complex edits accessible and efficient.

Use Cases:

• Creative Image Manipulation: Ideal for artists and designers who want to alter scenes with high precision.

• Object-Based Adjustments: Use for targeted modifications in complex images, such as replacing or repositioning objects in visual storytelling.

• Educational and Training Applications: Supports experiments in object recognition and relationships within images for visual AI research.

With SGEdit, users can enjoy unparalleled control over image modifications, thanks to the powerful combination of scene graphs and generative AI, making it an ideal tool for creatives, researchers, and AI enthusiasts alike.