arXiv Open Access 2024

Magic Insert: Style-Aware Drag-and-Drop

Nataniel Ruiz Yuanzhen Li Neal Wadhwa Yael Pritch Michael Rubinstein +2 lainnya
Lihat Sumber

Abstrak

We present Magic Insert, a method for dragging-and-dropping subjects from a user-provided image into a target image of a different style in a physically plausible manner while matching the style of the target image. This work formalizes the problem of style-aware drag-and-drop and presents a method for tackling it by addressing two sub-problems: style-aware personalization and realistic object insertion in stylized images. For style-aware personalization, our method first fine-tunes a pretrained text-to-image diffusion model using LoRA and learned text tokens on the subject image, and then infuses it with a CLIP representation of the target style. For object insertion, we use Bootstrapped Domain Adaption to adapt a domain-specific photorealistic object insertion model to the domain of diverse artistic styles. Overall, the method significantly outperforms traditional approaches such as inpainting. Finally, we present a dataset, SubjectPlop, to facilitate evaluation and future progress in this area. Project page: https://magicinsert.github.io/

Penulis (7)

N

Nataniel Ruiz

Y

Yuanzhen Li

N

Neal Wadhwa

Y

Yael Pritch

M

Michael Rubinstein

D

David E. Jacobs

S

Shlomi Fruchter

Format Sitasi

Ruiz, N., Li, Y., Wadhwa, N., Pritch, Y., Rubinstein, M., Jacobs, D.E. et al. (2024). Magic Insert: Style-Aware Drag-and-Drop. https://arxiv.org/abs/2407.02489

Akses Cepat

Lihat di Sumber
Informasi Jurnal
Tahun Terbit
2024
Bahasa
en
Sumber Database
arXiv
Akses
Open Access ✓