Fashion-Diffusion Dataset: Dubun Dubun Hotuna Masu Ingantacciyar Zane-zane na AI

Tsarin Abubuwan Ciki

1. Gabatarwa

Haɗuwar Hankalin Wucin Gadi (AI) da ƙirar zane-zane yana wakiltar wani sauyi mai canzawa a cikin hangen nesa na kwamfuta da masana'antu na ƙirƙira. Duk da yake samfuran rubutu-zuwa-hoto (T2I) kamar DALL-E, Stable Diffusion, da Imagen sun nuna iyawa mai ban mamaki, aikace-aikacensu a cikin fagage na musamman kamar ƙirar zane-zane ya kasance cikin takurawa mai mahimmanci: rashin manyan, ingantattun, da kuma takamaiman bayanai na yanki.

Bayanan zane-zane da ake da su, kamar DeepFashion, CM-Fashion, da Prada, suna fama da iyakoki a cikin sikelin (sau da yawa <100k hotuna), ƙuduri (misali, 256x256), cikakkiyar fahimta (rashin cikakkun siffofi na mutum ko cikakkun bayanin rubutu), ko ƙayyadaddun bayanin bayanai. Wannan takarda ta gabatar da Dataset na Fashion-Diffusion, ƙoƙari na shekaru da yawa don cike wannan gibi. Ya ƙunshi fiye da dubun dubun hotuna masu inganci (768x1152) na zane-zane, kowannensu yana haɗe da cikakkun bayanin rubutu da ke rufe duka siffofin tufafi da na ɗan adam, waɗanda aka samo daga bambance-bambancen yanayin zane-zane na duniya.

2. Dataset na Fashion-Diffusion

2.1 Gina & Tattara Dataset

An fara shi a cikin 2018, ginin dataset ya haɗa da tattarawa da tsarawa mai zurfi daga babban ma'ajiyar hotunan tufafi masu inganci. Babban abin banbance shine mayar da hankali kan bambance-bambancen duniya, tattara hotuna daga yanayi daban-daban na yanki da al'adu don haɗa yanayin zane-zane na duniya, ba kawai salon Yammacin duniya ba.

Tsarin ya haɗa da hanyoyin atomatik da na hannu. An bi tattarawa ta farko da tacewa mai tsauri don inganci da dacewa. An yi amfani da dabarar bayanin haɗin gwiwa, yana amfani da duka gano batun atomatik/rarraba da tabbatar da hannu ta ƙwararrun masu ƙirar tufafi don tabbatar da daidaito da cikakkun bayanai.

2.2 Bayanin Bayanai & Siffofi

Tare da haɗin gwiwar ƙwararrun zane-zane, ƙungiyar ta ayyana cikakkiyar ilimin halittu na siffofi masu alaƙa da tufafi. Dataset na ƙarshe ya haɗa da 8,037 siffofi masu lakabi, yana ba da damar sarrafa ƙayyadaddun sarrafa tsarin samar da T2I. Siffofi sun haɗa da:

• Cikakkun Bayanan Tufafi: Rukuni (riga, riga, wando), salo (bohemian, minimalist), masana'anta (silk, denim), launi, tsari, wuyan wuya, tsayin hannun riga.
• Yanayin ɗan Adam: Matsayi, nau'in jiki, jinsi, rukunin shekaru, hulɗa da tufafin.
• Yanayi & Mahallin: Lokaci (na yau da kullun, na yau da kullun), saitin.

Kowane hoto yana haɗe da ɗaya ko fiye da cikakkun bayanin rubutu, wanda ya haifar da nau'i-nau'i 1.59M na rubutu da hoto, yana ƙara wadata daidaitawar ma'ana mai mahimmanci don horar da samfuran T2I.

2.3 Ƙididdiga & Siffofin Dataset

• Sikelin: Hotuna 1,044,491.
• Ƙudurin: Babban ƙuduri 768x1152, wanda ya dace don cikakken hangen nesa na ƙira.
• Nau'i-nau'i na Rubutu da Hotuna: Bayanin 1,593,808.
• Bambance-bambance: Tushen yanki da al'adu daban-daban.
• Zurfin Bayanin Bayanai: 8,037 ƙayyadaddun siffofi.
• Mai Daidaitawa da ɗan Adam: Mayar da hankali kan cikakkun siffofi na mutum sanye da tufafi, ba kawai keɓaɓɓun abubuwan tufafi ba.

3. Ma'auni na Gwaji & Sakamako

3.1 Ma'aunin Kimantawa

Ma'aunin da aka gabatar yana kimanta samfuran T2I akan ginshiƙai da yawa ta amfani da ma'auni na yau da kullun:

• Nisan Fréchet Inception (FID): Yana auna kamanceceniya tsakanin rarraba hotunan da aka samar da na ainihi. Ƙananan yana da mafi kyau.
• Maki Inception (IS): Yana tantance inganci da bambancin hotunan da aka samar. Babba yana da mafi kyau.
• CLIPScore: Yana kimanta daidaitawar ma'ana tsakanin hotunan da aka samar da umarnin rubutu na shigarwa. Babba yana da mafi kyau.

3.2 Nazarin Kwatance

An kwatanta samfuran da aka horar akan Fashion-Diffusion da waɗanda aka horar akan wasu fitattun bayanan zane-zane (misali, DeepFashion-MM). Kwatancen ya nuna tasirin ingancin dataset da sikelin akan aikin samfurin.

3.3 Sakamako & Aiki

Sakamakon gwaji ya nuna fifikon samfuran da aka horar akan dataset na Fashion-Diffusion:

• FID: 8.33 (Fashion-Diffusion) vs. 15.32 (Tushe). Ingantacciyar ~46%, yana nuna hotunan da aka samar sun fi zama na hoto kuma sun dace da ainihin bayanai.
• IS: 6.95 vs. 4.7. Ingantacciyar ~48%, yana nuna mafi kyawun ingancin hoto da bambancin da aka gane.
• CLIPScore: 0.83 vs. 0.70. Ingantacciyar ~19%, yana nuna mafi kyawun daidaitawar ma'ana na rubutu da hoto.

Bayanin Jadawali (Tunani): Jadawali mai suna "Kwatancen Aikin Samfurin T2I" zai nuna nau'i-nau'i uku na sanduna don FID, IS, da CLIPScore. Sandunan "Fashion-Diffusion" za su kasance mafi girma sosai (don IS, CLIPScore) ko ƙasa (don FID) fiye da sandunan "Dataset na Tushe", yana tabbatar da fifikon ƙididdiga da aka ruwaito a cikin rubutu.

4. Tsarin Fasaha & Hanyoyin Aiki

4.1 Tsarin Haɗa Rubutu zuwa Hotuna

Binciken yana amfani da samfuran watsawa, na yanzu na zamani don samar da T2I. Tsarin yawanci ya haɗa da:

1. Rufewa Rubutu: Ana rufe umarnin rubutu na shigarwa zuwa wakilci mai ɓoye ta amfani da samfuri kamar CLIP ko T5.
2. Tsarin Watsawa: Tsarin U-Net yana sake cire hayaniyar Gaussian bazuwar akai-akai, yana bin jagorancin abubuwan da aka haɗa na rubutu, don samar da hoto mai daidaituwa. Tsarin yana bayyana ta hanyar gaba (hayaniya) da juyawa (cire hayaniya) Markov.
3. Sarrafa Ƙayyadaddun Bayanai: Cikakkun lakabin siffofi a cikin Fashion-Diffusion suna ba da damar daidaita tsarin watsawa akan takamaiman siffofi, yana ba da damar sarrafa ainihin abubuwan zane-zane da aka samar.

4.2 Tushen Lissafi

Ginshiƙin samfuran watsawa ya haɗa da koyon juyar da tsarin hayaniya na gaba. Idan aka ba da ma'auni na bayanai $x_0$ (hoton ainihi), tsarin gaba yana samar da jerin abubuwan da ke ƙara hayaniya $x_1, x_2, ..., x_T$ sama da matakai $T$:

$q(x_t | x_{t-1}) = \mathcal{N}(x_t; \sqrt{1-\beta_t} x_{t-1}, \beta_t I)$

inda $\beta_t$ ke tsarin bambance-bambance. Tsarin juyawa, wanda aka ƙayyade ta hanyar hanyar sadarwar jijiyoyi $\theta$, yana koyon cire hayaniya:

$p_\theta(x_{t-1} | x_t) = \mathcal{N}(x_{t-1}; \mu_\theta(x_t, t), \Sigma_\theta(x_t, t))$

Horon ya haɗa da inganta ƙananan iyaka na bambance-bambance. Don samarwa na sharadi (misali, tare da rubutu $y$), samfurin yana koyon $p_\theta(x_{t-1} | x_t, y)$. Ingantattun nau'i-nau'i masu daidaito a cikin Fashion-Diffusion suna ba da sigina mai ƙarfi na horo don koyon wannan rarraba na sharadi $p_\theta$ a cikin yankin zane-zane.

5. Babban Fahimta & Ra'ayi na Manazarta

6. Tsarin Aikace-aikace & Nazarin Hali

Tsarin don Ƙirar Zane-zane Mai Taimakon AI:

1. Shigarwa: Mai zane yana ba da taƙaitaccen bayani na yau da kullun (misali, "rigar bazara mai gudana, tsakiyar tsayi a cikin chiffon lavender tare da hannun riga mai kumbura, don liyafar lambu") ko zaɓi takamaiman siffofi daga ilimin halittu.
2. Samarwa: Samfurin watsawa (misali, daidaitaccen Stable Diffusion) da aka horar akan Fashion-Diffusion yana samar da ra'ayoyin gani masu inganci da yawa.
3. Gyara: Mai zane yana zaɓe da maimaitawa, yana iya amfani da fasahar shigar ciki ko img2img don gyara takamaiman yankuna (misali, canza wuyan wuya, daidaita tsayi).
4. Fitarwa: Ƙirar gani ta ƙarshe don ƙirar samfuri ko ƙirar kadara ta dijital.

Nazarin Hali Ba tare da Lamba ba: Hasashen Yanayi & Saurin Ƙirar Samfuri
Dillalin saurin zane-zane yana son samun riba daga wani sabon yanayi na "cottagecore" aesthetics da aka gano ta hanyar nazarin kafofin watsa labarun. Ta amfani da tsarin T2I mai ƙarfin Fashion-Diffusion, ƙungiyar ƙirar su ta shigar da umarni kamar "cottagecore linen pinafore dress, smocked bodice, prairie aesthetic" kuma ta samar da ɗaruruwan bambance-bambancen ƙira na musamman a cikin sa'o'i. Ana yin bitar su da sauri, ana zaɓar manyan 10 don samfurin dijital, kuma lokutan jigilar kaya daga gano yanayi zuwa samfuri ana yanke su daga makonni zuwa kwanaki, yana inganta amsa kasuwa sosai.

7. Aikace-aikace na Gaba & Jagorori

• Zane-zane Mai Keɓancewa: Haɗa ma'aunin jiki na musamman na mai amfani da abubuwan da ake so don samar da ƙira na tufafi na musamman, masu dacewa da mutum.
• Gwada Wuta ta Wucin Gadi & Zane-zane na Metaverse: Yin aiki azaman tushen dataset don samar da tufafin dijital na gaske don avatars a cikin duniyoyin wucin gadi da dandamali na zamantakewa.
• Ƙira Mai Dorewa: Ingantaccen kayan aiki na AI da samar da tsarin sharar gida mara sharar gida wanda aka sanar da shi ta cikakkun siffofin tufafi.
• Kayan Aikin Haɗin Gina: Mataimakan ƙirar AI na ainihin lokaci, masu tattaunawa inda masu zane za su iya gyara ra'ayoyi akai-akai ta hanyar tattaunawa.
• Binciken Zane-zane Mai Tsaka-tsaki: Ba da damar neman abubuwan tufafi ta amfani da zane-zane, yare mai bayyanawa, ko ma hotunan da aka loda na salon da ake so, wanda ke da ƙarfi ta sararin haɗin rubutu da hoto da aka koya daga dataset.
• Daidaita Da'a & Son Kai: Aikin gaba dole ne ya mayar da hankali kan bincike da rage son zuciya na dataset don tabbatar da wakilci mai adalci a cikin nau'ikan jiki, kabilu, da al'adu, hana ci gaba da ra'ayoyin masana'antar zane-zane.

8. Nassoshi

1. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., & Ommer, B. (2022). Haɗa Hotuna Masu Ƙuduri Mai Girma tare da Samfuran Watsawa na Latent. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
2. Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Fassarar Hotuna-zuwa-Hoto mara Haɗin gwiwa ta amfani da Cibiyoyin Adawa na Ci gaba da Ci gaba. Proceedings of the IEEE International Conference on Computer Vision (ICCV).
3. Liu, Z., Luo, P., Qiu, S., Wang, X., & Tang, X. (2016). DeepFashion: Ƙarfafa Ƙwararrun Gane Tufafi da Maido da Cikakkun Bayanai. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
4. Cibiyar AI Now. (2019). Nakasa, Son Kai, da AI. An samo daga https://ainowinstitute.org
5. Ge, Y., Zhang, R., Wang, X., Tang, X., & Luo, P. (2021). DeepFashion-MM: Dataset na Haɗa Rubutu zuwa Hotuna don Zane-zane. ACM Multimedia.
6. Yu, J., Zhang, L., Chen, Z., et al. (2024). Inganci da Ƙididdiga: Bayyana Dubun Dubun Hotuna Masu Inganci don Haɗa Rubutu zuwa Hotuna a cikin Ƙirar Zane-zane. arXiv:2311.12067v3.