Style2Vec: Koyon Wakilcin Kayayyakin Kaya daga Tsarin Salo

1. Gabatarwa

Tare da saurin girma na kasuwar kaya ta kan layi, akwai bukatu mai mahimmanci don ingantattun tsarin shawara. Hanyoyin tacewa na haɗin gwiwa na gargajiya, waɗanda suka dogara da tarihin siye na mai amfani (kima), ba su dace da kaya ba. Tarihin mai amfani na iya ƙunsar salo daban-daban (misali, riguna na yau da kullun da jeans na yau da kullun), wanda ke sa ya zama ba zai yiwu a koyi fasali masu daidaituwa, masu ƙayyadaddun salo don abubuwa ko kayan sawa na ɗaiɗaikun mutane ba. Kalubalen ainihi shine ƙirƙirar samfurin ma'anar "dacewar salo" mai sauƙi, sau da yawa na zahiri tsakanin abubuwa.

Wannan takarda ta gabatar da Style2Vec, sabon samfurin wakilci mai rarrabawa don kayayyakin kaya. An yi wahayi daga ma'anar rarrabawa a cikin NLP (misali, Word2Vec), yana koyon shigar abubuwa daga "tsarin salo" da mai amfani ya tsara—tarin tufafi da kayan haɗi waɗanda suka zama kayan sawa masu haɗaka. Babban ƙirƙira shine amfani da Cibiyoyin Jijiyoyin Convolutional (CNNs) azaman ayyukan tsinkaya daga hotunan abu zuwa vectors masu shigarwa, wanda ke shawo kan matsalar yalwa inda abubuwa na ɗaiɗaiku suka bayyana a cikin ƴan tsarin salo.

2. Hanyar Aiki

2.1. Tsarin Matsala & Tsarin Salo

An ayyana tsarin salo azaman tarin abubuwa (misali, jaket, riga, wando, takalmi, jaka) waɗanda tare suka zama kayan sawa guda ɗaya, masu haɗaka. Yana kama da "jimla" a cikin NLP, yayin da kowane abu na kaya na musamman shine "kalma". Manufar samfurin ita ce koyon aiki $f: I \rightarrow \mathbb{R}^d$ wanda ke zana hoton abu $I$ zuwa vector na salo na latent mai girma $d$, ta yadda abubuwan da ke cikin tsarin salo ɗaya suna da vectors masu kama a cikin sararin shigarwa.

2.2. Tsarin Style2Vec

Samfurin yana amfani da Cibiyoyin Jijiyoyin Convolutional (CNNs) daban-daban guda biyu:

CNN Shigarwa ($\text{CNN}_i$): Yana sarrafa hoton abin da ake nufi wanda ake koyon wakilcinsa.
CNN Mahallin ($\text{CNN}_c$): Yana sarrafa hotunan abubuwan mahallin (sauran abubuwa a cikin tsarin salo ɗaya).

Dukansu cibiyoyin sadarwa suna zana hotunan shigar da su zuwa sararin shigarwa ɗaya mai girma $d$. Wannan hanyar cibiyar sadarwa biyu tana ba da damar samfurin ya bambanta tsakanin rawar abin da ake nufi da mahallinsa yayin koyo.

2.3. Manufar Horarwa

An horar da samfurin ta amfani da manufar koyo mai kwatanta da aka yi wahayi daga tsalle-gram tare da samfurin mara kyau. Don tsarin salo da aka bayar $S = \{i_1, i_2, ..., i_n\}$, manufar ita ce haɓaka yuwuwar ganin kowane abu na mahallin $i_c$ idan aka ba da abin da ake nufi $i_t$. Aikin manufa don nau'i-nau'i ɗaya (abin da ake nufi, mahallin) shine:

$$ J(\theta) = \log \sigma(\mathbf{v}_{i_t} \cdot \mathbf{v}_{i_c}) + \sum_{k=1}^{K} \mathbb{E}_{i_k \sim P_n} [\log \sigma(-\mathbf{v}_{i_t} \cdot \mathbf{v}_{i_k})] $$

inda $\mathbf{v}_{i} = \text{CNN}(I_i)$ shine shigarwar abu $i$, $\sigma$ shine aikin sigmoid, kuma $P_n$ shine rarraba amo don samfurin mara kyau na $K$ misalai marasa kyau.

3. Tsarin Gwaji

3.1. Bayanan Gwaji

An horar da samfurin akan tsarin salo 297,083 da mai amfani ya ƙirƙira da aka tattara daga shahararren gidan yanar gizon kaya. Kowane saiti yana ƙunsar hotunan abubuwa da yawa daga nau'ikan daban-daban (saman, ƙasa, takalmi, kayan haɗi).

Kididdigar Bayanan Gwaji

Jimlar Tsarin Salo: 297,083

Matsakaicin Abubuwa a kowane Saiti: ~5-7

Nau'ikan Abu: Daban-daban (tufafi, takalmi, kayan haɗi)

3.2. Samfurori na Asali

An kwatanta aiki da samfurori na asali da yawa:

Mai Tushen Nau'i: Amfani da nau'ikan abubuwa masu zafi guda ɗaya azaman fasali.
Mai Tushen Sifa: Amfani da sifofin gani da aka ƙera da hannu (launi, tsari).
Fasalin CNN: Amfani da fasalin CNN da aka riga aka horar (misali, ResNet) daga hotunan abu na ɗaiɗaiku, ba tare da la'akari da mahallin saiti ba.
Word2Vec na Gargajiya akan Nau'ikan: Kula da nau'ikan abu azaman "kalmomi" a cikin "jumloli" na tsarin salo.

3.3. Ma'aunin Kimantawa

An yi amfani da hanyoyin kimantawa na farko guda biyu:

Gwajin Kwatancen Kaya: Yayi kama da gwajin "sarki - namiji + mace = sarauniya" a cikin shigar kalmomi. Yana kimanta idan vectors da aka koya sun ɗauki alaƙar ma'ana (misali, "takalmin ƙafa - hunturu + bazara = sandal").
Rarraba Salo: Amfani da fasalin Style2Vec da aka koya azaman shigarwa ga mai rarrabawa don hasashen alamun salo da aka riga aka ayyana (misali, na yau da kullun, punk, kasuwanci na yau da kullun). Ana amfani da daidaito azaman ma'auni.

4. Sakamako & Bincike

4.1. Gwajin Kwatancen Kaya

Style2Vec ya yi nasarar warware kwatancen kaya iri-iri, yana nuna cewa shigarsa tana ɗaukar ma'ana mai wadata fiye da nau'ikan asali. Misalai sun haɗa da canje-canje masu alaƙa da:

Lokacin Shekara: Abu na hunturu → Abu na bazara.
Tsari: Abu na yau da kullun → Abu na yau da kullun.
Launi/Tsari: Abu mai launi mai ƙarfi → Abu mai tsari.
Silhouette/Siffa: Abu da aka dace → Abu maras kyau.

Wannan yana nuna samfurin ya koyi wakilcin da aka raba inda takamaiman ma'auni ko kwatance a cikin sararin vector sun dace da sifofin salo masu fassara.

4.2. Aikin Rarraba Salo

Lokacin da aka yi amfani da shi azaman fasali don mai rarrabawar salo, shigarwar Style2Vec ta fi duk hanyoyin asali sosai. Babban fahimta shine cewa fasalin da aka koya daga haɗuwa a cikin tsarin salo sun fi hasashen alamun salo gaba ɗaya fiye da fasalin daga hotuna na ɗaiɗaiku (CNN baselines) ko metadata (nau'i/sifa baselines). Wannan yana tabbatar da ainihin hasashe cewa salo dukiya ce ta alaƙa mafi kyau a koya daga mahallin.

Mahimman Fahimta

Mahallin shine Sarki: Salo ba dukiya ce ta cikin abu ba amma yana fitowa daga alaƙarsa da wasu abubuwa.
Cin nasara akan Yalwa: Amfani da CNNs azaman cibiyoyin sadarwa masu horarwa yana magance matsalar yalwar bayanai da ke cikin ɗaukar kowane abu na musamman azaman alamar discrete.
Ma'ana mai Wadata: Sararin shigarwa yana tsara abubuwa tare da fassarar salo masu fassara da yawa, yana ba da damar yin tunani mai rikitarwa.

5. Cikakkun Bayanai na Fasaha & Tsarin Lissafi

Babban ƙirƙira yana cikin daidaita tsarin Word2Vec don yankin gani. Bari $D = \{S_1, S_2, ..., S_N\}$ ya zama gawa na tsarin salo. Don tsarin salo $S = \{I_1, I_2, ..., I_m\}$, inda $I_j$ hoto ne, muna samfurin abin da ake nufi $I_t$ da abu na mahallin $I_c$ daga $S$.

Ana ƙididdige shigarwa kamar haka: $$\mathbf{v}_t = \text{CNN}_i(I_t; \theta_i), \quad \mathbf{v}_c = \text{CNN}_c(I_c; \theta_c)$$ inda $\theta_i$ da $\theta_c$ su ne sigogin shigar da CNNs na mahallin, bi da bi. Ana horar da cibiyoyin sadarwa har zuwa ƙarshe ta hanyar haɓaka aikin manufa $J(\theta)$ da aka ayyana a Sashe na 2.3 a cikin duk nau'ikan (abin da ake nufi, mahallin) a cikin bayanan. Bayan horo, kawai CNN Shigarwa ($\text{CNN}_i$) ana amfani da shi don samar da shigarwar Style2Vec na ƙarshe don kowane sabon hoton abu.

6. Tsarin Bincike: Nazarin Lamari Ba tare da Lamba ba

Yanayi: Dandalin kasuwancin kaya na kan layi yana son inganta widget ɗin shawararsa na "Cika Kallon".

Hanyar Gargajiya: Widget ɗin yana ba da shawarar abubuwa bisa ga mitar sayayya tare ko alamun nau'i ɗaya (misali, "abokan ciniki waɗanda suka sayi wannan blazer suma sun sayi waɗannan wando"). Wannan yana haifar da shawarwari na gama gari, sau da yawa masu rashin dacewa da salo.

Hanyar da Style2Vec ya Ba da Damar:

Samar da Shigarwa: Duk abubuwan da ke cikin kasida ana sarrafa su ta hanyar CNN Shigarwa da aka horar don samun vectors ɗin Style2Vec.
Tsarin Tambaya: Mai amfani ya ƙara wando na chino navy da farar takalmi zuwa keken su. Dandalin ya matsakaici vectors ɗin Style2Vec na waɗannan abubuwa biyu don ƙirƙirar "vector tambaya" wanda ke wakiltar tsarin salo mai farawa.
Binciken Maƙwabta Mafi Kusa: Tsarin yana bincika sararin shigarwa don abubuwan da vectors ɗinsu suka fi kusa da vector tambaya. Ya dawo da, misali, rigar Oxford mai launin shuɗi mai haske, suweter mai tsiri, da bel na zane.
Sakamako: Shawarwari ba kawai ana sayayya tare ba amma suna da haɗakar salo tare da abubuwan da mai amfani ya zaɓa, suna haɓaka kallon yau da kullun, mai wayo. Dandalin zai iya bayyana shawarwari ta hanyar kwatanta: "Mun ba da shawarar wannan rigar saboda ta cika kallon ku na yau da kullun, kamar yadda blazer ke cika na yau da kullun."

Wannan tsarin yana canza dabarun shawara daga haɗin kai na ƙididdiga zuwa dacewar salo na ma'ana.

7. Ra'ayi na Mai Binciken Masana'antu

Fahimta ta Asali: Style2Vec ba wani samfurin shigarwa kawai bane; yana juyawa daga ƙirƙirar ɗanɗano mai amfani zuwa ƙirƙirar ma'anar abu a cikin mahallin salo. Takarda ta gano kuskuren asali a cikin amfani da tacewa na haɗin gwiwa na gargajiya ga kaya: tarihin siye na mai amfani sigina ne mai hayaniya, salo da yawa. Ta mayar da hankali kan kayan sawa (tsarin salo) azaman rukunin atomic na salo, suna ƙetare wannan hayaniyar kuma suna ɗaukar ainihin kaya—wanda ke haɗawa da alaƙa. Wannan ya yi daidai da manyan abubuwan da ke faruwa a cikin AI suna matsawa zuwa tunani na alaƙa da na jadawali, kamar yadda aka gani a cikin samfurori kamar Cibiyoyin Jijiyoyin Jadawali (GNNs) da aka yi amfani da su ga hanyoyin sadarwar jama'a ko jadawalin ilimi.

Kwararar Ma'ana: Hujja tana da ban sha'awa. 1) Matsala: Shawarar tushen tarihin mai amfani ta kasa aiki don salo. 2) Fahimta: An ayyana salo ta haɗuwar abu a cikin kayan sawa. 3) Aro: Hasashen rarrabawa na NLP (kalmomi a cikin mahalli iri ɗaya suna da ma'ana iri ɗaya). 4) Daidaitawa: Maye gurbin kalmomi da hotunan abu, jimloli da tsarin salo. 5) Warware Yalwa: Amfani da CNNs azaman masu ɓoyewa masu horarwa maimakon teburin duba. 6) Tabbatarwa: Nuna shigarwar tana aiki ta hanyar kwatanta da ayyukan rarrabawa. Ma'ana tana da tsabta kuma zaɓin injiniya (CNNs biyu, samfurin mara kyau) daidaitattun dabarun da aka tabbatar.

Ƙarfi & Kurakurai:

Ƙarfi: Babban ƙarfin takarda shine bayyananniyar ra'ayi da canja wurin yanki mai tasiri. Amfani da CNNs don sarrafa shigarwar gani da yalwa yana da kyau. Gwajin kwatancen kaya ƙwaƙƙwaran ma'auni ne na kimantawa mai fahimta wanda nan da nan ke isar da iyawar samfurin, kamar yadda takardar Word2Vec ta asali ta yi wa NLP.
Kurakurai & Gaps: Samfurin a zahiri yana da amsawa da bayyani, ba ƙirƙira ba. Yana koyo daga tsarin salo da mai amfani ya ƙirƙira, yana iya ƙarfafa shahararrun salo ko na al'ada kuma yana fama da haɗuwar avant-garde ko sababbi—iyaka sananne na hanyoyin rarrabawa. Hakanan yana ƙetare al'amarin keɓancewa. Salo na "punk" na iya bambanta da naku. Kamar yadda aka lura a cikin babban aikin kan tacewa na haɗin gwiwa na jijiyoyi ta He et al. (2017, WWW), manufa ta ƙarshe ita ce aikin keɓancewa. Style2Vec yana ba da kyakkyawan wakilcin abu amma bai ƙirƙira yadda takamaiman mai amfani ke hulɗa da wannan sararin salo ba.

Fahimta masu Aiki:

Ga Masu Bincike: Mataki na gaba nan da nan shine haɗakarwa. Haɗa shigarwar abu na mahallin mai wayo na Style2Vec tare da na'urar keɓancewa ta mai amfani (misali, tsarin shawarar jijiyoyi). Bincika ɗan gajeren harbi ko sifili-shot salo koyo don karya son zuciya na shahara.
Ga Masu Aiki (Kasuwancin Kan Layi, Ayyukan Salo): Aiwatar da wannan samfurin azaman sabis na gindi don daidaita kayan sawa, salon wardrobe na zahiri, da bincike-ta-salo. Komawar zuba jari a bayyane yake: ƙara matsakaicin ƙimar oda ta hanyar ingantattun shawarwari na "cika kallon" da ingantaccen haɗin gwiwar abokin ciniki ta hanyar kayan aikin binciken salo mai hulɗa ("nemo abubuwan da suke salo kamar wannan").
Ɗaukar Dabarun: Makomar kaya AI tana cikin tsarin multi-modal, mahalli-mai wayo. Style2Vec mataki ne mai mahimmanci fiye da binciken gani kawai (kamar wanda Bayanan DeepFashion suka yi) da tacewa na haɗin gwiwa kawai. Dandalin da zai yi nasara shine wanda zai iya haɗa wannan nau'in fahimtar salo na ma'ana tare da ƙirar son ra'ayi na mai amfani ɗaya kuma watakila ma iyawar ƙirƙira don ƙirƙirar sabbin salo na zahiri, kamar yadda samfurori kamar DALL-E 2 ko Stable Diffusion ke samar da hotuna daga umarnin rubutu, amma an takura ta hanyar yiwuwar kaya.

8. Aikace-aikace na Gaba & Hanyoyin Bincike

Style2Vec Na Musamman: Ƙara samfurin don koyon shigar salo na musamman na mai amfani, yana ba da damar "salo a gare ku" maimakon kawai "salo gabaɗaya". Wannan na iya haɗawa da tsarin gine-gine biyu wanda ke haɗa masu ɓoyewa na abu da mai amfani.
Koyon Salo na Tsakanin Hanyoyi: Haɗa bayanin rubutu (taken samfur, bita na mai amfani) da bayanan kafofin watsa labarai (post ɗin Instagram tare da hashtags) tare da hotuna don ƙirƙirar wakilcin salo mai wadata, mai yawa.
Aikace-aikace na Ƙirƙirar Salo: Amfani da sararin salo da aka koya azaman tsarin sharadi don cibiyoyin sadarwar adawa (GANs) kamar StyleGAN ko samfurori masu yaduwa don samar da sabbin ƙirar tufafi waɗanda suka dace da salo da ake nufi, ko don "gwada" salo daban-daban ta hanyar sarrafa shigar abu. Bincike a cikin fassarar hoto-zuwa-hoto, kamar CycleGAN (Zhu et al., 2017), yana nuna yuwuwar canza bayyanar abu a cikin yankuna, wanda za a iya jagoranta ta hanyar kwatancen Style2Vec.
Hasashen Salon Salon Salon: Bin diddigin juyin halittar cibiyoyin vector na salo akan lokaci don hasashen abubuwan da ke tasowa, kamar yadda aka yi amfani da shigar kalmomi don bin sauyin ma'ana a cikin harshe.
Kaya mai Dorewa: Ba da shawarar abubuwa masu haɗakar salo na biyu ko haya ta hanyar nemo maƙwabta mafi kusa a cikin sararin Style2Vec, yana haɓaka tattalin arzikin kaya na madauwari.

9. Nassoshi

Lee, H., Seol, J., & Lee, S. (2017). Style2Vec: Koyon Wakilcin Kayayyakin Kaya daga Tsarin Salo. arXiv preprint arXiv:1708.04014.
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Ingantaccen Ƙididdiga na Wakilcin Kalma a cikin Sararin Vector. arXiv preprint arXiv:1301.3781.
He, X., Liao, L., Zhang, H., Nie, L., Hu, X., & Chua, T. S. (2017) Tace Haɗin gwiwar Jijiyoyi. A cikin Proceedings of the 26th International Conference on World Wide Web (shafi na 173–182).
Liu, Z., Luo, P., Qiu, S., Wang, X., & Tang, X. (2016). DeepFashion: Ƙarfafa Ƙwaƙƙwaran Gane Tufafi da Maido da Bayanai Masu Wadata. A cikin Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
Zhu, J. Y., Park, T., Isola, P., & Efros, A. A. (2017). Fassarar Hoton da ba a haɗa shi ba ta amfani da Cibiyoyin Sadarwar Adawa na Tsarin Zagayowar. A cikin Proceedings of the IEEE International Conference on Computer Vision (ICCV).
Karras, T., Laine, S., & Aila, T. (2019). Tsarin Ƙirƙirar Salo don Cibiyoyin Sadarwar Adawa. A cikin Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).