Ngumsebenzi obalulekileyo nonqwenelekayo kumbono wekhompyuter kunye nemizobo ukuvelisa iifilimu zoyilo lwemifanekiso ekwizinga eliphezulu.
Nangona iimodeli ezininzi ezisebenzayo ze-portrait image toonification esekwe kwi-StyleGAN enamandla ziye zacetywa, ezi ndlela zijolise kumfanekiso zinemiqobo ecacileyo xa zisetyenziswa neevidiyo, ezinje ngobungakanani besakhelo esisisigxina, imfuneko yokulungelelaniswa kobuso, ukungabikho kweenkcukacha ezingezizo ubuso. , kunye nokungahambelani kwexesha.
Isakhelo soguqulo se-VToonify sisetyenziselwa ukujongana nobunzima obulawulwayo obuphezulu bodluliselo lwesitayile somfanekiso wevidiyo.
Siza kuphonononga olona phononongo lwamva nje lwe-VCoonify kweli nqaku, kubandakanya ukusebenza kwayo, imiqobo, kunye nezinye izinto.
Yintoni iVtoonify?
Isakhelo se-VToonify sivumela ukuhanjiswa kwesitayile sevidiyo esinesoso esiphakamileyo.
I-VToonify isebenzisa i-StyleGAN's's mid- and high-resolution layers ukwenza imizobo yobugcisa obuphezulu obusekwe kumxholo we-multi-scale ofunyenwe yi-encoder ukugcina iinkcukacha zesakhelo.
Uyilo olunesiphumo sokuguqula ngokupheleleyo luthatha ubuso obungahambelaniyo kwiimuvi ezinobungakanani obuguquguqukayo njengegalelo, okukhokelela kwimimandla yobuso obupheleleyo kunye neentshukumo zokwenyani kwisiphumo.
Esi sikhokelo siyahambelana nemodeli yangoku esekwe kwi-StyleGAN ye-toonification yemifanekiso, evumela ukuba yandiswe kwi-toonification yevidiyo, kwaye izuze ilifa leempawu ezinomtsalane ezinjengombala ohlengahlengiswayo kunye nokuqina ngokwezifiso.
le isifundo yazisa amanyathelo amabini e-VToonify asekwe kwi-Toonify kunye ne-DualStyleGAN yokuhanjiswa kwesitayile sevidiyo esekwe kwingqokelela kunye nesekwe kumzekelo, ngokulandelelana.
Ukufunyaniswa kovavanyo olubanzi kubonisa ukuba isakhelo esicetywayo se-VToonify sigqwesa iindlela ezikhoyo zokwenza imiboniso bhanyabhanya ekumgangatho ophezulu, ehambelana okwethutyana yobugcisa obunemilinganiselo eguquguqukayo.
Abaphandi babonelela nge Incwadi yamanqaku kaGoogle Colab, ukuze wenze izandla zakho zibe mdaka kuyo.
Ingaba isebenza kanjani?
Ukufezekisa ukuhanjiswa kwesitayile sevidiyo esinokulungiswa okuphezulu kwesisombululo esiphezulu, i-VToonify idibanisa iingenelo zesakhelo sokuguqulela umfanekiso kunye nesakhelo esekwe kwi-StyleGAN.
Ukwamkela ubungakanani begalelo obahlukeneyo, isixokelelwano soguqulelo lwemifanekiso sisebenzisa uthungelwano oluguquguqukayo ngokupheleleyo. Uqeqesho olusuka ekuqaleni, kwelinye icala, lenza ukuba ugqithiso oluphezulu kunye nokulawulwa kwesimbo esilawulwayo kungenzeki.
Imodeli ye-StyleGAN eqeqeshwe kwangaphambili isetyenziswe kwisakhelo esisekelwe kwi-StyleGAN kwisisombululo esiphezulu kunye nokuhanjiswa kwesitayela esilawulwayo, nangona kunqunyelwe ubungakanani bemifanekiso eqingqiweyo kunye nelahleko yeenkcukacha.
I-StyleGAN ilungisiwe kwisakhelo esixutyiweyo ngokucima igalelo lalo elinobungakanani obusisigxina kunye neeleya ezinesombululo esisezantsi, okukhokelela kuyilo olupheleleyo lwe-encoder-generator uyilo olufana nelo lwesakhelo sokuguqulela umfanekiso.
Ukugcina iinkcukacha zesakhelo, qeqesha i-encoder ukukhupha iimpawu zomxholo we-multi-scale yesakhelo sokufaka njengemfuno eyongezelelweyo yomxholo kwi-generator. I-Vtoonify izuza i-StyleGAN imodeli yesimbo solawulo olubhetyebhetye ngokuyibeka kwijenereyitha ukuze ikhuphe zombini idatha kunye nemodeli.
Unyino lweSitayileGAN kunye neVtoonify ecetywayo
Imizobo yobugcisa ixhaphakile kubomi bethu bemihla ngemihla kunye nakumashishini oyilo afana nobugcisa, Imidiya yokuncokola ii-avatars, iimuvi, intengiso yokuzonwabisa, njalo njalo.
Ngophuhliso lwe ukufunda okunzulu itekhnoloji, ngoku kunokwenzeka ukwenza imizobo yobugcisa ekumgangatho ophezulu ukusuka kwiifoto zobuso bokwenyani usebenzisa ukudluliselwa kwesitayile somfanekiso ozenzekelayo.
Kukho iindlela ezahlukeneyo eziyimpumelelo ezenzelwe ukuhanjiswa kwesitayile esekwe kumfanekiso, uninzi lwazo lufikeleleka ngokulula kubasebenzisi abaqalayo ngohlobo lwezicelo zeselula. Imathiriyeli yevidiyo ngokukhawuleza ibe yeyona nto iphambili kwimithombo yeendaba zentlalo kule minyaka idlulileyo.
Ukunyuka kwemithombo yeendaba zentlalo kunye neefilimu ze-ephemeral zonyuse imfuno yokuhlelwa kwevidiyo okutsha, njengokudluliselwa kwesitayile sevidiyo yomfanekiso, ukuvelisa iividiyo eziyimpumelelo nezinomdla.
Ubuchwephesha obukhoyo obujolise kumfanekiso bunobubi obubalulekileyo xa busetyenziswa kwiimuvi, zinciphisa ukusetyenziswa kwazo kwisitayile sevidiyo esizenzekelayo.
I-StyleGAN ngumqolo oqhelekileyo wokuphuhlisa imodeli yokuhanjiswa kwesitayile somfanekiso ngenxa yomthamo wayo wokwenza ubuso obukumgangatho ophezulu kunye nolawulo lwesitayile esilungelelanisiweyo.
Inkqubo esekwe kwi-StyleGAN (ekwabizwa ngokuba yi-toonification yomfanekiso) ifaka ikhowudi yobuso bokwenyani kwisithuba esifihlakeleyo se-StyleGAN emva koko isebenzise ikhowudi yesimbo enesiphumo kwenye i-StyleGAN elungiswe kakuhle kwiseti yedatha yobugcisa ukwenza inguqulelo eyenziwe ngesitayile.
I-StyleGAN yenza imifanekiso enobuso obulungelelanisiweyo kunye nobukhulu obumiselweyo, obungakhethi buso obuguqukayo kwimifanekiso yehlabathi yokwenyani. Ukunqampuna kobuso kunye nokulungelelaniswa kwividiyo ngamanye amaxesha kubangela ubuso obuthile kunye nezenzo ezigwenxa. Abaphandi babiza lo mbandela we-StyleGAN 'isithintelo sesityalo esisisigxina.'
Kubuso obungahambelaniyo, kucetyiwe i-StyleGAN3; nangona kunjalo, ixhasa kuphela iseti yobungakanani bomfanekiso.
Ngaphaya koko, uphononongo lwakutsha nje lufumanise ukuba ukufaka iikhowudi ubuso obungalungelelanisiweyo kunzima kunobuso obulungelelanisiweyo. Ukufakwa kweekhowudi ebusweni okungalunganga kuyingozi kugqithiso lwesimbo somfanekiso, okukhokelela kwimiba efana notshintsho lwesazisi kunye nezinto ezingekhoyo kwizakhelo ezihlaziyiweyo nezinesitayile.
Njengoko kuxoxiwe, ubuchule obusebenzayo bokudlulisa isitayile sevidiyo yomfanekiso kufuneka ijongane nale miba ilandelayo:
- Ukugcina iintshukumo zokwenyani, le ndlela kufuneka ikwazi ukujongana nobuso obungalungelelanisiweyo kunye nobukhulu bevidiyo obahlukeneyo. Ubungakanani bevidiyo enkulu, okanye i-engile ebanzi yokujonga, inokubamba ulwazi oluthe kratya ngelixa igcina ubuso ukuba bungahambi kwisakhelo.
- Ukukhuphisana nezixhobo zanamhlanje zeHD ezisetyenziswa ngokuqhelekileyo, ividiyo enesisombululo esiphezulu iyafuneka.
- Ulawulo lwesimbo esiguquguqukayo kufuneka lunikezelwe kubasebenzisi ukuba batshintshe kwaye bakhethe ukhetho lwabo xa bephuhlisa inkqubo yokunxibelelana yabasebenzisi eyinyani.
Ngaloo njongo, abaphandi bacebisa i-VToonify, isakhelo se-hybrid entsha ye-toonification yevidiyo. Ukoyisa isithintelo sesityalo esisisigxina, abaphandi bafunda kuqala ukulingana kwenguqulelo kwi-StyleGAN.
I-VToonify idibanisa izibonelelo zoyilo olusekwe kwi-StyleGAN kunye nesakhelo sokuguqulela umfanekiso ukufezekisa ukuhanjiswa kwesitayile sevidiyo esinokulungiswa okuphezulu kwesisombululo esiphezulu.
Oku kulandelayo ngamagalelo amakhulu:
- Abaphandi baphanda isithintelo se-StyleGAN sesityalo esisisigxina kwaye bacebise isisombululo esisekwe kuguqulo olulinganayo.
- Abaphandi babonisa isakhelo esipheleleyo se-VToonify esilawulwayo esinesisombululo esiphezulu sesitayile sevidiyo esixhasa ubuso obungalungelelanisiweyo kunye nobukhulu obahlukeneyo bevidiyo.
- Abaphandi bakha i-VToonify kwi-backbones ye-Toonify kunye ne-DualStyleGAN kwaye bagxininise i-backbones ngokwemiqathango yomibini idatha kunye nemodeli ukwenzela ukuba ukuhanjiswa kwesitayela sevidiyo ye-portrait-based based and exemplar-based.
Ukuthelekisa iVtoonify kunye nezinye iimodeli zanamhlanje
Yenza kwakhona
Isebenza njengesiseko sokuhanjiswa kwesitayile esekwe kwingqokelela kubuso obulungelelanisiweyo kusetyenziswa i-StyleGAN. Ukufumana kwakhona iikhowudi zesitayile, abaphandi kufuneka balungelelanise ubuso kwaye batyale iifoto ezingama-256256 zePSP. I-Toonify isetyenziselwa ukuvelisa umphumo owenziwe ngesitayela ngeekhowudi ze-1024 * 1024.
Ekugqibeleni, baphinda balungelelanise isiphumo kwividiyo kwindawo yayo yokuqala. Indawo engenziwanga isimbo imiselwe kumnyama.
I-DualStyleGAN
Ngumqolo wogqithiselo olusekwe kwimodeli esekwe kwiSitayileGAN. Basebenzisa iindlela ezifanayo zedatha zangaphambili kunye nasemva kokulungiswa njengeToonify.
I-Pix2pixHD
Yimodeli yokuguqulela umfanekiso ukuya kumfanekiso oqhele ukusetyenziswa ukucudisa iimodeli eziqeqeshelwe ukuhlelwa kwesisombululo esiphezulu. Iqeqeshelwa ukusebenzisa idatha edityanisiweyo.
Abaphandi basebenzisa i-pix2pixHD njengomzekelo owongezelelweyo wamagalelo emephu kuba isebenzisa imephu yokwahlulahlula.
Isindululo soMyalelo wokuQala
I-FOM ngumzekelo oqhelekileyo wopopayi. Yaqeqeshwa kwimifanekiso ye-256256 kwaye iqhuba kakubi kunye nobunye ubungakanani bemifanekiso. Ngenxa yoko, abaphandi baqala ukukala iifreyimu zevidiyo ukuya kuma-256*256 ukuze iFOM ibe oopopayi kwaye emva koko balinganise iziphumo kubungakanani bazo bokuqala.
Uthelekiso olufanelekileyo, iFOM isebenzisa isakhelo sokuqala esinesitayile sendlela yayo njengomfanekiso wesitayile sereferensi.
DaGAN
Yimodeli yopopayi yobuso be-3D. Basebenzisa ukulungiswa kwedatha efanayo kunye neendlela zokulandela emva kweFOM.
eziluncedo
- Ingasetyenziswa kwezobugcisa, ii-avatar zemidiya yoluntu, iimuvi, intengiso yokuzonwabisa, njalo njalo.
- I-Vtoonify inokusetyenziswa kwakhona kwi-metaverse.
Imida
- Le ndlela yokusebenza ikhupha zombini idatha kunye nemodeli ukusuka kwi-backbones esekelwe kwi-StyleGAN, okukhokelela kwidatha kunye nemodeli yokukhetha.
- I-artifacts ibangelwa ubukhulu becala ngumahluko wobungakanani phakathi kommandla wobuso obunesitayile kunye namanye amacandelo.
- Esi sicwangciso asiphumelelanga kangako xa sijongana nezinto ezikummandla wobuso.
isiphelo
Okokugqibela, i-VToonify sisikhokelo se-toonification yevidiyo elawulwa yisitayile esinesisombululo esiphezulu.
Esi sikhokelo sifezekisa ukusebenza kakuhle ekuphatheni iividiyo kwaye senza ulawulo olubanzi kwisitayile sesakhiwo, isitayile sombala, kunye nenqanaba lesitayile ngokunciphisa imodeli ye-StyleGAN esekwe kwimifanekiso yethoni ngokwemiqathango yazo zombini. idatha yokwenziwa kunye nezakhiwo zothungelwano.
Shiya iMpendulo