Usenokuba uvile malunga nokuba zinamandla kangakanani iimodeli ze-AI ze-text-to-image kule minyaka imbalwa idlulileyo. Kodwa ngaba ubusazi ukuba itekhnoloji efanayo inokunceda ukwenza umtsi ukusuka kwi-2D ukuya kwi-3D?
Iimodeli ze-3D ezenziwe nge-AI zinemeko yosetyenziso olubanzi kwindawo yanamhlanje yedijithali. Iimidlalo yevidiyo kunye nefilimu ixhomekeke kumagcisa anezakhono ze-3D kunye nesoftware yokubumba efana neBlender ukwenza i-asethi ye-3D ukuze izalise imiboniso eyenziwe ngekhompyuter.
Nangona kunjalo, ngaba kunokwenzeka ukuba ishishini linokusebenzisa umatshini wokufunda ukwenza i-asethi ye-3D ngomzamo omncinci, ngokufanayo nendlela amagcisa e-2D namhlanje aqala ngayo ukwamkela iteknoloji efana ne-DALL-E kunye Phakathi kohambo?
Eli nqaku liza kuphonononga i-algorithm yenoveli ezama ukwenza imodeli esebenzayo yokubhaliweyo ukuya kwi-3D isebenzisa ekhoyo iimodeli zokusasaza.
Yintoni i Ukuphupha?
Umba omnye ophambili ngokudala imodeli yosasazo eyenza i-asethi ye-3D ngokuthe ngqo kukuba akukho datha eninzi ye-3D ekhoyo. Iimodeli zokusasazwa kwe-2D ziye zaba namandla ngenxa yobuninzi bedatha yemifanekiso efunyenwe kwi-intanethi. Okufanayo akunakuthethwa nge-asethi ye-3D.
Ezinye iindlela zokuvelisa ze-3D zisebenza malunga noku kunqongophala kwedatha ngokusebenzisa le ntaphane yedatha ye-2D.
I-DreamFusion ngumzekelo ovelisayo onokudala imifuziselo ye-3D esekelwe kwinkcazo yombhalo onikiweyo. Imodeli ye-DreamFusion isebenzisa imodeli yokusasazwa kwesicatshulwa-kumfanekiso oqeqeshwe kwangaphambili ukuvelisa iimodeli ezinokwenyani ze-dimensional ezintathu ukusuka kwi-text prompions.
Nangona kungekho datha yoqeqesho lwe-3D, le ndlela ivelise i-asethi ye-3D ehambelanayo kunye nembonakalo ephezulu yokuthembeka kunye nobunzulu.
Ingaba isebenza kanjani?
I-algorithm ye-DreamFusion iqukethe iimodeli ezimbini eziphambili: imodeli yokusabalalisa i-2D kunye ne-a inethiwekhi yomnatha enokuguqula imifanekiso ye-2D ibe ngumboniso odityanisiweyo we-3D.
Imodeli kaGoogle yoMfanekiso wokuguqulela kuMfanekiso
Inxalenye yokuqala ye-algorithm yimodeli yokusabalalisa. Lo mzekelo unoxanduva lokuguqula umbhalo ube yimifanekiso.
Fungisa yimodeli yosasazo enokuvelisa isampulu enkulu yokwahluka komfanekiso wento ethile. Kule meko, ukuhluka kwemifanekiso yethu kufuneka kugubungele zonke ii-angles ezinokwenzeka zento enikeziweyo. Umzekelo, ukuba sifuna ukuvelisa imodeli ye-3D yehashe, besiya kufuna imifanekiso ye-2D yehashe kuzo zonke ii-engile ezinokubakho. Injongo kukusebenzisa i-Imagen ukubonelela ngolwazi oluninzi kangangoko kunokwenzeka (imibala, imiboniso, ukuxinana) kwimodeli elandelayo kwi-algorithm yethu.
Ukudala iiModeli ze3D ngeNeRF
Okulandelayo, i-Dreamfusion isebenzisa imodeli eyaziwa ngokuba yi-a I-Neural Radiance Field okanye i-NeRF ukwenza ngokwenene imodeli ye-3D ukusuka kwiseti yomfanekiso owenziweyo. Ii-NeRFs ziyakwazi ukwenza imiboniso entsonkothileyo ye-3D enikwe iseti yedatha yemifanekiso ye-2D.
Makhe sizame ukuqonda ukuba isebenza njani iNeRF.
Imodeli ijolise ekudaleni umsebenzi oqhubekayo we-volumetric scene elungiselelwe ukusuka kwidatha enikeziweyo yemifanekiso ye-2D.
Ukuba imodeli yenza umsebenzi, yintoni igalelo kunye nemveliso?
Umsebenzi wescene uthatha indawo ye-3D kunye nesalathiso sokujonga i-2D njengegalelo. Umsebenzi ke ukhupha umbala (ngendlela ye RGB) kunye noxinaniso lwevolyum ethile.
Ukuvelisa umfanekiso we-2D ngokwembono ethile, imodeli iyakuvelisa iseti yamanqaku e-3D kwaye iqhube ezo ngongoma ngomsebenzi wescene ukubuyisela iseti yombala kunye namaxabiso oxinaniso lwevolyum. Iindlela zokubonisa umthamo ziya kuthi emva koko ziguqule ezo xabiso zibe yimveliso yomfanekiso we-2D.
Ukusebenzisa i-NeRF kunye ne-2D Diffusion Models kunye
Ngoku siyazi ukuba isebenza njani i-NeRF, makhe sibone ukuba le modeli inokuvelisa njani iimodeli ezichanekileyo ze-3D kwimifanekiso yethu eyenziweyo.
Kumyalezo ngamnye onikiweyo, i-DreamFusion iqeqesha i-NeRF engacwangciswanga ukusuka ekuqaleni. Ukuphinda-phindwa ngakunye kukhetha indawo yekhamera engacwangciswanga kwiseti yolungelelwaniso olungqukuva. Cinga ngemodeli evalelwe kwiglasi yeglasi. Ngalo lonke ixesha sisenza umfanekiso omtsha wemodeli yethu ye-3D, siya kukhetha indawo engacwangciswanga kwisigaba sethu njengendawo yokujonga imveliso yethu. I-DreamFusion iya kuphinda ikhethe indawo yokukhanya engaqhelekanga l ukusetyenziselwa unikezelo.
Nje ukuba sinekhamera kunye nendawo yokukhanya, imodeli yeNeRF iya kunikezelwa. I-DreamFusion nayo iya kukhetha ngokungakhethiyo phakathi konikezelo olunemibala, unikezelo olungaxutywanga, kunye nonikezelo lwe-albedo ngaphandle komthunzi.
Siye sakhankanya ngaphambili ukuba sifuna imodeli yethu yokubhaliweyo ukuya kumfanekiso (Umfanekiso) ukuvelisa imifanekiso eyaneleyo ukwenza isampuli emele.
Iyenza njani le nto iDreamfusion?
I-Dreamfusion iguqula ngokulula igalelo loncedo kancinane ukufezekisa ii-engile ezicetywayo. Umzekelo, sinokufikelela kwii-engile zokuphakama okuphezulu ngokufaka "imbono engaphezulu" kwi-prompt yethu. Sinokuvelisa ezinye ii-engile ngokufakela amabinzana afana “nombono wangaphambili”, “umbono wecala”, kunye “nombono wangemva”.
Imiboniso inikezelwa ngokuphindaphindiweyo ukusuka kwindawo engacwangciswanga yekhamera. Olu nikezelo emva koko ludlule kumanqaku okulahlekelwa umsebenzi. Indlela elula yokwehla ngethambeka iya kuyiphucula kancinane Imodeli ye3D ide ingqinelane nemeko echazwe sisicatshulwa.
Nje ukuba sinikeze imodeli ye-3D sisebenzisa i-NeRF, sinokusebenzisa i Ukuhamba ngeeCubes algorithm ukuvelisa i-3D mesh yemodeli yethu. Lo mesh unokungeniswa kumazwe angaphandle kwi-3D abanikezeli abaziwayo okanye isoftware yomfuziselo.
Imida
Ngelixa imveliso ye-DreamFusion inomtsalane ngokwaneleyo kuba isebenzisa iimodeli ezikhoyo zokusasazwa kwesicatshulwa-kumfanekiso ngendlela yenoveli, abaphandi baye baqaphela imida embalwa.
Umsebenzi welahleko we-SDS uye wabonwa ukuba uvelise iziphumo ezigqithisileyo kunye ne-over-smoothed. Ungaqwalasela oku kumbala ongeyondalo kunye nokunqongophala kweenkcukacha ezichanekileyo ezifunyenwe kwiziphumo.
I-algorithm ye-DreamFusion nayo ilinganiselwe ngesisombululo semveliso yemodeli ye-Imagen, eyi-64 x 64 pixels. Oku kukhokelela kwiimodeli ezihlanganisiweyo ezingenazo iinkcukacha eziphucukileyo.
Okokugqibela, abaphandi baye baqaphela ukuba kukho umceli mngeni okhoyo ekwenziweni kweemodeli ze-3D ukusuka kwidatha ye-2D. Zininzi ezinokwenzeka iimodeli ze-3D esinokuzenza kwiseti yemifanekiso ye-2D, eyenza ulungiselelo lubenzima kakhulu kwaye ludideke.
isiphelo
Unikezelo lwe-3D lwe-DreamFusion lusebenza kakuhle kakhulu ngenxa yokukwazi kweemodeli zokusasazwa kwesicatshulwa ukuya kumfanekiso ukwenza nayiphi na into okanye indawo. Iyamangalisa indlela inethiwekhi ye-neural inokuqonda ngayo indawo kwindawo ye-3D ngaphandle kwedatha yoqeqesho lwe-3D. Ndincoma ukufunda i iphepha elipheleleyo ukufunda ngakumbi malunga neenkcukacha zobugcisa be-algorithm ye-DreamFusion.
Ngethemba, le teknoloji iya kuphucuka ukuze ekugqibeleni yenze iimodeli ezinokwenyani ze-3D. Yiba nomfanekiso ngqondweni wemidlalo yevidiyo epheleleyo okanye ukulinganisa okusebenzisa iimeko-bume ezenziwe nge-AI. Inokunciphisa umqobo wokungena kubaphuhlisi bomdlalo wevidiyo ukwenza ihlabathi le-3D elintywilayo!
Yeyiphi indima ocinga ukuba i-text-to-3D iya kudlala kwixa elizayo?
Shiya iMpendulo