Kungenzeka ukuthi uzwile ukuthi anamandla kangakanani amamodeli we-AI wokuguqula umbhalo uye esithombeni eminyakeni embalwa edlule. Kodwa bewazi yini ukuthi ubuchwepheshe obufanayo bungasiza ukwenza ukugxuma kusuka ku-2D kuye ku-3D?
Amamodeli e-3D akhiqizwe nge-AI anokusetshenziswa okubanzi endaweni yanamuhla yedijithali. Amageyimu evidiyo futhi ifilimu ithembele kumaciko anekhono e-3D nesofthiwe yokumodela njenge-Blender ukuze adale amafa e-3D ukuze agcwalise izigcawu ezikhiqizwe ngekhompuyutha.
Kodwa-ke, kungenzeka yini ukuthi imboni ingasebenzisa ukufunda ngomshini ukuze idale amafa e-3D ngomzamo omncane, okufana nendlela abaculi be-2D namuhla abaqala ngayo ukusebenzisa ubuchwepheshe obufana ne-DALL-E kanye Uhambo lwaphakathi?
Lesi sihloko sizohlola i-algorithm yenoveli ezama ukudala imodeli ephumelelayo yombhalo uye ku-3D isebenzisa ekhona amamodeli wokusabalalisa.
Kuyini I-Dreamfusion?
Inkinga eyodwa enkulu ngokudala imodeli yokusabalalisa ekhiqiza amafa e-3D ngokuqondile ukuthi ayikho idatha eningi ye-3D etholakalayo. Amamodeli wokusabalalisa we-2D abe namandla kakhulu ngenxa yobuningi bedathasethi yezithombe ezitholakala ku-inthanethi. Okufanayo akunakushiwo ngezimpahla ze-3D.
Amanye amasu okukhiqiza e-3D asebenza kulokhu kuntuleka kwedatha ngokusebenzisa le nsada yedatha ye-2D.
I-DreamFusion iyimodeli ekhiqizayo engakha amamodeli e-3D ngokusekelwe encazelweni yombhalo enikeziwe. Imodeli ye-DreamFusion isebenzisa imodeli yokusabalalisa umbhalo-kuya-isithombe eqeqeshwe ngaphambilini ukuze ikhiqize amamodeli angokoqobo ezinhlangothi ezintathu kusukela kukwaziswa kombhalo.
Naphezu kokungenayo idatha yokuqeqeshwa kwe-3D, le ndlela ikhiqize izimpahla ze-3D ezihambisanayo ezinokubukeka kokwethembeka okuphezulu nokujula.
Isebenza kanjani?
I-algorithm ye-DreamFusion iqukethe amamodeli amabili amakhulu: imodeli yokusabalalisa ye-2D kanye ne- inethiwekhi ye-neural engaguqula izithombe ze-2D zibe isigcawu esihlangene se-3D.
Imodeli ye-Google yokuguqula umbhalo ube yisithombe se-Imagen
Ingxenye yokuqala ye-algorithm imodeli yokusabalalisa. Le modeli inesibopho sokuguqula umbhalo ube yizithombe.
Imagen imodeli yokusabalalisa engakhiqiza isampula enkulu yokuhluka kwesithombe sento ethile. Kulokhu, ukuhluka kwezithombe zethu kufanele kufake wonke ama-engeli okungenzeka wento enikeziwe. Isibonelo, uma sifuna ukukhiqiza imodeli ye-3D yehhashi, sizofuna izithombe ze-2D zehhashi kuwo wonke ama-engeli angaba khona. Umgomo uwukusebenzisa i-Imagen ukuze unikeze ulwazi oluningi ngangokunokwenzeka (imibala, ukuboniswa, ukuminyana) kumodeli elandelayo ku-algorithm yethu.
Ukudala amamodeli we-3D nge-NeRF
Okulandelayo, i-Dreamfusion isebenzisa imodeli eyaziwa ngokuthi a Inkundla ye-Neural Radiance noma i-NeRF ukuze idale imodeli ye-3D kusukela kusethi yesithombe esikhiqiziwe. Ama-NeRF ayakwazi ukudala izigcawu eziyinkimbinkimbi ze-3D ezinikezwe idathasethi yezithombe ze-2D.
Ake sizame ukuqonda ukuthi i-NeRF isebenza kanjani.
Imodeli ihlose ukudala umsebenzi oqhubekayo wesigcawu se-volumetric olungiselelwe kusukela kudathasethi enikeziwe yezithombe ze-2D.
Uma imodeli idala umsebenzi, kuyini okokufaka nokukhiphayo?
Umsebenzi wesigcawu uthatha indawo ye-3D kanye nesiqondiso sokubuka se-2D njengokufaka. Umsebenzi ube usukhipha umbala (ngendlela ye-RGB) kanye nokuminyana kwevolumu ethile.
Ukuze ukhiqize isithombe se-2D ngombono othile, imodeli izokhiqiza isethi yamaphoyinti e-3D futhi isebenzise lawo maphuzu ngomsebenzi wesigcawu ukuze ibuyisele isethi yombala namanani wokuminyana kwevolumu. Amasu okunikeza ivolomu azobe eseguqula lawo manani abe okokukhiphayo kwesithombe se-2D.
Ukusebenzisa I-NeRF kanye ne-2D Diffusion Models Ndawonye
Manje njengoba sesiyazi ukuthi i-NeRF isebenza kanjani, ake sibone ukuthi le modeli ingawakhiqiza kanjani amamodeli e-3D anembile ezithombeni zethu ezikhiqiziwe.
Emyalweni ngamunye wombhalo onikeziwe, i-DreamFusion iqeqesha i-NeRF eqalwe ngokungahleliwe ukusuka ekuqaleni. Ukuphindaphinda ngakunye kukhetha indawo yekhamera engahleliwe kusethi yezixhumanisi eziyindilinga. Cabanga ngemodeli ebiyelwe endaweni yengilazi. Ngaso sonke isikhathi uma senza isithombe esisha semodeli yethu ye-3D, sizokhetha iphoyinti elingahleliwe ku-sphere yethu njengendawo ephambili yokukhiphayo. I-DreamFusion izophinde ikhethe indawo yokukhanya engahleliwe l ukusetshenziselwa ukuhlinzeka.
Uma sesinekhamera nendawo yokukhanya, kuzonikezwa imodeli ye-NeRF. I-DreamFusion izophinde ikhethe ngokungahleliwe phakathi kokunikeza okunemibala, ukunikezwa okungathungwanga, nokunikezwa kwe-albedo ngaphandle kokufiphazwa.
Sike sasho ngaphambilini ukuthi sifuna imodeli yethu yokuguqula umbhalo uye-esithombeni (Isithombe) ukuthi sikhiqize izithombe ezanele ukuze sakhe isampula elimele.
I-Dreamfusion ikufeza kanjani lokhu?
I-Dreamfusion imane iguqule ukwaziswa kokufaka kancane ukuze kuzuzwe ama-engeli ahlosiwe. Isibonelo, singafinyelela ama-engeli aphezulu ngokufaka "ukubuka okungaphezulu" ekwazisweni kwethu. Singakwazi ukukhiqiza amanye ama-engeli ngokuhlanganisa imishwana efana nokuthi “ukubuka ngaphambili”, “ukubuka okuseceleni”, kanye “nokubuka emuva”.
Izigcawu zinikezwa ngokuphindaphindiwe ukusuka ezindaweni ezingahleliwe zekhamera. Lokhu kuhumusha bese kudlula kumsebenzi wokulahlekelwa kwe-distillation wamaphuzu. Indlela elula yokwehla kwe-gradient izothuthukisa kancane kancane Imodeli ye-3D kuze kufane nesigcawu esichazwe umbhalo.
Uma sesinikeze imodeli ye-3D sisebenzisa i-NeRF, singasebenzisa i I-algorithm ye-Marching Cubes ukukhipha i-3D mesh yemodeli yethu. Le mesh ingase ingeniswe kuzinikezeli ezidumile ze-3D noma isofthiwe yokumodela.
Ukulinganiselwa
Ngenkathi okuphumayo kwe-DreamFusion kuhlaba umxhwele ngokwanele njengoba kusebenzisa amamodeli wokusabalalisa umbhalo kuya esithombeni ngendlela yenoveli, abacwaningi baphawule imikhawulo embalwa.
Umsebenzi wokulahlekelwa kwe-SDS uye waqashelwa ukukhiqiza imiphumela egcwele ngokweqile kanye neshelelezi ngokweqile. Ungakubona lokhu ebaleni elingelona ngokwemvelo kanye nokuntuleka kwemininingwane enembayo etholakala kokuphumayo.
I-algorithm ye-DreamFusion nayo inqunyelwe ukulungiswa kokuphuma kwemodeli ye-Imagen, okungamaphikseli angu-64 x 64. Lokhu kuholela kumamodeli ahlanganisiwe angenayo imininingwane ecolekile.
Okokugcina, abacwaningi baqaphele ukuthi kunenselelo engokwemvelo ekuhlanganiseni amamodeli e-3D kusuka kudatha ye-2D. Kunamamodeli amaningi e-3D esingawenza kusukela kusethi yezithombe ze-2D, okwenza ukwenza kube nzima kakhulu futhi kungaqondakali.
Isiphetho
Ukuhumusha kwe-3D kwe-DreamFusion kusebenza kahle kakhulu ngenxa yekhono lamamodeli okusabalalisa umbhalo-kuya-isithombe ukudala noma iyiphi into noma isigcawu. Kuyamangaza ukuthi inethiwekhi ye-neural ingasiqonda kanjani isigcawu esikhaleni se-3D ngaphandle kwanoma iyiphi idatha yokuqeqeshwa kwe-3D. Ngincoma ukufunda i- iphepha lonke ukuze ufunde kabanzi mayelana nemininingwane yezobuchwepheshe ye-algorithm ye-DreamFusion.
Ngethemba, lobu buchwepheshe buzothuthuka ukuze ekugcineni kwakhe amamodeli e-3D angokoqobo ngesithombe. Cabanga ngayo yonke imidlalo yevidiyo noma izifaniso ezisebenzisa izindawo ezikhiqizwe yi-AI. Kungehlisa umgoqo wokungena kubathuthukisi begeyimu yevidiyo ukuze bakhe imihlaba ye-3D egxilile!
Iyiphi indima ocabanga ukuthi amamodeli we-text-to-3D azoyidlala ngokuzayo?
shiya impendulo