Iwe unogona kunge wakanzwa nezve simba sei mameseji-kune-mufananidzo AI modhi ave mumakore mashoma apfuura. Asi waizviziva here kuti iyo tekinoroji imwe chete inogona kubatsira kusvetuka kubva ku2D kuenda ku3D?
AI-yakagadzirwa 3D modhi ine yakakura yekushandisa kesi mune yanhasi dhijitari mamiriro. Video mitambo uye firimu rinotsamira kune vane hunyanzvi 3D maartist uye modelling software seBlender kugadzira 3D assets kuti iburitse mapikicha anogadzirwa nekombuta.
Nekudaro, zvinoita here kuti iyo indasitiri inogona kushandisa muchina kudzidza kugadzira 3D assets pasina kushanda nesimba, zvakafanana nemaitiro e2D artists nhasi ari kutanga kutora tekinoroji yakaita seDALL-E uye. midjourney?
Ichi chinyorwa chichaongorora chinyorwa chegorgorithm inoedza kugadzira iyo inoshanda mavara-ku-ku-3D modhi uchishandisa iripo kupararira mienzaniso.
Chii Dreamfusion?
Imwe nyaya hombe nekugadzira modhi yekuparadzira iyo inogadzira 3D assets zvakananga ndeyekuti hapana chakawanda che3D data iripo. 2D diffusion modhi dzave nesimba nekuda kweiyo yakakura dataset yemifananidzo inowanikwa painternet. Zvakafanana hazvigone kutaurwa ne 3D assets.
Mamwe maitiro ekugadzira e3D anoshanda achitenderedza uku kushomeka kwedata nekutora mukana weiyo yakawanda ye2D data.
DreamFusion imhando yekugadzira iyo inogona kugadzira 3D modhi zvichibva pane yakapihwa rondedzero yemavara. Iyo DreamFusion modhi inoshandisa pre-yakadzidziswa mameseji-kune-mufananidzo diffusion modhi kugadzira echokwadi matatu-dimensional modhi kubva kune zvinyorwa zvinokurudzira.
Kunyangwe isina 3D yekudzidziswa data, nzira iyi yakagadzira inowirirana 3D assets ine yakakwirira-kutendeseka chitarisiko uye kudzika.
Chinoshanda sei?
Iyo DreamFusion algorithm ine maviri makuru modhi: a 2D diffusion modhi uye a neural network iyo inogona kushandura mifananidzo ye2D kuita yakabatana 3D chiitiko.
Google's Imagen Text-to-Image Model
Chikamu chekutanga chegorgorithm ndiyo diffusion modhi. Iyi modhi ine basa rekushandura mavara kuita mifananidzo.
Imagen imhando yekuparadzira iyo inogona kuburitsa mufananidzo wakakura wekusiyana kwemufananidzo wechimwe chinhu. Muchiitiko ichi, misiyano yedu yemifananidzo inofanira kuvhara makona ese anobvira echinhu chakapihwa. Semuyenzaniso, kana taida kugadzira 3D modhi yebhiza, taizoda mifananidzo yebhiza ye2D kubva kumakona ese. Chinangwa ndechekushandisa Imagen kupa ruzivo rwakawanda sezvinobvira (mavara, maratidziro, density) yemuenzaniso unotevera mune yedu algorithm.
Kugadzira 3D Models neNeRF
Tevere, Dreamfusion inoshandisa modhi inozivikanwa sea Neural Radiance Field kana NeRF kuti inyatso gadzira iyo 3D modhi kubva kune yakagadzirwa mufananidzo seti. NeRFs inokwanisa kugadzira yakaoma 3D zviratidziro zvakapihwa dhata remifananidzo ye2D.
Ngatiedzei kunzwisisa kuti NeRF inoshanda sei.
Iyo modhi ine chinangwa chekugadzira inoenderera volumetric scene basa rakagadziridzwa kubva kune yakapihwa dataset yemifananidzo ye2D.
Kana iyo modhi ikagadzira basa, chii chinopinza uye chinobuda?
Basa rechiitiko rinotora munzvimbo ye 3D uye 2D yekuona nzira sekuisa. Basa racho rinobva raburitsa ruvara (muchimiro cheRGB) uye yakananga vhoriyamu density.
Kugadzira mufananidzo we2D kubva pane imwe maonero, modhi yacho inoburitsa seti ye3D mapoinzi uye womhanyisa iwo mapoinzi kuburikidza nechiitiko chiitiko kudzorera seti yeruvara uye vhoriyamu density values. Maitiro ekupa mavhoriyamu anozoshandura iwo makoshero kuita 2D mufananidzo unobuda.
Kushandisa NeRF uye 2D Diffusion Models Pamwe chete
Zvino zvatave kuziva mashandiro anoita NeRF, ngationei kuti modhi iyi inogona kuburitsa sei emhando dze 3D kubva pamifananidzo yedu yakagadzirwa.
Kune yega yega yakapihwa mameseji ekukurumidza, DreamFusion inodzidzisa isina kurongeka NeRF kubva kutanga. Imwe neimwe iteration inosarudza isina kujairika kamera chinzvimbo mune seti ye spherical coordinates. Funga nezvemuenzaniso wakakomberedzwa mugirazi sphere. Pese patinogadzira mufananidzo mutsva we 3D modhi yedu, isu tichasarudza isina kurongeka poindi mundima yedu senzvimbo yekuona yekubuda kwedu. DreamFusion ichasarudzawo nzvimbo isina kurongeka yechiedza l kushandisa pakupa.
Kana tave nekamera uye chinzvimbo chechiedza, modhi yeNeRF ichapihwa. DreamFusion ichaitawo zvisizvo sarudza pakati peanopa ane mavara, asina mavara ekupa, uye kupa kwealbedo pasina shading.
Isu takambotaura kuti tinoda kuti yedu yemavara-kune-mufananidzo modhi (Imagen) ibudise mifananidzo yakakwana kugadzira mumiriri wemuenzaniso.
Dreamfusion inoita sei izvi?
Dreamfusion inongo shandura iyo yekupinza yekuchimbidza zvishoma kuti iwane ma angles anodiwa. Semuyenzaniso, tinogona kuwana maekiseni akakwirira nekuisa "pamusoro pekutarisa" kune yedu kukurumidza. Tinogona kugadzira mamwe makona nekuwedzera mitsara yakadai se "kumberi kwekuona", "kudivi rekuona", uye "kumashure maonero".
Zviratidziro zvinodzokororwa zvichiitwa kubva kune zvisina kujairika kamera nzvimbo. Aya madudziro anobva apfuura nepakati pechibodzwa distillation kurasikirwa basa. Iyo yakapusa gradient descent nzira inovandudza zvishoma nezvishoma 3D modhi kusvikira yaenderana nechiitiko chinotsanangurwa nerugwaro.
Kana tangopa iyo 3D modhi tichishandisa NeRF, tinogona kushandisa iyo Kufamba Cubes algorithm kuburitsa 3D mesh yemodhi yedu. Iyi mesh inogona kubva kunze kwenyika kune yakakurumbira 3D renderers kana modelling software.
Nokuremara
Nepo kuburitswa kweDreamFusion kuchikatyamadza zvakakwana sezvo ichishandisa mameseji-kune-mufananidzo ekudhirowa mamodheru nenzira itsva, vaongorori vakaona zvishoma zvipimo.
Iyo SDS yekurasikirwa basa yakacherechedzwa kuburitsa yakawandisa uye yakanyanyisa-yakapfava mhedzisiro. Iwe unogona kuona izvi mune zvisiri zvechisikigo coloring uye kushomeka kweiyo chaiyo ruzivo inowanikwa mune zvakabuda.
Iyo DreamFusion algorithm inoganhurirwawo nekugadziriswa kweiyo Imagen modhi yakabuda, inova 64 x 64 pixels. Izvi zvinotungamira kune akagadzirwa modhi asina ruzivo rwakanaka.
Chekupedzisira, vaongorori vakaona kuti pane dambudziko riripo mukugadzira mhando dze3D kubva ku2D data. Kune akawanda anogoneka emhando dze3D dzatinogona kugadzira kubva kune seti yemifananidzo ye2D, izvo zvinoita kuti optimization iome uye kunyange kusanzwisisika.
mhedziso
DreamFusion's 3D renders inoshanda zvakanyanya nekuda kwekugona kwemavara-kune-image diffusion modhi kugadzira chero chinhu kana chiitiko. Zvinoshamisa kuti neural network inganzwisisa sei chiitiko mu 3D nzvimbo isina chero 3D yekudzidzisa data. Ndinokurudzira kuverenga bepa rose kuti udzidze zvakawanda nezve tekinoroji yeiyo DreamFusion algorithm.
Ndinovimba, iyi tekinoroji ichavandudza kuti pakupedzisira igadzire mafoto-echokwadi 3D modhi. Fungidzira mitambo yese yemavhidhiyo kana simulations inoshandisa AI-yakagadzirwa nharaunda. Inogona kuderedza chipingamupinyi chekupinda chevagadziri vemutambo wevhidhiyo kuti vagadzire inonyudza 3D nyika!
Ibasa ripi raunofunga kuti mavara-ku-3D mamodheru achaita mune ramangwana?
Leave a Reply