Table of Contents[Hide][Show]
Koj feem ntau yuav paub tias lub computer tuaj yeem piav qhia ib daim duab.
Piv txwv li, ib daim duab ntawm tus dev ua si nrog koj cov menyuam tuaj yeem txhais tau tias 'dab thiab menyuam yaus hauv lub vaj.' Tab sis koj puas paub tias txoj kev sib txawv nyob ib puag ncig tam sim no ua tau thiab? Koj ntaus qee cov lus, thiab lub tshuab tsim cov duab tshiab.
Tsis zoo li Google tshawb nrhiav, uas tshawb nrhiav cov duab uas twb muaj lawm, qhov no yog txhua yam tshiab. Nyob rau hauv xyoo tas los no, OpenAI tau yog ib lub koom haum ua lag luam, tshaj tawm cov txiaj ntsig zoo.
Lawv cob qhia lawv cov algorithms ntawm cov ntawv loj thiab cov duab databases. Lawv tau luam tawm ib daim ntawv ntawm lawv cov qauv duab GLIDE, uas tau kawm txog ntau pua lab tus duab. Nyob rau hauv cov nqe lus ntawm photorealism, nws outperforms lawv ua ntej 'DALL-E' qauv.
Hauv cov ntawv tshaj tawm no, peb yuav saib ntawm OpenAI's GLIDE, yog ib qho ntawm ntau qhov kev nyiam ua haujlwm uas tsom mus tsim thiab hloov cov duab tiag tiag nrog cov ntawv qhia kev nthuav dav. Cia peb pib.
Yuav ua li cas yog Qhib AI Glide?
Txawm hais tias feem ntau cov duab tuaj yeem piav qhia hauv cov lus, tsim cov duab los ntawm cov ntawv nkag yuav tsum muaj kev paub tshwj xeeb thiab lub sijhawm tseem ceeb.
Tso cai rau tus neeg sawv cev AI los tsim cov duab tiag tiag los ntawm cov lus hais lus ntuj tsis tsuas yog tso cai rau tib neeg los tsim cov khoom siv pom kev nplua nuj thiab ntau haiv neeg nrog qhov yooj yim uas tsis tau pom dua, tab sis kuj tso cai rau kev kho kom yooj yim dua thiab kev tswj xyuas cov duab zoo.
GLIDE tuaj yeem siv los hloov kho cov duab uas twb muaj lawm los ntawm kev siv cov ntawv sau lus ntuj kom ntxig cov khoom tshiab, tsim duab ntxoov ntxoo thiab kev xav, ua haujlwm. duab inpainting, thiab hais txog.
Nws kuj tseem tuaj yeem tig cov kab kos duab kos duab rau hauv cov duab tiag tiag, thiab nws muaj qhov tshwj xeeb xoom-cov qauv tsim thiab kho lub peev xwm rau cov xwm txheej nyuaj.
Kev tshawb fawb tsis ntev los no tau ua pov thawj tias qhov ua tau raws li cov qauv diffusion tuaj yeem tsim cov duab hluavtaws zoo, tshwj xeeb tshaj yog thaum ua ke nrog kev taw qhia uas sib npaug ntau yam thiab kev ncaj ncees.
OpenAI luam tawm ib coj diffusion qauv nyob rau hauv lub Tsib Hlis, uas tso cai rau cov qauv diffusion yuav tsum nyob rau hauv daim ntawv lo ntawm ib tug classifier. GLIDE txhim kho ntawm qhov kev vam meej no los ntawm kev coj kev nthuav dav mus rau qhov teeb meem ntawm cov ntawv sau cov duab tsim.
Tom qab kev cob qhia 3.5 billion parameter GLIDE diffusion qauv siv cov ntawv nyeem encoder kom ua raws li cov lus piav qhia, cov kws tshawb fawb tau sim ob txoj kev qhia lwm txoj hauv kev: CLIP kev taw qhia thiab kev qhia tsis pub cais tawm.
CLIP yog ib qho txheej txheem uas tuaj yeem ua tau rau kev kawm sib koom ua ke sawv cev ntawm cov ntawv nyeem thiab cov duab uas muab cov qhab nia raws li qhov ze ntawm daim duab rau ib daim ntawv.
Pab neeg no siv lub tswv yim no hauv lawv cov qauv diffusion los ntawm kev hloov cov khoom faib nrog CLIP qauv uas "qhia" cov qauv. Meanwhile, classifier-free kev taw qhia yog ib lub tswv yim rau directing diffusion qauv uas tsis muaj kev cob qhia ntawm ib tug cais cais.
GLIDE Architecture
Lub GLIDE architecture muaj peb yam: Ablated Diffusion Model (ADM) tau cob qhia los tsim cov duab 64 × 64, cov qauv ntawv (transformer) uas cuam tshuam cov duab tsim los ntawm cov ntawv nyeem tam sim, thiab cov qauv tsim tawm uas hloov peb lub 64 × 64 me me. cov duab kom txhais tau ntau dua 256 x 256 pixels.
Thawj ob lub ntsiab lus ua haujlwm ua ke los tswj cov txheej txheem tsim cov duab kom nws tsim nyog muaj kev cuam tshuam cov ntawv nyeem sai, thaum lub tom kawg yuav tsum ua kom cov duab peb tsim kom nkag siab yooj yim dua. Qhov project GLIDE tau tshwm sim los ntawm ib tshaj tawm hauv 2021 uas qhia tau hais tias ADM cov tswv yim ua tau zoo tshaj tam sim no nrov, lub xeev-of-the-art generative qauv nyob rau hauv cov nqe lus ntawm daim duab qauv zoo.
Rau ADM, cov kws sau ntawv GLIDE tau ua haujlwm tib yam ImageNet 64 x 64 qauv raws li Dhariwal thiab Nichol, tab sis nrog 512 channels es tsis txhob 64. Tus qauv ImageNet muaj kwv yees li 2.3 billion tsis raws li qhov tshwm sim ntawm qhov no.
Pab pawg GLIDE, tsis zoo li Dhariwal thiab Nichol, xav kom muaj kev tswj hwm ncaj qha ntau dua ntawm cov txheej txheem tsim cov duab, yog li lawv tau ua ke cov qauv pom nrog lub ntsej muag hloov pauv. GLIDE muab qee qhov kev tswj hwm ntawm daim duab tsim cov txheej txheem tso tawm los ntawm kev ua cov ntawv sau cov lus qhia.
Qhov no ua tiav los ntawm kev cob qhia tus qauv hloov pauv ntawm cov ntaub ntawv loj loj ntawm cov duab thiab cov ntawv sau (zoo ib yam li qhov ua haujlwm hauv DALL-E project).
Cov ntawv nyeem tau pib nkag mus rau hauv K tokens txhawm rau txhawm rau kho nws. Tom qab ntawd, lub tokens yog loaded rau hauv ib tug transformer qauv. Cov zis ntawm lub transformer tuaj yeem siv tau ob txoj hauv kev. Rau ADM qauv, qhov kawg token embedding yog siv es tsis txhob ntawm chav kawm embedding.
Qhov thib ob, lub token embeddings 'txheej txheem kawg - ib tug series ntawm feature vectors - yog projected ntawm nws tus kheej mus rau qhov ntev ntawm txhua txheej xim nyob rau hauv tus qauv ADM thiab concatenated rau txhua lub ntsiab lus mloog.
Hauv kev muaj tiag, qhov no ua rau ADM tus qauv tsim cov duab los ntawm kev sib txuas tshiab ntawm cov ntawv nyeem zoo sib xws hauv qhov tshwj xeeb thiab cov duab tiag tiag, raws li nws qhov kev nkag siab ntawm cov lus nkag thiab lawv cov duab cuam tshuam. Qhov no text-encoding transformer muaj 1.2 billion tsis thiab ua hauj lwm 24 seem blocks nrog ib tug dav ntawm 2048.
Thaum kawg, tus qauv upsampler diffusion muaj nyob ib ncig ntawm 1.5 billion tsis thiab nws txawv ntawm cov qauv yooj yim uas nws cov ntawv encoder me dua, nrog rau qhov dav ntawm 1024 thiab 384 lub hauv paus raws, piv rau cov qauv hauv paus. Cov qauv no, raws li lub npe qhia, pab txhim kho cov qauv txhawm rau txhim kho kev txhais lus rau ob lub tshuab thiab tib neeg.
Diffusion qauv
GLIDE tsim cov duab siv nws tus kheej version ntawm ADM (ADM-G rau "guided"). Tus qauv ADM-G yog ib qho kev hloov kho ntawm diffusion U-net qauv. Ib tug diffusion U-net qauv txawv heev los ntawm ntau cov duab synthesis cov tswv yim xws li VAE, GAN, thiab transformers.
Lawv tsim Markov saw ntawm diffusion cov kauj ruam kom maj mam txhaj cov suab nrov rau hauv cov ntaub ntawv, thiab tom qab ntawd kawm kom thim rov qab cov txheej txheem diffusion thiab rov tsim cov ntaub ntawv xav tau los ntawm lub suab nrov ib leeg. Nws ua haujlwm nyob rau hauv ob theem: pem hauv ntej thiab rov qab diffusion.
Txoj kev xa mus rau pem hauv ntej, muab cov ntaub ntawv taw qhia los ntawm cov qauv kev faib tawm tiag tiag, ntxiv cov suab nrov me me rau cov qauv dhau ntawm cov kauj ruam ua ntej. Raws li cov kauj ruam nce nyob rau hauv qhov loj thiab mus kom ze infinity, tus qauv poob tag nrho cov yam ntxwv pom tau thiab qhov sib lawv liag pib zoo li ib qho isotropic Gaussian nkhaus.
Thaum lub sij hawm rov qab diffusion theem, tus qauv diffusion kawm kom thim rov qab qhov cuam tshuam ntawm lub suab nrov ntxiv ntawm cov duab thiab coj cov duab tsim rov qab mus rau nws cov duab qub los ntawm kev sim ua kom zoo ib yam li thawj cov qauv kev faib tawm.
Tus qauv ua tiav tuaj yeem ua tau nrog lub suab nrov Gaussian tiag tiag thiab ua kom sai. Txoj kev ADM-G txawv ntawm ib qho ua ntej hauv tus qauv, CLIP lossis hloov pauv hloov pauv, cuam tshuam rau theem rov qab los ntawm kev siv cov ntawv nyeem tam sim tokens uas tau nkag.
Glide muaj peev xwm
1. Tsim duab
Qhov nrov tshaj plaws thiab siv dav siv ntawm GLIDE tej zaum yuav yog cov duab synthesis. Txawm hais tias cov duab yog me me thiab GLIDE muaj teeb meem nrog tsiaj / tib neeg cov ntaub ntawv, lub peev xwm rau ib-steev duab ntau lawm yuav luag tag nrho.
Nws tuaj yeem tsim cov duab ntawm cov tsiaj, cov neeg nto moo, toj roob hauv pes, vaj tse, thiab ntau ntxiv, thiab nws tuaj yeem ua tau ntau yam kev kos duab zoo li kev yees duab tiag tiag. Cov kws sau ntawv ntawm cov kws tshawb fawb tau lees tias GLIDE muaj peev xwm los txhais lus thiab hloov kho ntau yam ntawm cov ntaub ntawv nkag mus rau hauv hom kev pom, raws li pom hauv cov qauv hauv qab no.
2. Glide inpainting
GLIDE's tsis siv neeg yees duab inpainting yog arguably siv fascinating tshaj plaws. GLIDE tuaj yeem nqa ib daim duab uas twb muaj lawm los ua cov tswv yim, ua tiav nrog cov ntawv qhia hauv siab rau cov chaw uas yuav tsum tau hloov kho, thiab tom qab ntawd ua kom muaj kev hloov pauv rau cov khoom ntawd yooj yim.
Nws yuav tsum tau siv ua ke nrog tus qauv kho, xws li SDEdit, los tsim cov txiaj ntsig zoo dua. Nyob rau hauv lub neej yav tom ntej, apps uas coj kom zoo dua ntawm lub peev xwm zoo li no tej zaum yuav yog ib qho tseem ceeb nyob rau hauv kev tsim code-dawb duab-hloov mus kom ze.
xaus
Tam sim no uas peb tau dhau los ntawm cov txheej txheem, koj yuav tsum nkag siab qhov tseem ceeb ntawm GLIDE ua haujlwm li cas, nrog rau qhov dav ntawm nws cov peev xwm hauv kev tsim duab thiab hloov kho duab.
Sau ntawv cia Ncua