Zvikamu zvinoverengeka zvepasirese zviri kutanga kuisa mari zvakanyanya mukudzidza kwemichina (ML).
Mhando dzeML dzinogona kutangwa uye kushandiswa nezvikwata zvenyanzvi, asi chimwe chezvipingamupinyi zvikuru kuendesa ruzivo rwakawanikwa kune inotevera modhi kuitira kuti maitiro awedzere.
Kuvandudza uye kumisikidza maitiro anosanganisirwa mumodhiyo yehupenyu manejimendi, matekiniki eMLOps ari kuramba achishandiswa nezvikwata zvinogadzira mhando dzekudzidza dzemuchina.
Ramba uchiverenga kuti uwane zvimwe nezve mamwe akanakisa MLOps maturusi uye mapuratifomu aripo nhasi uye maitiro avanogona kuita kuti kudzidza kwemuchina kuve nyore kubva pachishandiso, mugadziri, uye maitiro ekuona.
Chii chinonzi MLOps?
Maitiro ekugadzira marongero, zvimiro, uye akanakisa maitiro emamodhi ekudzidza muchina anozivikanwa se "muchina kudzidza mashandiro," kana "MLOps."
MLOps ine chinangwa chekuvimbisa hupenyu hwese hwekuvandudza kweML - kubva pakubata pamuviri kusvika pakutumirwa - inonyorwa zvine hungwaru uye inotungamirwa kuti iwane mhedzisiro yakanaka pane kuisa nguva yakawanda nezviwanikwa mairi pasina zano.
Chinangwa cheMLOps ndechekubatanidza maitiro akanakisa nenzira inoita kuti kuvandudzwa kwekudzidza kwemuchina kuwedzere kuvashandisi veML nevagadziri, pamwe nekuwedzera kunaka uye kuchengetedzeka kweML modhi.
Vamwe vanotaura nezveMLOps se "DevOps yekudzidza muchina" sezvo ichibudirira kushandisa DevOps misimboti kune imwe nzvimbo yakasarudzika yekuvandudza tekinoroji.
Iyi inzira inobatsira yekufunga nezveMLOps nekuti, seDevOps, inosimbisa kugovana ruzivo, kubatana, uye maitiro akanaka pakati pezvikwata nemidziyo.
MLOps inopa vanogadzira, data sainzi, uye zvikwata zvekushanda zvine hurongwa hwekushandira pamwe uye, semhedzisiro, kugadzira ine simba kwazvo ML modhi.
Sei Uchishandisa MLOps Zvishandiso?
Maturusi eMLOps anogona kuita mabasa akasiyana siyana echikwata cheML, zvisinei, anowanzo patsanurwa kuita mapoka maviri: kutungamira kwepuratifomu uye kutonga kwechikamu chega.
Nepo zvimwe zvigadzirwa zveMLOps zvinongotarisa pane imwechete yakakosha basa, senge data kana metadata manejimendi, mamwe maturusi anotora yakawedzera-inosanganisa zvese uye inopa MLOps chikuva chekudzora akati wandei eML lifecycle.
Tsvaga mhinduro dzeMLOps dzinobatsira timu yako kutonga idzi nzvimbo dzekuvandudza dzeML, kungave uri kutsvaga nyanzvi kana chishandiso chakawedzera:
- Kubata data
- Kugadzira uye kuenzanisa
- Management yemapurojekiti nenzvimbo yebasa
- ML modhi yekuendesa uye kuenderera mberi nekuchengetedza
- Lifecycle manejimendi kubva pakutanga kusvika kumagumo, ayo anowanzo kupihwa neakazara-sevhisi MLOps mapuratifomu.
MLOps Zvishandiso
1. MLFlow
Muchina wekudzidza lifecycle inodzorwa neyakavhurika-sosi chikuva MLflow uye inosanganisira yepakati modhi kunyoresa, kuendesa, uye kuyedza.
MLflow inogona kushandiswa nechero saizi timu, yega uye pamwe chete. Maraibhurari haana chekuita nechokushandisa.
Chero mutauro wepurogiramu uye raibhurari yekudzidza yemuchina inogona kuishandisa.
Kuita kuti zvive nyore kudzidzisa, kuendesa, uye kubata muchina kudzidza maapplication, MLFlow inopindirana nehuwandu hwemichina yekudzidza masisitimu, kusanganisira. TensorFlow uye Pytorch.
Pamusoro pezvo, MLflow inopa ari nyore kushandisa maAPI anogona kuverengerwa mune chero aripo muchina ekudzidza zvirongwa kana maraibhurari.
MLflow ine zvinhu zvina zvakakosha zvinofambisira kuronda nekuronga zviedzo:
- MLflow Tracking - API uye UI yekutema muchina wekudzidza kodhi paramita, shanduro, metrics, uye zvigadzirwa pamwe nekuzotevera kuratidza nekusiyanisa zvabuda.
- MLflow Projects - yekurongedza muchina yekudzidza kodhi mune ino shandiswazve, inodhinda fomati yekuchinjisa kukugadzira kana kugovana nemamwe masayendisiti data.
- MLflow Models - kuchengetedza uye kutumira mamodheru kune akasiyana emhando yekushumira uye inference masisitimu kubva kumaraibhurari akasiyana eML.
- MLflow Model Registry - chitoro chepakati chemodhi chinogonesa kutonga kwemubatanidzwa kwehupenyu hwese hweMLflow, kusanganisira kushandura modhi, shanduko yenhanho, uye zvirevo.
2. KubeFlow
Iyo ML toolbox yeKubernetes inonzi Kubeflow. Kurongedza uye kubata Docker midziyo, zvinobatsira mukuchengetedza kwe michina yekudzidza masisitimu.
Nekurerutsa kumhanya orchestration uye deployment yemashini yekudzidza workflows, inosimudzira scalability yemamodhi ekudzidza muchina.
Icho chirongwa chakavhurika-sosi chinosanganisira boka rakanyatsosarudzwa rematurusi ekuwedzera uye masisitimu akarongedzerwa kune akasiyana ML zvinodiwa.
Yakareba ML yekudzidzisa mabasa, kuyedza manyore, kudzokorora, uye DevOps matambudziko anogona kubatwa neKubeflow Pipelines.
Kwematanho akati wandei ekudzidza muchina, kusanganisira kudzidziswa, kugadzirwa kwepombi, uye kugadzirisa kwe Jupyter zvinyorwa, Kubeflow inopa hunyanzvi masevhisi uye kubatanidzwa.
Zvinoita kuti zvive nyore kubata uye kuteedzera hupenyu hweAI yako mitoro yebasa pamwe nekuisa muchina kudzidza (ML) modhi uye mapaipi edata kumasumbu eKubernetes.
Inopa:
- Zvinyorwa zvekushandisa iyo SDK kudyidzana neiyo system
- mushandisi interface (UI) yekudzora uye yekutarisa kumhanya, mabasa, uye kuyedza
- Kugadzira nekukurumidza kugadzirisa-kusvika-kumagumo mhinduro pasina kuvaka patsva nguva yega yega, uye kushandisazve zvikamu nemapaipi.
- Sechinhu chakakosha cheKubeflow kana sekumisikidzwa kwakamira, Kubeflow Pipelines inopihwa.
3. Data Version Control
Yakavhurika-sosi vhezheni yekudzora mhinduro yemapurojekiti ekudzidza muchina inonzi DVC, kana Dhata Version Control.
Chero mutauro waunosarudza, chishandiso chekuyedza chinobatsira pakutsanangura pombi.
DVC inoshandisa kodhi, vhezheni yedata, uye kugadzirwazve kuti ikubatsire kuchengetedza nguva kana wawana dambudziko neshanduro yekutanga yeML yako modhi.
Pamusoro pezvo, unogona kushandisa mapaipi eDVC kudzidzisa modhi yako uye kuigovera kunhengo dzechikwata chako. Kurongeka kukuru kwedata uye shanduro inogona kubatwa neDVC, uye iyo data inogona kuchengetwa nenzira iri nyore kuwanikwa.
Kunyangwe ichisanganisira zvimwe (zvishoma) zviyedzo zvekuteedzera maficha, zvinonyanya kutarisa pane data uye pombi shanduro uye manejimendi.
Inopa:
- Iyo yekuchengetedza agnostic, saka zvinokwanisika kushandisa akasiyana emhando dzekuchengetedza.
- Inopa tracking stats zvakare.
- nzira isati yavakwa yekubatanidza ML matanho muDAG uye kumhanyisa pombi yese kubva pakutanga kusvika pakupedzisira.
- Imwe neimwe ML modhi yekusimudzira inogona kuteverwa uchishandisa kodhi yayo yese uye data kubva.
- Reproducibility nekuchengetedza nekutendeka gadziriro yekutanga, data rekuisa, uye kodhi yepurogiramu yekuyedza.
4. Pachyderm
Pachyderm ishanduro-yekudzora chirongwa chekudzidza muchina uye sainzi yedata, yakafanana neDVC.
Uyezve, nekuti yakagadzirwa uchishandisa Docker uye Kubernetes, inogona kuita uye kutumira Machine Kudzidza maapplication pane chero makore papuratifomu.
Pachyderm inoita vimbiso yekuti chidimbu chega chega che data chinopedzwa kuita muchina wekudzidza modhi chinogona kuteedzerwa kumashure nekushandurwa.
Inoshandiswa kugadzira, kugovera, kubata, uye kuchengeta ziso pamamodhi ekudzidza muchina. Modhi registry, modhi manejimendi system, uye CLI bhokisi rekushandisa zvese zvinosanganisirwa.
Vagadziri vanogona kuita otomatiki uye kuwedzera yavo muchina kudzidza lifecycle vachishandisa Pachyderm's data hwaro, iyo zvakare inovimbisa kudzokorora.
Iyo inotsigira yakaomesesa yekutonga data zviyero, inodzikisa kugadziridza data uye mutengo wekuchengetedza, uye inobatsira mabhizinesi mukuunza yavo data sainzi zvirongwa kuti vatengese nekukurumidza.
5. Polyaxon
Uchishandisa iyo Polyaxon chikuva, mapurojekiti ekudzidza muchina uye zvakadzama kudzidza maapplication anogona kudzokororwa uye kudzorwa pamusoro pehupenyu hwavo hwese.
Polyaxon inokwanisa kubata uye kutonga chishandiso, uye inogona kuiswa mune chero data data kana gore mupi. Zvakadai seTorch, Tensorflow, uye MXNet, inotsigira ese anonyanya kufarirwa zvakadzika masisitimu ekudzidza.
Kana zvasvika pakurongeka, Polyaxon inoita kuti iwe ugone kushandisa zvakanyanya sumbu rako nekuronga mabasa uye bvunzo kuburikidza neCLI yavo, dashboard, SDKs, kana REST API.
Inopa:
- Unogona kushandisa yakavhurika-sosi vhezheni izvozvi, asi inosanganisirawo sarudzo dzekambani.
- Kunyangwe ichivhara hupenyu hwakazara, kusanganisira kumhanya orchestration, inokwanisa zvimwe zvakawanda.
- Iine magwaro ereferenzi ehunyanzvi, ekutanga nhungamiro, zvekushandisa pakudzidza, zvinyorwa, tutorials, changelogs, uye nezvimwe, ipuratifomu yakanyatso nyorwa.
- Nedeshibhodhi yezviyedzo, zvinogoneka kuramba wakatarisa, kuronda, uye kuongorora kuyedza kwega kwega.
6. kometi
Comet ipuratifomu yekudzidza meta muchina inoteedzera, inosiyanisa, inotsanangura, uye inovandudza kuyedza uye modhi.
Zvese zvekuedza kwako zvinogona kuonwa nekuenzaniswa munzvimbo imwechete.
Inoshanda kune chero basa rekudzidza muchina, chero kupi kodhi yako inoitwa, uye nechero raibhurari yekudzidza muchina.
Comet inokodzera mapoka, vanhu, masangano ezvidzidzo, mabhizinesi, uye chero ani zvake anoda kukurumidza kuona zviedzo, kufambisa basa, uye kuita zviedzo.
Data masayendisiti nezvikwata zvinogona kuronda, kujekesa, kuvandudza, uye kuenzanisa zviedzo nemamodheru vachishandisa inozvigashira uye yegore-yakavakirwa meta-muchina yekudzidza chikuva Comet.
Inopa:
- Kugona kwakawanda kuripo kuti nhengo dzechikwata dzigovane mabasa.
- Iyo ine akati wandei kusanganisa kunoita kuti zvive nyore kuibatanidza kune mamwe matekinoroji
- Inoshanda zvakanaka nemaraibhurari eML aripo
- Inochengetedza mushandisi manejimendi
- Kuenzanisa kwekuedza kunogoneswa, kusanganisira kuenzanisa kwekodhi, hyperparameters, metrics, kufanotaura, kutsamira, uye masisitimu metrics.
- Inopa akasiyana mamodule ekuona, odhiyo, zvinyorwa, uye tabular data iyo inokutendera iwe kuona masampuli.
7. Opt
Optuna isystem yekuzvimiririra hyperparameter optimization inogona kuiswa kune ese ari maviri muchina kudzidza uye kudzidza kwakadzama pamwe nemamwe minda.
Iyo ine akasiyana-siyana ekucheka-kumucheto algorithms kubva kwaunogona kusarudza (kana kubatanidza), inoita kuti zvive nyore kwazvo kugovera kudzidziswa pamakomputa akawanda, uye inopa inoyevedza mhedzisiro yekuona.
Maraibhurari akakurumbira ekudzidza muchina sePyTorch, TensorFlow, Keras, FastAI, sci-kit-dzidza, LightGBM, uye XGBoost zvese zvakabatanidzwa nazvo.
Inopa ekucheka-kumucheto algorithms ayo anogonesa vatengi kuwana mhedzisiro nekukurumidza nekudzikisa nekukurumidza masampuli asingataridzike achivimbisa.
Ichishandisa Python-based algorithms, inotsvaga otomatiki iyo yakakodzera hyperparameter. Optuna inokurudzira parallelized hyperparameter kutsvaga pane dzakawanda shinda pasina kuchinja iyo yekutanga kodhi.
Inopa:
- Inotsigira kudzidziswa kwakagoverwa pasumbu pamwe chete nekombuta imwe (yakawanda-maitiro) (multi-node)
- Iyo inotsigira akati wandei ekucheka matekiniki ekumhanyisa convergence (uye shandisa shoma compute)
- Iyo ine akasiyana-siyana ekuona ane simba, senge chimedu chidimbu, contour plot, uye parallel coordination.
8. Kedro
Kedro ndeye yemahara Python chimiro chekunyora kodhi iyo inogona kuvandudzwa uye kuchengetedzwa kune data sainzi mapurojekiti.
Iyo inounza mazano kubva kune akanakisa maitiro muinjiniya yesoftware kune kodhi yekudzidza yemuchina. Python ndiyo hwaro hweiyi workflow orchestration tool.
Kuita kuti yako ML maitiro ave nyore uye anyatsojeka, unogona kuvandudza, chengetedzo, uye modular workflows.
Kedro inosanganisa misimboti yeinjiniya software senge modularity, kupatsanurwa kwemabasa, uye kushandura munzvimbo yekudzidza yemuchina.
Pahwaro hweCookiecutter Data Sayenzi, inopa yakajairika, inochinjika purojekiti chimiro.
Huwandu hweakareruka ekubatanidza data anoshandiswa kuchengetedza uye kurodha data kune akati wandei faira masisitimu uye faira mafomati, anotungamirwa nedata catalog. Inoita kuti mapurojekiti ekudzidza emuchina ashande uye anoita kuti zvive nyore kugadzira pombi yedata.
Inopa:
- Kedro inobvumira kuparadzirwa kwega kana kushandiswa kwega muchina.
- Iwe unogona otomatiki kutsamira pakati pePython kodhi uye kuyerera kwekuona kwekushanda uchishandisa pombi abstraction.
- Kuburikidza nekushandiswa kwemodular, reusable kodhi, tekinoroji iyi inofambisa kubatana kwechikwata pamatanho akasiyana uye inovandudza chibereko munzvimbo yekodha.
- Chinangwa chikuru ndechekukunda zvipingamupinyi zveJupyter notebook, one-off scripts, uye glue-code nekunyora inochengeteka data science programming.
9. BentoML
Kuvaka muchina kudzidza API endpoints inogadzirwa nyore neBentoML.
Inopa yakajairwa asi yakapfupikiswa masisitimu ekufambisa akadzidza emuchina ekudzidza modhi mukugadzira.
Inokugonesa kurongedza mamodheru akadzidzwa kuti ashandiswe mugadziriro yekugadzira, uchiadudzira uchishandisa chero ML chimiro. Ose ari maviri ekunze batch anoshumira uye online API anoshandira anotsigirwa.
Iyo yepamusoro-inoshanda modhi sevha uye inoshanduka yekufamba-famba zvinhu zveBentoML.
Uyezve, sevha inopa adaptive micro-batching. Nzira yakabatana yekuronga mamodheru uye kucherekedza nzira dzekuendesa kunopihwa neUI dashboard.
Pachave pasina server downtime nekuti iyo inoshanda sisitimu ndeye modular uye iyo gadziriso inodzokororwa. Iyo ipuratifomu inochinjika yekupa, kuronga, uye kutumira ML modhi.
Inopa:
- Iyo ine modular dhizaini inochinjika.
- Inogonesa kutumirwa kune akati wandei mapuratifomu.
- Haikwanise kubata yega yakachinjika kuyera.
- Inogonesa imwe modhi fomati, modhi manejimendi, modhi kurongedza, uye yepamusoro-inoshanda modhi inoshumira.
10. seldon
Masayendisiti edata anogona kugadzira, kuendesa, uye kutonga modhi yekudzidza muchina uye kuyedza pachiyero paKubernetes vachishandisa yakavhurika-sosi Seldon Core chimiro.
TensorFlow, sci-kit-dzidza, Spark, R, Java, uye H2O angori mashoma ematurusi ezvishandiso anotsigirwa nawo.
Iyo zvakare inopindirana neKubeflow uye RedHat's OpenShift. Iyo Seldon musimboti inoshandura michina yekudzidza modhi (ML modhi) kana mitauro yekuputira (mitauro yakaita sePython, Java, nezvimwewo) mukugadzira REST/GRPC microservices.
Imwe yeakanakisa MLOps maturusi ekuvandudza maitiro ekudzidza muchina ndeiyi.
Zviri nyore kuisa ML modhi uye bvunzo yekushandisa uye kuchengetedzeka uchishandisa Seldon Core.
Inopa:
- Model deployment inogona kuitwa nyore neimwe nzira dzinoverengeka, senge canary deployment.
- Kuti unzwisise kuti sei mafambiro chaiwo akaitwa, shandisa mienzaniso inotsanangura.
- Kana nyaya dzamuka, ramba wakatarira mhando dzekugadzira uchishandisa yambiro system.
mhedziso
MLOps inogona kubatsira kuita kuti kudzidzira kwemuchina kuve nani. MLOps inogona kukurumidzira kutumira, kuita kuti kuunganidzwa kwedata uye kugadzirisa zvireruke, uye kugadzirisa kudyidzana pakati peinjiniya nemasainzi edata.
Kuti iwe usarudze chishandiso cheMLOps chinonyatso kuenderana nezvido zvako, iyi positi yakaongorora gumi akakurumbira MLOps mhinduro, mazhinji acho akavhurika-sosi.
Leave a Reply