Imikhakha eminingana yomhlaba isiqala ukutshala imali eningi kakhulu ekufundeni komshini (ML).
Amamodeli e-ML angaqala aqaliswe futhi asetshenziswe amaqembu ochwepheshe, kodwa enye yezithiyo ezinkulu ukudlulisela ulwazi oluzuziwe kumodeli elandelayo ukuze izinqubo zandiswe.
Ukuze uthuthukise futhi ulinganise izinqubo ezihilelekile ekulawuleni imodeli yomjikelezo wempilo, amasu e-MLOps aya ngokuya esetshenziswa amaqembu adala amamodeli okufunda omshini.
Qhubeka nokufunda ukuze uthole okwengeziwe mayelana namanye amathuluzi nezinkundla ezingcono kakhulu ze-MLOps ezitholakalayo namuhla nokuthi zingakwenza kanjani ukufunda ngomshini kube lula kusukela kuthuluzi, unjiniyela, nokubuka kwenqubo.
Yini ama-MLOps?
Indlela yokudala izinqubomgomo, izinkambiso, nemikhuba ehamba phambili yamamodeli okufunda omshini yaziwa ngokuthi “imisebenzi yokufunda ngomshini,” noma “ama-MLOps.”
Ama-MLOps ahlose ukuqinisekisa wonke umjikelezo wokuphila wokuthuthukiswa kwe-ML - kusukela ekukhulelweni kuya ekusetshenzisweni - ibhalwe ngokucophelela futhi ilawulwa ukuze kube nemiphumela engcono kakhulu kunokutshala isikhathi esiningi nezisetshenziswa kuyo ngaphandle kwesu.
Umgomo we-MLOps uwukuhlanganisa izinqubo ezihamba phambili ngendlela eyenza ukuthuthukiswa kokufunda komshini kukhuphuke kakhulu kuma-opharetha we-ML nonjiniyela, kanye nokuthuthukisa ikhwalithi nokuvikeleka kwamamodeli e-ML.
Abanye babiza ama-MLOps ngokuthi “i-DevOps yokufunda ngomshini” njengoba isebenzisa ngempumelelo izimiso ze-DevOps emkhakheni okhethekile wokuthuthukiswa kobuchwepheshe.
Lena indlela ewusizo yokucabanga nge-MLOps ngoba, njenge-DevOps, igcizelela ukwabelana ngolwazi, ukubambisana, nemikhuba ehamba phambili phakathi kwamaqembu namathuluzi.
I-MLOps ihlinzeka onjiniyela, ososayensi bedatha, namaqembu okusebenza ngohlaka lokubambisana futhi, njengomphumela, ukukhiqiza amamodeli e-ML anamandla kakhulu.
Kungani Usebenzisa Amathuluzi we-MLOps?
Amathuluzi e-MLOps angenza imisebenzi eminingi yeqembu le-ML, nokho, ngokuvamile ahlukaniswa abe amaqembu amabili: ukuphathwa kwenkundla nokuphathwa kwengxenye ngayinye.
Nakuba eminye imikhiqizo ye-MLOps igxila kuphela kumsebenzi owodwa oyinhloko, njengokuphatha idatha noma imethadatha, amanye amathuluzi asebenzisa isu elihlanganisa konke futhi anikeze inkundla ye-MLOps yokulawula izici ezimbalwa zomjikelezo wempilo we-ML.
Bheka izixazululo ze-MLOps ezisiza ithimba lakho ekuphatheni lezi zindawo zokuthuthukisa i-ML, noma ngabe ufuna uchwepheshe noma ithuluzi elibanzi kakhulu:
- Ukuphathwa kwedatha
- Ukuklama nokumodela
- Ukuphathwa kwamaphrojekthi kanye nendawo yokusebenza
- Ukuthunyelwa kwemodeli ye-ML nokugcinwa okuqhubekayo
- Ukuphathwa kwe-Lifecycle kusuka ekuqaleni kuya ekugcineni, okuvamise ukunikezwa izinkundla ze-MLOps ezisebenza ngokugcwele.
Amathuluzi we-MLOps
1. MLFlow
Umjikelezo wokuphila wokufunda komshini ulawulwa yinkundla yomthombo ovulekile i-MLflow futhi ihlanganisa ukubhaliswa kwemodeli emaphakathi, ukusetshenziswa, nokuhlola.
I-MLflow ingasetshenziswa yinoma yiliphi iqembu losayizi, kokubili ngabanye nangokuhlangene. Imitapo yolwazi ayinalutho kuleli thuluzi.
Noma yiluphi ulimi lokuhlela nomtapo wolwazi wokufunda womshini ungawusebenzisa.
Ukuze kwenziwe kube lula ukuqeqesha, ukuphakela, nokuphatha izinhlelo zokufunda zomshini, i-MLFlow isebenzisana nenani lezinhlaka zokufunda zomshini, okuhlanganisa I-TensorFlow kanye nePytorch.
Ukwengeza, i-MLflow inikeza ama-API asebenziseka kalula angafakwa kunoma yiziphi izinhlelo zokufunda zomshini ezikhona noma amalabhulali.
I-MLflow inezici ezine ezibalulekile ezenza kube lula ukuhlola nokuhlela:
- I-MLflow Tracking – i-API ne-UI yamapharamitha ekhodi yokufunda yomshini wokugawula, izinguqulo, amamethrikhi, nama-artifacts kanye nokubonisa nokuqhathanisa imiphumela kamuva.
- I-MLflow Projects – ikhodi yokufunda yomshini wokupakisha ngefomethi esebenzisekayo, ekhiqizekayo ukuze idluliselwe ekukhiqizweni noma ukwabelana nabanye ososayensi bedatha
- Amamodeli we-MLflow - ukugcinwa kanye nokuthumela amamodeli kuhlu lwamamodeli wokuphakela kanye nezinhlelo zokukhomba ezivela emitatsheni yolwazi ye-ML ehlukahlukene
- I-MLflow Model Registry – imodeli yesitolo esimaphakathi evumela ukuphathwa ngokubambisana kwayo yonke impilo yemodeli ye-MLflow, okuhlanganisa ukuguqulwa kwemodeli, ukushintshwa kwesiteji, nezichasiselo.
2. IKubeflow
Ibhokisi lamathuluzi le-ML le-Kubernetes libizwa ngokuthi i-Kubeflow. Ukupakisha nokuphatha iziqukathi ze-Docker, izinsiza ekunakekeleni izinhlelo zokufunda zomshini.
Ngokwenza lula i-orchestration yokuqalisa nokuphakwa kokugeleza komsebenzi wokufunda komshini, kuthuthukisa ukusitika kwamamodeli okufunda omshini.
Kuyiphrojekthi yomthombo ovulekile ehlanganisa iqembu elikhethwe ngokucophelela lamathuluzi ahambisanayo nezinhlaka ezenzelwe izidingo ezahlukene ze-ML.
Imisebenzi yokuqeqesha ye-ML ende, ukuhlola okwenziwa ngesandla, ukuphindaphinda, nezinselele ze-DevOps kungasingathwa nge-Kubeflow Pipelines.
Ngezigaba ezimbalwa zokufunda komshini, okuhlanganisa ukuqeqeshwa, ukuthuthukiswa kwamapayipi, nokugcinwa kwe Jupyter notebook, I-Kubeflow inikeza izinsizakalo ezikhethekile nokuhlanganiswa.
Kwenza kube lula ukuphatha nokulandelela impilo yonke yemisebenzi yakho ye-AI kanye nokuthumela amamodeli okufunda ngomshini (ML) namapayipi edatha kumaqoqo e-Kubernetes.
Inikeza:
- Amanothibhuku okusebenzisa i-SDK ukuze uhlanganyele nesistimu
- isixhumi esibonakalayo somsebenzisi (i-UI) sokulawula nokuqapha ukusebenza, imisebenzi, nokuhlola
- Ukuklama ngokushesha izixazululo ezisuka ekupheleni ngaphandle kokwakha kabusha isikhathi ngasinye, futhi usebenzise kabusha izingxenye namapayipi.
- Njengengxenye eyinhloko ye-Kubeflow noma njengokufakwa okuzimele, i-Kubeflow Pipelines inikezwa.
3. Ukulawulwa Kwenguqulo Yedatha
Isixazululo sokulawula inguqulo yomthombo ovulekile wamaphrojekthi wokufunda womshini sibizwa nge-DVC, noma Ukulawulwa Kwenguqulo Yedatha.
Noma ngabe yiluphi ulimi olukhethayo, ithuluzi lokuhlola elisiza ekuchazeni ipayipi.
I-DVC isebenzisa ikhodi, inguqulo yedatha, nokwenza kabusha ukuze ikusize wonge isikhathi lapho uthola inkinga ngenguqulo yangaphambili yemodeli yakho ye-ML.
Ukwengeza, ungasebenzisa amapayipi e-DVC ukuze uqeqeshe imodeli yakho futhi usabalalise kumalungu eqembu lakho. Ukuhlelwa kwedatha enkulu nokuguqulwa kungaphathwa yi-DVC, futhi idatha ingagcinwa ngendlela efinyeleleka kalula.
Nakuba ihlanganisa izici ezithile (ezilinganiselwe) zokulandelela ukuhlolwa, igxile kakhulu kudatha nenguqulo yamapayipi nokuphatha.
Inikeza:
- Kuyinto yokugcina i-agnostic, ngakho-ke kungenzeka ukusebenzisa izinhlobo ezihlukahlukene zokugcina.
- Ihlinzeka ngezibalo zokulandelela futhi.
- indlela eyakhelwe ngaphambili yokuhlanganisa izigaba ze-ML ku-DAG futhi isebenzise lonke ipayipi kusukela ekuqaleni kuze kube sekugcineni
- Ukuthuthukiswa kwemodeli ngayinye ye-ML kungalandelwa kusetshenziswa ikhodi yayo yonke kanye nokuvela kwedatha.
- Ukukhiqiza kabusha ngokulondoloza ngokwethembeka ukulungiselelwa kokuqala, idatha yokokufaka, kanye nekhodi yohlelo yokuhlolwa.
4. I-Pachyderm
I-Pachyderm iwuhlelo lokulawula inguqulo lokufunda ngomshini nesayensi yedatha, efana ne-DVC.
Ukwengeza, ngoba yakhiwe kusetshenziswa I-Docker ne-Kubernetes, ingakwazi ukusebenzisa futhi isebenzise izinhlelo zokusebenza zokufunda ngomshini kunoma iyiphi iplatifomu yamafu.
I-Pachyderm yenza iziqinisekiso zokuthi ucezu ngalunye lwedatha olusetshenziswa kumodeli yokufunda yomshini lungalandelelwa futhi lwenziwe inguqulo.
Isetshenziselwa ukudala, ukusabalalisa, ukuphatha, kanye nokubheka amamodeli okufunda omshini. Ukubhaliswa okuyimodeli, isistimu yokuphatha imodeli, nebhokisi lamathuluzi le-CLI konke kufakiwe.
Onjiniyela bangakwazi ukuzenzela futhi banwebe umjikelezo wokuphila wokufunda komshini besebenzisa isisekelo sedatha ye-Pachyderm, esiqinisekisa ukuphindaphinda.
Isekela izindinganiso eziqinile zokuphatha idatha, yehlisa izindleko zokucubungula nokugcinwa kwedatha, futhi isiza amabhizinisi ekuletheni imizamo yawo yesayensi yedatha ukuze imakethwe ngokushesha okukhulu.
5. I-Polyaxon
Ngokusebenzisa inkundla ye-Polyaxon, amaphrojekthi okufunda ngomshini kanye nezinhlelo zokusebenza zokufunda ezijulile zingaphindwa futhi zilawulwe kuwo wonke umjikelezo wazo wempilo.
I-Polyaxon iyakwazi ukusingatha nokuphatha ithuluzi, futhi ingafakwa kunoma yisiphi isikhungo sedatha noma umhlinzeki wamafu. Okufana ne-Torch, Tensorflow, ne-MXNet, esekela zonke izinhlaka zokufunda ezijulile ezaziwa kakhulu.
Uma kukhulunywa nge-orchestration, i-Polyaxon ikuvumela ukuthi wenze ngokugcwele iqoqo lakho ngokuhlela imisebenzi nezivivinyo nge-CLI, ideshibhodi, ama-SDK, noma i-REST API.
Inikeza:
- Ungasebenzisa inguqulo yomthombo ovulekile njengamanje, kodwa futhi ihlanganisa izinketho zenkampani.
- Yize ihlanganisa umjikelezo wempilo ophelele, okuhlanganisa nokushaya i-orchestration, ikwazi ukwenza okwengeziwe.
- Ngemibhalo yezethenjwa yobuchwepheshe, imihlahlandlela yokuqalisa, izinto zokufunda, imanyuwali, okokufundisa, ama-changelog, nokunye, kuyinkundla ebhalwe kahle kakhulu.
- Ngedeshibhodi yemininingwane yokuhlolwa, kuyenzeka ukuthi uhlale ubhekile, ulandelele, futhi uhlole isilingo ngasinye sokuthuthukisa.
6. Comet
I-Comet iyinkundla yokufunda komshini we-meta elandelela, eqhathanisa, echazayo, futhi ethuthukisa izivivinyo namamodeli.
Konke ukuhlola kwakho kungabonwa futhi kuqhathaniswe endaweni eyodwa.
Isebenza kunoma yimuphi umsebenzi wokufunda womshini, noma kuphi lapho kwenziwa khona ikhodi yakho, nanoma yimuphi umtapo wolwazi wokufunda womshini.
Inkanyezi enomsila ifanele amaqembu, abantu ngabanye, izikhungo zezemfundo, amabhizinisi, nanoma ubani omunye ofisa ukubona ngokushesha ukuhlola, ukwenza lula umsebenzi, nokwenza izivivinyo.
Ososayensi bedatha namaqembu bangakwazi ukulandelela, bacacise, bathuthukise, futhi baqhathanise izivivinyo namamodeli besebenzisa inkundla yokufunda ezibambele yona kanye nesekelwe emafini ye-meta-machine yokufunda i-Comet.
Inikeza:
- Amakhono amaningi akhona ukuze amalungu eqembu abelane ngemisebenzi.
- Inokuhlanganiswa okuningana okwenza kube lula ukuyixhumanisa nobunye ubuchwepheshe
- Isebenza kahle nemitapo yolwazi ye-ML yamanje
- Inakekela ukuphathwa komsebenzisi
- Ukuqhathaniswa kokuhlolwa kunikwe amandla, okufaka ukuqhathaniswa kwekhodi, amapharamitha aphezulu, amamethrikhi, izibikezelo, ukuncika, namamethrikhi esistimu.
- Ihlinzeka ngamamojula ahlukile wokubona, umsindo, umbhalo, nedatha yethebula ekuvumela ukuthi ubone ngeso lengqondo amasampula.
7. I-Optuna
I-Optuna iwuhlelo lwe-autonomous hyperparameter optimization engasetshenziswa kukho kokubili ukufundwa komshini nokufunda okujulile kanye nezinye izinkambu.
Iqukethe ama-algorithms asezingeni eliphezulu ongakhetha kuwo (noma ukuxhumanisa), yenza kube lula kakhulu ukusabalalisa ukuqeqeshwa kumakhompyutha amaningi, futhi inikeza imiphumela ekhangayo yokubonwa.
Imitapo yolwazi yokufunda yomshini edumile njenge-PyTorch, i-TensorFlow, i-Keras, i-FastAI, i-sci-kit-learn, i-LightGBM, ne-XGBoost yonke ihlanganiswe nayo.
Ihlinzeka ngama-algorithm ahlabayo avumela amakhasimende ukuthi athole imiphumela ngokushesha kakhulu ngokunciphisa ngokushesha amasampula angabonakali ethembisa.
Isebenzisa ama-algorithms asuselwa kuPython, isesha ngokuzenzakalelayo ama-hyperparameter afanelekile. I-Optuna ikhuthaza ukusesha kwe-hyperparameter okuhambisanayo emicu eminingi ngaphandle kokushintsha ikhodi yoqobo.
Inikeza:
- Isekela ukuqeqeshwa okusabalalisiwe kuqoqo kanye nekhompyutha eyodwa (izinqubo eziningi) (ama-multi-node)
- Isekela amasu okunquma amaningana ukusheshisa ukuhlangana (futhi usebenzise ikhompuyutha encane)
- Inokuhlukahluka okubonakalayo okunamandla, okufana nesakhiwo socezu, isakhiwo sekhonta, kanye nezixhumanisi ezihambisanayo.
8. I-Kedro
I-Kedro iwuhlaka lwamahhala lwePython lwekhodi yokubhala engabuyekezwa futhi igcinwe kumaphrojekthi wesayensi yedatha.
Iletha imibono esuka kumikhuba engcono kakhulu yobunjiniyela besofthiwe iye kwikhodi yokufunda yomshini. I-Python iyisisekelo saleli thuluzi lokuhamba komsebenzi.
Ukuze wenze izinqubo zakho ze-ML zibe lula futhi zicace kakhudlwana, ungathuthukisa ukugeleza komsebenzi okuphindaphindekayo, okulondolozekayo, nokumodulayo.
I-Kedro ihlanganisa izimiso zobunjiniyela besofthiwe njengemodularity, ukuhlukaniswa kwezibopho, nokuguqulela endaweni yokufunda yomshini.
Ngokwesisekelo se-Cookiecutter Data Science, inikeza uhlaka lwephrojekthi oluvamile, oluguquguqukayo.
Inani lezixhumi zedatha ezilula ezisetshenziselwa ukugcina nokulayisha idatha kumasistimu amaningana wamafayela namafomethi wefayela, aphethwe ikhathalogi yedatha. Kwenza amaphrojekthi okufunda ngomshini asebenze kakhudlwana futhi kwenza kube lula ukwakha ipayipi ledatha.
Inikeza:
- I-Kedro ivumela ukuthunyelwa komshini ohlakazekile noma uwedwa.
- Ungakwazi ukushintsha ngokuzenzakalelayo ukuncika phakathi kwekhodi ye-Python nokubonwa kokugeleza komsebenzi usebenzisa ukuhunyushwa kwepayipi.
- Ngokusetshenziswa kwekhodi yemodular, esebenziseka kabusha, lobu buchwepheshe busiza ukubambisana kweqembu kumazinga ahlukahlukene futhi buthuthukise ukukhiqiza endaweni yokubhala amakhodi.
- Umgomo oyinhloko uwukunqoba izithiyo zamabhukwana e-Jupyter, izikripthi eziphuma kanye, kanye nekhodi yeglue ngokubhala uhlelo lwesayensi yedatha olugcinekayo.
9. I-BentoML
Iziphetho ze-API yokufunda komshini wokwakha zenziwe zaba lula nge-BentoML.
Ihlinzeka ngengqalasizinda evamile kodwa efingqiwe yokuhambisa amamodeli okufunda omshini ekukhiqizeni.
Ikwenza ukwazi ukupakisha amamodeli afundiwe ukuze asetshenziswe kusilungiselelo sokukhiqiza, uwatolike usebenzisa noma yiluphi uhlaka lwe-ML. Kokubili ukukhonza kwenqwaba okungaxhunyiwe ku-inthanethi nokusetshenziswa kwe-API eku-inthanethi kuyasekelwa.
Iseva yemodeli esebenza kahle kakhulu kanye nokugeleza komsebenzi okuvumelana nezimo izici ze-BentoML.
Ukwengeza, iseva inikeza i-adaptive micro-batching. Indlela ehlangene yokuhlela amamodeli kanye nokulandelela izinqubo zokuphakelwa kunikezwa ideshibhodi ye-UI.
Ngeke kube khona isikhathi sokuphumula seseva ngoba indlela yokusebenza iyimodular futhi ukulungiselelwa kungasetshenziswa kabusha. Kuyinkundla evumelana nezimo yokuhlinzeka, ukuhlela, kanye nokuthumela amamodeli e-ML.
Inikeza:
- Inomklamo we-modular ovumelana nezimo.
- Inika amandla ukusetshenziswa kuzo zonke izinkundla ezimbalwa.
- Ayikwazi ukuphatha ngokuzenzakalelayo ukukala okuvundlile.
- Inika amandla ifomethi yemodeli eyodwa, ukuphathwa kwamamodeli, ukupakishwa kwamamodeli, kanye nokunikezwa kwemodeli esebenza kahle kakhulu.
10. USeldon
Ososayensi bedatha bangadala, basebenzise, futhi balawule amamodeli okufunda omshini nokuhlola esikalini ku-Kubernetes basebenzisa uhlaka lomthombo ovulekile lwe-Seldon Core.
I-TensorFlow, i-sci-kit-learn, i-Spark, i-R, i-Java, ne-H2O zingamakhithi amathuluzi ambalwa nje asekelwa yiyo.
Iphinde ixhumane ne-Kubeflow kanye ne-OpenShift ye-RedHat. I-Seldon core iguqula amamodeli okufunda omshini (amamodeli e-ML) noma izisonga zolimi (izilimi ezifana ne-Python, i-Java, njll.) ibe yizinsiza ezincane zokukhiqiza ze-REST/GRPC.
Elinye lamathuluzi angcono kakhulu e-MLOps okuthuthukisa izinqubo zokufunda ngomshini yileli.
Kulula ukufaka amamodeli e-ML futhi uhlole ukusebenziseka nokuphepha usebenzisa i-Seldon Core.
Inikeza:
- Ukuthunyelwa kwemodeli kungenziwa kube lula ngezinye izindlela ezimbalwa, njengokuthunyelwa kwe-canary.
- Ukuze uqonde ukuthi kungani izibikezelo ezithile zenziwe, sebenzisa izichazi zemodeli.
- Uma kuphakama izinkinga, bheka amamodeli okukhiqiza usebenzisa isistimu yokuxwayisa.
Isiphetho
Ama-MLOps angasiza ukwenza imisebenzi yokufunda yomshini ibe ngcono. Ama-MLOps angasheshisa ukuthunyelwa, enze ukuqoqwa kwedatha nokulungisa iphutha kube lula, futhi athuthukise ukusebenzisana phakathi konjiniyela nososayensi bedatha.
Ukuze ukhethe ithuluzi le-MLOps elivumelana kangcono nezidingo zakho, lokhu okuthunyelwe kuhlole izixazululo eziyi-10 ezidumile ze-MLOps, eziningi zazo eziwumthombo ovulekile.
shiya impendulo