Amacandelo aliqela ehlabathi aqala ukutyala imali kakhulu kufundo ngoomatshini (ML).
Iimodeli ze-ML zinokusungulwa kwaye ziqhutywe ngamaqela eengcaphephe, kodwa omnye wemiqobo emikhulu kukudlulisela ulwazi olufunyenweyo kwimodeli elandelayo ukuze iinkqubo zandiswe.
Ukuphucula kunye nokulinganisa iinkqubo ezibandakanyekayo kulawulo lwemodeli yobomi, iindlela ze-MLOps ziya zisetyenziswa ngakumbi ngamaqela adala imodeli yokufunda koomatshini.
Qhubeka ufunda ukuze ufumane ngakumbi malunga nezona zixhobo zibalaseleyo ze-MLOps kunye namaqonga akhoyo namhlanje kunye nendlela abanokwenza ngayo ukufunda koomatshini kube lula kwisixhobo, umphuhlisi, kunye nombono wenkqubo.
Yintoni iMLOps?
Ubuchule bokuyila imigaqo-nkqubo, izithethe, kunye nezona ndlela zilungileyo zemodeli yokufunda koomatshini zaziwa “njengemisebenzi yokufunda ngomatshini,” okanye “iiMLOps.”
I-MLOps ijolise ekuqinisekiseni yonke i-lifecycle yophuhliso lwe-ML - ukusuka ekukhawulweni ukuya ekuhanjisweni - ibhalwe ngokucokisekileyo kwaye ilawulwa ngezona ziphumo ezilungileyo kunokuba kutyalwe ixesha elininzi kunye nezibonelelo kuyo ngaphandle kwesicwangciso.
Injongo ye-MLOps kukulungelelanisa iindlela ezilungileyo ngendlela eyenza uphuhliso lokufunda koomatshini lube lukhulu ngakumbi kubasebenzisi be-ML kunye nabaphuhlisi, kunye nokuphucula umgangatho kunye nokhuseleko lweemodeli zeML.
Abanye babhekisa kwi-MLOps njenge "DevOps yokufunda koomatshini" kuba isebenzisa ngempumelelo imigaqo ye-DevOps kwicandelo elikhethekileyo lophuhliso lwetekhnoloji.
Le yindlela eluncedo yokucinga nge-MLOps kuba, njenge-DevOps, igxininisa ukwabelana ngolwazi, intsebenziswano, kunye nezenzo ezilungileyo phakathi kwamaqela kunye nezixhobo.
I-MLOps ibonelela abaphuhlisi, izazinzulu zedatha, kunye namaqela okusebenza ngesakhelo sokusebenzisana kwaye, ngenxa yoko, ukuvelisa iimodeli zeML ezinamandla kakhulu.
Kutheni Usebenzisa izixhobo zeMLOps?
Izixhobo ze-MLOps zinokwenza uluhlu olubanzi lwemisebenzi kwiqela le-ML, nangona kunjalo, zihlala zihlulwe zibe ngamaqela amabini: ulawulo lweqonga kunye nolawulo lwecandelo lomntu ngamnye.
Nangona ezinye iimveliso ze-MLOps zijolise kuphela kumsebenzi ongundoqo omnye, njengedatha okanye ulawulo lwemethadatha, ezinye izixhobo zisebenzisa isicwangciso esiquka konke kwaye zibonelela ngeqonga le-MLOps lokulawula iinkalo ezininzi ze-ML lifecycle.
Jonga izisombululo zeMLOps ezinceda iqela lakho ekulawuleni ezi ndawo zophuhliso lweML, nokuba ufuna ingcaphephe okanye isixhobo esibanzi ngakumbi:
- Ukuphathwa kwedatha
- Uyilo kunye nomzekelo
- Ulawulo lweeprojekthi kunye nendawo yokusebenza
- Ukuhanjiswa kwemodeli yeML kunye nokugcinwa okuqhubekayo
- Ulawulo lwe-Lifecycle ukusuka ekuqaleni ukuya ekupheleni, olunikezelwa ngokuqhelekileyo ngamaqonga eenkonzo ezipheleleyo ze-MLOps.
Izixhobo zeMLOps
1. MLFlow
Ubomi bomjikelo wokufunda umatshini ulawulwa yi-open-source platform MLflow kwaye ibandakanya ukubhaliswa kwemodeli ephakathi, ukuthunyelwa, kunye nokulinga.
I-MLflow ingasetyenziswa naliphi na iqela lobungakanani, ngokuzimeleyo nangokudibeneyo. Amathala eencwadi akananxaxheba kwisixhobo.
Naluphi na ulwimi lwenkqubo kunye nethala leencwadi lokufunda koomatshini linokulisebenzisa.
Ukwenza kube lula ukuqeqesha, ukuhambisa, kunye nokulawula izicelo zokufunda koomatshini, i-MLFlow isebenzisana nenani lezicwangciso zokufunda zoomatshini, kubandakanywa. TensorFlow kunye nePytorch.
Ukongezelela, i-MLflow inikezela nge-APIs ekulula ukuyisebenzisa enokuthi ifakwe kuyo nayiphi na inkqubo ekhoyo yokufunda koomatshini okanye iilayibrari.
I-MLflow ineempawu ezine eziphambili eziququzelela imifuniselo yokulandela umkhondo nokucwangcisa:
- I-MLflow Tracking-i-API kunye ne-UI yokugawulwa kwekhowudi yokufunda iparameters, iinguqulelo, iimetrics, kunye nezinto zakudala kunye nokubonisa nokuchasanisa iziphumo.
- IiProjekthi zeMLflow - ikhowudi yokufunda yomatshini wokupakisha kwifomathi enokuphinda isetyenziswe, iphinda iphinde iphinde iphinde iphinde iphinde iphinde iphinde iphinde iphinde iphinde isetyenziswe kwimveliso okanye ukwabelana nabanye oososayensi bedatha.
- IiModeli zeMLflow-ukugcina kunye nokuhambisa iimodeli kuluhlu lweemodeli ezisebenzayo kunye neenkqubo zokuthelekelela ezivela kumathala eencwadi eML.
- UBhaliso lweModeli yeMLflow – ivenkile eyimodeli esembindini eyenza ukuba ulawulo lwentsebenziswano ye-MLflow ubomi bubonke, kubandakanywa uguqulelo lwemodeli, utshintsho lweqonga, kunye namanqakwana.
2. IKubeflow
Ibhokisi yezixhobo zeML yeKubernetes ibizwa ngokuba yiKubeflow. Ukupakishwa kunye nokulawula izikhongozeli zeDocker, izibonelelo kugcino lwe iinkqubo zokufunda koomatshini.
Ngokwenza lula ukuqhutywa kwe-orchestration kunye nokusasazwa kokuhamba komsebenzi wokufunda koomatshini, kukhuthaza ukuxhaphaka kweemodeli zokufunda koomatshini.
Yiprojekthi yomthombo ovulekileyo equka iqela elikhethwe ngononophelo lwezixhobo ezincedisayo kunye nezicwangciso ezilungiselelwe iimfuno ezahlukeneyo zeML.
Imisebenzi yoqeqesho emide ye-ML, ukulinga ngesandla, ukuphindaphinda, kunye nemingeni ye-DevOps inokuphathwa ngeKubeflow Pipelines.
Kwizigaba ezininzi zokufunda koomatshini, kubandakanywa uqeqesho, uphuhliso lwemibhobho, kunye nokugcinwa kwe Iincwadana zeJupyter, I-Kubeflow inikezela ngeenkonzo ezizodwa kunye nokudibanisa.
Yenza kube lula ukulawula kunye nokulandelela ubomi bomthwalo wakho we-AI kunye nokuthumela iimodeli zokufunda ngoomatshini (ML) kunye nemibhobho yedatha kumaqela e-Kubernetes.
Inika:
- Iincwadana zamanqaku zokusebenzisa i-SDK ukusebenzisana nesistim
- ujongano lomsebenzisi (UI) lokulawula kunye nokubeka iliso okuqhubayo, imisebenzi, kunye nemifuniselo
- Ukuyila ngokukhawuleza izisombululo zokuphela ukuya-ekupheleni ngaphandle kokuphinda kwakhiwe ixesha ngalinye, kwaye uphinde usebenzise izixhobo kunye nemibhobho.
- Njengenxalenye ephambili ye-Kubeflow okanye njengofakelo oluzimeleyo, i-Kubeflow Pipelines inikezelwa.
3. Ulawulo lweNguqulelo yeDatha
Isisombululo solawulo lwenguqulelo evulelekileyo yeeprojekthi zokufunda ngomatshini sibizwa ngokuba yiDVC, okanye uLawulo lweNguqulelo yeDatha.
Nokuba loluphi ulwimi olukhethayo, sisixhobo sokulinga esinceda kwingcaciso yombhobho.
I-DVC isebenzisa ikhowudi, uguqulelo lwedatha, kunye nokuveliswa kwakhona ukuze kukuncede wonge ixesha xa ufumanisa umcimbi ngoguqulelo lwangaphambili lwemodeli yakho yeML.
Ukongeza, ungasebenzisa imibhobho ye-DVC ukuqeqesha imodeli yakho kwaye uyisasaze kumalungu eqela lakho. Umbutho omkhulu wedatha kunye noguqulelo lunokuphathwa yi-DVC, kwaye idatha ingagcinwa ngendlela efikeleleke ngokulula.
Nangona ibandakanya ezinye (ezilinganiselwe) iimpawu zokulandelela umfuniselo, ikakhulu igxile kwidatha kunye noguqulelo lwemibhobho kunye nolawulo.
Inika:
- I-agnostic yokugcina i-agnostic, ngoko ke kunokwenzeka ukuba uqeshe iintlobo ezahlukeneyo zokugcina.
- Inika izibalo zokulandelela ngokunjalo.
- indlela eyakhiwe kwangaphambili yokujoyina izigaba zeML kwiDAG kwaye iqhube umbhobho wonke ukusuka ekuqaleni ukuya ekugqibeleni.
- Uphuhliso lwemodeli nganye ye-ML inokulandelwa kusetyenziswa ikhowudi yayo yonke kunye nokuvela kwedatha.
- Ukuveliswa kwakhona ngokugcina ngokuthembekileyo ukucwangciswa kokuqala, idatha yegalelo, kunye nekhowudi yeprogram yovavanyo.
4. Pachyderm
I-Pachyderm yinkqubo yokulawula uguqulelo lokufunda koomatshini kunye nesayensi yedatha, efana ne-DVC.
Ukongeza, ngenxa yokuba yenziwe kusetyenziswa Docker kunye neKubernetes, inokuphumeza kwaye ifake izicelo zokuFunda ngoomatshini kulo naliphi na iqonga lelifu.
I-Pachyderm yenza iziqinisekiso ukuba isicatshulwa ngasinye sedatha esisetyenzisiweyo kwimodeli yokufunda yomatshini sinokulandelelwa emva kwaye siguqulelwe.
Isetyenziselwa ukwenza, ukuhambisa, ukulawula, kunye nokubeka iliso kwiimodeli zokufunda koomatshini. Irejistri eyimodeli, inkqubo yolawulo lwemodeli, kunye nebhokisi yesixhobo ye-CLI zonke zibandakanyiwe.
Abaphuhlisi banokuzenza ngokuzenzekelayo kwaye bandise ubomi babo bokufunda ngomatshini usebenzisa isiseko sedatha ye-Pachyderm, eqinisekisa ukuphindaphinda.
Ixhasa imigangatho yolawulo lwedatha engqongqo, yehlisa ukusetyenzwa kwedatha kunye neendleko zokugcina, kwaye inceda amashishini ekuziseni amanyathelo abo esayensi yedatha ukuthengisa ngokukhawuleza.
5. Polyaxon
Ukusebenzisa iqonga lePolyaxon, iiprojekthi zokufunda koomatshini kunye nezicelo zokufunda ezinzulu zinokuphinda ziphindwe kwaye zilawulwe kumjikelezo wobomi babo bonke.
I-Polyaxon iyakwazi ukusingatha kunye nokulawula isixhobo, kwaye inokufakwa kuyo nayiphi na indawo yedatha okanye umboneleli wefu. NjengeTorch, Tensorflow, kunye ne-MXNet, exhasa zonke ezona zikhokelo zokufunda zinzulu.
Xa kuziwa kwiokhestra, iPolyaxon ikuvumela ukuba wenze uninzi lweqela lakho ngokucwangcisa imisebenzi kunye novavanyo nge-CLI yabo, ideshibhodi, ii-SDKs, okanye i-REST API.
Inika:
- Ungasebenzisa inguqulelo yomthombo ovulekileyo ngoku, kodwa ikwabandakanya ukhetho lweshishini.
- Nangona igubungela umjikelo opheleleyo wobomi, kubandakanya neokhestra yokubaleka, iyakwazi okungaphezulu.
- Ngamaxwebhu ereferensi yobugcisa, izikhokelo zokuqalisa, izixhobo zokufunda, iincwadana, ii-tutorials, ii-changelogs, kunye nokunye, liqonga elibhalwe kakuhle kakhulu.
- Ngedeshibhodi yemibono yovavanyo, kuyenzeka ukuba uhlale ujonge, ulandelele, kwaye uvavanye umfuniselo ngamnye wokuphucula.
6. unomgca
I-Comet liqonga lokufunda ngoomatshini bemeta abalandelela, bathelekise, bacacise, kwaye baphucule imifuniselo kunye neemodeli.
Yonke imifuniselo yakho inokubonwa kwaye ithelekiswe kwindawo enye.
Isebenza kuwo nawuphi na umsebenzi wokufunda ngomatshini, naphi na ikhowudi yakho eyenziwayo, kunye nalo naliphi na ithala leencwadi lokufunda ngoomatshini.
I-Comet ifanelekile kumaqela, abantu, amaziko emfundo, amashishini, kunye nabani na ongomnye onqwenela ukujonga ngokukhawuleza imifuniselo, ukulungelelanisa umsebenzi, kunye nokwenza imifuniselo.
Izazinzulu zedatha kunye namaqela angalandelela, acacise, aphucule, kwaye athelekise imifuniselo kunye neemodeli zisebenzisa iqonga lokuzibamba kunye nelifu elisekelwe kwi-meta-machine yeComet.
Inika:
- Izakhono ezininzi zikhona ukuze amalungu eqela abelane ngemisebenzi.
- Ineendibaniselwano ezininzi ezenza kube lula ukuyidibanisa nobunye ubugcisa
- Isebenza kakuhle ngamathala eencwadi eML angoku
- Ikhathalela ulawulo lomsebenzisi
- Ukuthelekisa imifuniselo yenziwe, kubandakanywa ukuthelekisa ikhowudi, i-hyperparameters, i-metrics, uqikelelo, ukuxhomekeka, kunye ne-system metrics.
- Ibonelela ngeemodyuli ezahlukeneyo zombono, iaudio, umbhalo, kunye nedatha yetheyibhile ekuvumela ukuba ubone iisampuli.
7. I-Optuna
I-Optuna yinkqubo ye-autonomous hyperparameter optimization enokuthi isetyenziswe ekufundeni koomatshini kunye nokufunda nzulu kunye namanye amacandelo.
Iqulethe iintlobo ngeentlobo zealgorithms onokuthi ukhethe kuyo (okanye uqhagamshele), yenza kube lula kakhulu ukusasaza uqeqesho kwiikhompyuter ezininzi, kwaye inikezela ngokubonwa kweziphumo ezinomtsalane.
Iilayibrari zokufunda koomatshini abadumileyo njengePyTorch, TensorFlow, Keras, FastAI, sci-kit-learn, LightGBM, kunye neXGBoost zonke zidibene nayo.
Ibonelela nge-algorithms yokusika evumela abathengi ukuba bafumane iziphumo ngokukhawuleza ngokunciphisa ngokukhawuleza iisampulu ezingabonakali zithembisa.
Isebenzisa i-algorithms esekwe kwiPython, ikhangela ngokuzenzekelayo iihyperparameter ezifanelekileyo. I-Optuna ikhuthaza uphendlo lwehyperparameter enxuseneyo kwimisonto emininzi ngaphandle kokuguqula ikhowudi yoqobo.
Inika:
- Ixhasa uqeqesho olusasazwayo kwi-cluster kunye nekhompyuter enye (iinkqubo ezininzi) (ii-multi-node)
- Ixhasa iindlela ezininzi zokucheba ukukhawulezisa ukudibanisa (kwaye usebenzise ikhompuyutha encinci)
- Inemibono eyahlukahlukeneyo enamandla, efana neploti yesilayi, iploti yecontour, kunye nolungelelwaniso oluhambelanayo.
8. Kedro
I-Kedro yinkqubo ye-Python yamahhala yekhowudi yokubhala enokuthi ihlaziywe kwaye igcinwe kwiiprojekthi zesayensi yedatha.
Izisa iingcamango ezivela kwiindlela ezilungileyo zobunjineli besoftware ukuya kwikhowudi yokufunda koomatshini. I-Python isisiseko sesi sixhobo sokuhamba komsebenzi.
Ukwenza iinkqubo zakho ze-ML zibe lula kwaye zichaneke ngakumbi, unokuphuhlisa ukuveliswa kwakhona, ukugcinwa, kunye nokuhamba kwemodyuli.
I-Kedro ibandakanya imigaqo yobunjineli besoftware efana nokumodareyitha, ukwahlulwa koxanduva, kunye noguqulelo kwindawo yokufunda koomatshini.
Ngokusekelwe kwiSayensi yeNkcukacha yeCookiecutter, ibonelela ngesakhelo seprojekthi esiqhelekileyo, esiguquguqukayo.
Uninzi lwezixhumi zedatha ezilula ezisetyenziselwa ukugcina nokulayisha idatha kwiinkqubo ezininzi zeefayile kunye neefomathi zefayile, zilawulwa yikhathalogu yedatha. Yenza iiprojekthi zokufunda koomatshini zisebenze ngakumbi kwaye yenza kube lula ukwakha umbhobho wedatha.
Inika:
- I-Kedro ivumela ukusasazwa komatshini osasazekileyo okanye wedwa.
- Unokwenza ngokuzenzekelayo ukuxhomekeka phakathi kwekhowudi yePython kunye nokubonwa kokuhamba komsebenzi usebenzisa ukukhutshwa kombhobho.
- Ngokusetyenziswa kweemodyuli, ikhowudi enokusetyenziswa kwakhona, le teknoloji iququzelela ukusebenzisana kweqela kumanqanaba ahlukeneyo kwaye iphucula imveliso kwindawo yokubhala ikhowudi.
- Eyona njongo iphambili kukukoyisa ukungaphumeleli kweencwadi zamanqaku zeJupyter, izikripthi zodwa, kunye nekhowudi yeglue ngokubhala inkqubo yesayensi yedatha egcinayo.
9. I-Bentoml
Isiphelo se-API sokufunda ngomatshini senziwe lula ngeBentoML.
Ibonelela ngesiseko esiqhelekileyo kodwa esicuthiweyo ukuhambisa iimodeli zokufunda zoomatshini kwimveliso.
Ikwenza ukwazi ukupakisha imifuziselo efundiweyo ukuze isetyenziswe kubume bemveliso, ukuyitolika usebenzisa nasiphi na isakhelo seML. Zombini iibhetshi ezisebenza ngaphandle kweintanethi kunye nokusebenza kwe-API kwi-intanethi kuyaxhaswa.
Umncedisi wemodeli yokusebenza okuphezulu kunye nokuhamba komsebenzi oguquguqukayo zizinto zeBentoML.
Ukongeza, iseva ibonelela nge-adaptive micro-batching. Indlela emanyeneyo yokulungiselela imifuziselo kunye nokugcina umkhondo weenkqubo zokusasazwa kubonelelwa yideshibhodi ye-UI.
Akuyi kubakho xesha lokuphumla kwiseva kuba indlela yokusebenza iyimodyuli kwaye uqwalaselo luyaphinda lusetyenziswe. Liqonga eliguquguqukayo lokubonelela, ukulungelelanisa, kunye nokuthunyelwa kweemodeli zeML.
Inika:
- Inoyilo lwemodyuli oluguquguqukayo.
- Ivumela ukusasazwa kwiiplatifti ezininzi.
- Ayinakuphatha ngokuzenzekelayo ukukala okuthe tye.
- Inika amandla ifomathi yemodeli enye, ulawulo lwemodeli, ukupakishwa kwemodeli, kunye nemodeli yokusebenza ephezulu.
10. USeldon
Izazinzulu zedatha zinokudala, zisebenzise, kwaye zilawule iimodeli zokufunda zoomatshini kunye nemifuniselo kwisikali kwi-Kubernetes isebenzisa isakhelo esivulelekileyo seSeldon Core.
I-TensorFlow, i-sci-kit-learn, i-Spark, i-R, i-Java, kunye ne-H2O zimbalwa nje zezixhobo zezixhobo ezixhaswa yiyo.
Ikwanxibelelana neKubeflow kunye neRedHat's OpenShift. Undoqo weSeldon uguqula imifuziselo yokufunda koomatshini (imifuziselo yeML) okanye izisongelo zolwimi (iilwimi ezifana nePython, iJava, njl. njl.) zibe yimveliso REST/GRPC microservices.
Esinye sezona zixhobo zeMLOps zokuphucula iinkqubo zokufunda ngomatshini yile.
Kulula ukufaka iimodeli zeML kunye novavanyo lokusebenziseka kunye nokhuseleko usebenzisa iSeldon Core.
Inika:
- Ukuhanjiswa kwemodeli kunokwenziwa lula ngeendlela ezininzi, ezifana nokuhanjiswa kwe-canary.
- Ukuze uqonde ukuba kutheni uqikelelo oluthile lwenziwe, sebenzisa iimodeli zokucacisa.
- Xa kuvela imiba, gcina iliso kwiimodeli zemveliso usebenzisa inkqubo yokulumkisa.
isiphelo
Ii-MLOps zinokunceda ukwenza imisebenzi yokufunda koomatshini ibe ngcono. Ii-MLOps zinokukhawulezisa ukuthunyelwa, zenze ukuqokelelwa kwedatha kunye nokulungisa iimpazamo zibe lula, kwaye ziphucule intsebenziswano phakathi kweenjineli kunye nososayensi bedatha.
Ukuze ukhethe isixhobo se-MLOps esizifanela kakuhle iimfuno zakho, esi sithuba sihlolisise izisombululo ezili-10 ezithandwayo ze-MLOps, uninzi lwazo zingumthombo ovulekileyo.
Shiya iMpendulo