Wonke umntu oye wazama uphuhliso lokufunda koomatshini uyayiqonda indlela ekunzima ngayo. Ngaphandle kweengxaki eziqhelekileyo ekuphuhlisweni kwesoftware, ukufunda ngomatshini (ML) uphuhliso luzisa ukubulawa kwemiqobo eyongezelelweyo.
Amakhulu ezixhobo zomthombo ovulekileyo ziyafumaneka ukunceda kwinqanaba ngalinye le-ML lifecycle, ukusuka ekulungiseleleni idatha ngoqeqesho lwemodeli.
Ngokungafaniyo nophuhliso lwesoftware yemveli, xa amaqela ekhetha isixhobo esinye kwinyathelo ngalinye, ngeML ufuna ukuphonononga sonke isixhobo esikhoyo (umzekelo, i-algorithm) ukubona ukuba iyaphucula na iziphumo.
Ngenxa yoko, abaphuhlisi beML kufuneka basebenzise kwaye bavelise amakhulu ethala leencwadi.
Ii-algorithms zokufunda koomatshini ziqulathe amawaka eeparamitha ezinokwenziwa ngokwezifiso, kwaye kunzima ukuchonga ukuba zeziphi iiparamitha, ikhowudi, kunye nedatha eye yangena kumfuniselo ngamnye ukuvelisa imodeli, nokuba usebenza wedwa okanye kwiqela.
Ngaphandle kokubeka iliso okufanelekileyo, amaqela ahlala esokola ukufumana ikhowudi efanayo ukuba isebenze kwakhona. Nokuba uyisazi sedatha esidlulisela ikhowudi yakho yoqeqesho kwinjineli ukuze isetyenziswe kwimveliso, okanye ubuyela emsebenzini wakho wangaphambili ukuze uxilonge ingxaki, ukuhlehla amanyathelo okuhamba komsebenzi weML kubalulekile.
Ukusa imodeli kwimveliso kunokuba nzima ngenxa yeendlela ezininzi zokusasaza kunye neemeko ezingqongileyo ekufuneka zisetyenziswe (umzekelo, i-REST ukukhonza, i-batch inference, okanye ii-apps eziphathwayo). Akukho ndlela iqhelekileyo yokuhambisa iimodeli ukusuka kulo naliphi na ithala leencwadi ukuya kwesinye sezi zixhobo, kwaye ke ukusasazwa olutsha ngalunye luzisa umngcipheko.
Ngenxa yale miba, kuyacaca ukuba uphuhliso lweML kufuneka luphucuke kakhulu ukuze luzinze, luqikeleleke, kwaye lusetyenziswe ngokubanzi njengophuhliso lwesoftware yemveli.
ML Imingeni
- Kukho ubuninzi bezixhobo ezahlukeneyo. Amakhulu ezisombululo zesoftware ziyafumaneka ukunceda kwinqanaba ngalinye lomatshini wokufunda ubomi, ukusuka ekulungiseleleni idatha ukuya kuqeqesho lwemodeli. Ngapha koko, ngokungafaniyo nophuhliso lwesoftware yemveli, xa amaqela ekhetha isixhobo esinye kwinyathelo ngalinye, ekufundeni koomatshini (ML), uhlala ufuna ukuphonononga zonke izixhobo ezikhoyo (umzekelo, i-algorithm) ukubona ukuba iyaphucula iziphumo. Ngenxa yoko, abaphuhlisi beML kufuneka basebenzise kwaye bavelise amakhulu ethala leencwadi.
- Kunzima ukugcina umkhondo wemifuniselo. Ii-algorithms zokufunda koomatshini ziqulathe amawaka eeparamitha ezinokwenziwa ngokwezifiso, kwaye kunzima ukuchonga ukuba zeziphi iiparamitha, ikhowudi, kunye nedatha eye yangena kumfuniselo ngamnye ukuvelisa imodeli, nokuba usebenza wedwa okanye kwiqela.
- Kunzima ukuphumeza ukufunda ngomatshini. Ukusa imodeli kwimveliso kunokuba nzima ngenxa yeendlela ezininzi zokusasaza kunye neemeko ezingqongileyo ekufuneka zisetyenziswe (umzekelo, i-REST ukukhonza, i-batch inference, okanye ii-apps eziphathwayo). Akukho ndlela iqhelekileyo yokususa imifuziselo nakweliphi na ithala leencwadi ukuya nakwesiphi na kwezi zixhobo. Ngaloo ndlela, ukuthunyelwa olutsha ngalunye luzisa umngcipheko.
Yintoni i MLflow?
I-MLflow liqonga elivulelekileyo lomjikelo wobomi bokufunda ngomatshini. Isekwe kwingqikelelo ye-interface evulekileyo, ephakamisa izinto ezininzi ezinokuthi zithintelwe ezivumela iziseko zangoku kunye ne-algorithms yokufunda koomatshini ukuba idityaniswe ngokulula kunye nenkqubo.
Oku kuthetha ukuba ukuba ungumphuhlisi ofuna ukusebenzisa i-MLflow kodwa usebenzisa isakhelo esingaxhaswanga, uyilo oluvulekileyo lojongano lwenza kube lula ukudibanisa eso sikhokelo kwaye uqale ukusebenza kunye neqonga. Ngokwenyani, oku kuthetha ukuba iMLflow yenzelwe ukusebenza nayo nayiphi na yokufunda umatshini ithala leencwadi okanye ulwimi.
Ngaphezu koko, i-MLflow ikhuthaza ukuphindaphinda, oku kuthetha ukuba uqeqesho olufanayo okanye ikhowudi yokufunda yomatshini wokuvelisa ijoliswe ukuba iqhube kunye neziphumo ezifanayo ezizimeleyo kwimeko-bume, nokuba kwilifu, kwindawo yokusebenzela yendawo, okanye kwincwadana.
Ekugqibeleni, i-MLflow yakhelwe ukulinganisa, ngoko inokusetyenziswa liqela elincinci leenzululwazi zedatha kunye nenkampani enkulu enamakhulu oochwephesha bokufunda ngomatshini.
I-MLflow iyahambelana nalo naliphi na ithala leencwadi lokufunda koomatshini, i-algorithm, isixhobo sokusasaza, okanye ulwimi. Ikwanazo ezi zinto zilandelayo:
- Yenzelwe ukusebenza nayo nayiphi na inkonzo yelifu.
- Izikali kwidatha enkulu nge-Apache Spark.
- I-MLflow iyahambelana neenkqubo ezahlukeneyo zokufunda zomatshini ovulekileyo, kubandakanya iApache Spark, TensorFlow, yaye I-SciKit-Funda.
Ukuba sele unayo ikhowudi, iMLflow ingasetyenziswa nayo. Ungabelana ngesakhelo sakho kunye neemodeli phakathi kwamashishini kuba kunjalo Vula Umnikezi.
Amacandelo e-MLflow: Asebenza njani?
I-MLflow liqonga elikhululekileyo nelivulelekileyo lokulawula umjikelo wobomi be-ML, ebandakanya ukuvavanywa, ukuveliswa kwakhona, ukuhanjiswa, kunye nerejista yemodeli enye. Okwangoku, iMLflow inamacandelo amane:
1. MLflow Tracking
Ndizakuqala nge-MLflow Tracking. I-MLflow ixhasa ukuqokelelwa kweekhonsepthi ezahlukeneyo ezibalulekileyo ezidityaniswe kwindawo esembindini yoqeqesho lwemetadata yokulandelela uvimba. Ingcinga yokuqala yingqokelela yeehyperparameters ezibalulekileyo okanye amaqhosha oqwalaselo aphembelela ukusebenza kwemodeli. Ukusebenzisa i-MLflow's APIs kunye nenkonzo yokulandela umkhondo inokugcina zonke ezi.
Abasebenzisi banokurekhoda idatha yokusebenza ukuze bafumane ukuqonda kwimpumelelo yeemodeli zabo zokufunda koomatshini. Ngaphaya koko, ukuphinda-phinda, i-MLflow ivumela abasebenzisi ukuba babhale ikhowudi yomthombo ethile eyayisetyenziswa ukwenza imodeli kunye noguqulelo lwayo ngokudibanisa ngokuqinileyo neGit ukubopha yonke imodeli kwi-hash ethile yokuzibophelela.
I-MLflow ingasetyenziselwa ukulogisha izinto zakudala, ezizizo naziphi na iifayile ezingafunekiyo ezibandakanya uqeqesho, idatha yovavanyo, kunye neemodeli ngokwazo ukuze ziphinde zenziwe kwakhona.
Oku kuthetha ukuba ukuba ndingumphuhlisi osandula ukuqeqesha imodeli, ndiyakwazi ukuyiqhubela kwinkonzo yokulandelela indawo esembindini, kwaye omnye woogxa bam angayilayisha kamva kwaye aqhubeke nokuqeqesha kunye nokulinga okanye ukuvelisa loo modeli ukuhlangabezana nesidingo esithile. .
Xa usenza ikhowudi yakho yokufunda ngomatshini kwaye emva koko ujonge iziphumo, ukulandela umkhondo yi-API ekuvumela ukuba ungene kwiiparamitha, iinguqulelo zekhowudi, iimetrics, kunye neefayile zemveliso. Ibhalwe kwiPython, R, kunye neJava, phakathi kwezinye iilwimi. Ikwafikeleleka njenge-REST API, enokuthi isetyenziswe ukwakha ii-apps ngaphezulu kwayo.
Ezona mpawu
- Abaphuhlisi abaninzi basebenzisa i-MLflow kwi-PC yabo yendawo, apho i-backend kunye nokugcinwa kwe-artifact yabelana ngolawulo kwi-disc.
- Abasebenzisi abaninzi bakwasebenzisa iSQLite, idatabase ehambelana neSQLAlchemy, ukusebenzisa iMLflow kwiiPC zabo zasekhaya.
- I-MLflow ikwaxhasa ulwakhiwo lwezakhiwo olusasaziweyo. Umncedisi wokulandelela, ivenkile ye-backend, kunye nevenkile ye-artifact zonke zibanjwe kwiiseva ezahlukeneyo kwezi.
- Ukuba ukubaleka kuqaliswe yiProjekthi yeMLflow, i-git commit hash isetyenzisiwe. I-MLflow Python, i-R, iJava, kunye ne-REST APIs ingasetyenziselwa ukuloga idatha ukuze iqhube.
Ngolwazi oluthe kratya, ungajonga igosa maxwebhu.
2. Iiprojekthi zeMLFlow
Emva kokuba sidlule kumacandelo okulandela umkhondo, ndingathanda ukuthetha malunga neeprojekthi zeMLflow, ezilulungiselelo lokupakisha oluphindaphindwayo lweeseshoni zoqeqesho lwemodeli ngaphandle komxholo wophumezo.
Amashishini asebenzisa uluhlu olubanzi lweetekhnoloji zoqeqesho lokufunda koomatshini, kodwa asebenzisa ezi zixhobo zoqeqesho kwiindidi ezahlukeneyo zeemeko. Ngokomzekelo, banokuthi baqhube ikhowudi yabo yoqeqesho kwilifu, kwi-PC yendawo, okanye kwi-notebook.
Oku kukhokelela kwingxaki yokuba iziphumo zokufunda koomatshini kunzima ukuziphindaphinda. Rhoqo, ikhowudi yoqeqesho efanayo ayiphumeleli okanye inike iziphumo ezifanayo kwiindawo ezimbini ezahlukeneyo.
Isisombululo esibonelelwe yi-MLflow yinkcazo yeprojekthi yoqeqesho oluzimeleyo olubandakanya yonke ikhowudi yokuqeqeshwa komatshini wokufunda, kunye nokuxhomekeka kwelayibrari yenguqulelo, izicwangciso, kunye noqeqesho kunye novavanyo lwedatha.
I-MLflow iqinisekisa ukuveliswa kwakhona kuyo yonke imixholo yokwenziwa ngokuchaza ngokucacileyo yonke isethi yeemfuno zenkqubo yokuqeqeshwa komatshini wokufunda. Ifezekisa oku ngokufaka onke loo mathala kwaye ifezekise inkqubo efanayo ikhowudi esebenza kuyo.
Iprojekthi yeMLflow ayikho enye into ngaphandle koluhlu. Luluhlu olubandakanya ikhowudi yoqeqesho, inkcazo yokuxhomekeka kwethala leencwadi, kunye nenye idatha eyimfuneko ngeseshoni yoqeqesho, kunye nale fayile yoqwalaselo yokuzikhethela.
Ezi mfuno zamathala eencwadi zinokuchazwa ngeendlela ezahlukeneyo. Abasebenzisi banakho, umzekelo, ukubonelela ngemeko ye-anaconda efomathiweyo ye-YAML ukudwelisa iimfuno zabo zethala leencwadi lekhowudi yoqeqesho. I-MLflow iya kusebenzisa ikhowudi yoqeqesho ngaphakathi kwesikhongozeli. Kwimeko enjalo, banokubandakanya isikhongozeli seDocker.
Ekugqibeleni, i-MLflow ine-interface yomgca womyalelo (CLI) wokuqhuba ezi projekthi, kunye nePython, kunye neJava APIs. Ezi projekthi zinokuqhutywa kwisixokelelwano sasekhaya somsebenzisi ngokunjalo nakwiindlela ezahlukeneyo zokuseta ezikude ezifana nomcwangcisi weDatabricks kunye noKubernetes. Iiprojekthi zeMLflow zikuvumela ukuba upakishe ikhowudi yesayensi yedatha ngendlela ephindaphindwayo nesebenzisekayo, ikakhulu ngokusekwe kwimigangatho.
Icandelo leeprojekthi libandakanya i-API kunye nezixhobo zomyalelo wokulawula iiprojekthi. Ezi zakhono ziqinisekisa ukuba iiprojekthi zinokudityaniswa kunye ukwenza iinkqubo zokufunda koomatshini.
Ezona mpawu
- I-MLflow ixhasa ubume beprojekthi, kubandakanya indawo yesikhongozeli seDocker, indawo yeConda, kunye nemeko yenkqubo.
- Nawuphi na uvimba weGit okanye uvimba weefayili wasekuhlaleni unokuqwalaselwa njengeprojekthi yeMLflow; ngokuzenzekela; ungasebenzisa naliphi na iqokobhe okanye Umbhalo wePython kuluhlu njengendawo yokungena yeprojekthi.
- Ukuxhomekeka kwe-non-Python, njengamathala eencwadi eJava, kunokubanjwa kusetyenziswa izikhongozeli zeDocker.
- Unokufumana ulawulo olukhulu kwiProjekthi ye-MLflow ngokongeza ifayile yeprojekthi kulawulo lweengcambu zeprojekthi, eyifayile yokubhaliweyo kwi-syntax ye-YAML.
Ngolwazi oluthe kratya, ungajonga igosa maxwebhu.
3. Iimodeli zeMLflow
Ngoku, ndingathanda ukuxoxa ngeemodeli zeMLflow, imodeli yenjongo eqhelekileyo exhasa uluhlu olubanzi lweemeko zemveliso. Isizathu seemodeli zeMLflow ngoku siyafana neso seeprojekthi.
Kwakhona, siyabona ukuba iimodeli zinokuveliswa kusetyenziswa uluhlu olubanzi lwezixhobo, kodwa zinokuveliswa okanye zifakwe kuluhlu olukhulu lweemeko, ngokuchasene neendawo zoqeqesho.
Ezi zicwangciso zibandakanya izixhobo zokusebenza ngexesha langempela, njenge-Kubernetes okanye i-Amazon SageMaker, kunye nokusasazwa kunye ne-batch scoring, njenge-Spark. Ngaphaya koko, amanye amashishini anokukhetha ukusebenzisa iimodeli njengenkonzo yewebhu ye-RESTful esebenza kumzekelo welifu olungiselelwe kwangaphambili.
Imodeli ye-MLflow, njengeprojekthi, sisakhiwo solawulo. Ibandakanya ifayile yoqwalaselo kwaye, ngeli xesha, imodeli ye-artifact ye-serialized kunokuba ikhowudi yoqeqesho. Ikwabandakanya le seti yokuxhomekeka ukuphinda-phinda njengeprojekthi. Ngeli xesha, siza kuqwalasela ukuxhomekeka kovavanyo kumxholo wendawo yeConda.
Ukongeza, i-MLflow ibandakanya izixhobo zokuvelisa imodeli yeemodeli zokulinganisa kwifomathi yeMLflow ukusuka kuluhlu lwesakhelo esidumileyo. Ekugqibeleni, i-MLflow yongeza i-deploys, i-APIs yokuvelisa kunye nokudibanisa nayiphi na imodeli ye-MLflow kuluhlu lweenkonzo, kwaye ezi APIs zifumaneka kwi-Python, Java, R, kunye nefomethi ye-CLI.
Iimodeli licandelo elinolwakhiwo olusemgangathweni lweemodeli zokupakisha ezinokuthi zisetyenziswe kwaye ziqondwe ngezixhobo ezisezantsi ezinje ngeeseva zokuphonononga okanye Izitena zedatha iqonga ibhetshi inferencing. Eli candelo ligcina iiyure zekhowudi ye-bespoke xa upakisha imodeli yemveliso.
IModeli yeMLflow ngumgangatho wokupakisha iimodeli zokufunda koomatshini kwiintlobo ezahlukeneyo ezaziwa ngokuba "ziincasa." I-MLflow ibonelela ngezixhobo ezininzi zokukunceda ekubekeni iindidi ngeendidi zeemodeli. IModeli nganye yeMLflow igcinwa njengoluhlu oluqulathe iifayile ezingafunekiyo kunye nefayile yenkcazo yemodeli yeML enoluhlu lwencasa enokuthi isetyenziswe kuyo.
Ezona mpawu
- Zonke izixhobo zokusasaza ezakhelwe ngaphakathi zeMLflow zibonelela ngeencasa ezininzi “eziqhelekileyo”, ezifana nencasa ye “Python function” echaza indlela yokuqhuba imodeli njengomsebenzi wePython.
- IModeli nganye yeMLflow iqulathe uluhlu oluqulathe iifayile ezingafunekiyo, kunye nefayile yomfuziselo weML kwingcambu yolawulo echaza iincasa ezininzi zemodeli.
- Xa ugcina imodeli, iMLflow ikuvumela ukuba ukhankanye iparamitha yemeko-bume yeConda equlethe ukuxhomekeka komfuziselo. Ukuba akukho bume beConda buxeliweyo, imeko-bume engagqibekanga esekwe kwincasa yemodeli iyakhiwa. Emva koko, indawo yeConda igcinwa kwi conda.yaml.
Ngolwazi oluthe kratya, ungajonga igosa maxwebhu.
4. UBhaliso loMfanekiso weMLflow
Irejistri eyimodeli yindawo yokugcina yokufunda koomatshini bokufunda (ML) imifuziselo. IRegistry yeModel yenziwe nge-APIs kunye nesicelo esisekelwe kwiwebhu esisetyenziselwa ukugcina iimodeli kwizigaba ezahlukeneyo njengeqela. Imodeli yoMda, iModel Versioning, i-Easy Stage Transition, kunye ne-Annotation zimbalwa zesakhono ezikhoyo kwiRegistry Model.
Irejista yemodeli, ngaphezu kweemodeli ngokwazo, iqulethe ulwazi (imethadatha) malunga neenkcukacha kunye nemisebenzi yoqeqesho esetyenziselwa ukwakha imodeli. Kubalulekile ukugcina umkhondo wala magalelo afunekayo ukwenza umnombo weemodeli zeML. Kule meko, imodeli yobhaliso isebenza ngokufanayo kwiisoftware eziqhelekileyo ulawulo lwenguqulo iinkqubo (umzekelo, iGit, SVN) kunye neendawo zokugcina izinto zakudala (umz., Artifactory, PyPI).
IRegistry yeModeli yinkqubo evumela izazinzulu zedatha kunye neenjineli zokufunda ngomatshini ukuba zipapashe, zivavanye, zibeke iliso, zilawule, kwaye zisasaze iimodeli zabo zentsebenziswano namanye amaqela. Ngokusisiseko, imodeli yobhaliso iyasetyenziswa xa ugqibile isigaba sakho sovavanyo kwaye ulungele ukwabelana ngeziphumo zakho neqela kunye nabachaphazelekayo.
I-MLflow Model Registry inikeza i-API kunye ne-interface yomsebenzisi yokulawula iimodeli zakho kunye nobomi babo ukusuka kwindawo ephakathi. Umda wemodeli, imodeli yoguqulelo, amanqakwana, kunye notshintsho lweqonga zonke ziyafumaneka ngobhaliso.
Kwi-MLflow, imodeli ebhalisiweyo yiyo enegama elilodwa kunye nemethadatha, iinguqulelo zemodeli, izigaba zenguqu, kunye nemodeli yomgca. Imodeli enye okanye ngaphezulu inokufumaneka kwimodeli ebhalisiweyo. Imodeli entsha ithathwa njengenguqulo 1 xa ibhalisiwe kwirejista. Le nguqulo ilandelayo yongezwa kuyo nayiphi na imodeli entsha enegama elifanayo.
Unokwabela inyathelo elinye kuyo nayiphi na imodeli yenguqulelo nangaliphi na ixesha. Nangona kunjalo, izigaba kufuneka zinikezelwe phantsi kwezigaba ze-MLflow ezichazwe ngokusemthethweni, ezifana nesiteji, ukuveliswa, kunye nokugcinwa kweenkcukacha. Uguqulelo lwemodeli lunokutshintshwa ukusuka kwinqanaba ukuya kwelinye.
I-MLflow ikuvumela ukuba usebenzise i-markdown ukuchaza imodeli yomgangatho ophezulu kunye noguqulelo ngalunye oluthile. Unokubandakanya iinkcazo kunye nolunye ulwazi olufanelekileyo, njengeenkcazo ze-algorithm, indlela yokusebenza, kunye neeseti zedatha ezisetyenzisiweyo.
Ezona mpawu
- Ukufikelela kwimodeli yobhaliso nge-UI okanye i-API xa ubamba eyakho i-MLflow umncedisi, kufuneka usebenzise i-database-backed backend store.
- IsiBhalo soMfanekiso sinokufikelelwa ngencasa yemodeli ye-MLflow okanye ujongano lwe-API yokuLandelwa koMthengi weMLflow. Unokwenza, umzekelo, ubhalise imodeli ngexesha lovavanyo lweMLflow okanye emva kwayo yonke imifuniselo yakho.
- Ayinguye wonke umntu oza kuqalisa ukuqeqesha iimodeli zabo esebenzisa iMLflow. Ngenxa yoko, unokuba neemodeli ezithile eziqeqeshiweyo ngaphambi kokusebenzisa iMLflow. Kunokuba uphinde uqeqeshe iimodeli, unqwenela ukubhalisa iimodeli zakho ezigciniweyo kunye neRegistry yeModeli.
Ngolwazi oluthe kratya, ungajonga igosa maxwebhu.
isiphelo
I-MLflow sesona sixhobo sibalaseleyo kwaye sikhula rhoqo seML. Unokuyisebenzisa ecaleni kwezixhobo zakho zangoku kunye namaqonga.
Ixhasa iilwimi ezininzi zokucwangcisa, kubandakanywa iPython, iJava, kunye neR. Unokuphinda ulandelele ngokukhawuleza, ugcine, kwaye uthelekise iinguqulelo zeemodeli ezahlukeneyo enkosi kuyilo olusebenziseka lula.
Zama iMLflow kwaye usazise ngamava akho!
Shiya iMpendulo