Wese munhu akaedza muchina kudzidza kuvandudza anonzwisisa kuti zvakaoma sei. Kunze kwematambudziko akajairwa mukugadzirwa kwesoftware, kudzidza kwemichina (ML) kunounza huwandu hwezvimwe zvipingamupinyi.
Mazana ezvishandiso zvakavhurwa sosi aripo kuti abatsire nenhanho yega yega yeML lifecycle, kubva pakugadzirira data kuburikidza nekudzidziswa kwemuenzaniso.
Kusiyana nekugadzirwa kwesoftware yechinyakare, kana zvikwata zvichisarudza chishandiso chimwe padanho rega rega, neML iwe unowanzo kuda kuongorora chishandiso chiripo (semuenzaniso, algorithm) kuona kana ichivandudza mhedzisiro.
Nekuda kweizvozvo, vanogadzira ML vanofanirwa kushandisa uye kugadzira mazana emaraibhurari.
Michina yekudzidza algorithms ine zviuru zvezvimiro zvinogoneka, uye zvakaoma kuona kuti ndeapi paramita, kodhi, uye data zvakapinda mukuyedza kwega kwega kugadzira modhi, ungave uchishanda wega kana muchikwata.
Pasina kunyatsotarisisa, zvikwata zvinowanzonetsekana kuti kodhi imwechete ishande zvakare. Kunyangwe iwe uri sainzi wedata uchiendesa kodhi yako yekudzidzira kune injinjini yekushandisa kugadzira, kana kuti uri kudzokera kubasa rako rekare kuti uongorore dambudziko, kudzoka nhanho dzeML workflow kwakakosha.
Kufambisa modhi kune kugadzira kunogona kuve kwakaoma nekuda kweakawanda nzira dzekutumira uye nharaunda dzinofanirwa kushandiswa (semuenzaniso, REST kushumira, batch inference, kana nharembozha). Iko hakuna nzira yakajairika yekufambisa mamodheru kubva kune chero raibhurari kuenda kune chero eaya maturusi, uye nekudaro kutsva kwega kwega kunounza njodzi.
Nekuda kwenyaya idzi, zviri pachena kuti ML budiriro inofanirwa kuvandudza zvakanyanya kuti ive yakagadzikana, inofanotaurwa, uye inoshandiswa zvakanyanya sechinyakare software kuvandudza.
ML Matambudziko
- Kune akawanda maturusi akasiyana. Mazana emhinduro dzesoftware anowanikwa kubatsira nenhanho yega yega yemuchina wekudzidza hupenyu, kubva pakugadzirira data kusvika pakudzidziswa kwemuenzaniso. Uyezve, kusiyana nechinyakare kusimudzira software, kana zvikwata zvichisarudza chishandiso chimwe padanho rega rega, mukudzidza muchina (ML), iwe unowanzo kuda kuongorora chishandiso chese chiripo (semuenzaniso, algorithm) kuona kana ichivandudza mhedzisiro. Nekuda kweizvozvo, vanogadzira ML vanofanirwa kushandisa uye kugadzira mazana emaraibhurari.
- Zvakaoma kuchengeta ongororo. Michina yekudzidza algorithms ine zviuru zvezvimiro zvinogoneka, uye zvakaoma kuona kuti ndeapi paramita, kodhi, uye data zvakapinda mukuyedza kwega kwega kugadzira modhi, ungave uchishanda wega kana muchikwata.
- Zvakaoma kushandisa muchina kudzidza. Kufambisa modhi kune kugadzira kunogona kuve kwakaoma nekuda kweakawanda nzira dzekutumira uye nharaunda dzinofanirwa kushandiswa (semuenzaniso, REST kushumira, batch inference, kana nharembozha). Iko hakuna nzira yakajairika yekufambisa mamodheru kubva kune chero raibhurari kune chero yezvishandiso izvi. Nokudaro, kushandiswa kutsva kwega kwega kunounza njodzi.
Chii MLflow?
MLflow ipuratifomu yakavhurika-sosi yemuchina wekudzidza hupenyu kutenderera. Iyo yakavakirwa pane yakavhurika interface pfungwa, ichikurudzira akawanda akakosha abstractions ayo anobvumira zvazvino zvivakwa uye muchina kudzidza algorithms kuti ibatanidzwe nyore nehurongwa.
Izvi zvinoreva kuti kana iwe uri mugadziri anoda kushandisa MLflow asi ari kushandisa isina kutsigirwa dhizaini, yakavhurika interface dhizaini inoita kuti zvive nyore kubatanidza iyo chimiro uye kutanga kushanda nepuratifomu. Mukuita, izvi zvinoreva kuti MLflow inoitirwa kushanda nechero machine learning raibhurari kana mutauro.
Uyezve, MLflow inosimudzira kudzokorora, zvinoreva kuti kodhi imwe chete yekudzidzira kana kugadzira muchina wekudzidza inoitirwa kuti imhanye nemhedzisiro yakafanana yakazvimirira kubva kune nharaunda, ingave iri mugore, panzvimbo yenzvimbo yekushandira, kana mubhuku rekunyorera.
Chekupedzisira, MLflow yakavakirwa scalability, saka inogona kushandiswa neboka diki revasainzi vedata pamwe nekambani hombe ine mazana evadzidzisi vemuchina.
MLflow inoenderana nechero raibhurari yekudzidza muchina, algorithm, chishandiso chekutumira, kana mutauro. Iyo ine zvakare zvinotevera zvakanakira:
- Yakagadzirirwa kushanda nechero cloud service.
- Chikero kune yakakura data neApache Spark.
- MLflow inoenderana neyakavhurika-sosi yekudzidza masisitimu, kusanganisira Apache Spark, TensorFlow, uye SciKit-Dzidza.
Kana iwe uchitova nekodhi, MLflow inogona kushandiswa nayo. Iwe unogona kugovera yako chimiro uye modhi pakati pemabhizinesi nekuti ndizvo pachena-mabviro.
MLflow Zvikamu: Vanoshanda sei?
MLflow inzvimbo yemahara uye yakavhurika-sosi yekutonga iyo ML lifecycle, iyo inosanganisira kuyedza, kuberekazve, kutumira, uye imwe modhi registry. Parizvino, MLflow ine zvikamu zvina:
1. MLflow Tracking
Ini ndichatanga neMLflow Tracking. MLflow inotsigira kuunganidzwa kweakasiyana akakosha ma concepts akabatana nepakati pekudzidzisa metadata yekutevera repository. Pfungwa yekutanga muunganidzwa weakakosha hyperparameters kana mapfumo ekumisikidza anopesvedzera kuita kwemuenzaniso. Kushandisa MLflow's APIs uye yepakati yekutevera sevhisi inogona kuchengetedza zvese izvi.
Vashandisi vanogona zvakare kurekodha data rekuita kuti vawane nzwisiso mubudiriro yemamodhi avo ekudzidza muchina. Uyezve, yekudzokorora, MLflow inobvumira vashandisi kunyora iyo chaiyo sosi kodhi yakashandiswa kugadzira modhi pamwe neshanduro yayo nekubatanidza zvakasimba neGit kusungira modhi yega yega kune chaiyo kuzvipira hashi.
MLflow inogona kushandiswa kurodha zviwanikwa, ari chero mafaera akasarudzika anosanganisira kudzidziswa, bvunzo data, uye modhi pachayo kuti iberekezve.
Izvi zvinoreva kuti kana ndiri mugadziri achangobva kudzidzisa modhi, ndinogona kuramba ndichienda kune yepakati yekutevera sevhisi, uye mumwe wevandinoshanda navo anogona kuiisa gare gare uye kana kuenderera mberi nekudzidzisa nekuyedza kana kugadzira iyo modhi kuti isangane nechinodiwa. .
Paunenge uchiita kodhi yako yekudzidza muchina uye wozoona zvabuda, kuteedzera iAPI iyo inokutendera iwe kuti utore paramita, kodhi shanduro, metrics, uye mafaera ekubuda. Zvakanyorwa muPython, R, uye Java, pakati pemimwe mitauro. Inowanikwawo seREST API, inogona kushandiswa kuvaka maapplication pamusoro payo.
Key Features
- Vazhinji vanogadzira vanoshandisa MLflow paPC yavo yemuno, uko iyo yekumashure uye yekuchengetera artifact inogovera dhairekitori pane disc.
- Vazhinji vashandisi vanoshandisawo SQLite, SQLAlchemy-inoenderana dhatabhesi, kumhanya MLflow pamaPC avo emunharaunda.
- MLflow zvakare inotsigira akaparadzirwa zvivakwa. Iyo yekutevera server, backend chitoro, uye artifact chitoro zvese zvakagarwa pamaseva akasiyana mune izvi.
- Kana kumhanya kwakatangwa neMLflow Project, iyo git commit hash yakashandiswa. Iyo MLflow Python, R, Java, uye REST APIs inogona kushandiswa kunyora data kuti iite.
Kuti uwane rumwe ruzivo, unogona kutarisa kune mukuru mapepa.
2. MLFlow Projects
Mushure mekunge tapfuura nepakati pezvikamu zvekutevera, ndinoda kutaura nezve MLflow mapurojekiti, ayo ari anodzokororwa ekurongedza chimiro chemuenzaniso wekudzidzisa zvikamu zvisinei nemamiriro ekuita.
Mabhizinesi anoshandisa huwandu hwakakura hwemakina ekudzidzira tekinoroji, asi ivo zvakare vanoshandisa aya maturusi ekudzidzisa mune akasiyana seti yemamiriro. Semuenzaniso, vanogona kunge vachiita kodhi yavo yekudzidzira pagore, paPC yemuno, kana mubhuku rekunyorera.
Izvi zvinotungamira kune dambudziko rekuti kudzidza kwemuchina kwakaoma kutevedzera. Kazhinji, iyo yakafanana kodhi kodhi haiite kana kuburitsa mhedzisiro yakafanana munzvimbo mbiri dzakasiyana.
Mhinduro yakapihwa neMLflow ndeyekuzvimiririra yekudzidzisa kodhi purojekiti tsananguro iyo inosanganisira ese emuchina wekudzidzira kodhi kodhi, pamwe neayo vhezheni raibhurari zvinoenderana, marongero, uye kudzidziswa uye bvunzo data.
MLflow inova nechokwadi chekuberekana mukati memaitiro ekuita nekutsanangura zvakajeka seti yese yezvinodiwa zvemuchina wekudzidzira maitiro. Inoita izvi nekuisa ese emaraibhurari aya uye kuzadzisa iyo yakafanana sisitimu iyo kodhi iri kushanda mairi.
Iyo MLflow chirongwa hachisi chimwe chinhu kunze kwedhairekitori. Iro dhairekitori rinosanganisira kodhi yekudzidziswa, iyo raibhurari yekutsamira tsananguro, uye imwe dhata inodiwa nechikamu chekudzidzisa, pamwe neiyi sarudzo yekumisikidza faira.
Izvi zvinodikanwa zveraibhurari zvinogona kutsanangurwa nenzira dzakasiyana siyana. Vashandisi vanogona, semuenzaniso, kupa YAML-yakagadziridzwa anaconda nharaunda yakatarwa kunyora yavo yekudzidziswa kodhi raibhurari zvinodiwa. MLflow ichaita kodhi yekudzidzisa mukati memudziyo. Mumamiriro ezvinhu akadaro, vanogona zvakare kusanganisira mudziyo weDocker.
Pakupedzisira, MLflow ine yekuraira-mutsara interface (CLI) yekumhanyisa mapurojekiti aya, pamwe nePython, uye Java APIs. Aya mapurojekiti anogona kuitiswa pamushandisi wemuno sisitimu pamwe neakasiyana siyana ari kure senge Databricks basa scheduler uye Kubernetes. MLflow mapurojekiti anobvumidza iwe kurongedza data sainzi kodhi nenzira inodzokororwa uye inogona kushandiswazve, zvakanyanya zvichienderana nezviyero.
Chikamu chepurojekiti chinosanganisira API pamwe chete nemirairo-mutsara zvishandiso zvekutarisira mapurojekiti. Aya masimba anovimbisa kuti mapurojekiti anogona kusungwa pamwe chete kuti agadzire maitiro ekudzidza muchina.
Key Features
- MLflow inotsigira nharaunda dzeprojekiti, kusanganisira Docker mudziyo nharaunda, Conda nharaunda, uye system nharaunda.
- Chero Git repository kana dhairekitori remunharaunda rinogona kutorwa seMLflow chirongwa; by default; unogona kushandisa chero shell kana Python script mudhairekitori senzvimbo yekupinda purojekiti.
- Zvisiri-Python zvinotsamira, zvakaita semaraibhurari eJava, zvinogona kutorwa uchishandisa Docker midziyo.
- Iwe unogona kuwana hukuru hwekutonga pamusoro peMLflow Project nekuwedzera faira reprojekiti kumudzi weprojekiti dhairekitori, iri faira remavara muYML syntax.
Kuti uwane rumwe ruzivo, unogona kutarisa kune mukuru mapepa.
3. MLflow Models
Zvino, ini ndoda kukurukura MLflow modhi, yakajairika-chinangwa modhi fomati inotsigira akasiyana siyana ekugadzira mamiriro. Chikonzero cheMLflow modhi ikozvino chakangofanana neicho chezvirongwa.
Zvakare, tinoona kuti mamodheru anogona kugadzirwa pachishandiswa maturusi akasiyana-siyana, asi anogonawo kugadzirwa kana kuiswa munzvimbo dzakawanda, kusiyana nenzvimbo dzekudzidzira.
Aya marongero anosanganisira maturusi e-chaiyo-nguva yekushandira, seKubernetes kana Amazon SageMaker, pamwe nekushambadzira uye batch scoring, seSpark. Uyezve, mamwe mabhizinesi anogona kusarudza kuendesa mamodheru se RESTful web sevhisi inomhanya pane yakafanogadzirirwa gore.
Iyo MLflow modhi, senge chirongwa, idhairekitori chimiro. Inosanganisira faira yekumisikidza uye, panguva ino, serialized modhi artifact pane yekudzidzisa kodhi. Inosanganisirawo seti iyi yekutsamira kwekudzokorora sepurojekiti. Panguva ino, isu tichatarisa muyedzo yekutsamira mumamiriro ezvinhu enzvimbo yeConda.
Pamusoro pezvo, MLflow inosanganisira maturusi ekugadzira emhando yekuenzanisa modhi muMLflow fomati kubva kune dzakasiyana siyana dzakakurumbira masisitimu. Chekupedzisira, MLflow inowedzera deploys, APIs yekugadzira uye kubatanidza chero MLflow modhi kune akasiyana masevhisi, uye aya maAPIs anowanikwa muPython, Java, R, uye CLI fomati.
Mamodheru chikamu chine chimiro chakajairwa chekurongedza mamodheru anogona kushandiswa uye kunzwisiswa neakadzika maturusi senge inferencing maseva kana iyo. data zvidhinha batch inferencing chikuva. Ichi chikamu chinochengetedza maawa ebespoke kodhi kana uchirongedza modhi yekugadzira.
Iyo MLflow Model chiyero chekurongedza modhi yekudzidza muchina mumhando dzakasiyana dzinozivikanwa se "flavours." MLflow inopa akawanda maturusi ekukubatsira mukuendesa akasiyana marudzi emamodheru. Imwe neimwe MLflow Model inochengeterwa sedhairekitori rine mafaera asina tsananguro pamwe neML modhi descriptor faira rine runyoro rwezvinonhuwirira maanogona kushandiswa.
Key Features
- Yese maturusi eMLflow akavakirwa-mukati ekutumira anopa akawanda "standard" flavour, senge "Python function" flavour inotsanangura maitiro ekumhanyisa modhi sePython basa.
- Imwe neimwe MLflow Model ine dhairekitori rine mafaera eanopokana, pamwe neML modhi faira pamudzi wedhairekitori inotsanangura akawanda emhando inonaka.
- Paunenge uchichengeta modhi, MLflow inokutendera kuti utaure yeConda nharaunda paramende ine zvinotsamira zvemuenzaniso. Kana pasina nharaunda yeConda yakatsanangurwa, nharaunda yakasarudzika yakavakirwa pamhando yemhando inovakwa. Mushure meizvozvo, nharaunda yeConda inochengetwa mu conda.yaml.
Kuti uwane rumwe ruzivo, unogona kutarisa kune mukuru mapepa.
4. MLflow Model Registry
Modhi registry inzvimbo yekudzidza muchina (ML) modhi. Model Registry inoumbwa neAPIs uye webhu-based application iyo inoshandiswa kuchengetedza mamodheru muzvikamu zvakasiyana sechikwata. Model Lineage, Model Versioning, Easy Stage Transition, uye Annotation angori mashoma emano anowanikwa muModel Registry.
Modhi registry, kuwedzera kune iwo mamodheru pachawo, ine ruzivo (metadata) nezve data uye mabasa ekudzidzisa anoshandiswa kugadzira modhi. Izvo zvakakosha kuti utarise izvi zvinodikanwa zvekupinza kugadzira mutsara weML modhi. Panyaya iyi, registry yemodhi inoshanda zvakafanana kune yakajairwa software's shanduro control masisitimu (semuenzaniso, Git, SVN) uye artifact repositories (semuenzaniso, Artifactory, PyPI).
Iyo Model Registry dhizaini inobvumira masayendisiti edatha uye mainjiniya ekudzidza muchina kushambadza, kuyedza, kutarisa, kutonga, uye kugovera modhi yavo yekushandira pamwe nezvimwe zvikwata. Chaizvoizvo, registry yemhando inoshandiswa kana wapedza chikamu chako chekuyedza uye wagadzirira kugovera zvawawana nechikwata uye vanobatanidzwa.
Iyo MLflow Model Registry inopa API uye mushandisi interface yekutonga mamodheru ako uye hupenyu hwavo kubva panzvimbo yepakati. Modhi mutsara, modhi shanduro, zvirevo, uye nhanho shanduko zvese zviripo kuburikidza neregistry.
MuMLflow, modhi yakanyoreswa ndiyo ine zita rakasiyana uye metadata, mhando dzemhando, nhanho dzekuchinja, uye mutsara wemuenzaniso. Imwe kana akawanda emhando shanduro inogona kuwanikwa mune yakanyoreswa modhi. Muenzaniso mutsva unoonekwa sevhezheni 1 kana yakanyoreswa murejista. Iyo inotevera vhezheni inowedzerwa kune chero mhando nyowani ine zita rimwechete.
Iwe unogona kupa nhanho imwe kune chero mhando yemhando chero nguva. Nekudaro, nhanho dzinofanirwa kupihwa pasi peMLflow zvikamu zvakatsanangurwa zviri pamutemo, senge staging, kugadzira, uye kuchengetwa. Shanduro yemuenzaniso inogona kuchinjwa kubva pane imwe nhanho kuenda kune imwe.
MLflow inobvumidza iwe kushandisa makidown kuratidza ese epamusoro-level modhi uye yega yega vhezheni. Iwe unogona kusanganisira tsananguro pamwe nerumwe ruzivo rwakakodzera, senge algorithm tsananguro, nzira, uye dhatabhesi rinoshandiswa.
Key Features
- Kuti uwane iyo modhi registry kuburikidza neUI kana API paunenge uchitambira yako MLflow server, unofanirwa kushandisa dhatabhesi-yakatsigirwa backend chitoro.
- Model Registry inogonawo kuwanikwa kuburikidza neMLflow modhi flavour kana iyo MLflow Client Tracking API interface. Unogona, semuenzaniso, kunyoresa modhi panguva yeMLflow kuyedza kana mushure mekuedza kwako kwese.
- Havasi vese vanozotanga kudzidzisa mamodheru avo vachishandisa MLflow. Nekuda kweizvozvo, unogona kunge uine mamwe mamodheru akadzidziswa usati washandisa MLflow. Pane kudzidzisazve mamodheru, unongoda kunyoresa ako akachengetwa mamodheru neModel Registry.
Kuti uwane rumwe ruzivo, unogona kutarisa kune mukuru mapepa.
mhedziso
MLflow yakanaka uye inogara ichikura ML lifecycle chishandiso. Iwe unogona kuishandisa pamwe chete nemidziyo yako yazvino uye mapuratifomu.
Inotsigira mitauro yakawanda yekuronga, kusanganisira Python, Java, uye R. Unogona zvakare kukurumidza kuronda, kuchengetedza, uye kuenzanisa mhando dzemhando dzakasiyana nekuda kwekugadzirwa kwayo kwemushandisi.
Edza MLflow uye tizivise ruzivo rwako!
Leave a Reply