Magawo angapo apadziko lonse lapansi ayamba kuyika ndalama zambiri pakuphunzira makina (ML).
Mitundu ya ML imatha kukhazikitsidwa ndikuyendetsedwa ndi magulu a akatswiri, koma chopinga chachikulu ndikusamutsa chidziwitso chomwe mwapeza kupita kuchitsanzo china kuti njira ziwonjezeke.
Kuwongolera ndikuwongolera njira zomwe zikukhudzidwa ndi kasamalidwe ka moyo wachitsanzo, njira za MLOps zikugwiritsidwa ntchito kwambiri ndi magulu omwe amapanga makina ophunzirira makina.
Pitirizani kuwerenga kuti mudziwe zambiri za zida zabwino kwambiri za MLOps ndi nsanja zomwe zilipo masiku ano komanso momwe angapangire kuphunzira pamakina kukhala kosavuta kuchokera pachida, wopanga mapulogalamu, ndi njira zamachitidwe.
Kodi MLOps ndi chiyani?
Njira yopangira mfundo, zikhalidwe, ndi machitidwe abwino amitundu yophunzirira pamakina imadziwika kuti "machine learning operations," kapena "MLOps."
MLOps ikufuna kutsimikizira moyo wonse wa chitukuko cha ML - kuchokera pa kubadwa mpaka kutumizidwa - imalembedwa bwino ndikuyendetsedwa kuti pakhale zotsatira zabwino kwambiri m'malo moyika nthawi yambiri ndi zothandizira popanda njira.
Cholinga cha MLOps ndikukhazikitsa njira zabwino kwambiri zomwe zimapangitsa kukula kwa kuphunzira pamakina kwa ogwiritsa ntchito ndi opanga ma ML, komanso kupititsa patsogolo mtundu ndi chitetezo chamitundu ya ML.
Ena amatcha MLOps ngati "DevOps yophunzirira makina" popeza imagwiritsa ntchito bwino mfundo za DevOps pagawo laukadaulo laukadaulo.
Iyi ndi njira yothandiza yoganizira za MLOps chifukwa, monga DevOps, imatsindika kugawana nzeru, mgwirizano, ndi machitidwe abwino pakati pa magulu ndi zida.
MLOps imapatsa opanga, asayansi a data, ndi magulu ogwirira ntchito ndi chimango chogwirizira, motero, kupanga mitundu yamphamvu kwambiri ya ML.
Chifukwa Chiyani Mukugwiritsa Ntchito Zida za MLOps?
Zida za MLOps zimatha kugwira ntchito zosiyanasiyana kwa gulu la ML, komabe, nthawi zambiri zimagawidwa m'magulu awiri: kasamalidwe ka nsanja ndi kasamalidwe kagawo kakang'ono.
Ngakhale kuti zinthu zina za MLOps zimayang'ana pa ntchito imodzi yokha, monga deta kapena metadata management, zida zina zimagwiritsa ntchito njira zowonjezera zonse ndikupereka nsanja ya MLOps kulamulira mbali zingapo za moyo wa ML.
Yang'anani mayankho a MLOps omwe amathandizira gulu lanu kuyang'anira madera otukuka a ML, kaya mukuyang'ana katswiri kapena chida chokulirapo:
- Kusamalira deta
- Kupanga ndi kutsanzira
- Kuwongolera ma projekiti ndi malo antchito
- Kutumiza kwachitsanzo cha ML ndikusamalira mosalekeza
- Kuwongolera kwa Lifecycle kuyambira koyambira mpaka kumapeto, komwe kumaperekedwa ndi nsanja za MLOps.
Zida za MLOps
1. MLFlow
Moyo wophunzirira makina umayendetsedwa ndi nsanja yotseguka ya MLflow ndipo imaphatikizapo kulembetsa kwachitsanzo chapakati, kutumiza, ndi kuyesa.
MLflow itha kugwiritsidwa ntchito ndi gulu lililonse lakukula, payekhapayekha komanso palimodzi. Malaibulale alibe mphamvu pa chida.
Chiyankhulo chilichonse cha pulogalamu ndi laibulale yophunzirira makina imatha kuzigwiritsa ntchito.
Kuti zikhale zosavuta kuphunzitsa, kutumiza, ndi kuyang'anira makina ophunzirira makina, MLFlow imagwirizana ndi njira zingapo zophunzirira makina, kuphatikizapo TensorFlow ndi Pytorch.
Kuphatikiza apo, MLflow imapereka ma API osavuta kugwiritsa ntchito omwe amatha kuphatikizidwa mumapulogalamu kapena malaibulale omwe alipo.
MLflow ili ndi zinthu zinayi zofunika zomwe zimathandizira kutsata ndikukonzekera kuyesa:
- MLflow Tracking - API ndi UI yowerengera makina owerengera mitengo, mitundu, ma metrics, ndi zinthu zakale komanso kuwonetsa ndi kusiyanitsa zotsatira.
- MLflow Projects - makina opangira makina ophunzirira m'njira yogwiritsidwanso ntchito, yopangikanso kuti asamutsire kupanga kapena kugawana ndi asayansi ena a data.
- MLflow Models - kusunga ndi kutumiza zitsanzo kumitundu ingapo yotumikira ndi ma inference system kuchokera kuma library osiyanasiyana a ML
- MLflow Model Registry - sitolo yachitsanzo yapakati yomwe imathandizira kasamalidwe kaumoyo wamtundu wa MLflow nthawi yonse ya moyo wake, kuphatikiza masinthidwe achitsanzo, kusintha kwa siteji, ndi ndemanga.
2. KubeFlow
Bokosi la zida la ML la Kubernetes limatchedwa Kubeflow. Kuyika ndi kuyang'anira zotengera za Docker, zothandizira pakukonza makina ophunzirira makina.
Mwa kuphweka kayimbidwe kake ndi kutumizidwa kwa makina ophunzirira makina, kumalimbikitsa kuchulukira kwa mitundu yophunzirira makina.
Ndi pulojekiti yotseguka yomwe imaphatikizapo gulu losankhidwa bwino la zida zowonjezera ndi maziko ogwirizana ndi zosowa zosiyanasiyana za ML.
Ntchito zophunzitsira zazitali za ML, kuyesa pamanja, kubwereza, ndi zovuta za DevOps zitha kuyendetsedwa ndi Kubeflow Pipelines.
Kwa magawo angapo ophunzirira makina, kuphatikiza kuphunzitsa, kukonza mapaipi, ndi kukonza Zolemba za Jupyter, Kubeflow imapereka ntchito zapadera komanso kuphatikiza.
Zimapangitsa kuti zikhale zosavuta kuyang'anira ndikutsata moyo wa ntchito zanu za AI komanso kutumiza zitsanzo zamakina ophunzirira (ML) ndi mapaipi a data kumagulu a Kubernetes.
Imapereka:
- Mabuku ogwiritsira ntchito SDK kuti agwirizane ndi dongosolo
- mawonekedwe ogwiritsira ntchito (UI) owongolera ndi kuyang'anira kuthamanga, ntchito, ndi kuyesa
- Kupanga mwachangu mayankho akumapeto-kumapeto popanda kumanganso nthawi iliyonse, ndikugwiritsanso ntchito zida ndi mapaipi.
- Monga gawo lofunikira la Kubeflow kapena ngati kukhazikitsa koyimirira, Kubeflow Pipelines amaperekedwa.
3. Data Version Control
Njira yotsegulira yotsegulira mapulojekiti ophunzirira makina imatchedwa DVC, kapena Data Version Control.
Chilankhulo chilichonse chomwe mungasankhe, ndi chida choyesera chomwe chimathandizira kutanthauzira kwa mapaipi.
DVC imagwiritsa ntchito ma code, kusintha kwa data, ndi kuchulukirachulukira kuti ikuthandizeni kusunga nthawi mukazindikira vuto ndi mtundu wakale wa ML yanu.
Kuphatikiza apo, mutha kugwiritsa ntchito mapaipi a DVC kuti muphunzitse chitsanzo chanu ndikuchigawa kwa mamembala a gulu lanu. Kukonzekera kwakukulu kwa deta ndi kumasulira kungathe kuthandizidwa ndi DVC, ndipo deta ikhoza kusungidwa m'njira yosavuta.
Ngakhale zimaphatikizanso zinthu zina (zochepa) zoyeserera zoyeserera, zimayang'ana kwambiri pakusintha kwamapaipi ndi kasamalidwe.
Imapereka:
- Ndi yosungirako agnostic, kotero ndizotheka kugwiritsa ntchito mitundu yosiyanasiyana yosungirako.
- Imaperekanso ziwerengero zotsata.
- njira zopangira zolumikizira magawo a ML kukhala DAG ndikuyendetsa mapaipi onse kuyambira koyambira mpaka kumapeto.
- Kukula kwamtundu uliwonse wa ML kumatha kutsatiridwa pogwiritsa ntchito ma code ake onse komanso kupezeka kwa data.
- Kuberekanso mwa kusunga mokhulupirika kasinthidwe koyambirira, zolowetsamo, ndi code ya pulogalamu yoyesera.
4. Pachyderm
Pachyderm ndi pulogalamu yowongolera makina ophunzirira makina ndi sayansi ya data, yofanana ndi DVC.
Kuphatikiza apo, chifukwa idapangidwa pogwiritsa ntchito Docker ndi Kubernetes, imatha kuchita ndikuyika mapulogalamu a Machine Learning papulatifomu iliyonse yamtambo.
Pachyderm imatsimikizira kuti chidutswa chilichonse cha data chomwe chimagwiritsidwa ntchito ngati makina ophunzirira makina chikhoza kutsatiridwa ndikusinthidwa.
Amagwiritsidwa ntchito popanga, kugawa, kuyang'anira, ndi kuyang'anira mitundu yophunzirira makina. Registry yachitsanzo, kasamalidwe kachitsanzo, ndi bokosi la zida za CLI zonse zikuphatikizidwa.
Madivelopa amatha kupanga okha ndikukulitsa makina awo ophunzirira makina pogwiritsa ntchito maziko a data a Pachyderm, omwe amatsimikiziranso kubwereza.
Imathandizira miyezo yokhazikika yaulamuliro wa data, imachepetsa mtengo wokonza ndi kusunga, ndikuthandizira mabizinesi kubweretsa njira zawo za sayansi ya data kuti agulitse mwachangu.
5. Polyaxon
Pogwiritsa ntchito nsanja ya Polyaxon, mapulojekiti ophunzirira makina ndi ntchito zophunzirira mozama zitha kutsatiridwa ndikuwongolera moyo wawo wonse.
Polyaxon imatha kuchititsa ndi kuyang'anira chida, ndipo chitha kuyikidwa mu data center kapena cloud provider. Monga Torch, Tensorflow, ndi MXNet, zomwe zimathandizira njira zonse zodziwika bwino zophunzirira mwakuya.
Zikafika pakuyimba, Polyaxon imakuthandizani kuti mupindule kwambiri ndi gulu lanu pokonza ntchito ndi mayeso kudzera pa CLI, dashboard, SDKs, kapena REST API.
Imapereka:
- Mutha kugwiritsa ntchito pulogalamu yotsegulira pompano, koma imaphatikizanso zosankha zamakampani.
- Ngakhale imakhudza moyo wathunthu, kuphatikiza kuyimba kwa orchestration, imatha kuchita zambiri.
- Ndi zolemba zaukadaulo, malangizo oyambira, zida zophunzirira, zolemba, maphunziro, zosintha, ndi zina zambiri, ndi nsanja yolembedwa bwino kwambiri.
- Ndi dashboard yazidziwitso zoyeserera, ndizotheka kuyang'anitsitsa, kuyang'anira, ndikuwunika kuyesa kulikonse.
6. nyenyezi
Comet ndi nsanja yophunzirira makina a meta omwe amatsata, kusiyanitsa, kufotokoza, ndikusintha zoyeserera ndi mitundu.
Zoyeserera zanu zonse zitha kuwoneka ndikufananizidwa pamalo amodzi.
Imagwira ntchito iliyonse yophunzirira makina, kulikonse komwe ma code anu amachitidwa, komanso ndi laibulale iliyonse yophunzirira makina.
Comet ndiyoyenera magulu, anthu, mabungwe amaphunziro, mabizinesi, ndi wina aliyense amene akufuna kuwona mwachangu zoyeserera, kuwongolera ntchito, ndi kuyesa.
Asayansi ndi magulu a data amatha kutsata, kumveketsa bwino, kukonza, ndi kufanizira zoyeserera ndi zitsanzo pogwiritsa ntchito nsanja yophunzirira makina yodzipangira okha komanso yozikidwa pamtambo ya Comet.
Imapereka:
- Pali kuthekera kochuluka kuti mamembala azitha kugawana ntchito.
- Ili ndi zophatikiza zingapo zomwe zimapangitsa kuti zikhale zosavuta kuzilumikiza ndi matekinoloje ena
- Imagwira ntchito bwino ndi malaibulale amakono a ML
- Imasamalira kasamalidwe ka ogwiritsa ntchito
- Kuyerekeza zoyeserera kumathandizidwa, kuphatikiza kufananiza kwa ma code, hyperparameters, metrics, kulosera, kudalira, ndi ma metrics adongosolo.
- Amapereka ma module apadera a masomphenya, zomvera, zolemba, ndi tabular zomwe zimakupatsani mwayi wowonera zitsanzo.
7. Optuna
Optuna ndi kachitidwe ka autonomous hyperparameter kukhathamiritsa komwe kungagwiritsidwe ntchito pophunzira pamakina ndi kuphunzira mozama komanso magawo ena.
Ili ndi ma aligorivimu osiyanasiyana otsogola omwe mungasankhe (kapena kulumikiza), imapangitsa kuti ikhale yosavuta kugawa maphunziro pamakompyuta ambiri, ndipo imapereka mawonekedwe owoneka bwino.
Malaibulale ophunzirira makina otchuka monga PyTorch, TensorFlow, Keras, FastAI, sci-kit-lern, LightGBM, ndi XGBoost onse amaphatikizidwa nayo.
Imakhala ndi ma aligorivimu otsogola omwe amathandizira makasitomala kupeza zotsatira mwachangu pochepetsa mwachangu zitsanzo zomwe sizikuwoneka bwino.
Pogwiritsa ntchito ma algorithms a Python, imangosaka ma hyperparameter abwino. Optuna imalimbikitsa kusaka kofananira kwa hyperparameter mu ulusi wambiri popanda kusintha nambala yoyambirira.
Imapereka:
- Imathandizira maphunziro ogawidwa pamagulu komanso pakompyuta imodzi (njira zambiri) (ma node ambiri)
- Imathandizira njira zingapo zochepetsera kuti zifulumizitse kulumikizana (ndikugwiritsa ntchito makompyuta ochepa)
- Ili ndi mawonekedwe osiyanasiyana amphamvu, monga chiwembu cha magawo, chiwembu cha contour, ndi ma coordinates ofanana.
8. Kedro
Kedro ndi pulogalamu yaulere ya Python yolembera khodi yomwe imatha kusinthidwa ndikusungidwa pama projekiti a sayansi ya data.
Zimabweretsa malingaliro kuchokera pamachitidwe abwino kwambiri muukadaulo wamapulogalamu mpaka pamakina ophunzirira makina. Python ndiye maziko a chida ichi chowongolera mayendedwe.
Kuti njira zanu za ML zikhale zosavuta komanso zolondola, mutha kupanga zosinthika, zosungika, komanso zoyendera.
Kedro imaphatikizanso mfundo zaumisiri wamapulogalamu monga kusinthasintha, kulekanitsa maudindo, ndikusintha malo ophunzirira makina.
Pamaziko a Cookiecutter Data Science, imapereka dongosolo lofanana, losinthika la projekiti.
Zolumikizira zingapo zosavuta zomwe zimagwiritsidwa ntchito kusungira ndi kuyika deta pamafayilo angapo ndi mafayilo amafayilo, zimayendetsedwa ndi kalozera wa data. Zimapangitsa mapulojekiti ophunzirira pamakina kukhala othandiza kwambiri ndikupangitsa kukhala kosavuta kupanga mapaipi a data.
Imapereka:
- Kedro imalola makina obalalitsidwa kapena osungulumwa okha.
- Mutha kusinthira kudalira pakati pa nambala ya Python ndi mawonekedwe akuyenda kwa ntchito pogwiritsa ntchito mapaipi.
- Pogwiritsa ntchito ma modular, reusable code, teknolojiyi imathandizira mgwirizano wamagulu pamagulu osiyanasiyana ndikuwongolera zokolola m'malo olembera.
- Cholinga chachikulu ndikuthana ndi zovuta zomwe zili m'mabuku a Jupyter, zolemba zamtundu umodzi, ndi glue-code polemba madongosolo asayansi a data.
9. BentoML
Kumanga makina omaliza a API amapangidwa kukhala kosavuta ndi BentoML.
Amapereka njira yokhazikika koma yofupikitsidwa kuti asunthire makina ophunzirira makina kuti apange.
Zimakuthandizani kuti muphatikize mitundu yophunzirira kuti mugwiritse ntchito popanga, kuwatanthauzira pogwiritsa ntchito dongosolo lililonse la ML. Kutumikira kwapaintaneti komanso API yapaintaneti imathandizidwa.
Seva yachitsanzo yogwira ntchito kwambiri komanso kusinthasintha kwa ntchito ndi mawonekedwe a BentoML.
Kuphatikiza apo, seva imapereka adaptive micro-batching. Njira yogwirizana yokonzekera ma model ndi kutsata njira zotumizira zimaperekedwa ndi UI dashboard.
Sipadzakhalanso nthawi yopumira ya seva chifukwa makina ogwiritsira ntchito ndi okhazikika ndipo kasinthidwe kake kamagwiritsidwanso ntchito. Ndi nsanja yosinthika yoperekera, kukonza, ndi kutumiza mitundu ya ML.
Imapereka:
- Ili ndi mawonekedwe osinthika omwe amatha kusintha.
- Imathandizira kutumizidwa pamapulatifomu angapo.
- Sizingagwire ntchito yopingasa makulitsidwe.
- Imathandizira mtundu umodzi wachitsanzo, kasamalidwe kachitsanzo, kuyika kwachitsanzo, komanso kugwiritsa ntchito bwino kwambiri.
10. Seldon, PA
Asayansi a data amatha kupanga, kutumiza, ndi kuyang'anira makina ophunzirira ndi zoyeserera pamlingo wa Kubernetes pogwiritsa ntchito maziko otseguka a Seldon Core.
TensorFlow, sci-kit-learn, Spark, R, Java, ndi H2O ndi ochepa chabe mwa zida zomwe zimathandizidwa ndi izo.
Imalumikizananso ndi Kubeflow ndi RedHat's OpenShift. Seldon core imasintha mitundu yophunzirira yamakina (mitundu ya ML) kapena zokutira zilankhulo (zilankhulo monga Python, Java, ndi zina zotero) kukhala REST/GRPC microservices.
Chimodzi mwazida zabwino kwambiri za MLOps zowongolera njira zophunzirira makina ndi ichi.
Ndiosavuta kuyikamo mitundu ya ML ndikuyesa kugwiritsiridwa ntchito ndi chitetezo pogwiritsa ntchito Seldon Core.
Imapereka:
- Kutumiza kwachitsanzo kumatha kupangidwa kukhala kosavuta ndi njira zingapo, monga kutumizidwa kwa canary.
- Kuti mumvetse chifukwa chake maulosi enieni anapangidwira, gwiritsani ntchito ofotokozera zitsanzo.
- Mavuto akabuka, yang'anani mitundu yopangira pogwiritsa ntchito makina ochenjeza.
Kutsiliza
MLOps ingathandize kupanga makina ophunzirira bwino. MLOps imatha kufulumizitsa kutumizidwa, kupanga kusonkhanitsa deta ndikuwongolera mosavuta, ndikuwongolera mgwirizano pakati pa mainjiniya ndi asayansi a data.
Kuti musankhe chida cha MLOps chomwe chikugwirizana ndi zosowa zanu, positiyi idasanthula mayankho 10 otchuka a MLOps, omwe ambiri ndi otseguka.
Siyani Mumakonda