Okuqukethwe[Fihla][Bonisa]
Idatha ikhona yonke indawo eduze nawe. Ngomqondo wangempela, kuthonya zonke izici zebhizinisi lakho. Kungase kuzwakale sengathi asikho isikhathi esanele sokuhlola imininingwane yokuthi isebenzela ibhizinisi lakho kahle kangakanani uma umatasatasa ngezinqumo zendlela yokuphatha idatha yakho.
Qaphela lokhu. Inhlangano yakho isebenzisa idatha amahora angama-24 ngosuku. Ngakho-ke ukuqonda ukuthi ivelaphi, yafika kanjani lapho, nokuthi ihamba kanjani enkampanini kubalulekile ukuze uqonde ukubaluleka kwayo.
Uhlu lwedatha lubaluleka kulesi simo. Kulula ukuqonda ukuthi idatha yakhiwa kanjani, yavelaphi, futhi iyaphi lapho singakwazi ukulandelela umsuka, ukufuduka, nezinguquko zedatha.
Kulokhu okuthunyelwe, sizobhekisisa i-Data Lineage, ukuthi isebenza kanjani, izimo zayo zokusebenzisa, amasu, nokunye okuningi.
Iyini i-Data Lineage?
Uhlu lwedatha lusebenza njengohlobo lwepasipoti yedijithali. I-akhawunti ebanzi kakhulu yohambo lwedatha, echaza zonke izitobhi zakhona, ukuchezuka, nokulungiswa kusuka kumsuka wayo kuya endaweni yayo ekugcineni.
INgokuyisisekelo, uhlu lwedatha luchaza umsuka, ukuguqulwa, nokusetshenziswa kocezu lwedatha kuwo wonke amasistimu nezinkundla. Isebenza njengethuluzi lomseshi ngokunikeza abasebenzisi ulwazi mayelana nendlela idatha ekhiqizwa ngayo, ukuthi isuka kuphi, nokuthi yasetshenziswa kanjani. Lolu lwazi lwenza abasebenzisi bakwazi ukubona nokuxazulula izinkinga ezingaba khona.
Uhlu lwedatha luyisisetshenziswa esiyigugu sezinkampani ezincike kudatha ukuze ziqhube imisebenzi yazo ngoba kuvumela abasebenzisi ukuphendula imibuzo ebalulekile efana nokuthi ubani, ini, nini, futhi kuphi.
Uhlu lwedatha, uma sikubeka kalula, umkhondo wedatha wokugcina oqinisekisa ukunemba kwedatha, ukuphelela, nokungaguquguquki kuyilapho unikeza umbono ocacile nomfishane wendlela egcwele yedatha.
Isebenza kanjani i-Data Lineage?
Umugqa wedatha imephu yomgwaqo esenza sikwazi ukulandela ucezu lwedatha ukusuka lapho luqala khona ukuya ekugcineni. Cabangela iphoyinti ledatha njengomhambi, futhi iphasiphothi yayo ibe wuhlu lwedatha yayo ukuze uqonde kangcono ukuthi isebenza kanjani.
Imithombo yedatha, ukuguqulwa kwedatha, ukugcinwa kwedatha, nokuphuma kwedatha kwenza izingxenye ezine eziyinhloko zepasipoti.
Amasistimu amaningi, izinhlelo zokusebenza, nezinkundla lapho idatha isuka khona zimelelwa imithombo yedatha, esebenza njengesiqalo sohambo lwedatha. Ukuguqulwa kwedatha yisigaba esilandelayo, futhi umugqa wedatha ushadi ukuqhubeka kwedatha ukusuka kule mithombo ukuya kuyo.
Ukuguqulwa kwedatha kubhekisela ekubunjweni, ekulungiseni nasekusetshenzisweni kwedatha ukuze kuhlangatshezwane nezidingo zabasebenzisi. Isebenza njengendawo yokuphumula phakathi nohambo lwedatha, iyilungiselela umlenze olandelayo.
Idatha ibe isigcinwa ngaphambi kokuya endaweni yayo yokugcina. Ingagcinwa kumaseva wamafu, kusizindalwazi, noma olunye uhlobo lwedivayisi yokugcina. Uhlu lwedatha lulandela umkhondo walapho idatha igcinwa khona, kanye nokuthi ivikelwa kanjani, yenziwe ikhophi yasenqolobaneni, futhi ibuyiselwe kanjani.
Isinyathelo sokugcina siwukuphuma kwedatha, okuyilapho idatha ithunyelwa khona ukuze isetshenziswe. Imibiko, i-infographics, nanoma yiluphi olunye uhlobo lomkhiqizo wedatha ingase isetshenziselwe ukwethula. Umugqa wedatha ulandela okukhiphayo futhi uqinisekisa ukuvumelana, ukunemba, nokuphelela kwedatha.
Uhlu lwedatha ngokuyisisekelo lusebenza ngokurekhoda isigaba ngasinye sohambo lwedatha, kusukela ekuqaleni kwayo kuya ekuphumeni kwayo, nokuqinisekisa ukuthi luhlala luthembekile, lungaguquguquki, futhi lulungisa yonke indlela. Uhlu lwedatha lusiza izinhlangano ukuthi zenze izinqumo ezifundile, zilungise izinkinga, futhi zithobele izibopho zomthetho ngokunikeza umbono ogcwele wokuba khona kwedatha.
Ukuze uqonde amafa edatha nokuthi ahamba kanjani emgqeni wedatha, imethadatha iyingxenye ebalulekile yenqubo yohlu lwedatha.
Ungabona ukuthi idatha iguqulwa futhi isetshenziswe kanjani ngaphakathi kwenhlangano usebenzisa amathuluzi omugqa wedatha, aphakamisa imethadatha ukunikeza ukuvezwa okubonakalayo kokugeleza kwedatha. Lokhu kuvumela abasebenzisi ukuthi bahlole amandla edatha ukubasiza benze izinqumo ezinolwazi olungcono.
Izinhlobo Zomugqa Wedatha
Kunezindlela ezintathu eziyisisekelo zomugqa wedatha: umugqa wedatha oya phambili, umugqa wedatha obuyela emuva, kanye ne-bi-directional data lineage.
Dlulisa Umugqa Wedatha
Njengomgwaqo wendlela eyodwa, umugqa wedatha oya phambili ufaka ukulandelela ucezu lwedatha ukusuka lapho luqala khona ukuya endaweni yalo yokugcina. Kusukela emthonjeni wedatha, ilandela idatha njengoba idlula ezinguqukweni ezimbalwa namasistimu okugcina ukuze ifinyelele okukhiphayo.
Ukuqonda ukucutshungulwa nokuguqulwa kwedatha kanye nanoma yiziphi izinkinga okungenzeka zivele endleleni kusiza ngokuba nohlu lwedatha lwalolu hlobo. Zonke izinyathelo ziholela kwesilandelayo; kufana nokulandela umkhondo wemvuthuluka yesinkwa.
Uhlu Lwedatha Esemuva
Uhlu lwedatha ebuyela emuva lufana nohambo lokuhlehla lapho silandelela okuphumayo kwedatha emuva emthonjeni wakho. Inqubo iqala endaweni yokugcina yedatha futhi ihlehlela emuva ngamasu ahlukahlukene okugcina nokuguqula ize ifinyelele emthonjeni wedatha.
Ukuhlonzwa komthombo wangempela wedatha, ukuqondwa kokuguqulwa kwayo, nokuqinisekiswa kokunemba nokuphelela kwayo konke kungenzeka ngosizo lwalolu hlobo lokulandelana kwedatha. Isebenza njengethuluzi lomseshi, esivumela ukuthi silandele indlela yedatha emuva.
I-Bi-directional Data Lineage
Umugqa wezindlela ezimbili, uhlu lwedatha olunezinhlangothi ezimbili luhlanganisa izinzuzo zomugqa wedatha oya phambili nangemuva. Inikeza umbono ophelele womzila wedatha ngokuyilandelela ukusuka emthonjeni wayo ukuya lapho iya khona kanye nokusuka kuleyo ndawo kuya lapho iqala khona.
Ukuze unqume umthombo wangempela wedatha, uqonde ukuthi yashintshwa kanjani, futhi uqinisekise ikhwalithi yayo, ukungaguquguquki, nokuphelela ngaso sonke isikhathi, kuyasiza ukulandelela uhlu lwedatha. Ngolwazi lwesikhathi sangempela ngendawo yayo nesimo, kufana nokuba nesilandeleli se-GPS sedatha.
Ukuqaliswa Komugqa Wedatha
Ukusebenzisa umugqa wedatha enhlanganweni ngokuvamile kubandakanya izigaba ezilandelayo.
Chaza imithombo yedatha
Amasistimu nezizindalwazi ezigcina idatha ofisa ukuyilandela kufanele zonke zikhonjwe. Ukuze wenze lokhu, kufanele uqale ukhombe imithombo yedatha eyahlukene, okuhlanganisa amafayela, ama-API, namasevisi wamafu.
Qoqa imethadatha
Isigaba esilandelayo ukuthola imininingwane mayelana nedatha, okuhlanganisa indawo yayo, ifomethi, kanye nokuhleleka kwayo. Ukuqonda izici zedatha nendlela esetshenziswa ngayo kwenziwa nokwenzeka yile methadatha.
Thola amaphutha edatha
Kulula ukuqonda ukuthi idatha ibuyekezwa futhi isetshenziswe kanjani ngaphakathi kwenhlangano uma ukugeleza kwedatha kumephuzwa kusukela emthonjeni wayo kuya endaweni eya kuyo, okuhlanganisa noma yiziphi izinguquko noma ukucutshungulwa okwenzeka endleleni.
Landelela ukufinyelela kwedatha
Ukugcina ukuphepha kwedatha nokuthobelana, landelela, futhi urekhode ukuthi ubani ofinyelela idatha.
Gcina futhi ubone ngeso lengqondo uhlu lozalo
Sebenzisa amathuluzi okubona ngeso lengqondo ukwethula uhlu ukuze uthole ukuqonda okulula nokuhlaziya. Gcina imethadatha eqoqiwe kanye nolwazi lokugeleza kwedatha endaweni yokugcina eyodwa.
Sebenzisa isixazululo esizenzakalelayo
Ungaqinisekisa ukuthi uhlu lwedatha luyaqoqwa futhi luqashwe nge-automation, okuzosiza futhi ukunciphisa amaphutha futhi kukhuphule umkhiqizo.
Buyekeza & Buyekeza
Yenza ukuthi amarekhodi ohlu alungile futhi akhona njalo, futhi uwabuyekeze ngokufanele.
Inqubo yokusebenzisa ingase idinge ukushintshwa noma yengezwe ezigabeni kuye ngezidingo eziyingqayizivele kanye nemikhawulo yenhlangano ngayinye.
I-Data Lineage Techniques
Uzalo Olususelwe kwiphethini
Ngale ndlela, uhlu lwenziwa ngaphandle kokuxhumana nohlelo olukhiqize noma oluguqule idatha. Ukuhlolwa kwemethadatha kwamathebula, amakholomu, nemibiko yebhizinisi konke kuyingxenye yakho. Ihlola uhlu ngokubheka amathrendi kusetshenziswa le methadatha.
Ngokwesibonelo, kungenzeka ukuthi ikholomu kumadathasethi amabili anegama elifanayo namanani edatha afanayo imele idatha efanayo ezigabeni ezihlukene zokuba khona kwayo. Ishadi lomugqa wedatha libe selisetshenziswa ukuxhuma lawo makholomu amabili.
Uzalo olususelwe kwiphethini lunenzuzo enkulu yokuzimela kwezobuchwepheshe ngoba luvele luhlole idatha, hhayi izindlela zokucubungula idatha. Noma ibuphi ubuchwepheshe besizindalwazi, okuhlanganisa i-Oracle, i-MySQL, ne-Spark, bungakwazi ukubusebenzisa ngendlela efanayo. I-drawback ukuthi le ndlela ayinembile ngaso sonke isikhathi.
Lapho i-logic yokucubungula idatha ifihliwe kukhodi yekhompuyutha futhi ingabonakali kalula kumethadatha efundeka umuntu, ingase ingabunaki ubudlelwano phakathi kwamasethi edatha.
Lineage by Data Tagging
Le ndlela isekelwe embonweni wokuthi injini yokuguqula imaka noma umaka idatha. Ilandelela umaki kusukela ekuqaleni kuye ekugcineni ukuze kutholwe uhlu lozalo. Le ndlela ingaphumelela kuphela uma unethuluzi lokuguqula elinokwethenjelwa elilawula konke ukudluliswa kwedatha futhi ujwayelene nesakhiwo sokumaka ithuluzi elisisebenzisayo.
Noma ngabe ithuluzi elinjalo belingaba khona, ayikho idatha edaliwe noma eshintshiwe ngaphandle kwayo engaba ngaphansi kohlu ngokumaka idatha. Kulinganiselwe kulokhu ukwenza uhlu lwedatha kumasistimu edatha avaliwe.
Uzalo Oluzimele
Amanye amabhizinisi anendawo yedatha ehlanganisa ukugcinwa kwemethadatha, ukucubungula ukucubungula, nokuphathwa kwedatha okuyinhloko (MDM). Lezi zilungiselelo ngokuvamile zihlanganisa a idatha echibini lapho yonke idatha igcinwa kuyo yonke impilo yayo.
Uzalo lungahlinzekwa ngokwemvelo ngalolu hlobo lwesistimu eziqukethwe ngokwalo ngaphandle kwesidingo sezinsiza ezengeziwe. Nokho, njengoba kunjalo nangendlela yokumaka idatha, uhlu lozalo ngeke lwazi noma yini eyenzekayo ngaphandle kwale ndawo elawulwayo.
I-Data Lineage by Parsing
Uhlobo lohlu oluyinkimbinkimbi kakhulu yilolo olufunda ingqondo yokucubungula idatha ngokuzenzakalela. Ukuze uthole ukulandelela okuphelele, ekupheleni ukuya ekupheleni, le ndlela ihlehlisa onjiniyela ingqondo yokuguqulwa kwedatha.
Njengoba lesi sixazululo kufanele siqonde konke izilimi zokuhlela namathuluzi asetshenziselwa ukuguqula nokuthutha idatha, ukuthunyelwa kwayo kuyinkimbinkimbi. Lokhu kungase kusebenzise ingqondo ye-extract-transform-load (ETL), i-SQL- ne-Java-based solutions, amafomethi wedatha amadala, izixazululo ezisekelwe ku-XML, nezinye izindlela.
Izimo Zokusetshenziswa Komugqa Wedatha
Ukumodela idatha
Izinkampani kufanele zisungule izakhiwo zedatha eziyisisekelo ezizisekelayo ukuze zibone ngeso lengqondo izinto eziningi zedatha nokuxhumana phakathi kwazo ngaphakathi kwenkampani. Lokhu kuxhumana kwenziwa imodeli kusetshenziswa ukulandelana kwedatha, okubonisa futhi ukuncika okuningi okukhona ku-ecosystem yedatha.
Njengoba idatha ishintsha ngokuhamba kwesikhathi, imithombo emisha yedatha ivela njalo, edinga ukuhlanganiswa kwedatha entsha, njll. Ngenxa yalokhu, amamodeli wedatha evamile wamafemu wokuphatha idatha yawo kufanele ngokufanayo ashintshe ukuze abonise indawo ezungezile.
Compliance
Uhlu lwedatha lunikeza indlela yokuthobela yokucubungula, ukuthuthukisa ukuphathwa kwengozi, nokuqinisekisa ukuthi idatha igcinwa futhi isingathwe ngokuvumelana nezinqubomgomo nemithetho yokuphatha idatha.
Ukuhlaziywa Komthelela
Imiphumela yezinguquko ezithile zebhizinisi, njenganoma yikuphi ukubika okwehlayo, kungabonwa kusetshenziswa amathuluzi omugqa wedatha. Uhlu lwedatha, isibonelo, lungasiza abaphathi ekunqumeni ukuthi mangaki amadeshibhodi ukushintsha kwegama okungathinta futhi, ngenxa yalokho, bangaki abantu abafinyelela lokho kubika.
Ukuthuthwa kwedatha
Izinhlangano zisebenzisa ukuthuthwa kwedatha ukuze ziqonde ukuthi idatha ikuphi nokuthi isinesikhathi esingakanani ikhona ngaphambi kokuyidlulisela kusistimu entsha yokugcina noma yenze isofthiwe entsha.
Uhlu lwedatha lusiza amaqembu ukuthi alungiselele ukuthuthukiswa kwesistimu noma ukufuduka ngokuwanikeza ukubuka konke kokuthi idatha ihambe kanjani kuyo yonke inhlangano. Lokhu kusheshisa ukudluliselwa endaweni entsha yesitoreji kukonke.
Ukwengeza, kunikeza amaqembu ithuba lokususa isistimu yedatha ngokufaka kungobo yomlando noma ngokususa idatha ephelelwe yisikhathi noma engenamsebenzi. Ngokwenza kanjalo, isistimu yedatha izosebenza kangcono lilonke futhi idinga ukuphathwa okuncane kwedatha.
Izinselelo Zokusebenzisa Uhlu Lwedatha
- Ukuphepha Kwedatha: Ukuvikeleka kwedatha yinto eyinhloko ekukhathazayo ngenkathi kwakhiwa uhlu lwedatha. Ukuze ulandele uhambo lwedatha ukusuka lapho iqala khona ukuya endaweni yayo yokugcina, ukufinyelela kudatha ebucayi kufanele kunikezwe, futhi le datha kufanele ivikelwe ekufinyeleleni okungagunyaziwe nasekuphuleni umthetho.
- Ukushoda Kokumiswa Okujwayelekile: Esinye sezithiyo eziyinhloko ekwamukeleni uhlu lwedatha ukuntuleka kwamazinga. Njengoba izinkundla eziningi, izinhlelo zokusebenza, nezinhlelo zisebenzisa izindlela ezihlukile zokulandelela nokurekhoda ukutholakala kwedatha, kungaba nzima ukuhlanganisa isithombe esihlangene sohambo lwedatha.
- Izithako Zedatha: Izisele zedatha enye inkinga ephakamayo ngenkathi kusetshenziswa ukulandelana kwedatha. Uma idatha isatshalaliswa kuzo zonke izinhlelo zokusebenza namasistimu, kungaba inselele ukulandelela uhambo lwayo ukusuka kokunye kuya kokunye. Lokhu kungase kuholele kuhlu lwedatha olunganembile noma olungaphelele.
Isiphetho
Sengiphetha, ukulandelana kwedatha kuyingxenye ebalulekile yawo wonke amabhizinisi aqhutshwa yidatha. Inikeza umbono obanzi wendlela yedatha ukusuka lapho iqala khona ukuya endaweni yayo yokugcina, iqinisekisa ukunemba kwayo, ukuphelela, nokungaguquguquki.
I-future data lineage automation kanye nokumiswa kulindeleke ukuthi kukhule, kwenze ukuqaliswa nokugcinwa kwezinhlangano kube lula. Ekugcineni, ukubaluleka kohlu lwedatha angeke kugcizelelwe.
Inikeza izinkampani amathuluzi eziwadingayo ukuze zenze izinqumo ezihlakaniphile, ziqhube imisebenzi yazo ngokuphumelelayo, futhi zizuze impumelelo.
shiya impendulo