Zviri Mukati[Viga][Ratidza]
Data iri kwese kwese kwakakupoteredza. Mupfungwa chaiyo, zvinopesvedzera chikamu chose chebhizinesi rako. Zvinogona kunzwa sekunge hapana nguva yakakwana yekuongorora iwo chaiwo ekuti riri kushandira bhizinesi rako zvakanaka sei kana uchinge wakabatikana nesarudzo dzemabatiro e data rako.
Cherechedza izvi. Sangano rako riri kushandisa data maawa makumi maviri nemana pazuva. Saka kunzwisisa kwayakabva, kuti yakasvika sei, uye kuti iri kufamba sei nekambani kwakakosha kuti unzwisise kukosha kwayo.
Mutsara we data unova wakakosha mumamiriro ezvinhu aya. Zviri nyore kunzwisisa kuti data yakaumbwa sei, kwayakabva, uye kwairi kuenda kana isu tichigona kuronda kwakabva, kutama, uye shanduko yedata.
Mune ino positi, isu tichanyatso tarisa kuData Lineage, kuti inoshanda sei, mashandisiro ayo makesi, matekiniki, uye zvimwe zvakawanda.
Chii chinonzi Data Lineage?
Dhata mutsara unoshanda semhando yedhijitari pasipoti. Ndiyo nhoroondo yakazara kwazvo yerwendo rwedata, inotsanangura zvese zvayakamira, kutsaudzira, uye kugadziridzwa kubva kwayakabva kusvika kwainozoenda.
In essence, mutsara wedata unotsanangura kwakabva, kugadziridzwa, uye kushandiswa kwechidimbu che data pane akawanda masisitimu nemapuratifomu. Inoshanda sechokushandisa chemutikitivha nekupa vashandisi ruzivo rwekuti data rakagadzirwa sei, kwarakabva, uye mashandisirwo arakaitwa. Ruzivo urwu runogonesa vashandisi kuona nekugadzirisa chero matambudziko angangoitika.
Mutsara wedata chinhu chakakosha kumakambani anovimba nedata kuti aite mashandiro avo nekuti inobvumira vashandisi kupindura kumibvunzo yakakosha senge ndiani, chii, rini, uye kupi.
Mutsara wedhata, kutaura zviri nyore, iyo yekupedzisira data trail inovimbisa chokwadi chedata, kukwana, uye kuenderana uku ichipa yakajeka uye yakajeka maonero eiyo data yakazara nzira.
Data Lineage inoshanda sei?
Mutsara wedhata ndiyo mepu yemumugwagwa inotigonesa kutevedzera chidimbu che data kubva painotangira kusvika painoperera. Funga nezve data data semufambi, uye pasipoti yayo kuti ive iyo data mutsara kuti unzwisise zviri nani mashandiro ayo.
Zvinyorwa zvedata, shanduko yedata, chengetedzo yedata, uye kuburitsa data zvinogadzira zvinhu zvina zvakakosha zvepasipoti.
Iwo akawanda masisitimu, maapplication, uye mapuratifomu kubva kunobva data anomiririrwa nedata masosi, ayo anoshanda senzvimbo dzekutanga dzerwendo rwe data. Shanduko yedata ndiyo inotevera nhanho, uye mutsara wedata unotaridza kufambira mberi kwedata kubva kune aya masosi kuenda kwairi.
Shanduko yedata inoreva kuumbwa, kugadzirisa, uye kushandura data kuti isangane nezvinodiwa nemushandisi. Inoshanda sekuzorora kwekumira panguva yerwendo rwe data, ichiigadzirira gumbo rinotevera.
Iyo data inozochengetwa isati yaenda kunzvimbo yayo yekupedzisira. Inogona kuchengetwa pamaseva egore, dhatabhesi, kana imwe mhando yemudziyo wekuchengetera. Mutsara wedata unochengeta kwainochengeterwa data, pamwe nekudzivirirwa kwairi, kutsigirwa, uye kudzoserwa.
Nhanho yekupedzisira ndeyekuburitsa data, uko ndiko kunotumirwa data kuti ishandiswe. Mishumo, infographics, kana chero imwe mhando yechigadzirwa chedata inogona kushandiswa kuiratidza. Mutsara wedata unochengeta zvakabuda uye unovimbisa kuenderana, huchokwadi, uye kuzara kweiyo data.
Mutsara wedata unonyanya kushanda nekurekodha nhanho yega yega yerwendo rwedata, kubva payakatanga kusvika pakubuda kwayo, uye kuve nechokwadi chekuti inoramba yakavimbika, inowirirana, uye ichigadzirisa nzira yese. Mutsara wedata unobatsira masangano kuita sarudzo dzakadzidziswa, kugadzirisa matambudziko, uye kutevedzera zvisungo zvemutemo nekupa tarisiro yakazara yekuvapo kwedata.
Kuti unzwisise iyo data asset uye mafambiro aanoita kuburikidza nepombi yedata, metadata chikamu chakakosha cheiyo data mutsara maitiro.
Iwe unogona kuona kuti data rinoshandurwa sei uye rinoshandiswa mukati mesangano uchishandisa data lineage maturusi, ayo anowedzera metadata kuti ape mufananidzo unooneka wekuyerera kwedata. Izvi zvinogonesa vashandisi kuongorora iyo data inogona kuvabatsira kuita sarudzo dzine ruzivo.
Mhando dzeData Lineage
Kune matatu mafomu ekutanga edhata mutsara: yekumberi data mutsara, yekumashure data mutsara, uye bi-directional data mutsara.
Pamberi Data Lineage
Sezvakaita nemugwagwa wenzira imwe, mutsara wedata wemberi unosanganisira kuronda chidimbu che data kubva painotangira kusvika painoperera. Kutanga kubva kune data sosi, inoteedzera iyo data sezvo ichipfuura nepakati akati wandei shanduko uye ekuchengetedza masisitimu kuti isvike pakubuda kwayo.
Kunzwisisa kugadzirisa uye kushandurwa kwedata pamwe nechero matambudziko angave amuka munzira anofambiswa nekuva nemutsara wedata werudzi urwu. Nhanho imwe neimwe inotungamira kune inotevera; zvakafanana nekutevera nzira yezvimedu zvechingwa.
Backward Data Lineage
Mutsara wedata rekudzokera kumashure wakafanana nerwendo rwekudzokera kumashure uko isu tinoronda zvakabuda kubva kune data kumashure kwayakabva. Maitiro acho anotanga panzvimbo yekupedzisira yedata uye anodzokera kumashure kuburikidza neakasiyana ekuchengetedza uye shanduko nzira kudzamara yasvika kune data sosi.
Kuzivikanwa kwekwakabva data, kunzwisiswa kweshanduko yayo, uye ongororo yekururama kwayo uye kukwana kwayo zvese zvinogoneka nerubatsiro rwerudzi urwu rwemutsara wedata. Inoshanda sechishandiso chemutikitivha, ichitibvumira kutevera nzira yedata kumashure.
Bi-directional Data Lineage
Mugwagwa une nzira mbiri, bi-directional data mutsara unosanganisa zvakanakira zvemberi uye kumashure data mutsara. Inopa maonero akazara enzira yedata nekuitevera kubva kwairi kuenda kwairi kuenda uye kubva kunzvimbo iyoyo kusvika kwainotangira.
Kuti uone kwakabva data, kunzwisisa kuti yakashandurwa sei, uye kuvimbisa hunhu hwayo, kuenderana, uye kukwana munzira yese, zvinobatsira kuronda dzinza re data. Neruzivo rwechokwadi-nguva nezve nzvimbo uye chimiro, zvakaita sekuva neGPS tracker yedata.
Kuitwa kweData Lineage
Kuita mutsara wedata musangano kazhinji kunosanganisira zvikamu zvinotevera.
Tsanangura mabviro e data
Iwo masisitimu uye dhatabhesi inobata iyo data yaunoda kuteedzera inofanira kuzivikanwa yese. Kuti uite izvi, unofanirwa kutanga waona kwakasiyana data masosi, kusanganisira mafaera, APIs, uye makore masevhisi.
Unganidza metadata
Nhanho inotevera ndeyekuwana ruzivo nezve data, kusanganisira nzvimbo yaro, fomati, uye sangano. Kunzwisisa maumbirwo e data uye mashandisirwo ainoitwa kunogoneka neiyi metadata.
Ziva data kukanganisa
Zviri nyore kunzwisisa kuti data inovandudzwa sei uye inoshandiswa mukati mesangano kana kuyerera kwedata kwakarongwa kubva kwairi kuenda kwairi kuenda, kusanganisira chero shanduko kana kugadzirisa kunoitika munzira.
Tevera kuwana data
Kuchengetedza kuchengetedza kwedata uye kutevedzera, kuteedzera, uye kurekodha uyo anowana iyo data.
Chengetedza uye fungidzira mutsara
Shandisa maturusi ekuona kuratidza mutsara wekunzwisisa kuri nyore uye kuongorora. Chengetedza iyo yakaunganidzwa metadata uye data kuyerera ruzivo mune imwechete repository.
Shandisa mhinduro yega
Iwe unogona kuona kuti mutsara wedata uri kuunganidzwa uye kutariswa kuburikidza neautomation, izvo zvichabatsirawo kuderedza kukanganisa uye kuwedzera chibereko.
Wongororo & Update
Ita kuti zvinyorwa zvemutsara ndezvechokwadi uye zvazvino nguva nenguva, uye zvigadzirise sezvakakodzera.
Maitiro ekuita angangoda kugadziridzwa kana kuwedzerwa kuzvikamu zvinoenderana nezvinodiwa zvakasiyana uye miganhu yesangano rega rega.
Data Lineage Techniques
Pattern-based Lineage
Neiyi nzira, mutsara unoitwa pasina kupindirana nehurongwa hwakagadzira kana kushandura data. Metadata ongororo yematafura, makoramu, uye mishumo yebhizinesi zvese chikamu chayo. Inoongorora mutsara nekutsvaga mafambiro uchishandisa iyi metadata.
Semuenzaniso, zvine mukana wekuti koramu mumaseti maviri ane zita rimwechete uye yakafanana data data inomiririra iyo yakafanana data pazvikamu zvakasiyana zvekuvapo kwayo. A data lineage chart inobva yashandiswa kubatanidza aya makoramu maviri.
Patani-yakavakirwa mutsara ine yakakosha bhenefiti yekuzvimiririra tekinoroji nekuti inongotarisa data, kwete nzira dzekugadzirisa data. Chero tekinoroji yedatabase, kusanganisira Oracle, MySQL, uye Spark, inogona kuishandisa nenzira imwechete. Dambudziko nderokuti nzira iyi haisi nguva dzose yakarurama.
Kana iyo data yekugadzirisa logic yakavigwa mukombuta yekombuta uye isiri pachena mumetadata inoverengwa nevanhu, inogona nguva nenguva kufuratira hukama pakati pedataset.
Dzinza neData Tagging
Iyi nzira inotemerwa pane pfungwa yekuti injini yekushandura inobata kana neimwe nzira inocherekedza data. Inoteedza tag kubva kumavambo kusvika kumagumo kuti iwane dzinza. Iyi nzira inogona kubudirira chete kana uine chakavimbika chekushandura chishandiso chinobata ese ekufambisa data uye iwe unojairana nematagi chimiro chinoshandiswa nechombo.
Kunyangwe dai chishandiso chakadaro chaizovepo, hapana data rakagadzirwa kana kuchinjwa pasina iro raigona kuiswa pasi pemutsara kuburikidza nedata tagging. Izvo zvinogumira mune izvi kuita mutsara wedata pane yakavharwa data system.
Kuzvimiririra Mutsara
Mamwe mabhizinesi ane nharaunda yedata inosanganisira kuchengetedza metadata, kugadzirisa logic, uye master data management (MDM). Aya marongero anowanzo sanganisira a data lake uko data rese rinochengetwa mukati mehupenyu hwayo hwese.
Lineage inogona kupihwa zvakasikwa nerudzi urwu rwekuzvimiririra system pasina chinodiwa chekuwedzera zviwanikwa. Nekudaro, sezvakangoita nenzira yekumaka data, mutsara hauzoziva chero chinhu chinoitika kunze kweiyi nharaunda yakadzorwa.
Data Lineage by Parsing
Iyo yakanyanya kuomesesa mhando yemutsara ndeiya inoverenga data-kugadzirisa logic otomatiki. Kunyatso, kupedzisa-kusvika-kumagumo, nzira iyi inodzosera mainjiniya iyo data shanduko logic.
Sezvo mhinduro iyi inofanirwa kunzwisisa zvese zve kuronga mitauro uye zvishandiso zvinoshandiswa kushandura uye kutakura iyo data, kuiswa kwayo kwakaoma. Izvi zvinogona kushandisa extract-transform-load (ETL) logic, SQL- uye Java-based solutions, data data formats, XML-based solutions, uye mamwe maitiro.
Data Lineage Use Cases
Data modelling
Makambani anofanirwa kumisikidza epasi data zvimiro zvinovatsigira kuti vaone zvinhu zvakawanda zve data uye kubatana kuri pakati pazvo mukati mekambani. Izvi zvinongedzo zvinomisikidzwa uchishandisa mutsara wedata, uyo unoratidzawo zvakawanda zvinotsamira zviripo mu data ecosystem.
Sezvo data richichinja nekufamba kwenguva, zvinyorwa zvitsva zvinogara zvichionekwa, zvinoda kubatanidzwa kwedata kutsva, nezvimwewo. Nekuda kweizvi, mafemu 'general data modhi yekutarisa data ravo anofanirawo kushanduka kuratidza nharaunda.
kutevedzera
Mutsara wedata unopa nzira yekutevedzera yekuongorora, kusimudzira manejimendi enjodzi, uye kuve nechokwadi kuti data rinochengetwa uye rinobatwa zvinoenderana nemitemo yekutonga data nemitemo.
Impact Kuongorora
Mhedzisiro yeimwe shanduko yebhizinesi, senge chero yakadzika yekubika, inogona kuoneka uchishandisa data mutsara maturusi. Mutsara wedhata, semuenzaniso, unogona kubatsira vatariri mukuona kuti mangani madhibhodhi ekuchinja zita angakanganisa uye, nekudaro, vangani vanhu vanowana iyo yekushuma.
Kutama kwedata
Masangano anoshandisa kutama-dhata kuti anzwisise kuti data riri papi uye kuti rave nguva yakareba sei risati raendesa kune imwe nzira yekuchengetedza kana kushandisa software nyowani.
Mutsara wedata unobatsira zvikwata kugadzirira kukwidziridzwa kwehurongwa kana kutama nekuvapa tarisiro yekuti data rakafamba sei musangano rese. Izvi zvinomhanyisa kuendesa kunzvimbo itsva yekuchengetedza zvachose.
Pamusoro pezvo, inopa zvikwata mukana wekubvisa iyo data system nekuchengeta kana kubvisa data rekare kana risina basa. Nekuita izvi, iyo data system ichaita zvirinani zvakazara uye inoda kushoma manejimendi yedata.
Matambudziko Ekuita Data Lineage
- Kuchengetedzwa kwedata: Kuchengetedzeka kwedata chinhu chinonyanya kunetseka paunenge uchivaka mutsara wedata. Kutevera rwendo rwedata kubva payaitangira kusvika kwainoperera, kuwana data rakadzama kunofanirwa kupihwa, uye iyi data inofanirwa kuchengetedzwa kubva kune kusingatenderwe kuwana nekutyorwa.
- Kushaikwa kweKumisikidzwa: Chimwe chezvipingamupinyi zvekutanga kumbundira mutsara wedata kushaikwa kwemazinga. Sezvo akawanda mapuratifomu, maapplication, uye masisitimu anoshandisa yakasarudzika nzira dzekutevera uye kurekodha dhata, zvinogona kuve zvakaoma kubatanidza pamwechete mufananidzo wakabatana werwendo rwedata.
- Data Silos: Data silos imwe nyaya inomuka panguva yekushandisa data mutsara. Kana data ichipararira kune akati wandei maapplication uye masisitimu, zvinogona kuve zvakaoma kuronda rwendo rwayo kubva kune imwe kuenda kune imwe. Izvi zvinogona kutungamirira kumutsara wedata usina kururama kana usina kukwana.
mhedziso
Mukupedzisa, mutsara wedata chikamu chakakosha chebhizinesi rega rega rinofambiswa nedata. Inopa maonero akazara enzira yedata kubva painotangira kusvika painogumira, ichivimbisa huchokwadi, huzere, uye kuenderana.
Remangwana data mutsara otomatiki uye kumisikidza zvinotarisirwa kuwedzera, zvichiita kuti kuita uye kugadzirisa kwemasangano kuve nyore. Pakupedzisira, kukosha kwemutsara wedata hakugone kusimbiswa.
Inopa makambani maturusi avanoda kuita sarudzo dzakangwara, kuita mabasa avo nemazvo, uye kubudirira.
Leave a Reply