Hive kayan aikin Binciken Babban Bayanai ne da ake amfani da shi sosai a cikin kasuwancin, kuma wuri ne mai ban sha'awa don farawa idan kun kasance sababbi ga Babban Bayanai. Wannan darasi na Apache Hive yana tafiya ta tushen tushen Apache Hive, dalilin da yasa hive ya zama dole, fasalinsa, da duk wani abu da yakamata ku sani.
Bari mu fara fahimtar tsarin Hadoop wanda aka gina Apache Hive akansa.
Hadoop Apache
Apache Hadoop kyauta ne kuma Bude-source dandamali don adanawa da sarrafa manyan bayanan bayanai masu girma daga gigabytes zuwa petabytes. Hadoop yana ba da damar tara kwamfutoci da yawa don tantance manyan bayanai a layi daya, maimakon buƙatar babbar kwamfuta ɗaya don adanawa da tantance bayanan.
MapReduce da Hadoop Rarraba Fayil ɗin Fayil ɗin Rarraba biyu ne daga cikin abubuwan haɗin gwiwa:
- Taswirar Kasa – MapReduce wata dabara ce ta tsara shirye-shirye don sarrafa ɗimbin ɗimbin tsari, tsararru, da bayanan da ba a tsara su ba kan tarin kayan masarufi.
- HDFS – HDFS (Tsarin Fayil Rarraba Hadoop) yanki ne na tsarin Hadoop wanda ke adanawa da sarrafa bayanai. Tsarin fayil ne mai jurewa kuskure wanda ke gudana akan daidaitaccen kayan aiki
Daban-daban ƙananan ayyuka (kayan aiki) a cikin yanayin yanayin Hadoop, gami da Sqoop, Pig, da Hive, ana amfani da su don taimakawa samfuran Hadoop.
- hive – Hive wani tsari ne na rubuta rubutun salon SQL waɗanda ke yin lissafin MapReduce.
- Alade - Alade yaren shirye-shirye ne na tsari wanda za'a iya amfani dashi don ƙirƙirar rubutun don tafiyar matakai na MapReduce.
- Sqoop - Sqoop kayan aiki ne don shigo da bayanai tsakanin HDFS da RDBMS.
Mene ne Apache Kado?
Apache Hive shine tushen budewa bayanan ajiya shirin don karantawa, rubutu, da sarrafa manyan bayanan da aka adana kai tsaye a cikin Tsarin Fayil ɗin Rarraba Apache Hadoop (HDFS) ko wasu tsarin adana bayanai kamar Apache HBase.
Masu haɓaka SQL na iya amfani da Hive don ƙirƙirar maganganun Hive Query Language (HQL) don neman bayanai da bincike waɗanda suka yi daidai da maganganun SQL na yau da kullun. An ƙirƙira shi don sauƙaƙe shirye-shiryen MapReduce ta hanyar kawar da buƙatar koyo da rubuta doguwar lambar Java. Madadin haka, zaku iya rubuta tambayoyinku a cikin HQL, kuma Hive zata gina taswirar kuma ta rage muku ayyukan.
Ƙirƙirar hanyar SQL mai kama da Apache Hive ta zama Matsayin Zinariya don yin bincike na ad-hoc, taƙaitawa, da kuma nazarin bayanan Hadoop. Lokacin da aka haɗa cikin gajimare hanyoyin sadarwar kwamfuta, Wannan bayani yana da mahimmanci mai mahimmanci kuma mai daidaitawa, wanda shine dalilin da ya sa kamfanoni da yawa, ciki har da Netflix da Amazon, suna ci gaba da haɓakawa da inganta Apache Hive.
Tarihi
A lokacin da suke Facebook, Joydeep Sen Sarma da Ashish Sooo sun hada Apache Hive. Dukansu sun fahimci cewa don samun fa'ida daga Hadoop, dole ne su ƙirƙira wasu rikitattun ayyuka na Taswirar Java-Rage. Sun fahimci ba za su iya ilmantar da ƙungiyoyin aikin injiniya da ke haɓaka cikin sauri kan ƙwarewar da suke buƙata don yin amfani da Hadoop a duk faɗin kamfanin. Injiniyoyin injiniya da manazarta akai-akai suna amfani da SQL azaman mahaɗin mai amfani.
Yayin da SQL zai iya biyan yawancin buƙatun nazari, masu haɓakawa kuma sun yi niyyar haɗa shirye-shiryen Hadoop. Apache Hive ya taso ne daga waɗannan manufofi guda biyu: Harshen bayyanawa na tushen SQL wanda kuma ya baiwa masu haɓakawa damar kawo nasu rubutun da shirye-shiryen lokacin da SQL bai isa ba.
Hakanan an haɓaka shi don riƙe metadata na tsakiya (Hadoop-based) game da duk bayanan da ke cikin kamfanin don sauƙaƙe gina ƙungiyoyin bayanan.
Ta yaya Apache Hive ke aiki?
A taƙaice, Apache Hive yana canza shirin shigar da aka rubuta a cikin yaren HiveQL (SQL-like) zuwa ɗaya ko fiye Java MapReduce, Tez, ko Spark ayyuka. (Duk waɗannan injunan kisa sun dace da Hadoop YARN.) Bayan haka, Apache Hive yana tsara bayanai cikin tebur don Tsarin Fayil ɗin Rarraba Hadoop HDFS) kuma yana aiwatar da ayyuka akan gungu don samun amsa.
data
An tsara allunan Apache Hive kamar yadda aka tsara teburi a cikin bayanan alaƙa, tare da raka'o'in bayanai masu girma daga girma zuwa ƙarami. Databases suna kunshe ne da teburi da aka karkasu zuwa kashi-kashi, wadanda aka kara rabasu zuwa bokiti. Ana amfani da HiveQL (Hive Query Language) don samun damar bayanan, wanda za'a iya canzawa ko haɗawa. An jera bayanan tebur a cikin kowane rumbun adana bayanai, kuma kowane tebur yana da nasa kundin adireshin HDFS.
Architecture
Yanzu za mu magana game da mafi muhimmanci al'amari na Hive Architecture. Abubuwan da ke cikin Apache Hive sune kamar haka:
Metastore - Yana kiyaye bayanan kowane tebur, kamar tsarinsa da wurinsa. Hakanan ana haɗa metadata na bangare a cikin Hive. Wannan yana bawa direba damar ci gaba da lura da ci gaban saitin bayanai daban-daban da aka bazu a cikin tari. Ana adana bayanan a cikin tsarin RDBMS na al'ada. Hive metadata yana da matuƙar mahimmanci ga direba don kula da bayanan. A madadin uwar garken yana kwafin bayanai akai-akai domin a iya dawo dasu a yayin asarar bayanai.
Direba – Ana karɓar bayanan HiveQL ta direba, wanda ke aiki azaman mai sarrafawa. Ta hanyar kafa zaman, direban ya fara aiwatar da bayanin. Yana lura da tsawon rayuwar mai zartarwa da ci gabansa. Yayin aiwatar da bayanin HiveQL, direba yana adana metadata da ake buƙata. Hakanan yana aiki azaman wurin tattara bayanai ko sakamakon tambaya biyo bayan Rage tsari.
Mai tarawa - Yana aiwatar da harhada tambayoyin HiveQL. Tambayar yanzu an canza zuwa tsarin aiwatarwa. An jera ayyukan a cikin shirin. Hakanan ya haɗa da matakan da MapReduce dole ne ya ɗauka don samun sakamako kamar yadda tambayar ta fassara. An juyar da tambayar zuwa Bishiyar Haɗaɗɗen Magana ta Hive's compiler (AST). Yana canza AST zuwa Hotunan Acyclic Directed bayan duba dacewa da kurakuran lokaci-lokaci (DAG).
Mai ingantawa - Yana inganta DAG ta hanyar yin canje-canje daban-daban akan tsarin aiwatarwa. Yana haɗa sauye-sauye don ingantaccen aiki, kamar juya bututun haɗin kai zuwa haɗin haɗin gwiwa guda ɗaya. Don inganta sauri, mai ingantawa na iya raba ayyuka, kamar amfani da canji zuwa bayanai kafin yin aikin ragewa.
Mai aiwatarwa - Mai aiwatarwa yana gudanar da ayyukan lokacin da aka gama haɗawa da haɓakawa. Ana aiwatar da ayyukan ta hanyar Executor.
CLI, UI, da Thrift Server - Ƙididdigar layi na umarni (CLI) shine mai amfani da mai amfani wanda ke ba da damar mai amfani na waje don sadarwa tare da Hive. Sabar thrift ta Hive, mai kama da ka'idojin JDBC ko ODBC, yana ba abokan ciniki na waje damar sadarwa tare da Hive ta hanyar hanyar sadarwa.
Tsaro
Apache Hive an haɗa shi tare da tsaro na Hadoop, wanda ke amfani da Kerberos don amincin abokin ciniki da uwar garken. HDFS tana ba da izini ga sabbin fayilolin da aka ƙirƙira a cikin Apache Hive, yana ba ku damar amincewa da mai amfani, ƙungiya, da sauransu.
key siffofin
- Hive yana goyan bayan tebur na waje, waɗanda ke ba ku damar sarrafa bayanai ba tare da adana su a cikin HDFS ba.
- Hakanan yana ba da damar rarraba bayanai a matakin tebur don ƙara saurin gudu.
- Apache Hive yana da kyau ya dace da ƙarancin ƙirar ƙirar Hadoop.
- Hive yana sa taƙaita bayanai, tambaya, da bincike cikin sauƙi.
- HiveQL baya buƙatar kowane ƙwarewar shirye-shirye; sauƙin fahimtar tambayoyin SQL ya wadatar.
- Hakanan zamu iya amfani da Hive don gudanar da tambayoyin ad-hoc don tantance bayanai.
- Yana da ma'auni, saba, kuma mai daidaitawa.
- HiveQL baya buƙatar kowane ƙwarewar shirye-shirye; sauƙin fahimtar tambayoyin SQL ya wadatar.
amfanin
Apache Hive yana ba da damar rahotannin ƙarshen rana, kimanta ma'amala na yau da kullun, binciken ad-hoc, da nazarin bayanai. Cikakken fahimtar da Apache Hive ya bayar yana ba da fa'idodi ga gasa kuma yana sauƙaƙa muku don amsa buƙatun kasuwa.
Anan ga wasu fa'idodin samun irin waɗannan bayanan a shirye suke:
- Sauƙi na amfani - Tare da yaren sa kamar SQL, bayanan tambaya yana da sauƙin fahimta.
- Gaggauta shigar da bayanai - Saboda Apache Hive yana karanta makircin ba tare da tabbatar da nau'in tebur ko ma'anar tsari ba, ba dole ba ne a karanta, tantancewa, da jera bayanai zuwa fayafai a cikin tsarin bayanan bayanan. Sabanin haka, a cikin bayanan al'ada, dole ne a inganta bayanai duk lokacin da aka ƙara shi.
- Maɗaukakin scalability, sassauƙa, da ingantaccen farashi - Saboda an adana bayanai a cikin HDFS, Apache Hive na iya ɗaukar 100s na petabytes na bayanai, yana mai da shi zaɓi mafi girma fiye da na yau da kullun. Apache Hive, azaman sabis na Hadoop na tushen gajimare, yana bawa abokan ciniki damar jujjuya sama da saukar da sabar kama-da-wane da sauri don saduwa da canjin ayyuka.
- Babban ƙarfin aiki - Manyan bayanai na iya ɗaukar tambayoyi har 100,000 a kowace awa.
gazawar
- Gabaɗaya, tambayoyin Apache Hive suna da jinkiri sosai.
- Tallafin biyan kuɗi yana da iyaka.
- Ba a samun tambayoyin ainihin-lokaci da sauye-sauye-jere a cikin Apache Hive.
- Babu goyon baya ga ra'ayi na zahiri.
- A cikin hive, sabuntawa da share ayyuka ba su da tallafi.
- Ba a yi nufin OLTP (tsari na tsaka-tsakin kan layi ba).
Farawa tare da Apache Hive
Apache Hive shine abokin Hadoop mai ƙarfi wanda ke sauƙaƙa da daidaita ayyukan ku. Don samun fa'ida daga Apache Hive, haɗin kai mara kyau yana da mahimmanci. Mataki na farko shine zuwa ga yanar.
1. Shigarwa Hive daga Bargawar Saki
Fara da zazzage mafi ƙarancin kwanciyar hankali na Hive daga ɗayan madubin zazzagewar Apache (duba Sakin Hive). Sannan dole ne a cire kayan kwalta. Wannan zai haifar da babban fayil mai suna hive-xyz (inda xyz shine lambar sakin):
Saita madaidaicin yanayin HVE_HOME don nunawa ga kundin adireshin shigarwa:
A ƙarshe, ƙara $HIVE_HOME/bin zuwa naku PATH
:
2. Gudun Hiwa
Hive yana amfani da Hadoop, don haka:
- dole ne ku sami Hadoop a hanyar ku KO
3. DLL aiki
Ƙirƙirar Teburin Hive
yana samar da tebur mai suna pokes tare da ginshiƙai biyu, na farko wanda shine lamba kuma na biyu shine kirtani.
lilo ta Tables
Lissafin Duk Tables
Canje-canje da Cire Tebur
Ana iya canza sunayen tebur kuma ana iya ƙara ko musanya ginshiƙai:
Yana da kyau a lura cewa SAUYA COLUMNS yana maye gurbin duk ginshiƙan da ke akwai yayin da kawai ke canza tsarin tebur ba bayanai ba. Dole ne a yi amfani da ɗan asalin SerDe a cikin tebur. Hakanan za'a iya amfani da SAUKI COLUMNS don cire ginshiƙai daga tsarin tsarin tebur:
Zazzage Tebura
Akwai ƙarin ƙarin ayyuka da fasali da yawa a cikin Apache Hive waɗanda zaku iya koya game da su ta ziyartar gidan yanar gizon hukuma.
Kammalawa
Ma'anar hive shine tsarin tsarin bayanai don tambaya da bincike don manyan bayanan da aka gina a saman Apache Hadoop. Kwararru sun zaɓe shi fiye da sauran shirye-shirye, kayan aiki, da software tunda an ƙirƙira ta musamman don cikakkun bayanai na Hive kuma yana da sauƙin amfani.
Fata wannan koyawa tana taimaka muku farawa tare da Apache Hive kuma ku sa ayyukanku su fi inganci. Bari mu sani a cikin sharhi.
Leave a Reply