Isiqulatho[Fihla][Bonisa]
Izicelo ze-intanethi ezinkulu ziye zahamba indlela ende kumashumi amabini eminyaka adlulileyo. Olu tshintsho luyitshintshile indlela esibona ngayo uphuhliso lwesoftware. I-Facebook, i-Instagram, kunye ne-Twitter, umzekelo, onke ngamaqonga anokunyuka.
Ezi nkqubo kufuneka zakhelwe ukulawula umthamo omkhulu wetrafikhi kunye nedatha kuba iibhiliyoni zabantu zizisebenzisa ngaxeshanye kwihlabathi liphela. Eli lixesha uyilo lwenkqubo ungena emfanekisweni.
Inkqubo yokuseka uyilo, ujongano, kunye nedatha yenkqubo ehlangabezana nemilinganiselo ethile yaziwa ngokuba yinkqubo yoyilo. Ngeenkqubo ezibambeneyo nezisebenzayo, uyilo lwenkqubo luyanelisa iimfuno zeshishini lakho okanye umbutho.
Nje ukuba inkampani okanye umbutho wakho umisele imigaqo yawo, ungaqala ukuzibandakanya kuyilo lwenkqubo yomzimba ehlangabezana neemfuno zabathengi bakho.
Nokuba ukhetha ukuhamba nophuhliso lwebespoke, izisombululo zorhwebo, okanye indibaniselwano yezi zimbini, indlela oyila ngayo inkqubo yakho iya kugqiba ukuba uyakha njani.
Siza kujonga ngokweenkcukacha uyilo lwenkqubo yoluhlu lwexesha le-Twitter kwesi sithuba, sigqibezele ngesifundo. Masiqalise.
Inyathelo 1: Yenza inkcazo ngemeko yokusetyenziswa kunye nemiqobo
Sebenzisa imeko
- Umsebenzisi ufaka i-tweet.
- Inkonzo ithumela izaziso zokutyhala kunye nee-imeyile kubalandeli beetweets.
- Ixesha lomsebenzisi lijongwe (umsebenzi osuka kumsebenzisi)
- Umsebenzisi ujonga ixesha lasekhaya (umsebenzi osuka kubantu abalandelayo umsebenzisi)
- Amagama angundoqo akhangelwa ngumsebenzisi.
- Inkonzo iyafikeleleka ngokwenene.
Ngaphandle kwendawo
- IiTweets zithunyelwa kwi-Twitter Firehose kunye neminye imijelo esebenzisa le nkonzo.
- Inkonzo isusa i-tweets ngokusekelwe kwimimiselo yokubonakala komsebenzisi.
- Ukuba umsebenzisi akalandeli umntu ophendulwa kuye, fihla impendulo.
- Qwalasela i 'fihla retweets' ukhetho.
- Uhlalutyo
Imiqobo kunye neengqikelelo
Iingqikelelo zikaRhulumente
- I-traffic ayisasazwanga ngokulinganayo.
- Kufuneka kube lula ukuthumela i-tweet.
- Ngaphandle kokuba unezigidi zabalandeli, ukuthumela i-tweet kubo bonke abalandeli bakho kufuneka kukhawuleze.
- Kukho i-100 yezigidi zabasebenzisi abasebenzayo.
- I-15 yeebhiliyoni zeetweets nyanga nganye okanye i-500 yezigidi zeetweets yonke imihla
- I-tweet nganye ine-fanout ye-10 yokuhanjiswa ngokomndilili.
- Yonke imihla, i-fanout ihambisa ii-tweets ezibhiliyoni ezi-5.
- IFanout ihambisa iitweets ezibhiliyoni ezingama-150 ngenyanga.
- Iibhiliyoni ezingama-250 zezicelo zokufunda ngenyanga
- I-10 yeebhiliyoni zophando lwenyanga
Ixesha
- Umda wexesha kufuneka ube lula ukuwujonga.
- I-Twitter imalunga nokufunda kunokubhala.
- Lungiselela ukufunda itweet ngokukhawuleza
- Ukusetyenziswa kweTweet kudla ixesha.
ukufuna
- Inkqubo yokukhangela kufuneka ikhawuleze.
- Kuchitha ixesha ukukhangela.
Bala ukusetyenziswa
Ubungakanani betweet nganye:
- 8 bytes tweet id
- 32 bytes umsebenzisi-id
- I-140 bytes yombhalo
- imidiya – umndilili we-10 KB
- Iyonke: ~10 KB
Rhoqo ngenyanga, i-150 TB yomxholo omtsha we-tweet yenziwa.
- * 500 million tweets yonke imihla * 30 iintsuku ngenyanga * 10 KB tweet nganye
- Kwiminyaka emithathu, bekukho i-5.4 PB yomxholo omtsha we-tweet.
Kukho izicelo ezifundwayo ezili-100,000 ngomzuzwana ngamnye.
- * (izicelo ezingama-400 ngesekhondi / izicelo ezibhiliyoni ezi-1 ngenyanga) izicelo ezingama-250 ezigidigidi zifundwa nyanga nganye
Kukho ii-tweets ezingama-6,000 kwisekhondi nganye.
- * (izicelo ezingama-400 ngesekhondi / izicelo ezibhiliyoni ezi-1 ngenyanga) ii-tweets ezibhiliyoni ezili-15 ngenyanga
Kwi- fanout, ii-tweets ezingamawaka angama-60 zithunyelwa ngomzuzwana ngamnye.
- IFanout ihambisa i-150 yeebhiliyoni zeetweets nyanga nganye* (izicelo ezingama-400 ngesekhondi / izicelo ezibhiliyoni ezi-1 ngenyanga).
Izicelo ezingama-4,000 zolwazi ngomzuzwana ngamnye
- * (izicelo ezingama-400 ngesekhondi / izicelo ezibhiliyoni ezi-1 ngenyanga) iibhiliyoni ezili-10 zikhangelwa nyanga nganye
Uguqulo oluthile oluluncedo
- Nyanga nganye, kudlula imizuzwana ezizigidi ezi-2.5.
- Izicelo ze-2.5 yezigidi ngenyanga kwisicelo esi-1 ngesekhondi
- Izicelo ezizizigidi ezili-100 ngenyanga x izicelo ezingama-40 ngesekhondi
- Izicelo ze-1 yebhiliyoni ngenyanga = izicelo ze-400 ngesekhondi
Inyathelo lesi-2: Umzobo okwinqanaba eliphezulu
Inyathelo lesi-3: Ukucacisa amacandelo aphambili
Sinokuzigcina ii-tweets zomsebenzisi ezizezakhe ukugcwalisa ixesha lomsebenzisi (umsebenzi osuka kumsebenzisi) kwiziko ledatha elinxulumeneyo ukuba bangenise i-tweet. Kunzima ngakumbi ukuhambisa iitweets kunye nokuphuhlisa umda wexesha lasekhaya (umsebenzi osuka kumntu ngamnye olandelwa ngumsebenzisi).
Isiseko sedatha esiqhelekileyo sobudlelwane siya koyiswa ngokukhuphela ii-tweets kubo bonke abalandeli (ii-tweets ezingamawaka angama-60 zihanjiswa ngomzuzwana ngamnye). Mhlawumbi siya kufuna ukuhamba ngokukhawuleza-bhala ukugcinwa kwedatha njengesiseko sedatha seNoSQL okanye iMemory Cache.
Ukufunda i-1 MB ngokulandelelanayo kwimemori kuthatha malunga ne-250 microseconds, kodwa ukufunda kwi-SSD kuthatha amaxesha angama-4 ubude, kwaye ukufunda kwidiski kuthatha amaxesha angama-80 ubude.
Ugcino lweNto lunokusetyenziswa ukugcina idatha efana nemifanekiso kunye neevidiyo.
- Iseva yewebhu, esebenza njenge-proxy engasemva, ifumana i-tweet evela kuMthengi.
- Isicelo sithunyelwe kumncedisi we Bhala API ngumncedisi we Web.
- I-Writing API igcina i-tweet kwi-database ye-SQL kumgca wexesha lomsebenzisi.
Inkonzo yeFan-Out iqhagamshelwa yi Bhala API, kwaye yenza le misebenzi ilandelayo.
- Ifumana abalandeli bomsebenzisi kwiCache yeMemori ngokubuza iNkonzo yeGrafu yoMsebenzisi.
- Kwi-Memory Cache, i-tweet igcinwa kumda wexesha wasekhaya wabalandeli bomsebenzisi.
- Abalandeli be-1,000 = i-1,000 yokujonga kunye nokufaka = ukusebenza kwe-O (n).
- I-tweet igcinwe kwiNkonzo yeSalathiso soPhando ukuze ikhangelwe ngokukhawuleza.
- UGcino lweNto lusetyenziselwa ukugcina imidiya.
- Ithumela izilumkiso zokutyhala kubalandeli ngeNkonzo yeSaziso.
- Ukuthumela izilumkiso nge-asynchronously, isebenzisa Umgca.
Singasebenzisa uluhlu lweRedis lwemveli olunesi sakhiwo silandelayo ukuba iMemory Cache yethu yiRedis:
Umgca wexesha lasekhaya lomsebenzisi uya kuhlaziywa kunye ne-tweet entsha, eya kugcinwa kwi-Memory Cache. Siza kusebenzisa i-REST API yoluntu ilandelayo:
Umgca wexesha lomsebenzisi ujongwa ngumsebenzisi.
- Iseva yeWebhu ifumana isicelo somda wexesha lomsebenzisi kuMthengi.
- Isicelo sithunyelwe kwi-Read API iseva yi-Web Server.
- I-Read API ibuza i-SQL Database yexesha lomsebenzisi.
I-REST API iya kusebenza ngokufanayo kumgca wexesha lasekhaya, ngaphandle kokuba zonke iitweets ziya kuvela kumsebenzisi kunabantu ababalandelayo.
Umsebenzisi ukhangela amagama angundoqo:
- Iseva yeWeb ifumana isicelo sokukhangela kuMxumi.
- Isicelo sithunyelwe kwiSeva ye-API yoPhendlo yiSeva yeWebhu.
Inyathelo 4: Ixesha le-Twitter
Ukudala umgca wexesha ngumsebenzi onzima. Umncedisi wokwenza umgca wexesha odibanisa kwiwebhu okanye iiseva zesicelo ziyafuneka.
Ngalo lonke ixesha umsebenzisi esayina, inkonzo yexesha elide igcina umkhondo weetweets ezintsha ezivela kubasebenzisi kwitafile yomlandeli kunye nokuhlaziya okanye ukuhlaziya ixesha lomsebenzisi.
Asisebenzisi naluphi na uhlobo lwenkqubo yokubeka apha; endaweni yoko, sicinga ukuba iitweets ezi-5 eziphezulu ezivela kubalandeli babasebenzisi zinikezelwe kumgca wexesha ngokulandelelana kwexesha lokudala. Singakwazi ukugcina i-50-tweet yokuhlaziya ukucima. Sisayeka ukuhlaziya okanye ukwakha umda wexesha emva kokuba kufikelelwe kumda ade umsebenzisi ahlaziye iphepha.
I-latency ephezulu kunye neenkxalabo zokusebenza ziya kuvela ekudalweni kwabasebenzisi abaphilayo. Endaweni yoko, ukwenza umjelo ongaxhunyiwe kwi-intanethi onokuthi uboniswe ngoko nangoko yeyona ndlela ilungileyo yokuphucula ukusebenza. Qhuba abancedisi bexesha abazinikeleyo abachwetheza umncedisi wesicelo rhoqo ukuhlaziya ukondla ngokusekelwe kwixesha eyenziwe ngayo.
I-algorithm yokubeka inqanaba kufuneka ithathele ingqalelo imiqondiso ebalulekileyo kwaye ibonelele ngobunzima bokuqinisekisa ukuba ixesha lomsebenzisi alilawulwa zizinto ezisuka kwenye okanye ezininzi zeeakhawunti abazilandelayo.
Ngokuchanekileyo, sinokukhetha iimpawu ezinxulumene nokufaneleka kwayo nayiphi na into yokutya, njengenani lokuthandwa, izimvo, izabelo, kunye nexesha lokuhlaziya. Nganye kwezi khrayitheriya kufuneka isetyenziswe ukukala i-tweet, kwaye eso nqanaba kufuneka sisetyenziswe ukubonisa ii-tweets kumgca wexesha.
Ngaba kufuneka sihlale sibalumkisa abasebenzisi xa umxholo omtsha weendaba zabo ufumaneka? Abasebenzisi banokukufumana kuluncedo ukwaziswa xa idatha entsha ifumaneka. Kwizixhobo eziphathwayo, nangona kunjalo, xa ukusetyenziswa kwedatha kubiza kakhulu, kunokumosha i-bandwidth.
Ngenxa yoko, sinokukhetha ukungatyhala idatha kwizixhobo eziphathwayo kwaye endaweni yoko sivumele abasebenzisi ukuba "batsale ukuze bahlaziye" ukuthunyelwa okutsha.
Inyathelo lesi-5: Uyilo lokukala
Into enokuba ngumqobo yiNkonzo yeFanout. Abasebenzisi be-Twitter abanezigidi zabalandeli kuya kufuneka balinde imizuzu emininzi ukuze iitweets zabo ziphume. Oku kunokubangela ugqatso ngeempendulo kwi-tweet, esinokuyiphepha ngoku-odola kwakhona ii-tweets ngexesha lokunikwa.
Sinokuthintela ukusasazwa kweetweets ezivela kubantu abanenani elikhulu labalandeli. Endaweni yoko, sisenokwenza ukhangelo lweetweets ezisuka kubantu abalandelwa kakhulu, sidibanise iziphumo zophendlo kunye neziphumo zomda wexesha lasekhaya labasebenzisi, emva koko sicwangcise ngokutsha iitweets ngexesha lokunikwa.
Izincedisi ezongezelelweyo ziquka:
- Gcina kuphela amakhulu ambalwa eetweets kwiMemori Cache kumda wexesha wasekhaya ngalinye.
- Kwi-Memory Cache, kuphela ulwazi lwexesha lasekhaya lwabasebenzisi abasebenzayo olugciniweyo.
- Singakwazi ukuphinda silwakhe uluhlu lweziganeko kwi-SQL Database ukuba umsebenzisi akazange asebenze kwiintsuku ezingama-30 ezandulelayo.
- Ukufumanisa ukuba ngubani na umsebenzisi, sebenzisa iNkonzo yeGrafu yoMsebenzisi.
- Yongeza i-tweets kwi-Memory Cache ngokuyibuyisela kwi-SQL Database.
- Inkonzo yoLwazi lweTweet inokugcina kuphela iitweets ezixabisa inyanga.
- KwiNkonzo yoLwazi loMsebenzisi, ngabasebenzisi abasebenzayo kuphela abagciniweyo.
- Ukugcina i-latency isezantsi, iQela loPhando liya kufuna kakhulu ukugcina iitweets kwinkumbulo.
isiphelo
Nangona i-Twitter iyintlangano enkulu, inomsebenzi ongcono ukuqonda uyilo lwenkqubo. Ndenze konke okusemandleni am ukukunika isishwankathelo somgangatho ophezulu wexesha le-Twitter.
Ndiyathemba ukuba ufumene ulwazi oluluncedo kuyo kwaye unokuyisebenzisa kakuhle.
Shiya iMpendulo