Okuqukethwe[Fihla][Bonisa]
Izicelo ze-inthanethi ezinkulu zihambe ibanga elide kule minyaka engamashumi amabili edlule. Lezi zindlela ezintsha ziguqule imibono yethu yokuthuthukiswa kwesofthiwe. I-Facebook, Instagram, ne-Twitter, ngokwesibonelo, zonke ziyizinkundla eziyingozi.
Lezi zinhlelo kumele zakhelwe ukuphatha umthamo omkhulu wethrafikhi nedatha njengoba izigidigidi zabantu zizisebenzisa ngesikhathi esifanayo emhlabeni wonke. Lesi yisikhathi umklamo wesistimu kungena esithombeni.
Inqubo yokusungula i-architecture, interfaces, kanye nedatha yesistimu ehlangabezana nemibandela ethile yaziwa ngokuthi idizayini yesistimu. Ngezinhlelo ezihlangene nezisebenza kahle, ukwakheka kwesistimu kwanelisa izimfuno zebhizinisi noma inhlangano yakho.
Uma inkampani noma inhlangano yakho isinqume izindlela zayo, ungaqala ukuzifaka ekwakhiweni kwesistimu ebonakalayo ehlangabezana nezidingo zabathengi bakho.
Ukuthi ukhetha ukuhamba nokuthuthukiswa kwe-bespoke, izixazululo zentengiso, noma inhlanganisela yakho kokubili, indlela oyiklama ngayo isistimu yakho izonquma ukuthi ulwakha kanjani.
Sizobheka kabanzi ukwakheka kwesistimu yomugqa wesikhathi we-Twitter kulokhu okuthunyelwe, siqedele ngesifundo. Ake siqale.
Isinyathelo 1: Chaza isimo sokusetshenziswa nezingqinamba
Sebenzisa icala
- Umsebenzisi ulayisha i-tweet.
- Isevisi ithumela izaziso zohlelo lokusebenza nama-imeyili kubalandeli bama-tweets.
- Umugqa wesikhathi somsebenzisi uyabukwa (umsebenzi ovela kumsebenzisi)
- Umsebenzisi ubheka umugqa wesikhathi wasekhaya (umsebenzi ovela kubantu abalandelayo umsebenzisi)
- Amagama angukhiye aseshwa ngumsebenzisi.
- Isevisi iyafinyeleleka ngempela.
Ingaphandle kwendawo
- Ama-Tweets athunyelwa ku-Twitter Firehose nakweminye imifudlana esebenzisa le sevisi.
- Isevisi isusa ama-tweets ngokusekelwe kuzilungiselelo zokubonakala komsebenzisi.
- Uma umsebenzisi futhi engalandeli umuntu ophendulwa kuye, fihla impendulo.
- Qaphela inketho 'yokufihla ama-retweets'.
- Ama-Analytics
Izithiyo nokuqagela
Imibono Yesifunda
- Ithrafikhi ayihlakazwanga ngokulinganayo.
- Kufanele kube lula ukuthumela i-tweet.
- Ngaphandle uma unezigidi zabalandeli, ukuthumela i-tweet kubo bonke abalandeli bakho kufanele kusheshe.
- Kunabasebenzisi abasebenzayo abayizigidi eziyi-100.
- Ama-tweets ayizigidi eziyizinkulungwane eziyi-15 inyanga ngayinye noma ama-tweets ayizigidi ezingama-500 nsuku zonke
- I-tweet ngayinye ine-fanout yokulethwa kweziyi-10 ngokwesilinganiso.
- Nsuku zonke, i-fanout iletha ama-tweets ayizigidi eziyizinkulungwane ezingu-5.
- UFanout uletha ama-tweets ayizigidi eziyizinkulungwane eziyi-150 njalo ngenyanga.
- 250 billion izicelo zokufunda njalo ngenyanga
- 10 billion ukusesha kwanyanga zonke
esibekelwe
- Umugqa wesikhathi kufanele ube lula ukuwusebenzisa.
- I-Twitter imayelana nokufunda kunokubhala.
- Lungiselela ukufunda i-tweet ngokushesha
- Ukusetshenziswa kwe-Tweet kudla isikhathi.
Ukucinga
- Inqubo yokusesha kufanele isheshe.
- Kudla isikhathi ukusesha.
Bala ukusetshenziswa
Usayizi we-tweet ngayinye:
- 8 bytes tweet id
- 32 bytes umsebenzisi-id
- Amabhayithi angu-140 ombhalo
- imidiya – isilinganiso esingu-10 KB
- Isamba: ~10 KB
Njalo ngenyanga, kukhiqizwa u-150 TB wokuqukethwe okusha kwe-tweet.
- * Ama-tweets ayizigidi ezingama-500 nsuku zonke * izinsuku ezingama-30 ngenyanga * 10 KB ku-tweet ngayinye
- Eminyakeni emithathu, kube ne-5.4 PB yokuqukethwe okusha kwe-tweet.
Kunezicelo ezifundwayo eziyi-100,000 umzuzwana ngamunye.
- * (Izicelo ezingama-400 ngomzuzwana / izicelo eziyizigidi eziyizinkulungwane eziyi-1 ngenyanga) funda izicelo eziyizigidi eziyizinkulungwane ezingama-250 inyanga ngayinye
Kunama-tweets angu-6,000 umzuzwana ngamunye.
- * (Izicelo ezingama-400 ngomzuzwana / izicelo eziyizigidi eziyizinkulungwane eziyi-1 ngenyanga) ama-tweets ayizigidi eziyizinkulungwane eziyi-15 njalo ngenyanga
Ku-fanout, kuthunyelwa ama-tweets ayizinkulungwane ezingama-60 njalo ngomzuzwana.
- UFanout uletha ama-tweets ayizigidi eziyizinkulungwane eziyi-150 inyanga ngayinye* (izicelo ezingama-400 ngomzuzwana / izicelo eziyisigidigidi esi-1 ngenyanga).
Izicelo ezingu-4,000 zolwazi njalo ngomzuzwana
- * (Izicelo ezingu-400 ngomzuzwana / izicelo eziyizigidi eziyizinkulungwane ezingu-1 ngenyanga) ukusesha okuyizigidi eziyizinkulungwane ezingu-10 inyanga ngayinye
Ukuguqulwa okuthile okuwusizo
- Nyanga zonke, kudlula imizuzwana eyizigidi ezingu-2.5.
- Izicelo eziyizigidi ezi-2.5 ngenyanga ngesicelo esisodwa ngomzuzwana
- Izicelo eziyizigidi eziyi-100 ngenyanga x izicelo ezingama-40 ngomzuzwana
- Izicelo eziyizigidi eziyizinkulungwane eziyi-1 ngenyanga = izicelo ezingama-400 ngomzuzwana
Isinyathelo sesi-2: Umdwebo wezinga eliphezulu
Isinyathelo sesi-3: Ukuchaza izingxenye ezibalulekile
Singagcina ama-tweets omsebenzisi ukuze agcwalise umugqa wesikhathi somsebenzisi (umsebenzi ovela kumsebenzisi) kusizindalwazi esihlobene uma behambisa i-tweet. Kunzima kakhulu ukuletha ama-tweets nokuthuthukisa umugqa wesikhathi wasekhaya (umsebenzi ovela kubantu abalandelayo umsebenzisi).
Isizindalwazi esijwayelekile sobudlelwano singakhungatheka ngokusakaza ama-tweets kubo bonke abalandeli (ama-tweets ayizinkulungwane ezingama-60 alethwa umzuzwana ngamunye). Cishe sizofuna ukuhamba nesitoreji sedatha esibhala ngokushesha njengesizindalwazi se-NoSQL noma i-Memory Cache.
Ukufunda u-1 MB ngokulandelana kwenkumbulo kuthatha cishe ama-microseconds angu-250, kodwa ukufunda ku-SSD kuthatha izikhathi ezi-4 ubude, futhi ukufunda kudiski kuthatha izikhathi ezingu-80 ubude.
Isitolo Sezinto singasetshenziswa ukugcina idatha efana nezithombe namavidiyo.
- I-Web Server, esebenza njenge-proxy ehlehlayo, ithola i-tweet evela ku-Client.
- Isicelo sithunyelwa kuseva ye-Writ API yi-Web Server.
- I-Writ API igcina i-tweet kusizindalwazi se-SQL emugqeni wesikhathi somsebenzisi.
Isevisi ye-Fan-Out ithintwa yi-Writ API, futhi yenza imisebenzi elandelayo.
- Ithola abalandeli bomsebenzisi Kunqolobane Yenkumbulo ngokubuza Isevisi Yegrafu Yomsebenzisi.
- Kunqolobane Yenkumbulo, i-tweet igcinwa emugqeni wesikhathi wasekhaya wabalandeli bomsebenzisi.
- 1,000 abalandeli = 1,000 lookups and inserts = O(n) operation.
- I-tweet igcinwa ku-Search Index Service ukuze iseshwe ngokushesha.
- Isitolo Sezinto sisetshenziselwa ukugcina imidiya.
- Ithumela izexwayiso zohlelo lokusebenza kubalandeli ngesevisi Yezaziso.
- Ukuthumela izexwayiso ngendlela efanayo, isebenzisa Umugqa.
Singasebenzisa uhlu lwendabuko lwe-Redis ngesakhiwo esilandelayo uma i-Memory Cache yethu i-Redis:
Umugqa wesikhathi wasekhaya womsebenzisi uzobuyekezwa nge-tweet entsha, ezogcinwa Kunqolobane Yenkumbulo. Sizosebenzisa i-REST API elandelayo yomphakathi:
Umugqa wesikhathi somsebenzisi ubukwa ngumsebenzisi.
- Iseva Yewebhu ithola isicelo somugqa wesikhathi somsebenzisi esivela Kuklayenti.
- Isicelo sithunyelwa kuseva ye-Read API yi-Web Server.
- I-Read API ibuza i-SQL Database yesikhathi somsebenzisi.
I-REST API izosebenza ngokufanayo kumugqa wesikhathi wasekhaya, ngaphandle kokuthi wonke ama-tweets azosuka kumsebenzisi kunabantu abalandelayo.
Umsebenzisi usesha amagama angukhiye:
- Iseva Yewebhu ithola isicelo sokucinga esivela Eklayentini.
- Isicelo sithunyelwa kuseva ye-Search API yi-Web Server.
Isinyathelo sesi-4: Umugqa wesikhathi we-Twitter
Ukudala umugqa wesikhathi kuwumsebenzi onzima. Iseva ekhiqiza umugqa wesikhathi exhuma kuwebhu noma amaseva ohlelo lokusebenza iyadingeka.
Ngaso sonke isikhathi uma umsebenzisi engena ngemvume, isevisi yomugqa wesikhathi igcina ithrekhi yama-tweets amasha kakhulu avela kubasebenzisi kuthebula lomlandeli futhi ibuyekeza noma ivuselele umugqa wesikhathi somsebenzisi.
Asisebenzisi noma yiluphi uhlobo lwesistimu yezinga lapha; esikhundleni salokho, sithatha ngokuthi ama-tweets angu-5 aphezulu avela kubalandeli bomsebenzisi ethulwa emugqeni wesikhathi ngokulandelana kwesikhathi sokudala. Singagcina ukunqanyulwa kokuvuselela kwama-tweets angama-50. Sisayeka ukuvuselela noma ukwakha umugqa wesikhathi ngemva kokuba lowo mkhawulo usufinyelelwe kuze kube yilapho umsebenzisi evuselela ikhasi.
Ukubambezeleka okuphezulu nokukhathazeka kokusebenza kuzovela ekudaleni okuphakelayo komsebenzisi bukhoma. Kunalokho, ukudala ukusakaza okungaxhunyiwe ku-inthanethi okungase kwethulwe ngokushesha kuyindlela engcono kakhulu yokuthuthukisa ukusebenza. Qalisa amaseva omugqa wesikhathi azinikezele acindezela iseva yohlelo lokusebenza njalo ukuze uvuselele okuphakelayo ngokusekelwe esikhathini esidalwe ngaso.
I-algorithm yezinga kufanele icabangele amasignali abalulekile futhi inikeze isisindo sokuqinisekisa ukuthi umugqa wesikhathi somsebenzisi awubuswa izinto ezivela ku-akhawunti eyodwa noma amaningi awalandelayo.
Ngokunemba kakhudlwana, singakhetha izici ezihlobene nokufaneleka kwanoma iyiphi into ephakelayo, njengenombolo yokuthandwa, amazwana, ukwabelana, nesikhathi sokubuyekeza. Ngayinye yalezi zindlela kufanele isetshenziselwe ukukala i-tweet, bese lelo zinga kufanele lisetshenziswe ukukhombisa ama-tweets emugqeni wesikhathi.
Ingabe kufanele sihlale sixwayisa abasebenzisi uma okuqukethwe okusha kwezindaba zabo kutholakala? Abasebenzisi bangakuthola kuzuzisa ukwaziswa lapho idatha entsha itholakala. Kumadivayisi eselula, nokho, lapho ukusetshenziswa kwedatha kubiza kakhulu, kungamosha umkhawulokudonsa.
Njengomphumela, singakhetha ukungaphusheli idatha kumadivayisi eselula futhi esikhundleni salokho sivumele abasebenzisi ukuthi "Badonse ukuze Uvuselele" ngokuthunyelwe okusha.
Isinyathelo sesi-5: Idizayini yokukala
Okungase kube yinkinga yiFanout Service. Abasebenzisi be-Twitter abanezigidi zabalandeli kuzodingeka balinde imizuzu embalwa ukuze ama-tweets abo akhishwe. Lokhu kungase kubangele umjaho onezimpendulo ku-tweet, esingagwema ngoku-oda kabusha ama-tweets ngesikhathi sokunikezwa.
Singakwazi futhi ukuvimbela ukusabalalisa ama-tweets avela kubantu abanenani elikhulu labalandeli. Kunalokho, singase siseshe ama-tweets avela kubantu abalandelwa kakhulu, sihlanganise imiphumela yosesho nemiphumela yomugqa wesikhathi wasekhaya wabasebenzisi, bese sihlela kabusha ama-tweets ngesikhathi sokunikezwa.
Izithuthukisi ezengeziwe zifaka:
- Gcina kuphela amakhulu ambalwa ama-tweets Kunqolobane Yenkumbulo ngomugqa wesikhathi wasekhaya ngamunye.
- Kunqolobane Yenkumbulo, kulondolozwa kuphela ulwazi lomugqa wesikhathi wasekhaya wabasebenzisi abasebenzayo.
- Singakwazi ukwakha kabusha ukulandelana kwezikhathi kusuka Kusizindalwazi se-SQL uma umsebenzisi ubengasebenzanga ezinsukwini ezingama-30 ezandulele.
- Ukuze uthole ukuthi umsebenzisi ungubani, sebenzisa Isevisi Yegrafu Yomsebenzisi.
- Engeza ama-tweets ku-Memory Cache ngokuwakhipha ku-SQL Database.
- Isevisi ye-Tweet Info ingonga kuphela ama-tweets anenani lenyanga.
- Kusevisi yolwazi lomsebenzisi, abasebenzisi abasebenzayo kuphela abalondoloziwe.
- Ukuze ugcine ukubambezeleka kuphansi, Iqoqo Lokusesha lingadinga kakhulu ukugcina ama-tweets enkumbulweni.
Isiphetho
Nakuba i-Twitter iyinhlangano enkulu, inengcono ukuqonda ukwakheka kwesistimu. Ngenze konke okusemandleni ami ukukunikeza umbono wezinga eliphezulu womugqa wesikhathi we-Twitter.
Ngethemba ukuthi uthole ulwazi oluwusizo kuyo futhi ungalisebenzisa ngokunenzuzo.
shiya impendulo