Table of Contents[Hide][Show]
Instagram Pub yog lub platform rau kev sib qhia thiab txuas nrog tib neeg thiab yam tseem ceeb rau koj. Thaum koj qhib Instagram lossis rov kho koj cov khoom noj, cov duab thiab cov yeeb yaj kiab uas peb xav tias koj yuav txaus siab yuav tshwm rau sab saum toj.
Cov xov xwm pub yog ib phau ntawm cov khoom uas muaj cov ntawv nyeem, cov duab, lossis cov yeeb yaj kiab uas tsim los ntawm lwm lub koom haum hauv qhov system uas yog tsom rau koj nyeem. Nws ib txwm hloov pauv, thaum lwm lub koom haum tab tom ua cov ntawv tshiab.
Hauv cov ntawv tshaj tawm no, peb yuav saib ze rau qhov tsim qauv ntawm Instagram cov khoom noj. Yog li, cia peb pib.
1. Yuav tsum tau ua
Kev tsim nyog ua haujlwm
- Tus neeg siv xov xwm pub yog tsim los ntawm cov ntawv tshaj tawm los ntawm lwm qhov chaw hauv qhov system uas tus neeg siv tau ua raws lossis nyiam.
- Cov ntawv nyeem, duab, thiab yeeb yaj kiab tuaj yeem pom hauv cov ntawv tshaj tawm.
- Tus neeg siv xov xwm pub yuav tsum tau hloov kho nrog cov ntawv tshaj tawm tshiab tsim los ntawm lwm tus.
Cov txheej txheem tsis ua haujlwm
- Kev tsim cov ntawv xov xwm yuav tsum tshwm sim hauv lub sijhawm tiag tiag. Cov neeg siv kawg yuav tsum tau ntsib tsuas yog 12 vib nas this ntawm kev ncua.
- Ntxiv ib tsab xov xwm tshiab: Nws yuav tsum siv sijhawm tsis pub dhau 5 vib nas this rau qhov kev tshaj tawm tshiab kom tshwm sim hauv xov xwm pub thov tom qab nws xa mus rau qhov system.
2. Kev kwv yees muaj peev xwm
- Raws li lub Peb Hlis 2021, lub ntiaj teb cov pej xeem tsuas yog 7.8 billion tus neeg. Nws qhia tias 21% ntawm lub ntiaj teb cov pej xeem yog Facebook DAU (Cov Neeg Siv Txhua Hnub) thiab 32% yog Facebook MAU (Monthly Active User) (Lub hli Active User). Uas yog amazing.
- Cia peb ua piv txwv qhov system peb tab tom tsim muaj 1 billion DAU los ua kom yooj yim dua.
- Xav tias ib tug neeg ua raws 500 tus neeg lossis kev lag luam hauv Facebook. Ib pawg lossis nplooj ntawv yuav raug suav hais tias yog ib qho chaw.
Kwv yees tsheb
Xav tias ib tus neeg siv rub tawm cov xov xwm pub 10 zaug txhua hnub nyob rau nruab nrab. Yog li nws yog kwv yees li 116K QPS thiab 1e10 thov txhua hnub.
Kev kwv yees ntawm Cia
Xav tias peb khaws 500 cov lus los ntawm txhua tus neeg siv xov xwm pub rau hauv lub cim xeeb ntawm qhov nruab nrab kom rov qab tau sai, thiab txhua qhov tshaj tawm yog 1KB loj. Yog li 500 KB rau ib tus neeg siv, 500 TB rau tag nrho DAUs, thiab 5000 computers nrog 100 GB RAM txhua.
3. APIs rau Systems
userId (GUID): tus neeg siv uas nws cov xov xwm pub tau txais.
Cov teb hauv qab no muaj nyob rau hauv cov kev xaiv xaiv parameter:
- afterPostId (GUID): tau txais cov xov xwm pub los ntawm tus ncej tom qab no. Yog tias tsis tau teev tseg, tau txais cov ntawv tshaj tawm tsis ntev los no.
- suav (tus naj npawb): ntau tshaj ntawm cov ntawv tshaj tawm uas txhua qhov kev thov tuaj yeem rov qab los. Lub backend teeb tsa tus lej siab tshaj plaws yog tias tsis muaj qhov nkag.
- excludeReplies (boolean): tiv thaiv cov lus teb los ntawm kev suav nrog hauv xov xwm pub.
- JSON xa rov qab muaj cov npe ntawm cov xov xwm pub khoom.
4. Tsim ib lub database
Qhov chaw
- cov neeg siv
- entityId, lub npe, lus piav qhia, thiab lub sijhawm teev tseg yog txhua qhov yuav tsum tau ua.
- Cov nram qab no yuav tsum muaj: PostId, lub npe, ntawv nyeem, authorId, thiab timestamp.
- timestamp, url, thiab mediaId
kev sib raug zoo
- Lwm cov neeg siv lossis cov koom haum tuaj yeem ua raws li tus neeg siv. (m:n)
- Sau-Post: Ob tus neeg siv thiab cov koom haum tuaj yeem tsim cov ntawv tshaj tawm. Nco ntsoov tias tsuas yog Cov Neeg Siv Khoom tuaj yeem tsim Cov Ntawv rau lub hom phiaj ntawm kev yooj yim. (1:n; authorId yog embeddable).
- Txhua tus ncej yog nrog los ntawm qee hom kev tshaj tawm. (1:n)
5. Kev Tsim Kho Siab
Architecture
Workflows
Pub ntau lawm
Thaum Jay thov nws cov kwj dej, lub kaw lus yuav ua cov hauv qab no:
- Rov qab tau cov IDs ntawm txhua tus neeg thiab tej yam uas Jay ua raws.
- Aggregate posts: muab cov IDs, tau txais cov ntawv tshaj tawm tsis ntev los no, nrov, thiab muaj feem cuam tshuam.
- Muab cov ntawv teev npe raws li lawv qhov cuam tshuam thiab sijhawm.
- Cache: txuag cov khoom tsim thiab xa Jay rau saum 20 cov ntawv tshaj tawm.
- Thaum Jay tau nyeem thawj 20 nqe lus tiav lawm, lwm qhov kev thov raug xa mus kom tau txais 20 cov lus tom ntej.
Pub kev faib tawm
Xav tias Jay tab tom ua raws Aayush thiab tias Aayush tshaj tawm txhua yam tshiab. Jay's xov xwm pub yuav tsum tau hloov kho los ntawm qhov system:
- Rov qab tau cov IDs ntawm Aayush cov thwjtim.
- Ntxiv cov ntawv tshiab: Ntxiv Aayush cov ntawv tshaj tawm rau cov xov xwm pub ntawm cov neeg ua raws ID.
- Muab cov ntawv teev npe raws li lawv qhov cuam tshuam thiab sijhawm.
- Hloov kho lub cache ntawm qhov qeb duas.
- Cov neeg ua raws yuav tsum tau ceeb toom thaum cov ntawv tshaj tawm tshiab.
cheebtsam
Cov neeg siv kev sib txuas yog tswj hwm los ntawm lub vev xaib servers.
Cov txheej txheem qhia saum toj no yog ua los ntawm daim ntawv thov server.
Cache thiab database:
- Relational database neeg siv / chaw
- Relational database (post)
- Duab/video attribute: Aayush storge
- Relational database metadata
Cov kev pabcuam tus kheej:
- Pub ntau lawm
- Kev ceeb toom ntawm feeds
6. Tsim kom ntxaws
Tiam ntawm pub
Ntxuam tawm nyeem naive siv:
Cov teeb meem nrog qhov kev siv tsis zoo no suav nrog:
- Cov neeg siv nrog coob tus phooj ywg / cov neeg ua raws yuav pom qhov kev poob qis vim tias peb yuav tsum sib koom ua ke, sib koom ua ke, thiab qeb ntau ntawm kev tshaj tawm.
- Thaum tus neeg siv thauj lawv nplooj ntawv, peb tsim lub sijhawm. Qhov no tuaj yeem ua rau qeeb thiab muaj ntau latency.
- Txhua qhov xwm txheej hloov tshiab yuav ua rau muaj kev hloov pauv tshiab rau txhua tus neeg ua raws rau qhov hloov tshiab nyob. Qhov no tuaj yeem ua rau qeeb qeeb hauv peb qhov Kev Pabcuam Newsfeed Generation.
Peb tuaj yeem tsim cov sijhawm ua ntej thiab khaws cia hauv lub cim xeeb kom ua tau zoo.
Offline Production (Fan-out sau)
Peb tuaj yeem muaj cov servers tshwj xeeb uas niaj hnub tsim thiab khaws cov neeg siv cov ntawv xov xwm hauv lub cim xeeb. Peb tuaj yeem xa cov xov xwm pub dawb los ntawm qhov tsim ua ntej, qhov chaw khaws tseg thaum twg tus neeg siv xav tau.
Muaj pes tsawg cov khoom pub rau tus neeg siv khoom noj yuav tsum khaws cia hauv lub cim xeeb?
Kho raws li koj tus cwj pwm siv.
Peb puas yuav tsum ua cov ntawv xov xwm rau txhua tus neeg siv (thiab khaws cia hauv nco)?
- Rau cov neeg uas tsis nkag mus ntau zaus.
- LRU-based caching yog ib txoj hauv kev yooj yim.
- Ib qho kev daws teeb meem zoo dua yog xav paub seb cov neeg siv nkag mus li cas. Thaum twg? Hnub twg koj tham txog?
Tshaj tawm ntawm pub
Fanout yog tus txheej txheem xa ntawv mus rau tag nrho koj cov thwjtim.
Fanout nyeem (rub)
Thaum koj thov xov xwm pub, lub kaw lus tau txais kev thov nyeem. Fanout nyeem xa ib daim ntawv thov nyeem rau tag nrho koj cov thwjtim, thov kom lawv nyeem lawv cov ntsiab lus.
pros:
- Cov txheej txheem ntawm kev sau ntawv yog pheej yig.
- Thaum nyeem cov ntaub ntawv, nws yooj yim dua los siv ntau yam aggregation algorithms.
cons:
- Rau ib tus neeg uas muaj ntau cov thwjtim, kev nyeem ntawv yog qhov kim heev.
- Cov neeg siv yuav tsis pom cov ntaub ntawv tshiab kom txog thaum lawv rub nws.
- Thaum peb rub mus nqa cov ntawv tshaj tawm tsis ntev los no tsis tu ncua, nws nyuaj rau nrhiav qhov kev rub tawm kom raug, thiab feem ntau rub cov lus thov yuav rov qab los teb qhov khoob, nkim peev txheej.
Fanout sau (push)
Kev thov sau yog ua rau lub kaw lus thaum koj xa ib tsab ntawv tshiab. Cov ntawv thov raug xa tawm mus rau tag nrho koj cov thwjtim los hloov kho lawv cov ntawv xov xwm siv fanout sau.
ib
- Cov txheej txheem nyeem yog pheej yig.
con
- Rau ib tus neeg siv nrog ntau lab tus followers, lub sau ntawv txheej txheem yog kim heev.
Qhov qeb duas ntawm lub Feed
Tsis yog tsuas yog xaj cov khoom noj raws sijhawm, hnub no cov txheej txheem qeb duas kuj tseem sim ua kom ntseeg tau tias cov khoom uas muaj feem cuam tshuam ntau dua yog qhov tseem ceeb.
- Xaiv yam uas tuaj yeem pab koj txiav txim siab qhov cuam tshuam ntawm cov khoom noj, xws li tus naj npawb ntawm cov nyiam, cov lus pom, thiab kev sib koom, lub sijhawm cov khoom tau hloov kho zaum kawg yog tias tsab xov xwm muaj cov duab lossis yeeb yaj kiab, thiab lwm yam.
- Xam cov qhab nia raws li tus yam ntxwv.
- Siv cov qhab nia los qeb duas cov ntawv.
Teem KPIs zoo li cov neeg siv khaws cia, cov nyiaj tau los tshaj tawm, thiab lwm yam kom pom tias peb qhov kev ua haujlwm zoo li cas.
xaus
Txawm tias muaj tseeb tias Instagram lossis nws niam txiv ua lag luam Facebook yog lub tuam txhab loj, nws muaj kev nkag siab zoo dua tsim qauv.
Kuv sim kuv qhov nyuaj tshaj plaws los muab koj cov ntsiab lus siab ntawm Instagram pub.
Kuv vam tias nws yuav pab tau thiab koj yuav muab nws siv tau zoo.
Sau ntawv cia Ncua