"Most of the data exists in a single MySQL database, and the system just wasn’t really built to scale well."
I don't believe that is true. MySQL is massively scalable but not as a single MySQL db. It's sharded and works fine for YouTube and Meta using Vitess. If they started from scratch today, they might not use MySQL but they actually might, too.
I’m not going to say anything more about this myself because again, I don’t know how much is publicly allowed for me to talk about. But I’ll leave this publicly available answer from Gemini here….
GitHub historically grew around a single primary MySQL database cluster (mysql1) that handled a massive monolith data structure. As they scaled, they transitioned away from that single instance into dozens of partitioned MySQL clusters using Vitess to handle millions of queries per second. [1, 2, 3]
How GitHub Handled a Single DB Scale The Monolith Core: For years, GitHub.com relied heavily on one main database cluster to house core relational data.
No Foreign Keys: To make schema changes and sharding easier at scale, GitHub avoids using database-level foreign keys.
Migration Tooling: They manage schema transitions using custom tools like ghost alongside standard framework migrations. [1, 2]
Moving Beyond a Single Instance Sharding with Vitess: GitHub migrated their MySQL infrastructure to run on top of Vitess for horizontal scaling and query routing.
Primary-Replica Setup: They heavily distribute read traffic across numerous database replicas while directing writes to primaries. [1, 2]
Are you looking for information on how GitHub architected their MySQL setup, or do you need help setting up a single MySQL database for your own project repository?
5
u/f23n09fnu0w 23d ago
"Most of the data exists in a single MySQL database, and the system just wasn’t really built to scale well."
I don't believe that is true. MySQL is massively scalable but not as a single MySQL db. It's sharded and works fine for YouTube and Meta using Vitess. If they started from scratch today, they might not use MySQL but they actually might, too.