What happens to data lost during the 30 minute recovery window for a shard if transactions have happened and money has been exchanged but theres no longer a record or order id in the dB?
You are saying that while copying Shop A from one shard to another, they are write-locking the records that are part of the current batch that's being copied over while also keeping a binlog to capture recent updates on these records like a CDC (Change Data Capture) process.
If they are locking these transactions, then what happens during a sale surge for an item that has only 1 unit left? Are people locked out of the payment checkout process until the copy is complete? If the transactions are write-locked, then what's the point of having a binlog for this record if it's not gonna be able to capture recent changes? Having a binlog and write-locking seem to be mutually exclusive.
Can you clarify what happens to this final unit when multiple people are fighting over its purchase during a surge sale while the record(s) for this item is being copied over to another shard?
It's just a unique identifier, so data can be split between database instances, and moved from one instance to another (aka, shards)
Each shard can contain one or more shops, so if there are 2 shops, with their identifiers as myshop12 and yourshop23. You would have tables like myshop12_plugins myshop12_orders, yourshop23_plugins, yourshop23_orders
When the tables with the prefix name of "yourshop23_" get larger than some threshold, a new database "shard" is created, all the shop data gets copied over into its own shard.
Many shops may start off as sharing the same shard, till certain requirements are met, then they get their own shards
What happens to data lost during the 30 minute recovery window for a shard if transactions have happened and money has been exchanged but theres no longer a record or order id in the dB?
You are saying that while copying Shop A from one shard to another, they are write-locking the records that are part of the current batch that's being copied over while also keeping a binlog to capture recent updates on these records like a CDC (Change Data Capture) process.
If they are locking these transactions, then what happens during a sale surge for an item that has only 1 unit left? Are people locked out of the payment checkout process until the copy is complete? If the transactions are write-locked, then what's the point of having a binlog for this record if it's not gonna be able to capture recent changes? Having a binlog and write-locking seem to be mutually exclusive.
Can you clarify what happens to this final unit when multiple people are fighting over its purchase during a surge sale while the record(s) for this item is being copied over to another shard?
Very interesting- GCP-MySQL- Replication 👍
The last line of the [Read Consistency with Database Replication](https://blog.bytebytego.com/i/148705574/read-consistency-with-database-replication) should use the word "inconsistencies" instead of "consistencies".
"Its main drawback was that intermittent server outages can introduce read `consistencies` but this tradeoff was acceptable to them."
Very interesting
1. How are related reads detected? Is it devs' headache to pass a correct consistent_read_id to each query?
2. How do they prepare to BF each year?
3. Where do they store and how do they serve images uploaded by the tenants? GCS + CDN or something trickier?
Good insight
Didn't quiet get the shop_id shard concept
It's just a unique identifier, so data can be split between database instances, and moved from one instance to another (aka, shards)
Each shard can contain one or more shops, so if there are 2 shops, with their identifiers as myshop12 and yourshop23. You would have tables like myshop12_plugins myshop12_orders, yourshop23_plugins, yourshop23_orders
When the tables with the prefix name of "yourshop23_" get larger than some threshold, a new database "shard" is created, all the shop data gets copied over into its own shard.
Many shops may start off as sharing the same shard, till certain requirements are met, then they get their own shards