Does EdgeDB scale horizontally, at least with tables, and are there plans to provide even further horizontal sclaing through real sharding splitting up tables?

[kokoskiwi]

Hi,
e.g I have a Twitter clone with the tables/types "Users" "Tweets" and Biography"

Does EdgeDB offer basic horizontal scaling by making it possible to have each table/type on an own hardware/server? If no, why [are there problems by implementing that? one server per table shouldn't be a problem or not?], and when will it be possible/implemented?

If yes, does EdgeDB offer even real sharding(=horizontal scaling) by splitting up whole tables into multiple hardware/servers/machines (e.g if there are billions of tweets and the type/table doesn't fit into one machine/SSD/HDD anymore)?

If no, are there plans to build this solution? If yes, when?
If no: if my app grows over night extremely big and successful, and one machine can't fit in the tweets-table/type anymore, what shall I do? I think there's nothing I could do than shutting down my app, employing engineers, and develop that real sharding. Or?

[amaster507]

To quote @elprans from discord:

The answer is actually fairly simple: we'll continue to lean into Postgres capabilities there. Postgres today has native sharding, logical replication etc, so our job is to make sure all that integrates nicely into the frontend.

[NikolayS]

Where can I read more about Postgres' native sharding?:)

[kokoskiwi]

Where can I read more about Postgres' native sharding?:)

https://wiki.postgresql.org/wiki/Built-in_Sharding
https://www.highgo.ca/2020/03/12/horizontal-scalability-sharding-in-postgresql-core-missing-pieces-of-the-puzzle/

[NikolayS]

Right but both resources just describe the problem, ideas, and some (very long-lasting) efforts towards solution.

I'm afraid it's far from being solved to say that it's natively supported. But there are many works in progress, of course.

[kokoskiwi]

for anyone interested: the devs are EXTREMELY friendly and helpful
i asked them on discord today after the answer amaster quoted, this:

thanks a lot for the reply!! what do you think how long it will take until sharding will be available? 3-6 months? or a year? or maybe several years?

their reply:

This isn't a concrete ETA yet, but then, again, the hard part of partitioning will be handled by Postgres, so I wouldn't be surprised if this happens within a couple of releases. And before you get scared, we're planning to have reasonably short release cycles 🙂

amasters reply:

I really love the 6 week releases. I did read all of the blogs... like one said, should've had better things to do, but I really enjoyed learning how things lrogressed and seeing some of the early decisions that were made. You learn alot about a company by going back into their history.

So, as I understood now, in like 18 weeks, or better said 4 1/2 months, or 6 months, we might get the horizontal sharding feature :D

[jsimonlane]

Has there been any more news on this -- that is, an updated timeline on the scalability of EdgeDB?

[elprans]

Support for partitioning is planned for a release early next year.

[clungzta]

Any updates on this front? In my view, it would transform EdgeDB from constrained to fully scalable.