RDF Star and scalability

[patstevens4]

Hello! I just discovered your project while researching alternatives to Apache Jena and commercial offerings. I'm coming from an LPG background, and we're evaluating modeling and serving our data in common core.

1. What is the state or roadmap for RDF Star support in qlever?
2. Is any horizontal sharding available today? Or any tools to help shard data correctly across multiple qlever instances? (We have several TB of data, and a quick SWAG that each TB = ~25 billion triples)
3. Any CDC support planned?

[hannahbast]

@1: RDF Star will eventually be supported, but it's not the very next thing on our agenda. I wonder: Since RDF-start is really only syntactic sugar for standard RDF: what's your use case?

@2: Horizontal sharding of data comes with a heavy cost in performance, which should be avoided whenever possible. Therefore, QLever's approach has always been to scale on a single machine. For example, https://qlever.cs.uni-freiburg.de/uniprot handles 200 billion triples on a commodity PC with little RAM. How many triples do you have all in all? What does make sense for high query loads is to replicate the data among multiple machines and QLever does of course support that.

@3: Can you be more specific?

[patstevens4]

Hi Hannah, appreciate the responses!

1. Support our analyst users consuming the data. We've got multi dimensional attributes we need to apply around data access control tagging, data quality/confidence, temporal/location, and just the ability to more compactly apply additional metadata to stuff like edges. We'd prefer not to use named graphs for all of those. Additionally this article claims graphDb saw -40% data volume and response times after modeling their data in RDF*.
2. We don't know how many triples we'll have yet, but we have ~20+ TB of data in a variety of forms coming our way
3. We're interested in the ability to stream data changes from the database for complex event processing. Write Ahead Log (WAL) based Change Data Capture (CDC) is the preferred way that many databases do this.

[patstevens4]

Hi Hannah, appreciate the responses! (moving response to root so hopefully you get notified :)

1. Support our analyst users consuming the data. We've got multi dimensional attributes we need to apply around data access control tagging, data quality/confidence, temporal/location, and just the ability to more compactly apply additional metadata to stuff like edges.
2. We'd prefer not to use named graphs for all of those. Additionally this article claims graphDb saw -40% data volume and response times after modeling their data in RDF*.
3. We don't know how many triples we'll have yet, but we have ~20+ TB of data in a variety of forms coming our way
   We're interested in the ability to stream data changes from the database for complex event processing. Write Ahead Log (WAL) based Change Data Capture (CDC) is the preferred way that many databases do this.

[patstevens4]

Hello, happy new year! Any word on when RDF * support may be added?