Absolute form conversion #3378

jnsrnhld · 2024-04-08T12:39:40Z

No description provided.

jnsrnhld

In der README hab ich nochmal versucht zusammenzufassen, wie der Prozess in SQL funktioniert. Ich hoffe das hilft etwas.

backend/src/main/java/com/bakdata/conquery/apiv1/query/EditorQuery.java

backend/src/main/java/com/bakdata/conquery/sql/conversion/cqelement/CQExternalConverter.java

thoniTUB

Danke für die ReadMe, das hat gut geholfen

...end/src/main/java/com/bakdata/conquery/apiv1/query/concept/specific/external/CQExternal.java

backend/src/main/java/com/bakdata/conquery/sql/conversion/dialect/HanaSqlFunctionProvider.java

backend/src/main/java/com/bakdata/conquery/sql/conversion/forms/AbsoluteStratification.java

.../main/java/com/bakdata/conquery/sql/conversion/forms/PostgresStratificationTableFactory.java

backend/src/main/java/com/bakdata/conquery/sql/conversion/forms/README.md

backend/src/main/java/com/bakdata/conquery/sql/conversion/model/QueryStepJoiner.java

backend/src/test/resources/tests/sql/form/ABSOLUT/ALIGNMENT/YEAR QUARTER.json

backend/src/test/resources/tests/sql/form/ABSOLUT/ALIGNMENT/year quarter expected.csv

This reverts commit 55c33eb.

thoniTUB · 2024-04-10T06:40:25Z

backend/src/test/resources/tests/sql/form/ABSOLUT/ALIGNMENT/year_quarter_expected.csv

+result,resolution,index,date_range,Alter
+1,complete,,2012-06-16/2014-12-17,57
+1,year,1,2012-06-16/2013-03-31,56
+1,year,2,2013-04-01/2014-03-31,
+1,year,3,2014-04-01/2014-12-17,


Danke dir! Stratifizierung und Altersberechnung sehen korrekt aus

backend/src/main/java/com/bakdata/conquery/sql/conversion/forms/StratificationTableFactory.java

…s/README.md Co-authored-by: Torben Meyer <torben.meyer@bakdata.com>

…ect/HanaSqlFunctionProvider.java Co-authored-by: Torben Meyer <torben.meyer@bakdata.com>

jnsrnhld · 2024-04-15T11:59:27Z

@awildturtok @thoniTUB Wir sind bei dem Alignment DAY und der EntityDate Query an die Grenzen unseres Ansatzes gestoßen und haben nochmal einen neuen Ansatz ausgearbeitet. Damit ändert sich die Art und Weise, wie wir die Stratifizierungstabellen erstellen nochmal deutlich. Die README enthält dazu auch die aktualisierten Beispiele.

…sion' into sql/feature/absolute-form-conversion

awildturtok

Ich hab mir noch nicht alles angeschaut aber mir raucht jetzt erstmal der Kopf

backend/src/main/java/com/bakdata/conquery/models/error/ErrorMessages.java

backend/src/main/java/com/bakdata/conquery/sql/conversion/cqelement/CQExternalConverter.java

...rc/main/java/com/bakdata/conquery/sql/conversion/cqelement/concept/AggregationSelectCte.java

backend/src/main/java/com/bakdata/conquery/sql/conversion/cqelement/concept/EventFilterCte.java

backend/src/main/java/com/bakdata/conquery/sql/conversion/forms/StratificationFunctions.java

backend/src/main/java/com/bakdata/conquery/sql/conversion/model/LogicalOperation.java

awildturtok · 2024-04-22T13:12:57Z

backend/src/main/java/com/bakdata/conquery/sql/conversion/model/ColumnDateRange.java

+		if (this.isSingleColumnRange() != right.isSingleColumnRange()) {
+			throw new UnsupportedOperationException("Can only join ColumnDateRanges of same type");
+		}


in Hana können wir ja eh keine singleColumnRanges machen oder? Können wir dann vlt einen Standard erzwingen? Also in Hana müssen es compounds sein, in psql müssen es singleColumnRanges sein

Siehe oben, können wir gerne darüber sprechen weil mir das auch nicht gefällt wie es aktuell ist, aber sehe ich nicht im Rahmen dieses PRs, weil das mMn. sehr viele Änderungen wären.

backend/src/test/java/com/bakdata/conquery/integration/IntegrationTest.java

awildturtok · 2024-04-23T10:31:06Z

...end/src/main/java/com/bakdata/conquery/sql/conversion/forms/HanaStratificationFunctions.java

+		);
+	}
+
+	private Field<Integer> getQuartersInMonths(Field<Date> date, int quarterOffset) {


monthsInQuarters

backend/src/main/java/com/bakdata/conquery/sql/conversion/forms/README.md

awildturtok · 2024-04-23T10:42:03Z

backend/src/main/java/com/bakdata/conquery/sql/conversion/model/QueryStepJoiner.java

@@ -24,7 +26,7 @@ public class QueryStepJoiner {
 	public static QueryStep joinChildren(
 			Iterable<CQElement> children,
 			ConversionContext context,
-			LogicalOperation logicalOperation,
+			com.bakdata.conquery.sql.conversion.model.JoinType logicalOperation,


warum fully Qualified?

hm, jetzt hast du zwei Klassen, die Join-Type heißen und hier ein 1:1 mapping davon, magst du die vlt zusammenziehen?

Hab es nochmal umbenannt: wir hatten die Klasse damals eingeführt und so benannt, weil es im Grund ein Subset vom JOOQ Enum JoinType ist. Deswegen auch der lange Qualifier an der einen Stelle.

awildturtok · 2024-04-23T11:07:55Z

backend/src/main/java/com/bakdata/conquery/sql/conversion/query/AbsoluteFormQueryConverter.java

+		if (queriesToJoin.size() == 1) {
+			QueryStep convertedFeature = queriesToJoin.get(0);
+			return createFinalSelect(form, stratificationTable, convertedFeature, childContext);
+		}


sowas ist gerne mal quelle für bugs, kann der code unten explizit nicht mit N=1 umgehen?

Ich strukturiere das nochmal um, sodass der Check in der Method selbst stattfindet. Also joinSteps() joined nur, wenn size > 1, ansonsten wird der einzige Step zurückgegeben.

…s/README.md Co-authored-by: awildturtok <1553491+awildturtok@users.noreply.github.com>

jnsrnhld · 2024-04-23T14:38:14Z

@thoniTUB Ich musste nochmal dein Review anfordern, andernfalls kann ich nicht mergen wegen den requested changes 😄

thoniTUB

@jnsrnhld Alles klar :)

jnsrnhld added backend Project SQL labels Apr 8, 2024

Implement absolute form conversion

941c5f4

jnsrnhld force-pushed the sql/feature/absolute-form-conversion branch from c8d8831 to 941c5f4 Compare April 8, 2024 12:44

jnsrnhld commented Apr 8, 2024

View reviewed changes

backend/src/main/java/com/bakdata/conquery/apiv1/query/EditorQuery.java Outdated Show resolved Hide resolved

backend/src/main/java/com/bakdata/conquery/sql/conversion/cqelement/CQExternalConverter.java Show resolved Hide resolved

jnsrnhld requested a review from awildturtok April 8, 2024 13:37

awildturtok requested a review from thoniTUB April 8, 2024 13:39

thoniTUB requested changes Apr 9, 2024

View reviewed changes

jnsrnhld added 2 commits April 9, 2024 14:13

Cleanup

389a4fe

Fix alignment

5c5cc49

thoniTUB reviewed Apr 9, 2024

View reviewed changes

backend/src/test/resources/tests/sql/form/ABSOLUT/ALIGNMENT/year quarter expected.csv Outdated Show resolved Hide resolved

jnsrnhld added 2 commits April 9, 2024 21:08

No-Op commit

55c33eb

Revert "No-Op commit"

c3f8f46

This reverts commit 55c33eb.

thoniTUB reviewed Apr 10, 2024

View reviewed changes

backend/src/main/java/com/bakdata/conquery/sql/conversion/forms/StratificationTableFactory.java Outdated Show resolved Hide resolved

jnsrnhld and others added 12 commits April 10, 2024 09:51

refactoring

12b80e2

New approach

135d7a1

...

74d093e

Cleanup

cbfc427

Adjust docs

17e0c70

Update backend/src/main/java/com/bakdata/conquery/sql/conversion/form…

f62ec0c

…s/README.md Co-authored-by: Torben Meyer <torben.meyer@bakdata.com>

Update backend/src/main/java/com/bakdata/conquery/sql/conversion/form…

4dcaa4e

…s/README.md Co-authored-by: Torben Meyer <torben.meyer@bakdata.com>

Update backend/src/main/java/com/bakdata/conquery/sql/conversion/dial…

e90e729

…ect/HanaSqlFunctionProvider.java Co-authored-by: Torben Meyer <torben.meyer@bakdata.com>

Refactoring

d3f8eb2

Fix

8f509cf

Merge branch 'develop' into sql/feature/absolute-form-conversion

e16e731

Fix

c162133

jnsrnhld and others added 2 commits April 15, 2024 17:30

Merge branch 'develop' into sql/feature/absolute-form-conversion

623af5f

Fix error message

9ce680c

jnsrnhld and others added 6 commits April 16, 2024 09:53

Cleanup

e830a15

Merge branch 'develop' into sql/feature/absolute-form-conversion

5bc4b01

Merge branch 'develop' into sql/feature/absolute-form-conversion

7dcf72a

Fix calculating date diff counts

5f3dffb

Merge remote-tracking branch 'origin/sql/feature/absolute-form-conver…

27e6d88

…sion' into sql/feature/absolute-form-conversion

Merge branch 'develop' into sql/feature/absolute-form-conversion

e6b39e9

awildturtok requested changes Apr 22, 2024

View reviewed changes

PR review changes

4e90719

awildturtok approved these changes Apr 23, 2024

View reviewed changes

jnsrnhld and others added 5 commits April 23, 2024 15:52

Update backend/src/main/java/com/bakdata/conquery/sql/conversion/form…

6508ffd

…s/README.md Co-authored-by: awildturtok <1553491+awildturtok@users.noreply.github.com>

Cleanup

fd01238

Use Offset for HANA functions

f1f9311

Merge branch 'develop' into sql/feature/absolute-form-conversion

2ca950e

Cleanup postgres functions

b6496b7

jnsrnhld requested a review from thoniTUB April 23, 2024 14:37

awildturtok approved these changes Apr 23, 2024

View reviewed changes

jnsrnhld enabled auto-merge (squash) April 23, 2024 18:19

Further cleanup

4aba25e

thoniTUB approved these changes Apr 24, 2024

View reviewed changes

jnsrnhld merged commit d4291e9 into develop Apr 24, 2024
6 checks passed

delete-merged-branch bot deleted the sql/feature/absolute-form-conversion branch April 24, 2024 14:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Absolute form conversion #3378

Absolute form conversion #3378

jnsrnhld commented Apr 8, 2024

jnsrnhld left a comment

thoniTUB left a comment

thoniTUB Apr 10, 2024

jnsrnhld commented Apr 15, 2024

awildturtok left a comment

awildturtok Apr 22, 2024

jnsrnhld Apr 22, 2024

awildturtok Apr 23, 2024

awildturtok Apr 23, 2024

awildturtok Apr 23, 2024

jnsrnhld Apr 23, 2024

awildturtok Apr 23, 2024

jnsrnhld Apr 23, 2024

jnsrnhld commented Apr 23, 2024

thoniTUB left a comment

Absolute form conversion #3378

Absolute form conversion #3378

Conversation

jnsrnhld commented Apr 8, 2024

jnsrnhld left a comment

Choose a reason for hiding this comment

thoniTUB left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jnsrnhld commented Apr 15, 2024

awildturtok left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jnsrnhld commented Apr 23, 2024

thoniTUB left a comment

Choose a reason for hiding this comment