Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DBZ-7362 Support DECFLOAT in Db2 connector #129

Merged
merged 3 commits into from
Jan 25, 2024
Merged

DBZ-7362 Support DECFLOAT in Db2 connector #129

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2fb555d
DBZ-7362 Support DECFLOAT in Db2 connector
leoloel Jan 22, 2024
d233f0c
DBZ-7362 Format code
jpechane Jan 22, 2024
fcd2344
DBZ-7362 DECFLOAT is precise decimal type
jpechane Jan 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions src/main/java/io/debezium/connector/db2/Db2DefaultValueConverter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,8 @@ private static Map<Integer, DefaultValueMapper> createDefaultValueMappers(Db2Con
result.put(Types.DECIMAL, nullableDefaultValueMapper());
result.put(Types.DOUBLE, nullableDefaultValueMapper((c, v) -> Double.parseDouble(v)));
result.put(Types.REAL, nullableDefaultValueMapper((c, v) -> Float.parseFloat(v)));
// decfloat type
result.put(Types.OTHER, nullableDefaultValueMapper(decfloatDefaultValueMapper()));

// Date and time
result.put(Types.DATE, nullableDefaultValueMapper(castTemporalFunctionCall(connection, Types.DATE)));
Expand Down Expand Up @@ -210,4 +212,13 @@ private static String unquote(String value) {
}
return value;
}

public static DefaultValueMapper decfloatDefaultValueMapper() {
return (column, value) -> {
if (Db2ValueConverters.matches(column.typeName().toUpperCase(), "DECFLOAT")) {
return Double.parseDouble(value);
}
return nullableDefaultValueMapper(null);
};
}
}
22 changes: 22 additions & 0 deletions src/main/java/io/debezium/connector/db2/Db2ValueConverters.java
View file
Open in desktop
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not reuse the convertDecimal function? I think the contents of convertDecimal and convertDecfloat are the same.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, it is not. This is taken form PostgreSQL to hadnle potential issue with trailing zeroes. We don't know if it is problem here but it is kind of defensive measure.

Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ public SchemaBuilder schemaBuilder(Column column) {
case Types.TINYINT:
// values are an 8-bit unsigned integer value between 0 and 255, we thus need to store it in short int
return SchemaBuilder.int16();
case Types.OTHER:
if (matches(column.typeName().toUpperCase(), "DECFLOAT")) {
return SchemaBuilder.float64();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the real datatype coming from the database? It seems to me according to docs that it is either 16 or 32 bit signed floating point decimal. How does it differ from REAL/FLOAT? Does it makes sense to map it to either FLOAT32 or FLOAT64 based on the value? What Java class is returned by the JDBC driver when small and large DECFLOATs are used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The range of a decimal floating point number is either 16 or 34 digits of precision. You can refer to this manual. https://www.ibm.com/docs/en/db2-for-zos/12?topic=numbers-decimal-floating-point-decfloat

Create table like this,
create table TEST.T1
(
ID INTEGER not null primary key,
C_DECFLOAT DECFLOAT(34)
);

Then insert a record, INSERT INTO TEST.T1 (ID, C_DECFLOAT) VALUES (1, DECFLOAT(1.234));
Finally execute the select statement，select C_DECFLOAT from TEST.T1. The result is 1.234, without padding the 0's at the end.

To be honest, I don’t quite understand the difference between decfloat and float types.

}
default:
return super.schemaBuilder(column);
}
Expand All @@ -64,6 +68,10 @@ public ValueConverter converter(Column column, Field fieldDefn) {
case Types.TINYINT:
// values are an 8-bit unsigned integer value between 0 and 255, we thus need to store it in short int
return (data) -> convertSmallInt(column, fieldDefn, data);
case Types.OTHER:
if (matches(column.typeName().toUpperCase(), "DECFLOAT")) {
return (data) -> super.convertDouble(column, fieldDefn, data);
}
default:
return super.converter(column, fieldDefn);
}
Expand All @@ -82,4 +90,18 @@ protected Object convertTimestampWithZone(Column column, Field fieldDefn, Object
return super.convertTimestampWithZone(column, fieldDefn, data);
}

/**
* Determine if the uppercase form of a column's type exactly matches or begins with the specified prefix.
* Note that this logic works when the column's {@link Column#typeName() type} contains the type name followed by parentheses.
*
* @param upperCaseTypeName the upper case form of the column's {@link Column#typeName() type name}
* @param upperCaseMatch the upper case form of the expected type or prefix of the type; may not be null
* @return {@code true} if the type matches the specified type, or {@code false} otherwise
*/
protected static boolean matches(String upperCaseTypeName, String upperCaseMatch) {
if (upperCaseTypeName == null) {
return false;
}
return upperCaseMatch.equals(upperCaseTypeName) || upperCaseTypeName.startsWith(upperCaseMatch + "(");
}
}
5 changes: 3 additions & 2 deletions src/test/java/io/debezium/connector/db2/AbstractDb2DefaultValueIT.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,8 +125,9 @@ public void shouldHandleFloatPointDefaultTypes() throws Exception {
}

List<ColumnDefinition> columnDefinitions = Arrays.asList(
// todo: DECFLOAT is currently not supported.
// new ColumnDefinition("val_decfloat", "decfloat", "3.14", "6.28", 3.14, 6.28, AssertionType.FIELD_DEFAULT_EQUAL),
new ColumnDefinition("val_decfloat", "decfloat",
"3.14", "6.28", 3.14d,
6.28d, AssertionType.FIELD_DEFAULT_EQUAL),
new ColumnDefinition("val_decimal", "decimal(5,2)",
"3.14", "6.28",
BigDecimal.valueOf(3.14), BigDecimal.valueOf(6.28),
Expand Down
Loading