Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DBZ-7362 Support DECFLOAT in Db2 connector #129

Merged
merged 3 commits into from
Jan 25, 2024
Merged

Conversation

leoloel
Copy link
Contributor

@leoloel leoloel commented Jan 22, 2024

No description provided.

Copy link

Welcome as a new contributor to Debezium, @leoloel. Reviewers, please add missing author name(s) and alias name(s) to the COPYRIGHT.txt and Aliases.txt respectively.

@@ -52,6 +52,10 @@ public SchemaBuilder schemaBuilder(Column column) {
case Types.TINYINT:
// values are an 8-bit unsigned integer value between 0 and 255, we thus need to store it in short int
return SchemaBuilder.int16();
case Types.OTHER:
if (matches(column.typeName().toUpperCase(), "DECFLOAT")) {
return SchemaBuilder.float64();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the real datatype coming from the database? It seems to me according to docs that it is either 16 or 32 bit signed floating point decimal. How does it differ from REAL/FLOAT? Does it makes sense to map it to either FLOAT32 or FLOAT64 based on the value? What Java class is returned by the JDBC driver when small and large DECFLOATs are used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The range of a decimal floating point number is either 16 or 34 digits of precision. You can refer to this manual. https://www.ibm.com/docs/en/db2-for-zos/12?topic=numbers-decimal-floating-point-decfloat

Create table like this,
create table TEST.T1
(
ID INTEGER not null primary key,
C_DECFLOAT DECFLOAT(34)
);

Then insert a record, INSERT INTO TEST.T1 (ID, C_DECFLOAT) VALUES (1, DECFLOAT(1.234));
Finally execute the select statement,select C_DECFLOAT from TEST.T1. The result is 1.234, without padding the 0's at the end.

To be honest, I don’t quite understand the difference between decfloat and float types.

@leoloel
Copy link
Contributor Author

leoloel commented Jan 24, 2024

Db2OnlineDefaultValueIT>AbstractDb2DefaultValueIT.shouldHandleDateTimeDefaultTypes:201->AbstractDb2DefaultValueIT.shouldHandleDefaultValuesCommon:215->AbstractDb2DefaultValueIT.testDefaultValuesAlterTableAdd:491->AbstractDb2DefaultValueIT.assertSchemaFieldWithDefaultCurrentDate:542->AbstractDb2DefaultValueIT.assertSchemaFieldValueWithDefault:538->AbstractDb2DefaultValueIT.lambda$assertSchemaFieldWithDefaultCurrentDate$2:550 [Unexpected field value: AVAL_DATE_SYSDATE_NONNULL]

I tested this test locally and the result was successful. I don’t know why the check that
Cross Maven CI / build (pull_request) failed.

@jpechane
Copy link
Contributor

@leoloel DECFLOAT seems to be precise numeric type similar to DECIMAL. Even JDBC driver returns its value as BigDecimal. Unfortunately it is not of fixed scal so it must be mapped to VariableScaleDecimal like for example Postgres - see https://debezium.io/documentation/reference/2.5/connectors/postgresql.html#postgresql-decimal-types

Could you please give the PR look if it is OK for you? If yes I am going to mege it after CI excution. In that case would you be willing to send a PR with Db2 docs update for this datatype mapping?

@jpechane
Copy link
Contributor

Test failures are unrelated.

@leoloel
Copy link
Contributor Author

leoloel commented Jan 25, 2024

ok, I will try it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not reuse the convertDecimal function? I think the contents of convertDecimal and convertDecfloat are the same.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, it is not. This is taken form PostgreSQL to hadnle potential issue with trailing zeroes. We don't know if it is problem here but it is kind of defensive measure.

@leoloel
Copy link
Contributor Author

leoloel commented Jan 25, 2024

Could you please check this pr? debezium/debezium#5189

@leoloel DECFLOAT seems to be precise numeric type similar to DECIMAL. Even JDBC driver returns its value as BigDecimal. Unfortunately it is not of fixed scal so it must be mapped to VariableScaleDecimal like for example Postgres - see https://debezium.io/documentation/reference/2.5/connectors/postgresql.html#postgresql-decimal-types

Could you please give the PR look if it is OK for you? If yes I am going to mege it after CI excution. In that case would you be willing to send a PR with Db2 docs update for this datatype mapping?

@jpechane jpechane merged commit ead7281 into debezium:main Jan 25, 2024
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants