Releases: AbsaOSS/atum
Releases · AbsaOSS/atum
Atum v3.9.0
Atum v3.8.3
Atum v3.8.2
Atum v3.8.1
Due to an error, this version is identical to v3.8.0. Follow the next release for feature updates.
Atum v3.8.0
Release notes
- Atum packages for Scala 2.12 (
atum_2.12
,atum_model_2.12
, andatum-s3-sdk-extension_2.12
) are now built against Spark 3.2.1. - Scala 2.11 version remains the same as in v3.7.0 (built for Spark 2.4)
Issues fixed
- #127 Update to Spark 3.2.1
Atum v3.7.0
Release notes
- The file permissions on HDFS which the _INFO file will be created with are now driven by (implemented in #114):
- typesafe config key
atum.hdfs.info.file.permissions
(e.g. setatum.hdfs.info.file.permissions=711
) - default HDFS
FsPermissions
withumask
applied
- typesafe config key
Please note, that the target umask
will work as a cap for your desired permissions (e.g. having umask=022
, your desired 777
permissions will become 755
)
Issues fixed
- #113 Making _INFO file permission configurable
Atum v3.6.0
Atum v3.5.1
Atum v3.5.0
Release notes
- Improvement: implicit
SparkSessionWrapper
renamed toAtumSparkSessionWrapper
in order to prevent name clashing. - Generalization:
S3Location
Utils moved to AbsaOSS/commons, so a migration for S3Location-related routines is needed:- needs dependency
za.co.absa.commons:commons_2.11:0.0.27
(for Scala 2.12 usecommons_2.12
) za.co.absa.atum.location.S3Location
->za.co.absa.commons.s3.S3Location
za.co.absa.atum.location.S3Location.StringS3LocationExt
->za.co.absa.commons.s3.SimpleS3Location.SimpleS3LocationExt
- address small API differences:
"string".toS3Location
->"string".toSimpleS3Location
s3Location.s3String
->s3Location.asSimpleS3LocationString
- needs dependency
- Added support for Apache Spark 3.1.X
- Atum model now has json4s dependencies as provided dependencies to accommodate multiple Spark versions. More details in the README.
Issues fixed
- #69 Atum implicit rename
- #71 S3Location moved to AbsaOSS/commons
Atum v3.4.1
Release notes
- Bugfix: Atum's
model
works without extra dependencies or Spark being present.
Issues fixed
- #63 Atum model's serialization missing dependencies