Skip to content

Atum v3.5.0

Compare
Choose a tag to compare
@dk1844 dk1844 released this 07 Apr 14:13
· 44 commits to master since this release

Release notes

  • Improvement: implicit SparkSessionWrapper renamed to AtumSparkSessionWrapper in order to prevent name clashing.
  • Generalization: S3Location Utils moved to AbsaOSS/commons, so a migration for S3Location-related routines is needed:
    • needs dependency za.co.absa.commons:commons_2.11:0.0.27 (for Scala 2.12 use commons_2.12)
    • za.co.absa.atum.location.S3Location -> za.co.absa.commons.s3.S3Location
    • za.co.absa.atum.location.S3Location.StringS3LocationExt -> za.co.absa.commons.s3.SimpleS3Location.SimpleS3LocationExt
    • address small API differences:
      • "string".toS3Location -> "string".toSimpleS3Location
      • s3Location.s3String -> s3Location.asSimpleS3LocationString
  • Added support for Apache Spark 3.1.X
  • Atum model now has json4s dependencies as provided dependencies to accommodate multiple Spark versions. More details in the README.

Issues fixed