Skip to content

Releases: spotify/scio

Release v0.10.3

26 May 18:40
a0b3a05
Compare
Choose a tag to compare

"Dallasium"

🚀 Enhancements

🐛 Bug Fixes

🌱 Dependency Updates

Contributors to this release

@anish749, @iezer, @jordiolivares, @jto, @kndarp, @mkrajc, @mlachkar, @nevillelyh, @regadas and @scala-steward

Release v0.10.2

08 Apr 16:59
18aaa84
Compare
Choose a tag to compare

"Cantabrigia"

The release has the fix #3716 for the critical coder performance regression issue leading to an abnormal CPU usage due to a coder logic being evaluated per every element.

🐛 Bug Fixes

🚀 Enhancements

📗 Documentation

  • Remove internal release steps from the How-to-Release.md (#3738) @stormy-ua

🌱 Dependency Updates

Contributors to this release

@nevillelyh, @regadas and @stormy-ua

Release v0.10.1

01 Apr 19:42
7c94085
Compare
Choose a tag to compare

"Belli Horizontis"

🚀 Enhancements

🐛 Bug Fixes

  • Fix infinite recursion in PubsubIO.readAvro (#3690) @moscowart
  • Fix header/footer in TextIO.textOut (#3715) @shomolomo

🌱 Dependency Updates

Contributors to this release

@annicai, @jto, @moscowart, @mrkm4ntr, @nevillelyh, @psobot, @regadas, @shomolomo and @wayoung

Release v0.10.0

03 Mar 18:44
11cc428
Compare
Choose a tag to compare

"Aquae Sextiae"

🚀 Enhancements

🐛 Bug Fixes

🌱 Dependency Updates

Contributors to this release

@clairemcginty, @jordiolivares, @mrkm4ntr, @nevillelyh, @regadas and @scala-steward

Release v0.10.0-beta3

29 Jan 18:35
6f6c7ba
Compare
Choose a tag to compare
Release v0.10.0-beta3 Pre-release
Pre-release

🚀 Enhancements

🐛 Bug Fixes

  • Upgrade Sparkey to work around hash collision bug. (#3532) @psobot
  • TestInput is not thread safe and can result in flaky tests. (#3512) @rteabeault

📗 Documentation

🌱 Dependency Updates

Contributors to this release

@clairemcginty, @DeaconDesperado, @dennisylyung, @mattnworb, @mrkm4ntr, @nevillelyh, @psobot, @regadas, @rteabeault, @scala-steward and @syodage

v0.9.6

16 Dec 22:50
Compare
Choose a tag to compare

"Specialis Revelio"

🐛 Bug Fix release!

  • Upgrade Sparkey to work around hash collision bug. (#3532)

From sparkey-java#3.2.1:

Fixed bug where creating hash files would erroneously lose some keys. The bug only applies to cases where: construction mode was SORTING and there existed hash collisions (so typically only when hash mode is 32 bits and the number of keys is more than 100000). The bug was introduced along with SORTING in 2.3.0.

Scio users may have been impacted by this bug if:

  • writing any Sparkey files from their pipelines (.asSparkey, .asSparkeySideInput, etc)
  • writing between 1M and 8,388,608 elements per individual Sparkey file or shard

v0.10.0-beta2

19 Nov 16:01
a05a535
Compare
Choose a tag to compare
v0.10.0-beta2 Pre-release
Pre-release

🚀 Enhancements

🐛 Bug Fixes

📗 Documentation

🏗️ Build Improvements

🔧 Refactorings

🌱 Dependency Updates

Contributors to this release

@clairemcginty, @alexclare, @dependabot, @dependabot[bot], @jasmineytchen, @martinbomio, @moscowart, @nevillelyh, @regadas, @scala-steward, @sfines-clgx, @syodage and @viktorjonsson

v0.10.0-beta1

30 Oct 17:30
Compare
Choose a tag to compare
v0.10.0-beta1 Pre-release
Pre-release

See v0.10.0 Migration Guide for detailed instructions.

Improvements

  • Add scio-redis (#3386)
  • Add RedisMutation support for RedisIO (#3331) (#3391)
  • Add ApproxDistinctCount trait and two HLL++ beam extension implementations. (#3361)
  • Add Predicate for SMB source, fix #3398 (#3402)
  • Allow SortedBucketPreKeyedSink key validation with null keys (#3439)
  • Add missing sortMergeGroupByKey API with custom parallelism (#3430)
  • Unblock MergeAndWriteBucketsSource while in progress (#3407)
  • Add SColletion withResource functions (#3389)
  • Make BQ inherit labels from Beam/Dataflow job (#3375)
  • Simplify Bigquery Format typeclass and bubble up Avro coder (#3401)
  • Support $LATEST replace for BQ Table type (#3376)
  • Integrate docs in CI (#3392)
  • Add 0.10.0 migration guide (#3436)
  • Add scalafix rules for 0.10 (#3432)
  • Clarify extra.csv.CsvIO docs (#3435)
  • Update scio.g8 steps
  • Add install instructions for sbt. (#3433)

Bug Fixes

  • Avoid Iterable backed by List when running locally (#3408)
  • Remove Coder context bound from read (#3384)
  • Avoid fallback Coder on GenericRecord read (#3385)
  • Add missing pubsub coders to avoid fallback (#3382)
  • Remove prompt before interp is created (#3404)
  • Fix repl windows character escape (#3405)
  • Sanitize staged file path string (#3406)
  • Remove unused fields in AvroIO (#3383)
  • Fix JavaAsyncLookupDoFn doc link (#3381)

Dependency Updates

  • Update Beam to 2.25.0 (#3438)
  • Update sbt-avro to 3.2.0 (#3434)
  • Update magnolify-avro, magnolify-bigtable, ... to 0.3.0 (#3431)
  • Update elasticsearch-rest-client, ... to 7.9.3 (#3422)
  • Update sbt-explicit-dependencies to 0.2.15 (#3417)
  • Update joda-time to 2.10.8 (#3424)
  • Update sbt-mima-plugin to 0.8.1 (#3414)
  • Update sbt to 1.4.1 (#3415)
  • Update scalafmt-core to 2.7.5 (#3410)
  • Update mysql-connector-java to 8.0.22 (#3412)
  • Update sbt-mdoc to 2.2.10 (#3413)
  • Update junit to 4.13.1 (#3394)
  • Update caffeine to 2.8.6 (#3395)
  • Bump Featran to 0.7.0, re-enable failing TF tests (#3380)

v0.10.0-alpha1

02 Oct 20:57
Compare
Choose a tag to compare
v0.10.0-alpha1 Pre-release
Pre-release

Breaking Changes

  • Clean up coder implicits propagation #3056 (#3170)
  • Move Google Cloud IOs to scio-google-cloud-platform (#3340)
  • Deprecate ScioContext pubsubIO methods (#3345)
  • Remove EOLed elasticsearch5 support (#3217)
  • Remove readAll* API deprecated since 0.8 (#3216)
  • Remove async DoFn aliases deprecated since 0.8
  • Remove key-value transform code deprecated since 0.8 (#3213)
  • Remove scio-extra code deprecated since 0.8 (#3212)

Improvements

  • Add internal composite transfrom (#3159)

Bug fixes

  • Remove BeamCoders unchecked warning (#3343)

v0.9.5

02 Oct 17:48
Compare
Choose a tag to compare

"Colovaria"

There are no breaking changes in this release, but some were introduced with v0.9.0:

See v0.9.0 Migration Guide for detailed instructions.

Improvements

  • Add custom GenericJson pretty print (#3367)
  • scio-parquet to support dynamic destinations for windowed scollections (#3356)
  • Support $LATEST replacement for Query (#3357)
  • Mutable ScalableBloomFilter (#3339)
  • Add specialized TupleCoders (#3350)
  • Add nullCoder on Record and Disjunction coders (#3349)

Bug Fixes

  • Support null-key records in smb writes (#3359)
  • Fix serialization struggles in SMB transform API (#3342)
  • Grammar / spelling fixes in migration guides (#3358)
  • Remove unused macro import (#3353)
  • Remove unused BaseSeqLikeCoder implicit (#3344)
  • Filter out potentially included env directories (#3322)
  • Simplify LowPriorityCoders (#3320)
  • Remove unused and not useful Coder implicit trait (#3319)
  • Make javaBeanCoder lower prio (#3318)

Dependency Updates

  • Update Beam to 2.24.0 (#3325)
  • Update scalafmt-core to 2.7.3 (#3364)
  • Update elasticsearch-rest-client, ... to 7.9.2 (#3347)
  • Update hadoop libs to 2.8.5 (#3337)
  • Update sbt-scalafix to 0.9.21 (#3335)
  • Update sbt-mdoc to 2.2.9 (#3327)
  • Update sbt-avro to 3.1.0 (#3323)
  • Update mysql-socket-factory to 1.1.0 (#3321)
  • Update scala-collection-compat to 2.2.0 (#3312)
  • Update sbt-mdoc to 2.2.8 (#3313)