Skip to content

taskqueue-1.0.0

Compare
Choose a tag to compare
@talkiq-ops talkiq-ops released this 12 Jan 21:33
· 945 commits to master since this release
d15540f

This release migrates to Cloudtasks v2beta2. We've taken the opportunity to clean up the TaskManager and TaskQueue usage in response to this change, with the non-backwards-compatible changes described below.

Major changes

  • deserialize_task has been replaced with decode and encode. Usage:
await tq.insert(encode(payload))
leased_payload = decode((await tq.lease(num_tasks=1))['tasks'][0]['pullMessage']['payload'])
assert payload == leased_payload
  • TaskQueue has been re-written from the ground up. Notable changes:
    • the constructor now accepts a location parameter, which you can find in your Cloudtasks dashboard
    • insert_task, delete_task, and renew_taskhave been renamed toinsert, delete, and renew
    • A bunch of new methods have been added for interfacing with task entities: ack, cancel, drain, get, lease, list, and renew
    • The task entity format has changed with the new v2beta2 API! The most important changes are as follows:
      • id has been re-branded as name. Accordingly, TaskQueue methods now accept names rather than ids.
      • Some operations (ack, cancel, renew) can only be performed by the consumer which originally leased that task. Accordingly, those methods now require task entities be passed in, rather than just names, to prove that the consumer is indeed the task leaser.
      • task['payloadBase64'] has been moved to task['pullMessage']['payload'] (it is still base64-encoded, see above encode and decode convenience methods. task['pullMessage']['tag'] is now exposed.
  • TaskManager constructor has changed.
    • task_queue has been replaced with project, service_file, and taskqueue(eg. theTaskManagerinstance will construct its ownTaskQueuefrom these parameters).location, session, and tokenare optional values which will be passed to theTaskQueue` constructor.
    • poll_interval and max_poll_interval have been replaced with backoff_base and backoff_max_value, and an additional parameter backoff_factor has been added. We now backoff exponentially according to the backoff_factor, rather than linearly as before.
    • deadletter_upsert has been renamed to deadletter_insert_function
    • The worker method no longer decodes tasks as json objects. This allows for support of non-json-ified payloads. If your tasks are json-encoded, the first line of your worker method should now be ... = json.loads(payload)
OldTaskManager(self, task_queue, worker, deadletter_upsert=None, lease_seconds=10, poll_interval=1, max_poll_interval=30.0, max_concurrency=20, batch_size=1, retry_limit=50)

NewTaskManager(self, project, service_file, taskqueue, worker, backoff_base=2, backoff_factor=1.1, backoff_max_value=60, batch_size=1, deadletter_insert_function=None, lease_seconds=10, location=LOCATION, max_concurrency=20, retry_limit=None, session=None, token=None)
  • TaskManager and TaskQueue stat collection methods have been disabled. Google plans to re-introduce these in the future, at which point we will pass those through here.
  • LocalTaskManager has been removed, since its only real use was in stats collection. It may be re-introduced when stats collection is re-enabled, but we recommend simply mocking TaskManager when running tests which would have previously used the LocalTaskManager.
  • This library no longer relies on gcloud-aio-core, which will gradually be phased out of all gcloud-aio-* libraries, or ujson. It does, though, now rely on aiohttp version 2.0.0 or greater.
  • As always, see the smoke tests for more information