requester pays support

Release Notes

For: uc-cdis/fence
Notes since tag: 4.0.0
Notes to tag/commit: 66e0123
Generated: 2019-07-31

New Features

• new optional/configurable support for accessing Google buckets w/ requester
  pays enabled (#668)
• allow configuration of default billing projects for accessing Google
  buckets w/ requester pays enabled (#668)
• allow configuration for automatic billing permission assignment for Google
  service accounts to ease data access for Google buckets w/ requester pays
  enabled (#668)
• add userProject field to Google signed urls if a billing project is
  provided (by user or by default configuration) (#668)
• give Google service accounts billing permission to default billing project
  for Data Access Method: Temporary Service Account Credentials (if
  configured to do so) (#668)

Bug Fixes

• Re-add SAs to all GBAGs in PATCH endpoint (#670)
• don't overwrite top-level user.yaml mapping for fence project to arborist
  resource if no resource path field in individual user projects (#667)
• correctly get resource mapping in user sync (#666)
• in usersync _read_file, init Popen with text mode so that output stream is
  string not bytestring (#665)
• Fix loading privacy policy file in python3 (#659)
• Fix caching (#659)

Improvements

• create project if not exist for google bucket link fence-create script
  (#657)
• add google organization whitelist to force_default_if_none list in
  config.py (#662)

Dependency Updates

• cirrus to 1.1.0 (#668)

Deployment Changes

• configuration added for requester pays Google Buckets. if this is needed,
  you'll need to update configuration. default config does not support
  requester pays bucket access (#668)

Python 3

Release Notes

For: uc-cdis/fence

Notes since tag: 3.3.0

Notes to tag/commit: 4.0.0

Generated: 2019-07-12

Breaking Changes

• Python 3 instead of python 2 (#646)

Dependency Updates

• Remove unused flask-postgres-session dependency (#646)
• Remove enum34 dependency (#646)
• Use python 3 versions of cirrus (1.0.0), storage-client, cdisutilstest
  (1.0.0), userdatamodel (2.0.0), cdispyutils (1.0.0) and authutils (4.0.0)
  (#646)

Privacy policy feature

Release Notes

For: uc-cdis/fence

Notes since tag: 3.2.4

Notes to tag/commit: 3.3.0

Generated: 2019-07-12

New Features

• Add /privacy-policy endpoint, which serves the text of the new privacy
  policy in either markdown or HTML depending on the content-type headers in
  the request. (#649)

fix issue with dbgap resource / user.yaml paths

Merge pull request #656 from uc-cdis/fix/dbgap-combination

Fix/dbgap combination

Don't create duplicate arborist records for dbgap users

Merge pull request #655 from uc-cdis/fix/dbgap-sync-case

Fix/dbgap sync case

Fix google validation cronjob script

Merge pull request #654 from uc-cdis/fix/google-cron

Fix/google cron

3.2.1

Release Notes

For: uc-cdis/fence

Notes since tag: 3.2.0

Notes to tag/commit: 7196556

Generated: 2019-07-01

Breaking Changes

• remove --verbose option to fence-create CLI, prefers checking the
  DEBUG value from config now to determine whether or not to output debug
  logs (#647)

Bug Fixes

• fix issue with service account registration where validation checks new
  access and previous access when trying to update an SA (so if updating to
  restrict to a subset of previous access, validation may fail and not allow
  the update) (#651)

Improvements

• service account validation will only verify that users on the Google
  Project have access to the projects the active registered service account
  have access to (in other words, expired service account project access
  won't be checked since they don't actually have access to data) (#651)
• fence will now consider authz field on indexd record to determine if a file
  is public or not (for different signed url behavior). previously only
  checked acl field (#653)

dbGaP AuthZ Sync Support

Release Notes

For: uc-cdis/fence
Notes since tag: 3.1.0
Notes to tag/commit: 6317617
Generated: 2019-06-25

New Features

• dbgap syncing now updates resources in arborist (#641)
• SESSION_COOKIE_DOMAIN is now configurable (#640)
• Client policies can be managed in the user.yaml as long as the client
  already exists in fence (#642)
• use standard base image (#638)

Bug Fixes

• Re-add openssh back to image for sftp access to work correctly (#644)
• fix error with logger if no user.yaml provided (#639)
• can now successfully call fence-create sync --sync_from_dbgap True
  without providing a valid user.yaml (e.g. just a dbgap sync) (#637)

Improvements

• allow not providing a user.yaml (e.g. just a pure dbgap sync). previously
  you had to pass an empty user.yaml (#641)
• dbgap study to arborist resource namespace configuration (#641)
• specify upstream idp to require user re-auth. caveat is some IdPs wouldn't
  support it (#643)
• The login endpoints now verify that the redirect provided is valid
  according to fence: it redirects back to the Gen3 application, to an OAuth
  client, or to some other approved URL from the configuration. (#540)
• Deployment changes (#540)
• Added optional new LOGIN_REDIRECT_WHITELIST config variable to allow for
  redirecting to specific domains for login/logout. (#540)
• Base class for different IDPs for Login (#593)
• Update default configuration with details about setting up ORCID &
  Microsoft OAuth 2 Clients (#593)
• Better logging for sftp connections for dbgap syncing (#637)
• filter SAWarning about not reflecting partial indices (#636)

Choose Identity Provider in request

Release Notes

For: uc-cdis/fence
Notes since tag: 3.0.0
Notes to tag/commit: 3.1.0
Generated: 2019-05-31

Improvements

• Support providing alternative identity provider in oidc flow (#635)
• use new gen3config instead of having all config code in fence (#592)
• validate user.yaml files via gen3users tests before running usersync (#629)

Dependency Updates

• bump sqlalchemy 0.9.9 to 1.3.3 (#630)
• bump userdatamodel 1.5.0 to 1.5.1 (#630)
• bump authlib from 0.9 to 0.11 (#628)

Authorization and Sync Update: New Arborist Integration

Release Notes

For: uc-cdis/fence
Notes since tag: 2.8.2
Notes to tag/commit: 798dfe3
Generated: 2019-05-20

New Features

• Configure new clients with arborist policies using --arborist and
  --policies flags for fence-create (#608)
• Use arborist for authorization checks on downloading data files from
  indexd, as long as the authz field is present in the index record (#608)
• swagger document for multipart upload presigned url (#616)
• Allow setting policies for anonymous and logged-in groups in the
  user.yaml. (#625)
• Use arborist to check permission for indexd record upload/download on
  records when supported (send "rbac" field from the indexd record to
  arborist) (#606)

Breaking Changes

• Users' RBAC policies are now owned by arborist, not fence, so the tables
  are removed from fence/userdatamodel. (#608)
• The usersync is updated to work specifically with the new arborist version.
  (#608)
• In general, this and following versions of fence should be deployed only
  with
  arborist>=2.0.0
  (#608)
• Remove (unused) RBAC blueprint (#606)

Bug Fixes

• Fix typos in exception handler (#626)

Improvements

• Update user sync for compatibility with changes to arborist. (#613)
• Policies now owned by arborist (#613)
• Arborist needs (read-only) copy of users. (#613)
• User info endpoint will return policy list from arborist for that user, if
  available. (#604)