Templates - Second Pass

[petey]

Background See #470

• Wrapping around the steps inside template. For example: pretest, posttest. (feat: Add support for wrapping steps in template config-parser#39)
  Inside template:

   steps: 
       - test: echo test
       - publish: echo publish

Inside user's screwdriver.yaml:

main:
   template: mytemplate@1
   steps: 
       - pretest: echo hello
       - posttest: echo goodbye

Translates into:

   steps:
       - pretest: echo hello
       - test: echo test
       - posttest: echo goodbye
       - publish: echo publish

• Override step: (feat: support overriding steps for templates config-parser#40)

main:
   template: mytemplate@1
   steps: 
       - publish: echo not publishing

Translate into:

   steps:
       - test: echo test
       - publish: echo not publishing

• Version tags (WIP): crud api for tagging template versions, e.g. npm's dist_tags allow fetch on template version by tag. mytemplate@latest (feat: add template tag data-schema#157, fix(616): Add remove schema for template tags data-schema#158, feat(616): Add template tag model models#179, feat(616): support tag in templates [2] config-parser#41, feat(616): Can get a template using template name and tag #646, feat: Add create and remove template tag routes #647)
• Restrict API calls for template writes to admins, and template owners
• Version Lifespan: After a set period of time [configurable: default 6 months], a given version of a template should display a warning message in the build that the template is no longer being updated/maintained. ([feature] Clear deprecated sign of templates/commands  #1490)
• Template namespaces (e.g.: chef/publish, screwdriver/checkout, etc)
• Template keywords: Listed in annotations of template.yaml, e.g. language:node.js, publishing, aws. Primarily to allow discovery of templates by searching by keyword. Keywords should apply to all versions of a template.
• Documentation in guide:
  • create, validate, and publish (fix(470): Add job template docs guide#90)
  • wrapping around steps (fix: add documentation for wrapping steps guide#95)
  • overriding steps (fix: Add docs for replacing a step guide#96)
  • tagging (feat(616): Add script to tag templates template-main#8, fix(616): Make file executable template-main#9, fix(616): Updating a tag should not throw an error template-main#10, fix(616): Add template tags docs guide#105)
  • lifespan
  • namespaces
  • keywords
• Blog post

Nice to have:

• Publish multiple templates from one repo

Changelog

2017-7-6

• Added PR link for wrapping steps

2017-7-10

• Added PR link for create, validate, and publish documentation
• Added documentation requirements for wrapping and overriding steps

2017-7-13

• Added PR links for data schema and models for tagging

2017-7-14

• Added PR link for tagging API, tags in config-parser

2017-7-24

• Added checkbox for template namespacing

2017-7-28

• Added links to template tagging module and documentation PRs

2018-3-19

• Added link to template namespace issue

[d2lam]

Template tagging

For template tagging, we decided to have a new table called templateTags.
This will be a map of tag and version.
1 version can be mapped to multiple tags, but 1 tag can only be mapped to 1 version.

Example: mytemplate@1.2.0 can be tagged as stable and latest

The templateTags table will look like this

id	name	tag	version
1	mytemplate	stable	1.2.0
2	mytemplate	latest	1.2.0

If they retag stable with version 1.1.0, then the table will look like

id	name	tag	version
1	mytemplate	stable	1.1.0
2	mytemplate	latest	1.2.0

Permissions

We will only allow build scope for now. Only the pipeline that first creates the template has the permission to tag it.

[FenrirUnbound]

Some notes with using it as of writing:

1. Tagging in a build isn't easy

The publishing operation does not make it easy to determine what version was published. The only indicator about the new version is this message:

Template FenrirUnbound/my-template@1.2.18 was successfully published

In order to determine the exact version that was published, we need to:

• save the output of the publish to a file
• parse the file for the exact version that was published
• save that version to be used with the tagging operation

This problem kind of goes away if the template owner explicitly defines a major.minor.patch version in their sd-template.yaml. The oddities with this method is that you have to be sure to change it in two places, and you can't run more than one build without a code change.

2. Defining the --name value should be implied

Having an option for --name to be provided is nice to explicitly define the template to tag. It would be more convenient to have that value determined by the sd-template.yaml file. The publish operation determines the name based on the sd-template.yaml file so I would expect that the tagging operation would infer the same value.