Rework analyzer docs (#460)

* Add backticks to analysis types

* Rework feedback guidelines

building/config.json

@@ -689,13 +689,6 @@
     "title": "Analyzers",
     "blurb": ""
   },
-  {
-    "uuid": "48775c5f-b449-4c91-b792-b81a8ef4ec04",
-    "slug": "tooling/analyzers/feedback-guidelines",
-    "path": "building/tooling/analyzers/feedback-guidelines.md",
-    "title": "Feedback Guidelines",
-    "blurb": ""
-  },
   {
     "uuid": "787e4243-b97a-4b55-9038-7667e5c3788a",
     "slug": "tooling/analyzers/comments",

building/tooling/analyzers/README.md

@@ -18,3 +18,4 @@ You can use the following documents to learn more about building an analyzer:
 - [Creating a Analyzer from scratch](/docs/building/tooling/analyzers/creating-from-scratch)
 - [The Analyzer interface.](/docs/building/tooling/analyzers/interface)
 - [How to build a Docker image with Docker for local testing and deployment](/docs/building/tooling/analyzers/docker)
+- [Writing Analyzer comments](/docs/building/tooling/analyzers/comments)

building/tooling/analyzers/comments.md

@@ -1,36 +1,18 @@
-**NOTE: This spec is currently being updated.**
+# Writing Analyzer Comments
 
----
+This document provides information and guidelines on how to write comments produced by the analyzer.
 
-# Comment Copy Guidelines
+## Contents
 
-This document sets guidelines for the comments produced by the analyzer. The
-[Feedback guidelines](/docs/building/tooling/analyzers/feedback-guidelines) talk about _when_ to comment and
-_what_ to comment. This document talks mostly about _how_ to comment.
+The contents for analyzer comments are stored as Markdown document in the [exercism/website-copy][git-website-copy] repo.
+Comments contain a pointer-string, formatted as `<track-slug>.<exercise-slug>.<comment-slug>`, that links to a specific Markdown document in the `website-copy` repo.
 
-## Working directory for contents
+For example, `ruby.two-fer.string-interpolation` refers to https://github.com/exercism/website-copy/blob/main/analyzer-comments/ruby/two-fer/string_interpolation.md.
 
-Comments for the analyzer live in the [`exercism/website-copy`][git-website-copy]
-repository. Changes should be made via a PR on this repository, only changing
-the comments for one analyzer at a time. These will get the
-[`type/analyzer-comments` label][git-website-copy-label].
-
-### Generic comments
-
-Some comments are language specific and not exercise specific. These should be
-extracted into a generic comments file, in order to not duplicate commentary,
-which would lead to either outdated commentary or shotgun surgery when a change
-is made.
-
-## CI
-
-Because the comments don't live in the same repository as the analyzer, each
-analyzer should have CI that checks if the comments used in that specific
-analyzer (those that can become output), are comments on the `main` branch, on
-the [`exercism/website-copy`][git-website-copy] repository.
-
-At the moment of writing, [this issue][issue-ci-comments] tracks the status of any
-generalization of this CI, if any.
+```exercism/note
+If a comment is language specific and _not_ exercise specific, replace `<exercise-slug>` with `general`.
+E.g. `ruby.general.string-explicit_return`
+```
 
 ## Wording
 
@@ -49,23 +31,34 @@ generalization of this CI, if any.
   understand x", because even though something was said, the person doesn't
   necessarily understand it.
 
-### Guidance
+## Guidance
+
+- **Aim for fluency, not proficiency**: the goal of a language track on Exercism is to give people a way to achieve a high level of [fluency](/docs/using/product/fluency) at a low level of proficiency.
+  We're aiming for fluency in the syntax, idioms, and the standard library of the language.
+- **Suggest idiomatic code** when possible, where idiomatic is code that would be written by nearly all developers (non-hobbyists) who write code in that language.
+  If a non-idiomatic suggestion is made, mention this and explain why the suggestion might still be useful.
+- **Name the difference** between what the person is doing, and what is 'idiomatic' in the language.
+- **Use the proper terms** and nomenclature, so people can recognize those concepts elsewhere, and also can research them by themselves.
+- **Don't give the solution** as general advice for all Exercism mentoring.
+  Learning sticks when people discover the answer for themselves. That's a tremendously exhilarating experience, and the emotional kick makes it memorable.
+  However, if the discovery **doesn't** trigger a dopamine hit, it makes total sense to show what it looks like.
+  For example, we might choose to provide a small improvement on an approved solution will be a lot less exciting than giving someone a learning point on a solution that is being disapproved and therefore may warrant an example rather than a link.
+- **Order comments by importance**, the first comment being the most important, and the last comment being the least important.
+- **Keep the number of comments manageable**.
+  Aim for one to three comments per iteration.
+- **Don't add the same comment twice** in one analysis.
+  Adding the same comment with different parameters attached to it is _not_ considered a duplicate.
+- **Consider only commenting on formatting** if formatting or linting is integral to the language.
+  When possible, guide students towards tools for auto-formatting and/or link to any official style guide.
+
+### First few exercises
 
 For the first few exercises of a track, the following is _extra_ important:
 
-- **Name the difference** between what the person is doing, and what is
-  'idiomatic' in the language.
-- **Use the proper terms** and nomenclature, so people can recognize those
-  concepts elsewhere, and also can research them by themselves.
 - **Keep it relatively short**, avoiding a wall of text or overwhelming them
   with advice. If they have a great experience in the first exercise, they'll
   come back and you'll have many more opportunities to provide feedback on all
   of the things that you noticed.
-- **Don't give the solution** as general advice for all Exercism mentoring.
-  Learning sticks when people discover the answer for themselves. That's a
-  tremendously exhilarating experience, and the emotional kick makes it
-  memorable. However, if the discovery **doesn't** trigger a dopamine hit, it
-  makes total sense to show what it looks like. For example, we might choose to provide a small improvement on an approved solution will be a lot less exciting than giving someone a learning point on a solution that is being disapproved and therefore may warrant an example rather than a link.
 - **Don't overly explain a concept**: don't go in depth about the underlying
   mechanisms of compilers and things. In this case it's more about this being
   one of the first exercises in the language track, and at this stage feedback
@@ -135,6 +128,16 @@ on type later. For simpler cases, it's more common to rely on `errors.New` or
 custom errors might be interesting.
 ```
 
+## CI
+
+Because the comments don't live in the same repository as the analyzer, each
+analyzer should have CI that checks if the comments used in that specific
+analyzer (those that can become output), are comments on the `main` branch, on
+the [`exercism/website-copy`][git-website-copy] repository.
+
+At the moment of writing, [this issue][issue-ci-comments] tracks the status of any
+generalization of this CI, if any.
+
 [git-website-copy]: https://github.com/exercism/website-copy/tree/main/analyzer-comments
 [issue-ci-comments]: https://github.com/exercism/automated-mentoring-support/issues/51
 [git-website-copy-label]: https://github.com/exercism/website-copy/pulls?q=is%3Aopen+is%3Apr+label%3Atype%2Fanalyzer-comments

building/tooling/analyzers/interface.md

@@ -44,15 +44,14 @@ The `analysis.json` file should be structured as followed:
 
 ### `summary` (optional)
 
-The summary field is a text (not markdown) field that summarizes the output.
+The `summary` field is a text (not markdown) field that summarizes the output.
 It might say something like "Your solution is nearly there - there's just two small changes you can make." or "The code works great, but there's a little bit of linting that needs doing.".
 This summary is rendered on the website above the comments.
 
 ### `comments`
 
-Comments are keys into `website-copy/automated-comments/`, e.g. [`ruby.general.explicit_return -> automated-comments/ruby/general/explicit_return.md`](https://github.com/exercism/website-copy/blob/47af5b309ac263629ca5c52904046f81e0cc8def/automated-comments/ruby/general/explicit_return.md).
-
-Then can be structured either as single pointer strings (e.g. the last example above) or using a JSON Object to specify the follow keys:
+The `comments` field is an array of comments that link to Markdown documents in the [`exercism/website-copy`][website-copy-repo] (see [Writing Analyzer comments][writing-analyzer-comments] for more information).
+Each value in the array is either a pointer-string or a JSON object with the following format:
 
 #### `comment`
 
@@ -77,11 +76,14 @@ The following `type`s are valid:
 
 Comments without a type field default to `informative `.
 
-Currently in the website, we soft-block on essential comments, encourage students to complete actionable comments before marking as complete on Practice Exercises (but not Concept Exercises), but don't suggest any action on `informative` or `celebratory`.
+Currently in the website, we soft-block on `essential` comments, encourage students to complete `actionable` comments before marking as complete on Practice Exercises (but not Concept Exercises), but don't suggest any action on `informative` or `celebratory`.
 However, in the future we may choose to add emojis or indicators to other types, or group them separately.
 
 ## Debugging
 
 The contents of `stdout` and `stderr` from each run will be persisted into files that can be viewed later.
 
 You may write an `analysis.out` file that contains debugging information you want to later view.
+
+[website-copy-repo]: https://github.com/exercise/website-copy
+[writing-analyzer-comments]: /docs/building/tooling/analyzers/comments