BUG's in relref shortcode:

[The-going]

• A permanent link to the chapter cannot be obtained for all pages. Links are handled correctly for the title pages or `_index.md ' files and if such a page does not exist, an error message or a warning appears in the log, depending on the global settings. This is the correct operation of the code.
• For the rest of the pages. If the page exists, then we have a link to this page, and not to the chapter that is on this page. If the page does not exist, then we get a link to the current page, i.e. to itself. And no messages in the log or changes in the appearance of the link. This is unacceptable behavior.

This will not be good at all if the content of a multilingual site is organized in one directory, i.e. /_index.en.md ,_index.fr.md ,about.en.md ,about.fr.md

There is another problem of bad links, but it relates to the short code 'mermaid'. If you do something like:

{{< mermaid align="left" >}}
flowchart LR;
    S("testing") --> X["Test-1"]
    style S fill:#99ccff;
    click S "{{% relref "/core/_index.md#about-testing" %}}"
    click X "{{% relref "/core/testing.md#test-1" %}}"
{{< /mermaid >}}

For site: http://localhost:1313/website/
The link will look like: http://localhost:1313/core/testing/#test-1
And there should be: http://localhost:1313/website/core/testing/#test-1
For this construction, everything is fine in the text of the page: ["Test-1"]({{% relref "/core/testing.md#test-1" %}})

[McShelby]

So this issue is actually two.

For part I: Could you give me an example repo/zip and detailed instructions how to reproduce this

For part II: This is a hassle in the way Hugo generates sites and treat links. It already showed up serveral times in the past as with search and shortcode links. The implemented solution there is a bit tricky/hacky and can not be applied to this issue. So at the moment I don't have a solution for you.

[The-going]

For part II:

link in 'mermaid'. I found the place where this problem is located and will fix it later. This is a separate conversation, because it is related to other parts of the code.

[The-going]

OK, I've almost solved these problems. We will skip the last part about how the short code 'mermaid' handles links. I found a solution, but it only works for one content structure, i.e. a special case.

I will make an example of a test page to reproduce and a possible version of the code a little later today.

[McShelby]

As mentioned in #147:

For part I: That's a Hugo "issue" - if at least. I doubt that Hugo knows something about your headings/fragment identifiers and therefore has no chance to warn for it.

[The-going]

I doubt that Hugo knows something about your headings/fragment identifiers

Hugo knows everything about the files in the /content folder and about the page on the site itself, which it processes if it exists.
The problem is that I don't understand the syntax of commands well and have to write a lot of debugging code. It is for this reason that everything was delayed.

[McShelby]

But Hugo doesn't know about your headings. This is processed by the Markdown-Converter and never touched afterwards by Hugo. Therefore it can not warn if a fragment specifier isn't contained in your resulting HTML.

[The-going]

I'm going to tell you about everything in order

[The-going]

Let's try to understand the code that exists today.

{{- if in (.Get 0) "/_index.md" -}}
    {{- $paths := (split (.Get 0) "_index.md") -}}
    {{- $pagepath := index $paths 0 -}}
    {{- $anchor := index $paths 1 -}}
    {{- with .Site.GetPage "section" (trim $pagepath "/") -}}
        {{- ( printf "%s%s" $pagepath $anchor ) | relLangURL -}}
    {{- end -}}
{{- else -}}
    {{- with .Site.GetPage "section" (.Get 0) }}
        {{- .RelPermalink -}}
    {{- else -}}
        {{- .Get 0 | relref .Page -}}
    {{- end -}}
{{- end -}}

The first condition is a special case for a sequence of characters /_index.md .
The condition searches for this sequence in a string. I.e. for /_index.en.md the condition will not be met.
Or can it be fulfilled? It's a mystery to me.

Next, inside the condition, we initialize two variables $pagepath and $anchor.
This must be moved to the very beginning of the script. Then we can use them inside all subsequent conditional constructions.

{{- else -}}
    {{- with .Site.GetPage "section" (.Get 0) }}

This construction is performed if we have received an html page ready for publication and inside the condition we are on the territory of this page and all its attributes, including content, are available to us. But (Get 0) is not available, for example, or variables from the previous condition.
If $anchor is initialized at the beginning of the script and contains the value #chapter-1, we will not be able to find it on the page. He's not there. But there is a line like: <h2 id="chapter-1">Chapter 1</h2>

{{- .RelPermalink -}}

This line will return the correct link to the page only.

All of the above can be checked using a debugging short code:

<!-- First we need to initialize the necessary variables -->
{{ $hurl := (.Site.BaseURL) }}
<!-- current path for debugging -->
{{ $curpathf := $.Page.File}}
<!-- Split the file path and the ID key -->
{{ $spls := (split (.Get 0) "#") }}
<!-- We will print it for verification -->
{{ $anchor := index $spls 1 }}

{{ $pagepath := index $spls 0 }}

{{-  printf "\n* [ Get 0 = %s ]\n" (.Get 0) -}}
{{-  printf "\n* [ BaseURL = %s ]\n" $hurl -}}
{{-  printf "* [ relURL = %s ]\n" ( $hurl | relURL ) -}}
{{-  printf "* [ anchor = %s ]\n" $anchor -}}
{{-  printf "* [ current path Page = %s ]\n" $curpathf -}}
<!-- Get the page if it exists -->
{{- with .Site.GetPage $pagepath -}}
	{{- printf "( .Site.GetPage %s = TRUE )\n\n" $pagepath -}}
	<!-- Get the Anchor if it exists in page -->
	{{- if in ( .Content ) $anchor -}}
		{{- printf "* Anchor \"%s\" is exsist\n\n" $anchor -}}
	{{- else -}}
		{{ warnf "Do not get anchor [%s] for page [%s]" $anchor $pagepath }}
		{{- printf "* Anchor \"%s\" is **not** exist\n\n" $anchor -}}
	{{- end -}}
{{- else -}}
	{{ warnf "Do not get page with [%s] argument" $pagepath }}
{{- end -}}
{{- printf "%s\n" "---" -}}
{{- with .Site.GetPage $pagepath -}}
	{{- printf "* %s\n" (findRE "<h2.*?>(.|\n)*?</h2>" (.Content)) -}}
{{- end -}}
{{- printf "%s\n" "---" -}}

[The-going]

I still have an open question how to display "broken" bad links. Whether or not they can be colored red, if the code returns a link to the page from which it was called.

[The-going]

Add an additional check for the existence of an anchor in the index.

<!-- First we need to initialize the necessary variables -->
{{ $hurl := (.Site.BaseURL) }}
<!-- current path for debugging -->
{{ $curpathf := $.Page.File}}
<!-- Split the file path and the ID key -->
{{ $spls := (split (.Get 0) "#") }}
<!-- We will print it for verification -->

{{ $anchor := index $spls 1 }}
{{ $pagepath := index $spls 0 }}

{{-  printf "\n* [ Get 0 = %s ]\n" (.Get 0) -}}
{{-  printf "\n* [ BaseURL = %s ]\n" $hurl -}}
{{-  printf "* [ relURL = %s ]\n" ( $hurl | relURL ) -}}
{{-  printf "* [ anchor = %s ]\n" $anchor -}}
{{-  printf "* [ current path Page = %s ]\n" $curpathf -}}

<!-- Get the page if it exists -->
{{- with .Site.GetPage $pagepath -}}
	{{- printf "( .Site.GetPage %s = TRUE )\n\n" $pagepath -}}
	<!-- Get the Anchor if it exists in page -->
	{{- if index $spls 1 -}}
		{{-  printf "* The anchor [ %s ] on the chapter exists and can be checked\n" $anchor -}}
		{{- if in ( .Content ) $anchor -}}
			{{- printf "* Anchor \"%s\" is exsist\n\n" $anchor -}}
		{{- else -}}
			{{ warnf "Do not get anchor [%s] for page [%s]" $anchor $pagepath }}
			{{- printf "* Anchor \"%s\" is **not** exist\n\n" $anchor -}}
		{{- end -}}
	{{- else -}}
		{{-  printf "* The anchor on the chapter **DOES NOT** exist. There is no need to check\n" -}}
	{{- end -}}
{{- else -}}
	{{ warnf "Do not get page with [%s] argument" $pagepath }}
{{- end -}}

{{- printf "%s\n" "---" -}}
{{- with .Site.GetPage $pagepath -}}
	{{- printf "* %s\n" (findRE "<h2.*?>(.|\n)*?</h2>" (.Content)) -}}
{{- end -}}
{{- printf "%s\n" "---" -}}

[The-going]

@McShelby Yesterday I finished my version of the relref short code.
You can see and check how this code works on the testpage branch of my repository.

This code works well on your example site and for a small number of translated pages of the site I'm currently working on.
But when I turn on this code for the whole site, the "fast" Hugo goes down and ends on timeout. 7 minutes is not enough for him.

I have a large number of cross-references. And links to anchors that are generated during page rendering, i.e. they are not in the source file, but the author has indicated links to them. And this is a really big problem.

[McShelby]

My guess is the regex. I'am not a big fan of scanning the content. Although a lot of Hugo's internal mechanism are regex-based, it's usually not a good idea to use a regex to "parse" HTML.

[The-going]

My guess is the regex. I'am not a big fan of scanning the content.

I am a big fan of regular expressions and I made the script on awk for the task under discussion in 15 minutes. But... but here the problem is different. The problem is that I do not know the algorithm by which hugo processes pages. In what order does he take these pages? But I assume the following. Hugo took the first file. It contains ten links to other pages or anchors on these pages. For each link, a short code is called that tries to get the finished page and check the existence of an anchor on it. I.e. hugo starts processing these ten pages. On each of these ten pages there are also links to the other 10 pages. The second iteration gives the number 100. The website I am translating has a total of 132 pages in English. Three iterations cover the entire site, but this does not happen. `Hugo' lies down and can't do anything. 14 minutes of work allowed us to process ~ 60 pages. We should change our strategy here.

[McShelby]

I also love regex, too. But only for problems it can really solve. Parsing HTML is not really one of them. Although I must admit I do this myself from time to time ;-) (Did this in a now discontinued public project - and it was a pain)

From my experience with the handling (and issues) of using a page's scratch, I assume that Hugo processes pages in parallel or at least in undeterminate order.

Not sure here, but maybe there is unlimited recursion involved. If you are calling GetPage Hugo may have to process this page first. If it has a link back to the initial page, you may be hung in a loop

[The-going]

I fully agree with your conclusion. The idea of checking pages from a short code while running hugo is a very bad thing. Moreover, his task is to make the right switch, and that's all.

[The-going]

The strategy for checking broken links on pages can be as follows:

• 1. Allow hugo to process the entire site.
• 2. and last of all, call the verification script, which will search through all the pages already ready for publication.
     It should be a separate short code.
     Will it be possible to call it on the very last empty page?
     Or how can it be called?

[McShelby]

I've created issue #156 out of this discussion. Sadly it did not convert the discussion to issue comments.

[The-going]

I will transfer the result of my research to #156 and we continue.

[McShelby]

The opened issue ended in a breaking change.

I've added extensive migration instructions to the documentation with the essentials of this conversation.