Check links in toml and json files

[angelikatyborska]

We're adding the same link checker to the Elixir repo to catch dead links (PR) and I noticed it's failing on some URLs that come from this repository, three URLs from https://users.csc.calpoly.edu. How could that be? Well, this repository contains exercise metadata in toml files, which contain source URLs, which do not get checked.

Adding the check for toml and json files increases the number of links checked from 98 to 212.

This is a draft because I'm not sure yet how to fix those three URLs... I will just ignore them.

| ✗ [ERR] https://users.csc.calpoly.edu/~jdalbey/103/Projects/ProgrammingPractice.html | Failed: Network error: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1921: (unable to get local issuer certificate)
```~

[wolf99]

Really great that you've identified this!
If I get time I'll investigate also, but otherwise Iooking forward to other's thoughts on it.

[wolf99]

I see that the elixir track is adding checking of JSON files also.
Would it be good to add that here too?

[angelikatyborska]

Yes, absolutely!

What I tried to far: apt update && apt upgrade ca-certificates, but that didn't help :(

[angelikatyborska]

I'll just add that server to ignored links. It looks like it's SSL certificate is misconfigured or something.

The PR can be reviewed, but it requires #2293 to be merged first to have green CI.

[kytrinyx]

I've rebased onto main.

[kytrinyx]

This is a nice addition, thank you!

[kytrinyx]

All the twitter links are timing out right now. I tried re-running the link checking job, but it's still timing out on the twitter links. Let's try rerunning it later to see if it manages to get through it.

[angelikatyborska]

I forgot about this PR 🤦

All Twitter links are still timing out. I stopped using Twitter a while ago, but I think currently not logged in users are only allowed to see pages for single tweets. They are not allowed to see profile pages for example. I'm not allowed to access https://twitter.com/jeg2. And scripts aren't allow to access anything. curl -I on a valid tweet URL always returns 403.

I think the only solution is for us to remove all Twitter links from this repo. People cannot access them anyway.

Unrelated, the web archive is returning 503s today. I hope that's temporary.

[ErikSchierboom]

All Twitter links are still timing out. I stopped using Twitter a while ago, but I think currently not logged in users are only allowed to see pages for single tweets. They are not allowed to see profile pages for example. I'm not allowed to access https://twitter.com/jeg2. And scripts aren't allow to access anything. curl -I on a valid tweet URL always returns 403.

That is absolutely bonkers, but it is true! 🤦

I think the only solution is for us to remove all Twitter links from this repo. People cannot access them anyway.

I agree. We can always restore them if something changes.