Migrate Linux data source to Homebrew/homebrew-core

[issyl0]

• Very WIP, opening this as very draft to a) chart my progress and b) give visibility to others who might want to do parts of Migrate to homebrew-core for Linux #566.
• At the moment this is just me messing around trying to remember what I did here when I implemented Linux formulae on this site - that was two years ago! 😱

My vague plan here:

• The low-hanging fruit, README, index page.
• Remove formula-linux/ and see what breaks. (This is where I currently am at the time of writing this.)
• Check we've got all the necessary Linux things on the macOS (I should say "formula") pages now.
• The rest of the things in Migrate to homebrew-core for Linux #566, unless anyone else gets to them first.

Jekyll is still a mystery to me, so I can't promise this'll be fast.

[EricFromCanada]

I've implemented the redirects for the HTML pages; namely, /formula-linux/ => /formula/ and /formula-linux/ƒ => /formula/ƒ. The JSON API, however, can't be properly redirected with what we have on GitHub Pages. Since redirect_from simply generates an HTML page with a meta refresh value in the old location, API users expecting to find a JSON file in the old locations will instead find HTML that they (probably) won't know what to do with.

[SMillerDev]

Could we just copy the api files and send this as a feature request to github?

[MikeMcQuaid]

The JSON API, however, can't be properly redirected with what we have on GitHub Pages. Since redirect_from simply generates an HTML page with a meta refresh value in the old location, API users expecting to find a JSON file in the old locations will instead find HTML that they (probably) won't know what to do with.

I think this is fine, we can just drop these files.

Could we just copy the api files and send this as a feature request to github?

Let's not. It won't be implemented in a meaningful timeframe and will bloat this repo until then.

[MikeMcQuaid]

This makes sense to me!

[Bo98]

Not that it blocked this or anything, but I've informed Repology that the Linux data source is now deprecated and will soon be no more.

[MikeMcQuaid]

@Bo98 Thanks!

[Bo98]

Done: https://repology.org/news

[issyl0]

Thanks for the help, Eric, and for ticking the checklist items in the issue. There's quite a lot complete here! ✨

Over this weekend I'll work on item 2, the formula page: splitting up the views by macOS and Linux dependencies/caveats/bottles and the analytics tables.

[issyl0]

On the "show macOS and Linux analytics in separate tables", I'd quite like to show them in one table with multiple columns inside for macOS and Linux, but I couldn't work out how to do it with all the loops and different paths for analytics and analytics-linux. Maybe that can be a future goal!

[EricFromCanada]

The issue there is how the stats table markup iterates through each formula's invocation form that appears in each data source, whether just the formula name or with options. To combine the tables would require creating a merged list of these from each data source before fetching the values for each form, if they exist. But, since in core the only possibilities are either [formula] or [formula] --HEAD, the loop could check for each form on every loop and only print the row if a value was present in either data source.

[EricFromCanada]

With for loops implemented and analytics-linux added, that's a wrap for the JSON templates. Although the same kind of looping could be added to the formula HTML template, it probably needs a complete redesign.

[issyl0]

3. the Linux/macOS dependencies, analytics (and perhaps: caveats, anything else?) should be separated when they differ between OSs (but still displayed on the same page). This requires updating the formula API to differentiate between macOS and Linux dependencies, conflicting formulae, keg_only status, and (possibly) caveat text.

This is the last thing to do here, and it feels quite big, and this PR is extremely large already. I know if we don't ship step 3 with steps 1, 2, 4 and 5 we'll be lacking functionality for a bit. But if people are pulling Linux data from this API now they're getting out of date stuff anyway). All that to say, does anyone else have more time than me to work on this part? @Rylan12?

[SMillerDev]

Am I missing something or is the easiest solution to stop splitting Linux and macOS analytics reporting?

[EricFromCanada]

Am I missing something or is the easiest solution to stop splitting Linux and macOS analytics reporting?

No, that part is done. The issue now is that anything in formulae that's potentially affected by an on_linux block isn't reflected in its JSON representation, so the formula API (and therefore the formulae.brew.sh pages) are showing Mac-only info. I'd say we're good to ship what we have and follow up with more once the formula API is updated.

[Rylan12]

This is the last thing to do here, and it feels quite big, and this PR is extremely large already. I know if we don't ship step 3 with steps 1, 2, 4 and 5 we'll be lacking functionality for a bit. But if people are pulling Linux data from this API now they're getting out of date stuff anyway). All that to say, does anyone else have more time than me to work on this part? @Rylan12?

Eventually, yes, I'd be happy to work on this a bit. At the moment, though, I'm a bit swamped with HOMEBREW_INSTALL_FROM_API issues and school (midterms start again soon 😭). If I end up with some more time and start to work on it, I'll leave a comment.

[MikeMcQuaid]

This is the last thing to do here, and it feels quite big, and this PR is extremely large already. I know if we don't ship step 3 with steps 1, 2, 4 and 5 we'll be lacking functionality for a bit. But if people are pulling Linux data from this API now they're getting out of date stuff anyway). All that to say, does anyone else have more time than me to work on this part? @Rylan12?

Yes, I'm fine to ship without 3 for sure.

[MikeMcQuaid]

❤️. Rebase this and merge whenever you get a chance (because there will be inevitable conflicts soon after otherwise).

[issyl0]

I think GitHub is confused by the sheer size of this PR. There were no conflicts when I rebased on master locally.

[SMillerDev]

Thanks @issyl0 and @EricFromCanada ! Without your contributions it'd be impossible to keep homebrew going. You can feel good knowing that you've made the world a lot better for homebrew users around the world! 👍 🎉

[Rylan12]

I think GitHub is confused by the sheer size of this PR. There were no conflicts when I rebased on master locally.

What I usually do to avoid problems like this is only commit the changes that are to the generation files and not the actual site files. Then, once the PR is merged, the automatic generation is triggered and the updated site is pushed as an additional commit.