A Library Fetcher/Updater Script via cURL

[tajmone]

Since PB doesn't have a package manager (and it's unlikely it will ever have a native one), I've been wondering about the possible solutions to allow end users to update the RegEx Engine library.

Probably the easiest solution would be to provide a shell script that uses cURL to fetch the latest library modules, based on a release tag supplied as a parameter. Since this script would leverage Git, it assumes that even Windows users have access to Bash (via Git Bash), and cURL is natively available on all the target OSs, Windows included (Win 10 or above).

So, the end user will only have to place this script in his working folder, and by executing it he/she will be able to download all the required modules, or update them to a specific release tag.

The script should also support checking which is the latest release tag, to automatically update the modules to their latest version (unless a specific version tag is passed).

This should work quite efficiently, especially in third party repositories, where a main script can be used to invoke the fetcher script with the required library version, thus allowing the repo owner to control which library version the project should use at any time. The actual RegEx modules could even be Git ignored, since the script is used to fetch them and there's no actual need to include them in the project, given that end users will end up having the same modules version as long they run the script whenever its updated.

This IMO is the simplest workaround to the lack of a package manager, and cURL is flexible enough to allow smart fetching of files according to need.

[SicroAtGit]

Good idea!

Bash (via Git Bash)

I think Bash (via Git Bash) should not be a requirement. Everyone has PureBasic installed, so why not use that for the task? PureBasic can execute compiled code directly.

The PureBasic compiler does not provide a way to define parameters that will be passed to the compiled program at execution time, but it is possible to create constants at compile time:

$ pbcompiler updater.pb --constant version="2.0"

The script should also support checking which is the latest release tag

For this we can use the GitHub API (Get the latest release) with the native PB function HTTPRequest().

If we plan to create such a tool, we should create a separate repository for this project and discuss development there.

[tajmone]

Everyone has PureBasic installed, so why not use that for the task? PureBasic can execute compiled code directly.

The problem is that PB doesn't add the compiler path to the system PATH env. var (at least, not under Windows), so you can't just invoke it from a shell/CMD like a system command, you actually need to open the PB source in the IDE, which is a hassle.

Another reason would be script automation, because different OSs store the PB compiler in different paths (and only if the default settings are accepted when installing, but it could be installed in a custom path, as is often the case for Win OS users).

So, using PB for this task probably makes sense only if it's going to be a generic updater IMO, i.e. not just for this project, for in that case the user will add the compiled binary to the system PATH.

For this we can use the GitHub API (Get the latest release) with the native PB function HTTPRequest().

If there's a way of doing it by leveraging pure Git commands is better, so it wan't be restricted to GitHub only. Also, the GH API tends to change frequently, so in a year time it might bump a major version (I'm not entirely sure about this, but last time I looked into it I did notice that it had bumped up MAJOR versions quite fast).

If we plan to create such a tool, we should create a separate repository for this project and discuss development there.

Agree. But maybe in the meantime we could keep using this thread to elaborate on the general features of the tool and the dev strategy, so we'll have a clearer view in mind when it comes to creating a dedicated repository (from having a good name for it, up to a solid features plan for v1.0). But this is definitely something needed, in general, because right now PB really suffers from the lack of any tools to manage dependencies.

[tajmone]

Git vs GH API

Actually, I might be wrong about favoring Git over the GitHub API, since end users of the modules manager might not have Git installed at all — i.e. if the goal of this tool should be to allow end users to install and update third party modules without needing Git to be installed.

The GH API allows the PB tool to bypass Git entirely but, as mentioned, it also restricts the tool to GitHub repositories (which is not a small restriction). In theory, the tool could support the API of other Git hosting services, but that would add a significant burden in terms of programming and (most of all) maintenance (not to mention that simpler, self-hosted Git services might not have such an API at all).

Does the GH API require a GitHub account for basic read operations? or can it operate without credentials? If an account is required, then we might as well assume that the end user has Git installed, and hence look for a Git-based solution, which is more "universal".

[SicroAtGit]

The problem is that PB doesn't add the compiler path to the system PATH env. var

Yes, with the PureBasic IDE there is the feature to register multiple PB compiler versions and then select which compiler to use in the compiler options of the respective code. The problem is that all PB compiler versions have the same filenames, which would lead to a filename conflict in PATH.

Another reason would be script automation, because different OSs store the PB compiler in different paths (and only if the default settings are accepted when installing, but it could be installed in a custom path, as is often the case for Win OS users).

If someone has set up automation, he/she will probably have put the PB compiler in PATH. But clearly, we cannot assume that this is the case.

So, using PB for this task probably makes sense only if it's going to be a generic updater IMO, i.e. not just for this project, for in that case the user will add the compiled binary to the system PATH.

Yes. If it were just for this project, I couldn't imagine anyone taking the effort to install Git just to automate the downloading/updating of this module.

If there's a way of doing it by leveraging pure Git commands is better, so it wan't be restricted to GitHub only.

If we plan to have a generic updater, I agree that it can have Git as a requirement.

However, with the GitHub API, it would be easier to determine the release tags, because not all tags are also releases.

Also, the GH API tends to change frequently, so in a year time it might bump a major version (I'm not entirely sure about this, but last time I looked into it I did notice that it had bumped up MAJOR versions quite fast).

When sending a request to the GitHub API, you can specify which API version to use in the HTTP header data. So I think that older API versions will be supported for a while.

But maybe in the meantime we could keep using this thread to elaborate on the general features of the tool and the dev strategy, so we'll have a clearer view in mind when it comes to creating a dedicated repository (from having a good name for it, up to a solid features plan for v1.0).

Ok, until the name of the tool and what core features it should have is decided, I agree to continue the discussion here.

But this is definitely something needed, in general, because right now PB really suffers from the lack of any tools to manage dependencies.

Yes, we both have this discussion for a very long time now. I'm still toying with the idea of building a full package manager, but it's too time-consuming.

Does the GH API require a GitHub account for basic read operations?

The functions we would need do not require an authorization token to be specified in the HTTP request header.

[SicroAtGit]

I thought about it again today.

Even if it is more time-consuming, we should rather create a full package manager. It feels more right to me. Since the idea has been floating in my head for a long time, I also have a good picture of how we could create such one.

Also, once the name of the package manager is decided, we should create an empty repository where we continue the development discussion. I think it's better if the initial development discussions are also in the appropriate project repository.

What do you think about the name PureGet for a package manager for PureBasic? As far as I know, the prefix Pure was sometimes used with things that were PureBasic related. With Google, I have not found any tools that already have this name, so it would be ideal.

[tajmone]

I also have a good picture of how we could create such one.

Excellent. Although I did give it some thought from time to time, I didn't come up with a clear strategy yet. The main deterrent has always been the fact that PB versioning scheme doesn't really provide info about backward compatibility.

I think it's better if the initial development discussions are also in the appropriate project repository.

You should be able to move a discussion to another repository with the new Beta features enabled.

PureGet sounds fine to me — simple and direct to the point.

With Google, I have not found any tools that already have this name, so it would be ideal.

Unfortunately there's a language called Pure, so the "pure" prefix is often associated to tools for that language:

https://agraef.github.io/pure-lang/

It's also been around for quite some time, and it's more popular than PB on GitHub.
But probably that wouldn't be a problem for PB users, since the chances that they are using both languages are slim. Also, discovery shouldn't be a problem since this would be a package manager, and it would help finding packages that are PB related only, and as for the package manager itself my guess is that people would simple know were to find it.

But on GH I did bump into many repos starting with pure- that were for the Pure language, while seeking PB projects, so it's worth keeping this in mind.

As far as I know, the prefix Pure was sometimes used with things that were PureBasic related.

That's true, but now that SpiderBasic is out, and because often the same source can compile with both, it's harder to stick to either pure or spider as prefixes, unlike before. The only common name part they share is basic, but that would be too generic. I never managed to find an alternative that fits both, the only thing that came to my mind was Fantaise, since they are both by the same company, but that being the company name I'm not sure how well it would go down.

[SicroAtGit]

I have now created the PureGet repository for the package manager project.

There I will start a new discussion and describe how I envision the package manager in detail.

[tajmone]

Why didn't you create it at FossyCats?

[SicroAtGit]

Moved. Right, it's better.