Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gitea fetcher for flake inputs #11467

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 75 additions & 0 deletions src/libfetchers/github.cc
Original file line number Diff line number Diff line change
Expand Up @@ -569,8 +569,83 @@ struct SourceHutInputScheme : GitArchiveInputScheme
}
};

struct GiteaInputScheme : GitArchiveInputScheme
{
std::string_view schemeName() const override { return "gitea"; }

std::optional<std::pair<std::string, std::string>> accessHeaderFromToken(const std::string & token) const override
{
// Gitea supports OAuth2 tokens and HTTP Basic
// Authentication. The former simply specifies the token, the
// latter can use the token as the password. Only the first
// is used here. See
// https://docs.gitea.com/development/api-usage#authentication
return std::pair<std::string, std::string>("Authorization", fmt("token %s", token));
}

std::string getHost(const Input & input) const
{
return maybeGetStrAttr(input.attrs, "host").value_or("codeberg.org");
}

std::string getOwner(const Input & input) const
{
return getStrAttr(input.attrs, "owner");
}

std::string getRepo(const Input & input) const
{
return getStrAttr(input.attrs, "repo");
}

RefInfo getRevFromRef(nix::ref<Store> store, const Input & input) const override
{
auto host = getHost(input);
auto url = fmt("https://%s/api/v1/repos/%s/%s/commits?sha=%s", host, getOwner(input), getRepo(input), *input.getRef());
Copy link
Member

@Mic92 Mic92 Sep 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about http with the s? Might be used for testing.


Headers headers = makeHeadersWithAuthTokens(*input.settings, host);

auto json = nlohmann::json::parse(
readFile(
store->toRealPath(
downloadFile(store, url, "source", headers).storePath)));

return RefInfo {
.rev = Hash::parseAny(std::string { json[1]["sha"] }, HashAlgorithm::SHA1),
.treeHash = Hash::parseAny(std::string { json[1]["commit"]["tree"]["sha"] }, HashAlgorithm::SHA1)
};
}

DownloadUrl getDownloadUrl(const Input & input) const override
{
auto host = getHost(input);

Headers headers = makeHeadersWithAuthTokens(*input.settings, host);

// If we have no auth headers then we default to the public archive
// urls so we do not run into rate limits.
const auto urlFmt = headers.empty() ? "https://%s/%s/%s/archive/%s.tar.gz" : "https://%s/api/v1/repos/%s/%s/archive/%s.tar.gz";

const auto url = fmt(urlFmt, host, getOwner(input), getRepo(input),
input.getRev()->to_string(HashFormat::Base16, false));

return DownloadUrl { url, headers };
}

void clone(const Input & input, const Path & destDir) const override
{
auto host = getHost(input);
Input::fromURL(*input.settings, fmt("git+https://%s/%s/%s.git",
host, getOwner(input), getRepo(input)))
.applyOverrides(input.getRef(), input.getRev())
.clone(destDir);
}
};


static auto rGitHubInputScheme = OnStartup([] { registerInputScheme(std::make_unique<GitHubInputScheme>()); });
static auto rGitLabInputScheme = OnStartup([] { registerInputScheme(std::make_unique<GitLabInputScheme>()); });
static auto rSourceHutInputScheme = OnStartup([] { registerInputScheme(std::make_unique<SourceHutInputScheme>()); });
static auto rGiteaInputScheme = OnStartup([] { registerInputScheme(std::make_unique<GiteaInputScheme>()); });

}
27 changes: 27 additions & 0 deletions src/nix/flake.md
Original file line number Diff line number Diff line change
Expand Up @@ -364,6 +364,33 @@ Currently the `type` attribute can be one of the following:
* `sourcehut:~misterio/nix-colors/182b4b8709b8ffe4e9774a4c5d6877bf6bb9a21c`
* `sourcehut:~misterio/nix-colors/21c1a380a6915d890d408e9f22203436a35bb2de?host=hg.sr.ht`

* `gitea`: Similar to `github`, is a more efficient way to fetch
Gitea/Forgejo repositories. The default host is `codeberg.org`.
The following attributes are required:

* `owner`: The owner of the repository.

* `repo`: The name of the repository.

Like `github`, these are downloaded as tarball archives.

The URL syntax for `gitea` flakes is:

`gitea:<owner>/<repo>(/<rev-or-ref>)?(\?<params>)?`

`<rev-or-ref>` works the same as `github`. Either a branch or tag name
(`ref`), or a commit hash (`rev`) can be specified.

Since Gitea/Forgejo allows for self-hosting, you can specify `host` as
a parameter, to point to any instances other than `codeberg.org`.

Some examples:

* `gitea:redict/redict`
* `gitea:redict/redict/main`
* `gitea:redict/redict/a4c81102327bc2c74d229784a1d1dd680c708918`
* `gitea:lix-project/lix?host=git.lix.systems`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we actually should default to codeberg and not just make the host part of the url? This doesn't look much more complex and promotes self-hosting (which is in the spirit of gitea):

Suggested change
* `gitea:lix-project/lix?host=git.lix.systems`
* `gitea:git.lix.systems/lix-project/lix`

Copy link
Member

@Mic92 Mic92 Sep 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we should also support this?

  • gitea:git.lix.systems/some-subdirectory/lix-project/lix

Also some-subdirectory could become a url parameter

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we should support sub-directory. Though, I'd like to keep the host argument, so the URL doesn't feel to far off compared to the existing github: and sourcehut: URL types.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

* `gitea:git.lix.systems/some-subdirectory/lix-project/lix`

Also some-subdirectory could become a url parameter

What do you mean by subdirectories?

Subdirectories in the gitea repository are already supported with the dir url parameter.

Copy link
Author

@tilmanmixyz tilmanmixyz Sep 10, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we actually should default to codeberg and not just make the host part of the url? This doesn't look much more complex and promotes self-hosting (which is in the spirit of gitea):

I don't think we should actually default to codeberg becuase. I just use codeberg and wanted to quickly simplify my flakes.

I currently have an implementation which requires the specification of the host in the url in the gitea-fetcher-explicit-url branch of my fork.
But currently it makes the host parameter useless since it requires it in the scheme.


# Flake format

As an example, here is a simple `flake.nix` that depends on the
Expand Down
Loading