Support private GitLab repositories

[ceddlyburge]

Hi There

After a lot of trawling the internet I am fairly sure that SourceLink does not support private GitLab repositories. Based on my reading I think this is because GitCrendentialManager will only use Basic Authentication, and GitLab will not accept Basic Authentication.

Firstly, can someone confirm that this is the case?

Secondly, is there a plan to fix this? SourceLink is near useless to us unless we can use it with private repositores. My initial thought is that its not acceptable to ask GitLab to support Basic Authentication, being as it is basically a bad idea.

Thanks

Cedd

[tmat]

@chuckries

[chuckries]

I can confirm that this is the case: GitLab does not support raw source file access using Basic Authentication. Additionally, Git Credential Manager for Windows does not support GitLab's OAuth mechanism.

Today there is not a good plan moving forward. Personally I hope that GCM will support GitLab's OAuth and we can use some personal access token mechanism, as is done for GitHub via GCM. The branded login/OAuth support for GitHub and BitBucket was contribued to GCM by people from those companies. I hope that GitLab will work with GCM to provide support.

As for Basic Auth being a bad idea: basic auth is very common in git scenarios and is mitigated by HTTPS. GitHub and BitBucket both support basic auth for APIs (as long as 2FA is disabled). When in 2FA scenarios, GCM makes use of Personal Access Tokens for GitHub.

[ceddlyburge]

Hi there, thanks for the confirmation, I think its useful to have it written down definitively.

I take your point about https, although our policy is that everyone must have two factor auth enabled.

Authentication always seems to be a pain, even with the (Microsoft I think) standard OAuth. It feels like it is something that we have got wrong as an industry.

[swadamson]

Have subscribed to updates on this thread as this is something we need as well.

[TFTomSun]

I've written a small proxy application that I start locally and modified the source link target to localhost. Within the little proxy app I do the authentication (access token based), download the file and forward it to the debugger. That approach works with enterprise on premise gitlab installations, but the disadvantage is that you can only debug if that little proxy tool is running. Is anyone interested in that kind of workaround?
Currently I write the localhost link into the pdb during the build of the package library. A better approach might be a "on the fly modification of the pdbs". That means, only write that localhost based address into the pdbs that are shipped with the nuget package, when I use that tool.

[olivier-spinelli]

Yes it is... but not for on premise Gitlab. When you wrote "works with enterprise on premise gitlab", did you mean ONLY or ALSO?

[TFTomSun]

I think it works for the public gitlab as well. I don't see any reason why it shouldn't work. The special case I had, was a enterprise specific authentication proxy that prevented me from downloading the files from the on premise gitlab instance via a simple http request.

[olivier-spinelli]

We would be interested and may test the public gitlab access.

[OlegLevitsky]

Hello there. I've got the same problem, so...
First of all, @chuckries , thnx for so good explanation, it's brilliant!
@ceddlyburge , @swadamson , @TFTomSun , @olivier-spinelli
Please, try the following:

1. Open Windows Credential Manager (Control Panel\All Control Panel Items\Credential Manager)
2. Navigate to the 'Windows Credentials' tab
3. Add a new Generic credential with the desired network address in the following format:
   git:http://your.gitlab.com/
4. Specify user and password
   Even if you will add the user as 'PersonalAccessToken' and password as a generated token - Visual Studio works with it.
   It looks like Git Credential Manager does the same. At least, for GitHub connection.

[bcrosnier]

To my knowledge, Visual Studio still uses basic authentication to retrieve source files. If the credentials inside Visual Studio aren't enough, it falls back to GCM credentials as #281 (comment) points out (if the option is enabled in Debugging/Options).

Since Gitlab doesn't support basic authentication anyway, even if the GCM credentials are valid, having valid GCM credentials isn't going to help.

This is going to work with other providers like Github, Bitbucket and ADO because:

• Github supports basic auth using a PAT,
• Bitbucket supports basic auth using a PAT,
• Azure DevOps partially supports basic auth using a PAT.

Gitlab suppports other means of authentication with a PAT, such as:

• Having the PAT as a GET query parameter, which is a workaround used by Using private repository on self Hosted gitlab instance #135 (comment)
• Making the request with a Private-Token header,
• Making the request with a Bearer token (Authorization: Bearer header).

All means of authentication using a PAT with Gitlab are documented here.

For basic authentication, Gitlab is tracking this since Aug 18, 2017 (gitlab-org/gitlab#19189 Allow access to raw files using HTTP Basic Auth)

And for GCM itself:

• gitlab-org/gitlab#19178 Add support for Git Credential Manager on Windows
• Add support for GitLab and GitLab self-hosted instances microsoft/Git-Credential-Manager-for-Windows#480

If you self-host Gitlab, you can use https://github.com/rgl/gitlab-source-link-proxy, which is a proxy that passes requests on. I suppose it could work for the usual Gitlab too, but it requires editing both the SourceLink URL and the GCM authorities, which is everything but user-friendly.

I see two solutions moving forward:

• Gitlab could resolve the aforementioned issues, supporting at least basic auth for the scope of this issue,
• The Visual Studio Product Team could make a special exception for Gitlab and use their own authentication mechanism instead of basic auth.

Unfortunately, both Gitlab issues are stale, and I don't see the VSPT adding Gitlab-specific code, so I suspect this issue will stay open for another couple of years.

Edit: And as of today I don't even know if dotnet/sourcelink has any relation with how Visual Studio retrieves the source, besides setting the URL. Maybe the maintainers can answer this, but it might be more appropriate to poke the VSDC directly.

[olivier-spinelli]

@bcrosnier are you real? I'll definitely ask for the same OS as yours next christmas...

[lyling]

Hello there. I've got the same problem, so...
First of all, @chuckries , thnx for so good explanation, it's brilliant!
@ceddlyburge , @swadamson , @TFTomSun , @olivier-spinelli
Please, try the following:

1. Open Windows Credential Manager (Control Panel\All Control Panel Items\Credential Manager)
2. Navigate to the 'Windows Credentials' tab
3. Add a new Generic credential with the desired network address in the following format:
   git:http://your.gitlab.com/
4. Specify user and password
   Even if you will add the user as 'PersonalAccessToken' and password as a generated token - Visual Studio works with it.
   It looks like Git Credential Manager does the same. At least, for GitHub connection.

@OlegLevitsky
Dont your mind tell more detail ? whats the generated token ?

[darjanbogdan]

@lyling it's an actual value of the PersonalAccessToken, nevertheless it doesn't work if you are using gitlab.com

[AaronLayton]

Tracking this issue also

[ourdark]

mark, this is critical issue for devops

[sthames42]

Working in Windows, I've managed to solve this with a Proxy Server. Perhaps it can help someone else.

[AaronLayton]

Working in Windows, I've managed to solve this with a Proxy Server. Perhaps it can help someone else.

Hmm thanks @sthames42 I will give this a go tomorrow and report back :-)

[darjanbogdan]

GCM team managed to add GitLab support. Even though I haven't had chance to test it yet, it should be possible to use source linking with GitLab.com from now on.

More info:

• https://gitlab.com/gitlab-org/gitlab/-/issues/225215#note_898150816
• https://github.com/GitCredentialManager/git-credential-manager/releases/tag/v2.0.692
• https://github.com/GitCredentialManager/git-credential-manager/blob/main/docs/gitlab.md

[chuckries]

@darjanbogdan thanks for the info! I will look into this again.

[chuckries]

I had a chance to look at the new GitLab support in GCM. I was able to get an OAuth2 token from it, but there are still several issues integrating with Visual Studio Source Link. For the purposes of this conversation, I am speaking about GitLab.com, and not a self-hosted instance. I do not have access to a self-hosted GitLab for testing, I am only able to test against public.

First, two statements of fact:

• VS uses header-based authentication for Source Link requests. For GCM scenarios, this header is typically treated as "Basic" auth. This could be extended in the future to supply "Bearer" in known scenarios, but the flexibility to change this doesn't currently exist.
• The Microsoft.SourceLink.GitLab package generates web-based URLs (as opposed to API based URLs) for GitLab. This can be seen here. This is equivalent to the URL seen when clicking the "raw view" on a file in the GitLab website.

These two things combine to make the tokens provided by GCM for GitLab currently incompatible with VS Source Link. As far as I can tell, the web-based URL's do not accept header-based auth. Trying to use the token from GCM as either "Basic" or "Bearer" tokens gives a redirect to a sign in page which results in a 503.

I also took a look at using GitLab API based URL requests with the token provided by GCM. Unfortunately, this returns 401 Unauthorized for everything I tried. I believe the issue is the scope of the OAuth token obtained by GCM:

• GCM GitLab uses the write_repository scope. According to GitLab docs, this scope "Grants read-write access to repositories on private projects using Git-over-HTTP (not using the API).". I believe this means that the token will do all things git.exe related (clone, push, etc), but is not useful for Source Link, which would require API read access.
• I was unable to change the scopes and test. I believe this is because the GCM GitLab Oauth Application owned by GitLab is not configured to provide any other scopes, which makes sense.

Conclusions:

• If GitLab can enable header based authentication to raw web URLs, authentication to existing Source Link'd packages may some day work.
• If header based auth to web URLs will never be available, the Microsoft.SourceLink.GitLab package should be updated to produce API based URLs. Visual Studio could potentially translate the web-based URL to an API URL. cc @tmat
• In either case, it does not appear that the tokens provided by GCM are valid for anything but Git over HTTP (i.e. git.exe CLI operations) and will not work for Source Link. The scope used by GCM must be expanded in order to provide authentication for Source Link.

If anybody from GitLab or the Git Credential Manager project can speak to these points, I would really appreciate some insights.

[christianerbsmehl]

Since the missing read_repository scope has now been added (see git-ecosystem/git-credential-manager#669), things should work now with the latest version of Git Credential Manager and private Gitlab instances.
I followed this guide but however, I didn't manage to get it working properly. The Gitlab login dialog appears, and I am able to sign in to my private Gitlab instance. Scopes are also set correctly:

But somehow, SourceLink is still not able to download source files and shows following error:

Source Link Error:
git-credential-manager-core.exe: Could not obtain credentials. Process failed with exit code -1.
ERROR: The request returned an unexpected mime type (text/html).

My assumtion is, that Gitlab still rejects the access token it gets from SourceLink/Git Credential Manager and redirects to the login page (we are using SSO with Active Directory). Is there any way to verify this behavoir?

Additionally, I came across this issue but I don't understand if this could still be a showstopper. Any thoughts on this?

[czd890]

I can access https://gitlab.com/my_private_repo/raw/b80730a8f00280424cb3fb1d8d18e0d6fedb7d78/******.cs in my chrome.
but source link not working in visual studio 17.3.2. it show ERROR: The request failed with code 503 : "Service Temporarily Unavailable".

'/_/******.cs'. Checksum: SHA256 {70 d2 31 78 cf 7d b1 6a 14 b f7 96 e5 12 d5 4f 7 7a 18 57 1c 25 6f 43 57 24 7 3f 2 a0 83 82}
Looking in script documents for '/_/******.cs'...
Looking in the projects for '/_/******.cs'.
The file was not found in a project.
Searching for documents embedded in the symbol file.
An embedded document was not found.
Looking for source using Source Link (https://gitlab.com/*path*/******.cs)...
Source Link errors:
ERROR: The request failed with code 503 : "Service Temporarily Unavailable".
The file was not found with Source Link.
Looking for source using source server...
The debugger will ask the user to find the file: /_/******.cs.
The user pressed Cancel in the Find Source dialog. The debug source files settings for the active solution have been modified so that the debugger will not ask the user to find the file: /_/******.cs.
The debugger could not locate the source file '/_/******.cs'.

Applications has been added.

git clone https://gitlab.com/my_private_repo is working

[peeveen]

I tried using a proxy approach as suggested by a few users above, but even though it successfully pulled the code from GitLab, the debugging experience in Visual Studio was impossible. On every step-over/step-into attempt, VS seems to be attempting to validate the checksum of the source code, and failing due to a difference in encoding (UTF8/UTF8-BOM, and CRLF/LF). I'm struggling to think of a reliable way to translate the raw file content in my proxy service.

[jwfx]

I noticed two issues that lead me to believe that the GCM integration in VS is still broken (at least for GitLab).

1. When VS opens the GCM dialog after triggering Source Link, my perfectly valid Windows Credential Manager entry is just gone.

2. Additionally the GCM dialog itself does not work properly when triggered by VS. When using the "browser" OAuth scheme and clicking on the Sign in button on the GCM dialog, nothing ever happens. It should open a browser window.

I wouldn't be surprised if these two issues above also apply to the regular Token based auth.

@christianerbsmehl
Isn't the whole point of the VS GCM integration exercise not having to rely on basic auth? I might be wrong here.

[jwfx]

Ok, I took another stab at this with Fiddler.

It looks like VS is still trying to retrieve the file from GitLab using Basic Auth, which we already know is not supported

GitLab allows supplying the token via header or query string, see here.
Just either one needs to be implemented on VS side for this to work.

[Genbox]

Hi everyone.

I've spent some time deciphering the issues and found a simple solution that only requires installing a NuGet package.

Check it out: Genbox.GitLabSourceLink

It works the same way as Microsoft.SourceLink.GitLab, except it also works for self-managed GitLab instances.

I would have liked to contribute it to the official source link packages, but I saw no way to implement it due to limitations in how it produces sourcelink.json. Let me know if anyone from the sourcelink team wants to make it work in the official Microsoft.SourceLink.GitLab package.

[akak1977]

Hi! Very interesting! But the reference looks broken (404)... Or you point to the private repos.

[Genbox]

Hi! Very interesting! But the reference looks broken (404)... Or you point to the private repos.

What reference are you referring to? The examples in the repo only works when the code is hosted on a GitLab instance. It uses the URL of the git remote.

Edit: I'm a big doofus. The repository was private. I've fixed that now :)

[peeveen]

As of recent releases (certainly in 16.7), GitLab has started responding to unauthenticated file requests on non-public projects with a 404, rather than the expected 401 or 403. I think this kinda snookers SourceLink in a whole other way.

[Genbox]

As of recent releases (certainly in 16.7), GitLab has started responding to unauthenticated file requests on non-public projects with a 404, rather than the expected 401 or 403.

Just to be sure - you are talking about gitlab.com repos right? I ask because they have had that behavior on self-hosted GitLab instances forever. It begins to seem intentional, but it is buggy behavior.

[peeveen]

Just to be sure - you are talking about gitlab.com repos right? I ask because they have had that behavior on self-hosted GitLab instances forever. It begins to seem intentional, but it is buggy behavior.

Yeah, you're absolutely right ... I was getting confused about how our proxy works.
I know that this 404/401 thing isn't the cure-all for SourceLink on private GitLab, but I thought I'd log an issue anyway. They seem to have relented for their npm package feed, so maybe they're open to implementing that change elsewhere ...

[dosper7]

I'm having the same issue, local gitlab on prem and trying to have either the pdb or the sourcelink working and none of them are working. Is there any developments on this issue?
@davidfowl @DamianEdwards do you know if someone from VS team could try to help on this? 🥺

[Genbox]

Is there any developments on this issue?

I managed to create a solution. No proxies, no hacks. I have been using it at work (600+ projects) for some time now.

[DamianEdwards]

@tmat thoughts on this one?

[dosper7]

Is there any developments on this issue?

I managed to create a solution. No proxies, no hacks. I have been using it at work (600+ projects) for some time now.

In the readme you mentioned you've tested with PAT token. How? I mean, I have the nuget published with pdb symbols but now the client (as in, Visual studio) needs to authenticate the request in order to get the the files for debugging. That's the step that needs the PAT (I think). If it is, where and how to I set the PAT to be used by the client?

Am I'm missing something?

[Genbox]

@dosper7 please create an issue in the GitLabSourceLink repo so we don't make too much noise here :)

[dosper7]

Need some help understand what is going on, I've done the "playbook" and visual studio is still saying that it has decompiled instead of using the sourcelink (maybe I'm missing something).

Here is what I've tested:

My lib csproj

<PropertyGroup>
	<TargetFramework>net8.0</TargetFramework>
	<OutputType>library</OutputType>
	<ImplicitUsings>enable</ImplicitUsings>
	<Nullable>enable</Nullable>
		
	<DebugType>portable</DebugType>
	<PublishRepositoryUrl>true</PublishRepositoryUrl>
	<IncludeSymbols>true</IncludeSymbols>
	<DebugSymbols>true</DebugSymbols>
	<IsPackable>true</IsPackable>
	<DeterministicSourcePaths>true</DeterministicSourcePaths>
	<SymbolPackageFormat>snupkg</SymbolPackageFormat>
</PropertyGroup>

I'm using dotnet cli ->

dotnet pack src -c Release
dotnet nuget push

When I debug it and step into I have this:

I also notice that in the nuget folder I have the downloaded lib, when I check for nuget info validation i have this:

However, If I manually download the snupkg manually from the registry and put it in the same folder as the nupkg, the validation it's ok:

But I'm still unable to use the sourcelink, or at least it seems it's not working because I see in the modules window the info that it "decompiled" the lib.

So it seems it can be two issues:

• the debugger is not able to download the snupkg from (not an issue for this repo)
• even when I have the snupkg, source link seems to work (for private instances)

Does this information helps?
Do you need more info? Is there any location where source-link/debugger are dumping some logs so I can see what's going on?

This has been a bit frustrating, so any help or info I can provide to move forward please ping me 🙏

[KalleOlaviNiemitalo]

the debugger is not able to download the snupkg

AFAIK, the Visual Studio debugger does not support snupkg, but if you upload a snupkg file to NuGet.org, then there is a symbol server from which the debugger can request the PDB files. So if you are not using NuGet.org, then check whether your NuGet hosting service has a similar symbol-server feature, or perhaps embed the debug info into the DLL files so that it need not be downloaded separately.