Using private repository on self Hosted gitlab instance

[Ducatel]

Hi,

I'm currently migrating my process from gitlink because now I need to use portable PDB.

So I have my own gitlab instance and all of my repos need authorization for reading it.

In my csproj in tried to add:

<PackageReference Include="Microsoft.SourceLink.GitLab" Version="1.0.0-beta-63102-01" PrivateAssets="All" />
<SourceLinkGitLabHosts Include="gitlab.my-domain.com" />

But there is a way to pass a gitlab read access token ?

Thanks in advance for your help

[tmat]

@Ducatel You don't need to specify access token in the project. Authentication is performed by the debugger when it fetches the source file from the server.

That said, GitLab currently doesn't support the protocol the VS debugger uses to authenticate to git repositories.

See issues
https://gitlab.com/gitlab-org/gitlab-ce/issues/36654
microsoft/Git-Credential-Manager-for-Windows#480

BTW, you shouldn't need to specify SourceLinkGitLabHosts, unless you use multiple SourceLink packages in the project.

[Ducatel]

OK so, if I understand well, there is no solution yet ? There is a workarround ?

And issues doesn't speak about 2FA (2FA is mandatory on my gitlab instance ).
There is issue for that ?

Ps: I deleted SourceLinkGitLabHosts and the pdb still have link to my gitlab instance. So it's nice ;) (The remote gitlab server is automatically retrieved from .git folder ? )

[tmat]

@chuckries To comment on any workarounds and 2FA.

Ps: I deleted SourceLinkGitLabHosts and the pdb still have link to my gitlab instance. So it's nice ;) (The remote gitlab server is automatically retrieved from .git folder ? )

Yes, that is correct.

[chuckries]

@Ducatel Currently GitLab does not support the Source Link authentication mechanisms that are used by the Visual Studio Debugger. If this issue that @tmat mentioned is ever resolved, it will handle 2FA and the debugger will be able to use it.

The only workaround I know of for GitLab is running your own on premises Source Link proxy service. You would embed proxy URL's in the Source Link rather than the raw GitLab URLs. The proxy would need to be able to authenticate to GitLab using GitLab's OAuth mechanisms (which the debugger does not support). The proxy would then forward the Source Link request to GitLab with the appropriate authentication and return the result back to Visual Studio. This workaround is not ideal but today it is the only way to make Source Link work for private GitLab repos.

[Ducatel]

Hum, sound no good for me :D
Deploy a secure proxy for that is not easy in my network infrastructure...

Can I check in the source if it's possible to path the read token for the Gitlab link with a small trick (adding a configuration like SourceLinkGitLabHosts --> SourceLinkGitLabReadToken) ?

[tmat]

You mean include the access token to the URL stored in the SourceLink?

[Ducatel]

Yes in URL param like explained in gitlab doc

This solution seem's to not require lot of dev ;)

[tmat]

That seems like a bad idea. You'd be leaking your token to whomever has access to the PDB.

[chuckries]

In addition to the security hole, PAT's expire. The Source Link with embedded PAT would only work for so long before it would fail.

[Ducatel]

In my case there are no security hole.
My nugets feed is private and only accessible by dev computer.
So if you get the PDB you have access to dev computer and probably to source ...

In short, this is just a workaround not perfect (otherwise it not a workaround but a solution :D ) which can allow people to continue working and debugging.

[tmat]

If you want to do this for your own project you can write a target that runs before target GenerateSourceLinkFile and depends on targets $(SourceLinkUrlInitializerTargets). This target can update RepositoryUrl metadata of SourceRoot items and add the token.

[Ducatel]

ok, I don't really know how do what you say. 😄
Can you give me an example, or some docs I can read about that ?

[chuckries]

While I don't condone this, the following snippet should do the trick. Add it to the .csproj or a Directory.Build.targets.

  <Target Name="AppendSourceLinkPersonalAccessToken"
          BeforeTargets="GenerateSourceLinkFile"
          DependsOnTargets="$(SourceLinkUrlInitializerTargets)">
    <PropertyGroup>
        <GitLabPAT Condition="'$(GitLabPAT)'==''">deadbeefdeadbeef</GitLabPAT>
    </PropertyGroup>

    <ItemGroup>
      <SourceRoot>
        <SourceLinkUrl>%(SourceLinkUrl)?private_token=$(GitLabPAT)</SourceLinkUrl>
      </SourceRoot>
    </ItemGroup>
  </Target>

[Ducatel]

Yes with your code the pdb contain the private token.
Thanks for your help ;)

[Symbianx]

This issue should be open?
Obviously the workaround provided will not work in the long term and has a major security flaw.

@tmat What would be needed to support the GitLab authentication flow?

[tmat]

@Symbianx
microsoft/Git-Credential-Manager-for-Windows#480

[Spacefish]

Super unsafe solution: Reverse proxy in front of gitlab which just adds a private gitlab admin token to the URL if the user agent is "SourceLink" and the url contains "/raw/" shame on me

Probably best solution: Implement HTTP Basic auth in GitLab (see https://gitlab.com/gitlab-org/gitlab-ce/issues/36654 ) or do the same in a reverse proxy by acquiring a session cookie first and then accessing the /raw/ content with the session cookie set.

[mhofmann85]

Hi,

I recently had the a similar usecase with a C++-Project. I'm currently migrating from subversion to GitLab and I want to Debug Dumps from an execution if they crash in the field. Getting the correct pdb-files is done manually, this can be done with a symbol-server later. Never the less, if I'm debugging an exception, I want to have the correct source file from GitLab.

I Used the CMake scripts to generate a source-link.json file. I Simply rewrote the url to point to the /raw/ URL of Gitlab.

Everything works so far, except the authentification. Sourcelink would allways download the login-page as my final .cpp file. My solution so far is, to generate an session cookie for Gitlab in Visual Studio 2019. This can be done by logging in to gitlab via the internal web browser: View -> Other Windows -> Web browser.

This is maybe not the perfect solution, but in my usecase, it's still simpler than implementing some reverse proxy via localhost or anything desribed above. And, if Visual studio implements the missing feature, the "old" releases will still work, but without the manual login.

A Final small remark. If the credentials where wrong it is possible that the wrong file is cached, therefore check the C:\Users\youruser\AppData\Local\SourceServer for wrong files.

[mikhail-barg]

My solution so far is, to generate an session cookie for Gitlab in Visual Studio 2019. This can be done by logging in to gitlab via the internal web browser: View -> Other Windows -> Web browser.

@mhofmann85 thank you for the idea! works for me

[odalet]

Thanks alot @mhofmann85 you saved my day! It's hacky but less intrusive than other workarounds here!

[odalet]

@chuckries I think your snippet does not work any more: according to this issue it was a mistake in the first place to allow access to raw files this way. Your idea could maybe still work but would involve a rather convoluted reconstruction of the URL in order to translate it into a GitLab API format:

http(s)://mygitlab/proj1/proj2/proj3/raw/commit/dir1/dir2/file?private_token=msBuildProperty

should become

http(s)://mygitlab/api/v4/projects/proj1%2Fproj2%2Fproj3/repository/files/dir1%2Fdir2%2Ffile/raw?ref=commit&private_token=msBuildProperty

I must admit my MSBuild skills are not that high that I'd try to do this...

[juangburgos]

Even that would not work, because SRCSRV only accepts urls without query strings, si the urls cannot contain the ? query.

[juangburgos]

You can use this small app to rewrite Gitlab URLs to make sourcelink work:
https://github.com/juangburgos/GitlabRewritter

[odalet]

@mhofmann85's trick of using View -> Other Windows -> Web browser in VS in order to authenticate against a private GitLab instance does not work anymore in VS2022 as... the Web browser window was removed! So back to finding another workaround... 😡

[sthames42]

It doesn't work in 2016, either. Other than hosting your own local GitLab instance you can allow to be public within another security layer, the only way to do it, right now, is as @juangburgos suggested. I wrote GitlabProxy to handle this under IIS and it works flawlessly for me. Maybe it can help someone else.

[odalet]

@sthames42 2016?

I've managed to have the browser trick work again in VS2022 by basically recreating the Browser window in an extension...

[mikhail-barg]

@odalet Could you please share/publish the extension? It would be of a great help!

[mhofmann85]

@mikhail-barg I had no time to test it, but if i search in the Extension Manager, i find a WebBrowser Extension for VS2022. Maybe you can try that one.

[odalet]

@mikhail-barg , @mhofmann85 The extension you found is indeed the one I quickly crafted this WE; but I was too quick in replying it solves the issue:

IT DOES NOT SOLVE THE ISSUE 😭

However, I managed to find another workaround: Unless you are using Windows 11, you should still have the old Internet Explorer browser installed on your system.

Execute it (Windows+R then iexplore.exe) and log into your GitLab instance from it (do not mind the rendering: although crappy, the page works). Do not forget to check the "Remember me" box.

Once you are authenticated against GitLab in IE, downloading source files via SourceLink in VS2022 should work again...

Concerning my extension, 2 things worth noting:

• It's useless as a workaround for GitLab SourceLink as it is based on the WebView2 (aka Edge) control and hence shares nothing (cookies, session...) with the old IE browser
• For other uses, it still needs work as I'm not doing any cleanup of the browser control and bunches of Edge processes are created, never destroyed and may well hog the memory...

And then for Windows 11 users, all that's left is to recreate a Wpf or Winforms -based Browser by using the old WebBrowser control that is IE-based. For now the simplest way to obtain such a thing is to compile/adapt this: https://github.com/sphinxlogic/All-In-One-Code-Framework/tree/master/Visual%20Studio%202010/CSWebBrowserSuppressError

Hope this helps!

[mikhail-barg]

@tmat Thanks for the detailed info on the workaround! I will try it as soon as I face the issue again.