[FEATURE REQUEST] Allow to add our own GitHub token for requests

[EDM115]

📝 Provide a description of the new feature

We could have a new setting called github_token which would take a Github PAT. Even if this token have no rights, it would allow to authenticate on GitHub's API on the behalf of an user.
As explained here :

The primary rate limit for unauthenticated requests is 60 requests per hour.

So a new user that just browses the Marketplace, installs apps and applys will hit the limit very easily :

   ❯❯ C:\Windows\System32
    ❯ spicetify apply
spicetify v2.37.7
error Cannot fetch latest release info
error GitHub response: API rate limit exceeded for re.da.ct.ed. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)
...

Being authenticated gives us much more freedom :

You can use a personal access token to make API requests.
...
All of these requests count towards your personal rate limit of 5,000 requests per hour.

➕ Additional Information

This would fix reliably #2998 and #1981 (getting 403 errors and "Error parsing Markdown" in-app)
If the setting isn't set or the token is invalid, it would do an unauthenticated call just like it currently does. Else, it would do all GitHub requests authenticated.
Docs would probably need an update for this, maybe in the FAQ.

More overall info here

[rxri]

No. We won't. This ratelimit is enough unless you reload Spotify every 5 seconds. spicetify update throwing ratelimit? Just wait 15 minutes. 403 error means that there's no README. Also that's not related to CLI really :)

[EDM115]

403 error means that there's no README

wrong (tried with 2 random extensions) :

2024-09-01_15-21-01.1.mp4

[FrzMtrsprt]

This can also happen if you share the same ip address with someone who abuses the github api, or if you're in a really big subnet, which is quite common for those who can't access spotify without a vpn or those on campus.

[rxri]

Still am not sure how it's related to the cli itself. It would only apply to the update command which you don't really use frequently so this rate limit thingy wont affect you

[EDM115]

Still am not sure how it's related to the cli itself.

indeed it isn't but where tf is the repo with spicetify's actual source code ?
Out of the publicly available repos, I guess only 3 would be related to what's actually installed by the cli :

• https://github.com/spicetify/classmaps : probably for css, no readme whatsoever
• https://github.com/spicetify/modules : allegedly for v3, so probably doesn't concern our problem
• https://github.com/spicetify/hooks : the only one left, with no more info on the readme, no docs and commit names doesn't help at all

I'd be glad to open the issue in the appropriate repo and even make a PR for this, but god damn it's nowhere to be found.

It would only apply to the update command which you don't really use frequently

Last time I checked, spicetify backup, spicetify apply and others all triggers an update (or at least a check of it, which counts towards the hourly unauthenticated gh requests) :

   ❯❯ C:\Windows\System32
    ❯ spicetify backup
spicetify v2.38.3
info Spicetify up-to-date
info There is available backup.
info Clear current backup:
warning After clearing backup, Spotify cannot be backed up again.
info Please restore first then backup, run "spicetify restore backup" or re-install Spotify then run "spicetify backup".
                                                                                                                           ❯❯ C:\Windows\System32
    ❯ spicetify apply
spicetify v2.38.3
info Spicetify up-to-date
Overwriting themed assets:
OK
success Custom CSS is updated
success Theme's JS is updated
Applying additional modifications:
OK
Transferring custom apps:
OK
success Spotify is spiced up!
success Enabled DevTools!

also I read your announcement on discord so I definitely understand that this small problem is absolutely not a priority at all

[rxri]

Normal user won't run backup, apply or update 60 times in the hour, because it's literally not needed to do so. What you're looking for is https://github.com/spicetify/marketplace. It's pinned on the organization so I don't know how you couldn't find it. You listed 3 of v3 repos so yeah idk but marketplace is what you're looking for

[EDM115]

my bad, I thought that the marketplace repo was only holding responsibility for listing some extensions and themes and wasn't what actually implements spicetify's core functionalities

[rxri]

I'm sorry? Marketplace doesn't hold any of cli core functionalities. If you want to mitigate 403 http and rate limit errors in marketplace do it in marketplace's repo.

[WybeBosch]

Hey there, sorry to barge into this convo, but it seemed related to my issue.

I must be overlooking things:
The error message says read the docs, but i cant find it in the docs?

spicetify v2.37.7
Fetch latest release info:
error Cannot fetch latest release info
error GitHub response: API rate limit exceeded for 185.90.186.154. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)

*note, ah the error is from github itself. Yeah iknow how to fix auth issues by adding a personal acces token, but the cli doesnt support that as the person pointed out above.

Perhaps we could at something about it in the spicetify docs?

----- old oudated comment -----------

Where can i find in the docs more info about how to fix this? 

https://spicetify.app/docs/advanced-usage/command-line-interface
Contains no info about update going wrong

https://spicetify.app/docs/faq/#after-spotifys-update-running-spicetify-apply-or-spicetify-update-breaks-spotify
Is just about update breaking and having to apply backup

https://spicetify.app/docs/development/spotify-cli-flags
Looks handy, doesnt help me solve being rate limited by github.

Those are all the docs i can find related to cli or update

======================================

Reading this current thread the solution is: Wait?
im not spamming the github api or anything : s

Is spicetify update all what it does git clone the new release somewhere?
is this something i could do by hand perhaps? (So i dont get against the rate limit?)

[EDM115]

indeed it's github telling you to read their docs
and after all, spicetify is probably not the only app on my system (especially as a dev) which makes unauthenticated requests to github so that 60/h cap can be easily surpassed

[WybeBosch]

Is there another way to update Spicetify except for running spicetify update via the cli?
There isnt a hidden ui button or something right? I see the "notification" that theres an update, but its just telling me to run the CLI command

[Kovinda]

Any luck with this guys ?

[WybeBosch]

While i agree that nobody would run spicetify backup apply 60 times an hour, it seems that the API that is being called is shared across not just spicetify but other applications that use unauthenticated requests.

So if you have some other program that also uses the Api, its in fact very possible to exceed those 60 times an hour.

And very much so if you work in an office enviroment, where everyone shares the same IP adress,
and what do ya know they are all developers and they all have random programs on their laptops calling the github free api for random crap, then you end up hitting the rate limit quite fast.

Its also how i figured out that even random commands that spicetify does include a fetch for version update...

 spicetify enable-devtools
spicetify v2.38.7
error Cannot fetch latest release info
error GitHub response: API rate limit exceeded for xxx.xx.xxx.xxx. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)
success Enabled DevTools!

Why does enabling devtools, trigger a API request?
Im aware that this issue got closed, but im hoping that perhaps this can be reconsidered.

---

Iknow it would be quite a bit of effort to include a new config option, and then link the value from that to probably here

cli/src/cmd/update.go

Line 67 in 01f5cc5

resp2, err := http.Get(assetURL)

but then with

req, err := http.NewRequest("GET", assetURL, nil)
if err != nil {
	utils.Fatal(err)
}
req.Header.Set("Authorization", "Bearer YOUR_TOKEN")

resp2, err := http.DefaultClient.Do(req)
if err != nil {
	utils.Fatal(err)
}

and here

cli/jsHelper/spicetifyWrapper.js

Line 2458 in 01f5cc5

const res = await fetch("https://api.github.com/repos/spicetify/cli/releases/latest");

const res = await fetch('https://api.github.com/repos/spicetify/cli/releases/latest', {
    headers: {
        'Authorization': [${personalAccessToken}],
    },
});

[EDM115]

@WybeBosch that's exactly the point I tried to make but they won't listen...
the only way i'm thinking of is to have a program that listen and intercepts all network requests and add headers with your PAT for every request to the GitHub API, but honestly who would do it ? 😂

about the requests on every command, in your spicetify/config-xpui.ini set this to disable it (you would need to run spicetify update manually but at least not every command would run it) :

[Setting]
check_spicetify_update = 0

[EDM115]

While i agree that nobody would run spicetify backup apply 60 times an hour

Nobody does, but browsing the Marketplace does when you click on an extension/theme/app to display its description.
And considering the number of extensions/themes/apps nowadays, a new user can hit this limit really fast if they check every extension to know what they actually do (see the video at #3154 (comment)).

[rxri]

that's exactly the point I tried to make but they won't listen...

I'm listening. I responded to your request and I disagreed with it. I, not others, me. Regardless if we would want to implement it or not, there's no place to put this token because config-xpui.ini SHOULD NOT hold stuff like Authorization tokens. Many of our users share their dotfiles, so their token would be shared with their config files :^) On the other hand, how would cli share this token with the wrapper? Patch it into the javascript files? Yeah, that doesn't sound good at all. Like I said, I'm not changing my mind and we're not going to consider it.
As always, feel free to fork the project and add it manually as you wish.

And considering the number of extensions/themes/apps nowadays, a new user can hit this limit really fast if they check every extension to know what they actually do

Checking extension's README does not count towards the API ratelimit. Marketplace uses raw.githubusercontent and not GitHub's File content API.

[EDM115]

config-xpui.ini SHOULD NOT hold stuff like Authorization tokens. Many of our users share their dotfiles

very true, i share these files too.

On the other hand, how would cli share this token with the wrapper? Patch it into the javascript files?

or use an env var called spicetify_gh_pat or whatever, i mean not everything needs to be in files (this poses other issues tbh, and using a credentials manager for this task would be highly overkill ngl)

Checking extension's README does not count towards the API ratelimit

thx for the info ! but i still wonder why it wasn't able to fetch the content and dropped 429's, i don't think i was hitting the 5k limit either... mystery will never be solved

[WybeBosch]

Sorry if this conversation got a bit heated, this was never my intention, im very happy that people put so much effort and passion in a free tool.

I understand why you do not want the headache of adding support for a PersonalAccesToken.
Feel free to shootdown this PR, but i thought maybe if i put in the work and tested a solution.

It could provide something to read back on for the future, or atleast show that im invested in this topic
i created a PR that adds support for a GITHUB_TOKEN
#3271

thats read from the env from wherever the spicetify update command is ran

---

@EDM115 besides creating the PR, i also tried out your solution of intercepting network requests.. just for fun.
However thats not really feasible since spicetify uses https requests to acces github, which cant be modified to include the auth token.

However.. because i was bored i made that work too aswell.
https://github.com/WybeBosch/spicetify-funny-http-intercept/

Completely useless of course, but it works and is quite Funny.

---

I thank you both for your time, never meant to offend and i shall leave this topic for what it is and unsubscribe.
Feel free to lock the topic to prevent from getting further spam :)

[rxri]

thx for the info ! but i still wonder why it wasn't able to fetch the content and dropped 429's, i don't think i was hitting the 5k limit either... mystery will never be solved

because you're hitting the 429 on the endpoint to manifest. I think it's some stupid thing that fetches manifest when you want to click the readme am sure.

or use an env var called spicetify_gh_pat or whatever, i mean not everything needs to be in files (this poses other issues tbh, and using a credentials manager for this task would be highly overkill ngl)

this does not answer my question on how you would want to pass this token to marketplace. we can't read your env from spotify client