Statistics tab

[8ctopus]

Hallo Christian,

I've just started playing around with your package and like it. Thank you!
I've managed to get basic quote information such as trailing PE, however I fail to access the information that is found on the statistics page such as payout ratio. Looking at past issues, I have found #35 which is marked as completed, however I fail to fi
/nd any of it in the source code.

Also the query link shared, fails with

{
    "finance": {
        "result": null,
        "error": {
            "code": "Unauthorized",
            "description": "Invalid Crumb"
        }
    }
}

Update: I actually figured it out, adding &crumb=mdeVssfeRhi to the request url works. I got the crumb value from a browser session on yahoo finance.

[8ctopus]

To get the crumb, you need to provide two things: cookies and user agent:

curl 'https://query2.finance.yahoo.com/v1/test/getcrumb' \
  -H 'cookie: GUC=AQEBCAFlULBleUIhXwTg&s=AQAAAC76j7jx&g=ZU9e-w; A1=d=AQABBK0Lr2ICEI-bBud4SuIDLaB4bqaMNbAFEgEBCAGwUGV5Zc3ibmUB_eMBAAcIrQuvYqaMNbA&S=AQAAAggpViKy189d2OkdWsFnK_Y; A3=d=AQABBK0Lr2ICEI-bBud4SuIDLaB4bqaMNbAFEgEBCAGwUGV5Zc3ibmUB_eMBAAcIrQuvYqaMNbA&S=AQAAAggpViKy189d2OkdWsFnK_Y; A1S=d=AQABBK0Lr2ICEI-bBud4SuIDLaB4bqaMNbAFEgEBCAGwUGV5Zc3ibmUB_eMBAAcIrQuvYqaMNbA&S=AQAAAggpViKy189d2OkdWsFnK_Y; gpp=DBAA; gpp_sid=-1; gam_id=y-rpk1JRJE2uL6EDgVwC0_in.GgdLv339c~A; axids=gam=y-rpk1JRJE2uL6EDgVwC0_in.GgdLv339c~A&dv360=eS00Mm1IMHpoRTJ1RXhhNVlTUGtSVWRPdkZCWExrbjFkNH5B; tbla_id=ba643052-93dc-4ae8-a277-ee1cb7f7dab5-tuctc48e47a; cmp=t=1699788664&j=0&u=1---; PRF=t%3DZURN.SW%252BSREN.SW%26newChartbetateaser%3D0%252C1700910092680' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0'

From the cookies, only A1 is necessary. So this is the smallest request I could find:

curl 'https://query2.finance.yahoo.com/v1/test/getcrumb' \
  -H 'cookie: A1=d=AQABBK0Lr2ICEI-bBud4SuIDLaB4bqaMNbAFEgEBCAGwUGV5Zc3ibmUB_eMBAAcIrQuvYqaMNbA&S=AQAAAggpViKy189d2OkdWsFnK_Y' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0'

And to get A1 cookie:

curl -v -s 'https://finance.yahoo.com/quote/ZURN.SW' \
  -H 'accept-language: en-US,en;q=0.9,fr;q=0.8' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0' \
   1> /dev/null

Here's the response to the last request:

< HTTP/1.1 200 OK
< referrer-policy: no-referrer-when-downgrade
< strict-transport-security: max-age=31536000
< x-frame-options: SAMEORIGIN
< content-security-policy: frame-ancestors 'self' https://*.yahoo.com https://*.engadget.com https://*.pnr.ouryahoo.com https://pnr.ouryahoo.com https://*.search.aol.com https://*.onesearch.com https://*.publishing.oath.com https://*.aol.com; sandbox allow-downloads allow-forms allow-modals allow-same-origin allow-scripts allow-popups allow-popups-to-escape-sandbox allow-top-navigation-by-user-activation allow-presentation;
< cache-control: private, no-store, no-cache, max-age=0
< content-type: text/html; charset=utf-8
< vary: Accept-Encoding
< date: Mon, 13 Nov 2023 05:04:59 GMT
< x-envoy-upstream-service-time: 479
< server: ATS
< x-envoy-decorator-operation: finance-nodejs--mtls-production-ir2.finance-k8s.svc.yahoo.local:4080/*
< Age: 0
< Transfer-Encoding: chunked
< Connection: keep-alive
< Expect-CT: max-age=31536000, report-uri="http://csp.yahoo.com/beacon/csp?src=yahoocom-expect-ct-report-only"
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< Set-Cookie: A1=d=AQABBHuuUWUCECcV2H5ND6rsZYO4zo0HI2gFEgEBAQH_UmVbZc1o0CMA_eMAAA&S=AQAAAumF6MTURk-DLOB53xHM7Bk; Expires=Tue, 12 Nov 2024 11:04:59 GMT; Max-Age=31557600; Domain=.yahoo.com; Path=/; SameSite=Lax; Secure; HttpOnly
< Set-Cookie: A3=d=AQABBHuuUWUCECcV2H5ND6rsZYO4zo0HI2gFEgEBAQH_UmVbZc1o0CMA_eMAAA&S=AQAAAumF6MTURk-DLOB53xHM7Bk; Expires=Tue, 12 Nov 2024 11:04:59 GMT; Max-Age=31557600; Domain=.yahoo.com; Path=/; SameSite=None; Secure; HttpOnly
< Set-Cookie: A1S=d=AQABBHuuUWUCECcV2H5ND6rsZYO4zo0HI2gFEgEBAQH_UmVbZc1o0CMA_eMAAA&S=AQAAAumF6MTURk-DLOB53xHM7Bk; Domain=.yahoo.com; Path=/; SameSite=Lax; Secure

[scheb]

The feature was discussed in #35, but it never got implemented. If you'd like to see that feature, I'm open to PRs.

For getting a crumb value, there should already be a function in the library for that purpose.

[RobertSchiefele]

I tried to code the request that gets the A1-Cookie in in c#. My response was correct but it didn't contain a A1-Cookie

[8ctopus]

@RobertSchiefele Just tested my code today to see if anything changed and I found out that it is still working. So there is some issue on your side.

[Vladutu]

@8ctopus I've tried the curl command for the cookie and it's not present. Here is what I get. Can you try again on your side to see if you still have it or can see a problem on my side? Thank you

curl -v -s 'https://finance.yahoo.com/quote/ZURN.SW' \                                                                                                   ✔  3s 
  -H 'accept-language: en-US,en;q=0.9,fr;q=0.8' \
  -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0' \
   1> /dev/null
* Connected to finance.yahoo.com (2a00:1288:7c:800::4001) port 443 (#0)
* ALPN: offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
} [322 bytes data]
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* (304) (IN), TLS handshake, Unknown (8):
{ [19 bytes data]
* (304) (IN), TLS handshake, Certificate (11):
{ [4371 bytes data]
* (304) (IN), TLS handshake, CERT verify (15):
{ [79 bytes data]
* (304) (IN), TLS handshake, Finished (20):
{ [36 bytes data]
* (304) (OUT), TLS handshake, Finished (20):
} [36 bytes data]
* SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: C=US; ST=California; L=Sunnyvale; O=Oath Holdings Inc.; CN=*.fantasysports.yahoo.com
*  start date: Dec 12 00:00:00 2023 GMT
*  expire date: Jan 31 23:59:59 2024 GMT
*  subjectAltName: host "finance.yahoo.com" matched cert's "*.yahoo.com"
*  issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 High Assurance Server CA
*  SSL certificate verify ok.
* using HTTP/2
* h2 [:method: GET]
* h2 [:scheme: https]
* h2 [:authority: finance.yahoo.com]
* h2 [:path: /quote/ZURN.SW]
* h2 [accept: */*]
* h2 [accept-language: en-US,en;q=0.9,fr;q=0.8]
* h2 [user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0]
* Using Stream ID: 1 (easy handle 0x147809800)
> GET /quote/ZURN.SW HTTP/2
> Host: finance.yahoo.com
> Accept: */*
> accept-language: en-US,en;q=0.9,fr;q=0.8
> user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36 Edg/119.0.0.0
>
< HTTP/2 200
< referrer-policy: no-referrer-when-downgrade
< strict-transport-security: max-age=31536000
< x-frame-options: SAMEORIGIN
< content-security-policy: frame-ancestors 'self' https://*.yahoo.com https://*.engadget.com https://*.pnr.ouryahoo.com https://pnr.ouryahoo.com https://*.search.aol.com https://*.onesearch.com https://*.publishing.oath.com https://*.aol.com; sandbox allow-downloads allow-forms allow-modals allow-same-origin allow-scripts allow-popups allow-popups-to-escape-sandbox allow-top-navigation-by-user-activation allow-presentation;
< cache-control: private, no-store, no-cache, max-age=0
< content-type: text/html; charset=utf-8
< vary: Accept-Encoding
< date: Thu, 21 Dec 2023 10:50:02 GMT
< x-envoy-upstream-service-time: 662
< server: ATS
< x-envoy-decorator-operation: finance-nodejs--mtls-production-ir2.finance-k8s.svc.yahoo.local:4080/*
< age: 5
< expect-ct: max-age=31536000, report-uri="http://csp.yahoo.com/beacon/csp?src=yahoocom-expect-ct-report-only"
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
{ [8372 bytes data]
* Connection #0 to host finance.yahoo.com left intact

[RobertSchiefele]

@Vladutu I tried the YahooManager.cs class from @8ctopus and it worked fine on my side. Thanks to the author.

[Vladutu]

@RobertSchiefele what YahooManager.cs are you refering to? Maybe I'm not looking where I should but the yahoo-finance-api is PHP project, not c#. I saw that he was doing a call to https://fc.yahoo.com to get the cookie (which I was also doing) but that doesn't work anymore, the website is not responding (I got 502 connection timeout)

[RobertSchiefele]

@Vladutu sorry I got this Classs from another site, cannot remember where. But anyway here is the code

using Microsoft.Extensions.Caching.Memory;
using System.Net;

namespace My.YahooFinanceAPI
{
public class YahooManager
{
private static readonly object yahooCredentialsLocker = new object();

    private const string STOCK_MARKET_URL_SUMMARY = "https://query1.finance.yahoo.com/v10/finance/quoteSummary/{0}?formatted=true&lang=en-US&region=US&modules=assetProfile%2CbalanceSheetHistory%2CbalanceSheetHistoryQuarterly%2CcalendarEvents%2CcashflowStatementHistory%2CcashflowStatementHistoryQuarterly%2CdefaultKeyStatistics%2Cearnings%2CearningsHistory%2CearningsTrend%2CesgScores%2CfinancialData%2CfundOwnership%2CincomeStatementHistory%2CincomeStatementHistoryQuarterly%2CindexTrend%2CindustryTrend%2CinsiderHolders%2CinsiderTransactions%2CinstitutionOwnership%2CmajorDirectHolders%2CmajorHoldersBreakdown%2CnetSharePurchaseActivity%2Cprice%2CrecommendationTrend%2CsecFilings%2CsectorTrend%2CsummaryDetail%2CsummaryProfile%2CupgradeDowngradeHistory%2Cpageviews%2Cquotetype&ssl=true";
    
    private const string YahooFcUrl = "https://fc.yahoo.com";
    private const string YahooGetCrumbUrl = "https://query2.finance.yahoo.com/v1/test/getcrumb";
    private const string CacheKeyCookieContainer = "YahooCookieContainer";
    private const string CacheKeyCrumb = "YahooCrumb";

    private readonly IHttpClientFactory _httpClientFactory;
    private readonly IMemoryCache _memCache;

    public YahooManager(IHttpClientFactory httpClientFactory, IMemoryCache memCache)
    {
        _httpClientFactory = httpClientFactory;
        _memCache = memCache;
    }


    public async Task<string> GetYahooSymbolSummary(string symbol)
    {
        return await WebRequestGet(string.Format(STOCK_MARKET_URL_SUMMARY, symbol));
    }

    public async Task<string> WebRequestGet(string url)
    {
        (CookieContainer cookie, string crumb) credentials = new(null, null);
        lock (yahooCredentialsLocker)
        {
            int tryCounter = 0;
            while ((credentials.crumb == null || credentials.cookie == null) && tryCounter < 10)
            {
                Task.Delay(1000 * tryCounter).Wait();
                credentials = GetYahooCookie().ConfigureAwait(false).GetAwaiter().GetResult();
                tryCounter++;
            }//while
        }//lock

        string sAgent = $"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36";
        var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, url + "&crumb=" + credentials.crumb)
        {
            Headers =
                    {
                        { "Cookie",  credentials.cookie.GetCookieHeader(new Uri(YahooFcUrl))},
                        { "User-Agent",sAgent }
                    }
        };
        var httpClient = _httpClientFactory.CreateClient();
        var webResponse = await httpClient.SendAsync(httpRequestMessage);
        //webResponse.EnsureSuccessStatusCode();
        string responseContent = await webResponse.Content.ReadAsStringAsync();
        return responseContent;
    }//WebRequestGet

    private async Task<(CookieContainer cookie, string crumb)> GetYahooCookie()
    {
        CookieContainer m_cookieContainer = null;
        string m_crumb = null;
        _memCache.TryGetValue(CacheKeyCookieContainer, out m_cookieContainer);
        _memCache.TryGetValue(CacheKeyCrumb, out m_crumb);

        if (m_cookieContainer == null || m_crumb == null)
        {
            string sAgent = $"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36";

            m_cookieContainer = new CookieContainer();
            var uri = new Uri(YahooFcUrl);
            var httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, YahooFcUrl)
            {
                Headers =
                    {
                        { "User-Agent", sAgent }
                    }
            };
            var httpClient = _httpClientFactory.CreateClient();
            var httpResponseMessage = await httpClient.SendAsync(httpRequestMessage);
            IEnumerable<string> cookies;
            httpResponseMessage.Headers.TryGetValues("Set-Cookie", out cookies);
            if (cookies == null)
                return (null, null);

            foreach (var cookieValue in cookies)
                m_cookieContainer.SetCookies(uri, cookieValue);
            httpResponseMessage.Dispose();

            httpRequestMessage = new HttpRequestMessage(HttpMethod.Get, YahooGetCrumbUrl)
            {
                Headers =
                    {
                        { "Cookie",  m_cookieContainer.GetCookieHeader(uri) },
                        { "User-Agent",sAgent}
                    }
            };
            httpClient = _httpClientFactory.CreateClient();
            var crumbResponse = await httpClient.SendAsync(httpRequestMessage);
            crumbResponse.EnsureSuccessStatusCode();
            string responseContent = await crumbResponse.Content.ReadAsStringAsync();
            m_crumb = responseContent.Trim('"');

            var cacheEntryOptions = new MemoryCacheEntryOptions()
                .SetSlidingExpiration(TimeSpan.FromSeconds(60*5))
                .SetAbsoluteExpiration(TimeSpan.FromSeconds(60*10))
                .SetPriority(CacheItemPriority.Normal);
            _memCache.Set(CacheKeyCookieContainer, m_cookieContainer, cacheEntryOptions);
            _memCache.Set(CacheKeyCrumb, m_crumb, cacheEntryOptions);
        }//endif
        return (m_cookieContainer, m_crumb);
    }//GetYahooCookie


}

}

[Vladutu]

@RobertSchiefele thank you. I will have a look.

[Vladutu]

@RobertSchiefele Seems it's doing the same using "https://fc.yahoo.com" which for some reason doesn't work anymore for me. I even asked a friend to try it out (thought maybe my IP was blocked) but it was the same thing