Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable response caching and cookie storage #10

Open
geoffbeier opened this issue Sep 21, 2022 · 2 comments · May be fixed by #11
Open

Enable response caching and cookie storage #10

geoffbeier opened this issue Sep 21, 2022 · 2 comments · May be fixed by #11

Comments

@geoffbeier
Copy link

I just found this library, and it's great. It's saving me a ton of work. As I'm using it, there are two features I want:

  1. Response caching
  2. Cookie storage

Since pymfl uses requests to interact with the MFL API, a relatively small addition to the public interface is enough to make it so that applications can turn on caching (and even set different cache rules per-API) and store cookies between runs.

By adding a requests.Session parameter (defaulted to None) to the APIConfig.add_config_for_year_and_league_id and having pymfl hang onto that with the YearAPIConfig, then having the client base class use that session instead of the raw cookie value, callers who want caching and saving can get it, with no interface changes for callers who don't care about that.

Assuming a caller wants to use requests-cache for caching and a cookiejar from the standard library http.cookies, caching and saving cookies would then look something like this:

import os
import logging

from pymfl.api.config import APIConfig
from pymfl.api import CommonLeagueInfoAPIClient
import requests_cache
from http.cookiejar import LWPCookieJar

logger = logging.getLogger(__name__)

# This particular sample assumes the configuration data is stored in the environment,
# and the thing that runs this script calls `load_dotenv` from python-dotenv to make it so.

def run():
    try:
        mfl_user = os.environ["MFL_USER"]
        mfl_pass = os.environ["MFL_PASS"]
        mfl_league = os.environ["MFL_LEAGUE"]
        mfl_ua = os.environ["MFL_USER_AGENT"]
        mfl_cache = os.environ["MFL_CACHE"]
        mfl_cookies = os.environ["MFL_COOKIES"]
    except KeyError as e:
        logger.error(f"MFL parameters need to be configured in the environment:", e)
        return

    # cache things for 24 hours, for now, to prevent abusive requests while iterating in dev
    expiration_seconds = 60 * 60 * 24
    cookiejar = LWPCookieJar(mfl_cookies)
    if os.path.exists(mfl_cookies):
        cookiejar.load(ignore_discard=True)
    session = requests_cache.CachedSession(cache_name="mfl-cache", expire_after=expiration_seconds)
    session.cookies = cookiejar

    APIConfig.add_config_for_year_and_league_id(year=2022, league_id=mfl_league, username=mfl_user, password=mfl_pass,
                                                user_agent_name=mfl_ua, session=session)
    league_details = CommonLeagueInfoAPIClient.get_league(year=2022, league_id=mfl_league)
    cookiejar.save(ignore_discard=True)
    print(f"retrieved league details:\n{league_details}")
@geoffbeier
Copy link
Author

I have most of a PR done that makes this change, and plan to send it soon. I thought it'd be worth opening this issue to discuss it while I'm cleaning it up.

@geoffbeier geoffbeier linked a pull request Sep 21, 2022 that will close this issue
@joeyagreco
Copy link
Owner

Thanks for looking into this!

I'll leave comments on the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants