Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ✨ client connection options #534

Merged
merged 6 commits into from
Nov 14, 2024
Merged

Conversation

iwpnd
Copy link
Owner

@iwpnd iwpnd commented Nov 13, 2024

@alxwrd

using options pattern redis client connection options. options pattern doesn't seem to be too widely used, yet it allows me to abstract away any of the redis retry and backoff implementations. the final version might look a little different, but I think you can test run this one to see if that suffices to close #529.

closes #529

UPDATE:

run

docker-compose up -d

start this in one terminal

import asyncio

from pyle38 import Tile38
from pyle38.client_options import WithRetryExponentialBackoff, WithRetryOnError
from pyle38.errors import (
    Pyle38ConnectionError,
    Pyle38TimeoutError,
)

tile38 = Tile38(
    url="redis://localhost:9851",
    options=[
        WithRetryExponentialBackoff(30),
        WithRetryOnError(Pyle38TimeoutError, Pyle38ConnectionError),
    ]
)


async def main():
    while True:
        try:
            res = await tile38.set("fleet", "truck").point(1, 1).exec()
            print(res.dict())
        except Exception as e:
            print(e)

        await asyncio.sleep(1)


asyncio.run(main())

in another terminal restart the tile38 leader

docker-compose restart tile38-leader

as long as the leader comes back on within those 25 retries, the ConnectionError will not be raised.

using options pattern redis client connection options

closes #529
Copy link
Contributor

@alxwrd alxwrd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes works great ✨!

I've added a suggestion for the Tile38 interface, but either way works for me.

pyle38/tile38.py Outdated
def __init__(
self,
url=os.getenv("TILE38_LEADER_URI"),
follower_url=os.getenv("TILE38_FOLLOWER_URI"),
*opts: Callable[..., ClientOptions],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Having the options as *args makes the client interface a bit clunky as you can no longer use the key word args for url and follower_url and pass a list of options.

# Doesn't work
tile38 = Tile38(
    url="redis://localhost:9851",
    WithRetryExponentialBackoff(30),
    WithRetryOnError(Pyle38TimeoutError, Pyle38ConnectionError),
)

You can of course expand them inline,

tile38 = Tile38(
    url="redis://localhost:9851",
    *[
        WithRetryExponentialBackoff(30),
        WithRetryOnError(Pyle38TimeoutError, Pyle38ConnectionError),
    ]
)

but I think having another key word argument feels nicer?

Suggested change
*opts: Callable[..., ClientOptions],
options: List[Callable[..., ClientOptions]] = None,

And then of course:

    def __init__(
        self,
        url=os.getenv("TILE38_LEADER_URI"),
        follower_url=os.getenv("TILE38_FOLLOWER_URI"),
        options: List[Callable[..., ClientOptions]] = None,
    ):
        if not url:
            raise Tile38Error("No Tile38 url set")

        if options is None:
            options = []

Copy link
Owner Author

@iwpnd iwpnd Nov 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah my thoughts exactly. The way it is now is a little too golang’ish. Although I don’t like array as inputs either. .. unless I’m changing that follower_url as well. 😈

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im going for a less disruptive option. ill take on the follower refactor when I add the option for multiple follower instances.

@iwpnd iwpnd merged commit 651f7b3 into main Nov 14, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add ability to set retry on Redis connection
2 participants