Release 3.11.2 (#186)

3.11.2 (2024-11-29)
-------------------

**Fixed**
- SSE request block IO by default. Integrate better with urllib3-future
new SSE web extension.
- Passing a list instead of tuple for multipart file upload ends in
failure.

**Changed**
- urllib3-future lower bound version is raised to 2.12.900 to ensure
built-in support for SSE.
- help script now yield if websocket extra is present and actual
version.
- exception raised when no adapter were found now include a hint when
the intent was websocket and extra isn't installed.

.pre-commit-config.yaml

@@ -28,4 +28,4 @@ repos:
   -   id: mypy
       args: [--check-untyped-defs]
       exclude: 'tests/|noxfile.py'
-      additional_dependencies: ['charset_normalizer', 'urllib3.future>=2.11.900', 'wassima>=1.0.1', 'idna', 'kiss_headers']
+      additional_dependencies: ['charset_normalizer', 'urllib3.future>=2.12.900', 'wassima>=1.0.1', 'idna', 'kiss_headers']

HISTORY.md

@@ -1,6 +1,18 @@
 Release History
 ===============
 
+3.11.2 (2024-11-29)
+-------------------
+
+**Fixed**
+- SSE request block IO by default. Integrate better with urllib3-future new SSE web extension.
+- Passing a list instead of tuple for multipart file upload ends in failure.
+
+**Changed**
+- urllib3-future lower bound version is raised to 2.12.900 to ensure built-in support for SSE.
+- help script now yield if websocket extra is present and actual version.
+- exception raised when no adapter were found now include a hint when the intent was websocket and extra isn't installed.
+
 3.11.1 (2024-11-22)
 -------------------
 

README.md

@@ -7,7 +7,8 @@
 Niquests, is the “**Safest**, **Fastest[^10]**, **Easiest**, and **Most advanced**” Python HTTP Client. Production Ready!
 
 ✔️ **Try before you switch:** [See Multiplexed in Action](https://replit.com/@ahmedtahri4/Python#main.py)<br>
-📖 **See why you should switch:** [Read about 10 reasons why](https://medium.com/@ahmed.tahri/10-reasons-you-should-quit-your-http-client-98fd4c94bef3), and ["_Revived the promise made six years ago for Requests 3_"](https://medium.com/@ahmed.tahri/revived-the-promise-made-six-years-ago-for-requests-3-37b440e6a064)
+📖 **See why you should switch:** [Read about 10 reasons why](https://medium.com/@ahmed.tahri/10-reasons-you-should-quit-your-http-client-98fd4c94bef3), and ["_Revived the promise made six years ago for Requests 3_"](https://medium.com/@ahmed.tahri/revived-the-promise-made-six-years-ago-for-requests-3-37b440e6a064)<br>
+✨ **You were used to betamax, requests-mock, responses, ...?** [See how they still work! We got you covered.](https://niquests.readthedocs.io/en/latest/community/extensions.html)
 
 <details>
   <summary>👆 <b>Look at the feature table comparison</b> against <i>requests, httpx and aiohttp</i>!</summary>
@@ -47,6 +48,7 @@ Niquests, is the “**Safest**, **Fastest[^10]**, **Easiest**, and **Most advanc
 | `WebSocket over HTTP/2 and HTTP/3`         |     ✅[^13]     |     ❌     |       ❌       | ❌             |
 | `Automatic Ping for HTTP/2+`               |       ✅        |    N/A    |       ❌       | N/A           |
 | `Automatic Connection Upgrade / Downgrade` |       ✅        |    N/A    |       ❌       | N/A           |
+| `Server Side Event (SSE)`                  |       ✅        |     ❌     |       ❌       | ❌             |
 </details>
 
 <details>
@@ -75,14 +77,12 @@ Did you give up on HTTP/2 due to performance concerns? Think again! Do you reali
 Multiplexing and response lazyness open up a wide range of possibilities! Want to learn more about the tests? scripts? reasoning?
 
 Take a deeper look at https://github.com/Ousret/niquests-stats
-
-⚠️ Do the responsible thing with this library and do not attempt DoS remote servers using its abilities.
 </details>
 
 ```python
 >>> import niquests
 >>> s = niquests.Session(resolver="doh+google://", multiplexed=True)
->>> r = s.get('https://pie.dev/basic-auth/user/pass', auth=('user', 'pass'))
+>>> r = s.get('https://one.one.one.one')
 >>> r
 <ResponsePromise HTTP/3>
 >>> r.status_code
@@ -111,7 +111,7 @@ import asyncio
 
 async def main() -> None:
     async with niquests.AsyncSession(resolver="doh+google://") as s:
-        r = await s.get('https://pie.dev/basic-auth/user/pass', auth=('user', 'pass'), stream=True)
+        r = await s.get('https://one.one.one.one', stream=True)
         print(r)  # Output: <Response HTTP/3 [200]>
         payload = await r.json()
         print(payload)  # Output: {'authenticated': True, ...}
@@ -173,6 +173,7 @@ Niquests is ready for the demands of building scalable, robust and reliable HTTP
 - Trailers!
 - DNSSEC!
 - Async!
+- SSE!
 
 Need something more? Create an issue, we listen.
 

docs/user/quickstart.rst

@@ -1265,6 +1265,104 @@ See::
 
 .. note:: The given example are really basic ones. You may adjust at will the settings and algorithm to match your requisites.
 
+Server Side Event (SSE)
+-----------------------
+
+.. note:: Available since version 3.11.2+
+
+Server side event or widely known with its acronym SSE is a extremely popular method to stream continuously event
+from the server to the client in real time.
+
+Before this built-in feature, most way to leverage this were to induce a bit of hacks into your http client.
+
+Starting example
+~~~~~~~~~~~~~~~~
+
+Thanks to urllib3-future native SSE extension, we can effortlessly manage a stream of event.
+Here is a really basic example of how to proceed::
+
+    import niquests
+
+    if __name__ == "__main__":
+
+        r = niquests.post("sse://httpbingo.org/sse")
+
+        print(r)  # output: <Response HTTP/2 [200]>
+
+        while r.extension.closed is False:
+            print(r.extension.next_payload())  # ServerSentEvent(event='ping', data='{"id":0,"timestamp":1732857000473}')
+
+We purposely set the scheme to ``sse://`` to indicate our intent to consume a SSE endpoint.
+
+.. note:: ``sse://`` is using ``https://`` under the hood. To avoid using an encrypted connection, use ``psse://`` instead.
+
+You will notice that the program is similar to our ``WebSocket`` implementation. Excepted that the ``next_payload()``
+method returns by default a ``ServerSentEvent`` object.
+
+Extracting raw event
+~~~~~~~~~~~~~~~~~~~~
+
+In the case where your server weren't compliant to the defined web standard for SSE (e.g. add custom field/line style)
+you can extract a ``str`` instead of a ``ServerSentEvent`` object by passing ``raw=True`` into our ``next_payload()``
+method.
+
+As such::
+
+    while r.extension.closed is False:
+        print(r.extension.next_payload(raw=True))  # "event: ping\ndata: {"id":9,"timestamp":1732857471733}\n\n"
+
+.. warning:: As with WebSocket, ``next_payload`` method may return None if the server terminate the stream.
+
+Interrupt the stream
+~~~~~~~~~~~~~~~~~~~~
+
+A server may send event forever. And to avoid the awkward situation where your client receive unsolicited data
+you should at all time close the SSE extension to notify the remote peer about your intent to stop.
+
+For example, the following test server send events until you say to stop: ``sse://sse.dev/test``
+
+See how to stop cleanly the flow of events::
+
+    import niquests
+
+    if __name__ == "__main__":
+
+        r = niquests.post("sse://sse.dev/test")
+
+        events = []
+
+        while r.extension.closed is False:
+            event = r.extension.next_payload()
+
+            if event is None:  # the remote peer closed it himself
+                break
+
+            events.append(event)  # add the event to list
+
+            if len(events) >= 10:  # close ourselves SSE stream & notify remote peer.
+                r.extension.close()
+
+ServerSentEvent
+~~~~~~~~~~~~~~~
+
+.. note:: A ``ServerSentEvent`` object is returned by default with the ``next_payload()`` method. Or None if the server terminate the flow of events.
+
+It's a parsed SSE (single event). The object have nice shortcuts like:
+
+- ``payload.json()`` (any) to automatically unserialize passed json data.
+- ``payload.id`` (str)
+- ``payload.data`` (str) for the raw message payload
+- ``payload.event`` (str) for the event type (e.g. message, ping, etc...)
+- ``payload.retry`` (int)
+
+The full class source is located at https://github.com/jawah/urllib3.future/blob/3d7c5d9446880a8d473b9be4db0bcd419fb32dee/src/urllib3/contrib/webextensions/sse.py#L14
+
+Notes
+~~~~~
+
+SSE can be reached from HTTP/1, HTTP/2 or HTTP/3 at will. Niquests makes this very easy.
+Moreover every features like proxies, happy-eyeballs, hooks, etc.. can be used as you always did.
+
 -----------------------
 
 Ready for more? Check out the :ref:`advanced <advanced>` section.

pyproject.toml

@@ -8,7 +8,7 @@ description = "Niquests is a simple, yet elegant, HTTP library. It is a drop-in
 readme = "README.md"
 license-files = { paths = ["LICENSE"] }
 license = "Apache-2.0"
-keywords = ["requests", "http2", "http3", "QUIC", "http", "https", "http client", "http/1.1", "ocsp", "revocation", "tls", "multiplexed", "dns-over-quic", "doq", "dns-over-tls", "dot", "dns-over-https", "doh", "dnssec"]
+keywords = ["requests", "http2", "http3", "QUIC", "http", "https", "http client", "http/1.1", "ocsp", "revocation", "tls", "multiplexed", "dns-over-quic", "doq", "dns-over-tls", "dot", "dns-over-https", "doh", "dnssec", "websocket", "sse"]
 authors = [
   {name = "Kenneth Reitz", email = "me@kennethreitz.org"}
 ]
@@ -42,7 +42,7 @@ dynamic = ["version"]
 dependencies = [
     "charset_normalizer>=2,<4",
     "idna>=2.5,<4",
-    "urllib3.future>=2.11.900,<3",
+    "urllib3.future>=2.12.900,<3",
     "wassima>=1.0.1,<2",
     "kiss_headers>=2,<4",
 ]

requirements-dev.txt

@@ -5,5 +5,6 @@ pytest-httpbin>=2,<3
 pytest-asyncio>=0.21.1,<1.0
 httpbin==0.10.2
 trustme
-cryptography<40.0.0; python_version <= '3.7' and platform_python_implementation == 'PyPy'
+cryptography<40.0.0; python_version <= '3.8'
+cryptography<44; python_version > '3.8'
 werkzeug>=2.2.3,<=3.0.2

src/niquests/__version__.py

@@ -9,9 +9,9 @@
 __url__: str = "https://niquests.readthedocs.io"
 
 __version__: str
-__version__ = "3.11.1"
+__version__ = "3.11.2"
 
-__build__: int = 0x031101
+__build__: int = 0x031102
 __author__: str = "Kenneth Reitz"
 __author_email__: str = "me@kennethreitz.org"
 __license__: str = "Apache-2.0"

src/niquests/_async.py

@@ -324,8 +324,17 @@ def get_adapter(self, url: str) -> AsyncBaseAdapter:  # type: ignore[override]
         except ImportError:
             pass
 
+        # add a hint if wss:// fails when Niquests document its support.
+        # they probably forgot about the extra.
+        if url.startswith("ws://") or url.startswith("wss://"):
+            additional_hint = " Did you forget to install the extra for WebSocket? Run `pip install niquests[ws]` to fix this."
+        else:
+            additional_hint = ""
+
         # Nothing matches :-/
-        raise InvalidSchema(f"No connection adapters were found for {url!r}")
+        raise InvalidSchema(
+            f"No connection adapters were found for {url!r}{additional_hint}"
+        )
 
     async def send(  # type: ignore[override]
         self, request: PreparedRequest, **kwargs: typing.Any
@@ -579,10 +588,12 @@ async def _redirect_method_ref(x, y):
 
         if not stream:
             if isinstance(r, AsyncResponse):
-                await r.content
+                if r.extension is None:
+                    await r.content
                 _swap_context(r)
             else:
-                r.content
+                if r.extension is None:
+                    r.content
 
         return r
 

src/niquests/adapters.py

@@ -1201,7 +1201,8 @@ def on_post_connection(conn_info: ConnectionInfo) -> None:
                 extract_cookies_to_jar(session_cookies, sub_resp.request, sub_resp.raw)
 
         if not stream:
-            response.content
+            if response.extension is None:
+                response.content
 
         del self._promises[response_promise.uid]
 
@@ -2304,7 +2305,8 @@ async def on_post_connection(conn_info: ConnectionInfo) -> None:
                 extract_cookies_to_jar(session_cookies, sub_resp.request, sub_resp.raw)
 
         if not stream:
-            await response.content
+            if response.extension is None:
+                await response.content
             _swap_context(response)
 
         del self._promises[response_promise.uid]

src/niquests/help.py

@@ -4,7 +4,12 @@
 
 import json
 import platform
-import ssl
+
+try:
+    import ssl
+except ImportError:
+    ssl = None  # type: ignore
+
 import sys
 import warnings
 from json import JSONDecodeError
@@ -47,6 +52,11 @@
 except ImportError:
     ocsp_verify = None  # type: ignore
 
+try:
+    import wsproto  # type: ignore[import-not-found]
+except ImportError:
+    wsproto = None  # type: ignore
+
 
 def _implementation():
     """Return a dict with the Python implementation and version.
@@ -108,10 +118,15 @@ def info():
         "version": getattr(idna, "__version__", ""),
     }
 
-    system_ssl = ssl.OPENSSL_VERSION_NUMBER
-    system_ssl_info = {
-        "version": f"{system_ssl:x}" if system_ssl is not None else "N/A"
-    }
+    if ssl is not None:
+        system_ssl = ssl.OPENSSL_VERSION_NUMBER
+
+        system_ssl_info = {
+            "version": f"{system_ssl:x}" if system_ssl is not None else "N/A",
+            "name": ssl.OPENSSL_VERSION,
+        }
+    else:
+        system_ssl_info = {"version": "N/A", "name": "N/A"}
 
     return {
         "platform": platform_info,
@@ -139,6 +154,10 @@ def info():
             "version": wassima.__version__,
         },
         "ocsp": {"enabled": ocsp_verify is not None},
+        "websocket": {
+            "enabled": wsproto is not None,
+            "wsproto": wsproto.__version__ if wsproto is not None else None,
+        },
     }
 
 
@@ -177,6 +196,7 @@ def check_update(package_name: str, actual_version: str) -> None:
     "charset-normalizer": charset_normalizer.__version__,
     "wassima": wassima.__version__,
     "idna": idna.__version__,
+    "wsproto": wsproto.__version__ if wsproto is not None else None,
 }
 
 

src/niquests/models.py

@@ -57,10 +57,12 @@
     from urllib3.contrib.webextensions._async import (
         AsyncWebSocketExtensionFromHTTP,
         AsyncRawExtensionFromHTTP,
+        AsyncServerSideEventExtensionFromHTTP,
     )
     from urllib3.contrib.webextensions import (
         WebSocketExtensionFromHTTP,
         RawExtensionFromHTTP,
+        ServerSideEventExtensionFromHTTP,
     )
 else:
     from urllib3_future import (  # type: ignore[assignment]
@@ -82,10 +84,12 @@
     from urllib3_future.contrib.webextensions._async import (  # type: ignore[assignment]
         AsyncWebSocketExtensionFromHTTP,
         AsyncRawExtensionFromHTTP,
+        AsyncServerSideEventExtensionFromHTTP,
     )
     from urllib3_future.contrib.webextensions import (  # type: ignore[assignment]
         WebSocketExtensionFromHTTP,
         RawExtensionFromHTTP,
+        ServerSideEventExtensionFromHTTP,
     )
 
 from ._typing import (
@@ -885,15 +889,15 @@ def _encode_files(
             ft: str | None = None
             fh: HeadersType | None = None
 
-            if isinstance(fdescriptor, tuple):
+            if isinstance(fdescriptor, (tuple, list)):
                 # mypy and tuple length cmp not supported
                 # https://github.com/python/mypy/issues/1178
                 if len(fdescriptor) == 2:
-                    fn, fp = fdescriptor  # type: ignore[misc]
+                    fn, fp = tuple(fdescriptor)  # type: ignore[misc,assignment]
                 elif len(fdescriptor) == 3:
-                    fn, fp, ft = fdescriptor  # type: ignore[misc]
+                    fn, fp, ft = tuple(fdescriptor)  # type: ignore[misc,assignment]
                 else:
-                    fn, fp, ft, fh = fdescriptor  # type: ignore[misc]
+                    fn, fp, ft, fh = tuple(fdescriptor)  # type: ignore[misc,assignment]
             else:
                 if isinstance(fdescriptor, (str, bytes, bytearray)):
                     fn = fkey
@@ -1058,8 +1062,10 @@ def extension(
     ) -> (
         WebSocketExtensionFromHTTP
         | RawExtensionFromHTTP
+        | ServerSideEventExtensionFromHTTP
         | AsyncWebSocketExtensionFromHTTP
         | AsyncRawExtensionFromHTTP
+        | AsyncServerSideEventExtensionFromHTTP
         | None
     ):
         """Access the I/O after an Upgraded connection. E.g. for a WebSocket handler.