diff --git a/README.md b/README.md index 01e0c9e4f..f4de52022 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -Visit the [project website](https://mizosoft.github.io/methanol) for more info. +*note: documentation may contain updates for 1.8.0, which is not yet released* # Methanol @@ -7,16 +7,15 @@ Visit the [project website](https://mizosoft.github.io/methanol) for more info. [![Maven Central](https://img.shields.io/maven-central/v/com.github.mizosoft.methanol/methanol?style=flat-square)](https://search.maven.org/search?q=g:%22com.github.mizosoft.methanol%22%20AND%20a:%22methanol%22) [![Javadoc](https://img.shields.io/maven-central/v/com.github.mizosoft.methanol/methanol?color=blueviolet&label=Javadoc&style=flat-square)](https://mizosoft.github.io/methanol/api/latest/) -Java enjoys a neat, built-in [HTTP client](https://openjdk.java.net/groups/net/httpclient/intro.html). -However, it lacks key HTTP features like multipart uploads, caching and response decompression. -***Methanol*** comes in to fill these gaps. The library comprises a set of lightweight, yet powerful -extensions aimed at making it much easier & more productive to work with `java.net.http`. You can -say it's an `HttpClient` wrapper, but you'll see it almost seamlessly integrates with the standard -API you might already know. +Java enjoys a neat, built-in [HTTP client](https://openjdk.java.net/groups/net/httpclient/intro.html). However, it lacks key HTTP features like multipart uploads, caching and response decompression. +***Methanol*** comes in to fill these gaps. The library comprises a set of lightweight, yet powerful extensions aimed at making it much easier & more productive to work with `java.net.http`. +You can say it's an `HttpClient` wrapper, but you'll see it almost seamlessly integrates with the standard API you might already know. Methanol isn't invasive. The core library has zero runtime dependencies. However, special attention is given to object mapping, so integration with libraries like Jackson or Gson becomes a breeze. +[//]: # (There's also a nice DSL for Kotlin lovers!) + ## Installation ### Gradle diff --git a/USER_GUIDE.md b/USER_GUIDE.md deleted file mode 100644 index c641e1492..000000000 --- a/USER_GUIDE.md +++ /dev/null @@ -1,833 +0,0 @@ -**CAUTION**: This user guide is stale and won't be updated for future versions. Please visit the -[project website](https://mizosoft.github.io/methanol) instead. - -# Methanol's User Guide - -* [Overview](#overview) - * [Goals](#goals) - * [Non-goals](#non-goals) - * [Native reactiveness](#native-reactiveness) -* [Usage](#usage) - * [Response decompression](#response-decompression) - * [BodyDecoder](#bodydecoder) - * [Supported encodings](#supported-encodings) - * [Extending decompression support](#extending-decompression-support) - * [Request decoration](#request-decoration) - * [Transparent compression](#transparent-compression) - * [No overwrites](#no-overwrites) - * [MutableRequest](#mutablerequest) - * [MediaType](#mediatype) - * [Media ranges](#media-ranges) - * [MimeBodyPublisher](#mimebodypublisher) - * [Object conversion](#object-conversion) - * [BodyAdapter](#bodyadapter) - * [Installation](#installation) - * [Deferred conversion](#deferred-conversion) - * [Interceptors](#interceptors) - * [Invocation order](#invocation-order) - * [Reactive request dispatches](#reactive-request-dispatches) - * [Push promises](#push-promises) - * [Sending forms](#sending-forms) - * [Multipart bodies](#multipart-bodies) - * [WritableBodyPublisher](#writablebodypublisher) - * [Interruptible reading](#interruptible-reading) - * [Tracking progress](#tracking-progress) - -## Overview - -***Methanol*** is a lightweight library that complements `java.net.http` for a better HTTP -experience. Applications using Java's non-blocking HTTP client shall find it more robust and easier -to use with Methanol. - -### Goals - -* Provide useful lightweight HTTP extensions built on top of `java.net.http`. -* Enhance the client's API to make it more powerful and easier to use. -* Allow easier integration with other libraries commonly used in HTTP messaging. - -### Non-goals - -* Providing fancy low-level extensions like network interceptors or connection pooling control; we - can only go as far as allowed by the client's API. - -### Native reactiveness - -Most features provided by Methanol are [reactive-streams](http://www.reactive-streams.org/) -extensions. If you're wondering which of the wild reactive-streams implementations Methanol is -using, the answer is *none*. Methanol has its native `Flow.*` implementations. This was resolved due - to a couple of reasons: - -* Most prominent reactive-streams libraries are non-trivial dependencies; Methanol is meant to be - lightweight. Additionally, it doesn't *require* any of the fancy rx operators. -* Choosing an implementation over the other might be unsuitable for some applications. -* Integration can be rather provided via separate modules (e.g. -[methanol-jackson-flux](methanol-jackson-flux)). - -Java's `SubmissionPublisher` could have worked in some cases, but it has awkward buffering policies -and requires asynchronous scheduling of subscriber signals, so it wasn't an option. - -That being said, it's worth noting that Methanol's native `Flow.*` implementations are tested -thoroughly and validated against the [TCK][tck]. Additionally, they utilize optimizations common to -the reactive-streams world, making them fast and efficient. - -## Usage - -Methanol adheres to the standard `java.net.http` API by introducing very few new concepts. Using the -provided extensions shouldn't feel that different from normal `HttpClient` usage. If you're not -familiar with the client, make sure to skim through the [recipes][httpclient_recipies] to get an idea -of basic usage patterns. - -### Response decompression - -One caveat concerning the HTTP client is that it has [no native decompression support][so_question]. -A solution to this is to use an available `InputStream` decompressor (e.g. `GZIPInputStream`) -corresponding to the value of the `Content-Encoding` header. However, this forces us to always use -an `InputStream` body, which effectively throws away most of the flexibility the client's API -provides. We're back at the blocking realm again! We can definitely do better than this. - -With Methanol, all you need is to simply wrap your `BodyHandler` with -[`MoreBodyHandlers::decoding`][MoreBodyHandlers_decoding]: - -```java -HttpResponse response = client.send(request, MoreBodyHandlers.decoding(BodyHandlers.ofString())); -``` - -And that's it! The new `BodyHandler` will intercept the response, checking if a `Content-Encoding` -header is present. If so, the body is decompressed accordingly. Otherwise, it acts as a no-op and -delegates to your handler directly. - -Notes: - -* It doesn't matter which `BodyHandler` you are using; you can have whatever response body type you - want! -* If the response is compressed, your handler won't see any `Content-Encoding` or `Content-Length` - headers. This is simply because they'll be outdated in that case. - -#### BodyDecoder - -What makes this possible is the [`BodyDecoder`][BodyDecoder] interface. A `BodyDecoder` is a normal -`BodySubscriber` but with the added semantics of a `Flow.Processor`. It intercepts the flow of bytes -from the HTTP client and decodes each `List` individually. It then forwards the decoded -bytes into your downstream `BodySubscriber`, which itself converts those into the desired response -body. - -##### Scheduling signals - -A `BodyDecoder` has two modes for scheduling downstream signals: - -* **Synchronous**: The decoder processes and submits downstream items in the same thread supplying - the compressed bytes. This is normally a thread in the client's `Executor`. -* **Asynchronous**: The decoder dispatches downstream signals to a custom `Executor`. This can lead - to overlapped processing between the two subscribers (both can work asynchronously at different - rates). Note, however, that the overhead of managing the executor itself might overwhelm any - improvement gained from both subscribers running asynchronously, possibly resulting in decreased - performance. - -#### Supported encodings - -The core module has default support for deflate and gzip. There is also an optional -[module][methanol_brotli] providing support for [brotli][google_brotli]. - -#### Extending decompression support - -Adding support for more encodings is a matter of providing matching `BodyDecoder.Factory` -implementations. However, implementing the `Flow.Publisher` semantics of `BodyDecoder` can be a -challenge if you're not using a reactive-streams library. [`AsyncBodyDecoder`][AsyncBodyDecoder] is -provided for that purpose. This class, along with its sibling interface [`AsyncDecoder`][AsyncDecoder], -allows you to only focus on your decompression logic. For the sake of an example, let's add support -for the `identity` encoding (note that this is not really needed as `identity` is not a valid value -for `Content-Encoding`). - -Decoding is simply done as zero or more `decode(source, sink)` rounds finalized by one final round, -each with the currently available input. After the final round, `AsyncDecoder` must've completely -exhausted the source. - -```java -public class IdentityBodyDecoderFactory implements BodyDecoder.Factory { - private static final String IDENTITY = "identity"; - - public IdentityBodyDecoderFactory() {} - - @Override - public String encoding() { - return IDENTITY; - } - - @Override - public BodyDecoder create(BodySubscriber downstream) { - return new AsyncBodyDecoder<>(new IdentityDecoder(), downstream); - } - - @Override - public BodyDecoder create(BodySubscriber downstream, Executor executor) { - return new AsyncBodyDecoder<>(new IdentityDecoder(), downstream, executor); - } - - private static final class IdentityDecoder implements AsyncDecoder { - IdentityDecoder() {} - - @Override - public String encoding() { - return IDENTITY; - } - - @Override - public void decode(ByteSource source, ByteSink sink) { - while (source.hasRemaining()) { - sink.pushBytes(source.currentSource()); - } - } - - @Override - public void close() {} - } -} -``` - -Now make your implementation discoverable by Methanol's `ServiceLoader`. In case you're using the -module path, you can add a `provides...with` clause to your `module-info.java` as follows: - -```java -provides BodyDecoder.Factory with IdentityBodyDecoderFactory; -``` - -### Request decoration - -You can use the [`Methanol`][Methanol] `HttpClient` wrapper to decorate outgoing requests with -default properties. Think resolving with a base `URI`, adding default HTTP headers, default -timeouts, etc. - -```java -Methanol client = Methanol.newBuilder() - .userAgent("Will Smith") // Set a custom User-Agent - .baseUri("https://api.github.com") // To resolve each request's URI against - .defaultHeader("Accept", "application/vnd.github.v3+json") // Added to each request if not present - .requestTimeout(Duration.ofSeconds(5)) // Default request timeout to use if not set - .autoAcceptEncoding(true) // Transparent compression, this is true by default - .version(Version.HTTP_2) // Then configure the client as with HttpClient.Builder - ... - .build(); -``` - -You can also build from an existing `HttpClient`: - -```java -HttpClient baseClient = ... -Methanol client = Methanol.newBuilder(baseClient) - .defaultHeaders(...) - ... - .build(); -``` - -#### Transparent compression - -If transparent compression is enabled, the client will request a compressed response with all -supported schemes (available `BodyDecoder` providers). For example, if gzip, deflate and brotli are -supported, each request will have an `Accept-Encoding: br, deflate, gzip` header added. Of course, -the response will be automatically decompressed as well. - -#### No overwrites - -Default request properties are not set in a request that already has them. For example, for a client -with default header `Accept: text/html`, a request with an `Accept: application/json` header will -remain so. - -You may benefit from this in forcing the use of a specific compression scheme instead of all -supported ones (if transparent compression is enabled): - -```java -Methanol client = Methanol.create(); -MutableRequest request = MutableRequest.GET(uri) - .header("Accept-Encoding", "gzip"); // Will force gzip only -HttpResponse response = client.send(request, BodyHandlers.ofString()); -``` - -An exception to this is that a request with a [`MimeBodyPublisher`](#mimebodypublisher) will have -it's `Content-Type` header set, or overwritten if already there. This makes sense because a body -knows about its media type better than a containing request mistakenly setting a different one. - -### MutableRequest - -[`MutableRequest`][MutableRequest] is an `HttpRequest` that implements `HttpRequest.Builder` for -setting request's fields. This drops immutability in favor of more convenience when the request is -used immediately (which is typically the case): - -```java -HttpResponse response = client.send(MutableRequest.GET(uri), BodyHandlers.ofString()); -``` - -Additionally, it also allows setting a relative or empty `URI` (standard `HttpRequest.Builder` -doesn't). This is useful when the request is sent over a `Methanol` client with a base `URI` against -which the relative one is resolved. - -If you still want immutability, you can use `MutableRequest::build` for getting an immutable -`HttpRequest` snapshot. - -### MediaType - -Media types are the web's notion of file extensions. They play a very important role in identifying -how a response body should be approached. Methanol provides a [`MediaType`][MediaType] class for -representing and manipulating media types. - -```java -MediaType applicationJsonUtf8 = MediaType.of("application", "json", Map.of("charset", "UTF-8")); -MediaType parsedApplicationJsonUtf8 = MediaType.parse("application/json; charset=UTF-8"); -assertEquals(applicationJsonUtf8, parsedApplicationJsonUtf8); - -assertEquals("application", applicationJsonUtf8.type()); -assertEquals("json", applicationJsonUtf8.subtype()); -assertEquals("utf-8", applicationJsonUtf8.parameters().get("charset")); -assertEquals(Optional.of(StandardCharsets.UTF_8), applicationJsonUtf8.charset()); -``` - -#### Media ranges - -A `MediaType` also defines a [media range][media_ranges] to which a set of media types belong, -including itself. A media range is created like any other media type. You can check for inclusion in -a range with `MediaType::includes` or `MediaType::isCompatibleWith`. - -```java -MediaType anyTextType = MediaType.parse("text/*"); -MediaType textHtml = MediaType.parse("text/html"); -MediaType applicationJson = MediaType.parse("application/json"); - -assertTrue(anyTextType.hasWildcard()); -assertTrue(anyTextType.includes(textHtml)); -assertFalse(anyTextType.includes(applicationJson)); -assertTrue(anyTextType.isCompatibleWith(textHtml)); -assertTrue(textHtml.isCompatibleWith(anyTextType)); -``` - -The class also has `static final` definitions for commonly used media types and ranges. - -### MimeBodyPublisher - -`MimeBodyPublisher` is a mixin-style interface for body publishers that know their body's concrete -media type. The interface is recognized by [multipart bodies](#multipart-bodies) and the -[`Methanol`][Methanol] client in that any `MimeBodyPublisher` will have it's `Content-Type` header -implicitly added. You can adapt an existing `BodyPublisher` into a `MimeBodyPublisher` using -[`MoreBodyPublishers::ofMediaType`][MoreBodyPublishers_ofMediaType]: - -```java -static MimeBodyPublisher ofMimeBody(String body, MediaType mediaType) { - Charset charset = mediaType.charsetOrDefault(StandardCharsets.UTF_8); - BodyPublisher bodyPublisher = BodyPublishers.ofString(body, charset); - return MoreBodyPublishers.ofMediaType(bodyPublisher, mediaType); -} -``` - -### Object conversion - -It is often the case that an HTTP body is mappable to or from a higher-level entity that your code -understands. `BodyPublisher` and `BodySubscriber` APIs are designed with this in mind. However, -available implementations, especially available `BodyHandlers`, are not really that high-level. -Implementing your own can be a tiring and repetitive process, not to mention, for example, choosing -the correct handler for each response (e.g. `JsonHandler` for `application/json` or `XmlHandler` for - `application/xml`). Methanol does that for you! - -In case of response handling, all you need is to pass a `Class` for your desired type (assuming you -have the correct dependencies [installed](#installation)): - -```java -final Methanol client = Methanol.newBuilder() - .baseUri("https://api.github.com") - .defaultHeader("Accept", "application/vnd.github.v3+json") - .build(); - -GitHubUser getUser(String name) throws IOException, InterruptedException { - MutableRequest request = MutableRequest.GET("/users/" + name); - HttpResponse response = - client.send(request, MoreBodyHandlers.ofObject(GitHubUser.class)); - - return response.body(); -} - -static class GitHubUser { - public String login; - public long id; - public String bio; - // other fields omitted -} -``` - -Or pass a [`TypeRef`][TypeRef] if you wanna get fancier with generics: - -```java -List getUserFollowers(String userName) throws IOException, InterruptedException { - MutableRequest request = MutableRequest.GET("/users/" + userName + "/followers"); - HttpResponse> response = - client.send(request, MoreBodyHandlers.ofObject(new TypeRef>() {})); - - return response.body(); -} -``` - -For requests, just pass whatever `Object` you want to convert, along with a `MediaType` describing -which format to use for serialization. - -```java -String renderMarkdown(RenderRequest renderRequest) throws IOException, InterruptedException { - BodyPublisher requestBody = MoreBodyPublishers.ofObject(renderRequest, MediaType.APPLICATION_JSON); - // No need to set Content-Type header! - MutableRequest request = MutableRequest.POST("/markdown", requestBody) - .header("Accept", "text/html"); - HttpResponse response = client.send(request, BodyHandlers.ofString()); - - return response.body(); -} - -static class RenderRequest { - public String text, mode, context; -} -``` - -#### BodyAdapter - -[`BodyAdapter`][BodyAdapter] is the driver for this automatic object conversion mechanism. It's two -specialized subtypes, [`Encoder`][Encoder] and [`Decoder`][Decoder], act as factories for -`BodyPublisher` and `BodySubscriber` respectively. Given a `TypeRef` and an optional `MediaType`, -a `Decoder` returns a `BodySubscriber` responsible for converting the response body into a `T`. -Similarly, given an `Object` and optionally a `MediaType`, an `Encoder` returns a `BodyPublisher` -that streams that object's serialized form. - -#### Installation - -You add support for schemes and object types by installing matching `Decoder` and `Encoder` adapters. -Methanol has the following adapter modules: - -* [methanol-gson](methanol-gson): JSON conversion with Gson -* [methanol-jackson](methanol-jackson): JSON conversion with Jackson -* [methanol-jackson-flux](methanol-jackson-flux): Reactive JSON conversion with Jackson and Reactor -* [methanol-protobuf](methanol-protobuf): Support for Protocol Buffers -* [methanol-jaxb](methanol-jaxb): XML conversion with JAXB - -Note that implementations are not service-provided by default. See each module's `README` for how to -install. - -#### Deferred conversion - -Most messaging libraries support either reading from a streaming source (e.g. `InputStream` or a -`Reader`), or from a memory buffer (e.g. `byte[]` or `String`). Streaming sources are more efficient -because they do not require having the whole thing in memory prior to processing. - -##### The problem - -Streaming sources are also blocking. They assume you can either read something if it's there or keep -blocking till it's available. This simply doesn't fit in the reactive world, which is fundamentally -non-blocking (you request a message and get notified when it arrives). You can try to get around -this by using `BodySubscribers.mapping(BodySubscribers.ofInputStream(), ...)` with a -`Function`. However, this exposes your code to multiple problems, including starving -the client out of threads and even lurking deadlocks due to a pre-JDK13 -[bug][BodySubscribers_mapping_bug]. - -##### The solution - -Following [Javadoc's][BodyHandlers_mapping_jdk13] advice, `MoreBodyHandlers` declares `T` and -`Supplier` variants for dynamically handling a response. Use -[`MoreBodyHandlers::ofObject`][MoreBodyHandlers_ofObject] to get an `HttpResponse`, which will -typically buffer the body into memory then decode from there. This is fine for bodies with relatively -small sizes. Use [`MoreBodyHandlers::ofDeferredObject`][MoreBodyHandlers_ofDeferredObject] to get an -`HttpResponse>`, which will be completed immediately after headers are received, -*deferring* body processing till `Supplier::get` is called. This has better memory efficiency as it -reads from a streaming source, suiting cases where loading the whole body might cause problems. Be -aware however that processing in that case is done by the thread invoking the supplier. - -### Interceptors - -Interceptors allow you to monitor, mutate, retry or even short-circuit ongoing exchanges. You can -register one or more interceptors with `Methanol` to be invoked in order for each `send` or -`sendAsync` call. Here's an interceptor that logs each ongoing blocking or asynchronous exchange. - -```java -class LoggingInterceptor implements Interceptor { - private static final Logger LOG = Logger.getLogger(LoggingInterceptor.class.getName()); - - @Override - public HttpResponse intercept(HttpRequest request, Chain chain) - throws IOException, InterruptedException { - logRequest(request); - - return chain.withBodyHandler(loggingBodyHandler(request, chain.bodyHandler())) - .forward(request); - } - - @Override - public CompletableFuture> interceptAsync( - HttpRequest request, Chain chain) { - logRequest(request); - - return chain.withBodyHandler(loggingBodyHandler(request, chain.bodyHandler())) - .forwardAsync(request); - } - - private static void logRequest(HttpRequest request) { - LOG.info(() -> String.format("Sending %s%n%s", request, headersToString(request.headers()))); - } - - private static BodyHandler loggingBodyHandler( - HttpRequest request, BodyHandler bodyHandler) { - var beforeSend = Instant.now(); - return info -> { - LOG.info(() -> String.format( - "Completed %s %s with %d in %s%n%s", - request.method(), - request.uri(), - info.statusCode(), - Duration.between(beforeSend, Instant.now()), - headersToString(info.headers()))); - - return bodyHandler.apply(info); - }; - } - - private static String headersToString(HttpHeaders headers) { - return headers.map().entrySet().stream() - .map(entry -> entry.getKey() + ": " + String.join(", ", entry.getValue())) - .collect(Collectors.joining(System.lineSeparator())); - } -} -``` - -You then register the interceptor with `Methanol.Builder` as follows: - -```java -var client = Methanol.newBuilder() - ... - .interceptor(new LoggingInterceptor()) - .build(); -``` - -Because the HTTP client has both blocking and asynchronous APIs, we must implement two -`Interceptor` methods matching each. An interceptor is given a `Chain` so that it can forward -requests to its sibling, or to the underlying HTTP client in case it's the last interceptor. The -chain can also be used to perform `BodyHandler` and `PushPromiseHandler` transformations before -forwarding. - -You can use `Interceptor::create` to easily rewrite or decorate requests. For example, you may want -to enable the expect-continue feature for each ongoing POST request for a specific host. - -```java -var myHost = ...; -var interceptor = Interceptor.create( - req -> req.uri().getHost().equals(myHost) && req.method().equalsIgnoreCase("POST") - ? MutableRequest.copyOf(req).expectContinue(true) - : req); -var client = Methanol.newBuilder() - ... - .interceptor(interceptor) - .build(); -``` - -Interceptors can forward to the chain as many times as they want. Here's an interceptor that retries -each request up to 3 times in case of timeout. - -```java -static class RetryingInterceptor implements Interceptor { - private static final int MAX_RETRIES = 3; - - @Override - public HttpResponse intercept(HttpRequest request, Chain chain) - throws IOException, InterruptedException { - for (int retries = 0; ; retries++) { - try { - return chain.forward(request); - } catch(HttpTimeoutException e) { - if (retries >= MAX_RETRIES) throw e; - } - } - } - - @Override - public CompletableFuture> interceptAsync( - HttpRequest request, Chain chain) { - return withRetries(() -> chain.forwardAsync(request), new AtomicInteger()); - } - - private CompletableFuture> withRetries( - Supplier>> callOnRetry, - AtomicInteger retryCount) { - return callOnRetry.get() - .handle((r, x) -> handleRetry(r, x, callOnRetry, retryCount)) - .thenCompose(Function.identity()); - } - - private CompletableFuture> handleRetry( - HttpResponse response, Throwable error, - Supplier>> callOnRetry, AtomicInteger retryCount) { - if (response != null) return CompletableFuture.completedFuture(response); - - if (error instanceof CompletionException) error = error.getCause(); - - return error instanceof HttpTimeoutException && retryCount.incrementAndGet() <= MAX_RETRIES - ? withRetries(callOnRetry, retryCount) - : CompletableFuture.failedFuture(error); - } -} -``` - -The async version looks a bit awkward as we have to perform some async recursive lambda magic for -retries. - -> If you're on JDK 12+, `handle(...).thenCompse(Function.identity())` can be replaced with -> `exceptionallyCompose(...)` API. - -#### Invocation order - -Due to the fact that the client itself does request decoration and response body -transformation (i.e. decompression), interceptors are separated into two groups: *pre decoration* -and *post decoration* interceptors. The only difference is that the former gets invoked before any -default request properties or handler transformations are applied, while the latter gets invoked -right before relaying to the underlying HTTP client. Order of invocation for each group matches -addition order. You should be aware of this if you intend to do checksums or request/response body -transformation (i.e. encryption/decryption). - -You can add post decoration interceptors as follows: - -```java -var client = Methanol.newBuilder() - ... - .postDecorationInterceptor(new LoggingInterceptor()) - .build(); -``` - -### Reactive request dispatches - -For a truly reactive experience, one might want to dispatch async requests as -`Publisher>` sources. You can use `Methanol::exchnge` for this. - -```java -Methanol client = Methanol.create(); -Publisher> publisher = - client.exchange(MutableRequest.GET("https://example.com"), BodyHandlers.ofString()); - -publisher.subscribe(new Subscriber<>() { - @Override public void onSubscribe(Subscription subscription) { - subscription.request(Long.MAX_VALUE); - } - - @Override public void onNext(HttpResponse response) { - System.out.println("Response arrived: " + response.statusCode()); - System.out.println(response.body()); - } - - @Override public void onError(Throwable throwable) { - throwable.printStackTrace(); - } - - @Override public void onComplete() {} -}); -``` - -Here, the publisher acts as a `Mono` source which publishes the response when requested, then -completes the subscriber either normally if successful, or exceptionally if the request fails. - -#### Push promises - -You can also retrieve a `Publisher>` that publishes resources pushed by the server -in addition to the main response (only works with HTTP/2). Use with a reactive-streams library for -better control of the stream. - -```java -Methanol client = Methanol.create(); // default Version is HTTP_2 -MutableRequest request = MutableRequest.GET("https://http2.golang.org/serverpush"); -Publisher> publisher = - client.exchange( - request, - BodyHandlers.ofFile(Path.of("index.html")), - promise -> BodyHandlers.ofFile(Path.of(promise.uri().getPath()).getFileName())); -JdkFlowAdapter.flowPublisherToFlux(publisher) - .filter(res -> res.statusCode() == 200) - .map(HttpResponse::body) - .subscribe(System.out::println); -``` - -The function passed to `exchange(...)` must return a `BodyHandler` for handling the pushed -response. It may return `null` to reject the promise entirely. - -### Sending forms - -[`FormBodyPubliher`][FormBodyPublisher] implements the `application/x-www-form-urlencoded` format -often used with `
` HTML tags. Data in this body is added as a sequence of basic `String` -key-value pairs and serialized into a URL-encoded string. - -```java -FormBodyPublisher body = FormBodyPublisher.newBuilder() - .query("foo", "bar") - .query("baz", "qux") - .build(); -System.out.println("Encoded body: " + body.encodedString()); - -HttpRequest request = HttpRequest.newBuilder() - .POST(body) - .header("Content-Type", body.mediaType().toString()) - ... - .build(); -``` - -Note that `FormBodyPublisher` implements [`MimeBodyPublisher`](MimeWiki.md#mimebodypublisher) so it -knows it's request's `Content-Type`, which can be transparently added if using the -[`Methanol`][Methanol] client. - -### Multipart bodies - -[`MultipartBodyPublisher`][MultipartBodyPublisher] implements the more flexible `multipart` format. -This format is often used with file uploads and sending composite bodies with mixed schemes. A -multipart body has one or more parts separated by a boundary, where each part is itself another -`BodyPublisher` with `HttpHeaders` that describe it. The default multipart subtype is `form-data`, -but you can use any other subtype as well. `MultipartBodyPublisher.Builder`'s API is flexible enough -for adding any combination of body parts you want. The builder also has convenient methods for -directly adding form parts (those with a `Content-Disposition: form-data` header). - -```java -MultipartBodyPublisher body = -MultipartBodyPublisher.newBuilder() - .textPart("text_field", "Hello world!") - .filePart("file_field", Path.of("path/to/file")) - .formPart( - "json_field", - MoreBodyPublishers.ofMediaType( - BodyPublishers.ofString(...), MediaType.APPLICATION_JSON)) // explicitly specify part's Content-Type - .part(Part.create(HttpHeaders.of(...), BodyPublishers.ofInputStream(...))) // can use custom headers/bodies - .build(); -``` - -Note that if a file part is added without specifying a `MediaType`, the builder will ask the system -for one using the provided `Path`. If it fails to do so, `application/octet-stream` will be used. -You can explicitly specify a form-part's media type by adding it as a `MimeBodyPublisher`. - -### WritableBodyPublisher - -Not all APIs play well with non-blocking components like `BodyPublisher`. Many only support writing -into a blocking sink like an `OutputStream` or a `Reader`. Using such APIs can be easier with -[`WritableBodyPublisher`][WritableBodyPublisher], which allows you to stream the request body -through an `OutputStream` or a `WritableByteChannel`, possibly asynchronously. - -Let's say your sever supports compressed requests, and you want your file upload to be faster, so -you compress the request body with gzip. - -```java -final Methanol client = Methanol.create(); - -CompletableFuture> postAsync(Path file) { - WritableBodyPublisher requestBody = WritableBodyPublisher.create(); - MutableRequest request = MutableRequest.POST("https://example.com", requestBody) - .header("Content-Encoding", "gzip"); - - CompletableFuture.runAsync(() -> { - try (OutputStream gzipOut = new GZIPOutputStream(requestBody.outputStream())) { - Files.copy(file, gzipOut); - } catch (IOException ioe) { - requestBody.closeExceptionally(ioe); - } - }); - - return client.sendAsync(request, BodyHandlers.discarding()); -} -``` - -Note that `WritableBodyPublisher` acts as a pipe which connects `OutputStream` and -`Publisher` backends. It may buffer content temporarily in case the consumer can't keep -up with the producer, or till an inner buffer becomes full. You can use `WritableBodyPublisher::flush` -to make any buffered content consumable by the downstream. After writing content, call `close()` or -`closeExceptionally(Throwable)` to complete the request either normally or exceptionally. - -### Interruptible reading - -Another feature you might find useful if you like reading from blocking sources is support for -[interruptible channels][InterruptibleChannel]. These allow you to asynchronously close or interrupt -a reading operation in case it is not relevant anymore (e.g. when closing the application or -changing contexts). [`MoreBodySubscibers`][MoreBodySubscribers] provides interruptible -`ReadableByteChannel` and `Reader` implementations. - -This example uses a hypothetical component that reads the response from a `ReadableByteChannel`. -When the task is to be discarded, reader threads are interrupted by shutting down the containing -`ExecutorService`. This closes the channel and instructs it to immediately stop blocking. - -```java -class BodyProcessor { - final ExecutorService service = Executors.newCachedThreadPool(); - final HttpClient client = HttpClient.newHttpClient(); - - CompletableFuture processAsync(HttpRequest request, Consumer processAction) { - return client.sendAsync(request, MoreBodyHandlers.ofByteChannel()) - .thenApplyAsync(res -> { - ByteBuffer data = ByteBuffer.allocate(4 * 1024); - try (ReadableByteChannel channel = res.body()) { - while (channel.read(data.clear()) >= 0) { - processAction.accept(data.flip()); - } - } catch (ClosedByInterruptException | ClosedChannelException ignored) { - // The thread was interrupted due to pool shutdown - } catch (IOException e) { - throw new UncheckedIOException(e); - } - return null; - }, service); - } - - void terminate() { service.shutdownNow(); } -} -``` - -### Tracking progress - -Methanol introduces a simple API for tracking the progress of single upload or download operations. -You do that by attaching a [`Listener`][Listener] to a `BodyPublisher` or `BodyHandler` using a -configured [`ProgressTracker`][ProgressTracker]. To avoid getting too many events, you can configure -a tracker to signal progress only when a byte count or time threshold is exceeded. You can also set -a custom `Executor` for invoking listener callbacks. If the listener updates GUI for example, this -can be used to make sure it is executed in the GUI thread. - -```java -ProgressTracker tracker = ProgressTracker.newBuilder() - .bytesTransferredThreshold(5 * 1024) // at least 5KB is downloaded before any events - .timePassedThreshold(Duration.ofSeconds(1)) // at least 1 second passes before any events - .executor(SwingUtilities::invokeLater) // invoke in event dispatcher thread if updating UI - .build(); - -// Track upload -BodyPublisher requestBody = - tracker.tracking(BodyPublishers.ofString(...), p -> logUploadProgress(p)); -MutableRequest request = MutableRequest.POST(url, requestBody); - -// Track download -HttpResponse response = - client.send(request, tracker.tracking(BodyHandlers.ofString(), p -> logDownloadProgress(p))); -``` - -In case of [compressed responses](#response-decompression), the `Content-Length` header becomes -invalidated, which prevents calculation of progress percentage. On such case you can first send a -`HEAD` request with `Accept-Encoding: identity` to get the correct length. Then use that to track -download from a downstream `BodySubscriber`. See -[this JavaFX sample](methanol-samples/src/main/java/com/github/mizosoft/methanol/samples/DownloadProgress.java) -for an example. - -[tck]: -[httpclient_recipies]: -[so_question]: -[MoreBodyHandlers_decoding]: -[BodyDecoder]: -[methanol_brotli]: -[google_brotli]: -[AsyncBodyDecoder]: -[AsyncDecoder]: -[Methanol]: -[MutableRequest]: -[MediaType]: -[media_ranges]: -[MoreBodyPublishers_ofMediaType]: -[TypeRef]: -[BodyAdapter]: -[Encoder]: -[Decoder]: -[BodySubscribers_mapping_bug]: -[BodyHandlers_mapping_jdk13]: -[MoreBodyHandlers_ofObject]: -[MoreBodyHandlers_ofDeferredObject]: -[FormBodyPublisher]: -[MultipartBodyPublisher]: -[WritableBodyPublisher]: -[InterruptibleChannel]: -[MoreBodySubscribers]: -[ProgressTracker]: -[Listener]: diff --git a/buildSrc/src/main/kotlin/conventions/kotlin-library.gradle.kts b/buildSrc/src/main/kotlin/conventions/kotlin-library.gradle.kts index fc2325201..7df41bfd8 100644 --- a/buildSrc/src/main/kotlin/conventions/kotlin-library.gradle.kts +++ b/buildSrc/src/main/kotlin/conventions/kotlin-library.gradle.kts @@ -1,3 +1,25 @@ +/* + * Copyright (c) 2024 Moataz Abdelnasser + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + package conventions import extensions.javaModuleName @@ -24,7 +46,7 @@ tasks.withType().configureEach { tasks.withType { try { moduleName = project.javaModuleName - } catch (e: IllegalStateException) { - project.logger.warn("Couldn't get Java module name for Kotlin project (${project.name})", e) + } catch (_: IllegalStateException) { + project.logger.warn("Couldn't get Java module name for Kotlin project (${project.name})") } } diff --git a/docs/adapters.md b/docs/adapters.md new file mode 100644 index 000000000..31003a1d2 --- /dev/null +++ b/docs/adapters.md @@ -0,0 +1,196 @@ +# Adapters + +HTTP bodies are often mappable to high-level types that your code understands. Java's HttpClient was designed +with that in mind. However, available `BodyPublisher` & `BodySubscriber` implementations are too basic, and +implementing your own can be tricky. Methanol builds upon these APIs with an extensible object mapping mechanism +that treats your objects as first-citizen HTTP bodies. + +## Setup + +A serialization library can be integrated with Methanol through a corresponding adapter. +Adapters for the most popular serialization libraries are provided by separate modules. + +* [`methanol-gson`](adapters/gson.md): JSON with Gson +* [`methanol-jackson`](adapters/jackson.md): JSON with Jackson (but also XML, protocol buffers and other formats support by Jackson) +* [`methanol-jackson-flux`](adapters/jackson_flux.md): Streaming JSON with Jackson and Reactor +* [`methanol-jaxb`](adapters/jaxb.md): XML with JAXB +* [`methanol-jaxb-jakarta`](adapters/jaxb.md): XML with JAXB (Jakarta version) +* [`methanol-protobuf`](adapters/protobuf.md): Google's Protocol Buffers +* [`methanol-moshi`](adapters/moshi.md): JSON with Moshi, mainly for Kotlin + +We'll pick `methanol-jackson` for the examples presented here, which interact with GitHub's REST API. + +```java +var mapper = new JsonMapper(); +var adapterCodec = + AdapterCodec.newBuilder() + .basic() + .encoder(JacksonAdapterFactory.createEncoder(mapper, MediaType.APPLICATION_JSON)) + .decoder(JacksonAdapterFactory.createDecoder(mapper, MediaType.APPLICATION_JSON)) + .build(); +var client = + Methanol.newBuilder() + .adapterCodec(adapterCodec) + .baseUri("https://api.github.com/") + .defaultHeader("Accept", "application/vnd.github.v3+json") + .build(); +``` + +An `AdapterCodec` groups together one or more adapters, possibly targeting different mapping schemes. It helps `Methanol` +to select the correct adapter based on the request's or response's [`MediaType`](https://mizosoft.github.io/methanol/api/latest/methanol/com/github/mizosoft/methanol/MediaType.html). + +The [`basic()`][adaptercodec_basic_javadoc] calls adds the basic adapter, which encodes & decodes basic types like `String` & `InputStream`. + +## Receiving Objects + +To get an `HttpResponse`, give `Methanol::send` a `T.class`. + +```java + @JsonIgnoreProperties(ignoreUnknown = true) // We'll ignore most fields for brevity. +public record GitHubUser(String login, long id, String url) {} + +GitHubUser getUser(String username) throws IOException, InterruptedException { + return client.send(MutableRequest.GET("users/" + username), GitHubUser.class).body(); +} +``` + +If you want to get fancier with generics, pass a `TypeRef`. + +```java +@JsonIgnoreProperties(ignoreUnknown = true) // We'll ignore most fields for brevity. +public record GitHubIssue(String title, GitHubUser user, String body) {} + +List getIssues(String owner, String repo) throws IOException, InterruptedException { + return client.send( + MutableRequest.GET("repos/" + owner + "/" + repo + "/issues"), + new TypeRef>() {}).body(); +} +``` + +## Sending Objects + +Each `MutableRequest` can have a payload as its body. A payload is an arbitrary object that is not yet resolved into a `BodyPublisher`. +When the request is sent, the payload will be resolved with the client's `AdapterCodec`. + +```java +public record Markdown(String text, String context, String mode) {} + +String markdownToHtml(String text, String contextRepo) throws IOException, InterruptedException { + return client.send( + MutableRequest.POST("markdown", new Markdown(text, contextRepo, "gfm"), MediaType.APPLICATION_JSON), + String.class).body(); +} +``` + +A payload must be given along with a `MediaType` specifying the format with which it will be resolved. + +## Adapters + +An adapter provides an [`Encoder`][encoder_javadoc] and/or a [`Decoder`][decoder_javadoc] implementation. +An `Encoder` creates `BodyPublisher` instances that stream a given object's serialized form. +Similarly, a `Decoder` creates `BodySubscriber` instances that convert the response body into `T`. +Encoders & decoders are given [`Hints`][hints_javadoc] to customize their behavior. +One notable hint is the `MediaType`, which can be used to further describe the desired mapping format (e.g. specify a character set). + +### Example - An HTML Adapter + +Here's an adapter that uses [Jsoup][jsoup] to convert HTML bodies to `Document` objects and vise versa. +When you're writing adapters, it's a good idea to extend from [`AbstractBodyAdapter`][abstractbodyadapter_javadoc]. + +```java +public abstract class JsoupAdapter extends AbstractBodyAdapter { + JsoupAdapter() { + super(MediaType.TEXT_HTML); + } + + @Override + public boolean supportsType(TypeRef type) { + return type.rawType() == Document.class; + } + + public static final class Decoder extends JsoupAdapter implements BaseDecoder { + @Override + public BodySubscriber toObject(TypeRef typeRef, Hints hints) { + requireSupport(typeRef, hints); + var charset = hints.mediaTypeOrAny().charsetOrUtf8(); + var subscriber = BodySubscribers.mapping(BodySubscribers.ofString(charset), Jsoup::parse); + return BodySubscribers.mapping(subscriber, typeRef.exactRawType()::cast); // Safely cast Document to T. + } + } + + public static final class Encoder extends JsoupAdapter implements BaseEncoder { + @Override + public BodyPublisher toBody(T value, TypeRef typeRef, Hints hints) { + requireSupport(typeRef, hints); + var charset = hints.mediaTypeOrAny().charsetOrUtf8(); + var publisher = BodyPublishers.ofString(((Document) value).outerHtml(), charset); + return attachMediaType(publisher, hints.mediaTypeOrAny()); + } + } +} +``` + +!!! tip +Make sure your encoders call `AbstractBodyAdapter::attachMediaType` so the created `BodyPublisher` can be converted to a `MimeBodyPublisher`. +That way, requests get the correct `Content-Type` header added by `Methanol`. + +## Buffering vs Streaming + +Decoders typically load the whole response body into memory before deserialization. If your responses tend to have large bodies, +or you'd prefer the memory efficiency afforded by streaming sources, you can ask to get a `Supplier` instead. + +```java +@JsonIgnoreProperties(ignoreUnknown = true) // We'll ignore most fields for brevity. +public record GitHubUser(String login, long id, String url) {} + +GitHubUser getUser(String username) throws IOException, InterruptedException { + return client.send( + MutableRequest.GET("user/" + username), + new TypeRef>() {}).body().get(); +} +``` + +In such case, the response is completed as soon as all headers are read. If he decoder supports +streaming, the supplier will deserialize from a streaming source, typically an `InputStream` or a `Reader`. + +The way a `Decoder` implements streaming is by overriding `toDeferredObject` to return a `BodySubscriber>`. +Here's how it'd be properly implemented for our HTML adapter's decoder. + +```java +@Override +public BodySubscriber> toDeferredObject(TypeRef typeRef, Hints hints) { + requireSupport(typeRef, hints); + return BodySubscribers.mapping( + MoreBodySubscribers.ofReader(hints.mediaTypeOrAny().charsetOrUtf8()), + reader -> + () -> + typeRef + .exactRawType() // Get Class as Class + .cast( + Parser.htmlParser() + .parseInput( + new BufferedReader(reader), ""))); // Note the deferred parsing +} +``` + +## Legacy Adapters + +See [Legacy Adapter](legacy_adapters.md). + +[methanol_jackson]: https://github.com/mizosoft/methanol/tree/master/methanol-jackson + +[jsoup]: https://jsoup.org/ + +[encoder_javadoc]: ../api/latest/methanol/com/github/mizosoft/methanol/BodyAdapter.Encoder.html + +[decoder_javadoc]: ../api/latest/methanol/com/github/mizosoft/methanol/BodyAdapter.Decoder.html + +[bodyadapter_javadoc]: ../api/latest/methanol/com/github/mizosoft/methanol/BodyAdapter.html + +[hints_javadoc]: https://mizosoft.github.io/methanol/api/latest/methanol/com/github/mizosoft/methanol/BodyAdapter.Hints.html + +[mediatype_javadoc]: https://mizosoft.github.io/methanol/api/latest/methanol/com/github/mizosoft/methanol/MediaType.html + +[abstractbodyadapter_javadoc]: https://mizosoft.github.io/methanol/api/latest/methanol/com/github/mizosoft/methanol/adapter/AbstractBodyAdapter.html + +[adaptercodec_basic_javadoc]: https://mizosoft.github.io/methanol/api/latest/methanol/com/github/mizosoft/methanol/AdapterCodec.Builder.html#basic() diff --git a/docs/caching.md b/docs/caching.md index 273369b4e..450733f6d 100644 --- a/docs/caching.md +++ b/docs/caching.md @@ -1,48 +1,45 @@ # Caching -Methanol comes with an [RFC-compliant][rfc7234] HTTP cache that supports both disk & memory storage -backends. +Methanol comes with an [RFC-compliant][rfc7234] HTTP cache that supports disk & memory storage backends. +There's also an [extension](https://mizosoft.github.io/methanol/redis/) for Redis. ## Setup -An `HttpCache` is utilized by injecting it into a `Methanol` client. First, it needs to know where -it stores entries and how much space it can occupy. +An `HttpCache` is utilized by injecting it into a `Methanol` client. === "Disk" ```java - // Select a size limit thats suitable for your application + // Select a size limit thats suitable for your application. long maxSizeInBytes = 100 * 1024 * 1024; // 100 MBs - var cache = HttpCache.newBuilder() - .cacheOnDisk(Path.of("my-cache-dir"), maxSizeInBytes) + .cacheOnDisk(Path.of(".cache"), maxSizeInBytes) .build(); - // The cache intercepts requests you send through this client + // The cache intercepts requests you send through this client. var client = Methanol.newBuilder() .cache(cache) .build(); - // It's important that you close the disk cache after you're done + // Don't forget to close the cache when you're done! cache.close(); ``` === "Memory" ```java - // Select a size limit thats suitable for your application + // Select a size limit thats suitable for your application. long maxSizeInBytes = 50 * 1024 * 1024; // 50 MBs - var cache = HttpCache.newBuilder() .cacheOnMemory(maxSizeInBytes) .build(); - // The cache intercepts requests you send through this client + // The cache intercepts requests you send through this client. var client = Methanol.newBuilder() .cache(cache) .build(); - // No need to close, but doing so avoids surprises if you later switch to disk + // Don't forget to close the cache when you're done! cache.close(); ``` @@ -56,13 +53,28 @@ it stores entries and how much space it can occupy. an `IOException` if it's initialized with a directory that's already in use by another instance in the same or a different JVM. Note that you can use the same `HttpCache` with multiple clients. +An HTTP client can also be configured with a chain of caches, typically in the order of decreasing locality. +The chain is invoked in the given order, and a cache either returns the response if it has a suitable one, +or forwards to the next cache (or finally to the network) otherwise. + +```java +var memoryCache = HttpCache.newBuilder() + .cacheOnMemory(100 * 1024 * 1024) + .build(); +var diskCache = HttpCache.newBuilder() + .cacheOnDisk(Path.of(".cache"), 500 * 1024 * 1024) + .build(); +var client = Methanol.newBuilder() + .cacheChain(List.of(memoryCache, diskCache)) + .build(); +``` + ## Usage An HTTP cache is a transparent layer between you and the origin server. Its main goal is to save time & bandwidth by avoiding network if requested resources are locally retrievable. It does so -while preserving the typical HTTP client-server semantics. Thus, it should be OK for modules to -start using a cache-configured `Methanol` (and hence `HttpClient`) instance as a drop-in replacement -without further setup. +while preserving the typical HTTP client-server semantics. Thus, applications can start using a +cache-configured HTTP client instance as a drop-in replacement without further setup. ## CacheControl @@ -116,8 +128,8 @@ var cacheControl = CacheControl.newBuilder() You can specify how fresh you'd like the response to be by putting a lower bound on its freshness value. ```java -var cacheControl = CacheControl.newBuilder() - .minFresh(Duration.ofMinutes(10)) // Accept a response that stays fresh for at least the next 10 minutes +var cacheControl = CacheControl.newBuilder() + .minFresh(Duration.ofSeconds(30)) // Accept a response that stays fresh for at least the next 30 seconds .build(); ``` @@ -135,7 +147,7 @@ like `If-None-Match` & `If-Modified-Since`, if it can serve the stale response a If the server doesn't mind, the cache serves said response without re-downloading its payload. Otherwise, the response is re-fetched. -You can let the cache tolerate some stalness so it doesn't trigger revalidation. +You can let the cache tolerate some staleness so it doesn't trigger revalidation. === "Bounded Staleness" @@ -188,8 +200,7 @@ var cacheControl = CacheControl.newBuilder() .build(); ``` -A perfect use-case is when network is down or the app is offline. You'd want to get a cached -response if it's there or otherwise nothing. +A perfect use-case is when network is down. You may want to get a cached response if it's there or otherwise nothing. ### Prohibiting Storage @@ -210,8 +221,8 @@ immediately even if it's stale, but ensure it is freshened for later access. Tha `stale-while-revalidate` does. If the directive is found on a stale response, the cache serves it immediately provided it satisfies -allowed staleness. What's interesting is that an asynchronous revalidation is triggered and the response -is updated in background, keeping things fresh. +allowed staleness. Meanwhile, an asynchronous revalidation is triggered and the response is updated +in background, keeping things fresh. ## Invalidation @@ -219,18 +230,18 @@ is updated in background, keeping things fresh. ```java var cache = HttpCache.newBuilder() - .cacheOnDisk(Path.of("my-cache-dir"), 100 * 1024 * 1024) + .cacheOnDisk(Path.of(".cache"), 500 * 1024 * 1024) .build(); -// Remove the entry mapped to a particular URI +// Remove the entry mapped to a particular URI. cache.remove(URI.create("https://i.imgur.com/NYvl8Sy.mp4")); -// Remove the response variant matching a particular request +// Remove the response variant matching a particular request. cache.remove( MutableRequest.GET(URI.create("https://i.imgur.com/NYvl8Sy.mp4")) .header("Accept-Encoding", "gzip")); -// Remove specific entries by examining their URIs +// Remove specific entries by examining their URIs. var iterator = cache.uris(); while (iterator.hasNext()) { var uri = iterator.next(); @@ -239,7 +250,7 @@ while (iterator.hasNext()) { } } -// Remove all entries +// Remove all entries. cache.clear(); // Dispose of the cache by deleting its entries then closing it in an atomic fashion. @@ -252,7 +263,7 @@ cache.dispose(); Cache operation typically involves 3 scenarios. - * **Cache Hit**: The blessed scenario; everything was entirely served from cache and no network was +* **Cache Hit**: The desired scenario; everything was entirely served from cache and no network was used. * **Conditional Cache Hit**: The cache had to contact the origin to revalidate its copy of the response and the server decided it was valid. The cache uses server's response to update some @@ -269,7 +280,7 @@ can use to know which of the previous scenarios was the case. ```java var cache = HttpCache.newBuilder() - .cacheOnDisk(Path.of("my-cache-dir"), 100 * 1024 * 1024) + .cacheOnDisk(Path.of(".cache"), 500 * 1024 * 1024) .build(); var client = Methanol.newBuilder() .cache(cache) @@ -320,7 +331,7 @@ correspond to a specific `URI`. ```java var cache = HttpCache.newBuilder() - .cacheOnDisk(Path.of("my-cache-dir"), 100 * 1024 * 1024) + .cacheOnDisk(Path.of(".cache"), 500 * 1024 * 1024) .build(); var stats = cache.stats(); @@ -331,9 +342,9 @@ correspond to a specific `URI`. === "URI-specific Stats" ```java - // Per URI statistics aren't recorder by default + // Per URI statistics aren't recoded by default var cache = HttpCache.newBuilder() - .cacheOnDisk(Path.of("my-cache-dir"), 100 * 1024 * 1024) + .cacheOnDisk(Path.of(".cache"), 500 * 1024 * 1024) .statsRecorder(StatsRecorder.createConcurrentPerUriRecorder()) .build(); @@ -349,9 +360,11 @@ See [`HttpCache.Stats`][httpcache-stats] for all recorded statistics. * The cache only stores responses to GETs. This is typical for most caches. * The cache never stores [partial responses][partial-content-mdn]. * Only the most recent response [variant][vary-mdn] can be stored. -* The cache doesn't store responses that have a `Vary` header with any of the values: `Cookie`, - `Cookie2`, `Authorization`, `Proxy-Authroization`. That's because the `HttpClient` can implicitly - add these to requests, so Methanol won't be able to access their values to match responses against. +* The cache doesn't store responses that have a `Vary` header with any of the values: `Cookie`, + `Cookie2`, `Authorization`, `Proxy-Authroization`. The first two if the client has a configured + `CookieHandler`, the latter two if the client has a configured `Authentciator`. That's because + `HttpClient` can implicitly add these to requests, so Methanol won't be able to access their + values to match requests against. [rfc7234]: https://tools.ietf.org/html/rfc7234 [range-requests-mdn]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests diff --git a/docs/decompression.md b/docs/decompression.md index caa570efe..e942ba2ab 100644 --- a/docs/decompression.md +++ b/docs/decompression.md @@ -1,82 +1,63 @@ # Response Decompression -One caveat concerning Java's HTTP client is the lack of support for automatic response -decompression. A workaround is to use an available `InputStream` decompressor (e.g. `GZIPInputStream`) -that matches response's `Content-Encoding`. However, such approach is invasive as it forces us to deal -with `InputStreams`. +One caveat concerning Java's HTTP client is the lack of support for automatic response decompression. +A workaround is to use an available `InputStream` decompressor (e.g. `GZIPInputStream`) that matches response's `Content-Encoding`. +However, such approach is invasive as it forces you to deal with `InputStreams`. -The straightforward and recommended solution is to use -Methanol's [enhanced HTTP client](methanol_httpclient), -which gives you transparent response decompression for `gzip` & `deflate` out of the box. +The straightforward and recommended solution is to use [Methanol's HTTP client](methanol_httpclient.md), which gives you transparent response decompression for `gzip` & `deflate` out of the box. ```java final Methanol client = Methanol.create(); HttpResponse get(String url, BodyHandler bodyHandler) throws IOException, InterruptedException { - // No need to worry about adding Accept-Encoding and - // decompressing the response. The client does that for you! + // No need to worry about adding Accept-Encoding and decompressing the response, the client does that for you! return client.send(MutableRequest.GET(url), bodyHandler); } ``` -Read on if you're interested in knowing how that's accomplished or you want to extend decompression -support. +Read on if you're interested in knowing how that's accomplished, or you want to extend decompression support. ## Decoding BodyHandler The entry point to response body decompression is [`MoreBodyHandlers::decoding`][morebodyhandlers_decoding_javadoc]. -This method takes your desired `BodyHandler` and gives you one that decompresses the response body as -your handler's `BodySubscriber` receives it. +This method takes your desired `BodyHandler` and gives you one that decompresses the response body as your handler's `BodySubscriber` receives it. ```java var response = client.send(request, MoreBodyHandlers.decoding(BodyHandlers.ofString())); ``` - - -Note that it doesn't matter which `BodyHandler` you're using; you can have whatever response body -type you want. +It doesn't matter which `BodyHandler` you're using; you can have whatever response body type you want. ## BodyDecoder A [`BodyDecoder`][bodydecoder_javadoc] is a `BodySubscriber` with the added semantics of a `Flow.Processor`. -It intercepts the flow of bytes on its way down from the HTTP client, decoding each `List` -individually. The decoded bytes are forwarded to a downstream `BodySubscriber`, which converts them into the desired -response body. +It intercepts the flow of bytes on its way down from the HTTP client, decoding each `List` individually. +The decoded bytes are forwarded to a downstream `BodySubscriber`, which converts them into the desired response body. -A `BodyDecoder.Factory` associates itself with a defined encoding that's suitable as a `Content-Encoding` -directive. Given a downstream `BodySubscriber`, the factory creates a `BodyDecoder` that forwards the -response body after decoding it using the factory's encoding. For instance, a factory associated with -`gzip` creates decoders that decompress the response using the [gzip format][gzip-rfc]. +### BodyDecoder.Factory -### Factory Lookup +A `BodyDecoder.Factory` associates itself with a defined encoding that's suitable as a `Content-Encoding` directive. +It creates `BodyDecoder` instances that forward the decompressed response body to a downstream `BodySubscriber`. -Factories are installed as service-providers in the manner specified by Java's `ServiceLoader`. A -decoding `BodyHandler` looks up a factory associated with response's `Content-Encoding`. If found, -it's called to wrap user's `BodySubscriber`, so it receives the decompressed body. Otherwise, an -`UnsupportedOperationException` is thrown. +Factories are installed as service-providers in the manner specified by Java's `ServiceLoader`. +The handler returned by `MoreBodyHandlers::decoding` looks up a factory matching the response's `Content-Encoding` to wrap user's `BodySubscriber`. +If no such factory is found, an `UnsupportedOperationException` is thrown. ## Supported Encodings -The core module has support for `gzip` & `deflate` out of the box. There's also a separate -[module][methanol-brotli] providing support for [brotli]. +The core module has support for `gzip` & `deflate` out of the box. There's also a separate [module][methanol-brotli] for [brotli]. ## Extending decompression support -Adding support for more encodings or overriding supported ones is a matter of writing a `BodyDecoder` -implementation and providing a corresponding factory. However, implementing the decoder's `Flow.Publisher` -semantics can be tricky. Instead, implement an `AsyncDecoder` and wrap it in an `AsyncBodyDecoder`, so -you're only concerned with your decompression logic. +Adding support for more encodings or overriding supported ones is a matter of writing a `BodyDecoder` implementation and providing a corresponding factory. +However, implementing the decoder's `Flow.Publisher` semantics can be tricky. Instead, implement an `AsyncDecoder` and wrap it in an `AsyncBodyDecoder`, so +you're only concerned with the decompression logic. ### Writing an AsyncDecoder -Decoding is done as a number of `decode(source, sink)` rounds finalized by one final round, each -with the currently available input. After the final round, your `AsyncDecoder` must've completely -exhausted its source. Here's a decoder implementation that uses [jzlib] for `gzip` & `deflate` -decompression. +Decoding is done as a number of `decode(source, sink)` rounds finalized by one final round, each with the currently available input. +After the final round, your `AsyncDecoder` must've completely exhausted its source. +Here's a decoder implementation that uses [jzlib] for `gzip` & `deflate` decompression. ```java class JZlibDecoder implements AsyncDecoder { @@ -183,8 +164,8 @@ public static final class MyDecoderFactory implements BodyDecoder.Factory { } ``` -The next step is to declare your factory as a service-provider. For instance, here's an appropriate -`provides...with` declaration to put in `module-info.java` if your application uses Java modules. +The next step is to declare your factory as a service-provider. If your application uses Java modules, +you'd have a declaration like the following in your `module-info.java`. ```java module my.module { @@ -195,8 +176,13 @@ module my.module { ``` [gzip-rfc]: https://tools.ietf.org/html/rfc1952 + [methanol-brotli]: https://github.com/mizosoft/methanol/tree/master/methanol-brotli + [brotli]: https://github.com/google/brotli + [jzlib]: https://www.jcraft.com/jzlib/ + [morebodyhandlers_decoding_javadoc]: https://mizosoft.github.io/methanol/api/latest/methanol/com/github/mizosoft/methanol/MoreBodyHandlers.html#decoding(java.net.http.HttpResponse.BodyHandler) + [bodydecoder_javadoc]: https://mizosoft.github.io/methanol/api/latest/methanol/com/github/mizosoft/methanol/BodyDecoder.html diff --git a/docs/index.md b/docs/index.md deleted file mode 100644 index d90e23df1..000000000 --- a/docs/index.md +++ /dev/null @@ -1,42 +0,0 @@ -# Methanol - -[![CI status](https://img.shields.io/github/actions/workflow/status/mizosoft/methanol/build.yml?branch=master&logo=github&style=flat-square)](https://github.com/mizosoft/methanol/actions) -[![Coverage Status](https://img.shields.io/coveralls/github/mizosoft/methanol?style=flat-square)](https://coveralls.io/github/mizosoft/methanol?branch=master) -[![Maven Central](https://img.shields.io/maven-central/v/com.github.mizosoft.methanol/methanol?style=flat-square)](https://search.maven.org/search?q=g:%22com.github.mizosoft.methanol%22%20AND%20a:%22methanol%22) -[![Javadoc](https://img.shields.io/maven-central/v/com.github.mizosoft.methanol/methanol?color=blueviolet&label=Javadoc&style=flat-square)](https://mizosoft.github.io/methanol/api/latest/) - -Java enjoys a neat, built-in [HTTP client](https://openjdk.java.net/groups/net/httpclient/intro.html). -However, it lacks key HTTP features like multipart uploads, caching and response decompression. -***Methanol*** comes in to fill these gaps. The library comprises a set of lightweight, yet powerful -extensions aimed at making it much easier & more productive to work with `java.net.http`. You can -say it's an `HttpClient` wrapper, but you'll see it almost seamlessly integrates with the standard -API you might already know. - -Methanol isn't invasive. The core library has zero runtime dependencies. However, special attention -is given to object mapping, so integration with libraries like Jackson or Gson becomes a breeze. - -## Installation - -### Gradle - -```gradle -implementation 'com.github.mizosoft.methanol:methanol:1.7.0' -``` - -### Maven - -```xml - - com.github.mizosoft.methanol - methanol - 1.7.0 - -``` - -## Contributing - -See [CONTRIBUTING](CONTRIBUTING.md) - -## License - -[MIT](https://opensource.org/licenses/MIT) diff --git a/docs/interceptors.md b/docs/interceptors.md index 31c0ac154..160be7504 100644 --- a/docs/interceptors.md +++ b/docs/interceptors.md @@ -6,9 +6,8 @@ and to responses in their way back. ## Writing Interceptors -Interceptors sit between a `Methanol` client and its underlying `HttpClient`, referred to as its -backend. When registered, an interceptor is invoked each `send` or `sendAsync` -call. Here's an interceptor that logs requests and their responses. +Interceptors sit between a `Methanol` client and its underlying `HttpClient`, referred to as its backend. +When registered, an interceptor is invoked each `send` or `sendAsync` call. Here's an interceptor that logs requests and their responses. ```java public final class LoggingInterceptor implements Methanol.Interceptor { @@ -56,11 +55,9 @@ public final class LoggingInterceptor implements Methanol.Interceptor { } ``` -`HttpClient` has blocking and asynchronous APIs, so interceptors must implement two methods -matching each. An interceptor is given a `Chain` so it can forward requests to -its next sibling, or to the backend in case there's no more interceptors in the chain. The backend is -where requests finally get sent. Typically, an interceptor calls its chain's `forward` or `forwardAsync` -after it has done its job. +`HttpClient` has blocking and asynchronous APIs, so interceptors must implement two methods matching each. +An interceptor is given a `Chain` so it can forward requests to its next sibling, or to the backend in case there are no more interceptors in the chain. +The backend is where requests finally get sent. Typically, an interceptor calls its chain's `forward` or `forwardAsync` after it has done its job. If your interceptor only modifies requests, prefer passing a lambda to `Interceptor::create`. @@ -74,25 +71,20 @@ var expectContinueInterceptor = Interceptor.create(request -> ## Intercepting Bodies -A powerful property of interceptors is their control over how responses are received by their -caller. An interceptor can transform its chain's `BodyHandler` using -`Chain::withBodyHandler` before it forwards requests. A transformed `BodyHandler` typically applies the handler the chain previously -had, then wraps the resulted `BodySubscriber`, so it intercepts the response body as it's being received -by the caller. This is how `Methanol` does transparent decompression & cache writes. +A powerful property of interceptors is their control over how responses are received by their caller. +An interceptor can transform its chain's `BodyHandler` before it forwards requests. +A transformed `BodyHandler` typically applies the handler the chain previously had, then wraps the resulted `BodySubscriber`, so it intercepts the response body as it's being received by the caller. +This is how `Methanol` does transparent decompression & cache writes. -Note that this applies to requests too. You can transform a request body by wrapping its -`BodyPublisher`, if it's got any. `BodyPublisher` & `BodySubscriber` APIs can be nicely layered to -apply different transformations. - - +Note that this applies to requests too. You can transform a request body by wrapping its `BodyPublisher`. +`BodyPublisher` & `BodySubscriber` APIs can be nicely layered to apply different transformations. ## Invocation Order An interceptor can be either a *client* or a *backend* interceptor. Client interceptors sit between -the application and `Methanol`'s internal interceptors. They are called as soon as the client -receives a request. Backend interceptors sit between `Methanol` and its backend `HttpClient`. They -get invoked right before the request gets sent. This has a number -of implications. +the application and `Methanol`'s internal interceptors. They are called as soon as the client receives a request. +Backend interceptors sit between `Methanol` and its backend `HttpClient`. They get invoked right before the request gets sent. +This has a number of implications. ### Client Interceptors @@ -102,9 +94,8 @@ of implications. ### Backend Interceptors -* Observe the request after the client applies things like the base URI and default - headers. Additionally, they see - intermediate headers added by the client or the cache like `Accept-Encoding` & `If-None-Math`. +* Observe the request after the client applies things like the base URI and default headers. + Additionally, they see intermediate headers added by the client or the cache like `Accept-Encoding` & `If-None-Math`. * Receive the response body just as transmitted by the backend. For instance, a transformed `BodyHandler` receives a compressed body if the response comes with a `Content-Encoding` header. * May not always be invoked. This is the case when a cache decides it doesn't need network and hence @@ -117,8 +108,8 @@ of implications. ## Registration -You can register client and backend interceptors with `interceptor(...)` and `backendInterceptor(...)` -respectively. Interceptors in each group get invoked in registration order. +You can register client and backend interceptors with `interceptor(...)` and `backendInterceptor(...)` respectively. +Interceptors in each group get invoked in registration order. === "Client Interceptors" @@ -139,11 +130,10 @@ respectively. Interceptors in each group get invoked in registration order. ## Short-circuiting Both client & backend interceptors can refrain from forwarding a request. They're allowed to -short-circuit a request's path by returning a fabricated response. This makes them good candidates -for testing. You can mock responses with client interceptors to investigate requests just -as sent by your application. Moreover, responses can be mocked with backend interceptors to explore -requests as they get sent. This makes backend interceptors suitable for testing how your application -interacts with the cache. +short-circuit a request's path by returning a fabricated response. This makes them good candidates for testing. +You can mock responses with client interceptors to investigate requests just as sent by your application. +Moreover, responses can be mocked with backend interceptors to explore requests as they get sent. +This makes backend interceptors suitable for testing how your application interacts with the cache. ## Limitations diff --git a/docs/interruptible_reading.md b/docs/interruptible_reading.md deleted file mode 100644 index ed4b90c45..000000000 --- a/docs/interruptible_reading.md +++ /dev/null @@ -1,46 +0,0 @@ -# Interruptible Reading - -Reading from blocking sources like `InputStream` isn't always avoidable. Once they're needed, JDK's -`BodyHandlers::ofInputStream` can be used. However, reading -from such stream blocks your threads indefinitely, which causes troubles when you want to close the -application or change contexts amid reading. Methanol has support for [interruptible channels][interruptible-channel-jdk]. -These are asynchronously closeable and respond to thread interrupts. Using them, you can voluntarily -halt reading operations when they're not relevant anymore. - -`MoreBodySubscibers` has interruptible `ReadableByteChannel` and `Reader` implementations. Use JDK's -`Channels::newInputStream` to get an `InputStream` from an interruptible `ReadableByteChannel` when -input streams is what you need. - -## Example - Interruptible Body Processing - -Here's an example of a hypothetical component that processes the response from a `ReadableByteChannel`. -When the task is to be discarded, reader threads are interrupted by shutting down the owning `ExecutorService`. -This closes open channels and instructs them to halt blocking reads. - -```java -class BodyProcessor { - final ExecutorService executorService = Executors.newCachedThreadPool(); - final Methanol client = Methanol.create(); - - CompletableFuture processAsync(HttpRequest request, Consumer processAction) { - return client.sendAsync(request, MoreBodyHandlers.ofByteChannel()) - .thenApplyAsync(res -> { - var buffer = ByteBuffer.allocate(8 * 1024); - try (var channel = res.body()) { - while (channel.read(buffer.clear()) >= 0) { - processAction.accept(buffer.flip()); - } - } catch (ClosedByInterruptException | ClosedChannelException ignored) { - // The thread was interrupted due to ExecutorService shutdown - } catch (IOException e) { - throw new UncheckedIOException(e); - } - return null; - }, executorService); - } - - void terminate() { executorService.shutdownNow(); } -} -``` - -[interruptible-channel-jdk]: https://docs.oracle.com/en/java/javase/16/docs/api/java.base/java/nio/channels/InterruptibleChannel.html diff --git a/docs/legacy_adapters.md b/docs/legacy_adapters.md new file mode 100644 index 000000000..88af880fe --- /dev/null +++ b/docs/legacy_adapters.md @@ -0,0 +1,190 @@ +# Legacy Adapters + +Before version 1.8.0, [adapters](adapters.md) were required to be registered as service providers and used through +static methods, instead of being bundled in a per-client [`AdapterCodec`][adaptercodec_javadoc]. The latter is the +recommended way to use adapters. The legacy way is not deprecated but might be in the future. + +## Compatibility + +If you have adapters installed as service providers, they can still be used with the newer, non-static APIs. + +```java +record Person(String name) {} + +// The client will fall back to an AdapterCodec of installed adapters, if any. +var client = Methanol.create(); +var response = client.send( + MutableRequest.POST(".../echo", new Person("Bruce Lee"), MediaType.APPLICATION_JSON), + Person.class); +assertThat(response.body()).isEqualTo(new Person("Bruce Lee")); +``` + +## Installation + +The legacy way to register adapters is through [service providers][serviceloader_javadoc]. +How this is done depends on your project setup. We'll use `methanol-jackson` as an example. + +### Module Path + +Follow these steps if your project uses the Java module system. + +1. Add this class to your module: + + ```java + public class JacksonJsonProviders { + private static final ObjectMapper mapper = new ObjectMapper(); + + public static class EncoderProvider { + public static BodyAdapter.Encoder provider() { + return JacksonAdapterFactory.createJsonEncoder(mapper); + } + } + + public static class DecoderProvider { + public static BodyAdapter.Decoder provider() { + return JacksonAdapterFactory.createJsonDecoder(mapper); + } + } + } + ``` + +2. Add the corresponding provider declarations in your `module-info.java` file. + + ```java + requires methanol.adapter.jackson; + + provides BodyAdapter.Encoder with JacksonJsonProviders.EncoderProvider; + provides BodyAdapter.Decoder with JacksonJsonProviders.DecoderProvider; + ``` + +### Classpath + +Registering adapters from the classpath requires declaring the implementation classes in provider-configuration +files that are bundled with your JAR. You'll first need to implement delegating `Encoder` & `Decoder` +that forward to the instances created by `JacksonAdapterFactory`. Extending from `ForwardingEncoder` & +`ForwardingDecoder` makes this easier. + +You can use Google's [AutoService][autoservice] to generate the provider-configuration files automatically. + +#### Using AutoService + +First, [install AutoService][autoservice_getting_started]. + +##### Gradle + +```gradle +implementation "com.google.auto.service:auto-service-annotations:$autoServiceVersion" +annotationProcessor "com.google.auto.service:auto-service:$autoServiceVersion" +``` + +##### Maven + +```xml + + com.google.auto.service + auto-service-annotations + ${autoServiceVersion} + +``` + +Configure the annotation processor with the compiler plugin. + +```xml Person(String name) { +} + + maven-compiler-plugin + + + + com.google.auto.service + auto-service + ${autoServiceVersion} + + + + +``` + +Next, add this class to your project: + +```java +public class JacksonJsonAdapters { + private static final ObjectMapper mapper = new ObjectMapper(); + + @AutoService(BodyAdapter.Encoder.class) + public static class Encoder extends ForwardingEncoder { + public Encoder() { + super(JacksonAdapterFactory.createJsonEncoder(mapper)); + } + } + + @AutoService(BodyAdapter.Decoder.class) + public static class Decoder extends ForwardingDecoder { + public Decoder() { + super(JacksonAdapterFactory.createJsonDecoder(mapper)); + } + } +} +``` + +#### Manual Configuration + +You can also write the configuration files manually. First, add this class to your project: + +```java +public class JacksonJsonAdapters { + private static final ObjectMapper mapper = new ObjectMapper(); + + public static class Encoder extends ForwardingEncoder { + public Encoder() { + super(JacksonAdapterFactory.createJsonEncoder(mapper)); + } + } + + public static class Decoder extends ForwardingDecoder { + public Decoder() { + super(JacksonAdapterFactory.createJsonDecoder(mapper)); + } + } +} +``` + +Next, create two provider-configuration files in the resource directory: `META-INF/services`, +one for Person(String name) { +}the encoder and the other for the decoder. Each file must contain the fully qualified +name of the implementation class. + +Let's say the above class is in a package named `com.example`. You'll want to have one file for the +encoder named: + +``` +META-INF/services/com.github.mizosoft.methanol.BodyAdapter$Encoder +``` + +and contains the following line: + +``` +com.example.JacksonJsonAdapters$Encoder +``` + +Similarly, the decoder's file is named: + +``` +META-INF/services/com.github.mizosoft.methanol.BodyAdapter$Decoder +``` + +and contains: + +``` +com.example.JacksonJsonAdapters$Decoder +``` + +[jackson]: https://github.com/FasterXML/jackson + +[autoservice]: https://github.com/google/auto/tree/master/service + +[autoservice_getting_started]: https://github.com/google/auto/tree/master/service#getting-started + +[serviceloader_javadoc]: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/ServiceLoader.html + +[adaptercodec_javadoc]: https://mizosoft.github.io/methanol/api/latest/methanol/com/github/mizosoft/methanol/AdapterCodec.html diff --git a/docs/media_types.md b/docs/media_types.md index 36cd761f5..75610d552 100644 --- a/docs/media_types.md +++ b/docs/media_types.md @@ -1,9 +1,9 @@ # Mime -[Media types][mime-types-mdn] are the web's notion for file extensions. They're typically present in -requests and responses as `Content-Type` directives. Methanol's `MediaType` greatly facilitates the +[Media types][mime-types-mdn] are the web's notion for file extensions. They're present in +requests and responses as `Content-Type` directives. Methanol's `MediaType` facilitates the representation and manipulation of media types. - + ## MediaType You can create a `MediaType` from its individual components or parse one from a `Content-Type` string. @@ -32,8 +32,7 @@ You can create a `MediaType` from its individual components or parse one from a ### Media Ranges -A `MediaType` also defines a [media range][media-ranges-rfc] to which one or more media types belong, -including itself. +A `MediaType` also defines a [media range][media-ranges-rfc] to which one or more media types belong, including itself. ```java var anyTextType = MediaType.parse("text/*"); @@ -59,13 +58,11 @@ assertTrue(textHtml.isCompatibleWith(anyTextType)); ## MimeBodyPublisher -`MimeBodyPublisher` is a mixin-style interface that associates a `MediaType` with a `BodyPublisher`. -It's recognized by `Methanol` and [multipart bodies](multipart_and_forms.md#multipart-bodies) in that -it gets the appropriate `Content-Type` header implicitly added. +`MimeBodyPublisher` is a mixin-style interface that associates a `MediaType` with a `BodyPublisher`. +It's recognized by `Methanol` and [multipart bodies](multipart_and_forms.md#multipart-bodies) in that it gets the appropriate `Content-Type` header implicitly added. -You can adapt an arbitrary `BodyPublisher` into a `MimeBodyPublisher`. Here's a factory method that -creates `MimeBodyPublihers` for files. The file's media type is probed from the system, falling -back to `application/octet-stream` if that doesn't work. +You can adapt an arbitrary `BodyPublisher` into a `MimeBodyPublisher`. Here's a factory method that creates `MimeBodyPublihers` for files. +The file's media type is probed from the system, falling back to `application/octet-stream` if that doesn't work. ```java static MimeBodyPublisher ofMimeFile(Path file) throws FileNotFoundException { @@ -87,7 +84,6 @@ final Methanol client = Methanol.create(); HttpResponse post(String url, Path file, BodyHandler handler) throws IOException, InterruptedException { - // Request's Content-Type is implicitly added return client.send(MutableRequest.POST(url, ofMimeFile(file)), handler); } diff --git a/docs/methanol_httpclient.md b/docs/methanol_httpclient.md index e7750ba25..4f48420cf 100644 --- a/docs/methanol_httpclient.md +++ b/docs/methanol_httpclient.md @@ -1,77 +1,34 @@ # Enhanced HttpClient -Methanol has a special `HttpClient` that extends the standard one with interesting new features. +Methanol has a special `HttpClient` that extends the standard one with interesting features. Unsurprisingly, the client is named `Methanol`. ## Usage -In addition to [interceptors] and [caching], `Methanol` can apply default properties to your -requests. +In addition to [interceptors] and [caching], `Methanol` can apply default properties to your requests. Think resolving with a base URI, adding default request headers, default timeouts, etc. ```java var builder = Methanol.newBuilder() - .cache(...) - . - -interceptor(...) - . - -userAgent("Will Smith") // Custom User-Agent - . - -baseUri("https://api.github.com") // Base URI to resolve requests' URI against - . - -defaultHeader("Accept","application/json") // Default request headers - . - -requestTimeout(Duration.ofSeconds(20)) // Default request timeout - . - -headersTimeout(Duration.ofSeconds(5)) // Timeout for receiving response headers - . - -readTimeout(Duration.ofSeconds(5)) // Timeout for single reads - . - -autoAcceptEncoding(true); // Transparent response compression, this is true by default - -// Continue using as a standard HttpClient.Builder! -var client = builder.executor(...) - . - -executor(Executors.newFixedThreadPool(16)) - . - -connectTimeout(Duration.ofSeconds(30)) - ... - . - -build(); + .userAgent("Will Smith") // Custom User-Agent + .baseUri("https://api.github.com") // Base URI to resolve requests' URI against + .defaultHeader("Accept","application/json") // Default request headers + .requestTimeout(Duration.ofSeconds(20)) // Default request timeout + .headersTimeout(Duration.ofSeconds(5)) // Timeout for receiving response headers + .readTimeout(Duration.ofSeconds(5)) // Timeout for single reads + .autoAcceptEncoding(true); // Transparent response compression, this is true by default + +// Continue using as a standard HttpClient.Builder. +var client = builder.connectTimeout(Duration.ofSeconds(30)).build(); ``` -You can also build from an existing `HttpClient` instance. However, you can't install an `HttpCache` -in such case. +You can also build from an existing `HttpClient` instance. However, you can't install an `HttpCache` in such case. ```java -HttpClient prebuiltClient = ... -var client = Methanol.newBuilder(prebuiltClient) - .interceptor(...) - . - -userAgent("Will Smith") - ... - . - -build(); - +var prebuiltClient = HttpClient.newHttpClient(); +var client = Methanol.newBuilder(prebuiltClient).build(); ``` -!!! tip -`Methanol` is an `HttpClient`. It implements the same API like `send` & `sendAsync`, which you can -continue using as usual. - !!! note Default properties don't override those the request already has. For instance, a client with a default `Accept: text/html` will not override a request's `Accept: application/json`. @@ -80,15 +37,13 @@ default `Accept: text/html` will not override a request's `Accept: application/j If `autoAcceptEncoding` is enabled, the client complements requests with an `Accept-Encoding` header which accepts all supported encodings (i.e. available [`BodyDecoder`](decompression.md) providers). -Additionally, -the response is transparently decompressed according to its `Content-Encoding`. +Additionally, the response is transparently decompressed. Since `deflate` & `gzip` are supported out of the box, they're always included in `Accept-Encoding`. For instance, if [brotli][methanol-brotli] is installed, requests will typically have: -`Accept-Encoding: deflate, gzip, br`. +`Accept-Encoding: gzip, deflate, br`. If you want specific encodings to be applied, add `Accept-Encoding` as a default header or -explicitly -set one in your request. +explicitly set one in your request. === "Default Header" @@ -109,11 +64,9 @@ set one in your request. ### MimeBodyPublisher -`Methanol` automatically sets a request's `Content-Type` if it has a [ -`MimeBodyPublisher`](media_types.md#mimebodypublisher). +`Methanol` automatically sets a request's `Content-Type` if it has a [`MimeBodyPublisher`](media_types.md#mimebodypublisher). If the request already has a `Content-Type`, it's overwritten. This makes sense as a body knows its -media type -better than a containing request mistakenly setting a different one. +media type better than the containing request. ### Reactive Dispatching @@ -151,19 +104,19 @@ If you like reactive streams, use `Methanol::exchange`, which is like `sendAsync ## MutableRequest -`MutableRequest` is an `HttpRequest` that implements `HttpRequest.Builder` for settings request's -properties. This drops immutability in favor of some convenience when the request is sent -immediately. +`MutableRequest` is an `HttpRequest` with additional properties. It implements `HttpRequest.Builder` for settings request's fields. ```java -var response = client.send(MutableReqeust.GET(uri), BodyHandlers.ofString()); +var response = client.send( + MutableReqeust.GET(uri).header("Accept", "application/json"), + BodyHandlers.ofString()); ``` -Additionally, `MutableRequest` accepts relative URIs (standard `HttpRequest.Builder` doesn't). This -is a complementing feature to `Methanol`'s base URIs, against which relative ones are resolved. +`MutableRequest` accepts relative URIs (standard `HttpRequest.Builder` doesn't). +This complements `Methanol`'s base URIs, against which relative ones are resolved. !!! tip -You can use `MutableRequest::toImmutableRequest` to get an immutable `HttpRequest` snapshot. +Use `MutableRequest::toImmutableRequest` to get an immutable `HttpRequest` snapshot. [interceptors]: interceptors.md diff --git a/docs/multipart_and_forms.md b/docs/multipart_and_forms.md index 9b9961ac3..2b2e0edc7 100644 --- a/docs/multipart_and_forms.md +++ b/docs/multipart_and_forms.md @@ -4,15 +4,15 @@ Methanol has special `BodyPublisher` implementations for multipart uploads & for ## Multipart Bodies -`MultipartBodyPublisher` implements the flexible multipart format. A multipart body has one or more -parts. Each part has a `BodyPublisher` for its content and `HttpHeaders` that describe it. -`MultipartBodyPublisher.Builder`defaults to `multipart/form-data` if a multipart `MediaType` isn't -explicitly specified. There're special methods for adding parts with a `Content-Disposition: form-data` -header generated from a field name and an optional file name. These are referred to as form parts. +`MultipartBodyPublisher` implements the multipart format. A multipart body has one or more parts. +Each part has a `BodyPublisher` for its content and `HttpHeaders` that describe it. +`MultipartBodyPublisher.Builder`defaults to `multipart/form-data` if a multipart `MediaType` isn't explicitly specified. +There are special methods for adding parts with a `Content-Disposition: form-data` header generated from a field name and an optional file name. +These are referred to as form parts. ```java // Substitute with your client ID. Visit https://api.imgur.com/oauth2/addclient to get one. -static final String CLIENT_ID = System.getenv("IMGUR_CLIENT_ID"); +static final String CLIENT_ID = System.getenv("imgur.client.id"); final Methanol client = Methanol.create(); @@ -21,10 +21,10 @@ HttpResponse uploadGif() throws IOException, InterruptedException { .textPart("title", "Dancing stick bug") .filePart("image", Path.of("dancing-stick-bug.gif"), MediaType.IMAGE_GIF) .build(); - var request = MutableRequest.POST("https://api.imgur.com/3/image", multipartBody) - .header("Authorization", "Client-ID " + CLIENT_ID); - - return client.send(request, BodyHandlers.ofString()); + return client.send( + MutableRequest.POST("https://api.imgur.com/3/image", multipartBody) + .header("Authorization", "Client-ID " + CLIENT_ID), + BodyHandlers.ofString()); } ``` @@ -36,43 +36,34 @@ back to `application/octet-stream` if that doesn't work. ### Generic Form Parts -Use builder's `formPart` method to add a form part from an arbitrary `BodyPublisher`. It takes a field -name and an optional file name. +Use builder's `formPart` to add a form part from an arbitrary `BodyPublisher`. It takes a field name and an optional file name. ```java -// Substitute with your client ID. Visit https://api.imgur.com/oauth2/addclient to get one -static final String CLIENT_ID = System.getenv("IMGUR_CLIENT_ID"); +// Substitute with your client ID. Visit https://api.imgur.com/oauth2/addclient to get one. +static final String CLIENT_ID = System.getenv("imgur.client.id"); final Methanol client = Methanol.create(); -HttpResponse uploadPng(String title, InputStream pngImageInputStream) - throws IOException, InterruptedException { - var imagePart = MoreBodyPublishers.ofMediaType( - BodyPublishers.ofInputStream(() -> pngImageInputStream), MediaType.IMAGE_PNG); +HttpResponse uploadGif() throws IOException, InterruptedException { var multipartBody = MultipartBodyPublisher.newBuilder() - .textPart("title", title) - .formPart( - "image", title + ".png", MoreBodyPublishers.ofMediaType(imagePart, MediaType.IMAGE_PNG)) - .build(); - var request = MutableRequest.POST("https://api.imgur.com/3/image", multipartBody) - .header("Authorization", "Client-ID " + CLIENT_ID); - - return client.send(request, BodyHandlers.ofString()); + .textPart("title", "Dancing stick bug") + .formPart( + "image", title + ".png", MoreBodyPublishers.ofMediaType(imagePart, MediaType.IMAGE_PNG)) + .build(); + return client.send( + MutableRequest.POST("https://api.imgur.com/3/image", multipartBody) + .header("Authorization", "Client-ID " + CLIENT_ID), + BodyHandlers.ofString()); } ``` -!!! tip - You can use `formPart` to add a file part from something that's not a `Path` (e.g. `InputStream`) or - to override the part's `filename` property, which is not possible with `filePart`. - !!! tip Use `MoreBodyPublishers::ofMediaType` to pair an arbitrary `BodyPublisher` with its proper `MediaType` if you want a `Content-Type` header to be specified by the part. ## Form Bodies -Use `FormBodyPublisher` to send form data as a set of URL-encoded queries. Data is added as string -name-value pairs. +Use `FormBodyPublisher` to send form data as a set of URL-encoded queries. Data is added as string name-value pairs. ```java final Methanol client = Methanol.create(); @@ -81,11 +72,7 @@ HttpResponse sendQueries(String url, Map queries) throws IOException, InterruptedException { var builder = FormBodyPublisher.newBuilder(); queries.forEach(builder::query); - - var formBody = builder.build(); - var request = MutableRequest.POST(url, formBody); - - return client.send(request, BodyHandlers.ofString()); + return client.send(MutableRequest.POST(url, builder.build()), BodyHandlers.ofString()); } ``` diff --git a/docs/object_mapping.md b/docs/object_mapping.md deleted file mode 100644 index e249ee41a..000000000 --- a/docs/object_mapping.md +++ /dev/null @@ -1,271 +0,0 @@ -# Object Mapping - -HTTP bodies are often mappable to high-level entities that your code understands. Java's HttpClient -was designed with that in mind. However, available `BodyPublisher` & `BodySubscriber` implementations -are too basic, and implementing your own can be tricky. Methanol builds upon these APIs with an extensible -and easy-to-use object mapping mechanism that treats your objects as first-citizen HTTP bodies. - -## Setup - -Before sending and receiving objects over HTTP, Methanol needs to adapt to your desired mapping schemes. -Adapters for the most popular serialization libraries are provided in separate modules. - - * [`methanol-gson`](adapters/gson.md): JSON with Gson - * [`methanol-jackson`](adapters/jackson.md): JSON with Jackson (but also XML, protocol buffers and other formats support by Jackson) - * [`methanol-jackson-flux`](adapters/jackson_flux.md): Reactive JSON with Jackson and Reactor - * [`methanol-jaxb`](adapters/jaxb.md): XML with JAXB - * [`methanol-protobuf`](adapters/protobuf.md): Google's Protocol Buffers - -Adapters are dynamically located using Java's `ServiceLoader`. You can find clear installation steps -in each module. We'll see how to implement custom adapters as well. - -If you want to run examples presented here, get started by installing your favorite JSON adapter! - -## Receiving Objects - -To get an `HttpResponse`, give `MoreBodyHandlers` a `T.class` and it'll give you a `BodyHandler` -in return. - -```java hl_lines="8" -final Methanol client = Methanol.newBuilder() - .baseUri("https://api.github.com/") - .defaultHeader("Accept", "application/vnd.github.v3+json") - .build(); - -GitHubUser getUser(String username) throws IOException, InterruptedException { - var request = MutableRequest.GET("user/" + username); - var response = client.send(request, MoreBodyHandlers.ofObject(GitHubUser.class)); - - return response.body(); -} - -public static final class GitHubUser { - public String login; - public long id; - public String url; - - // Other fields omitted. - // Annotate with @JsonIgnoreProperties(ignoreUnknown = true) to run with Jackson. -} -``` - -If you want to get fancier with generics, use a `TypeRef`. - -```java hl_lines="9" -final Methanol client = Methanol.newBuilder() - .baseUri("https://api.github.com/") - .defaultHeader("Accept", "application/vnd.github.v3+json") - .build(); - -List getIssuesForRepo(String owner, String repo) throws IOException, InterruptedException { - var request = MutableRequest.GET("repos/" + owner + "/" + repo + "/issues"); - var response = client.send( - request, MoreBodyHandlers.ofObject(new TypeRef>() {})); - - return response.body(); -} - -public static final class GitHubIssue { - public String title; - public GitHubUser user; - public String body; - - // Other fields omitted. - // Annotate with @JsonIgnoreProperties(ignoreUnknown = true) to run with Jackson. -} - -public static final class GitHubUser { - public String login; - public long id; - public String url; - - // Other fields omitted. - // Annotate with @JsonIgnoreProperties(ignoreUnknown = true) to run with Jackson. -} -``` - -The right adapter is selected based on response's `Content-Type`. For instance, a response with -`Content-Type: application/json` causes Methanol to look for a JSON adapter. If such lookup -fails, an `UnsupportedOperationException` is thrown. - -## Sending Objects - -Get a `BodyPubilsher` for whatever object you've got by passing it in along with a `MediaType` describing -which adapter you prefer selected. - -```java hl_lines="7" -final Methanol client = Methanol.newBuilder() - .baseUri("https://api.github.com/") - .defaultHeader("Accept", "application/vnd.github.v3+json") - .build(); - -String renderMarkdown(RenderRequest renderRequest) throws IOException, InterruptedException { - var requestBody = MoreBodyPublishers.ofObject(renderRequest, MediaType.APPLICATION_JSON); - var request = MutableRequest.POST("markdown", requestBody); - var response = client.send(request, BodyHandlers.ofString()); - - return response.body(); -} - -public static final class RenderRequest { - public String text, mode, context; -} -``` - -## Adapters - -An adapter provides [`Encoder`][encoder_javadoc] and/or [`Decoder`][decoder_javadoc] implementations. -Both interfaces implement [`BodyAdapter`][bodyadapter_javadoc], which defines the methods necessary -for Methanol to know which object types the adapter believes it can handle, and in what scheme. An -`Encoder` creates a `BodyPublisher` that streams a given object's serialized form. Similarly, a `Decoder` -supplies `BodySubscriber` instances for a given `TypeRef` that convert the response body into `T`. -An optional `MediaType` is passed to encoders & decoders to further describe the desired mapping scheme -(e.g. specify a character set). - -### Example - An HTML Adapter - -Here's an adapter that uses [Jsoup][jsoup] to convert HTML bodies to parsed `Document` objects and -vise versa. When you're writing adapters, extend from `AbstractBodyAdapter` to get free media type -matching & other helpful functions. - -```java -public abstract class JsoupAdapter extends AbstractBodyAdapter { - JsoupAdapter() { - super(MediaType.TEXT_HTML); - } - - @Override - public boolean supportsType(TypeRef type) { - return type.rawType() == Document.class; - } - - public static final class Decoder extends JsoupAdapter implements BodyAdapter.Decoder { - @Override - public BodySubscriber toObject(TypeRef type, @Nullable MediaType mediaType) { - requireSupport(type); - requireCompatibleOrNull(mediaType); - - var charset = charsetOrUtf8(mediaType); - var subscriber = BodySubscribers.mapping(BodySubscribers.ofString(charset), Jsoup::parse); - return BodySubscribers.mapping(subscriber, type.exactRawType()::cast); // Safely cast Document to T - } - } - - public static final class Encoder extends JsoupAdapter implements BodyAdapter.Encoder { - @Override - public BodyPublisher toBody(Object object, @Nullable MediaType mediaType) { - requireSupport(object.getClass()); - requireCompatibleOrNull(mediaType); - - var charset = charsetOrUtf8(mediaType); - var publisher = BodyPublishers.ofString(((Document) object).outerHtml(), charset); - return attachMediaType(publisher, mediaType); - } - } -} -``` - -!!! tip - Make sure your encoders call `AbstractBodyAdapter::attachMediaType` so the created `BodyPublisher` - is converted to a `MimeBodyPublisher` if the given media type isn't null. That way, requests get - the correct `Content-Type` header added by `Methanol`. - -### Registration - -Declare your encoder & decoder implementations as service-providers in the manner specified by Java's -`ServiceLoader`. Here's the appropriate provider declarations for our Jsoup adapter to put in -`module-info.java`. - -```java -module my.module { - ... - - provides BodyAdapter.Decoder with JsoupAdapter.Decoder; - provides BodyAdapter.Encoder with JsoupAdapter.Encoder; -} -``` - -See any of the [supported adapters](#setup) for more registration methods. - -### Usage - -Now Methanol can send and receive HTML `Documents`! - -```java -final Methanol client = Methanol.create(); - -HttpResponse downloadHtml(String url) throws IOException, InterruptedException { - var request = MutableRequest.GET(url).header("Accept", "text/html"); - - return client.send(request, MoreBodyHandlers.ofObject(Document.class)); -} - - HttpResponse uploadHtml(String url, Document htmlDoc, BodyHandler responseHandler) - throws IOException, InterruptedException { - var requestBody = MoreBodyPublishers.ofObject(htmlDoc, MediaType.TEXT_HTML); - var request = MutableRequest.POST(url, requestBody); - - return client.send(request, responseHandler); -} -``` - -## Buffering vs Streaming - -`MoreBodyHandlers::ofObject` creates handlers that use `MoreBodySubscribers::ofObject` to obtain the -appropriate `BodySubscriber` from a chosen adapter. Such subscriber typically loads the whole response -into memory then decodes from there. If your responses tend to have large bodies, or you'd prefer the -memory efficiency afforded by streaming sources, `MoreBodyHandlers::ofDeferredObject` is the way to go. - -```java hl_lines="8" -final Methanol client = Methanol.newBuilder() - .baseUri("https://api.github.com/") - .defaultHeader("Accept", "application/vnd.github.v3+json") - .build(); - -GitHubUser getUser(String username) throws IOException, InterruptedException { - var request = MutableRequest.GET("user/" + username); - var response = client.send(request, MoreBodyHandlers.ofDeferredObject(GitHubUser.class)); - - return response.body().get(); -} - -public static final class GitHubUser { - public String login; - public long id; - public String url; - - // Other fields omitted. - // Annotate with @JsonIgnoreProperties(ignoreUnknown = true) to run with Jackson. -} -``` - -The handler results in an `HttpResponse>`. The response is completed as soon as all headers -are read. If the chosen decoder's `toDeferredObject` is implemented correctly, processing is deferred -till you invoke the supplier and the body is decoded from a streaming source, typically an `InputStream` -or a `Reader`. - -The `Decoder` interface has a naive default implementation for `toDeferredObject` that doesn't read -from a streaming source. Here's how it'd be properly implemented for our HTML adapter's decoder. - -```java hl_lines="10" -@Override -public BodySubscriber> toDeferredObject( - TypeRef type, @Nullable MediaType mediaType) { - requireSupport(type); - requireCompatibleOrNull(mediaType); - - var charset = charsetOrUtf8(mediaType); - BodySubscriber> subscriber = BodySubscribers.mapping( - MoreBodySubscribers.ofReader(charset), - reader -> () -> Parser.htmlParser().parseInput(new BufferedReader(reader), "")); // Note the deferred parsing - return BodySubscribers.mapping( - subscriber, - supplier -> () -> type.exactRawType().cast(supplier.get())); // Safely cast Document to T -} -``` - -[methanol_jackson]: https://github.com/mizosoft/methanol/tree/master/methanol-jackson -[jsoup]: https://jsoup.org/ -[encoder_javadoc]: ../api/latest/methanol/com/github/mizosoft/methanol/BodyAdapter.Encoder.html -[decoder_javadoc]: ../api/latest/methanol/com/github/mizosoft/methanol/BodyAdapter.Decoder.html -[bodyadapter_javadoc]: ../api/latest/methanol/com/github/mizosoft/methanol/BodyAdapter.html diff --git a/docs/progress_tracking.md b/docs/progress_tracking.md index 9105ce8ab..b55d5e85d 100644 --- a/docs/progress_tracking.md +++ b/docs/progress_tracking.md @@ -28,7 +28,7 @@ bytes transferred & time passed, both calculated since the last event. !!! tip You can use the builder to set an `Executor` that's used for dispatching progress events to your listener. That's useful in case your listener does something like GUI updates. - You'd want it to be invoked in the GUI thread rather than an arbitrary HTTP client thread. +You'd want it to be invoked in the GUI thread rather than an arbitrary HTTP client thread. ```java hl_lines="3" var tracker = ProgressTracker.newBuilder() @@ -109,5 +109,3 @@ progress is tracked by registering a `Listener` with a request's `BodyPublisher` } } ``` - -[comment]: <> (TODO mention multipart tracking?) diff --git a/docs/streaming_requests.md b/docs/streaming_requests.md new file mode 100644 index 000000000..abf2164f0 --- /dev/null +++ b/docs/streaming_requests.md @@ -0,0 +1,29 @@ +# Streaming Requests + +`MoreBodyPublishers` provides publishers for asynchronously streaming the request body into an `OutputStream` or a `WritableByteChannel`. + +Let's say your sever supports compressed requests. If you're sending a large file, you'd want to send it compressed. + +```java +final Methanol client = Methanol.create(); +final Executor executor = Executors.newVirtualThreadPerTaskExecutor(); + +HttpResponse postGzipped(Path file) { + return client.send( + MutableRequest.POST( + "https://example.com", + MoreBodyPublishers.ofOutputStream( + out -> Files.copy(file, out), executor)) + .header("Content-Encoding", "gzip"), + BodyHandlers.discarding()); +} +``` + +`MoreBodyPublishers::ofOutputStream` accepts a callback that takes the `OutputStream` to stream to. +The callback is executed on the given executor. The stream may buffer content temporarily in case the consumer can't keep up with the producer, or till an inner buffer becomes full. +You can use `OutputStream::flush`to make any buffered content available for consumption. The stream is closed automatically after the callback. +If the callback fails, the request is completed exceptionally. + +As of 1.8.0, this is the recommended way for streaming requests rather than using [`WritableBodyPublisher`][writablebodypublisher_javadoc] directly. + +[writablebodypublisher_javadoc]: https://mizosoft.github.io/methanol/api/latest/methanol/com/github/mizosoft/methanol/WritableBodyPublisher.html diff --git a/docs/writablebodypublisher.md b/docs/writablebodypublisher.md deleted file mode 100644 index 61c264f0e..000000000 --- a/docs/writablebodypublisher.md +++ /dev/null @@ -1,35 +0,0 @@ -# WritableBodyPublisher - -Using `WritableBodyPublisher`, you can stream the request body through an `OutputStream` or a `WritableByteChannel`, -possibly asynchronously. - -## Example - Gzipped Uploads - -Let's say your sever supports compressed requests. You'd want your file uploads to be faster, so you -compress the request body with gzip. - -```java -final Methanol client = Methanol.create(); - -CompletableFuture> postAsync(Path file) { - var requestBody = WritableBodyPublisher.create(); - var request = MutableRequest.POST("https://example.com", requestBody) - .header("Content-Encoding", "gzip"); - - CompletableFuture.runAsync(() -> { - try (var gzipOut = new GZIPOutputStream(requestBody.outputStream())) { - Files.copy(file, gzipOut); - } catch (IOException ioe) { - requestBody.closeExceptionally(ioe); - } - }); - - return client.sendAsync(request, BodyHandlers.discarding()); -} -``` - -`WritableBodyPublisher` acts as a pipe which connects `OutputStream` and `BodyPublisher` backends. -It may buffer content temporarily in case the consumer can't keep up with the producer, or till an -inner buffer becomes full. You can use `WritableBodyPublisher::flush`to make any buffered content -available for consumption. After you're done writing, call `close()` or `closeExceptionally(Throwable)` -to complete the request either normally or exceptionally. diff --git a/generate-docs.sh b/generate-docs.sh index f006bb554..629967a7d 100644 --- a/generate-docs.sh +++ b/generate-docs.sh @@ -1,5 +1,27 @@ #!/usr/bin/env bash +# +# Copyright (c) 2024 Moataz Abdelnasser +# +# Permission is hereby granted, free of charge, to any person obtaining a copy +# of this software and associated documentation files (the "Software"), to deal +# in the Software without restriction, including without limitation the rights +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +# copies of the Software, and to permit persons to whom the Software is +# furnished to do so, subject to the following conditions: +# +# The above copyright notice and this permission notice shall be included in all +# copies or substantial portions of the Software. +# +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +# SOFTWARE. +# + JAVADOC_SITE_PATH=api/latest # Fail the script if one command fails. @@ -12,6 +34,7 @@ python -m pip install mkdocs-material # Make the necessary files locatable by MkDocs. mkdir -p docs/adapters +cp -f README.md docs/index.md cp -f methanol-gson/README.md docs/adapters/gson.md cp -f methanol-jackson/README.md docs/adapters/jackson.md cp -f methanol-jackson-flux/README.md docs/adapters/jackson_flux.md diff --git a/gradle.properties b/gradle.properties index 1c7aa280c..c39f8770f 100644 --- a/gradle.properties +++ b/gradle.properties @@ -2,3 +2,4 @@ systemProp.org.gradle.internal.publish.checksums.insecure=true systemProp.javax.xml.parsers.SAXParserFactory=com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl systemProp.javax.xml.transform.TransformerFactory=com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl systemProp.javax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl +org.gradle.jvmargs=-Xmx4096m diff --git a/images/popcat.gif b/images/popcat.gif new file mode 100644 index 000000000..ca8be391a Binary files /dev/null and b/images/popcat.gif differ diff --git a/methanol-benchmarks/README.md b/methanol-benchmarks/README.md index 2d579c3d2..bc8f98a76 100644 --- a/methanol-benchmarks/README.md +++ b/methanol-benchmarks/README.md @@ -25,8 +25,8 @@ Compare Methanol's non-blocking decoders with available `InputStream` ones: | Brotli `BodyDecoder` | thrpt | 5 | 4186.791 | 213.283 | ops/s | | `BrotliInputStream` | thrpt | 5 | 2631.312 | 136.291 | ops/s | -Results show that `BodyDecoder` implementations are on par with available `InputStream` based -decoders. Note that the brotli benchmark is biased as it also compares native C vs pure Java implementations. +Results show that `BodyDecoder` implementations are on par with available `InputStream` based decoders. +Note that the brotli benchmark is biased as it also compares native C vs pure Java implementations. [jmh]: https://openjdk.java.net/projects/code-tools/jmh/ [benchmarks_maven]: https://mvnrepository.com/artifact/com.github.mizosoft.methanol/benchmarks/ diff --git a/methanol-gson/README.md b/methanol-gson/README.md index 376ce2ab9..d388fd675 100644 --- a/methanol-gson/README.md +++ b/methanol-gson/README.md @@ -6,8 +6,8 @@ Adapters for JSON using [Gson][gson]. ### Gradle -```gradle -implementation 'com.github.mizosoft.methanol:methanol-gson:1.7.0' +```kotlin +implementation("com.github.mizosoft.methanol:methanol-gson:1.7.0") ``` ### Maven @@ -20,164 +20,31 @@ implementation 'com.github.mizosoft.methanol:methanol-gson:1.7.0' ``` -The adapters need to be registered as [service providers][serviceloader_javadoc] so Methanol knows they're there. -The way this is done depends on your project setup. - -### Module Path - -Follow these steps if your project uses the Java module system. - -1. Add this class to your module: - - ```java - public class GsonProviders { - private static final Gson gson = new Gson(); - - public static class EncoderProvider { - public static BodyAdapter.Encoder provider() { - return GsonAdapterFactory.createEncoder(gson); - } - } - - public static class DecoderProvider { - public static BodyAdapter.Decoder provider() { - return GsonAdapterFactory.createDecoder(gson); - } - } - } - ``` - -2. Add the corresponding provider declarations in your `module-info.java` file. - - ```java - requires methanol.adapter.gson; - - provides BodyAdapter.Encoder with GsonProviders.EncoderProvider; - provides BodyAdapter.Decoder with GsonProviders.DecoderProvider; - ``` - -### Classpath - -Registering adapters from the classpath requires declaring the implementation classes in provider-configuration -files that are bundled with your JAR. You'll first need to implement delegating `Encoder` & `Decoder` -that forward to the instances created by `GsonAdapterFactory`. Extending from `ForwardingEncoder` & -`ForwardingDecoder` makes this easier. - -You can use Google's [AutoService][autoservice] to generate the provider-configuration files automatically, -so you won't bother writing them. - -#### Using AutoService - -First, [install AutoService][autoservice_getting_started]. - -##### Gradle - -```gradle -implementation "com.google.auto.service:auto-service-annotations:$autoServiceVersion" -annotationProcessor "com.google.auto.service:auto-service:$autoServiceVersion" -``` - -##### Maven - -```xml - - com.google.auto.service - auto-service-annotations - ${autoServiceVersion} - -``` - -Configure the annotation processor with the compiler plugin. - -```xml - - maven-compiler-plugin - - - - com.google.auto.service - auto-service - ${autoServiceVersion} - - - - -``` - -Next, add this class to your project: +## Usage ```java -public class GsonAdapters { - private static final Gson gson = new Gson(); - - @AutoService(BodyAdapter.Encoder.class) - public static class Encoder extends ForwardingEncoder { - public Encoder() { - super(GsonAdapterFactory.createEncoder(gson)); - } - } - - @AutoService(BodyAdapter.Decoder.class) - public static class Decoder extends ForwardingDecoder { - public Decoder() { - super(GsonAdapterFactory.createDecoder(gson)); - } - } -} -``` - -#### Manual Configuration - -You can also write the configuration files manually. First, add this class to your project: - -```java -public class GsonAdapters { - private static final Gson gson = new Gson(); - - public static class Encoder extends ForwardingEncoder { - public Encoder() { - super(GsonAdapterFactory.createEncoder(gson)); - } - } - - public static class Decoder extends ForwardingDecoder { - public Decoder() { - super(GsonAdapterFactory.createDecoder(gson)); - } - } -} -``` - -Next, create two provider-configuration files in the resource directory: `META-INF/services`, -one for the encoder and the other for the decoder. Each file must contain the fully qualified -name of the implementation class. +var gson = new Gson(); +var adapterCodec = + AdapterCodec.newBuilder() + .encoder(GsonAdapterFactory.createEncoder(gson)) + .decoder(GsonAdapterFactory.createDecoder(gson)) + .build(); +var client = + Methanol.newBuilder() + .adapterCodec(adapterCodec) + .build(); -Let's say the above class is in a package named `com.example`. You'll want to have one file for the -encoder named: +record Person(String name) {} -``` -META-INF/services/com.github.mizosoft.methanol.BodyAdapter$Encoder -``` - -and contains the following line: - -``` -com.example.GsonAdapters$Encoder +var bruceLee = new Person("Bruce Lee"); +var response = client.send( + MutableRequest.POST(".../echo", bruceLee, MediaType.APPLICATION_JSON), + Person.class); +assertThat(response.body()).isEqualTo(bruceLee); ``` -Similarly, the decoder's file is named: +## Legacy Adapters -``` -META-INF/services/com.github.mizosoft.methanol.BodyAdapter$Decoder -``` - -and contains: - -``` -com.example.GsonAdapters$Decoder -``` +See [Legacy Adapters](https://mizosoft.github.io/methanol/legacy_adapters/) [gson]: https://github.com/google/gson -[autoservice]: https://github.com/google/auto/tree/master/service -[autoservice_getting_started]: https://github.com/google/auto/tree/master/service#getting-started -[serviceloader_javadoc]: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/ServiceLoader.html diff --git a/methanol-jackson-flux/README.md b/methanol-jackson-flux/README.md index 31efa2ea1..a02c64675 100644 --- a/methanol-jackson-flux/README.md +++ b/methanol-jackson-flux/README.md @@ -5,14 +5,12 @@ Adapters for JSON & Reactive Streams using [Jackson][jackson] & [Reactor][reacto ## Decoding This adapter converts response bodies into publisher-based sources. Supported types are -`Mono`, `Flux`, `org.reactivestreams.Publisher` and `java.util.concurrent.Flow.Publisher`. For all -these types except `Mono`, the response body is expected to be a JSON array. The array is tokenized -into its individual elements, each mapped to the publisher's element type. +`Mono`, `Flux`, `org.reactivestreams.Publisher` and `java.util.concurrent.Flow.Publisher`. +For all these types except `Mono`, the response body is expected to be a JSON array. +The array is tokenized into its individual elements, each mapped to the publisher's element type. -Note that an `HttpResponse` handled with this adapter is completed immediately after the response headers -are received. Body completion is handled by the returned publisher source. Additionally, the decoder -always uses Jackson's non-blocking parser. This makes `MoreBodyHandlers::ofDeferredObject` redundant -with this decoder. +Note that an `HttpResponse` handled with this adapter is completed immediately after the response headers are received. +Body completion is handled by the returned publisher source. Additionally, the decoder always uses Jackson's non-blocking parser. ## Encoding @@ -25,7 +23,7 @@ With the exception of `Mono`, any subtype of `org.reactivestreams.Publisher` or ### Gradle ```gradle -implementation 'com.github.mizosoft.methanol:methanol-jackson-flux:1.7.0' +implementation("com.github.mizosoft.methanol:methanol-jackson-flux:1.7.0") ``` ### Maven @@ -38,165 +36,35 @@ implementation 'com.github.mizosoft.methanol:methanol-jackson-flux:1.7.0' ``` -The adapters need to be registered as [service providers][serviceloader_javadoc] so Methanol knows they're there. -The way this is done depends on your project setup. - -### Module Path - -Follow these steps if your project uses the Java module system. - -1. Add this class to your module: - - ```java - public class JacksonFluxProviders { - private static final ObjectMapper mapper = new ObjectMapper(); - - public static class EncoderProvider { - public static BodyAdapter.Encoder provider() { - return JacksonFluxAdapterFactory.createEncoder(mapper); - } - } - - public static class DecoderProvider { - public static BodyAdapter.Decoder provider() { - return JacksonFluxAdapterFactory.createDecoder(mapper); - } - } - } - ``` - -2. Add the corresponding provider declarations in your `module-info.java` file. - - ```java - requires methanol.adapter.jackson.flux; - - provides BodyAdapter.Encoder with JacksonFluxProviders.EncoderProvider; - provides BodyAdapter.Decoder with JacksonFluxProviders.DecoderProvider; - ``` - -### Classpath - -Registering adapters from the classpath requires declaring the implementation classes in provider-configuration -files that are bundled with your JAR. You'll first need to implement delegating `Encoder` & `Decoder` -that forward to the instances created by `JacksonAdapterFactory`. Extending from `ForwardingEncoder` & -`ForwardingDecoder` makes this easier. - -You can use Google's [AutoService][autoservice] to generate the provider-configuration files automatically, -so you won't bother writing them. - -#### Using AutoService - -First, [install AutoService][autoservice_getting_started]. - -##### Gradle - -```gradle -implementation "com.google.auto.service:auto-service-annotations:$autoServiceVersion" -annotationProcessor "com.google.auto.service:auto-service:$autoServiceVersion" -``` - -##### Maven - -```xml - - com.google.auto.service - auto-service-annotations - ${autoServiceVersion} - -``` - -Configure the annotation processor with the compiler plugin. - -```xml - - maven-compiler-plugin - - - - com.google.auto.service - auto-service - ${autoServiceVersion} - - - - -``` - -Next, add this class to your project: +## Usage ```java -public class JacksonFluxAdapters { - private static final ObjectMapper mapper = new ObjectMapper(); - - @AutoService(BodyAdapter.Encoder.class) - public static class Encoder extends ForwardingEncoder { - public Encoder() { - super(JacksonFluxAdapterFactory.createEncoder(mapper)); - } - } - - @AutoService(BodyAdapter.Decoder.class) - public static class Decoder extends ForwardingDecoder { - public Decoder() { - super(JacksonFluxAdapterFactory.createDecoder(mapper)); - } - } +var mapper = new JsonMapper(); +var adapterCodec = + AdapterCodec.newBuilder() + .encoder(JacksonFluxAdapterFactory.createEncoder(mapper)) + .decoder(JacksonFluxAdapterFactory.createDecoder(mapper)) + .build(); +var client = Methanol.newBuilder().adapterCodec(adapterCodec).build(); + +record Person(String name) { } -``` - -#### Manual Configuration - -You can also write the configuration files manually. First, add this class to your project: -```java -public class JacksonFluxAdapters { - private static final ObjectMapper mapper = new ObjectMapper(); - - public static class Encoder extends ForwardingEncoder { - public Encoder() { - super(JacksonFluxAdapterFactory.createEncoder(mapper)); - } - } - - public static class Decoder extends ForwardingDecoder { - public Decoder() { - super(JacksonFluxAdapterFactory.createDecoder(mapper)); - } - } -} +var bruceLee = new Person("Bruce Lee"); +var jackieChan = new Person("Jacki Chan"); +var response = + client.send( + MutableRequest.POST( + ".../echo", + Flux.just(bruceLee, jackieChan), + MediaType.APPLICATION_JSON), + new TypeRef>() {}); +assertThat(response.body().toIterable()).containsExactly(bruceLee, jackieChan); ``` -Next, create two provider-configuration files in the resource directory: `META-INF/services`, -one for the encoder and the other for the decoder. Each file must contain the fully qualified -name of the implementation class. - -Let's say the above class is in a package named `com.example`. You'll want to have one file for the -encoder named: - -``` -META-INF/services/com.github.mizosoft.methanol.BodyAdapter$Encoder -``` +## Legacy Adapters -and contains the following line: - -``` -com.example.JacksonFluxAdapters$Encoder -``` - -Similarly, the decoder's file is named: - -``` -META-INF/services/com.github.mizosoft.methanol.BodyAdapter$Decoder -``` - -and contains: - -``` -com.example.JacksonFluxAdapters$Decoder -``` +See [Legacy Adapters](https://mizosoft.github.io/methanol/legacy_adapters/) [jackson]: https://github.com/FasterXML/jackson [reactor]: https://github.com/reactor/reactor-core -[autoservice]: https://github.com/google/auto/tree/master/service -[autoservice_getting_started]: https://github.com/google/auto/tree/master/service#getting-started -[serviceloader_javadoc]: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/ServiceLoader.html diff --git a/methanol-jackson/README.md b/methanol-jackson/README.md index 9ac48722e..6a9fa373a 100644 --- a/methanol-jackson/README.md +++ b/methanol-jackson/README.md @@ -7,7 +7,7 @@ Adapters for [Jackson][jackson]. ### Gradle ```gradle -implementation 'com.github.mizosoft.methanol:methanol-jackson:1.7.0' +implementation("com.github.mizosoft.methanol:methanol-jackson:1.7.0") ``` ### Maven @@ -20,240 +20,83 @@ implementation 'com.github.mizosoft.methanol:methanol-jackson:1.7.0' ``` -The adapters need to be registered as [service providers][serviceloader_javadoc] so Methanol knows they're there. -The way this is done depends on your project setup. - -### Module Path - -Follow these steps if your project uses the Java module system. - -1. Add this class to your module: - - ```java - public class JacksonJsonProviders { - private static final ObjectMapper mapper = new ObjectMapper(); - - public static class EncoderProvider { - public static BodyAdapter.Encoder provider() { - return JacksonAdapterFactory.createJsonEncoder(mapper); - } - } - - public static class DecoderProvider { - public static BodyAdapter.Decoder provider() { - return JacksonAdapterFactory.createJsonDecoder(mapper); - } - } - } - ``` - -2. Add the corresponding provider declarations in your `module-info.java` file. - - ```java - requires methanol.adapter.jackson; - - provides BodyAdapter.Encoder with JacksonJsonProviders.EncoderProvider; - provides BodyAdapter.Decoder with JacksonJsonProviders.DecoderProvider; - ``` - -### Classpath - -Registering adapters from the classpath requires declaring the implementation classes in provider-configuration -files that are bundled with your JAR. You'll first need to implement delegating `Encoder` & `Decoder` -that forward to the instances created by `JacksonAdapterFactory`. Extending from `ForwardingEncoder` & -`ForwardingDecoder` makes this easier. - -You can use Google's [AutoService][autoservice] to generate the provider-configuration files automatically, -so you won't bother writing them. - -#### Using AutoService - -First, [install AutoService][autoservice_getting_started]. - -##### Gradle - -```gradle -implementation "com.google.auto.service:auto-service-annotations:$autoServiceVersion" -annotationProcessor "com.google.auto.service:auto-service:$autoServiceVersion" -``` - -##### Maven - -```xml - - com.google.auto.service - auto-service-annotations - ${autoServiceVersion} - -``` - -Configure the annotation processor with the compiler plugin. - -```xml - - maven-compiler-plugin - - - - com.google.auto.service - auto-service - ${autoServiceVersion} - - - - -``` - -Next, add this class to your project: +## Usage ```java -public class JacksonJsonAdapters { - private static final ObjectMapper mapper = new ObjectMapper(); - - @AutoService(BodyAdapter.Encoder.class) - public static class Encoder extends ForwardingEncoder { - public Encoder() { - super(JacksonAdapterFactory.createJsonEncoder(mapper)); - } - } - - @AutoService(BodyAdapter.Decoder.class) - public static class Decoder extends ForwardingDecoder { - public Decoder() { - super(JacksonAdapterFactory.createJsonDecoder(mapper)); - } - } -} -``` +var mapper = new JsonMapper(); +var adapterCodec = + AdapterCodec.newBuilder() + .encoder(JacksonAdapterFactory.createJsonEncoder(mapper)) + .decoder(JacksonAdapterFactory.createJsonDecoder(mapper)) + .build(); +var client = + Methanol.newBuilder() + .adapterCodec(adapterCodec) + .build(); -#### Manual Configuration +record Person(String name) {} -You can also write the configuration files manually. First, add this class to your project: - -```java -public class JacksonJsonAdapters { - private static final ObjectMapper mapper = new ObjectMapper(); - - public static class Encoder extends ForwardingEncoder { - public Encoder() { - super(JacksonAdapterFactory.createJsonEncoder(mapper)); - } - } - - public static class Decoder extends ForwardingDecoder { - public Decoder() { - super(JacksonAdapterFactory.createJsonDecoder(mapper)); - } - } -} -``` - -Next, create two provider-configuration files in the resource directory: `META-INF/services`, -one for the encoder and the other for the decoder. Each file must contain the fully qualified -name of the implementation class. - -Let's say the above class is in a package named `com.example`. You'll want to have one file for the -encoder named: - -``` -META-INF/services/com.github.mizosoft.methanol.BodyAdapter$Encoder -``` - -and contains the following line: - -``` -com.example.JacksonJsonAdapters$Encoder -``` - -Similarly, the decoder's file is named: - -``` -META-INF/services/com.github.mizosoft.methanol.BodyAdapter$Decoder +var bruceLee = new Person("Bruce Lee"); +var response = client.send( + MutableRequest.POST(".../echo", bruceLee, MediaType.APPLICATION_JSON), + Person.class); +assertThat(response.body()).isEqualTo(bruceLee); ``` -and contains: +## Formats -``` -com.example.JacksonJsonAdapters$Decoder -``` - -## Adapters for other formats - -The Jackson adapter doesn't only support JSON. You can pair whatever `ObjectMapper` implementation -with one or more `MediaTypes` to create adapters for any of the formats supported by Jackson. For -instance, here's a provider for a XML adapter. You'll need to pull in [`jackson-dataformat-xml`](https://github.com/FasterXML/jackson-dataformat-xml). You can install it as mentioned above. +`ObjectMapper` implementations can be paired with one or more `MediaTypes` to create adapters for any format supported by Jackson. +For instance, here's an adapter for XML that uses [`jackson-dataformat-xml`](https://github.com/FasterXML/jackson-dataformat-xml). ```java -public class JacksonXmlProviders { - private static final ObjectMapper mapper = new XmlMapper(); - - public static class EncoderProvider { - public static BodyAdapter.Encoder provider() { - return JacksonAdapterFactory.createEncoder(mapper, MediaType.TEXT_XML); - } - } - - public static class DecoderProvider { - public static BodyAdapter.Decoder provider() { - return JacksonAdapterFactory.createDecoder(mapper, MediaType.TEXT_XML); - } - } -} + var mapper = new XmlMapper(); +var adapterCodec = + AdapterCodec.newBuilder() + .encoder( + JacksonAdapterFactory.createEncoder( + mapper, MediaType.APPLICATION_XML, MediaType.TEXT_XML)) + .decoder( + JacksonAdapterFactory.createDecoder( + mapper, MediaType.APPLICATION_XML, MediaType.TEXT_XML)) + .build(); ``` For binary formats, you usually can't just plug in an `ObjectMapper` as a schema must be applied for each type. -For this reason you can use a custom `ObjectReaderFactory` and/or `ObjectWriterFactory`. For instance, here's a provider for a -[Protocol Buffers](https://github.com/FasterXML/jackson-dataformats-binary/tree/2.14/protobuf) adapter. +Thus, you would use a custom `ObjectReaderFactory` and/or `ObjectWriterFactory`. +Here's an adapter for [Protocol Buffers](https://github.com/FasterXML/jackson-dataformats-binary/tree/2.14/protobuf). You'll need to know what types are expected beforehand. ```java record Point(int x, int y) {} -public class JacksonProtobufProviders { - private static final ObjectMapper mapper = new ProtobufMapper(); - - /** - * We'll store our schemas in a map. You can implement this in other ways, like loading the - * protobuf files lazily when needed. - */ - private static final Map, ProtobufSchema> schemas; - - static { - try { - schemas = Map.of( - TypeRef.from(Point.class), - ProtobufSchemaLoader.std.parse( - """ - message Point { - required int32 x = 1; - required int32 y = 2; - } - """)); - } catch (IOException e) { - throw new ExceptionInInitializerError(e); - } - } - - public static class EncoderProvider { - public static BodyAdapter.Encoder provider() { - ObjectWriterFactory writerFactory = (mapper, type) -> mapper.writer(schemas.get(type)); - return JacksonAdapterFactory.createEncoder( - mapper, writerFactory, MediaType.APPLICATION_X_PROTOBUF); - } - } - - public static class DecoderProvider { - public static BodyAdapter.Decoder provider() { - ObjectReaderFactory readerFactory = - (mapper, type) -> mapper.readerFor(type.rawType()).with(schemas.get(type)); - return JacksonAdapterFactory.createDecoder( - mapper, readerFactory, MediaType.APPLICATION_X_PROTOBUF); - } - } -} -``` +var schemas = + Map.of( + TypeRef.of(Point.class), + ProtobufSchemaLoader.std.parse( + """ + message Point { + required int32 x = 1; + required int32 y = 2; + } + """)); +var mapper = new ProtobufMapper(); +var adapterCodec = + AdapterCodec.newBuilder() + .encoder( + JacksonAdapterFactory.createEncoder( + mapper, + (localMapper, typeRef) -> localMapper.writer(schemas.get(typeRef)), + MediaType.APPLICATION_X_PROTOBUF)) + .decoder( + JacksonAdapterFactory.createDecoder( + mapper, + (localMapper, typeRef) -> + mapper.readerFor(typeRef.rawType()).with(schemas.get(typeRef)), + MediaType.APPLICATION_X_PROTOBUF)); +``` + +## Legacy Adapters + +See [Legacy Adapters](https://mizosoft.github.io/methanol/legacy_adapters/) [jackson]: https://github.com/FasterXML/jackson -[autoservice]: https://github.com/google/auto/tree/master/service -[autoservice_getting_started]: https://github.com/google/auto/tree/master/service#getting-started -[serviceloader_javadoc]: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/ServiceLoader.html diff --git a/methanol-jaxb-jakarta/README.md b/methanol-jaxb-jakarta/README.md index 9f4bdc4ea..8f13510bf 100644 --- a/methanol-jaxb-jakarta/README.md +++ b/methanol-jaxb-jakarta/README.md @@ -7,7 +7,7 @@ Adapters for XML using Jakarta EE's [JAXB][jaxb]. ### Gradle ```gradle -implementation 'com.github.mizosoft.methanol:methanol-jaxb-jakarta:1.7.0' +implementation("com.github.mizosoft.methanol:methanol-jaxb-jakarta:1.7.0") ``` ### Maven @@ -20,165 +20,30 @@ implementation 'com.github.mizosoft.methanol:methanol-jaxb-jakarta:1.7.0' ``` -The adapters need to be registered as [service providers][serviceloader_javadoc] so Methanol knows -they're there. -The way this is done depends on your project setup. - -### Module Path - -Follow these steps if your project uses the Java module system. - -1. Add this class to your module: - - ```java - public class JaxbProviders { - public static class EncoderProvider { - public static BodyAdapter.Encoder provider() { - return JaxbAdapterFactory.createEncoder(); - } - } - - public static class DecoderProvider { - public static BodyAdapter.Decoder provider() { - return JaxbAdapterFactory.createDecoder(); - } - } - } - ``` - -2. Add the corresponding provider declarations in your `module-info.java` file. - - ```java - requires methanol.adapter.jaxb.jakarta; - - provides BodyAdapter.Encoder with JaxbProviders.EncoderProvider; - provides BodyAdapter.Decoder with JaxbProviders.DecoderProvider; - ``` - -### Classpath - -Registering adapters from the classpath requires declaring the implementation classes in -provider-configuration -files that are bundled with your JAR. You'll first need to implement -delegating `Encoder` & `Decoder` -that forward to the instances created by `JaxbAdapterFactory`. Extending from `ForwardingEncoder` & -`ForwardingDecoder` makes this easier. - -You can use Google's [AutoService][autoservice] to generate the provider-configuration files -automatically, -so you won't bother writing them. - -#### Using AutoService - -First, [install AutoService][autoservice_getting_started]. - -##### Gradle - -```gradle -implementation "com.google.auto.service:auto-service-annotations:$autoServiceVersion" -annotationProcessor "com.google.auto.service:auto-service:$autoServiceVersion" -``` - -##### Maven - -```xml - - com.google.auto.service - auto-service-annotations - ${autoServiceVersion} - -``` - -Configure the annotation processor with the compiler plugin. - -```xml - - maven-compiler-plugin - - - - com.google.auto.service - auto-service - ${autoServiceVersion} - - - - -``` - -Next, add this class to your project: - -```java -public class JaxbAdapters { - @AutoService(BodyAdapter.Encoder.class) - public static class Encoder extends ForwardingEncoder { - public Encoder() { - super(JaxbAdapterFactory.createEncoder()); - } - } - - @AutoService(BodyAdapter.Decoder.class) - public static class Decoder extends ForwardingDecoder { - public Decoder() { - super(JaxbAdapterFactory.createDecoder()); - } - } -} -``` - -#### Manual Configuration - -You can also write the configuration files manually. First, add this class to your project: +## Usage ```java -public class JaxbAdapters { - public static class Encoder extends ForwardingEncoder { - public Encoder() { - super(JaxbAdapterFactory.createEncoder()); - } - } - - public static class Decoder extends ForwardingDecoder { - public Decoder() { - super(JaxbAdapterFactory.createDecoder()); - } - } -} -``` - -Next, create two provider-configuration files in the resource directory: `META-INF/services`, -one for the encoder and the other for the decoder. Each file must contain the fully qualified -name of the implementation class. +var adapterCodec = + AdapterCodec.newBuilder() + .encoder(JaxbAdapterFactory.createEncoder()) + .decoder(JaxbAdapterFactory.createDecoder()) + .build(); +var client = + Methanol.newBuilder() + .adapterCodec(adapterCodec) + .build(); -Let's say the above class is in a package named `com.example`. You'll want to have one file for the -encoder named: +record Person(String name) {} -``` -META-INF/services/com.github.mizosoft.methanol.BodyAdapter$Encoder +var bruceLee = new Person("Bruce Lee"); +var response = client.send( + MutableRequest.POST(".../echo", bruceLee, MediaType.APPLICATION_XML), + Person.class); +assertThat(response.body()).isEqualTo(bruceLee); ``` -and contains the following line: +## Legacy Adapters -``` -com.example.JaxbAdapters$Encoder -``` - -Similarly, the decoder's file is named: - -``` -META-INF/services/com.github.mizosoft.methanol.BodyAdapter$Decoder -``` - -and contains: - -``` -com.example.JaxbAdapters$Decoder -``` +See [Legacy Adapters](https://mizosoft.github.io/methanol/legacy_adapters/) [jaxb]: https://eclipse-ee4j.github.io/jaxb-ri/ - -[autoservice]: https://github.com/google/auto/tree/master/service - -[autoservice_getting_started]: https://github.com/google/auto/tree/master/service#getting-started - -[serviceloader_javadoc]: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/ServiceLoader.html diff --git a/methanol-jaxb/README.md b/methanol-jaxb/README.md index 0621a54f7..62bc3c2dd 100644 --- a/methanol-jaxb/README.md +++ b/methanol-jaxb/README.md @@ -7,7 +7,7 @@ Adapters for XML using Java EE's [JAXB][jaxb]. ### Gradle ```gradle -implementation 'com.github.mizosoft.methanol:methanol-jaxb:1.7.0' +implementation("com.github.mizosoft.methanol:methanol-jaxb:1.7.0") ``` ### Maven @@ -20,158 +20,30 @@ implementation 'com.github.mizosoft.methanol:methanol-jaxb:1.7.0' ``` -The adapters need to be registered as [service providers][serviceloader_javadoc] so Methanol knows they're there. -The way this is done depends on your project setup. - -### Module Path - -Follow these steps if your project uses the Java module system. - -1. Add this class to your module: - - ```java - public class JaxbProviders { - public static class EncoderProvider { - public static BodyAdapter.Encoder provider() { - return JaxbAdapterFactory.createEncoder(); - } - } - - public static class DecoderProvider { - public static BodyAdapter.Decoder provider() { - return JaxbAdapterFactory.createDecoder(); - } - } - } - ``` - -2. Add the corresponding provider declarations in your `module-info.java` file. - - ```java - requires methanol.adapter.jaxb; - - provides BodyAdapter.Encoder with JaxbProviders.EncoderProvider; - provides BodyAdapter.Decoder with JaxbProviders.DecoderProvider; - ``` - -### Classpath - -Registering adapters from the classpath requires declaring the implementation classes in provider-configuration -files that are bundled with your JAR. You'll first need to implement delegating `Encoder` & `Decoder` -that forward to the instances created by `JaxbAdapterFactory`. Extending from `ForwardingEncoder` & -`ForwardingDecoder` makes this easier. - -You can use Google's [AutoService][autoservice] to generate the provider-configuration files automatically, -so you won't bother writing them. - -#### Using AutoService - -First, [install AutoService][autoservice_getting_started]. - -##### Gradle - -```gradle -implementation "com.google.auto.service:auto-service-annotations:$autoServiceVersion" -annotationProcessor "com.google.auto.service:auto-service:$autoServiceVersion" -``` - -##### Maven - -```xml - - com.google.auto.service - auto-service-annotations - ${autoServiceVersion} - -``` - -Configure the annotation processor with the compiler plugin. - -```xml - - maven-compiler-plugin - - - - com.google.auto.service - auto-service - ${autoServiceVersion} - - - - -``` - -Next, add this class to your project: +## Usage ```java -public class JaxbAdapters { - @AutoService(BodyAdapter.Encoder.class) - public static class Encoder extends ForwardingEncoder { - public Encoder() { - super(JaxbAdapterFactory.createEncoder()); - } - } - - @AutoService(BodyAdapter.Decoder.class) - public static class Decoder extends ForwardingDecoder { - public Decoder() { - super(JaxbAdapterFactory.createDecoder()); - } - } -} -``` - -#### Manual Configuration - -You can also write the configuration files manually. First, add this class to your project: - -```java -public class JaxbAdapters { - public static class Encoder extends ForwardingEncoder { - public Encoder() { - super(JaxbAdapterFactory.createEncoder()); - } - } - - public static class Decoder extends ForwardingDecoder { - public Decoder() { - super(JaxbAdapterFactory.createDecoder()); - } - } -} -``` - -Next, create two provider-configuration files in the resource directory: `META-INF/services`, -one for the encoder and the other for the decoder. Each file must contain the fully qualified -name of the implementation class. +var adapterCodec = + AdapterCodec.newBuilder() + .encoder(JaxbAdapterFactory.createEncoder()) + .decoder(JaxbAdapterFactory.createDecoder()) + .build(); +var client = + Methanol.newBuilder() + .adapterCodec(adapterCodec) + .build(); -Let's say the above class is in a package named `com.example`. You'll want to have one file for the -encoder named: +record Person(String name) {} -``` -META-INF/services/com.github.mizosoft.methanol.BodyAdapter$Encoder -``` - -and contains the following line: - -``` -com.example.JaxbAdapters$Encoder +var bruceLee = new Person("Bruce Lee"); +var response = client.send( + MutableRequest.POST(".../echo", bruceLee, MediaType.APPLICATION_XML), + Person.class); +assertThat(response.body()).isEqualTo(bruceLee); ``` -Similarly, the decoder's file is named: +## Legacy Adapters -``` -META-INF/services/com.github.mizosoft.methanol.BodyAdapter$Decoder -``` - -and contains: - -``` -com.example.JaxbAdapters$Decoder -``` +See [Legacy Adapters](https://mizosoft.github.io/methanol/legacy_adapters/) [jaxb]: https://javaee.github.io/jaxb-v2/ -[autoservice]: https://github.com/google/auto/tree/master/service -[autoservice_getting_started]: https://github.com/google/auto/tree/master/service#getting-started -[serviceloader_javadoc]: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/ServiceLoader.html diff --git a/methanol-kotlin/README.md b/methanol-kotlin/README.md index a18df077b..bee2ac164 100644 --- a/methanol-kotlin/README.md +++ b/methanol-kotlin/README.md @@ -1,3 +1,484 @@ # methanol-kotlin -Coming Soon! +*note: coming soon!* + +Kotlin extensions for Methanol, which include: + +- A DSL for HTTP requests. +- Adapters for [Kotlin Serialization](https://kotlinlang.org/docs/serialization.html). + +## Installation + +### Gradle + +```kotlin +implementation("com.github.mizosoft.methanol:methanol-kotlin:1.8.0") +``` + +## Usage + +Most types and functions in this module are defined as type aliases and extension functions to core Methanol & JDK HTTP client libraries. +They have a different, more Kotlin-like feel, however. The best way to get familiar is to go through the examples. +There's also the [KDocs](https://mizosoft.github.io/methanol/api/latest/methanol.kotlin/com.github.mizosoft.methanol.kotlin/index.html) +for a comprehensive list of what is provided. For advanced usage, it's a good idea to be familiar with the Java libraries this module extends. + +Almost everything in this module is configured with lambda expressions that are resolved against a particular [Spec](https://mizosoft.github.io/methanol/api/latest/methanol.kotlin/com.github.mizosoft.methanol.kotlin/-spec/index.html?query=annotation%20class%20Spec). +Look up the KDocs/source of a spec to know all what it can configure. + +Runnable code examples are linked at the end of each section. If you're going to copy from the snippets, add these imports: + +```kotlin +import com.github.mizosoft.methanol.* +import com.github.mizosoft.methanol.kotlin.* +``` + +### Get & Post String + +Let's get started by creating our client. + +```kotlin +val client = Client { + adapterCodec { + basic() + } +} +``` + +Here, we're configuring a [ClientSpec](https://mizosoft.github.io/methanol/api/latest/methanol.kotlin/com.github.mizosoft.methanol.kotlin/-client-spec/index.html). +The only thing we'll configure for now is the client's [`AdapterCodec`](https://mizosoft.github.io/methanol/api/latest/methanol/com/github/mizosoft/methanol/AdapterCodec.html), +which tells it how to map high level types to & from HTTP bodies. `basic()` is good enough for, well, basic types, like `String` & `InputStream`. +Trace through [`basicEncoder`](https://mizosoft.github.io/methanol/api/latest/methanol.kotlin/com.github.mizosoft.methanol.kotlin/-adapter-codec-spec/basic-encoder.html) & [`basicDecoder`](https://mizosoft.github.io/methanol/api/latest/methanol.kotlin/com.github.mizosoft.methanol.kotlin/-adapter-codec-spec/basic-decoder.html) +for all the supported basic types. + +#### Get String + +Now let's GET a string from a URL. + +```kotlin +suspend fun runGet() { + val response = client.get("https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt") + require(response.isSuccessful()) { "Unsuccessful response: $response - ${response.body()}" } + println(response.body()) +} +``` + +Known HTTP verbs have corresponding client-defined functions, like the `client::get` above. Each HTTP verb function returns +a `Response` whose `Response::body` is already converted into the specified type. These functions are suspending, meaning they run as part of a [coroutine](https://kotlinlang.org/docs/coroutines-overview.html). + +#### Post String + +HTTP verb functions take an optional request configuration block next to the URI. We can use it to specify the body of body-bearing requests. + +```kotlin +suspend fun runPost() { + val response: Response = client.post("https://api.github.com/markdown/raw") { + body( + """ + > He who has a ***why*** can bear almost any ***how***. + > - Friedrich Nietzsche + """.trimIndent(), + MediaType.TEXT_MARKDOWN + ) + } + require(response.isSuccessful()) { "Unsuccessful response: $response - ${response.body()}" } + println(response.body()) +} +``` + +Next to what we want to send, we pass `body` a [`MediaType`](https://mizosoft.github.io/methanol/api/latest/methanol/com/github/mizosoft/methanol/MediaType.html) +signifying the desired mapping format. This `MediaType` becomes the request's `Content-Type`. + +Note that we specified the response body type on the left of `client.post(...)`. We could have written `val response = client.post(...)`, +but that can hurt readability since `String` in this expression is what we're getting, not what we're posting; the latter is defined by `body`. You can use either way, though. + +[Runnable Example](https://github.com/mizosoft/methanol/blob/master/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/GetPostString.kt). + +### Get & Post JSON + +Let's get more sophisticated. The basic adapter is nice, but it ain't much. We can make the client understand JSON through Kotlin Serialization. +You'll first need to apply the serialization plugin in your build script & pull in `kotlinx-serialization-json` as specified [here](https://kotlinlang.org/docs/serialization.html#example-json-serialization). + +Now we redefine our client. + +```kotlin +val client = Client { + baseUri("https://api.github.com/") + defaultHeaders { + "Accept" to "application/vnd.github+json" + "X-GitHub-Api-Version" to "2022-11-28" + } + adapterCodec { + basic() + +KotlinAdapter.Encoder( + Json, MediaType.APPLICATION_JSON + ) + +KotlinAdapter.Decoder(Json { + ignoreUnknownKeys = true // For brevity, we'll skip most fields. + }, MediaType.APPLICATION_JSON) + } +} +``` + +This time our client has more configuration, most of which is self-descriptive. The interesting part is how we configure the `AdapterCodec`. +We add (hence the `+`) an encoder & a decoder that use `kotlinx-serialization-json`'s [`Json`](https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-json/kotlinx.serialization.json/-json/). +`KotlinAdapter.Encoder` & `KotlinAdapter.Decoder` are pluggable, and hence can work with all the [supported formats](https://kotlinlang.org/docs/serialization.html#formats). +You'll just need to pass the desired [SerialFormat](https://kotlinlang.org/api/kotlinx.serialization/kotlinx-serialization-core/kotlinx.serialization/-serial-format/), and one or more `MediaType`s signifying that format. + +Note that we keep the basic adapter to still be able to send & receive basic types. `AdapterCodec` will figure out which adapter to use. +You can also add adapters for other formats, say `application/protobuf`. The first adapter (in addition order) that can handle +the object type based on the given `MediaType` is selected. + +#### Get JSON + +Now it's a matter of type specification. + +```kotlin +@Serializable +data class Repository( + val description: String, + @SerialName("full_name") val fullName: String +) + +suspend fun runGet() { + val response = client.get>("users/mizosoft/starred?per_page=10") + response.body().forEach { repo -> + println("${repo.fullName}\n\t${repo.description}") + } +} +``` + +#### Post JSON + +For posts, specify the `MediaType` next to your payload. + +```kotlin +@Serializable +data class Markdown( + val text: String, + val context: String, + val mode: String +) + +suspend fun runPost() { + val response: Response = client.post("markdown") { + body( + Markdown( + "this code very fast: #437", + "torvalds/linux", + "gfm" + ), + MediaType.APPLICATION_JSON + ) + } + println(response.body()) +} +``` + +[Runnable Example]([Runnable Example](https://github.com/mizosoft/methanol/blob/master/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/GetPostJson.kt). + +### Multipart & Forms + +Now let's say we want to upload some cat memes to [Imgur](https://imgur.com/) using their [API](https://api.imgur.com/endpoints/image). + +#### Multipart Bodies + +`multipart/format-data` bodies are perfect for that task. We'll be making use of the JSON decoder from above to extract the image link. + +```kotlin +// You can get your own clien ID here: https://api.imgur.com/oauth2/addclient. +val imgurClientId = System.getProperty("imgur.client.id") + +val client = Client { + baseUri("https://api.imgur.com/3/") + defaultHeaders { + "Authorization" to "Client-ID $imgurClientId" + } + + adapterCodec { + basic() + +KotlinAdapter.Decoder(Json { + ignoreUnknownKeys = true + }, MediaType.APPLICATION_JSON) + } +} + +@Serializable +data class ImgurResponse(val status: Int, val success: Boolean, val data: T?) + +@Serializable +data class Image(val link: String) + +suspend fun multipartUpload() { + val response: Response> = client.post("image") { + multipartBody { + "image" to Path.of("images/popcat.gif") // File's Content-Type will be looked-up automatically. + "title" to "PopCat" + "description" to "A cat that pops" + } + } + require(response.body().success) { + "Unsuccessful response: $response - ${response.body()}" + } + println("Uploaded: ${response.body().data!!.link}") +} +``` + +The [`multipartBody`](https://mizosoft.github.io/methanol/api/latest/methanol.kotlin/com.github.mizosoft.methanol.kotlin/-request-body-spec/multipart-body.html) +block makes it easy to configure `multipart/form-data` bodies as key-value pairs. It can also be used to configure any kind of `multipart/*` body. + +#### Form Bodies + +It turns out that Imgur's upload API also accepts `application/x-www-form-urlencoded` submissions, which may result in more efficient uploads if we +have the image bytes in memory. [`formBody`](https://mizosoft.github.io/methanol/api/latest/methanol.kotlin/com.github.mizosoft.methanol.kotlin/-request-body-spec/form-body.html) is here to help. + +```kotlin +@OptIn(ExperimentalEncodingApi::class) // In order to use Kotlin's Base64. +suspend fun formUpload() { + val response: Response> = client.post("image") { + formBody { + "image" to Base64.Default.encode(Path.of("images/popcat.gif").readBytes()) + "type" to "base64" + "title" to "PopCat" + "description" to "A cat that pops" + } + } + require(response.body().success) { + "Unsuccessful response: $response - ${response.body()}" + } + println("Uploaded: ${response.body().data!!.link}") +} +``` + +When using `multipartBody` or `formBody`, the request's `Content-Type` will be set for you. + +[Runnable Example](https://github.com/mizosoft/methanol/blob/master/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/MultipartAndFormUploads.kt). + +### Caching + +Methanol provides an HTTP caching solution, which supports disk, memory & redis storage backends. + +```kotlin +val client = Client { + userAgent("Chuck Norris") + cache { + onDisk(Path.of(".cache"), 500 * 1024 * 1024) // Occupy at most 500Mb on disk. + } +} +``` + +The cache will be used automatically as you use the client. You can communicate with it using request's `Cache-Control`. + +```kotlin +suspend fun run() { + val response = + client.get("https://i.imgur.com/V79ulbT.gif", BodyHandlers.ofFile(Path.of("popcat.gif"))) { + cacheControl { + maxAge(5.seconds) // Override server's max-age. + } + } as CacheAwareResponse + println( + "$response - ${response.cacheStatus()} (Cached for ${response.headers()["Age"].firstOrNull() ?: -1} seconds)" + ) +} +``` + +Run this example multiple times within 5 seconds and then apart. + +You can also set up a chain of caches, typically in the order of decreasing locality. + +```kotlin +val redisUrl = System.getProperty("redis.url") + +val client = Client { + userAgent("Chuck Norris") + cacheChain { + +Cache { + inMemory(100 * 1024 * 1024) // Occupy at most 100Mb in memory. + } + +Cache { + onDisk(500 * 1024 * 204) // Occupy at most 500Mb on disk. + } + } +} +``` + +Here, we're calling the [`Cache`](https://mizosoft.github.io/methanol/api/latest/methanol.kotlin/com.github.mizosoft.methanol.kotlin/-cache.html) +constructor, and prepend a `+` to add it to the chain. The chain is invoked in the order of addition. + +In case of a single cache or a cache chain, it's always a good practice to close it when you're done. + +```kotlin +client.caches().close() +``` + +You can learn more about caching [here](https://mizosoft.github.io/methanol/caching/). + +[Runnable Example](https://github.com/mizosoft/methanol/blob/master/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/CachingClient.kt). + +### Interceptors + +So far, if we wanted to validate the response, we had to do so each time we get it. We may also want to log the request/response exchange, or send some metrics to a monitoring framework. This is a perfect usage for interceptors. + +```kotlin +object LoggingInterceptor : Interceptor { + val logger: System.Logger = System.getLogger(LoggingInterceptor::class.simpleName) + val requestIdGenerator = AtomicInteger() + + override suspend fun intercept( + request: Request, + chain: Interceptor.Chain + ): Response { + val requestId = requestIdGenerator.getAndIncrement() + val start = System.currentTimeMillis() + logger.log(Level.INFO) { + "$requestId: sending $request \n${request.headers().toHttpString()}" + } + + return chain.forward(request).also { response -> + logger.log(Level.INFO) { + "$requestId: received $response in ${(System.currentTimeMillis() - start).milliseconds} \n" + + request.headers().toHttpString() + } + require(response.isSuccessful()) { "Unsuccessful response: $response" } + } + } +} + +val client = Client { + interceptors { + +LoggingInterceptor + } + + userAgent("Dave Bautista") + adapterCodec { + basic() + } +} + +suspend fun run() { + client.get("https://httpbin.org/gzip") { + headers { + "Accept" to MediaType.APPLICATION_OCTET_STREAM.toString() + } + } +} +``` + +More than one interceptor can be added. Together, they form a chain that is invoked in the order of addition. + +Running this code gives: + +``` +Dec 06, 2024 1:10:47 PM com.github.mizosoft.methanol.samples.kotlin.ClientInterceptor$LoggingInterceptor intercept +INFO: 0: sending https://httpbin.org/gzip GET +Accept: application/octet-stream + +Dec 06, 2024 1:10:48 PM com.github.mizosoft.methanol.samples.kotlin.ClientInterceptor$LoggingInterceptor intercept +INFO: 0: received (GET https://httpbin.org/gzip) 200 in 1.631s +:status: 200 +access-control-allow-credentials: true +access-control-allow-origin: * +content-type: application/json +date: Fri, 06 Dec 2024 11:10:48 GMT +server: gunicorn/19.9.0 +``` + +If you squint, you'll notice that the request headers don't contain a `User-Agent`, although we've configured the client with one. +Additionally, the response lacks typical headers like `Content-Length` & `Content-Encoding`. + +Let's instead add `LoggingInterceptor` as what we'll call a backend interceptor. + +```kotlin +val client = Client { + backendInterceptors { + +LoggingInterceptor + } + + userAgent("Arnold Schwarzenegger") + adapterCodec { + basic() + } +} +``` + +Running this code gives: + +``` +Dec 06, 2024 2:02:55 PM com.github.mizosoft.methanol.samples.kotlin.ClientInterceptor$LoggingInterceptor intercept +INFO: 0: sending https://httpbin.org/gzip GET +Accept: application/octet-stream +Accept-Encoding: deflate, gzip +User-Agent: Dave Bautista + +Dec 06, 2024 2:02:56 PM com.github.mizosoft.methanol.samples.kotlin.ClientInterceptor$LoggingInterceptor intercept +INFO: 0: received (GET https://httpbin.org/gzip) 200 in 1.602s +:status: 200 +access-control-allow-credentials: true +access-control-allow-origin: * +content-encoding: gzip +content-length: 241 +content-type: application/json +date: Fri, 06 Dec 2024 12:02:56 GMT +server: gunicorn/19.9.0 +``` + +Now we can see a `User-Agent`, an `Accept-Encoding` added implicitly by the client, and the typical response headers. + +Client interceptors (which we added first within the `interceptors` block) see the request just as given to the client, +and the response after the client does some changes (e.g. decompression). Backend interceptors see the request +just before sending it, and the response just as received. + +Note that Kotlin's [`Interceptor`](https://mizosoft.github.io/methanol/api/latest/methanol.kotlin/com.github.mizosoft.methanol.kotlin/-interceptor/index.html?query=interface%20Interceptor) +is a different interface from core library's [`Methanol.Interceptor`](https://mizosoft.github.io/methanol/api/latest/methanol/com/github/mizosoft/methanol/Methanol.Interceptor.html). +This is so that Kotlin's interceptors can support coroutines; they are functionally the same, however. +You can learn more about interceptors [here](https://mizosoft.github.io/methanol/interceptors/#client-interceptors). + +[Runnable Example](https://github.com/mizosoft/methanol/blob/master/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/Interceptors.kt). + +#### Coroutines + +HTTP verb functions are suspending, which is also the case with `Interceptor::intercept`. In fact, the entire interceptor chain +along with the ultimate HTTP call all share the same [coroutine context](https://kotlinlang.org/docs/coroutine-context-and-dispatchers.html). This implies that cancelling the HTTP call also cancels whatever some interceptor is doing. + +```kotlin +val client = Client { + interceptors { + +object : Interceptor { + override suspend fun intercept( + request: Request, + chain: Interceptor.Chain + ): Response { + println("Invoking interceptor with ${coroutineContext[CoroutineName]}") + return try { + delay(1000L) + chain.forward(request) + } catch (e: CancellationException) { + println("Cancelled interceptor") + throw e + } + } + } + } + + adapterCodec { + basic() + } +} + +fun run() { + runBlocking(CoroutineName("MyCoroutine")) { + val job = launch { + client.get("https://httpbin.org/get") + } + delay(500) + job.cancel() + } +} +``` + +Running the code gives the following output. + +``` +Calling interceptor with CoroutineName(MyCoroutine) +Cancelled interceptor +``` diff --git a/methanol-moshi/README.md b/methanol-moshi/README.md index 2359e36c8..a11c79759 100644 --- a/methanol-moshi/README.md +++ b/methanol-moshi/README.md @@ -1,3 +1,45 @@ # methanol-moshi -Coming Soon! +Adapters for JSON using [moshi](https://github.com/square/moshi). + +## Installation + +### Gradle + +```gradle +implementation("com.github.mizosoft.methanol:methanol-moshi:1.7.0") +``` + +### Maven + +```xml + + com.github.mizosoft.methanol + methanol-moshi + 1.7.0 + +``` + +## Usage + +```kotlin +val moshi: Moshi = Moshi.Builder().build() +val client = Client { + adapterCodec { + +MoshiAdapter.Encoder(moshi, MediaType.APPLICATION_JSON) + +MoshiAdapter.Decoder(moshi, MediaType.APPLICATION_JSON) + } +} + +data class Person(val name: String) + +var bruceLee = Person("Bruce Lee") +val response: Response = client.post(".../echo") { + body(bruceLee, MediaType.APPLICATION_JSON) +} +assertThat(response.body()).isEqualTo(bruceLee) +``` + +## Legacy Adapters + +See [Legacy Adapters](https://mizosoft.github.io/methanol/legacy_adapters/) diff --git a/methanol-protobuf/README.md b/methanol-protobuf/README.md index 4c7a36459..4f6f68a79 100644 --- a/methanol-protobuf/README.md +++ b/methanol-protobuf/README.md @@ -12,7 +12,7 @@ Any subtype of `MessageLite` is supported by encoders & decoders. Decoders can o ### Gradle ```gradle -implementation 'com.github.mizosoft.methanol:methanol-protobuf:1.7.0' +implementation("com.github.mizosoft.methanol:methanol-protobuf:1.7.0") ``` ### Maven @@ -25,159 +25,25 @@ implementation 'com.github.mizosoft.methanol:methanol-protobuf:1.7.0' ``` -The adapters need to be registered as [service providers][serviceloader_javadoc] so Methanol knows they're there. -The way this is done depends on your project setup. - -### Module Path - -Follow these steps if your project uses the Java module system. - -1. Add this class to your module: - - ```java - public class ProtobufProviders { - public static class EncoderProvider { - public static BodyAdapter.Encoder provider() { - return ProtobufAdapterFactory.createEncoder(); - } - } - - public static class DecoderProvider { - public static BodyAdapter.Decoder provider() { - return ProtobufAdapterFactory.createDecoder(); - } - } - } - ``` - -2. Add the corresponding provider declarations in your `module-info.java` file. - - ```java - requires methanol.adapter.protobuf; - - provides BodyAdapter.Encoder with ProtobufProviders.EncoderProvider; - provides BodyAdapter.Decoder with ProtobufProviders.DecoderProvider; - ``` - -### Classpath - -Registering adapters from the classpath requires declaring the implementation classes in provider-configuration -files that are bundled with your JAR. You'll first need to implement delegating `Encoder` & `Decoder` -that forward to the instances created by `ProtobufAdapterFactory`. Extending from `ForwardingEncoder` & -`ForwardingDecoder` makes this easier. - -You can use Google's [AutoService][autoservice] to generate the provider-configuration files automatically, -so you won't bother writing them. - -#### Using AutoService - -First, [install AutoService][autoservice_getting_started]. - -##### Gradle - -```gradle -implementation "com.google.auto.service:auto-service-annotations:$autoServiceVersion" -annotationProcessor "com.google.auto.service:auto-service:$autoServiceVersion" -``` - -##### Maven - -```xml - - com.google.auto.service - auto-service-annotations - ${autoServiceVersion} - -``` - -Configure the annotation processor with the compiler plugin. - -```xml - - maven-compiler-plugin - - - - com.google.auto.service - auto-service - ${autoServiceVersion} - - - - -``` - -Next, add this class to your project: - -```java -public class ProtobufAdapters { - @AutoService(BodyAdapter.Encoder.class) - public static class Encoder extends ForwardingEncoder { - public Encoder() { - super(ProtobufAdapterFactory.createEncoder()); - } - } - - @AutoService(BodyAdapter.Decoder.class) - public static class Decoder extends ForwardingDecoder { - public Decoder() { - super(ProtobufAdapterFactory.createDecoder()); - } - } -} -``` - -#### Manual Configuration - -You can also write the configuration files manually. First, add this class to your project: +## Usage ```java -public class ProtobufAdapters { - public static class Decoder extends ForwardingDecoder { - public Decoder() { - super(ProtobufAdapterFactory.createDecoder()); - } - } - - public static class Encoder extends ForwardingEncoder { - public Encoder() { - super(ProtobufAdapterFactory.createEncoder()); - } - } -} -``` - -Next, create two provider-configuration files in the resource directory: `META-INF/services`, -one for the encoder and the other for the decoder. Each file must contain the fully qualified -name of the implementation class. - -Let's say the above class is in a package named `com.example`. You'll want to have one file for the -encoder named: - -``` -META-INF/services/com.github.mizosoft.methanol.BodyAdapter$Encoder -``` - -and contains the following line: - -``` -com.example.ProtobufAdapters$Encoder -``` - -Similarly, the decoder's file is named: - -``` -META-INF/services/com.github.mizosoft.methanol.BodyAdapter$Decoder -``` - -and contains: - -``` -com.example.ProtobufAdapters$Decoder +var adapterCodec = + AdapterCodec.newBuilder() + .encoder(ProtobufAdapterFactory.createEncoder()) + .decoder(ProtobufAdapterFactory.createDecoder()) + .build(); +var client = + Methanol.newBuilder() + .adapterCodec(adapterCodec) + .build(); + +var bruceLee = Person.newBuilder().setName("Bruce Lee").build(); +var response = client.send( + MutableRequest.POST(".../echo", bruceLee, MediaType.APPLICATION_XML), + MyMessage.class); +assertThat(response.body()).isEqualTo(bruceLee); ``` [protocol_buffers]: https://developers.google.com/protocol-buffers [message_extensions]: https://developers.google.com/protocol-buffers/docs/proto#extensions -[autoservice]: https://github.com/google/auto/tree/master/service -[autoservice_getting_started]: https://github.com/google/auto/tree/master/service#getting-started -[serviceloader_javadoc]: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/ServiceLoader.html diff --git a/methanol-redis/README.md b/methanol-redis/README.md index 6104d4fd3..c92c210cc 100644 --- a/methanol-redis/README.md +++ b/methanol-redis/README.md @@ -1,3 +1,82 @@ # methanol-redis -Coming Soon! +Redis storage extension for the HTTP cache, built on [lettuce](https://github.com/redis/lettuce). The extension supports +Redis Standalone & Redis Cluster setups. + +## Installation + +### Gradle + +```kotlin +implementation("com.github.mizosoft.methanol:methanol-redis:1.8.0") +``` + +### Maven + +```xml + + com.github.mizosoft.methanol + methanol-redis + 1.8.0 + +``` + +## Usage + +Plug in a [RedisStorageExtension](https://mizosoft.github.io/methanol/api/latest/methanol.redis/com/github/mizosoft/methanol/store/redis/RedisStorageExtension.html) instance. +The easiest way is to create one from a `RedisURI`. + +### Redis Standalone + +```java +var redisUrl = RedisURI.create("redis://localhost:6379"); +var cache = + HttpCache.newBuilder() + .cacheOn( + RedisStorageExtension.newBuilder() + .standalone(redisUri) + .build()) + .build(); +var client = Methanol.newBuilder().cache(cache).build(); +``` + +### Redis Cluster + +```java +var redisUris = List.of( + RedisURI.create("redis://localhost:6379"), + RedisURI.create("redis://localhost:6380"), + ...); +var cache = + HttpCache.newBuilder() + .cacheOn( + RedisStorageExtension.newBuilder() + .cluster(redisUris) + .build()) + .build(); +var client = Methanol.newBuilder().cache(cache).build(); +``` + +Don't forget to close the cache! + +You can also pass your own `Redis[Cluster]Client`, but then you'll be responsible for its closure. + +### `RedisConnectionProvider` + +Another way to create a `RedisStorageExtension` is to provide your implementation of [`RedisConnectionProvider`](https://mizosoft.github.io/methanol/api/latest/methanol.redis/com/github/mizosoft/methanol/store/redis/RedisConnectionProvider.html), +which is an abstraction for the provision and release of redis connections. Currently, the implementation relies on +a single connection during lifetime of the cache, and releases it when the cache is closed. `RedisConnectionProvider::close` is also invoked when the cache is closed. + +### Timeouts + +For better memory efficiency, the extension implements stream semantics upon Redis. This means that whatever +read or written is seen as a stream of bytes that is acquired progressively as chunks, rather than first loaded entirely +in memory. As the program utilizing the cache can fail at any time, this requires having timeouts so that temporary +streams that are not utilized anymore are deleted. There are two: + +- `editorLockInactiveTtlSeconds`: The number of seconds an uncommitted entry that is still being edited can live during + an editor's inactivity. Default is `5` seconds. +- `staleEntryInactiveTtlSeconds`: the number of seconds a deleted entry is allowed to live during an inactivity from + a potential concurrent reader. Default is `3` seconds. + +You can customize these with `RedisStorageExtension.Builder`. diff --git a/methanol-redis/src/main/java/com/github/mizosoft/methanol/store/redis/RedisStorageExtension.java b/methanol-redis/src/main/java/com/github/mizosoft/methanol/store/redis/RedisStorageExtension.java index 54c3e378c..34f0b37bd 100644 --- a/methanol-redis/src/main/java/com/github/mizosoft/methanol/store/redis/RedisStorageExtension.java +++ b/methanol-redis/src/main/java/com/github/mizosoft/methanol/store/redis/RedisStorageExtension.java @@ -51,8 +51,8 @@ static Builder newBuilder() { /** A builder of {@code RedisStorageExtension}. */ final class Builder { - private static final int DEFAULT_EDITOR_LOCK_INACTIVE_TTL_SECONDS = 8; - private static final int DEFAULT_STALE_ENTRY_INACTIVE_TTL_SECONDS = 4; + private static final int DEFAULT_EDITOR_LOCK_INACTIVE_TTL_SECONDS = 5; + private static final int DEFAULT_STALE_ENTRY_INACTIVE_TTL_SECONDS = 3; private @MonotonicNonNull RedisStorageExtensionFactory factory; private int editorLockInactiveTtlSeconds = DEFAULT_EDITOR_LOCK_INACTIVE_TTL_SECONDS; diff --git a/methanol-samples/kotlin/build.gradle.kts b/methanol-samples/kotlin/build.gradle.kts new file mode 100644 index 000000000..191143cc0 --- /dev/null +++ b/methanol-samples/kotlin/build.gradle.kts @@ -0,0 +1,10 @@ +plugins { + id("conventions.kotlin-library") + kotlin("plugin.serialization") version "2.1.0" +} + +dependencies { + implementation(project(":methanol-kotlin")) + implementation(project(":methanol-redis")) + implementation(libs.kotlinx.serialization.json) +} diff --git a/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/CachingClient.kt b/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/CachingClient.kt new file mode 100644 index 000000000..76a6ba439 --- /dev/null +++ b/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/CachingClient.kt @@ -0,0 +1,56 @@ +/* + * Copyright (c) 2024 Moataz Abdelnasser + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package com.github.mizosoft.methanol.samples.kotlin + +import com.github.mizosoft.methanol.CacheAwareResponse +import com.github.mizosoft.methanol.kotlin.BodyHandlers +import com.github.mizosoft.methanol.kotlin.Client +import com.github.mizosoft.methanol.kotlin.close +import com.github.mizosoft.methanol.kotlin.get +import java.nio.file.Path +import kotlin.time.Duration.Companion.seconds + +object CachingClient { + val client = Client { + userAgent("Chuck Norris") + cache { + onDisk(Path.of(".cache"), 500 * 1024 * 1024) // Occupy at most 500Mb on disk. + } + } + + suspend fun run() { + val response = + client.get("https://i.imgur.com/V79ulbT.gif", BodyHandlers.ofFile(Path.of("popcat.gif"))) { + cacheControl { + maxAge(5.seconds) // Override server's max-age. + } + } as CacheAwareResponse + println( + "$response - ${response.cacheStatus()} (Cached for ${response.headers()["Age"].firstOrNull() ?: -1} seconds)" + ) + } + + fun closeCache() { + client.caches().close() + } +} diff --git a/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/Coroutines.kt b/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/Coroutines.kt new file mode 100644 index 000000000..04a9ff3b7 --- /dev/null +++ b/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/Coroutines.kt @@ -0,0 +1,49 @@ +package com.github.mizosoft.methanol.samples.kotlin + +import com.github.mizosoft.methanol.kotlin.Client +import com.github.mizosoft.methanol.kotlin.Interceptor +import com.github.mizosoft.methanol.kotlin.Request +import com.github.mizosoft.methanol.kotlin.Response +import com.github.mizosoft.methanol.kotlin.get +import kotlinx.coroutines.CancellationException +import kotlinx.coroutines.CoroutineName +import kotlinx.coroutines.async +import kotlinx.coroutines.delay +import kotlinx.coroutines.runBlocking +import kotlin.coroutines.coroutineContext + +object Coroutines { + val client = Client { + interceptors { + +object : Interceptor { + override suspend fun intercept( + request: Request, + chain: Interceptor.Chain + ): Response { + println("Calling Interceptor::intercept with CoroutineName: ${coroutineContext[CoroutineName]}") + return try { + delay(1000L) + chain.forward(request) + } catch (e: CancellationException) { + println("Cancelled") + throw e + } + } + } + } + + adapterCodec { + basic() + } + } + + fun run() { + runBlocking(CoroutineName("MyCoroutine")) { + val deferred = async { + client.get("https://httpbin.org/get") + } + delay(500) + deferred.cancel() + } + } +} diff --git a/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/GetPostJson.kt b/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/GetPostJson.kt new file mode 100644 index 000000000..3beee614d --- /dev/null +++ b/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/GetPostJson.kt @@ -0,0 +1,60 @@ +package com.github.mizosoft.methanol.samples.kotlin + +import com.github.mizosoft.methanol.MediaType +import com.github.mizosoft.methanol.kotlin.* +import kotlinx.serialization.SerialName +import kotlinx.serialization.Serializable +import kotlinx.serialization.json.Json + +object GetPostJson { + val client = Client { + baseUri("https://api.github.com/") + defaultHeaders { + "Accept" to "application/vnd.github+json" + "X-GitHub-Api-Version" to "2022-11-28" + } + adapterCodec { + basic() + +KotlinAdapter.Encoder( + Json, MediaType.APPLICATION_JSON + ) + +KotlinAdapter.Decoder(Json { + ignoreUnknownKeys = true // For brevity, we'll skip most fields. + }, MediaType.APPLICATION_JSON) + } + } + + @Serializable + data class Repository( + val description: String, + @SerialName("full_name") val fullName: String + ) + + suspend fun runGet() { + val response = client.get>("users/mizosoft/starred?per_page=10") + response.body().forEach { repo -> + println("${repo.fullName}\n\t${repo.description}") + } + } + + @Serializable + data class Markdown( + val text: String, + val context: String, + val mode: String + ) + + suspend fun runPost() { + val response: Response = client.post("markdown") { + body( + Markdown( + "this code very fast: #437", + "torvalds/linux", + "gfm" + ), + MediaType.APPLICATION_JSON + ) + } + println(response.body()) + } +} diff --git a/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/GetPostString.kt b/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/GetPostString.kt new file mode 100644 index 000000000..860f074c3 --- /dev/null +++ b/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/GetPostString.kt @@ -0,0 +1,31 @@ +package com.github.mizosoft.methanol.samples.kotlin + +import com.github.mizosoft.methanol.MediaType +import com.github.mizosoft.methanol.kotlin.* + +object GetPostString { + val client = Client { + adapterCodec { + basic() + } + } + + suspend fun runGet() { + val response = client.get("https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt") + require(response.isSuccessful()) { "Unsuccessful response: $response - ${response.body()}" } + println(response.body()) + } + + suspend fun runPost() { + val response: Response = client.post("https://api.github.com/markdown/raw") { + body( + """ + > He who has a ***why*** to live can bear almost any ***how***. + > - Friedrich Nietzsche + """.trimIndent(), + MediaType.TEXT_MARKDOWN + ) + } + println(response.body()) + } +} diff --git a/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/Interceptors.kt b/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/Interceptors.kt new file mode 100644 index 000000000..d469919c9 --- /dev/null +++ b/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/Interceptors.kt @@ -0,0 +1,74 @@ +/* + * Copyright (c) 2024 Moataz Abdelnasser + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package com.github.mizosoft.methanol.samples.kotlin + +import com.github.mizosoft.methanol.MediaType +import com.github.mizosoft.methanol.kotlin.* +import java.lang.System.Logger.Level +import java.util.concurrent.atomic.AtomicInteger +import kotlin.time.Duration.Companion.milliseconds + +object Interceptors { + object LoggingInterceptor : Interceptor { + val logger: System.Logger = System.getLogger(LoggingInterceptor::class.simpleName) + val requestIdGenerator = AtomicInteger() + + override suspend fun intercept( + request: Request, + chain: Interceptor.Chain + ): Response { + val requestId = requestIdGenerator.getAndIncrement() + val start = System.currentTimeMillis() + logger.log(Level.INFO) { + "$requestId: sending $request \n${request.headers().toHttpString()}" + } + + return chain.forward(request).also { response -> + logger.log(Level.INFO) { + "$requestId: received $response in ${(System.currentTimeMillis() - start).milliseconds} \n" + + request.headers().toHttpString() + } + require(response.isSuccessful()) { "Unsuccessful response: $response" } + } + } + } + + val client = Client { + interceptors { + +LoggingInterceptor + } + + userAgent("Dave Bautista") + adapterCodec { + basic() + } + } + + suspend fun run() { + client.get("https://httpbin.org/gzip") { + headers { + "Accept" to MediaType.APPLICATION_OCTET_STREAM.toString() + } + } + } +} diff --git a/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/MultipartAndFormUploads.kt b/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/MultipartAndFormUploads.kt new file mode 100644 index 000000000..c24eba159 --- /dev/null +++ b/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/MultipartAndFormUploads.kt @@ -0,0 +1,67 @@ +package com.github.mizosoft.methanol.samples.kotlin + +import com.github.mizosoft.methanol.MediaType +import com.github.mizosoft.methanol.kotlin.Client +import com.github.mizosoft.methanol.kotlin.KotlinAdapter +import com.github.mizosoft.methanol.kotlin.Response +import com.github.mizosoft.methanol.kotlin.post +import kotlinx.serialization.Serializable +import kotlinx.serialization.json.Json +import java.nio.file.Path +import kotlin.io.encoding.Base64 +import kotlin.io.encoding.ExperimentalEncodingApi +import kotlin.io.path.readBytes + +object MultipartAndFormUploads { + // You can get your own clien ID here: https://api.imgur.com/oauth2/addclient. + val imgurClientId = System.getProperty("imgur.client.id") + + val client = Client { + baseUri("https://api.imgur.com/3/") + defaultHeaders { + "Authorization" to "Client-ID $imgurClientId" + } + adapterCodec { + basic() + +KotlinAdapter.Decoder(Json { + ignoreUnknownKeys = true + }, MediaType.APPLICATION_JSON) + } + } + + @Serializable + data class ImgurResponse(val status: Int, val success: Boolean, val data: T?) + + @Serializable + data class Image(val link: String) + + suspend fun multipartUpload() { + val response: Response> = client.post("image") { + multipartBody { + "image" to Path.of("images/popcat.gif") // File's Content-Type will be looked-up automatically. + "title" to "PopCat" + "description" to "A cat that pops" + } + } + require(response.body().success) { + "Unsuccessful response: $response - ${response.body()}" + } + println("Uploaded: ${response.body().data!!.link}") + } + + @OptIn(ExperimentalEncodingApi::class) // In order to use Kotlin's Base64. + suspend fun formUpload() { + val response: Response> = client.post("image") { + formBody { + "image" to Base64.Default.encode(Path.of("images/popcat.gif").readBytes()) + "type" to "base64" + "title" to "PopCat" + "description" to "A cat that pops" + } + } + require(response.body().success) { + "Unsuccessful response: $response - ${response.body()}" + } + println("Uploaded: ${response.body().data!!.link}") + } +} diff --git a/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/main.kt b/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/main.kt new file mode 100644 index 000000000..325757a60 --- /dev/null +++ b/methanol-samples/kotlin/src/main/kotlin/com/github/mizosoft/methanol/samples/kotlin/main.kt @@ -0,0 +1,27 @@ +/* + * Copyright (c) 2024 Moataz Abdelnasser + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ + +package com.github.mizosoft.methanol.samples.kotlin + +suspend fun main() { + // TODO run the examples. +} diff --git a/methanol/src/main/java/com/github/mizosoft/methanol/Methanol.java b/methanol/src/main/java/com/github/mizosoft/methanol/Methanol.java index 966a69b43..c17725d8c 100644 --- a/methanol/src/main/java/com/github/mizosoft/methanol/Methanol.java +++ b/methanol/src/main/java/com/github/mizosoft/methanol/Methanol.java @@ -589,7 +589,7 @@ public void close() { } } - /** An object that intercepts requests being sent over a {@code Methanol} client. */ + /** An object that intercepts requests before being sent and responses before being returned. */ public interface Interceptor { /** diff --git a/mkdocs.yml b/mkdocs.yml index ae6a881c2..0f9f776d0 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -64,15 +64,14 @@ nav: - 🔗 Javadoc: api/latest/ - Documentation: - Methanol HttpClient: methanol_httpclient.md - - Object Mapping: object_mapping.md + - Object Mapping: adapters.md - Interceptors: interceptors.md - Caching: caching.md - Decompression: decompression.md - Media Types: media_types.md - Multipart & Forms: multipart_and_forms.md - Progress Tracking: progress_tracking.md - - Interruptible Reading: interruptible_reading.md - - WritableBodyPublisher: writablebodypublisher.md + - Streaming Requests: streaming_requests.md - Kotlin DSL: kotlin.md - Redis: redis.md - Brotli: brotli.md @@ -84,5 +83,6 @@ nav: - methanol-jaxb-jakarta: adapters/jaxb_jakarta.md - methanol-protobuf: adapters/protobuf.md - methanol-moshi: adapters/moshi.md + - Legacy Adapters: legacy_adapters.md - Benchmarks: benchmarks.md - Change Log: CHANGELOG.md diff --git a/settings.gradle.kts b/settings.gradle.kts index d8db9df47..d536b2738 100644 --- a/settings.gradle.kts +++ b/settings.gradle.kts @@ -18,9 +18,11 @@ include("methanol-samples") include("methanol-samples:crawler") include("methanol-samples:download-progress") include("methanol-samples:upload-progress") +include("methanol-samples:kotlin") include("spring-boot-test") include("methanol-redis") include("methanol-kotlin") +include("methanol-kotlin-examples") include("methanol-moshi") // Load local properties while giving precedence to properties defined through CLI.