Observing an API JSON content payload

[mwpowellhtx]

Question, I have some API which I would like to observe. Most of the time, the API yields a "simple" result, one request, one response, some JSON content which I may iterate over. So I want to observe that collection, or items in the collection, as deserialized from the JSON.

Sometimes the data set may span over one or more requests. Some requests, array data JSON deserialized in each corresponding response. Pagination, after a sort, is governed by content deserialized from the response, number of pages, items per page, total number of items, and so on.

How would I observe that pattern? Or would it rather be an observer of an observer, if that makes sense?

Thinking what the subscriber might be, something like IObserver<IReadOnlyCollection<T>> versus IObserver<T>?

Clarification would be helpful how these patterns might be adapted opposite an Observable pattern.

Cheers thank you.

[mwpowellhtx]

Could further clarify a bit. Internally, my data source is HTTP request based. But it manages the authentication aspects, tokens or what not, Uri, and other aspects.

It's a bit complicated, not really something I can easily contain, to my knowledge. Unless there is an easy way to not only configure the outgoing request, including request content headers, authentication tokens or what not. Then I expect the response would be rather formulaic, pretty easy to digest afterwards. Request url is also cobbled together based upon that authentication.

Anyway, really, I just need the observable pattern wrapping the overall engagement, request, get the response back, parse through the JSON content. The benefit I aim to achieve is threading, obviously, freeing up the UI/UX responsiveness, until the last possible moment when the result is ready to be presented to the UI thread for its merging reconciliation process.

I wonder if there are web or HTTP based Observable extensions.

[regolas]

The pattern your describing is more of a "composable observer" rather than a strict "observer of an observer" Using IObserver<IReadOnlyCollection<T>> vs IObserver<T> depends on your specific use case Collection-level: When you want to process batches Item-level: When you want to process individual items There is public class ApiDataObserver<T> you can create and then build the observer from that You can then look simply or with more advanced detail, subscribe and so on. I have some sample code if you want to see a an example. Art

…

On Wed, Mar 26, 2025 at 12:45 PM Michael W Powell ***@***.***> wrote: Could further clarify a bit. Internally, my data source is HTTP request based. But it manages the *authentication* aspects, *tokens* or what not, Uri, and other aspects. It's a bit complicated, not really something I can easily contain, to my knowledge. Unless there is an easy way to not only configure the outgoing request, including request content headers, authentication tokens or what not. Then I expect the response would be rather formulaic, pretty easy to digest afterwards. Request url is also cobbled together based upon that authentication. Anyway, really, I just need the observable pattern wrapping the overall engagement, request, get the response back, parse through the JSON content. The benefit I aim to achieve is threading, obviously, freeing up the UI/UX responsiveness, until the last possible moment when the result is ready to be presented to the UI thread for its merging reconciliation process. I wonder if there are web or HTTP based Observable extensions. — Reply to this email directly, view it on GitHub <#2192 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADGQ6PTUSXGST2R7YUGAWBT2WLYUFAVCNFSM6AAAAABZ3BR4TGVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENRTGI2DQNY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[mwpowellhtx]

If I provide values FromAsync, do I still need to signal Subject<T> the relevant messages, next, completed, error, and so on? Or is FromAsync scaffold doing that work for me?

Thanks...

[mwpowellhtx]

Further from that, actually, what is the "unit of work" or measurement being signaled by next, completed, error, etc. Each item in the overall sequence?

[mwpowellhtx]

I think I'm grasping it. Near enough approximation, instead of values mapping to Newtonsoft.Json.Linq JArray, picture an API response JSON content parsed to corresponding, either JArray, sometimes JObject, or what not. Generically, I actually constrain that in terms of where TContainer : JContainer, which is either of those.

private static void VerifyObservableFrom(params int[] values)
{
    Task<IReadOnlyCollection<JArray>> GetDataSourceAsync()
    {
        IEnumerable<int[]> GetAllValues()
        {
            yield return values;
            yield return values.Select(x => x + 1).ToArray();
            yield return values.Select(x => x + 2).ToArray();
        }

        return Task.FromResult<IReadOnlyCollection<JArray>>([.. GetAllValues().Select(x => new JArray(x))]);
    }

    static void OnValues(IEnumerable<int[]> values)
    {
        Debug.WriteLine($"Observable {nameof(values)}: {Join(", ", values.SelectMany(x => x).Select(x => $"{x}"))}");
    }

    using var sub = Observable
        .FromAsync(GetDataSourceAsync)
        .Select(jArrayColl => jArrayColl.Select(jArray => jArray.Select(x => x.Value<int>()).ToArray()))
        .Subscribe(OnValues);
}

After parsing the JSON to token, then I Select(...) project the response that I actually want, and pass it along to subs, or pub, or whatever else I need to do.

[mwpowellhtx]

So in the above case, if I have a series of request-response pairs until the set is complete, then I gather those up first, then move along from there. The projection then is concerned about doing something with the JSON parsed instances.

[mwpowellhtx]

I got this one sorted out. Works pretty well.