Suggestion to Mark Asynchronous Operations in the Type System

[QuentinJanuel]

Before proceeding, I want to acknowledge that I have read the rationale provided in the documentation regarding the decision not to implement the synchronicity of operations in the type system. For reference, here is the relevant explanation from the documentation:

In the Effect library, there is no built-in way to determine in advance whether an effect will execute synchronously or asynchronously. While this idea was considered in earlier versions of Effect, it was ultimately not implemented for a few important reasons:

1. Complexity: Introducing this feature to track sync/async behavior in the type system would make Effect more complex to use and limit its composability.
2. Safety Concerns: We experimented with different approaches to track asynchronous Effects, but they all resulted in a worse developer experience without significantly improving safety. Even with fully synchronous types, we needed to support a fromCallback combinator to work with APIs using Continuation-Passing Style (CPS). However, at the type level, it’s impossible to guarantee that such a function is always called immediately and not deferred.

That said, I have not been able to locate discussions on this topic and would like to reopen the debate. My intention is twofold: first, to gain a deeper understanding of the reasons behind this decision, and second, to explore whether it could be reconsidered if viable solutions are proposed.

---

Motivations for This Feature

We should not have to rely on internal knowledge of the library to use it safely. One example illustrating this point is the creation of a cache in the global scope. A common approach might look like this:

const myCachedEffect = pipe(
  myEffect,
  Effect.cached,
  Effect.runSync
);

This works because Effect.cached is synchronous. However, the type system does not guarantee this behavior, and an update could just make it asynchronous. If such a change were introduced, the type system would not flag the issue, increasing the likelihood of pushing a bug to production.

Regarding the safety concerns mentioned in the documentation, I believe these can be addressed. For instance, the fromCallback function could default to being marked as asynchronous, while an alternative fromSyncCallback function could explicitly handle synchronous callbacks. Users would then be responsible for ensuring that they only use fromSyncCallback with genuinely synchronous operations. Although misuse would remain possible, such errors would be easier to detect, aligning with existing Effect library conventions—e.g., promise vs. tryPromise:

Use Effect.promise when you are sure the operation will not reject.

Similarly, the documentation for a synchronous variant could state:

Use Effect.fromSyncCallback when you are sure the operation is synchronous.

---

Proposal to Address Complexity

I have experimented with a method for marking asynchronous effects that, in my view, neither increases library complexity nor compromises composability. However, I recognize that I may have overlooked some aspects, so I welcome feedback. The approach involves introducing an async service to mark asynchronous effects:

// Do not export this, as users should not provide synchronicity to async operations.
class InternalAsync extends Context.Tag("Async")<InternalAsync, {}>() {}

const asyncService = Context.make(InternalAsync, {});

// Export this type to allow users to write types involving Async if needed.
export type Async = InternalAsync;

// Mark all internal async effects as such.
// e.g.
export const sleep: (d: Duration.DurationInput) => Effect<void, never, Async>;

// Allow async run functions to handle effects marked as Async.
export const runPromise: <A, E>(e: Effect<A, E, Async>) => Promise<A>;

// Enforce that sync run functions only handle effects not marked as Async.
export const runSync: <A, E>(effect: Effect<A, E>) => A;

// Examples:

// Automatically marked as Async.
const testAsync = gen(function* () {
  yield* sleep("1 second");
  return "done";
});

runPromise(testAsync); // Works.
runSync(testAsync); // Fails.

const testSync = E.sync(() => "done");

await runPromise(testSync); // Works.
runSync(testSync); // Works.

---

I apologize if this specific approach has already been considered and rejected for reasons I have not yet encountered. Nonetheless, I am genuinely interested in hearing thoughts on this proposal. If there are compelling arguments against this approach, I would greatly appreciate the opportunity to learn from them. Thank you for your time!

[QuentinJanuel]

I experimented with this suggestion to explore its limits concerning the composability of Effect. I observed that we still maintain valid interoperability between types, as follows:

• Option<A> is a subtype of Effect<A, NoSuchElementException>.
• Either<A, E> is a subtype of Effect<A, E>.
• Effect<A, E, R> is a subtype of Effect<A, E, R | Async>.
• Effect<A, E, R> is a subtype of Stream<A, E, R>.
• Effect<A, E, R | Async> is a subtype of Stream<A, E, R | Async>.

In other words, Option and Either are synchronous Effects; synchronous Effects are also asynchronous Effects, and Effects are Streams with synchronicity preserved.

This seems like a sensible default, as it also enables distinguishing between synchronous and asynchronous Streams while maintaining compatibility with Effect. Here's an example of how it might look:

const s1 = Stream.make(1, 2, 3, 4); // : Stream<number>
const s2 = s1.pipe(Stream.schedule(Schedule.spaced("1 second"))); // : Stream<number, never, Async>

const e1 = Stream.runDrain(s1); // : Effect<void>
const e2 = Stream.runDrain(s2); // : Effect<void, never, Async>

That said, if for some reason it is preferable not to encode the synchronicity of streams in the type system, we can adopt a simpler approach by making Effect<A, E, R | Async> a subtype of Stream<A, E, R>:

// src/Stream.ts
declare module "./Effect.js" {
  interface Effect<A, E, R> extends Stream<A, E, Exclude<R, Async>> {}
}

Finally, I wanted to verify whether TypeScript is capable of expressing types that strictly enforce either synchronous or asynchronous effects, regardless of other requirements. It turns out this is entirely feasible:

type AsyncEffect<A, E, R> = R extends Async ? E.Effect<A, E, R> : never;
type SyncEffect<A, E, R> = E.Effect<A, E, Exclude<R, Async>>;

const anyAsyncEffect = <A, E, R>(e: AsyncEffect<A, E, R>) => e;
const anySyncEffect = <A, E, R>(e: SyncEffect<A, E, R>) => e;
const anyEffect = <A, E, R>(e: E.Effect<A, E, R>) => e;

anyAsyncEffect(testAsync);
anyAsyncEffect(testSync); // Fails.

anySyncEffect(testAsync); // Fails.
anySyncEffect(testSync);

anyEffect(testAsync);
anyEffect(testSync);

These type aliases, SyncEffect and AsyncEffect, demonstrate how we can maintain clarity without increasing complexity for the user. In practice, though, most functions users write for Effect should work regardless of synchronicity, so this might even less affect the user experience.

[tim-smart]

Yes it is possible to have some kind of Async effect tracking - you will still have some false positives where an async effect is actually synchronous.

But it adds more complexity than what it is worth. In the majority of cases you don't need to think about the nature of an Effect, and in my opinion the extra noise in the types is not worth the type safety benefits.

[QuentinJanuel]

I don’t see these so-called “false positives” as problematic. To me, Effect<A, E, R | Async> represents an effect that has the potential to perform asynchronous operations, not one that is necessarily asynchronous. There is no issue lifting a synchronous effect to an asynchronous one; the critical requirement is to prevent the reverse. My reasoning is that while it’s important to know with certainty that an effect is synchronous in specific contexts (like when using runSync), we never need to ensure that an effect is strictly asynchronous. For example, using runPromise with a synchronous effect poses no problems.

This is analogous to saying that Effect<void, MyError> could have “false positives” where the effect doesn’t fail. I would interpret this type as an effect that can fail, not one that necessarily does. Similarly, the potential for asynchronicity doesn’t imply it must be realized in all cases.

That said, I can appreciate the argument about complexity potentially outweighing the benefits. However, as you mentioned, in the majority of cases, we don’t need to concern ourselves with the synchronicity of an effect. A key advantage of this proposal is that it aligns with this principle: if you don’t care about the synchronicity, the default typing Effect<A, E, R> already represents an effect of any synchronicity. As a result, I believe the additional type annotations wouldn’t introduce significant noise in most scenarios.

[QuentinJanuel]

To draw an analogy, Effect<A, E, never> is like NonEmptyArray, while Effect<A, E, Async> is like Array. Arrays can be empty, but non-empty arrays are still valid elements of Array without being considered false positives. Similarly, an effect marked as potentially asynchronous (Effect<A, E, Async>) can encompass purely synchronous effects without any issues.

In most cases, we don’t need to explicitly work with NonEmptyArray, yet the Effect library incorporates it in many internal functions for the few cases where it proves useful. This precedent suggests that having more precise types for effects where synchronicity matters could be beneficial in similar scenarios.

From a terminology perspective, it might make sense to rename the current Effect to NonAsyncEffect and define a new Effect type alias as follows:

type Effect<A, E, R> = NonAsyncEffect<A, E, R | Async>;

This adjustment would better reflect the distinction between the base synchronous type and its asynchronous superset. Internal functions wouldn’t need to make this distinction immediately. We could gradually migrate functions known to be synchronous to use NonAsyncEffect, enhancing the library's precision over time.

Even if only half of the types are migrated initially, while the other half remain loose, it would still represent a significant improvement in clarity and safety. The gradual adoption of more precise typings would allow for incremental refinements without imposing an immediate overhaul on the library or its users.

[tim-smart]

I'm still not convinced it is worth it. It also draws too much attention to async vs sync effects which the developer shouldn't really care about.

I'll let another contributor weigh in before moving any further with this issue. I have a feeling this design has already been attempted before.

[QuentinJanuel]

I see. Well, anyway, I do agree—although it's a nice-to-have, I wouldn't consider it crucial, so maybe it's just not worth it indeed 😕

[mikearnaldi]

Given that Async would be a service it is possible to provide such service and erase it from the signature, effectively getting to Effect<A, E, R> with an operation that is indeed async so this wouldn't really add safety while limiting composability (e.g. retry and other combinators will need to make a choice), Also there are constructors that can be async but don't have to like Effect.async so marking everything will also be semantically wrong. There's also fundamental reasons why not to mark the effects as async, Effect is a unified model of computation that doesn't require function coloring (that has know drawbacks), usage of runSync should be consider an exception and by all means not the norm, in a normal program you should have a single run at the top which is a runMain not a runSync so we can consider using async ops and then using runSync to be a code defect and defects are not tracked by the effect type.

This topic has already been discussed quite a few times if you want to see the previous discussions feel free to search on our Discord server, in short we don't believe in and we won't evaluate an addition of an Async service type, if a user wants to add a marker to their effects it is fully possible to do it in user-land.

Closing as not planned

[mikearnaldi]

Re attempting the design before I can confirm that it was attempted, at some point Effect was Effect<S, R, E, A> with S marking the sync/async, this would be safe from accidental provisioning but it ended up making some combinators not precise, e.g. if fork is sync or async depends by the scheduler, which can also change during execution, so to be sure we should safely assume async but then almost every effect becomes async at some point at the type level (even if being sync)

[QuentinJanuel]

It’s perfectly fine to aim for being "as precise as possible." For example, with functions like Effect.retry or Effect.async, we are compelled to mark them as Async, and that’s entirely okay. Async doesn’t mean the effect must be asynchronous—just that it can be. This is why I believe renaming Effect to NonAsyncEffect and introducing a new Effect type alias would make more sense. Similar to the relationship between NonEmptyArray and Array, it’s reasonable to return Array by default when compile-time precision isn’t possible.

Additionally, I previously mentioned not exporting the Async service to prevent accidental provision of asynchronous behavior. However, even if users were to explicitly provide it, that would clearly be intentional and therefore not problematic, in my opinion.

That said, I appreciate the rationale behind your explanation—thank you for taking the time to elaborate!

[mattiamanzati]

You also need to remember that Effect can yield in order to avoid stack overflows and starving the thread.
That means that something like
new Array(100000).map((_, i) => i).reduce((i, c) => Effect.map(c, sum => sum + i), Effect.succeed(0))
may appear completely synchronous even at type level given the suggested approach.
It will definetely not be, as the Effect runtime will "yield" after a number of operations set by the scheduler as Mike said.

[QuentinJanuel]

Oh I did not know that, that explains a lot thank you! 🙏

[tim-smart]

may appear completely synchronous even at type level given the suggested approach.\nIt will definetely not be

In the case of Effect.runSync, it uses a sync scheduler to ensure yields happen synchronously too.

[QuentinJanuel]

In the case of Effect.runSync, it uses a sync scheduler to ensure yields happen synchronously too.

Oh, I see! That means a NonAsyncEffect could theoretically still be defined. Rather than representing an effect that will necessarily run synchronously, it would instead signify an effect that is safe to run synchronously

That still seems useful, even if it’s only for very specific cases