Expand Disposable/AsyncDisposable to parallel Python's ExitStack/AsyncExitStack

[rbuckton]

I've been experimenting with a variation of Python's ExitStack that I mentioned here. I quite like the API it provides in tandem with using const when building up custom disposables, as it significantly helps with cleanup when something fails during construction:

class PluginHost {
  #disposables;
  #channel;
  #socket;

  constructor() {
    // create a DisposableStack that is disposed when the block exits.
    // if construction succeeds, we move everything out of `stack` and into
    // `#disposables` to be disposed later.
    using const stack = new DisposableStack();

    // create a IPC adapter around process.send/process.on("message").
    // when disposed, it unsubscribes from `process.on("message")`
    this.#channel = stack.use(new NodeProcessIpcChannelAdapter(process));
    
    // create a pseudo-websocket that sends and receives messages over
    // a NodeJS IPC channel
    this.#socket = stack.use(new NodePluginHostIpcSocket(this.#channel));

    // if we made it here, then there were no errors during construction and
    // we can safely move the disposables out of `stack` and into `#disposables`
    this.#disposables = stack.move();

    // if construction failed, then `stack` would be disposed before reaching the line above.
    // event handlers would be removed, allowing #channel and #socket to be GC'd
  }

  [Symbol.dispose]() {
    this.#disposables[Symbol.dispose]();
  }
}

I'm considering renaming the Disposable and AsyncDisposable globals in this proposal to DisposableStack and AsyncDisposableStack to more clearly denote the resource stack-like behavior and adding the use method (nee. enter, push, etc.) to allow new resources to be added, and the move method to move resources out of the stack and into a new one. Its a very convenient API when combined with using const.

Tentative API sketch (in TypeScript parlance):

interface Disposable {
  [Symbol.dispose](): void;
}

type DisposableLike = Disposable | (() => void) | null | undefined;

declare class DisposableStack {
  /**
   * Creates a new DisposableStack.
   * @param onDispose An optional callback to add as the first resource of the stack.
   */
  constructor(onDispose?: () => void);

  /**
   * Creates a DisposableStack from an iterable of disposable resources and/or disposal callbacks.
   */
  static from(iterable: Iterable<DisposableLike>): DisposableStack;

  /**
   * Add a disposable resource to the stack. 
   * @param resource A Disposable or a callback to execute on dispose, or null or undefined.
   * @returns The resource that was added.
   */
  use<T extends DisposableLike>(resource: T): T;
  /**
   * Add a resource to the stack with a callback to be executed when the resource is disposed. 
   * @param resource A value to track as if it were a disposable resource.
   * @param onDispose A callback that is evaluated when this DisposableStack is disposed. The resource
   * to be disposed is provided as the first argument.
   * @returns The resource that was added.
   */
  use<T>(resource: T, onDispose: (resource: T) => void): T;

  /**
   * Moves all resources out of this stack and into a new DisposableStack.
   */
  move(): DisposableStack;

  /**
   * Dispose all resources in this DisposableStack in the reverse order in which they were added.
   */
  [Symbol.dispose](): void;
}

interface AsyncDisposable {
  [Symbol.asyncDispose](): Promise<void>;
}

type AsyncDisposableLike = AsyncDisposable | Disposable | (() => void | PromiseLike<void>) | null | undefined;

declare class AsyncDisposableStack {
  /**
   * Creates a new AsyncDisposableStack.
   * @param onDispose An optional callback to add as the first resource of the stack.
   */
  constructor(onDispose?: () => void | PromiseLike<void>);

  /**
   * Creates an AsyncDisposableStack from an iterable of disposable resources and/or disposal callbacks.
   */
  static from(iterable: Iterable<AsyncDisposableLike | PromiseLike<AsyncDisposableLike>>): Promise<AsyncDisposableStack>;
  /**
   * Creates an AsyncDisposableStack from an async iterable of disposable resources and/or disposal callbacks.
   */
  static from(iterable: AsyncIterable<AsyncDisposableLike>): Promise<AsyncDisposableStack>;

  /**
   * Add a disposable resource to the stack. 
   * @param resource An AsyncDisposable, Disposable, or a callback to execute on dispose, or null or undefined.
   * @returns The resource that was added.
   */
  use<T extends AsyncDisposableLike>(resource: T): T;
  /**
   * Add a resource to the stack with a callback to be executed when the resource is disposed. 
   * @param resource A value to track as if it were a disposable resource.
   * @param onDispose A callback that is evaluated when this DisposableStack is disposed. The resource
   * to be disposed is provided as the first argument.
   * @returns The resource that was added.
   */
  use<T>(resource: T, onDispose: (resource: T) => void | PromiseLike<void>): T;

  /**
   * Moves all resources out of this stack and into a new AsyncDisposableStack.
   */
  move(): AsyncDisposableStack;

  /**
   * Dispose all resources in this AsyncDisposableStack in the reverse order in which they were added.
   */
  [Symbol.asyncDispose](): Promise<void>;
}

In particular:

• DisposableStack.use parallels ExitStack.enter_context/ExitStack.push/ExitStack.callback
  • NOTE: This proposal does not have an exact behavioral match for Python's __enter__
• DisposableStack.move parallels ExitStack.pop_all
• DisposableStack[Symbol.dispose] parallels ExitStack.close
• AsyncDisposableStack.use parallels AsyncExitStack.enter_async_context/AsyncExitStack.push_async_exit/AsyncExitStack.push_async_callback
  • NOTE: This proposal does not have an exact behavioral match for Python's __aenter__
• AsyncDisposableStack.move has no parallel in AsyncExitStack, but serves a similar purpose as ExitStack.pop_all
• AsyncDisposableStack[Symbol.asyncDispose] parallels AsyncExitStack.aclose

cc: @mhofman

[bergus]

Oh I'd love to have a stack in the standard library!

type DisposableLike = Disposable | (() => void) | null | undefined;
type AsyncDisposableLike = AsyncDisposable | Disposable | (() => void | PromiseLike<void>) | null | undefined;

I disagree that null and undefined should be allowed as arguments. If you try to use a resource and get back null or undefined, something went wrong - use() should throw an exception. With a stack, it's very easy to write code like const res = getResource(); if (res) stack.use(res);, we don't need to build this in. Perhaps a separate method const res = maybeUse(getResource())?
But I guess this needs to be consistent with the design decision whether using const x = null or using const x = { [Symbol.dispose]: null } should throw (which I'd also favour).

@param onDispose An optional callback to add as the first resource of the stack.

I see you took this from the original Disposable constructor. I'm not sure if makes a lot of sense on a stack? I'd rather keep these separated as Disposable and Disposable.Stack (same for AsyncDisposable and AsyncDisposable.Stack) - if using classes as namespaces is fine with the committee. Then the stack constructors wouldn't need to take any arguments.

// Creates a DisposableStack from an iterable of disposable resources and/or disposal callbacks.
static from(iterable: Iterable<DisposableLike>): DisposableStack;

Is this really a useful method? I can't currently imagine a good use case, but it seems like an easy way to shoot yourself in the foot with

using const stack = DisposableStack.from([getResourceA(), getResourceB()])

where the second allocation might throw. Or when the first returns an object that is not disposable and the DisposableStack.from might throw. Also how would you even access the resources on the stack variable? It seems like there is no way to get back the resources that you passed into stack.use() (or into from()) from the stack object.

 use<T>(resource: T, onDispose: (resource: T) => void): T;

💡 Oh I like that interface!

NOTE: This proposal does not have an exact behavioral match for Python's __enter__ or __aenter__

I don't think this is bad. Tbh in Python I've been confused when writing a context manager whether I should put the allocation in __init__ or in __enter__, and when using a context manager whether it is reusable (or even reentrant) or not.

Instead of some unclear things happening when an object is used as a disposable

const file = openFile();
{
    using const handle = file; // ???
    … // do something with the file handle
}
const lock = new Lock();
{
    using void lock; // wait, this acquired it???
    …
}

I much prefer being explicit

{
    using const file = openFile();
    … // do something with file
}
const lock = new Lock();
{
    using void lock.acquire();
    … // do something while the lock is acquired
}

(It's still up to the implementation of Lock whether acquire just does return this; and any locks are disposable at any time, or whether each acquire() call does return a new Acquisition object that you have to dispose to release the lock, but the usage is the same and quite obvious to me).

// Moves all resources out of this stack and into a new DisposableStack.
move(): DisposableStack;

I wonder if this should actually return a new DisposableStack.
A) Making DisposableStack subclassible would be rather ugly if move() had to return a this.constructor instance
B) The API would be just as nice if it simply returned a "disposer function", maybe even easier to use:

class PluginHost {
  #channel;
  #socket;
  [Symbol.dispose];
  constructor() {
    using const stack = new Disposable.Stack();
    this.#channel = stack.use(new NodeProcessIpcChannelAdapter(process));
    this.#socket = stack.use(new NodePluginHostIpcSocket(this.#channel));

    // if we made it here, then there were no errors during construction and
    // we can safely move the disposables out of `stack` and onto the plugin host itself:
    this[Symbol.dispose] = stack.move();
  }
}

You still could do this.#disposable = new Disposable(stack.move()) if you wanted to have an object.

[rbuckton]

Oh I'd love to have a stack in the standard library!

Just to clarify, if you are talking about a list-like stack, that's not what this is.

type DisposableLike = Disposable | (() => void) | null | undefined;
type AsyncDisposableLike = AsyncDisposable | Disposable | (() => void | PromiseLike<void>) | null | undefined;

I disagree that null and undefined should be allowed as arguments. If you try to use a resource and get back null or undefined, something went wrong - use() should throw an exception. With a stack, it's very easy to write code like const res = getResource(); if (res) stack.use(res);, we don't need to build this in. Perhaps a separate method const res = maybeUse(getResource())? But I guess this needs to be consistent with the design decision whether using const x = null or using const x = { [Symbol.dispose]: null } should throw (which I'd also favour).

This aligns with using const allowing null/undefined. Note that using const will throw for { [Symbol.dispose]: null }.

@param onDispose An optional callback to add as the first resource of the stack.

I see you took this from the original Disposable constructor. I'm not sure if makes a lot of sense on a stack?

I'm debating this. I found new Disposable(() => {}) to be a better UX than { [Symbol.dispose]: () => {} }, but it does feel a bit out of place on something named DisposableStack.

I'd rather keep these separated as Disposable and Disposable.Stack (same for AsyncDisposable and AsyncDisposable.Stack) - if using classes as namespaces is fine with the committee. Then the stack constructors wouldn't need to take any arguments.

Using classes as namespaces is something the committee has so far been against, having recently been opposed to ArrayBuffer.Resizable/SharedArrayBuffer.Growable in recent meetings.

// Creates a DisposableStack from an iterable of disposable resources and/or disposal callbacks.
static from(iterable: Iterable<DisposableLike>): DisposableStack;

Is this really a useful method? I can't currently imagine a good use case, but it seems like an easy way to shoot yourself in the foot with

using const stack = DisposableStack.from([getResourceA(), getResourceB()])

where the second allocation might throw. Or when the first returns an object that is not disposable and the DisposableStack.from might throw. Also how would you even access the resources on the stack variable? [...]

One purpose of .from is to provide an easy to use API that helps to avoid footguns when managing resources produced by an iterable/generator by ensuring that resources are correctly cleaned up when errors occur during iteration or of any resource in the iterable is not a valid disposable:

function* g() {
  yield getResourceA();
  yield getResourceB();
}
using const stack = DisposableStack.from(g());

It seems like there is no way to get back the resources that you passed into stack.use() (or into from()) from the stack object.

The resource passed into use is returned immediately, however, as the purpose of use is to track disposable effects:

using const stack = new DisposableStack();
const x = stack.use(getResourceX());
const y = stack.use(getResourceY());

There isn't a way to get them back from .from as there's no introspection of DisposableStack currently, and I'm not sure there needs to be. You would use from if you already have the disposables or don't need the values. Perhaps DisposableStack could have a [Symbol.iterator] as well, but I'm currently unconvinced.

NOTE: This proposal does not have an exact behavioral match for Python's __enter__ or __aenter__

I don't think this is bad. Tbh in Python I've been confused when writing a context manager whether I should put the allocation in __init__ or in __enter__, and when using a context manager whether it is reusable (or even reentrant) or not.

My preference is for an approach where the __enter__ behavior is an explicit invocation of a function or constructor, much like in your examples (though I would just have written using const void = new Lock(mutex) or using const void = mutex.lock() as opposed to splitting it into multiple steps).

// Moves all resources out of this stack and into a new DisposableStack.
move(): DisposableStack;

I wonder if this should actually return a new DisposableStack. A) Making DisposableStack subclassible would be rather ugly if move() had to return a this.constructor instance B) The API would be just as nice if it simply returned a "disposer function", maybe even easier to use: [...]

You still could do this.#disposable = new Disposable(stack.move()) if you wanted to have an object.

From my experimentation, having move() return a new DisposableStack is more convenient when composing resources. Reassigning Symbol.dispose seems to be a bit of an antipattern to me, as it would only be marginally safe to do so in a constructor. At any other time, you run the risk of a using const capturing the wrong [Symbol.dispose] method.

I don't imagine that subclassing DisposableStack will be any more or less onerous than subclassing ArrayBuffer or Promise.

[bergus]

Just to clarify, if you are talking about a list-like stack, that's not what this is.

Of course, I was referring to the DisposableStack that this proposal is concerned with, I'm totally happy with the push/pop methods of arrays for stack structures.

I'm debating this. I found new Disposable(() => {}) to be a better UX than { [Symbol.dispose]: () => {} }, but it does feel a bit out of place on something named DisposableStack.

Yeah, it does. On the other hand, I'm not sure how you intended the from and move methods to be implemented - would it be return new this(() => { … }), or const s = new this(); s.use(…); return s;?

Using classes as namespaces is something the committee has so far been against, having recently been opposed to ArrayBuffer.Resizable/SharedArrayBuffer.Growable in recent meetings.

Ah. I guess then we'd either have to introduce 4 new global bindings, or wait for builtin modules :-/

One purpose of .from is to provide an easy to use API that helps to avoid footguns when managing resources produced by an iterable/generator by ensuring that resources are correctly cleaned up when errors occur during iteration or of any resource in the iterable is not a valid disposable:

function* g() {
  yield getResourceA();
  yield getResourceB();
}
using const stack = DisposableStack.from(g());

Hm, I'm not convinced of that. I doubt there will be lots of resources produced by true iterators/generators. Why wouldn't you rather write

using const stack = new DisposableStack();
stack.use(getResourceA());
stack.use(getResourceB());

or even just

using void getResourceA();
using void getResourceB();

If it needs to be a function, I'd much rather have

function g() {
  using const stack = new DisposableStack();
  stack.use(getResourceA());
  stack.use(getResourceB());
  return stack.move();
}
using const stack = g();

than using a generator function. Accepting any iterable in DisposableStack.from(…), especially arrays, is a much bigger footgun in my eyes, it's so easy and inviting to use it in the wrong way. If you really do have a generator, you can always explicitly write

using const stack = new DisposableStack();
for (const res of g()) stack.use(res);

which isn't that much longer.

It seems like there is no way to get back the resources that you passed into stack.use() (or into from()) from the stack object.

There isn't a way to get them back from .from as there's no introspection of DisposableStack currently, and I'm not sure there needs to be. […] Perhaps DisposableStack could have a [Symbol.iterator] as well, but I'm currently unconvinced.

Exactly my thoughts. We could make it iterable, but I'd prefer it not to be. It's not meant to be a ResourceList, it's about tracking disposers.

The [move API] would be just as nice if it simply returned a "disposer function", maybe even easier to use: [...]

From my experimentation, having move() return a new DisposableStack is more convenient when composing resources. Reassigning Symbol.dispose seems to be a bit of an antipattern to me, as it would only be marginally safe to do so in a constructor. At any other time, you run the risk of a using const capturing the wrong [Symbol.dispose] method.

Yeah, you're right about the reassignment, but on the other hand I do see the primary (only?) use case of move() exactly in constructors and factory functions. Could we maybe have both?

function makePluginHost() {
  using const stack = new Disposable.Stack();
  return {
    channel: stack.use(new NodeProcessIpcChannelAdapter(process)),
    socket: stack.use(new NodePluginHostIpcSocket(this.#channel)),
    [Symbol.dispose]: stack.moveToFunction(),
  };
}

Or stack.moveTo({…}), working like Object.assign({…}, stack) but including the Symbol.dispose property?

I don't imagine that subclassing DisposableStack will be any more or less onerous than subclassing ArrayBuffer or Promise.

I've never been a proponent of having Promise be subclassible, imo it just makes the .then() implementation slow and prevents optimisations.
For subclassing DisposableStack, I guess there might actually be more use cases, but not necessarily good ones either. I'm more concerned with spec complexity here, what would it take to implement .move() if it could be called on a subclass instance? If we didn't have that, I imagine the method could just take the internal [[DisposableList]] slot and place it into the slot of a new instance.

[rbuckton]

I'm debating this. I found new Disposable(() => {}) to be a better UX than { [Symbol.dispose]: () => {} }, but it does feel a bit out of place on something named DisposableStack.

Yeah, it does. On the other hand, I'm not sure how you intended the from and move methods to be implemented - would it be return new this(() => { … }), or const s = new this(); s.use(…); return s;?

I'm not sure I understand what you mean here.

One purpose of .from is to provide an easy to use API that helps to avoid footguns when managing resources produced by an iterable/generator by ensuring that resources are correctly cleaned up when errors occur during iteration or of any resource in the iterable is not a valid disposable:

function* g() {
  yield getResourceA();
  yield getResourceB();
}
using const stack = DisposableStack.from(g());

Hm, I'm not convinced of that. I doubt there will be lots of resources produced by true iterators/generators. Why wouldn't you rather write
[...]
than using a generator function. Accepting any iterable in DisposableStack.from(…), especially arrays, is a much bigger footgun in my eyes, it's so easy and inviting to use it in the wrong way. If you really do have a generator, you can always explicitly write

I can see your point. One of the other motivations for .from is the Disposable.from API in VSCode, which is fairly heavily used. That said, it is a footgun to pass items from an array, since any error could result in resources not being freed. This isn't an issue so much for a VSCode Extension, since their extension host is out of process so they can restart it in the event of a crash.

There isn't a way to get them back from .from as there's no introspection of DisposableStack currently, and I'm not sure there needs to be. […] Perhaps DisposableStack could have a [Symbol.iterator] as well, but I'm currently unconvinced.

Exactly my thoughts. We could make it iterable, but I'd prefer it not to be. It's not meant to be a ResourceList, it's about tracking disposers.

I agree.

The [move API] would be just as nice if it simply returned a "disposer function", maybe even easier to use: [...]

From my experimentation, having move() return a new DisposableStack is more convenient when composing resources. Reassigning Symbol.dispose seems to be a bit of an antipattern to me, as it would only be marginally safe to do so in a constructor. At any other time, you run the risk of a using const capturing the wrong [Symbol.dispose] method.

Yeah, you're right about the reassignment, but on the other hand I do see the primary (only?) use case of move() exactly in constructors and factory functions. Could we maybe have both?

function makePluginHost() {
  using const stack = new Disposable.Stack();
  return {
    channel: stack.use(new NodeProcessIpcChannelAdapter(process)),
    socket: stack.use(new NodePluginHostIpcSocket(this.#channel)),
    [Symbol.dispose]: stack.moveToFunction(),
  };
}

Or stack.moveTo({…}), working like Object.assign({…}, stack) but including the Symbol.dispose property?

The Python documentation shows examples of ExitStack that do something like this:

with ExitStack() as stack:
  files = [stack.enter_context(open(fname)) for fname in filenames]
  close_files = stack.pop_all().close

We could add a bound .dispose method to DisposableStack for the function extraction case. While uncommon in the standard library,
there is precedent in builtins like Intl.NumberFormat.prototype.format which is bound during construction. That would let you do:

function makePluginHost() {
  using const stack = new DisposableStack();
  return {
    channel: stack.use(new NodeProcessIpcChannelAdapter(process)),
    socket: stack.use(new NodePluginHostIpcSocket(this.#channel)),
    [Symbol.dispose]: stack.move().dispose,
  };
}

I don't imagine that subclassing DisposableStack will be any more or less onerous than subclassing ArrayBuffer or Promise.

I've never been a proponent of having Promise be subclassible, imo it just makes the .then() implementation slow and prevents optimisations. For subclassing DisposableStack, I guess there might actually be more use cases, but not necessarily good ones either. I'm more concerned with spec complexity here, what would it take to implement .move() if it could be called on a subclass instance? If we didn't have that, I imagine the method could just take the internal [[DisposableList]] slot and place it into the slot of a new instance.

There seems to be a growing sentiment within a subset of TC39 that subclassing of builtins is problematic (specifically around Symbol.species-related security issues). It might be feasible to make DisposableStack not subclassible (i.e., have it throw if new.target is not %DisposableStack%, etc.).

[bergus]

We could add a bound .dispose method to DisposableStack for the function extraction case.

Oh, that's a perfect solution as well. Python has it easy with its autobound methods :-)

I'm not sure I understand what you mean here.

That was wrt subclassing. If move() does create a new instance, how does it install the disposables from the current stack in there? I can see basically three approaches there, which also depend on the constructor signature:

class DisposableStack
    #emptied = false;
    #entries = [];
    use(value, context = undefined) {
        if (this.#emptied) throw new Error("Already disposed or moved");
        if (typeof value == "function") {
            this.#entries.push({dispose: value, resource: context});
        } else if (value != null) {
            const dispose = value[Symbol.dispose];
            if (typeof dispose != "function") throw new Error("not a Disposable")
            this.#entries.push({dispose, resource: value});
        }
    }
    #dispose(...args) {
        if (!this.#entries.length) return;
        try {
             const { resource, dispose } = this.#entries.pop();
             %call(dispose, resource, ...args);
        } finally {
             this.#dispose(...args);
        }
    }
    [Symbol.dispose](...args) {
        if (this.#emptied) return;
        this.#emptied = true;
        this.#dispose(...args);
    }
    dispose = this[Symbol.dispose].bind(this);
    #empty() {
        if (this.#emptied) throw new Error("Already disposed or moved");
        this.#emptied = true;
    }
    // Variant move-A - constructor with argument
    move() {
        this.#empty();
        const C = %GetSpeciesConstructor(this);
        let called = false;
        return new C((...args) => {
             if (called) return;
             called = true;
             this.#dispose(...args);
        });
    }
    // Variant move-B - constructor without argument, but .use()
    move() {
        this.#empty();
        const C = %GetSpeciesConstructor(this);
        const result = new C();
        let called = false;
        result.use((...args) => {
             if (called) return;
             called = true;
             this.#dispose(...args);
        });
        return result;
    }
    // Variant move-C - no subclassing
    move() {
        this.#empty();
        const result = new DisposableStack();
        result.#entries.push(...this.#entries);
        this.#entries.length = 0;
        return result;
    }
}

[rbuckton]

I think it's more like this (based on 25.1.5.3 ArrayBuffer.prototype.slice (start, end)):

// Variant move-D: private slot (or field in this example) that is expected to have been initialized on the instance
move() {
  if (this.#disposed) throw new ReferenceError();
  const C = %GetSpeciesConstructor(this, %DisposableStack%);
  const newStack = new C();
  if (!(#entries in newStack)) throw new TypeError(); // RequireInternalSlot
  newStack.#entries = this.#entries;
  this.#entries = []; // NOTE: a DisposableStack is still usable after a move
  return newStack;
}

[rbuckton]

@bergus, @mhofman: I created #83 to amend the specification with DisposableStack/AsyncDisposableStack. Could you take a look?

[mhofman]

The example in the first post seem to imply a use case where the disposable resources are initialized in a block (the constructor), but their lifetime should actually extend past that block because the resources are saved outside the block into a longer lived container.

In my mind a block with using const implicitly adds the used disposables to a "hidden" stack scoped to the block. Something feels strange about having 2 ways to create and populate a stack, one syntactic and one API based, yet have no way to go between the 2. Should there be a way to "untether" the implicit stack created by a using block, aka move it's content to a stack object? Should there be a way to adopt a stack object into a block scope?