Nit: Could we call this checkpoint_dir ?

Also: Why does this default to your current directory ? I would've thought we default to no checkpointing unless a user explicitly requests to do so by providing a path to store the data.

Maybe this is a question of intent, but I imagine the users of this workflow infrastructure will always be saving their data. I would like to encourage that data always makes a trip to persistent storage before analysis and plotting to enforce separation of data generation from data processing. I've considered removing the return value from execute so users have to do it this way; but that's probably too drastic.

As such, the files aren't checkpoints(*) but rather the main point of the execute function.

(*) my view of checkpoint files is that they are a way of restarting a long computation from a known place, which is not the intent of the files written here.

Nit: missing "cg."

done

How does one identify runs if they lose the code after runtime ? Should we have a way to allow a user to specify a name they prefer ?

If you specify a run_id in the rt_config: QuantumRuntimeConfiguration it uses that.

Otherwise, you can look at your filesystem as the directory will be named according to the run_id.

If we expect this loop to grow in complexity the number of features/functionality that these Executables take on, do you think it might be worth structuring things a little more carefully. In this case something like:
allowing a user to provide a list of callbacks that we guarantee to call at the end of this loop iteration. These callbacks could include things like logging, saving, timing, or any other kind of unknown functionality one might want to put after each iteration here ?

Yes, yes, and yes.

The QuantumRuntimeConfiguration will learn new fields whose objects' methods serve as callbacks. Logging for sure will be factored out into a callback-like object.

My plan is to get this skeleton merged. The addition of callbacks with defaults and helper functions can be done stepwise and in a backwards-compatible manner.

The GroupResult only gets written once when it is empty ? Do we want another one after it's been filled ?

yeah this is weird. Let me try to write up the motivation for the current structure and think about how to make it less weird

Here's the motivation.

My proposed selection is to call cirq.to_json on the constituent parts of ExecutableGroupResult rather than ever saving the whole ExecutableGroupResult object. This is because of the different "lifetimes" of the constituent objects.

In this case, I'd like to introduce a bookkeeping dataclass

@dataclasses.dataclass
class ExecutableGroupFilesystemRecord:
    runtime_configuration_fn: str
    shared_runtime_info_fn: str
    executable_result_fns: List[str]

    def load(self):
        return ExecutableGroupResult(
            runtime_configuration=cirq.read_json(self.runtime_configuration_fn),
            shared_runtime_info=cirq.read_json(self.shared_runtime_info_fn),
            executable_results=[cirq.read_json(exe_fn) for exe_fn in self.executable_result_fns],
        )

to keep things under control.

Would you like me to implement that in the current PR or as a follow-on PR? It might be easier to review if this is kept in its own follow-on PR.