dynamic_DAG

Intro

The DAG builder constructs sub-graphs based on the current entry_func.

This eliminates complex if/else logic within EnvState-s to avoid python restarts for specialized modes like EntryFunc.func_start_app or EntryFunc.func_call_lib.

For example:

  • calling EntryFunc.func_start_app does not require most of venv creation and population steps

  • calling EntryFunc.func_call_lib must happen with no python restarts

  • both EntryFunc.func_start_app and EntryFunc.func_call_lib should not process any CLI args

  • while EntryFunc.func_boot_env should process CLI args, run through python restarts, and create venv

See also a more specific case: lib_access_to_config_data

DAG sub-graphs

Maintain a single EnvState list representing the "maximum" potential DAG.

Specialized modes simply use reduced sub-sets of this list.

DAG addressing

Use a 1-dimensional list of names (EnvState enum) to address DAG nodes.

But an N-dimensional space (one of the enums listed in GraphCoordinates) to address DAG sub-graphs.

In other words:

  • EnvState enum items are 1-dimensional space.

  • Bootstrap_* classes live in N-dimensional space (mapped from 1-dimensional to N-dimensional by factories).

Factory mapping

Each EnvState maps into a factory.

The factory instantiates one of the Bootstrap_* classes (based on GraphCoordinates).

The Bootstrap_* class then identifies its own dependencies (EnvState enums) to be instantiated via their own factories.

Dynamic DAG build

The full DAG is never constructed at once (statically).

Instead, DAG is built backwards from a specific target EnvState.

Multiple targets

Since GraphCoordinates are unknown until discovered (through environment variables or CLI args), the DAG sets new targets during execution.

For example, the role of proto_main shifts to:

  • pulling an initial target EnvState.state_func_boot_env_executed

  • executing that partial DAG: EnvState.state_func_boot_env_executed re-pulls the rest via RunStrategy

  • using the resulting context to determine the next target

GraphCoordinates

Separate:

  • EntryFunc: known immediately via the entry point - see entry_func:

    • boot_env (CLI processing)

    • start_app (no CLI processing)

    • API library (no CLI processing, no python restarts)

    • direct proto_main execution (similar to env_bootstrapper)

  • SubCommand: known only after CLI argument processing - see sub_command

TODO: FT_77_15_06_50.dynamic_DAG.md: sub_command should affect how GraphCoordinates are set, but should not be a coordinate itself.

EnvState-s

State names should remain stable and known at the start of a StateStride, though they can be extended in an append-only fashion.

See state_dependency.

Open TODOs

  • TODO: FT_77_15_06_50.dynamic_DAG.md: Rename GraphCoordinates to refer to what is currently called EnvContext.

  • TODO: FT_77_15_06_50.dynamic_DAG.md: Maybe Env* prefix in EnvContext and EnvState should be removed from such classes and replaced by more meningful.

  • TODO: FT_77_15_06_50.dynamic_DAG.md: Phase 2: simplify complex Bootstrap_* for specific GraphCoordinates-s. Adjust factories.

  • TODO: FT_77_15_06_50.dynamic_DAG.md: support different DAG depending on the SubCommand.

  • What is the role of protoprimer.primer_kernel.RunStrategy?

    At the moment, there is single strategy ExitCodeReporter (see EnvState.state_func_boot_env_executed).

    Do we want to make use of RunStrategy beyond that degenerate single case?

  • ExecMode enum: add items, explain, add FT_*.