5.4 Four-Stage Upstream Lifecycle

5.4.1 Plan

Planning previews an upstream task:

• Queries the flow definition.
• Reads the actual files in the BioLine data directory.
• Generates a dry-run specification and binding report.
• Reports matched inputs and defaults that will be used.
• Does not create a BioLine pipeline.

5.4.2 Submit

Submission converts the approved plan into a real task:

• Generates the final specification.
• Creates the BioLine pipeline.
• Writes the pipelineId to .pipeline_id.
• Records default usage and missing optional inputs in the upstream status material.

5.4.3 Poll

Polling reads and records the running state:

• Queries the current pipeline on each polling cycle.
• Writes each response to reproducibility/poll_*.json.
• Aggregates multiple sample states in batch mode.

5.4.4 Finalize

Finalisation converts cloud outputs into a local structure that downstream analysis can read:

• Downloads BioLine outputs.
• Organises outputs under stage/<category>/ according to output_to_stage.
• Writes stage/manifest.json.
• Creates the .fkit_done marker.