4.3 Data Preparation and Input Specifications

4.3.1 Common Analysis Patterns

The path is determined by discovered BioLine Flows and Tools, not by a fixed local name or image catalogue.

• Full analysis: Send inputs to a suitable BioLine Flow. After upstream success is finalized, discover a downstream Tool for statistics, plotting, or reporting.
• Upstream-only analysis: Run a BioLine Flow and finalize its outputs into stage for later tasks.
• Downstream-only analysis: Discover a suitable BioLine Tool version for prepared abundance, pathway, ASV, sequence, and metadata files.

Check every downstream Tool version’s bilingual summary and config. The summary should declare analysis_lane: downstream and fixed_command: false; the config must not contain a fixed analysis command.

4.3.2 Input File Requirements

Use the actual Flow/Tool interface and documentation for exact requirements. Common inputs include:

• Paired-end FASTQ for full or upstream tasks.
• A host reference genome in FASTA format when the inspected Flow requires it.
• An abundance, pathway, or ASV table whose sample IDs correspond to metadata.tsv.
• Model-specific formats documented in the exact deep-learning Tool version’s summary/config.

File type	Common use	Preparation
Paired-end FASTQ (_1.fq.gz + _2.fq.gz)	Upstream analysis	Keep one pair per sample with consistent names.
Host reference genome FASTA	Possible host-removal step	Use a version compatible with the sample host.
metadata.tsv	Groups, covariates, interpretation	Match sample_id to the input table or sample files.
Abundance, pathway, or ASV table	Downstream statistics or microbiome analysis	Confirm matrix orientation and format for the selected Tool.

4.3.3 Data Organisation and Naming

• Keep paired FASTQ filenames consistent so read 1 and read 2 can be identified.
• Match metadata.tsv sample_id values to filenames, sample names, or table columns.
• When host removal is required, prepare a compatible reference and confirm its version.
• For downstream-only work, organise abundance, pathway, or ASV tables with metadata.tsv and check sample order and missing values.
• Organise BioLine paths by project, cohort, or batch. The upstream Flow must be able to read the selected location.

4.3.4 BioLine and Local Input Boundaries

Tasks with an upstream stage use BioLine-readable data paths because a BioLine Flow cannot read an arbitrary local path. The execution lane lists BioLine files with fkit and binds them according to the inspected interface. Upstream FASTQ remains in BioLine’s controlled environment and is not copied back wholesale for local analysis. After successful finalization, download only the results or supporting inputs needed downstream into stage.

Downstream-only tasks may use prepared inputs within their configured boundary. Whether the input came from BioLine or a task-local directory, verify its format and permissions against the exact Tool version before running.

4.3.5 Data Checks and Consequences

Check	How to verify	Possible consequence
Paired reads	Compare with the Flow’s required input ports	A missing read may prevent execution.
Sample organization	Compare with batch binding and directory rules	Mixed files may not be assigned to the correct samples.
Sample size	Evaluate against study design and statistical method	Small groups limit statistical power.
Sequencing depth and quality	Review data reports and the Flow’s supported range	Low depth or quality may reduce detection stability.
Reference genome	Match host and reference version	A mismatch may cause failure or incorrect filtering.
metadata.tsv	Check sample_id, groups, and covariates	Key mismatches can block or misassign group analysis.
Matrix format	Confirm row/column orientation, format, and feature names from Tool docs	Incorrect formatting may prevent reading or misalign samples.

4.3.6 Before Submission

Before an upstream task, confirm FASTQ pairs, sample directories, reference genome, metadata, and the BioLine path with fkit. Before a downstream-only task, check table format and sample keys. For either path, discover existing BioLine Tools/Flows first and inspect the exact version’s documentation and resources. Use the separate authoring workflow if no suitable capability exists.

No historical image set has to be built or installed in advance. The selected BioLine Tool version is pulled on demand.