Pipeline¶

Overview¶

The pipeline automates the full data reduction workflow for eclipsing binary observations. Once started, it monitors a directory for incoming images from the telescope, waits until the observation session ends (no new files for a configurable timeout), then automatically runs data reduction, comparison star selection, and multi-aperture photometry in sequence.

The pipeline also provides:

Process locking — prevents two instances from running on the same directory simultaneously
Summary report — writes a [object_name]_pipeline_summary.txt file to the output folder at the end of every run, recording the runtime and status of each stage

Usage¶

To see all available options:

EB_pipeline -h

To start the pipeline:

EB_pipeline INPUT_DIR OUTPUT_DIR --ra HH:MM:SS.SS --dec DD:MM:SS.SS [options]

Required Inputs¶

INPUT_DIR: Path to the folder where incoming raw images from the telescope will appear. The pipeline monitors this folder for new files.
OUTPUT_DIR: Path to the folder where reduced images and all pipeline output files will be saved.
--ra: Right ascension of the target in the format HH:MM:SS.SS.
--dec: Declination of the target in the format DD:MM:SS.SS. For negative declinations use -DD:MM:SS.SS.

Optional Inputs¶

--time: How long in seconds the pipeline waits after the last new file before starting data reduction. Default is 3600 seconds (1 hour).
--loc: Location of the telescope. Accepted values are BSUO or any site listed in the Astropy sites list. Default is None.
--name: Name of the target object. Use underscores instead of spaces (e.g. NSVS_254037). This name is used for all output file names. Default is NSVS_254037.
--mem: Memory limit for the IRAF reduction stage in bytes. Default is 450e6 (450 MB). For example, to allow 800 MB use 800e6.
--gain: Gain of the camera in electrons per ADU. Default is 1.43.
--rdnoise: Readout noise of the camera in electrons. Default is 10.83.
--log-file: Optional path to write all log output to a file in addition to the terminal. Useful for keeping a record of the night’s run.

Output Files¶

The pipeline writes the following files to OUTPUT_DIR at the end of a successful run:

Reduced FITS images from the IRAF reduction stage
RADEC comparison star files for each filter
Light curve CSV files and plots for each filter
[name]_pipeline_summary.txt — a summary of the run including total runtime, per-stage duration, and any warnings encountered

The summary report looks like this:

============================================================
Pipeline Summary — NSVS_254037
============================================================
Total runtime : 1:23:47

Stages:
  IRAF Reduction                 0:45:12      [ok]
  Comparison Star Selection      0:02:15      [ok]
  Aperture Photometry            0:36:20      [ok]

No warnings.
============================================================

Process Locking¶

The pipeline creates a .pipeline.lock file in OUTPUT_DIR when it starts. This prevents a second instance from accidentally running on the same directory at the same time. If the pipeline exits unexpectedly, the lock is released automatically. If you are certain no other instance is running and the lock file remains, delete it manually and restart.

Example¶

A typical pipeline invocation:

EB_pipeline C:/folder1/raw_images C:/folder1/reduced_images \
    --ra 00:28:27.96 --dec 78:57:42.65 \
    --name NSVS_254037 \
    --time 3000 \
    --loc CTIO \
    --gain 1.43 \
    --rdnoise 10.83 \
    --log-file C:/folder1/pipeline_run.log

For a negative declination:

EB_pipeline C:/folder1/raw_images C:/folder1/reduced_images \
    --ra 00:28:27.96 --dec -12:34:56.78 \
    --name NSVS_254037

Note

The INPUT_DIR and OUTPUT_DIR positional arguments must come first, in that order, before any -- options. The --ra and --dec arguments are required and have no default values.

Warning

Do not use the same folder for both INPUT_DIR and OUTPUT_DIR. The reduction stage writes files to OUTPUT_DIR and monitoring INPUT_DIR for new files could behave unexpectedly if they overlap.