Pipeline

• 

  Shawn Booth 21 Nov 2025 07dccb528ab8733de117c7dd31e01c2966f011a5

  Merge remote-tracking branch 'origin/update-docs-build-and-packaging-setup' into test-suite-refactor
• 

  Rui Xue 21 Nov 2025 8ad9560bbb93952f480dd4f9828f57bdc1ad28ac m

  PIPE-2742: Work around library compatibility issue with Python 3.12.
  Newer Python 3.12 version (`3.12.12`) requires a newer version of SQLite that supports `sqlite3_deserialize`. The current Conda environment was linking to an older system library, causing ImportErrors. Fixing the SQLite version forces Conda-forge to install Python version `3.12.10` currently.

  • environment.yml (version 8ad9560bbb93952f480dd4f9828f57bdc1ad28ac) (diffs)
• 

  Rui Xue 21 Nov 2025 805e45de9c532bbec605acb0ced321cacbf3dbb3 m

  PIPE-1883: fix an issue caused by dated `SQLite` in the build image.

  • environment.yml (version 805e45de9c532bbec605acb0ced321cacbf3dbb3) (diffs)
• 

  Rui Xue 21 Nov 2025 b7f2cf16ce454f792cc90b8c46c685f309f63552 m

  PIPE-1669: refactor the state detection of xvfb, workers, and clients; minior change to `README.rst`.

  • docs/README.rst (version b7f2cf16ce454f792cc90b8c46c685f309f63552) (diffs)
  • pipeline/__init__.py (version b7f2cf16ce454f792cc90b8c46c685f309f63552) (diffs)
  • pipeline/infrastructure/daskhelpers.py (version b7f2cf16ce454f792cc90b8c46c685f309f63552) (diffs)
  • pipeline/infrastructure/mpihelpers.py (version b7f2cf16ce454f792cc90b8c46c685f309f63552) (diffs)
• 

  Rui Xue <rx.astro@gmail.com> 20 Nov 2025 b7a110777f83733a6a78326d2ede12feac504c2b m

  PIPE-1669: update Conda `environment.yml`. - Clean up package list - Verify compatibility with `Ubuntu 24.04.3 LTS` and `macOS 26.1`

  • environment.yml (version b7a110777f83733a6a78326d2ede12feac504c2b) (diffs)
• 

  Rui Xue 20 Nov 2025 983dde0b684f8b1ede9454864a358453d9502794 m

  PIPE-1669: update the `Conda` environment to the latest.

  • environment.yml (version 983dde0b684f8b1ede9454864a358453d9502794) (diffs)
• 

  Rui Xue 20 Nov 2025 356079c234a5c930a94df954757c5f45e35f830d m

  PIPE-1883: update `.github/workflows/sphinx.yml` to pick the ticket branch.

  • .github/workflows/sphinx.yml (version 356079c234a5c930a94df954757c5f45e35f830d) (diffs)
• 

  Rui Xue 20 Nov 2025 6299ba7b91c3d2eafb8497a3e27d0fa33531b690 m

  Merge remote-tracking branch 'origin/main' into update-docs-build-and-packaging-setup
• 

  Rui Xue 20 Nov 2025 d41e219cdf21d42c5415bbc58f0a029e77755dae m

  PIPE-2880: add pytest coverage combine instructions.

  • docs/source/develdocmd/pipeline_tests.md (version d41e219cdf21d42c5415bbc58f0a029e77755dae) (diffs)
• 

  Rui Xue 20 Nov 2025 ae3103b6394ff3eaba51ed8320bd8b0e1dcd5259 m

  Pull request #1699: PIPE-2916: support custom cutout sizes in VLASS imaging with precise field selection and refactor makecutoutimages to handle metadata-based cutout selections.
  Merge in PIPE/pipeline from PIPE-2916-vlass-cutout-imaging to main
  
  * commit '08f9ab58558eca035c000007c47c82c92140ba19':
    PIPE-2916: optimize the `imsize` value derived by the `imagin_imsize` method.
    PIPE-2914: Refactor makecutoutimages filename handling and logging
    PIPE-2916: support custom cutout sizes in VLASS imaging with precise field selection and refactor makecutoutimages to handle metadata-based cutout selections.
• 

  Rui Xue 20 Nov 2025 07c7b8e5488585cfda56d938c282553f7f6f33a3 m

  PIPE-1669: fix a missing `import` statement resulting from an interaction with PIPE-2279 changes.

  • pipeline/infrastructure/utils/framework.py (version 07c7b8e5488585cfda56d938c282553f7f6f33a3) (diffs)
• 

  Ajay Vibhute 20 Nov 2025 7f2c9d6e488e659979719f29a6ddcc162ac49cb7 m

  Pull request #1698: PIPE-2912 remove the backward compatibility wrapper function get scipy function for mad
  Merge in PIPE/pipeline from PIPE-2912-remove-the-backward-compatibility-wrapper-function-get_scipy_function_for_mad to main
  
  * commit 'e24cdb42e1b49cac1c42cc68878aa7127aebf1c2':
    PIPE-2912: removed compatibility.get_scipy_function_for_mad
    PIPE-2912: removed compatibility.py and compatibility_test.py
    PIPE-2912: remove backward compatibility wrapper function for calculating MAD
• 

  Rui Xue 19 Nov 2025 be08168c7e0385ecd6a32afb8749cca4617a4467 m

  PIPE-1001: update LM's contributon years.

  • Contributors.rst (version be08168c7e0385ecd6a32afb8749cca4617a4467) (diffs)
• 

  Rui Xue 19 Nov 2025 d0db9726163344e32f79041577b6e4e595d6cf03 m

  Merge remote-tracking branch 'origin/main' into update-docs-build-and-packaging-setup
  Resolved conflicts:
  * pipeline/domain/measurementset.py
  * pipeline/hsdn/tasks/importdata/importdata.py
• 

  Rui Xue 19 Nov 2025 bc0281080333ab32869482b338e08046c0fb90b4 m

  PIPE-1793/PIPE-1330: capture logging path should stay under the actual session working directory.

  • pipeline/infrastructure/utils/conf.py (version bc0281080333ab32869482b338e08046c0fb90b4) (diffs)
• 

  Rui Xue 19 Nov 2025 08f9ab58558eca035c000007c47c82c92140ba19 m

  PIPE-2916: optimize the `imsize` value derived by the `imagin_imsize` method.

  • pipeline/hif/heuristics/imageparams_base.py (version 08f9ab58558eca035c000007c47c82c92140ba19) (diffs)
• 

  Rui Xue 19 Nov 2025 3c067d71d8843d3cae9fd037ad97fd4861deb9f3 m

  PIPE-2914: Refactor makecutoutimages filename handling and logging
  - Refactor filename construction in `Makecutoutimages` to use `os.path.splitext` for deriving the base name, replacing repetitive string replacements.
  - Correct subimage boundary calcculation and simplify clamping logic using `min()` and `max()`.
  - Update logging statements to use lazy formatting for better performance and style compliance.

  • pipeline/hif/tasks/makecutoutimages/makecutoutimages.py (version 3c067d71d8843d3cae9fd037ad97fd4861deb9f3) (diffs)
• 

  Rui Xue 19 Nov 2025 3e705abd7fae8fb8b4c3470ca71ff631bf58f62f m

  Pull request #1702: PIPE-2698: fix a `string.format` use bug.
  Merge in PIPE/pipeline from PIPE-2698-str-formatting-fix to main
  
  * commit '9beb677e889ee835b6ef06701aeb8dba7b5082b4':
    PIPE-2698: deduplicate while preserving the order.
    PIPE-2698: fix a `string.format` use bug; minior codestyle updates.
• 

  Rui Xue 19 Nov 2025 9beb677e889ee835b6ef06701aeb8dba7b5082b4 m

  PIPE-2698: deduplicate while preserving the order.

  • pipeline/h/heuristics/tsysfieldmap.py (version 9beb677e889ee835b6ef06701aeb8dba7b5082b4) (diffs)
• 

  Rui Xue 19 Nov 2025 e48400dbd34988e4b89ff773bb9ae2fba20ff046 m

  PIPE-2698: fix a `string.format` use bug; minior codestyle updates.

  • pipeline/domain/measurementset.py (version e48400dbd34988e4b89ff773bb9ae2fba20ff046) (diffs)
• 

  Rui Xue 19 Nov 2025 4f6da8be6ec8f7ef43814aa5adc63eb8f9e9a71c m

  Pull request #1701: PIPE-2909/PIPE-2270: fix the indentation bug for the frequency grid overlapping check logic.
  Merge in PIPE/pipeline from PIPE-2270-PIPE-2909-fix-nchan-precheck-logic to main
  
  * commit '661b07a0fae3bced3dde113130d21685cc5010a1':
    PIPE-2909/PIPE-2270: fix the indentation bug for the frequency grid overlapping check logic.
• 

  Rui Xue 18 Nov 2025 661b07a0fae3bced3dde113130d21685cc5010a1 m

  PIPE-2909/PIPE-2270: fix the indentation bug for the frequency grid overlapping check logic.

  • pipeline/hif/tasks/tclean/tclean.py (version 661b07a0fae3bced3dde113130d21685cc5010a1) (diffs)
• 

  Rui Xue 18 Nov 2025 006e6e588c02f6e06db947b27c48f53966409dbe m

  PIPE-2916: support custom cutout sizes in VLASS imaging with precise field selection and refactor makecutoutimages to handle metadata-based cutout selections.
  - hif_editimlist: Add handling for `cutout_imsize` input. Calculate required image size including buffer using new `imaging_imsize` heuristic and store the requested cutout size in `misc_vlass` metadata.
  - hif_makeimages: Ensure `cutout_imsize` is retrieved safely from `misc_vlass` metadata.
  - hif_makecutoutimages: Refactor the subimage creation loop.
      - Create a map for efficient lookup of `cutout_imsize` metadata.
      - Switch to lazy logging for performance.
      - Use f-strings for string formatting.
  - heuristics: Add `imaging_imsize` to `ImageParamsHeuristics` to calculate image dimensions based on cutout size and primary beam.

  • pipeline/hif/heuristics/imageparams_base.py (version 006e6e588c02f6e06db947b27c48f53966409dbe) (diffs)
  • pipeline/hif/heuristics/imageparams_vlass_single_epoch_cube.py (version 006e6e588c02f6e06db947b27c48f53966409dbe) (diffs)
  • pipeline/hif/tasks/editimlist/editimlist.py (version 006e6e588c02f6e06db947b27c48f53966409dbe) (diffs)
  • pipeline/hif/tasks/makecutoutimages/makecutoutimages.py (version 006e6e588c02f6e06db947b27c48f53966409dbe) (diffs)
  • pipeline/hif/tasks/makeimages/makeimages.py (version 006e6e588c02f6e06db947b27c48f53966409dbe) (diffs)
• 

  Ajay Vibhute 18 Nov 2025 e24cdb42e1b49cac1c42cc68878aa7127aebf1c2 m

  PIPE-2912: removed compatibility.get_scipy_function_for_mad

  • pipeline/hif/tasks/makermsimages/display.py (version e24cdb42e1b49cac1c42cc68878aa7127aebf1c2) (diffs)
• 

  Rui Xue 18 Nov 2025 15423158bbbd11a2ce9905c74b315bf6becc8945 m

  Pull request #1697: PIPE-2914: preserve order of fields and spws.
  Merge in PIPE/pipeline from PIPE-2914-fix-the-random-order-in-the-mstransform-field-spec-of-hif_transformimagedata to main
  
  * commit '3b9be802483ff6024c56b7847090f884d4624be2':
    PIPE-2914: preserve order of fields and spws.
• 

  Rui Xue 18 Nov 2025 3b9be802483ff6024c56b7847090f884d4624be2 m

  PIPE-2914: preserve order of fields and spws.
  The hif_transformimagedata task now preserves the original order of fields and spws when collecting them from multiple imaging targets.

  • pipeline/hif/tasks/transformimagedata/transformimagedata.py (version 3b9be802483ff6024c56b7847090f884d4624be2) (diffs)
• 

  Ajay Vibhute 18 Nov 2025 4ec732e9d398b896d884763994cbfe07d76dd13a m

  PIPE-2912: removed compatibility.py and compatibility_test.py

  • pipeline/infrastructure/utils/compatibility.py (version 4ec732e9d398b896d884763994cbfe07d76dd13a) (diffs)
  • pipeline/infrastructure/utils/compatibility_test.py (version 4ec732e9d398b896d884763994cbfe07d76dd13a) (diffs)
• 

  Rui Xue 17 Nov 2025 6d67fd29d0c647e20600f44d64cca7631e209c9f m

  Pull request #1688: PIPE-2270: use the correct `deconvolver` selected by `specmode`.
  Merge in PIPE/pipeline from PIPE-2270-hif_editimlist-doesn-t-work-with-imaging_mode-vla to main
  
  * commit '3078d6af76d37cafcb2a90fb341a1f83ae540e1d':
    PIPE-2270: expand `processing_data_type` to include selfcal datatypes in `hif_editimlist`.
    PIPE-2909: allow imaging when the requested cube channel-grid stop frequency exceeds the actual SPW coverage.
    PIPE-2270: use the correct `deconvolver` selected by `specmode`.
• 

  Ajay Vibhute 17 Nov 2025 df7ba1385f9a858fb82b705f6539d5da47f86557 m

  PIPE-2912: remove backward compatibility wrapper function for calculating MAD

  • pipeline/hif/tasks/makecutoutimages/display.py (version df7ba1385f9a858fb82b705f6539d5da47f86557) (diffs)
  • pipeline/hsd/heuristics/rasterscan.py (version df7ba1385f9a858fb82b705f6539d5da47f86557) (diffs)
• 

  Shawn Booth 17 Nov 2025 a49a172872b618e33611160c9de0f43a3998fbfe m

  Pull request #1678: PIPE-2869/PIPE-2698: Tsys scan plot randomly changes for some datasets
  Merge in PIPE/pipeline from PIPE-2869-tsys-scan-plot-randomly-changes-for-some-sd-dataset to main
  
  * commit '70d1a29702c3bfcf17ee6e127cfd2d240fce9fa4':
    PIPE-2869: modified select_tsys_field to account for double-quoted field names and added unit test to test this use case; added unit test to exercise an invalid case
    PIPE-2869: changed ms.array_name for NRO telescope to TP
    PIPE-2869: fixed incorrect assigning of variable
    PIPE-2869: added unit tests to exercise the bugfix; included NRO is is_sd determination; added ms.array_name for NRO datasets; added Cycle 12 dates to get_alma_cycle_number and included NRO data in filter to return None; updated touched code to modern standards, including deferred logging, improved type hinting, and removal of legacy code styles
    PIPE-2698: modified MeasurementSet method get_alma_cycle_number to return None for VLA and EVLA datasets; removed 'observatory' parameter from tsys_scans_radec function, since this info is in the provided ms; added conditional to not consider ON_SOURCE Tsys intents for data older than cycle 3
    PIPE-2869: moved de-duplication method to the multiple intent conditional; fixed bug when mapping a string integer with the field ID
    PIPE-2869: added ATMOSPHERE intent to preferred Tsys solution mapping for SD; created select_tsys_field function for determining which Tsys field is used with the source; fixed ALMA cycle number comparison
• 

  Shawn Booth 14 Nov 2025 70d1a29702c3bfcf17ee6e127cfd2d240fce9fa4 m

  PIPE-2869: modified select_tsys_field to account for double-quoted field names and added unit test to test this use case; added unit test to exercise an invalid case

  • pipeline/infrastructure/displays/plotpointings.py (version 70d1a29702c3bfcf17ee6e127cfd2d240fce9fa4) (diffs)
  • pipeline/infrastructure/displays/plotpointings_test.py (version 70d1a29702c3bfcf17ee6e127cfd2d240fce9fa4) (diffs)
• 

  Rui Xue 14 Nov 2025 cf0a106791f89b935eb255560d948316f1e03eeb m

  Pull request #1692: PIPE-1001: update `LICENSE.md` with the ALMA partnership description.
  Merge in PIPE/pipeline from PIPE-1001-update-copyright-notice to main
  
  * commit '3cdd4e69571f11ea8807193abbcbad41d5e71711':
    PIPE-1001: update `LICENSE.md` with the ALMA partnership description.
• 

  Shawn Booth 13 Nov 2025 8569925ee4c5d9ed53eba9c14a719230b7c91f94 m

  PIPE-2869: changed ms.array_name for NRO telescope to TP

  • pipeline/hsdn/tasks/importdata/importdata.py (version 8569925ee4c5d9ed53eba9c14a719230b7c91f94) (diffs)
  • pipeline/infrastructure/displays/plotpointings.py (version 8569925ee4c5d9ed53eba9c14a719230b7c91f94) (diffs)
• 

  Rui Xue 07 Nov 2025 3078d6af76d37cafcb2a90fb341a1f83ae540e1d m

  PIPE-2270: expand `processing_data_type` to include selfcal datatypes in `hif_editimlist`.

  • pipeline/hif/tasks/editimlist/editimlist.py (version 3078d6af76d37cafcb2a90fb341a1f83ae540e1d) (diffs)
• 

  Rui Xue 06 Nov 2025 186dd12c0bc6ebbd689cd7c0dcfdc72646811101 m

  PIPE-2909: allow imaging when the requested cube channel-grid stop frequency exceeds the actual SPW coverage.

  • pipeline/hif/tasks/findcont/findcont.py (version 186dd12c0bc6ebbd689cd7c0dcfdc72646811101) (diffs)
  • pipeline/hif/tasks/tclean/tclean.py (version 186dd12c0bc6ebbd689cd7c0dcfdc72646811101) (diffs)
• 

  Rui Xue 06 Nov 2025 3cdd4e69571f11ea8807193abbcbad41d5e71711 m

  PIPE-1001: update `LICENSE.md` with the ALMA partnership description.

  • LICENSE.md (version 3cdd4e69571f11ea8807193abbcbad41d5e71711) (diffs)
• 

  Rui Xue 31 Oct 2025 b6ed73395b3852c345a120575af36357763e6329 m

  PIPE-2270: use the correct `deconvolver` selected by `specmode`.

  • pipeline/hif/tasks/editimlist/editimlist.py (version b6ed73395b3852c345a120575af36357763e6329) (diffs)
  • pipeline/hif/templates/tclean.mako (version b6ed73395b3852c345a120575af36357763e6329) (diffs)
• 

  Shawn Booth 28 Oct 2025 5d75f6be976212f7f1427f699e3f85635a8994eb m

  PIPE-2869: fixed incorrect assigning of variable

  • pipeline/hsdn/tasks/importdata/importdata.py (version 5d75f6be976212f7f1427f699e3f85635a8994eb) (diffs)
• 

  Shawn Booth 26 Oct 2025 f5c206fb73d91a8920977da925a3e829ac512b69 m

  PIPE-2869: added unit tests to exercise the bugfix; included NRO is is_sd determination; added ms.array_name for NRO datasets; added Cycle 12 dates to get_alma_cycle_number and included NRO data in filter to return None; updated touched code to modern standards, including deferred logging, improved type hinting, and removal of legacy code styles

  • pipeline/domain/measurementset.py (version f5c206fb73d91a8920977da925a3e829ac512b69) (diffs)
  • pipeline/hsdn/tasks/importdata/importdata.py (version f5c206fb73d91a8920977da925a3e829ac512b69) (diffs)
  • pipeline/infrastructure/displays/plotpointings.py (version f5c206fb73d91a8920977da925a3e829ac512b69) (diffs)
  • pipeline/infrastructure/displays/plotpointings_test.py (version f5c206fb73d91a8920977da925a3e829ac512b69) (diffs)
• 

  Shawn Booth 22 Oct 2025 8dfce92a0559f3321d261fe5943a080a535446a5 m

  PIPE-2698: modified MeasurementSet method get_alma_cycle_number to return None for VLA and EVLA datasets; removed 'observatory' parameter from tsys_scans_radec function, since this info is in the provided ms; added conditional to not consider ON_SOURCE Tsys intents for data older than cycle 3

  • pipeline/domain/measurementset.py (version 8dfce92a0559f3321d261fe5943a080a535446a5) (diffs)
  • pipeline/infrastructure/displays/plotpointings.py (version 8dfce92a0559f3321d261fe5943a080a535446a5) (diffs)
• 

  Shawn Booth 01 Oct 2025 f79f2d4cb2ebaa59c7dd02f3ec768518c505f4c3 m

  PIPE-2869: moved de-duplication method to the multiple intent conditional; fixed bug when mapping a string integer with the field ID

  • pipeline/h/heuristics/tsysfieldmap.py (version f79f2d4cb2ebaa59c7dd02f3ec768518c505f4c3) (diffs)
  • pipeline/infrastructure/displays/plotpointings.py (version f79f2d4cb2ebaa59c7dd02f3ec768518c505f4c3) (diffs)
• 

  Shawn Booth 30 Sep 2025 5072d6e6a0206e7b4951aba34321b1bd6f01c19e m

  PIPE-2869: added ATMOSPHERE intent to preferred Tsys solution mapping for SD; created select_tsys_field function for determining which Tsys field is used with the source; fixed ALMA cycle number comparison

  • pipeline/h/heuristics/tsysfieldmap.py (version 5072d6e6a0206e7b4951aba34321b1bd6f01c19e) (diffs)
  • pipeline/infrastructure/displays/plotpointings.py (version 5072d6e6a0206e7b4951aba34321b1bd6f01c19e) (diffs)