Code commits
Pipeline
-
Rui Xue 0ee2c21ad9a563e2a60beeb30b6ef67aa6252bc6
Merge remote-tracking branch 'origin/main' into PIPE-3269-add-parsl-adaptation-layer-in-pipeline
-
Rui Xue 03bb819e0f4a951d97ea64bb0c62ca441a082b52 m
Pull request #2046: Docs/add selfcal dr statement to main
Merge in PIPE/pipeline from docs/add_selfcal_dr_statement_to-main to main
* commit '3d176781bcf4d3b5eab11a5740864a627353880e':
Update clean threshold and automasking docs
Add a statement about the dynamic range correction factor applied for selfcal images to the hif_makeimages documentation. -
Rui Xue 3d176781bcf4d3b5eab11a5740864a627353880e m
Update clean threshold and automasking docs
- Convert automasking and dynamic range tables to reST list-tables with Unicode math symbols and compact header blocks.
- Add bibtex citation for the auto-multithresh reference.
- Link tclean via Intersphinx. -
Patrick Sheehan 004b7951361fa36f48f262eb542980a0885b3add m
Add a statement about the dynamic range correction factor applied for selfcal images to the hif_makeimages documentation.
-
Ajay Vibhute 0bd065b432fe85e7558e30814c1c2d0d56e1deb9 m
Pull request #1801: PIPE-3067: scripts to populate fits header keywords using information available in product directory
Merge in PIPE/pipeline from PIPE-3067-standalone-script-for-vlass-header-keywords to main
* commit 'c77e37077037a1828afe8c6b8af11d6f23b90b4b':
PIPE-3067: updated vlassbwn for se-cube and other imaging modes
PIPE-3067: updated VLASSPT, VLASSWP, VLASSPL, VLASSRJ
PIPE-3067: updated docstrings, added requirements and dry-run option
PIPE-3067: scripts to populate fits header keywords using information available in product directory -
Rui Xue 4bbbee5b965c40dc0e3b229aa32d6cf8cbc157de m
Pull request #1950: PIPE-3222/PIPE-3221/PIPE-889
Merge in PIPE/pipeline from PIPE-3222-redundant-edgechan-flag-command-in-h_tsysflag to main
* commit '6ecc99029229649dd32a98e3427f3a85754aaff8':
PIPE-3222: Deduplicate edgechans flag commands in h_tsysflag
PIPE-3221: Make applycal field ordering deterministic
PIPE-3221: Fix non-deterministic tsys spw iteration in tsysflag -
Rui Xue df36fe412134d0e89fcfcdc87a3edf3748f2b201 m
Pull request #2043: PIPE-3252: migrate to SciX API
Merge in PIPE/pipeline from PIPE-3252-update-bibtex-query-end-point-from-ads-to-scixplorer-transition to main
* commit '7069ad4d871a67029216005306848f17277e1ae0':
PIPE-3252: migrate to SciX API -
Rui Xue 7069ad4d871a67029216005306848f17277e1ae0 m
PIPE-3252: migrate to SciX API
- Migrate update_references.py to the NASA SciX API endpoint (api.scixplorer.org) with automatic fallback to adsabs.harvard.edu.
- Add SCIX_API_TOKEN and ~/.scix/token support to token discovery.
- Update reference synchronization documentation and refresh pipeline.bib from the SciX public library.- .skills/docs-build/SKILL.md (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- docs/source/devel/setup/pixi_tasks.md (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- docs/source/references/pipeline.bib (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- docs/source/references/readme.md (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- docs/source/releases.md (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- docs/source/users_guide/known_issues.md (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- pyproject.toml (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- scripts/README.md (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- scripts/update_references.py (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
-
Rui Xue 969aa3ff78c76b64383260f471bfa3d2c0ab0e9a m
Pull request #2042: PIPE-3252: update releases and known issues for CASA 6.7.4-12 patch
Merge in PIPE/pipeline from PIPE-3252-update-docs-for-vla-and-alma-patch-releases to main
* commit '523b6b9ec6c541dfe84ce6b44ceb8c9984df4c2d':
PIPE-3252: update releases and known issues for CASA 6.7.4-12 patch -
Rui Xue 523b6b9ec6c541dfe84ce6b44ceb8c9984df4c2d m
PIPE-3252: update releases and known issues for CASA 6.7.4-12 patch
- Note fixed issues (CAS-14850 and CAS-14874) in PL2026 known issues. -
Rui Xue aeb58b104b1d8fb14d4362574b1c1153142ef089 m
Pull request #2040: PIPE-3252: document DataType design
Merge in PIPE/pipeline from PIPE-3252-add-datatype-use-cases-notes to main
* commit '3c1c56edcfbfba77e87a6f4f60fedf1cf1b00f28':
PIPE-3252: update DataType reduction and restore workflows
PIPE-3252: refine datatype doc and diagram layout
PIPE-3252: document DataType design and lifecycle -
Rui Xue 3c1c56edcfbfba77e87a6f4f60fedf1cf1b00f28 m
PIPE-3252: update DataType reduction and restore workflows
Document ALMA and VLA reduction lifecycles and self-cal restore workflows with consolidated data nodes, line target application, and responsive Mermaid rendering.
- datatypes.md: add side-by-side ALMA and VLA lifecycles, clarify self-cal restore sequence, and fix Sphinx refs.
- furo_layout.js: support responsive container scaling for wide multi-column diagrams.
- conf.py: bump mermaid_version to 12.2.0. -
Rui Xue 102a12f9146d79b7d59da703fe67015ab10ea3a7 m
Pull request #2041: PIPE-3252: update releases table for `2026.2.0` and `2026.2.1`
Merge in PIPE/pipeline from PIPE-3252-update-docs-for-vla-and-alma-patch-releases to main
* commit '04e87fcdb8deb467821d4d612eaedf33201c8634':
PIPE-3252: update releases table for `2026.2.0` and `2026.2.1` -
Rui Xue 04e87fcdb8deb467821d4d612eaedf33201c8634 m
PIPE-3252: update releases table for `2026.2.0` and `2026.2.1`
- Update documentation references to target RTD slugs v2026.2.0 and v2026.2.1 using domain root-relative HTML anchors to prevent Sphinx cross-reference warnings.
- Update CASA version and download tarballs for 2026.2.1.55, and mark superseded 2026.2.0.27 August packages as struck-through. -
Rui Xue fc52a4bacf7c0998a62fe67cd2990beba2b18bde m
PIPE-3252: refine datatype doc and diagram layout
- Add API cross-references and prerequisite notes for `CORRECTED_DATA` initialization to the DataType reference documentation.
- Disable the unused ELK layout engine to reduce page bundle size.
- Update Mermaid container styling and dynamic aspect-ratio scaling to prevent Gantt chart horizontal collapse while bounding tall flowcharts.- docs/source/_static/custom_theme.css (version fc52a4bacf7c0998a62fe67cd2990beba2b18bde) (diffs)
- docs/source/_static/furo_layout.js (version fc52a4bacf7c0998a62fe67cd2990beba2b18bde) (diffs)
- docs/source/conf.py (version fc52a4bacf7c0998a62fe67cd2990beba2b18bde) (diffs)
- docs/source/devel/reference/datatypes.md (version fc52a4bacf7c0998a62fe67cd2990beba2b18bde) (diffs)
-
Takeshi Nakazato 78502f29806c81fc1659f7fe614584921e8c0e8c m
Pull request #1984: PIPE-3232 fix errors in log qa when weblog rendering is disabled
Merge in PIPE/pipeline from PIPE-3232-fix-errors-in-log_qa-when-weblog-rendering-is-disabled to main
* commit 'aae06797d595843586eed600982f3e71e7b7dd53':
PIPE-3232: add unit tests for `DISABLE_WEBLOG` and `DISABLE_QA` handling
PIPE-3232: introduce DISABLE_QA flag to control QA score generation
PIPE-3232: return empty plot dict from flagging tasks when weblog is disabled
PIPE-3232 update log_qa decorator
PIPE-3232 score_amp_vs_time_plots should skip QA evaluation if weblog is disabled
PIPE-3232 add/remove log_qa decorator depending on the return value of the function -
Rui Xue 1d2a4b07e290550e709635a2830a3eb3e69b841c m
PIPE-3252: document DataType design and lifecycle
- Add a developer reference guide detailing the Pipeline DataType design, covering in-memory domain tracking, on-disk persistence in MS HISTORY, reduction lifecycle stages, and downstream imaging selection.
- Also adjust Sphinx Mermaid layout configuration, centering styles, and interaction controls in custom theme assets to improve diagram readability.- docs/source/_static/custom_theme.css (version 1d2a4b07e290550e709635a2830a3eb3e69b841c) (diffs)
- docs/source/_static/furo_layout.js (version 1d2a4b07e290550e709635a2830a3eb3e69b841c) (diffs)
- docs/source/conf.py (version 1d2a4b07e290550e709635a2830a3eb3e69b841c) (diffs)
- docs/source/devel/reference/datatypes.md (version 1d2a4b07e290550e709635a2830a3eb3e69b841c) (diffs)
- docs/source/devel/reference/index.md (version 1d2a4b07e290550e709635a2830a3eb3e69b841c) (diffs)
-
Ajay Vibhute c77e37077037a1828afe8c6b8af11d6f23b90b4b m
PIPE-3067: updated vlassbwn for se-cube and other imaging modes
-
Rui Xue 396bb0fed9646d862bd0427e4a9df4d9bf365619 m
Pull request #2035: PIPE-3252: document observatory services and monitors
Merge in PIPE/pipeline from docs/observatory-services-monitoring to main
* commit 'ff257dbfc5ca70a9cc8b298a1af79105ce8c6e60':
PIPE-3252: Add maintainer information for Casaconfig and Pipeline Services monitors
PIPE-3252: document observatory services and monitors -
Rui Xue ff257dbfc5ca70a9cc8b298a1af79105ce8c6e60 m
PIPE-3252: Add maintainer information for Casaconfig and Pipeline Services monitors
-
Rui Xue e7cdaf510ed7aa7be9b6e36fcb28c9f0eaee1a57 m
PIPE-3252: document observatory services and monitors
Add documentation for external observatory web services, CASA runtime data, and operational health monitors (`casaconfig-monitor` and `pipeline-services-monitor`). -
Ajay Vibhute a23bf5d4430e6d2a81dbf666cfdf0ebee98bbc3c m
Pull request #2028: PIPE-3274: Update median delay QA to use delay values in ns instead of ms
Merge in PIPE/pipeline from PIPE-3274-main-typo-in-auto-qa-notes-in-hifv_testbpdcals to main
* commit '75b71fbd209a591dc1753ab4f0dba838e286d94c':
PIPE-3274: changed delay score to 1.0 from rendererutils.SCORE_THRESHOLD_SUBOPTIMAL
PIPE-3274: Update median delay QA to use delay values in ns instead of ms -
Rui Xue 6ecc99029229649dd32a98e3427f3a85754aaff8 m
Merge remote-tracking branch 'origin/PIPE-3221-flagging-command-order-from-hifa_tsysflag-might-not-be-deterministic' into PIPE-3222-redundant-edgechan-flag-command-in-h_tsysflag
-
Rui Xue 3a559c545d136093429d82edb2f2105d8d759c4b m
Merge remote-tracking branch 'origin/main' into PIPE-3222-redundant-edgechan-flag-command-in-h_tsysflag
-
Rui Xue 2ede876d89735978035e997d9c03da97a9fcf8bd m
Pull request #2014: PIPE-2949: to MAIN
Merge in PIPE/pipeline from PIPE-2949-hif_editimlist-resolves-spw_name-to-real-id-instead-of-virtual-id to main
* commit 'b31be0595fc9d529d7711c429207a54a5125bd8f':
PIPE-2949: resolve spw name to virtual ID and populate target vis in `editimlist`
PIPE-2949: use first ms for field/spw lookups -
Patrick Bos b49a22799e68a5cc7fdef6b9936e5552f6a5c7f4 m
Pull request #2027: Shorten AGENTS.md and split specific tasks into skills
Merge in PIPE/pipeline from egp-shorten-agents-md to main
* commit 'e857c0f99b3a53356d9a9a1d550777aaf5ec1c1f':
shorten AGENTS.md and split specific tasks out into skills
add test-unit pixi command output file to gitignore -
Ajay Vibhute 75b71fbd209a591dc1753ab4f0dba838e286d94c m
PIPE-3274: changed delay score to 1.0 from rendererutils.SCORE_THRESHOLD_SUBOPTIMAL
-
Ajay Vibhute 95592fb80df30dc6d2288c8f735dae540cfd237e m
Pull request #2030: PIPE-3013: adding editmode replace to template_hifa_cubeimage.xml
Merge in PIPE/pipeline from PIPE-3013-main-hifa_cubeimage-recipe-should-have-editmode-replace-parameter-set-for-hif_editimlist to main
* commit '3f3715c90f74ee8b00e2c93e5c861bb4804a3d82':
PIPE-3013: adding editmode replace to template_hifa_cubeimage.xml -
Rui Xue b376514f96fefbbcb64959d23ed12835ba0d1f8c m
Pull request #2025: PIPE-3275: fix scalar conversion crash in subband QA
Merge in PIPE/pipeline from PIPE-3275-numpy-2.5-crash-qa-of-hifa_bandpass to main
* commit 'aa06d07a366fd6393970c3a46ac21a6adddbdb15':
PIPE-3275: fix scalar conversion crash in subband QA -
Ajay Vibhute 3f3715c90f74ee8b00e2c93e5c861bb4804a3d82 m
PIPE-3013: adding editmode replace to template_hifa_cubeimage.xml
-
Ajay Vibhute fb0d8202b69a8e1baf21535190d53a13c81ac70d m
PIPE-3274: Update median delay QA to use delay values in ns instead of ms
-
E. G. Patrick Bos <egpbos@gmail.com> e857c0f99b3a53356d9a9a1d550777aaf5ec1c1f m
shorten AGENTS.md and split specific tasks out into skills
This reduces default context token use by about half. With all skills loaded, context reduction is still about 25%- .skills/commit-style/SKILL.md (version e857c0f99b3a53356d9a9a1d550777aaf5ec1c1f) (diffs)
- .skills/docs-build/SKILL.md (version e857c0f99b3a53356d9a9a1d550777aaf5ec1c1f) (diffs)
- .skills/python-code-standards/SKILL.md (version e857c0f99b3a53356d9a9a1d550777aaf5ec1c1f) (diffs)
- AGENTS.md (version e857c0f99b3a53356d9a9a1d550777aaf5ec1c1f) (diffs)
-
E. G. Patrick Bos <egpbos@gmail.com> 9a75c9748fd80459e367713fa2d6a584293fed4e m
add test-unit pixi command output file to gitignore
-
Rui Xue aa06d07a366fd6393970c3a46ac21a6adddbdb15 m
PIPE-3275: fix scalar conversion crash in subband QA
Extract a 0-dimensional scalar from subb_spk using .flat indexing in evalPerAntBP_Platform. In NumPy 2.0+, indexing a 2D slice with a flat argmax index returns a 1D array rather than a scalar, causing downstream string formatting to fail with TypeError. -
Rui Xue 4b0551017851ee500cf1f4f6d65c67328b48fb72 m
Pull request #2023: PIPE-3211: start casa/casampi tasks in INIT_CWD (similar to what has been done in `stack-man`)
Merge in PIPE/pipeline from PIPE-3211-start-casa-casampi-pixi-tasks-with-init-cwd to main
* commit '3c43082d7a61f56b515ae64ff6d5ed73ab1a123d':
PIPE-3211: update pixi tasks guide and environment specs
PIPE-3211: start casa/casampi tasks in INIT_CWD (similar to what has been done in `stack-man`) -
Rui Xue 3c43082d7a61f56b515ae64ff6d5ed73ab1a123d m
PIPE-3211: update pixi tasks guide and environment specs
Update pixi developer documentation to align with recent task and environment configurations:
- Document INIT_CWD working directory semantics and thread pool clamping for casa and casampi tasks.
- Add missing bundle-assets and update-references task descriptions.
- Remove duplicate environment table entry and update dated references.
- Sync conda migration examples with openmpi-mpicxx and casatasks pins.
- Clean up the common tasks quick reference table. -
Rui Xue fc436ff14b9fb879ef409889fe22c930b712091c m
PIPE-3211: start casa/casampi tasks in INIT_CWD (similar to what has been done in `stack-man`)
Start pixi tasks `casa` and `casampi` in the directory where pixi was invoked (`INIT_CWD`) instead of the workspace root. Default threading in `tasks.casa` to the lesser of available CPUs and 4 across Linux and macOS, while preserving explicit user overrides. -
Hajime Ezawa 1a4f5aa660536310ee611105ee310d066642389e m
Pull request #1967: PIPE-3105 hsd imaging possible improvement on the weblog layout for no detection of off line range
Merge in PIPE/pipeline from PIPE-3105-hsd_imaging-possible-improvement-on-the-weblog-layout-for-no-detection-of-off-line-range to main
* commit '225d8766fe66577455e9438ff6cdac281e41ae11':
PIPE-3105: bug fix
PIPE-3105: bug fix and corrections
PIPE-3105: optimized the default for min_gap_width of _pick_excess_ranges()
PIPE-3105: improve np.array masking
PIPE-3105: explicitly specify constant_values for np.pad
PIPE-3105: bug fix
PIPE-3105: add/correct type annotations
PIPE-3105: optimize/correct whitespaces
PIPE-3105: replace .format() to f-string, typo correction
PIPE-3105: output excess ranges to LOG
PIPE-3105: always show two plots regardless the detection status -
Rui Xue b31be0595fc9d529d7711c429207a54a5125bd8f m
PIPE-2949: resolve spw name to virtual ID and populate target vis in `editimlist`
- Resolve SPW names and center frequencies to virtual SPW IDs via `real2virtual_spw_id` in `hif_editimlist`.
- Populate imlist_entry['vis'] for all imaging modes so downstream `hif_makeimages`/`tclean` receives the expected MS list.
- Normalize vislist and guard reference MS resolution in editimlist.
- Add fallback in CleanTaskFactory to use image_heuristics.vislist if target['vis'] is unset. -
Rui Xue fea5b728022fea7766f6e3fa3d683f33cb4872e0 m
PIPE-2949: use first ms for field/spw lookups
Switch from inp.vis[-1] to inp.vis[0] for reference MS, which follows convention in other related tasks. -
Rui Xue 036f9701c1309560d2aeabbe58336600d231515f m
Pull request #2020: PIPE-3252 docs branching and rtd versioning
Merge in PIPE/pipeline from PIPE-3252-docs-branching-and-rtd-versioning to main
* commit '1de46b530b0126c4fa373a3cb8dd5160a0e0a14b':
docs: update Jira ticket and figure directives.
Add link to CAS-14874 on Jira
Update CASA 2026 known issues to include the CAS-14874 bug. -
Patrick Sheehan 4c652b4d83dda3657222a9f7652b50a00fbaf4ec m
Update CASA 2026 known issues to include the CAS-14874 bug.
-
Rui Xue 1de46b530b0126c4fa373a3cb8dd5160a0e0a14b m
docs: update Jira ticket and figure directives.
-
Patrick Sheehan ee57b4397297f156c41fc692b96abe0481c87656 m
Add link to CAS-14874 on Jira
-
Rui Xue e7b6405653ac1ea6c3d71cfc6da408d58e6f3a5b m
Pull request #2017: PIPE-3252: Document RTD branching strategy and version mapping
Merge in PIPE/pipeline from PIPE-3252-docs-branching-and-rtd-versioning to main
* commit 'd0d258de128b74e3e2d4cdde776a00598eddb77b':
PIPE-3252: Restructure RTD docs sections and add descriptions for the branch/tag-RTD-version mapping
PIPE-3252: Expand the docs build path in `.gitignore`
PIPE-3252: Document RTD branching strategy and version mapping -
Rui Xue d0d258de128b74e3e2d4cdde776a00598eddb77b m
PIPE-3252: Restructure RTD docs sections and add descriptions for the branch/tag-RTD-version mapping
-
Rui Xue 388f15af0cb4cb87ef3cf753ba74ba3ab5cd5226 m
PIPE-3252: Expand the docs build path in `.gitignore`
-
Rui Xue b69d2a417af62c280f002ac73fdffbfa772de35c m
PIPE-3252: Document RTD branching strategy and version mapping
-
Rui Xue 85a334903820b9841ba363dd5dec79fe92d113cd m
Pull request #2016: PIPE-3272: Add flux service query script and clairfy the new 4-digit "DataConditions" from 2026AUG
Merge in PIPE/pipeline from PIPE-3272-add-flux-service-query-script to main
* commit '4c62d794f7473aa7df7449b637c1f5ecfdd072e0':
PIPE-3272: reference ICT tickets for 4th DataConditions digit
PIPE-3272: update DataConditions parsing in dbfluxes for 4-digit codes
PIPE-3272: add `dbfluxes_query.py` script for catalogue lookups -
Rui Xue a7f8802d696f85d31b747a8dc52651366f1cf9c2 m
Pull request #1960: PIPE-3237: optimize unflagged data checks in `hif_findroi` and `ImageParamsHeuristics`
Merge in PIPE/pipeline from PIPE-3237-hif_findroi-very-inefficient-for-cases-with-many-fields-x-eb-x-spw to main
* commit '82fe783a77d453ed5300b9485da639fd9933cecc':
PIPE-3237: warn once when `hif_findroi` skips a virtual spw
PIPE-3237: simplify target field name list construction in `findroi.py`
PIPE-3237: optimize unflagged data checks in `hif_findroi` and `ImageParamsHeuristics` -
Rui Xue 4c62d794f7473aa7df7449b637c1f5ecfdd072e0 m
PIPE-3272: reference ICT tickets for 4th DataConditions digit
-
Rui Xue a3a8f9ac972247979d0280b51154721cd9104f16 m
PIPE-3272: update DataConditions parsing in dbfluxes for 4-digit codes
-
Rui Xue 293163dd05b17927cc2382fa157dbef71af89217 m
PIPE-3272: add `dbfluxes_query.py` script for catalogue lookups
-
Rui Xue df553e34d6e0e53244ac27c105e7fb0bf296f60f m
Pull request #1955: PIPE-684/PIPE-3218/PIPE-3255: VLA mosaic imaging
Merge in PIPE/pipeline from PIPE-684-vla-mosaic-imaging to main
* commit 'beed751da9a3c6555c661227f0e90d131b9ad0a7': (52 commits)
PIPE-684: handle per-MS field lists and streamline center_field_ids
PIPE-684: set VLA pblimit_image to 0.1 for mosaic non-cube imaging
PIPE-3247/PIPE-3248: Improve field name rendering with text wrapping
PIPE-684: consolidate flagged SPW warnings and skip fully flagged targets in makeimlist
PIPE-3218: display spectral window lists as ranges in `exportdata` tables
PIPE-684: Enable per-field mitigation parameter lookups in `makeimlist`
PIPE-684: Add optional field parameter to `hm_cell/hm_imsize` lookups
PIPE-684: improve `wrap_long_str` - track line length instead of using old modulo arithmetic
PIPE-684: fix multi-target product size mitigation with per-(field,spw) keying
PIPE-3218: fix HTML line breaks for long field names
PIPE-684: Fix product size calculation and add field filtering
PIPE-684: disable self-calibration heuristics for VLA mosaic targets
PIPE-3218: wrap long source names in exportdata.mako weblog tables
PIPE-3218: wrap long source names in `hifv_exportdata` weblog tables
PIPE-684: export `wrap_long_str` in pipeline.infrastructure.utils
PIPE-684: condense long composite field names in QA and log messages
PIPE-684: merge weblog table columns in findcont and VLA exportdata
PIPE-684: use `self.spwids` for field lookup and frequency calculation in VLA heuristics
PIPE-684: optimize logging and improve VLA field intent docs
PIPE-684: Refactor mosaic detection with modern Python standards and comprehensive tests
... -
Rui Xue 82fe783a77d453ed5300b9485da639fd9933cecc m
PIPE-3237: warn once when `hif_findroi` skips a virtual spw
-
Rui Xue 148f12af9c82315bbf9cd6b156b49b44be725d98 m
PIPE-3237: simplify target field name list construction in `findroi.py`
-
Hajime Ezawa 225d8766fe66577455e9438ff6cdac281e41ae11 m
PIPE-3105: bug fix
-
Hajime Ezawa 6a39bf2a14dd5632b7824bd2c58832a84801daa3 m
Merge branch 'main' of https://open-bitbucket.nrao.edu/scm/pipe/pipeline into PIPE-3105-hsd_imaging-possible-improvement-on-the-weblog-layout-for-no-detection-of-off-line-range
-
Hajime Ezawa 56745139edf63b1a6dd4dc351ecde473f08e2156 m
Pull request #1968: PIPE-3246: add overwrite=True to image_args
Merge in PIPE/pipeline from PIPE-3246-hsd_imaging-sometimes-hsd_imaging-crashes-with-runtimeerror-output-file-exists to main
* commit '1113f6eeab147278cd16547f836b6d5012ed49df':
PIPE-3246: add overwrite=True to image_args -
Rui Xue beed751da9a3c6555c661227f0e90d131b9ad0a7 m
Merge remote-tracking branch 'origin/main' into PIPE-684-vla-mosaic-imaging
-
Rui Xue ec0d64616bb1e25c69cdbdc3475801c422703a87 m
PIPE-684: handle per-MS field lists and streamline center_field_ids
- Support both string and per-MS sequence field selections in `center_field_ids`, fixing `AttributeError` from `.split()` when `inputs.field` is a list (e.g. in VLASS).
- Leverage `ms.get_fields(task_arg=...)` in `center_field_ids` to natively resolve field IDs, names, and selections via domain tools.
- Pass `fields` instead of flattened `field_names` in `phasecenter` to align with the per-MS heuristics contract.
- Add type hints and docstrings for `phasecenter` and `center_field_ids`. -
Hajime Ezawa 39d3d4222f0101c06a1a7bf7fd7816795370c983 m
PIPE-3105: bug fix and corrections
-
Rui Xue 302d80afc5c8d1f2c54c5f934078817de2cd77ba m
PIPE-684: set VLA pblimit_image to 0.1 for mosaic non-cube imaging
- Add optional `gridder` parameter to `pblimits` across base, VLA, and VLASS image heuristics classes.
- Update VLA heuristics to set `pblimit_image = 0.1` for non-cube mosaic imaging (keeping 0.2 for cube and -0.1 for single-field continuum).- pipeline/hif/heuristics/imageparams_base.py (version 302d80afc5c8d1f2c54c5f934078817de2cd77ba) (diffs)
- pipeline/hif/heuristics/imageparams_vla.py (version 302d80afc5c8d1f2c54c5f934078817de2cd77ba) (diffs)
- pipeline/hif/heuristics/imageparams_vlass_single_epoch_continuum.py (version 302d80afc5c8d1f2c54c5f934078817de2cd77ba) (diffs)
- pipeline/hif/tasks/tclean/tclean.py (version 302d80afc5c8d1f2c54c5f934078817de2cd77ba) (diffs)
-
Rui Xue 963956e8bd22165cc051817a887b04312b9ee253 m
PIPE-3247/PIPE-3248: Improve field name rendering with text wrapping
- Wrap long comma-separated field names weblogs of `findcont` and `uvcontsub`
- Save and present only first field name by parsing composite field names in joint mask sky plots -
Hajime Ezawa 5cbdb034e56a934f48b0646fe5a64a81c01fb13c m
PIPE-3105: optimized the default for min_gap_width of _pick_excess_ranges()
-
Hajime Ezawa 1113f6eeab147278cd16547f836b6d5012ed49df m
PIPE-3246: add overwrite=True to image_args
-
Hajime Ezawa 66f8cef3c33294e6c7b3c365bb204e8a86590f2c m
PIPE-3105: improve np.array masking
-
Hajime Ezawa 0d0624f895555487cbae89ca6d66f6ec1a62ead9 m
PIPE-3105: explicitly specify constant_values for np.pad
-
Hajime Ezawa ff61d16aa4b061eca66dadd0e8e407ec5996be65 m
PIPE-3105: bug fix
-
Hajime Ezawa 2cd98c4db2cebb5d9730411d612115d4b8064652 m
PIPE-3105: add/correct type annotations
-
Hajime Ezawa f173553d5ca12d31bef7c79a0053eabe491aaa77 m
PIPE-3105: optimize/correct whitespaces
-
Hajime Ezawa b8f5fbe45dc53e71ad398bdfebba18ee8019bdff m
PIPE-3105: replace .format() to f-string, typo correction
-
Hajime Ezawa 1d10f0b1efcfdf31fefe6d07965f5a22837c2ae0 m
Merge branch 'main' of https://open-bitbucket.nrao.edu/scm/pipe/pipeline into PIPE-3105-hsd_imaging-possible-improvement-on-the-weblog-layout-for-no-detection-of-off-line-range
-
Hajime Ezawa f806c297ba7084b81d8c36e31d5e3f7293549e44 m
PIPE-3105: output excess ranges to LOG
-
Rui Xue da22be2d53d62c4000bea3bf069e6e1ff56e4778 m
PIPE-684: consolidate flagged SPW warnings and skip fully flagged targets in makeimlist
- Aggregate completely flagged SPW warnings into a single consolidated message per EB/field using SPW ranges.
- Filter `field_intent_list` to skip imaging target generation and log warnings for fields that have no valid data within a receiver band.
- Add an explicit warning when no valid data remains across an entire receiver band. -
Rui Xue fa6824a49bda0ef9a3aa3bbdf576683fe62c0a2d m
PIPE-3218: display spectral window lists as ranges in `exportdata` tables
-
Rui Xue a2331a30330f62f5bd84a9ecd64599f36f6c978a m
PIPE-684: Enable per-field mitigation parameter lookups in `makeimlist`
Move `hm_cell` and `hm_imsize` parsing into `field_intent` loop to support field-specific custom mitigation parameters. Update dictionary keys from spwspec to `(field, spwspec)` tuples for field-specific lookups. -
Rui Xue 6ce2b6b36cd37892041044f1b4576fd4575e2d95 m
PIPE-684: Add optional field parameter to `hm_cell/hm_imsize` lookups
- Add field argument to `get_spw_hm_cell()` and `get_spw_hm_imsize()` for custom
field-specific mitigation parameter lookups
- Defaults to `self.field` for backward compatibility
- Reorder `hm_cell` parsing to after spw filtering for better code organization -
Rui Xue 670e29aca8932b3e3f8e760256e8e7c1870500eb m
PIPE-684: improve `wrap_long_str` - track line length instead of using old modulo arithmetic
-
Rui Xue d4978fd1728df6f0b0e6aaf61fb3e0336848e652 m
Merge remote-tracking branch 'origin/main' into PIPE-684-vla-mosaic-imaging
-
Rui Xue 963912718e33f9a2f72ba765774b003f2932c5a5 m
Merge branch 'PIPE-684-vla-mosaic-imaging' of https://open-bitbucket.nrao.edu/scm/pipe/pipeline into PIPE-684-vla-mosaic-imaging
-
Rui Xue 6a8be109f9b22e126cb5e773430a0c17e7c8c422 m
PIPE-684: fix multi-target product size mitigation with per-(field,spw) keying
Fix four bugs in mitigation parameter propagation:
- Dictionary overwriting: use .update() to accumulate params
- Wrong task instance: use local_makeimlist_inputs for second-level mitigation
- Conditional logic: prioritize mitigation over user defaults
- SPW-only keying: change to (field,spw) tuple for proper isolation- pipeline/hif/heuristics/checkproductsize.py (version 6a8be109f9b22e126cb5e773430a0c17e7c8c422) (diffs)
- pipeline/hif/tasks/checkproductsize/renderer.py (version 6a8be109f9b22e126cb5e773430a0c17e7c8c422) (diffs)
- pipeline/hif/tasks/makeimlist/makeimlist.py (version 6a8be109f9b22e126cb5e773430a0c17e7c8c422) (diffs)
-
Rui Xue 1a69811f2d2c52598eb8a9352bfba39903896042 m
PIPE-3218: fix HTML line breaks for long field names
Use `wrap_long_str(field, newline='<br>')` to properly render line breaks in long field names in the findcont table instead of invisible newlines. -
Rui Xue 95214d3a64821c117c187f3cd43768691b910db6 m
PIPE-684: Fix product size calculation and add field filtering
- Fix product size accumulation: sum all target sizes per SPW instead of last target only
- Fix max() on empty cubesizes dict with safe default
- Add 5% tolerance factor for product size limit checks
- Add field filtering support in VLA imaging heuristics with name/ID/source/pattern matching -
Rui Xue fc795b0a549b65680e113a5605a31c571999068e m
PIPE-3237: optimize unflagged data checks in `hif_findroi` and `ImageParamsHeuristics`
- Deduplicate target field names in hif_findroi using utils.deduplicate to avoid redundant probes across mosaic pointings
- Short-circuit the EB/field probe loop in hif_findroi as soon as unflagged data is confirmed for an SPW
- Break early in ImageParamsHeuristics.has_data once valid data is found for a field intent -
Rui Xue 912dd3dd9292793e05fd70a65e26038607f57e3b m
PIPE-684: disable self-calibration heuristics for VLA mosaic targets
- Disable self-calibration heuristics in `hif_selfcal` for VLA composite mosaic observations (multiple pointings) while preserving single-pointing VLA and ALMA mosaic self-calibration.
- Format skipped target warnings using `utils.condense_field_names` and `utils.find_ranges` for cleaner log output.
- Update self-calibration QA message to reflect unsupported VLA mosaic mode.
- Add `scal` argument docstring in `MakeImListInputs`. -
Rui Xue 5aa024e4bee0ea4643dc91739c0e4bd93630dbb6 m
PIPE-3218: wrap long source names in exportdata.mako weblog tables
- Use `utils.wrap_long_str` with `<br>` in `pipeline/h/templates/exportdata.mako` across all image tables to prevent long comma-separated mosaic source names from expanding table columns horizontally. -
Rui Xue b2d6cdac0bd96869e722a76fa8841205b206ffdc m
PIPE-3218: wrap long source names in `hifv_exportdata` weblog tables
-
Rui Xue d7e1c886146b70fea8425e0f1eaf2267e2351103 m
PIPE-684: export `wrap_long_str` in pipeline.infrastructure.utils
- Add `wrap_long_str` to `__all__` in `utils.py` so it is properly exposed in the top-level `pipeline.infrastructure.utils` package namespace.
- Add unit tests for `wrap_long_str` in `utils_test.py`. -
Rui Xue 61caed82fb7256a88d2cbd6b5bd7bdabb24f441e m
PIPE-684: condense long composite field names in QA and log messages
- Add `utils.condense_field_names` to abbreviate long comma-separated field lists for presentation and logging.
- Add unit test coverage for `condense_field_names` in `conversion_test.py`.
- Apply `condense_field_names` to log messages and QA score banners across `makeimlist`, `makeimages`, and `tclean`.
- Modernize message formatting to f-strings in `tclean/qa.py` and use lazy logging in `tclean.py`.- pipeline/hif/tasks/makeimages/makeimages.py (version 61caed82fb7256a88d2cbd6b5bd7bdabb24f441e) (diffs)
- pipeline/hif/tasks/makeimlist/makeimlist.py (version 61caed82fb7256a88d2cbd6b5bd7bdabb24f441e) (diffs)
- pipeline/hif/tasks/tclean/qa.py (version 61caed82fb7256a88d2cbd6b5bd7bdabb24f441e) (diffs)
- pipeline/hif/tasks/tclean/tclean.py (version 61caed82fb7256a88d2cbd6b5bd7bdabb24f441e) (diffs)
- pipeline/infrastructure/utils/conversion.py (version 61caed82fb7256a88d2cbd6b5bd7bdabb24f441e) (diffs)
- pipeline/infrastructure/utils/conversion_test.py (version 61caed82fb7256a88d2cbd6b5bd7bdabb24f441e) (diffs)
-
Rui Xue cc949427bb5b2336cfc85fc552fe977457f540e5 m
PIPE-684: merge weblog table columns in findcont and VLA exportdata
- Apply utils.merge_td_columns to the imaging summary table in hif_findcont.
- Introduce dedicated T2_4MDetailsVLAExportDataRenderer and template for hifv_exportdata to merge repeated source/spw rows in exported FITS image tables.
- Add defensive guards for missing image lists and dictionary key access.- pipeline/hif/tasks/findcont/renderer.py (version cc949427bb5b2336cfc85fc552fe977457f540e5) (diffs)
- pipeline/hif/templates/findcont.mako (version cc949427bb5b2336cfc85fc552fe977457f540e5) (diffs)
- pipeline/hifv/tasks/exportdata/__init__.py (version cc949427bb5b2336cfc85fc552fe977457f540e5) (diffs)
- pipeline/hifv/tasks/exportdata/renderer.py (version cc949427bb5b2336cfc85fc552fe977457f540e5) (diffs)
- pipeline/hifv/templates/exportdata.mako (version cc949427bb5b2336cfc85fc552fe977457f540e5) (diffs)
-
Rui Xue d2879d37ba29ca61808565a31b12584c644ce647 m
PIPE-684: use `self.spwids` for field lookup and frequency calculation in VLA heuristics
-
Rui Xue 948332397d4aef69335df5e92434fb14ab3d0b21 m
Merge remote-tracking branch 'origin/main' into PIPE-684-vla-mosaic-imaging
Resolve conflicts:
- `pipeline/hif/heuristics/imageparams_base.py` -
Rui Xue e16ab072f5445b683d8fedab9860a1b87d056cd1 m
PIPE-684: optimize logging and improve VLA field intent docs
- Convert LOG.info to lazy logging pattern (defer string interpolation)
- Add return type hint to field_intent_list: set[tuple[str, str]]
- Expand docstring with Args/Returns and mosaic behavior details
- Remove unnecessary `MosaicDetectionHeuristics()` instantiation
- Add inline comments documenting mosaic processing steps
- Clean up imports, blank lines, and quote consistency- pipeline/hif/heuristics/imageparams_base.py (version e16ab072f5445b683d8fedab9860a1b87d056cd1) (diffs)
- pipeline/hif/heuristics/imageparams_vla.py (version e16ab072f5445b683d8fedab9860a1b87d056cd1) (diffs)
- pipeline/hif/tasks/makeimlist/makeimlist.py (version e16ab072f5445b683d8fedab9860a1b87d056cd1) (diffs)
-
Rui Xue 5ef52151133ecf3b46bb6c421ca4208d17a5ece6 m
PIPE-684: Refactor mosaic detection with modern Python standards and comprehensive tests
- Convert MosaicDetectionHeuristics methods to @staticmethod (stateless design)
- Upgrade to Python 3.12+ type hints (PEP 604/585): list[], tuple[], dict[]
- Add Google-style docstrings with Args/Returns sections
- Add create_mosaic_groups() testing interface
- Refactor check_targets_for_mosaic() to reuse clustering logic
- Add comprehensive test suite (12 tests):
* Core clustering scenarios (low/high declination, independent fields)
* Edge cases (empty, single field, transitive closure)
* Parameter validation (overlap_tol)
* Unit tests for helper methods (merge_pairs, normalize_dec) -
Rui Xue c1e6f762cd2f0325aa46243b4050f19e74f3fb10 m
PIPE-684: Fix composite source detection and consolidate `wrap_long_str`
- Fix makeimlist.py: Check all fields for composite sources (comma-separated names), not just the first field. Prevents incorrect representative source reordering for mixed VLA field lists.
- Consolidate duplicate wrap_long_str implementations from renderer.py and makeimlist.mako into utils.py with configurable newline parameter.
- Standardize HTML: Use <br> instead of malformed </br> tag.
- Add Python 3.12+ type hints and Google-style docstring to wrap_long_str.- pipeline/hif/tasks/findcont/renderer.py (version c1e6f762cd2f0325aa46243b4050f19e74f3fb10) (diffs)
- pipeline/hif/tasks/makeimlist/makeimlist.py (version c1e6f762cd2f0325aa46243b4050f19e74f3fb10) (diffs)
- pipeline/hif/templates/makeimlist.mako (version c1e6f762cd2f0325aa46243b4050f19e74f3fb10) (diffs)
- pipeline/infrastructure/utils/utils.py (version c1e6f762cd2f0325aa46243b4050f19e74f3fb10) (diffs)
-
Rui Xue 791541abdfcfcba38726455e62e74446599de9cd m
PIPE-684: Add type hints and modernize docstrings in `mosaicoverlap.py`
-
Rui Xue 05b4b0cd3bf3de5a30af603de0c3f6e8c1ac652b m
PIPE-684: Refactor VLA mosaic field handling to be less invasive
Remove mosaic_sources parameter from base class cont_ranges_spwsel()
method and read composite mosaic field names directly from contfile
via ContFileHandler. This eliminates VLA-specific parameter pollution
of the base class interface.
Key changes:
- Read field names from contfile to handle composite mosaic sources
(e.g., "Field_A,Field_B") that don't exist in MS source table
- Remove tmpfield extraction that was breaking selfcal lookups by
splitting mosaic field names
- Use full field_intent[0] for selfcal parameter queries
- Deduplicate source names to avoid redundant dict initialization
This makes VLA mosaic support non-invasive while maintaining backward
compatibility with ALMA and single-field imaging.- pipeline/hif/heuristics/imageparams_base.py (version 05b4b0cd3bf3de5a30af603de0c3f6e8c1ac652b) (diffs)
- pipeline/hif/tasks/makeimlist/makeimlist.py (version 05b4b0cd3bf3de5a30af603de0c3f6e8c1ac652b) (diffs)
- pipeline/hif/tasks/tclean/tclean.py (version 05b4b0cd3bf3de5a30af603de0c3f6e8c1ac652b) (diffs)
-
Rui Xue 81b2d36c691c9ceaa10b424dcbfc68ea1a0ce1f6 m
PIPE-684: Refactor VLA mosaic detection to use direct spw->field mapping
Remove VLA-specific parameters from factory interface and simplify
by directly querying fields for current spws instead of deriving
band first. Keeps VLA logic isolated within VLA heuristics class.- pipeline/hif/heuristics/imageparams_factory.py (version 81b2d36c691c9ceaa10b424dcbfc68ea1a0ce1f6) (diffs)
- pipeline/hif/heuristics/imageparams_vla.py (version 81b2d36c691c9ceaa10b424dcbfc68ea1a0ce1f6) (diffs)
- pipeline/hif/tasks/makeimlist/makeimlist.py (version 81b2d36c691c9ceaa10b424dcbfc68ea1a0ce1f6) (diffs)
-
Hajime Ezawa 49c959405ffef5020caf82255ba4a8fe3bc83c38 m
PIPE-3105: always show two plots regardless the detection status
