Build: #8 failed
Job: Pipeline PR Test 6.7.4 failed
Code commits
Pipeline
-
Rui Xue 30e5aa8dae391bec613a3bcd438690de7b97d4ea
Merge remote-tracking branch 'origin/main' into PIPE-2941-refactor-pipeline-auto_selfcal-code-to-better-match-structure-of-gh-auto_selfcal-in
-
Rui Xue 0091d599c11a65dba59cd08a8b42a5697278d177 m
Pull request #2057: PIPE-3111: separate correctedampflag QA scores by intent and ensure deterministic ordering
Merge in PIPE/pipeline from PIPE-3111-fix-random-order-of-qa-scores-from-correctedampflag to main
* commit 'b62277a0e4dd638b9558cb7f4a6177e311e45bfc':
PIPE-3111: separate correctedampflag QA scores by intent and ensure deterministic ordering -
Rui Xue 9561135b97de612b809c9d884cf96dcb16a9a4c6 m
Pull request #2066: PIPE-3306: update github workflows
Merge in PIPE/pipeline from PIPE-3306-update-github-workflows to main
* commit '2099a082184b13f17896370d5e7aff4bb04b0c49':
PIPE-3306: update Bitbucket mirror cron and RTD docs
PIPE-3306: update Read the Docs workflow and automation rules
PIPE-3306: update Pixi workflow actions and runners -
Rui Xue 2099a082184b13f17896370d5e7aff4bb04b0c49 m
PIPE-3306: update Bitbucket mirror cron and RTD docs
- Lower the scheduled cron in sync-from-open-bitbucket.yml from 15 minutes to a 2-hour fallback now that pushes trigger the mirror via Bitbucket webhooks and a Cloudflare Worker relay.
- Update docs/source/devel/setup/readthedocs.md to document the webhook-driven mirror flow and 2-hour fallback schedule. -
Rui Xue 1043d17a90635a25886d6b1073c24811f829625f m
PIPE-3306: update Read the Docs workflow and automation rules
- Update `docs/source/devel/setup/readthedocs.md` to reflect the new Bitbucket @ NRAO to GitHub mirror workflow and Read the Docs branch building automations. -
Rui Xue 98eee0111660794b9d86cba15ea4e369c5b6d4b7 m
PIPE-3306: update Pixi workflow actions and runners
-
Patrick Bos 2a803cbc8b200e4ce3727d90e47f0628dc61bb7b m
Pull request #2031: PIPE-3189: reduce field.time samples in SQLD datasets
Merge in PIPE/pipeline from PIPE-3189-field-time-sqld to main
* commit '80f109819d196a7339ad449276a7cccfbf7b8e08':
PIPE-3189: stratify the decimation onto a uniform time grid
PIPE-3189: relax zd test tolerance to survive measures-data refreshes
PIPE-3189: decimate field.time in compute_zd_telmjd_for_ms -
Rui Xue 5894b60df56c96951f7a23aba7d213598661028e m
Pull request #2064: Mirror all Bitbucket branches and tags
Merge in PIPE/pipeline from ci/mirror-bitbucket-to-github to main
* commit 'b3e4587ebfe6070920dbdb248c4f59c1df396968':
Mirror all Bitbucket branches and tags -
Rui Xue b3e4587ebfe6070920dbdb248c4f59c1df396968 m
Mirror all Bitbucket branches and tags
Replace the main-only checkout and fast-forward merge workflow with a bare-repository fetch and atomic pruned push of all branches and tags from Bitbucket to GitHub. -
Devaky Kunneriath 9974cb59648b24a7fff882545cbff2a013b0fa27 m
Pull request #2062: Adding hifa_tsysflagcontamination() stage to all polcal and B2B recipes
Merge in PIPE/pipeline from PIPE-3303-add-hifa_tsysflagcontamination-to-b2b-and-full-polarization-recipes to main
* commit 'ded4ac779f47598aa4148b6591feccf7f0640883':
PIPE-3303: update recipe description and known issues
Adding hifa_tsysflagcontamination() stage to all polcal and B2B recipes -
Rui Xue ded4ac779f47598aa4148b6591feccf7f0640883 m
PIPE-3303: update recipe description and known issues
- Add hifa_tsysflagcontamination to hifa_polcalimage-recipe.md
- Mark PIPE-2407 as fixed in known_issues.md -
Devaky Kunneriath <devaky.k@gmail.com> ab0051f2004dc728adb228e2a1a43a345dd0d60c m
Adding hifa_tsysflagcontamination() stage to all polcal and B2B recipes
- pipeline/recipes/procedure_hifa_cal_diffgain.xml (version ab0051f2004dc728adb228e2a1a43a345dd0d60c) (diffs)
- pipeline/recipes/procedure_hifa_calimage_diffgain.xml (version ab0051f2004dc728adb228e2a1a43a345dd0d60c) (diffs)
- pipeline/recipes/procedure_hifa_polcal.xml (version ab0051f2004dc728adb228e2a1a43a345dd0d60c) (diffs)
- pipeline/recipes/procedure_hifa_polcal_totalintensity.xml (version ab0051f2004dc728adb228e2a1a43a345dd0d60c) (diffs)
- pipeline/recipes/procedure_hifa_polcalimage.xml (version ab0051f2004dc728adb228e2a1a43a345dd0d60c) (diffs)
- pipeline/recipes/procedure_hifa_polcalimage_totalintensity.xml (version ab0051f2004dc728adb228e2a1a43a345dd0d60c) (diffs)
-
Andres Guzman 52e772f577625a28faa951ccafd1a76d06e09bce m
Pull request #1987: PIPE-3257 improve tsysflagcontamination design scope
Merge in PIPE/pipeline from PIPE-3257-improve-tsysflagcontamination-design-scope to main
* commit '87763361a92605510e47f245f21d83990a4239b2':
Corrected some indices that went above the edge of the spw.
PIPE-3257: Clean up imports, staticmethod decorators, and dead code in Tsys contamination
recommendations on extern code by Rui
Correcting some bugs in both files of the extern code. PIPE-3257
Add context input to the functions that require it. Change long message to `Flagging not applied...`
Modifications of the three files associated with Tsys contamination. Description in PIPE-3257 and PIPEREQ-449 -
Rui Xue db53ef88f4b69453e37adcf95ac56bf27b2b6bff m
Pull request #2056: PIPE-3292: disable SCIPIPE_ROOTDIR during regression PPR-based testing runs
Merge in PIPE/pipeline from PIPE-3292-disable-scipipe_rootdir-during-regression-testing-runs to main
* commit '6bd243c6817d0e6e6491d4f3a1257dd88ee70ca5':
PIPE-3292: disable SCIPIPE_ROOTDIR during regression PPR-based testing runs -
Rui Xue b62277a0e4dd638b9558cb7f4a6177e311e45bfc m
PIPE-3111: separate correctedampflag QA scores by intent and ensure deterministic ordering
- Preserve input intent list order when filtering against MS intents in `CorrectedampflagQAHandler` to eliminate set hash-seed nondeterminism.
- Tag each per-intent QA score with applies_to.intent.
- Include intent in regression extractor data selection to disambiguate per-intent QA metric and score keys. -
Rui Xue 6bd243c6817d0e6e6491d4f3a1257dd88ee70ca5 m
PIPE-3292: disable SCIPIPE_ROOTDIR during regression PPR-based testing runs
-
Rui Xue bcbe7cdf996471859959552b1eb570c0e0aeab8c m
Pull request #2008: PIPE-2545: split scorecalculator into modules
Merge in PIPE/pipeline from PIPE-2545-split-scorecalculator.py-into-smaller-files to main
* commit '06cde5f7675c4249caa767af15afd6ec910a790f':
PIPE-2545: flatten qa submodule documentation
PIPE-2545: omit `scorecalculator` from api reference
PIPE-2545: streamline scorecalculator facade
PIPE-2545: split scorecalculator into modules -
Rui Xue 06cde5f7675c4249caa767af15afd6ec910a790f m
PIPE-2545: flatten qa submodule documentation
- Add `flat-module-template.rst` to document QA scoring functions directly on their submodule pages instead of generating individual autosummary stub pages.
- Update `pipeline.qa.rst` to use the flat template and omit internal helpers (`bpcal`, `checksource`, `gpcal`), focusing the API reference on the 5 score calculation submodules. -
Rui Xue 83b74cf77dfca0f05415cc84e121e101b8331867 m
PIPE-2545: omit `scorecalculator` from api reference
- Remove the scorecalculator facade from the `pipeline.qa` autosummary list to prevent duplicating all 116 scoring functions across doc pages.
- Add an admonition note explaining its role as a backward-compatible facade. -
Rui Xue 780661c709939297ca9603edd14824c147ced8d9 m
PIPE-2545: streamline scorecalculator facade
- Dynamically re-export submodule functions to reduce facade boilerplate, and add a TYPE_CHECKING getattr stub to eliminate IDE warnings.
- Remove redundant export tests and improve submodule export checks to verify cross-module symbol uniqueness and identity. -
Rui Xue 4902b0d621003769e8d80391d22e92f8ea4ed9d9 m
Merge remote-tracking branch 'origin/main' into PIPE-2545-split-scorecalculator.py-into-smaller-files
Resolved conflicts:
- pipeline/qa/scorecalculator_test.py
- pipeline/qa/scorecalculator.py -
Rui Xue bc947db09431022416a992b8a8742fbd1212344a m
Pull request #2054: Docs/update contributors list
Merge in PIPE/pipeline from docs/update-contributors-list to main
* commit '281f5973de6fe462fe037ccd19a571fa0334b77b':
docs(contributors): update start year for H.E.
Add DK into the contributors list. -
Rui Xue 281f5973de6fe462fe037ccd19a571fa0334b77b m
docs(contributors): update start year for H.E.
-
Remy Indebetouw <remy@virginia.edu> 69f06a97725f2c32090d1354b2ad98254b4e3978 m
Add DK into the contributors list.
-
Rui Xue 6d02febd52e34bcff87428cd28c0aeaec78dc1dc m
Pull request #2052: PIPE-3290: set min pixi version requirement in runtime env specs
Merge in PIPE/pipeline from PIPE-3290-set-min-pixi-version-requirement-in-runtime-env-specs to main
* commit '0435a7540000b02dd64304fbfbd105d51f663719':
PIPE-3290: Restrict CI LFS pulls to required test data
PIPE-3290: Centralize CI test data paths
PIPE-3290: Update CASA snapshots in development CI
PIPE-3290: Mitigate emSign certificate incident -
Rui Xue 0435a7540000b02dd64304fbfbd105d51f663719 m
PIPE-3290: Restrict CI LFS pulls to required test data
-
Rui Xue 5c6d13634a94637c8aa0bdc9fff85153af43741b m
PIPE-3290: Centralize CI test data paths
- Include the required regression MS and hash the directory list in cache keys so changes refresh cached data and trigger unit CI. -
Rui Xue 5001af31e59ce37b7dbcf7d4d27960629826fb6a m
PIPE-3290: Update CASA snapshots in development CI
- Keep release, tag, and manual runs locked.
- Separate concurrency groups by event and ref.- .github/workflows/env-setup-run-template-pixi.yml (version 5001af31e59ce37b7dbcf7d4d27960629826fb6a) (diffs)
- .github/workflows/test-unit-pixi.yml (version 5001af31e59ce37b7dbcf7d4d27960629826fb6a) (diffs)
- docs/source/devel/setup/pixi_tasks.md (version 5001af31e59ce37b7dbcf7d4d27960629826fb6a) (diffs)
-
Rui Xue 5c77ea9c494e0069d1aeac07e313df237165082c m
PIPE-3290: Mitigate emSign certificate incident
- Require patched Python, pip, certifi, and CA bundles, and refresh the cross-platform lockfile.
- Upgrade Pixi bootstrap versions and use locked dependencies in CI and Read the Docs.- .github/workflows/env-setup-run-template-pixi.yml (version 5c77ea9c494e0069d1aeac07e313df237165082c) (diffs)
- .readthedocs.yaml (version 5c77ea9c494e0069d1aeac07e313df237165082c) (diffs)
- docs/source/devel/setup/dependencies.md (version 5c77ea9c494e0069d1aeac07e313df237165082c) (diffs)
- docs/source/devel/setup/pixi_tasks.md (version 5c77ea9c494e0069d1aeac07e313df237165082c) (diffs)
- pixi.lock (version 5c77ea9c494e0069d1aeac07e313df237165082c) (diffs)
- pyproject.toml (version 5c77ea9c494e0069d1aeac07e313df237165082c) (diffs)
-
Rui Xue 8fb533bdf116ea5e7a6167285a7f1e1deaf4513e m
Pull request #2022: PIPE-3253: preserve in-band uniform cell size in `makeimlist`
Merge in PIPE/pipeline from PIPE-3253-makeimlist-preserve-in-band-default-cell to main
* commit 'c8c11ae226f0dd305bab02dec263a98f813fbcc9':
PIPE-3253: refactor cell calculation helpers in makeimlist and clean up heuristics
PIPE-3253: support telescope-specific cell determination via heuristics
PIPE-3253: preserve in-band uniform cell size in `makeimlist` -
Andres Guzman <andres.guzman@Andress-MacBook-Pro.local> 87763361a92605510e47f245f21d83990a4239b2 m
Corrected some indices that went above the edge of the spw.
-
Rui Xue c66cb91dcf6402eeb1d2ea063beba8dcbd20b9b1 m
Pull request #2038: PIPE-3283: Fix VLASS `makecutoutimages` weblog statistics
Merge in PIPE/pipeline from PIPE-3283-makecutoutimages-presenting-wrong-stats-values-in-the-ccip-workflow to main
* commit '3b562035097661380eba426b8ae9b9e480aa6d69':
PIPE-3283: Fix VLASS `makecutoutimages` weblog statistics -
Rui Xue db3fda4f9301bdbd886b6d836474aa08fbc4fd70 m
Pull request #2047: PIPE-3285: Refactor pyproject.toml; Update dependency management and environment features
Merge in PIPE/pipeline from PIPE-3285-update-plotms-index to main
* commit 'd0b5cceb8fb51484fc4cd9775fc9a60812d17b5b':
PIPE-3285: Refactor pyproject.toml; Update dependency management and environment features -
Rui Xue d0b5cceb8fb51484fc4cd9775fc9a60812d17b5b m
PIPE-3285: Refactor pyproject.toml; Update dependency management and environment features
- Clarified comments regarding common dependencies and their inheritance.
- Added extra PyPI index URLs for package resolution.
- Simplified dependency declarations for casampi, casaplotms, casatasks, and casatools.
- Introduced environment variable settings for Python warnings.aa
- Consolidated CASA version features into a more streamlined structure.
- Updated environment definitions to use feature lists instead of individual feature references.
- Adjusted the update-references task to a simpler command format. -
Rui Xue 03bb819e0f4a951d97ea64bb0c62ca441a082b52 m
Pull request #2046: Docs/add selfcal dr statement to main
Merge in PIPE/pipeline from docs/add_selfcal_dr_statement_to-main to main
* commit '3d176781bcf4d3b5eab11a5740864a627353880e':
Update clean threshold and automasking docs
Add a statement about the dynamic range correction factor applied for selfcal images to the hif_makeimages documentation. -
Rui Xue 3d176781bcf4d3b5eab11a5740864a627353880e m
Update clean threshold and automasking docs
- Convert automasking and dynamic range tables to reST list-tables with Unicode math symbols and compact header blocks.
- Add bibtex citation for the auto-multithresh reference.
- Link tclean via Intersphinx. -
Patrick Sheehan 004b7951361fa36f48f262eb542980a0885b3add m
Add a statement about the dynamic range correction factor applied for selfcal images to the hif_makeimages documentation.
-
Ajay Vibhute 0bd065b432fe85e7558e30814c1c2d0d56e1deb9 m
Pull request #1801: PIPE-3067: scripts to populate fits header keywords using information available in product directory
Merge in PIPE/pipeline from PIPE-3067-standalone-script-for-vlass-header-keywords to main
* commit 'c77e37077037a1828afe8c6b8af11d6f23b90b4b':
PIPE-3067: updated vlassbwn for se-cube and other imaging modes
PIPE-3067: updated VLASSPT, VLASSWP, VLASSPL, VLASSRJ
PIPE-3067: updated docstrings, added requirements and dry-run option
PIPE-3067: scripts to populate fits header keywords using information available in product directory -
Rui Xue 699eb1dcb419c129c820390ce47cc2043658a244 m
PIPE-3257: Clean up imports, staticmethod decorators, and dead code in Tsys contamination
-
Rui Xue 4bbbee5b965c40dc0e3b229aa32d6cf8cbc157de m
Pull request #1950: PIPE-3222/PIPE-3221/PIPE-889
Merge in PIPE/pipeline from PIPE-3222-redundant-edgechan-flag-command-in-h_tsysflag to main
* commit '6ecc99029229649dd32a98e3427f3a85754aaff8':
PIPE-3222: Deduplicate edgechans flag commands in h_tsysflag
PIPE-3221: Make applycal field ordering deterministic
PIPE-3221: Fix non-deterministic tsys spw iteration in tsysflag -
Rui Xue df36fe412134d0e89fcfcdc87a3edf3748f2b201 m
Pull request #2043: PIPE-3252: migrate to SciX API
Merge in PIPE/pipeline from PIPE-3252-update-bibtex-query-end-point-from-ads-to-scixplorer-transition to main
* commit '7069ad4d871a67029216005306848f17277e1ae0':
PIPE-3252: migrate to SciX API -
Rui Xue 7069ad4d871a67029216005306848f17277e1ae0 m
PIPE-3252: migrate to SciX API
- Migrate update_references.py to the NASA SciX API endpoint (api.scixplorer.org) with automatic fallback to adsabs.harvard.edu.
- Add SCIX_API_TOKEN and ~/.scix/token support to token discovery.
- Update reference synchronization documentation and refresh pipeline.bib from the SciX public library.- .skills/docs-build/SKILL.md (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- docs/source/devel/setup/pixi_tasks.md (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- docs/source/references/pipeline.bib (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- docs/source/references/readme.md (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- docs/source/releases.md (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- docs/source/users_guide/known_issues.md (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- pyproject.toml (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- scripts/README.md (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
- scripts/update_references.py (version 7069ad4d871a67029216005306848f17277e1ae0) (diffs)
-
Rui Xue 969aa3ff78c76b64383260f471bfa3d2c0ab0e9a m
Pull request #2042: PIPE-3252: update releases and known issues for CASA 6.7.4-12 patch
Merge in PIPE/pipeline from PIPE-3252-update-docs-for-vla-and-alma-patch-releases to main
* commit '523b6b9ec6c541dfe84ce6b44ceb8c9984df4c2d':
PIPE-3252: update releases and known issues for CASA 6.7.4-12 patch -
Rui Xue 523b6b9ec6c541dfe84ce6b44ceb8c9984df4c2d m
PIPE-3252: update releases and known issues for CASA 6.7.4-12 patch
- Note fixed issues (CAS-14850 and CAS-14874) in PL2026 known issues. -
Rui Xue aeb58b104b1d8fb14d4362574b1c1153142ef089 m
Pull request #2040: PIPE-3252: document DataType design
Merge in PIPE/pipeline from PIPE-3252-add-datatype-use-cases-notes to main
* commit '3c1c56edcfbfba77e87a6f4f60fedf1cf1b00f28':
PIPE-3252: update DataType reduction and restore workflows
PIPE-3252: refine datatype doc and diagram layout
PIPE-3252: document DataType design and lifecycle -
Rui Xue 3c1c56edcfbfba77e87a6f4f60fedf1cf1b00f28 m
PIPE-3252: update DataType reduction and restore workflows
Document ALMA and VLA reduction lifecycles and self-cal restore workflows with consolidated data nodes, line target application, and responsive Mermaid rendering.
- datatypes.md: add side-by-side ALMA and VLA lifecycles, clarify self-cal restore sequence, and fix Sphinx refs.
- furo_layout.js: support responsive container scaling for wide multi-column diagrams.
- conf.py: bump mermaid_version to 12.2.0. -
Rui Xue 102a12f9146d79b7d59da703fe67015ab10ea3a7 m
Pull request #2041: PIPE-3252: update releases table for `2026.2.0` and `2026.2.1`
Merge in PIPE/pipeline from PIPE-3252-update-docs-for-vla-and-alma-patch-releases to main
* commit '04e87fcdb8deb467821d4d612eaedf33201c8634':
PIPE-3252: update releases table for `2026.2.0` and `2026.2.1` -
Rui Xue 04e87fcdb8deb467821d4d612eaedf33201c8634 m
PIPE-3252: update releases table for `2026.2.0` and `2026.2.1`
- Update documentation references to target RTD slugs v2026.2.0 and v2026.2.1 using domain root-relative HTML anchors to prevent Sphinx cross-reference warnings.
- Update CASA version and download tarballs for 2026.2.1.55, and mark superseded 2026.2.0.27 August packages as struck-through. -
Rui Xue fc52a4bacf7c0998a62fe67cd2990beba2b18bde m
PIPE-3252: refine datatype doc and diagram layout
- Add API cross-references and prerequisite notes for `CORRECTED_DATA` initialization to the DataType reference documentation.
- Disable the unused ELK layout engine to reduce page bundle size.
- Update Mermaid container styling and dynamic aspect-ratio scaling to prevent Gantt chart horizontal collapse while bounding tall flowcharts.- docs/source/_static/custom_theme.css (version fc52a4bacf7c0998a62fe67cd2990beba2b18bde) (diffs)
- docs/source/_static/furo_layout.js (version fc52a4bacf7c0998a62fe67cd2990beba2b18bde) (diffs)
- docs/source/conf.py (version fc52a4bacf7c0998a62fe67cd2990beba2b18bde) (diffs)
- docs/source/devel/reference/datatypes.md (version fc52a4bacf7c0998a62fe67cd2990beba2b18bde) (diffs)
-
Takeshi Nakazato 78502f29806c81fc1659f7fe614584921e8c0e8c m
Pull request #1984: PIPE-3232 fix errors in log qa when weblog rendering is disabled
Merge in PIPE/pipeline from PIPE-3232-fix-errors-in-log_qa-when-weblog-rendering-is-disabled to main
* commit 'aae06797d595843586eed600982f3e71e7b7dd53':
PIPE-3232: add unit tests for `DISABLE_WEBLOG` and `DISABLE_QA` handling
PIPE-3232: introduce DISABLE_QA flag to control QA score generation
PIPE-3232: return empty plot dict from flagging tasks when weblog is disabled
PIPE-3232 update log_qa decorator
PIPE-3232 score_amp_vs_time_plots should skip QA evaluation if weblog is disabled
PIPE-3232 add/remove log_qa decorator depending on the return value of the function -
Rui Xue 1d2a4b07e290550e709635a2830a3eb3e69b841c m
PIPE-3252: document DataType design and lifecycle
- Add a developer reference guide detailing the Pipeline DataType design, covering in-memory domain tracking, on-disk persistence in MS HISTORY, reduction lifecycle stages, and downstream imaging selection.
- Also adjust Sphinx Mermaid layout configuration, centering styles, and interaction controls in custom theme assets to improve diagram readability.- docs/source/_static/custom_theme.css (version 1d2a4b07e290550e709635a2830a3eb3e69b841c) (diffs)
- docs/source/_static/furo_layout.js (version 1d2a4b07e290550e709635a2830a3eb3e69b841c) (diffs)
- docs/source/conf.py (version 1d2a4b07e290550e709635a2830a3eb3e69b841c) (diffs)
- docs/source/devel/reference/datatypes.md (version 1d2a4b07e290550e709635a2830a3eb3e69b841c) (diffs)
- docs/source/devel/reference/index.md (version 1d2a4b07e290550e709635a2830a3eb3e69b841c) (diffs)
-
Andres Guzman <andres.guzman@Andress-MacBook-Pro.local> 499c0b30430372d6dfe6086eee85e228c7bedccc m
recommendations on extern code by Rui
-
Ajay Vibhute c77e37077037a1828afe8c6b8af11d6f23b90b4b m
PIPE-3067: updated vlassbwn for se-cube and other imaging modes
-
Rui Xue 3b562035097661380eba426b8ae9b9e480aa6d69 m
PIPE-3283: Fix VLASS `makecutoutimages` weblog statistics
- Switch VLASS-coarsecube workflow `makecutoutimages` weblog to use `peak/madrms` to match weblog table headers.- pipeline/hif/tasks/makecutoutimages/makecutoutimages.py (version 3b562035097661380eba426b8ae9b9e480aa6d69) (diffs)
- pipeline/hif/templates/vlasscube_makecutoutimages.mako (version 3b562035097661380eba426b8ae9b9e480aa6d69) (diffs)
- pipeline/infrastructure/utils/utils.py (version 3b562035097661380eba426b8ae9b9e480aa6d69) (diffs)
- pipeline/infrastructure/utils/utils_test.py (version 3b562035097661380eba426b8ae9b9e480aa6d69) (diffs)
-
Rui Xue 6ecc99029229649dd32a98e3427f3a85754aaff8 m
Merge remote-tracking branch 'origin/PIPE-3221-flagging-command-order-from-hifa_tsysflag-might-not-be-deterministic' into PIPE-3222-redundant-edgechan-flag-command-in-h_tsysflag
-
Rui Xue 3a559c545d136093429d82edb2f2105d8d759c4b m
Merge remote-tracking branch 'origin/main' into PIPE-3222-redundant-edgechan-flag-command-in-h_tsysflag
-
E. G. Patrick Bos <e.g.p.bos@rug.nl> 80f109819d196a7339ad449276a7cccfbf7b8e08 m
PIPE-3189: stratify the decimation onto a uniform time grid
Replace the greedy minimum-gap selection in _decimate_timestamps with a
uniform target grid: every min_interval step is snapped to the nearest real
timestamp, duplicates are dropped and the endpoints are re-added explicitly.
Fully vectorised (searchsorted + where + union1d).
Why:
* The grid spacing is honoured exactly. A minimum-gap rule cannot reach a 1s
target from, e.g., 0.576s continuum sampling and has to round up to 1.152s,
under-sampling by a third; on SQLD it settled at 1.008s rather than 1.000s.
* Selection depends only on time, never on sample density, so bursts are
represented in proportion to their duration rather than to their number of
samples.
* It is vectorised: _decimate_timestamps goes from 0.018s (2.9% of the
decimated runtime, a Python loop over 190,080 elements) to 0.001s in the
profile over 3 calls.
The guarantee this buys is "at most one retained point per min_interval of
ELAPSED time" -- a bound on points per unit time, which is what bounds the
cost. It is deliberately not a guarantee that retained points are min_interval
apart: a grid node landing in a gap just past the end of a burst selects that
burst's final timestamp, so on the ticket's data 2.7% of adjacent pairs sit
closer than 0.5s (all of them at burst boundaries) while p50 = 0.992s and
everything outside the boundary pairs is >= 0.99s apart.
Add min_count=50 as a continuous cap on sparsity: the grid tightens to
span/(min_count-1) for a short field rather than falling back to no
decimation. A binary "keep everything below 50" fallback would have handed a
short-but-dense field all its samples back and re-opened the SQLD pathology
for short dwell times.
Measured on the ticket's dataset (SQLD, 3 fields x 63,360 timestamps):
190,080 -> 3,181 measures calls, 23.635s -> 0.429s (55x), ZD min/max
bit-identical on every field, the curve interpolated back onto the full time
base within 5.2e-08 deg (vs 4.7e-06 deg for the greedy rule, on a 12.67 deg
range), and the rendered zd_telmjd.png closer to main than the greedy variant
was (1373 vs 1430 pixels differing by > 8/255). t2-2-4.html and the other
three plots are byte-identical to main.
Honest caveat on the mean, which is what this revision of the message is for.
zd_avg is round(np.average(zd), 2) on t2-1_details, and on this dataset main's
density-weighted mean already coincides with the time-weighted mean to better
than 1e-4 deg -- all 33 bursts are the same length -- so there is no mean bias
for either decimator to recover. The decimated mean is a sample mean of 3,181
points and lands within 0.011 deg of main's per field (0.09% of the ZD range),
which does move the displayed second decimal (26.99 -> 26.98). The greedy rule
did better on that one statistic (+0.0046); the grid wins on curve fidelity
(~90x), on pixel proximity to main, on the vectorisation above, and on
min_count. The claim this message previously carried -- that zd_avg error drops
"from 1.25 to 0.003 in ZD-proportional units ... on the ticket's dataset" --
came from a synthetic experiment, not from this dataset, and is withdrawn.
zd_telmjd_test.py: 17 passed. -
Rui Xue c8c11ae226f0dd305bab02dec263a98f813fbcc9 m
PIPE-3253: refactor cell calculation helpers in makeimlist and clean up heuristics
- Modularize cell calculation and SPW observation checks into MakeImList helper methods.
- Clean up unused VLASS uniform_band_cell overrides and task comments.- pipeline/hif/heuristics/imageparams_base.py (version c8c11ae226f0dd305bab02dec263a98f813fbcc9) (diffs)
- pipeline/hif/heuristics/imageparams_vlass_quick_look.py (version c8c11ae226f0dd305bab02dec263a98f813fbcc9) (diffs)
- pipeline/hif/heuristics/imageparams_vlass_single_epoch_continuum.py (version c8c11ae226f0dd305bab02dec263a98f813fbcc9) (diffs)
- pipeline/hif/heuristics/imageparams_vlass_single_epoch_taper.py (version c8c11ae226f0dd305bab02dec263a98f813fbcc9) (diffs)
- pipeline/hif/tasks/makeimlist/makeimlist.py (version c8c11ae226f0dd305bab02dec263a98f813fbcc9) (diffs)
-
Rui Xue e54a410a056db2d69a3c415c6f48da104a8ec0e1 m
PIPE-3253: support telescope-specific cell determination via heuristics
- Add uniform_band_cell property to ImageParamsHeuristics (True for ALMA, False for VLA/VLASS).
- Update makeimlist Pass 1 to compute shared band-wide default cells only when uniform_band_cell is enabled.
- Update makeimlist Pass 2 to maintain independent per-field cell calculation for VLA/VLASS while preserving ALMA uniform in-band cells and size mitigation overrides.- pipeline/hif/heuristics/imageparams_base.py (version e54a410a056db2d69a3c415c6f48da104a8ec0e1) (diffs)
- pipeline/hif/heuristics/imageparams_vla.py (version e54a410a056db2d69a3c415c6f48da104a8ec0e1) (diffs)
- pipeline/hif/heuristics/imageparams_vlass_quick_look.py (version e54a410a056db2d69a3c415c6f48da104a8ec0e1) (diffs)
- pipeline/hif/heuristics/imageparams_vlass_single_epoch_continuum.py (version e54a410a056db2d69a3c415c6f48da104a8ec0e1) (diffs)
- pipeline/hif/heuristics/imageparams_vlass_single_epoch_taper.py (version e54a410a056db2d69a3c415c6f48da104a8ec0e1) (diffs)
- pipeline/hif/tasks/makeimlist/makeimlist.py (version e54a410a056db2d69a3c415c6f48da104a8ec0e1) (diffs)
-
Rui Xue cb8ffd84968e9a5ca7d560cb80e118cf33910dcc m
Merge remote-tracking branch 'origin/main' into PIPE-3253-makeimlist-preserve-in-band-default-cell
-
Rui Xue 2fd632ecd9f0ab5068be95e8c0bc6e20a40e567b m
PIPE-3253: preserve in-band uniform cell size in `makeimlist`
- Restore shared default cell across targets/calibrators within a receiver band based on the smallest synthesized beam
- Preserve per-target size mitigation overrides (e.g. custom pixperbeam/cell) when specified -
E. G. Patrick Bos <e.g.p.bos@rug.nl> 0fd43ae93457c9bd09299eaf788e92f7c9bb6e04 m
PIPE-3189: relax zd test tolerance to survive measures-data refreshes
zd_telmjd_test.py::test_zd_values used rel=5e-10, tighter than CASA's own
data-driven precision. The shared measurespath geodetic tables
(IERSeop2000, IERSpredict2000, TAI_UTC, IGRF, ...) were refreshed on
2026-10-05 20:04, which shifted every computed ZD by a constant
2.14e-08 deg. test_zd_values then fails on unmodified main as well --
verified against a pristine main worktree, identical failure, identical
diff -- so this is environment drift, not a PIPE-3189 regression.
Use rel=1e-5: ~10,000x the observed drift, yet still 17x tighter than the
~4e-3 deg that a one-second timestamp error would produce, so real
regressions are still caught. Timestamps themselves are asserted exactly
in test_telmjd_exact, which measures data cannot perturb. -
Patrick Bos b73367fc1282bcc4a1f9defe3a6e47627fd81bc2 m
PIPE-3189: decimate field.time in compute_zd_telmjd_for_ms
compute_zd_telmjd_for_ms() iterated over every timestamp in field.time,
doing one measures_doframe + measures_measure per timestamp. For SQLD
spectral-line windows the integration time is ~16ms, so a single field can
yield ~62k timestamps instead of ~1k, all for a weblog plot whose 1s
resolution is already more than adequate.
Add _decimate_timestamps() and apply it to field.time. It greedily keeps
timestamps at least 1s apart and always retains the first and last, so the
full time range is preserved.
The ZD/telmjd values are consumed only by the weblog renderer
(ZDTELMJDChart and the min/avg/max summary stats); they are never written
to a persisted product nor read by another task. Field.time itself is left
untouched, so hifv/tasks/syspower (which uses it for range filtering only)
is unaffected.
Verified on uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small.ms (helms30):
74 -> 55 timestamps, ZD min/max bit-identical, deviation from the full
curve ~1e-10 deg. -
Takeshi Nakazato aae06797d595843586eed600982f3e71e7b7dd53 m
Merge branch 'main' into PIPE-3232-fix-errors-in-log_qa-when-weblog-rendering-is-disabled
-
Rui Xue dcb69a11b89893bbb5c27893670a7a1a7c2897ce m
PIPE-2545: split scorecalculator into modules
-Decompose the monolithic scorecalculator module into domain-focused submodules (`common_scores`, `hif_scores`, `hifa_scores`, `hifv_scores`, and `hsd_scores`) to improve readability and long-term maintainability.
- Maintain full backward compatibility by converting scorecalculator into a facade that re-exports all scoring symbols and classes. Update package initialization and Sphinx documentation to expose the new submodules, and add export integrity unit tests.- docs/source/automodapi/pipeline.qa.rst (version dcb69a11b89893bbb5c27893670a7a1a7c2897ce) (diffs)
- docs/source/conf.py (version dcb69a11b89893bbb5c27893670a7a1a7c2897ce) (diffs)
- pipeline/qa/__init__.py (version dcb69a11b89893bbb5c27893670a7a1a7c2897ce) (diffs)
- pipeline/qa/common_scores.py (version dcb69a11b89893bbb5c27893670a7a1a7c2897ce) (diffs)
- pipeline/qa/hif_scores.py (version dcb69a11b89893bbb5c27893670a7a1a7c2897ce) (diffs)
- pipeline/qa/hifa_scores.py (version dcb69a11b89893bbb5c27893670a7a1a7c2897ce) (diffs)
- pipeline/qa/hifv_scores.py (version dcb69a11b89893bbb5c27893670a7a1a7c2897ce) (diffs)
- pipeline/qa/hsd_scores.py (version dcb69a11b89893bbb5c27893670a7a1a7c2897ce) (diffs)
- pipeline/qa/scorecalculator.py (version dcb69a11b89893bbb5c27893670a7a1a7c2897ce) (diffs)
- pipeline/qa/scorecalculator_test.py (version dcb69a11b89893bbb5c27893670a7a1a7c2897ce) (diffs)
-
Andres Guzman <andres.guzman@Andress-MacBook-Pro.local> 0097fea45d2bf9241d6fc4c847da31ba55147528 m
Correcting some bugs in both files of the extern code. PIPE-3257
-
Andres Guzman <andres.guzman@Andress-MacBook-Pro.local> 9f77c301fffee2a8307e46fdb271d4ae04d6455c m
Add context input to the functions that require it. Change long message to `Flagging not applied...`
-
Rui Xue ce99f6c83f9916cc278db40bc6f9fc1b19ad4f93 m
PIPE-3232: add unit tests for `DISABLE_WEBLOG` and `DISABLE_QA` handling
-
Rui Xue b9b91303c1f5cc136e138d6da858a67f9dd3dc54 m
PIPE-3232: introduce DISABLE_QA flag to control QA score generation
Define DISABLE_QA in basetask.py and guard pipelineqa.qa_registry.do_qa in Results.accept() to allow skipping QA evaluation when disabled. -
Rui Xue a70201b95aefa8acac4d43ae095f035368caea6b m
PIPE-3232: return empty plot dict from flagging tasks when weblog is disabled
Update create_plots() in bandpassflag, gfluxscaleflag, polcalflag, and targetflag to return a dictionary with 'time' and 'uvdist' keys instead of a tuple when DISABLE_WEBLOG is True, preventing downstream TypeErrors.- pipeline/hifa/tasks/bandpassflag/bandpassflag.py (version a70201b95aefa8acac4d43ae095f035368caea6b) (diffs)
- pipeline/hifa/tasks/gfluxscaleflag/gfluxscaleflag.py (version a70201b95aefa8acac4d43ae095f035368caea6b) (diffs)
- pipeline/hifa/tasks/polcalflag/polcalflag.py (version a70201b95aefa8acac4d43ae095f035368caea6b) (diffs)
- pipeline/hifa/tasks/targetflag/targetflag.py (version a70201b95aefa8acac4d43ae095f035368caea6b) (diffs)
-
Andres Guzman <andres.guzman@Andress-MacBook-Pro.local> 8fb7ba833c573a2e9c80734762e37339f2d56daa m
Modifications of the three files associated with Tsys contamination. Description in PIPE-3257 and PIPEREQ-449
- pipeline/extern/TsysDataClassFile.py (version 8fb7ba833c573a2e9c80734762e37339f2d56daa) (diffs)
- pipeline/extern/tsys_contamination.py (version 8fb7ba833c573a2e9c80734762e37339f2d56daa) (diffs)
- pipeline/hifa/tasks/tsysflag_contamination/tsysflagcontamination.py (version 8fb7ba833c573a2e9c80734762e37339f2d56daa) (diffs)
-
Takeshi Nakazato 0e3bdc4f300bf70de87781e6589fcd3990d74730 m
PIPE-3232 update log_qa decorator
- to support list of QAScore objects
- to log all QA messages rather than the first score for list/tuple inputs -
Takeshi Nakazato dad6c44787bd6a032985139f2f2c9b53e19f7751 m
PIPE-3232 score_amp_vs_time_plots should skip QA evaluation if weblog is disabled
-
Takeshi Nakazato bd44fd9fbd52306772d7bcea2f67eee7a0fcd281 m
PIPE-3232 add/remove log_qa decorator depending on the return value of the function
-
Rui Xue e6c2b25be9959541f2f46467b4d593d6ad8d754d m
PIPE-3222: Deduplicate edgechans flag commands in h_tsysflag
Edge channel detection is intent-independent. Compute one flagging
view per spw using the union of field IDs across all intents,
rather than duplicating views per (spw, intent) pair. -
Rui Xue 9e39bc66a2b65bcf8d736e666e89c986ae73cc0d m
PIPE-3221: Make applycal field ordering deterministic
Replace set-based deduplication with utils.deduplicate() to preserve field order in applycal CASA commands, ensuring reproducible task logs across runs. -
Rui Xue 5f3e75c0f865e27e0af42211d5efbb934bb89743 m
PIPE-3221: Fix non-deterministic tsys spw iteration in tsysflag
-
Ajay Vibhute f8b81355cc7eef5f7d85a7a6ce10793619445487 m
PIPE-3067: updated VLASSPT, VLASSWP, VLASSPL, VLASSRJ
-
Ajay Vibhute 9b75a37c75e4efc23c2b64b2fb9d5bd76b02ee90 m
PIPE-3067: updated docstrings, added requirements and dry-run option
-
Ajay Vibhute 6e07264808a7a234be1e7bf44dcfe61eed56818c m
PIPE-3067: scripts to populate fits header keywords using information available in product directory