Code commits
Pipeline
-
Rui Xue 357235e22099d79734b78442a6f6a47875775004
Merge remote-tracking branch 'origin/main' into PIPE-2601-for-alma-data-make-hif_lowgainflag-use-hifa_bandpass-almaphcorbandpass-bandpass-code
-
Ajay Vibhute 747bb8a04857939de9920a68106747c212a783c8 m
Pull request #1664: PIPE-2582: updated QA criteria for semiFinalBPdcals
Merge in PIPE/pipeline from PIPE-2582-hifv_semifinalbpdcals-qa-criteria to main
* commit '9b09ac10aa3c105cbc95aee66acc0e6dfab10eaf':
PIPE-2582: Changes suggested in PR review
PIPE-2582: present warning if spw is fully flagged, or mulitple antennas are fully flagged
PIPE-2582: Skipping the spectral line SPW fully flagged warning if the offending antenna is fully flagged
PIPE-2582: passing set(vis) to applies_to
PIPE-2582: added flaggedSolnApplycaldelay dictionary is structured, and other minior changes suggested during the PR review
PIPE-2582: minor changes suggested during PR review
PIPE-2582: minor bugfix
PIPE-2582: updated QA messages
PIPE-2582: maxout delay to 0.3
PIPE-2582: removed join from warning
PIPE-2582: added applies_to to QAScore
PIPE-2582: showing warning for fully flagged spw if it is spectral line spw
PIPE-2582: updated QA criteria -
Rui Xue 058d8aeea34d9aecc6b4a6a172339e1e4dea7d4a m
Pull request #1810: PIPE-3085: reduce MS open calls
Merge in PIPE/pipeline from PIPE-3085-improve-the-freq-frame--chan-conversion-speed-within-fluxcalflag to main
* commit 'ad315c171f1d6b940df0fe50f4c392b23effb04e':
PIPE-3085: Replace singleton ms tool with MSReader; add type hints
PIPE-3085: Extend MS open optimization to `correctedampflag`, `rawflagchans`, `gcorfluxscale`
PIPE-3085: Reduce MS open overhead in `hif_applycal` weblog rendering
PIPE-3085: hifa_fluxcalflag - cache SPW freq grids and use searchsorted in _get_chanrange -
Rui Xue ad315c171f1d6b940df0fe50f4c392b23effb04e m
PIPE-3085: Replace singleton ms tool with MSReader; add type hints
-
Ajay Vibhute 9b09ac10aa3c105cbc95aee66acc0e6dfab10eaf m
PIPE-2582: Changes suggested in PR review
-
Luke Maud ef7a76e314f8eb8de28402720b4c4c5e1bdf82db m
Pull request #1775: PIPE-3016 polarization calibration recipes second hifa bandpass after hifa lockrefant should not
Merge in PIPE/pipeline from PIPE-3016-polarization-calibration-recipes-second-hifa_bandpass-after-hifa_lockrefant-should-not to main
* commit '73c2dd1ef7a52b32229634b19b59610cd75013d2':
PIPE-3016 edit to move table unregister to within merge to context at end of stage run
PIPE-3016: minor codestyle changes.
PIPE-3016 typos and better commentary
PIPE-3016 add parameter to unregister the spwphaseup offset table -
Luca Di Mascolo 0e80eb93ef9ccb197fc019ce786fdd6d4558dfaf m
Pull request #1820: PIPE-2209: replace call to deprecated AntennaArray.baselines
Merge in PIPE/pipeline from PIPE-2209-eliminate-antennaarray.baselines-deprecation-warning to main
* commit '67aeda9c8e55a6aed96d04b61dc934e82d2960ed':
PIPE-2209: remove deprecated `baselines`/`_baselines` in AntennaArray
PIPE-2209: remove debugging leftover
PIPE-2209: replace call to deprecated AntennaArray.baselines -
Rui Xue 143268e351aabe649c573e0ba88ad8ba3073b691 m
Pull request #1818: PIPE-3089: Fix caltable filename inconsistency between serial and parallel modes
Merge in PIPE/pipeline from PIPE-3089-caltable-filenames-might-differ-between-serial-and-parallel-modes-for-some-alma-cal-tasks to main
* commit '5a560095bf4b576af51b2d4a67e4fa35279b7235':
PIPE-3089: set NamingTemplate.task on MPI/Dask workers -
Takeshi Nakazato 7b4dd9b34aff97f0f8ef8f42d93e1d88e9e8ed71 m
Pull request #1802: PIPE-2967 hsd k2jycal backup url for jy k db query
Merge in PIPE/pipeline from PIPE-2967-hsd_k2jycal-backup-url-for-jy-k-db-query to main
* commit 'aedad0e415d5cf24d1920ae51eae2aabaad2c2b8':
revert max casa version
PIPE-2967 removed unnecessary rstrip operation
PIPE-2967 safely remove trailing newline
PIPE-2967 replace "hosts" in the variable names with "urls"
reverse the order of log records to be displayed with original order
improvement of the analysis of warning messages
Set default backup hosts
CASA version requirement for getjyperkalma task
remove unnecessary filter
take care of the case of dbservice=False
Define backup_hosts as vis-dependent property so that it is listed in the weblog
Emit warning message per endpoint URL (per EB)
refactoring
Migrate to getjyperkalma to access Jy/K DB -
Hajime Ezawa 69c74d46d6e3ccce79f99b6281fc39816830ffda m
Pull request #1797: PIPE-2958 hsd imaging implement qa score based on the ratio of observed image rms to the theoretical
Merge in PIPE/pipeline from PIPE-2958-hsd_imaging-implement-qa-score-based-on-the-ratio-of-observed-image-rms-to-the-theoretical to main
* commit 'a0a1a262c66310e2ba8aa7111c0e10841400bb14':
PIPE-2958: improved the code for the thresholds of the relevant qascore
PIPE-2958: fix incomplete f-strings, remove unnecessary str() conversion
PIPE-2958: further correction
PIPE-2958: correct an accedental touch
PIPE-2958: align metric_name of missed-lines QA
PIPE-2958: allign wordings to Theoretical/Observed sensitivity in pubished materials
PIPE-2958: modified wordings in hsd_imaging weblog and QA messages
PIPE-2958: typo correction
PIPE-2958: skip processing QA score when result.sensitivity_info is None
PIPE-2958: minor docstring corrections
PIPE-2958: Implement a new QA score based on Observed/Theoretical sensitivities -
Shawn Booth b76d81a947b8daec4cd1740cd3ae3b395c2fb45a m
Pull request #1764: PIPE-3002: importdata weblog might not render correctly if a polcal field selected for a parallactic angle plot is present in one EB but not the other, within a single session
Merge in PIPE/pipeline from PIPE-3002-importdata-weblog-might-not-render-correctly-if-a-polcal-field-selected-for-a-parallactic to main
* commit 'b3cdb891bd8c5b1b00db9fa2bc02d97a6e7f86c4':
PIPE-3002: renamed variable inside list comprehension to prevent shadowing and confusion with the outside variable of the same name; added same safeguard in hifv/importdata/renderer as in hifa
added conditional intent filtering to plotms call; changed intent mapping to a list and removed the early mapping to CASA intents
PIPE-3002: removed unnecessary context property added to InputContainer
PIPE-3002: updated parallactic-angle intent filtering; fixed bug in vdp.InputContainer to make sure the context object is present for multi-MS runs -
Shawn Booth 16312f0381af4aede3180215779f238fc48418e6 m
Pull request #1800: PIPE-3079: Documentation: Context Use Cases and Improvements
Merge in PIPE/pipeline from context-use-cases to main
* commit 'd996090a04c869254ff5b3b5974206bbf57692c6':
PIPE-3079: consolidated implementation notes to reduce redundancy
PIPE-3079: removed sections 3 and 4 and appendix A; added clarification stating these use cases are retrospective
PIPE-3079: removed hanging header section
PIPE-3079: updated document language to be more reflective of current behavior and removed suggestions related to an unrelated future design; migrated GAP scenarios to observations and limitations, removing specific implementation suggestions
Docs: update context_use_cases improvements -
Shawn Booth d996090a04c869254ff5b3b5974206bbf57692c6 m
PIPE-3079: consolidated implementation notes to reduce redundancy
-
Shawn Booth 71d45c1fc13cf6714da96d32e71add044d0d7c93 m
PIPE-3079: removed sections 3 and 4 and appendix A; added clarification stating these use cases are retrospective
-
Takeshi Nakazato aedad0e415d5cf24d1920ae51eae2aabaad2c2b8 m
Merge branch 'main' into PIPE-2967-hsd_k2jycal-backup-url-for-jy-k-db-query
-
Hajime Ezawa a0a1a262c66310e2ba8aa7111c0e10841400bb14 m
PIPE-2958: improved the code for the thresholds of the relevant qascore
-
Takeshi Nakazato 674acc5fa300d5d875228aac3e96eba9a54934e1 m
revert max casa version
-
Rui Xue 6f549f5f134fe0ed4b4ae8043ff2a78b86ba71ce m
Pull request #1813: PIPE-3078: Remove clean mask deletion between `.iter1` and `.iter2` in continuum TARGET imaging
Merge in PIPE/pipeline from PIPE-3078-vla-imaging-pipeline-deletes-mask-between-iter1-and-iter2 to main
* commit '0d7c97da2e9984319ed164cc3493c19cf09afb57':
PIPE-3078: Remove clean mask deletion between `.iter1` and `.iter2` in continuum TARGET imaging -
Shawn Booth 313fa38dc4cef06a536c4255e605c158744432d1 m
PIPE-3079: removed hanging header section
-
Luca Di Mascolo <lucadimascolo@gmail.com> 67aeda9c8e55a6aed96d04b61dc934e82d2960ed m
PIPE-2209: remove deprecated `baselines`/`_baselines` in AntennaArray
-
Luke Maud af37277de39016b8db9af315c3021dca21597fe9 m
Pull request #1819: PIPE-1868 change wvrgcalflag score such that aca data with insufficient pm antennas for a solution
Merge in PIPE/pipeline from PIPE-1868-change-wvrgcalflag-score-such-that-aca-data-with-insufficient-pm-antennas-for-a-solution to main
* commit 'e43129fe6475b7b3ba6183b8de5cf33d50a47ef0':
PIPE-1868: Add PHgood and BPgood flags to WvrgcalResult; update score calculation in WvrgcalflagQAHandler
PIPE-1868 adding score update to blue 0.67 for good bandpass pre-WVR phase RMS
PIPE-1868 update logging
PIPE-1868 wvrgcal score to yellow inc ms name in QA -
Rui Xue e43129fe6475b7b3ba6183b8de5cf33d50a47ef0 m
PIPE-1868: Add PHgood and BPgood flags to WvrgcalResult; update score calculation in WvrgcalflagQAHandler
-
Ajay Vibhute 1752ede6e78945eb1f639517bc5396d5651cc0b9 m
Pull request #1815: PIPE-3068: clone PIPE-3040/PIPE-2461/PIPE-3091 changes to the main branch
Merge in PIPE/pipeline from PIPE-3068-clone-pipe-3040-and-pipe-2461-changes-to-the-main-branch to main
* commit 'c9ec29a7bedff884fee0f6ff2b1429b8c5ad3bac': (45 commits)
PIPE-3068: moving __future__ import to top of imaging.py
PIPE-3091: updated comments for vlass fits keywords
PIPE-3068: adding changes from PIPE-3060 and PIPE-3051
PIPE-3068: adding documentation for vlass fits header keywords
PIPE-3040: added key for alphaerror
PIPE-3040: added check on keys
PIPE-3040: updated peak and rms for .com.fits
PIPE-3040: computing peak and median for only stoke I images
PIPE-3040: removed unused variables, used file context manager to open fits file and updated get_vlass_image_type method
PIPE-3059: Fix TypeError when doing mask type checks
PIPE-2461: changes suggested in PR review
PIPE-2461: adding peak and rms from subimages
PIPE-2461: minor bugfix
PIPE-2461: moved header keyword update logic to _update_vlass_fits_header
PIPE-2461: update alpha image file not found warning in pbcor
PIPE-2461: update VLASSPL for .com.fits
PIPE-3043: Add warning for nmajor limit in `CleanBase`
PIPE-2461: showing alpha files skipping header update warning only if alpha image is expected
PIPE-2461: adding VLASSWP keyword
PIPE-2461: updated VLASSPL for VLASS cube stokes
... -
Ajay Vibhute c9ec29a7bedff884fee0f6ff2b1429b8c5ad3bac m
PIPE-3068: moving __future__ import to top of imaging.py
-
Ajay Vibhute 8550343130fe547b74e5358e29d35bb486477f6b m
PIPE-3091: updated comments for vlass fits keywords
-
Rui Xue 454b91e80c6962a09f89ae57b094da0fbc1bbc61 m
PIPE-3085: Extend MS open optimization to `correctedampflag`, `rawflagchans`, `gcorfluxscale`
Hoist casa_tools.MSReader outside the inner loops in three additional
tasks, following the same pattern established for hif_applycal weblog
rendering:
- hif_correctedampflag: open once for the intent × field × spw loop
in _run_flagging_iteration; thread ms_handle through
_evaluate_heuristic → _evaluate_heuristic_for_baseline_set →
mstools.read_channel_averaged_data_from_ms.
- hif_rawflagchans: open once for the spw loop in
RawflagchansData.prepare; call openms.reset() between iterations
instead of opening a fresh handle per SPW.
- hifa_gfluxscale: open once for the field × spw loop in
_derive_calvis_flux; pass ms_handle to mstools.compute_mean_flux.
Also: bare except: → except Exception:, lazy % log formatting.- pipeline/hif/tasks/correctedampflag/correctedampflag.py (version 454b91e80c6962a09f89ae57b094da0fbc1bbc61) (diffs)
- pipeline/hif/tasks/rawflagchans/rawflagchans.py (version 454b91e80c6962a09f89ae57b094da0fbc1bbc61) (diffs)
- pipeline/hifa/tasks/fluxscale/gcorfluxscale.py (version 454b91e80c6962a09f89ae57b094da0fbc1bbc61) (diffs)
-
Rui Xue 7673073843ca00326b758194ef083e23c7573149 m
PIPE-3085: Reduce MS open overhead in `hif_applycal` weblog rendering
Reduce the cost of opening measurement sets in get_brightest_field()
by reusing a single MS handle across all (field, spw) iterations instead
of opening and closing the MS once per combination. On lazy-import data
sets, where the DATA column is ASDM-backed, this cuts the applycal
post-processing time significantly (observed ~50 min saving on a
2496-call dataset).
- renderer.py: open MS once with casa_tools.MSReader and pass handle
down to mstools.compute_mean_flux via the new ms_handle kwarg.
- mstools.py: add keyword-only ms_handle parameter to both
read_channel_averaged_data_from_ms and compute_mean_flux; call
openms.reset() before each reuse to clear prior selection state. -
Luca Di Mascolo <lucadimascolo@gmail.com> b9348f477ae1b967972341d849ec172f718cf8e9 m
PIPE-2209: remove debugging leftover
-
Luke Maud 45f518c53da26de72c07978f7c1c9c8ae28128f2 m
PIPE-1868 adding score update to blue 0.67 for good bandpass pre-WVR phase RMS
-
Luca Di Mascolo <lucadimascolo@gmail.com> a71c2a5c4d64378e5b652014664ed2194a376785 m
PIPE-2209: replace call to deprecated AntennaArray.baselines
-
Rui Xue 5a560095bf4b576af51b2d4a67e4fa35279b7235 m
PIPE-3089: set NamingTemplate.task on MPI/Dask workers
NamingTemplate.task is set by basetask.execute() only when
is_top_level_task() is True, which is always False on MPI servers.
This caused worker-side data product filenames to lack the task name
prefix, breaking any logic relying on filename content (e.g. the
amplitude caltable lookup in hifa_gfluxscale).
In Tier0PipelineTask.get_executable(), resolve the Pipeline task name from
task_registry (with a ParallelFoo.Task fallback) and set
NamingTemplate.task before executing the task lambda. -
Ajay Vibhute bdeb273c7200de346af22d5784f3f2c3dab76c38 m
-
Shawn Booth b3cdb891bd8c5b1b00db9fa2bc02d97a6e7f86c4 m
PIPE-3002: renamed variable inside list comprehension to prevent shadowing and confusion with the outside variable of the same name; added same safeguard in hifv/importdata/renderer as in hifa
-
Ajay Vibhute 23f5af91074e9f683548dc5e4e72494494d61b8c m
PIPE-3068: resolved merge conflicts
-
Ajay Vibhute 04c27dccc0c7ab0c6f61d04a7e80c547d6168588 m
PIPE-3068: adding documentation for vlass fits header keywords
-
Rui Xue 0d7c97da2e9984319ed164cc3493c19cf09afb57 m
PIPE-3078: Remove clean mask deletion between `.iter1` and `.iter2` in continuum TARGET imaging
-
Luke Maud 94eaf9ae7e198ae39bb190060bbe4cbc9a41b7b6 m
PIPE-1868 update logging
-
Luke Maud a805a837a05a17b97c1ab7fc0e951cb053e98c9f m
PIPE-1868 wvrgcal score to yellow inc ms name in QA
-
Ajay Vibhute 459606c4ccc5b35ee3efca1182991f72dc41d584 m
PIPE-2582: present warning if spw is fully flagged, or mulitple antennas are fully flagged
-
Rui Xue 50fed352aa45cc77be74ca0dc00d3c028d594bfc m
PIPE-3085: hifa_fluxcalflag - cache SPW freq grids and use searchsorted in _get_chanrange
Open the MS once per prepare() call instead of once per line candidate,
and cache the cvelfreqs result per (vis, fieldid, spwid, refframe) via a
new _get_spw_freqs helper. Replace the O(N) linear channel scan with
np.searchsorted for O(log N) boundary lookup. Also convert %-style LOG
calls to lazy-format style and raise ValueError instead of Exception. -
Shawn Booth bc7209a88e14d8048c6427525765b4f5ca5c4331 m
PIPE-3079: updated document language to be more reflective of current behavior and removed suggestions related to an unrelated future design; migrated GAP scenarios to observations and limitations, removing specific implementation suggestions
-
Takeshi Nakazato ec08870085b982bbd3852c3c2fada3293a584d1a m
PIPE-2967 removed unnecessary rstrip operation
-
Takeshi Nakazato e4a4a28cdfb53232478226ee34e8a87094629829 m
PIPE-2967 safely remove trailing newline
-
Takeshi Nakazato 6f33a5ba730f28ba922f9382ca34c1a50544d127 m
PIPE-2967 replace "hosts" in the variable names with "urls"
-
Rui Xue 15bb0dc4b15b3cca2a4f9400a6f9c5fa959de9ae m
Pull request #1806: PIPE-2738: Update RTD setup
Merge in PIPE/pipeline from PIPE-2738-update-readthedocs-yaml to main
* commit 'f2cc52c1605ce1c02653e06d845bc3581eeaad47':
PIPE-2738: Update ReadTheDocs configuration to use Python 3.12 and Pixi for building documentation -
Ajay Vibhute 84c7162aa601647985e19da0c1ee23fbaf607fcb m
PIPE-2582:resolved conflicts
-
Rui Xue f2cc52c1605ce1c02653e06d845bc3581eeaad47 m
PIPE-2738: Update ReadTheDocs configuration to use Python 3.12 and Pixi for building documentation
Co-authored-by: Copilot <copilot@github.com> -
Luke Maud 73c2dd1ef7a52b32229634b19b59610cd75013d2 m
PIPE-3016: update merge conflic after PIPE-2903 merged back to branch and fixed lock_refant.py
-
Shawn Booth ab9d70ca7aba98a33a4ae6ae19c7c384672d4209 m
added conditional intent filtering to plotms call; changed intent mapping to a list and removed the early mapping to CASA intents
- pipeline/hifa/tasks/importdata/renderer.py (version ab9d70ca7aba98a33a4ae6ae19c7c384672d4209) (diffs)
- pipeline/hifv/tasks/importdata/renderer.py (version ab9d70ca7aba98a33a4ae6ae19c7c384672d4209) (diffs)
- pipeline/infrastructure/renderer/rendererutils.py (version ab9d70ca7aba98a33a4ae6ae19c7c384672d4209) (diffs)
-
Rui Xue 57e37988e5efa6c75a1e874b66154a64bf0fb28c m
Pull request #1805: PIPE-3077: doc update on regression test, pixi tasks
Merge in PIPE/pipeline from PIPE-3077-isolate-individual-regression-tests-from-casa-stateful-behavior to main
* commit '72a633b55efcf3af84583b240744337865ac7a6b':
PIPE-3077: Remove deprecated environment configuration files for pipeline and pipeline-vlass2026
PIPE-3077: `pyproject.toml` improve comments and move pandoc to docs env
PIPE-3077: Refactor Pixi task documentation for clarity and consistency
PIPE-3077: Update documentation and configuration for running tests with pixi and pytest -
Rui Xue 72a633b55efcf3af84583b240744337865ac7a6b m
PIPE-3077: Remove deprecated environment configuration files for pipeline and pipeline-vlass2026
-
Rui Xue 3222b0ed762be0d3baea6bcb0bece7e0235c76ec m
PIPE-3077: `pyproject.toml` improve comments and move pandoc to docs env
- Add inline rationale comments: poppler/imagemagick (Pipeline),
protobuf pin (casaplotms), openmpi-mpicxx (C++ MPI bindings),
nodefaults (not needed in pixi), casa674 indirect deps
- Move pandoc to [tool.pixi.feature.docs] — only needed for doc builds
- Add --junitxml=regression-results.xml to test-regression task
Co-authored-by: Copilot <copilot@github.com> -
Rui Xue 578d072259b387f9969b6121b5cbf12c630a3725 m
PIPE-3077: Refactor Pixi task documentation for clarity and consistency
Co-authored-by: Copilot <copilot@github.com> -
Hajime Ezawa 17fa6ef013802b70c36f33e81000cc300c489a16 m
PIPE-2958: fix incomplete f-strings, remove unnecessary str() conversion
-
Rui Xue de8bdaccb6bbf11018c8bce17185ae3a6fcccd72 m
PIPE-3077: Update documentation and configuration for running tests with pixi and pytest
Co-authored-by: Copilot <copilot@github.com> -
Takeshi Nakazato 60a4c0de699922601dcb4adecc5377495d6b9989 m
Pull request #1762: PIPE-2945 hsd_skycal should not crash if it failed to generate sky caltable for only one
Merge in PIPE/pipeline from PIPE-2945-hsd_skycal-sdpl-should-not-crash-if-it-failed-to-generate-sky-caltable-for-only-one to main
* commit '16ce874a58991b45f3955a2b0924440c2e5becae':
Skip generating elevation plot if the data is empty
Bug fix on removing extra files created by plotbandpass
Fix wrong scan number issue in amp vs freq summary plot
Skip computing elevation difference when no valid skycal solution exists
Improvement of warning message
refactoring
Remove obsolete debug log
Simply the implementation
Remove unused class
Refactoring
Fix typo
Error accurate error handling
PIPE-2945 consolidate per-fields sdcal calls into just a single call -
Takeshi Nakazato be7e43930885331dfcd1adcd6cee44316b3d9196 m
reverse the order of log records to be displayed with original order
-
Takeshi Nakazato df21a3a6f65a1c7c41121d0a9ccb70ccddde7f73 m
improvement of the analysis of warning messages
- remove line feed character from the analysis
- preserve the order of URLs in the log -
Rui Xue b49d03f06a1a212024a52fada5fa3e60699a3099 m
Pull request #1767: PIPE-2947: Implement NaN handling for `savgol_filter` and update requirements to unpin scipy version
Merge in PIPE/pipeline from PIPE-2947-update-heuristics to main
* commit '0c9b827f1b746ffb8167e59bff1ef6824776157f':
PIPE-2947: Implement NaN handling for `savgol_filter` and update requirements to unpin scipy version -
Rui Xue 0c9b827f1b746ffb8167e59bff1ef6824776157f m
Merge remote-tracking branch 'origin/main' into PIPE-2947-update-heuristics
-
Shawn Booth 5d62817fa7d0d41f848349fcf5ca3d7e2962cc02 m
Pull request #1736: PIPE-1787: Integrating Regression Test Documentation into RTD Source Code
Merge in PIPE/pipeline from integrate-docs-to-source-code to main
* commit 'e72af9e9f6dc3cdcab1f3251c0b9ae662207d915':
PIPE-1787: move table of contents to top of page
PIPE-1787: integrated regression testing instructions from confluence; updated to include information on component and unit tests -
Shawn Booth e72af9e9f6dc3cdcab1f3251c0b9ae662207d915 m
Merge remote-tracking branch 'origin/main' into integrate-docs-to-source-code
-
Shawn Booth 886479e3e1ba288a8b41f1373b6fae8bb465820c m
Docs: update context_use_cases improvements
Reframe document to present changes as improvements to the current pipeline context; remove RADPS and future/next-generation wording; add GAP-derived improvement items and improve consistency. -
Takeshi Nakazato e560fd1187fd2f5a21b6d87d8d1faa4a28902de6 m
Merge branch 'main' into PIPE-2967-hsd_k2jycal-backup-url-for-jy-k-db-query
-
Hajime Ezawa c91b1c98463d0291d1860e942f853be7cdcaab8c m
Merge branch 'main' of https://open-bitbucket.nrao.edu/scm/pipe/pipeline into PIPE-2958-hsd_imaging-implement-qa-score-based-on-the-ratio-of-observed-image-rms-to-the-theoretical
-
Takeshi Nakazato 16ce874a58991b45f3955a2b0924440c2e5becae m
Merge branch 'main' into PIPE-2945-hsd_skycal-sdpl-should-not-crash-if-it-failed-to-generate-sky-caltable-for-only-one
-
Takeshi Nakazato 19019e59b38674e3f5a2dcef53abe374862910b4 m
Pull request #1794: PIPE-3001 hsd baseline proper use of virtual real spw id in baseline worker.py
Merge in PIPE/pipeline from PIPE-3001-hsd_baseline-proper-use-of-virtual-real-spw-id-in-baseline-worker.py to main
* commit '72b7331ba1f531a7577dcc2efdfaaeae0dd9dfd7':
harmless change to trigger Bamboo build
Replace string format method with f-string
Update spw-related variable names to indicate they store real spw id
Replace "EB" with more generic word, "MS"
rename spw-related variables to distinguish virtual or real
Fix inconsistent order of the args
Add docstrings to some methods
Update parameter name accordingly
gather fitting configuration log per EB (per worker process)
bug fix on the use of local _wave_number
Remove extra comma
Clearly separate the use of real/virtual spw ids -
Takeshi Nakazato 20c1fde25f0a690cb78314659b1e9a8ed2020392 m
Merge branch 'main' into PIPE-2945-hsd_skycal-sdpl-should-not-crash-if-it-failed-to-generate-sky-caltable-for-only-one
-
Takeshi Nakazato 72b7331ba1f531a7577dcc2efdfaaeae0dd9dfd7 m
harmless change to trigger Bamboo build
-
Takeshi Nakazato 78659804c692531c049fdd943848342283554f73 m
Merge branch 'main' into PIPE-3001-hsd_baseline-proper-use-of-virtual-real-spw-id-in-baseline-worker.py
Co-authored-by: Copilot <copilot@github.com> -
Ryan Loomis d94fb1cb5caf9e7a9169818bb224b1445ad8ba02 m
Pull request #1787: PIPE-2617: make hif_findcont use uvrange
Merge in PIPE/pipeline from PIPE-2617-make-hif_findcont-use-uvrange to main
* commit '803f8d791bb8a562a28f97f032ba7939e678f16d':
PIPE-2617: make hif_findcont use uvrange -
Hajime Ezawa b73ab612106ce776cddef8ee50bfa5ac2eb9c984 m
PIPE-2958: further correction
-
Hajime Ezawa c2a221f3026b5a3195aea3121cd7818f9424ae76 m
PIPE-2958: correct an accedental touch
-
Hajime Ezawa c7d09ad1fceb2a7a2949c1a7fa8f89d5c3d57bda m
PIPE-2958: resolve the conflics with main
-
Shawn Booth a1e5544161d269358a369c16ed132a32904a2706 m
Pull request #1724: PIPE-2903 & PIPE-2939: Python Code Modernization, Import Optimization, and Cleanup
Merge in PIPE/pipeline from PIPE-2903-additional-refactoring-items-after-pr-1452 to main
* commit '83fe0c033e0abcb3a7237aa8768681ca58a63e1a':
PIPE-2903: reverted super reference calls to the parent object; changed type of QuantityDict value
PIPE-2903: fix failure introduced at merge
PIPE-2903: removed references to custom dictionaries in the all block that got reintroduced on merge with main
PIPE-2903: fixed docstring reference to the correct Jira ticket
PIPE-2903: removed defunct _cluster_details function replaced by deferred request
Replace naive datetimes with timezone-aware UTC datetimes throughout codebase
PIPE-2903: fixed timezone naive issues by adding UTC timezone
PIPE-2903: gathered all custom CASA type hints into single module casa_types
PIPE-2903: fixes to unit test failures associated with datetime changes from naive to UTC aware
PIPE-2903: converted callable type hint to Callable; removed commented out code and duplicated type hints from docstrings; updated numpy type hints
PIPE-2939: removed defunct pipeline/infrastructure/docutils module
PIPE-2903: updated type hints to use type MaskedArray instead of constructor; fixed inaccurate type hints in hsd/baselineflag; updated several NewType custom variables to use TypedDict; fixed several indentation levels; moved type-hint-only imports into TYPE_CHECKING; replaced Integral with Real for more accurate type hint
PIPE-2903: reverted changes to extern module; reverted several namespace imports that were incorrectly removed; reverted incorrectly moved imports to TYPE_CHECKING
PIPE-2903: moved more type-hint-only imports to TYPE_CHECKING; removed several unused imports; fixed several unbound/undefined variable errors; replaced deprecated imports with functionally-equivalent imports; fixed invalid type forms; added import of collections.abc library
PIPE-2903: moved more type-hint-only imports into TYPE_CHECKING; replaced typing module shims with direct module imports
PIPE-2903: removed deprecated datetime.utcnow function; migrated generic type hints to appropriate modules per PEP585; switched abstract base class usage per PEP 3119
PIPE-2903: moved many type-hint-only imports into TYPE_CHECKING; added future annotations and removed many string literals from type hints; updated many old style docstrings; removed object name from class definitions and super calls
PIPE-2903: updated type hinting standards per PEP484, PEP585, and PEP604, including replacing imports for built-ins and removing string literals and adding annotation forwarding -
Takeshi Nakazato fe9504fef685dc0b2798004a954353d43be3b20a m
Merge branch 'main' into PIPE-2967-hsd_k2jycal-backup-url-for-jy-k-db-query
-
Takeshi Nakazato 2a9841b35fa120ce5bba1de46904b5930e8b9ad9 m
Set default backup hosts
-
Hajime Ezawa 0d49685b8acc028c52b75f01eed7eebe4dca52c4 m
Merge branch 'main' into PIPE-2958-hsd_imaging-implement-qa-score-based-on-the-ratio-of-observed-image-rms-to-the-theoretical
-
Hajime Ezawa 44b079885db23b18acc8960295ef035399841925 m
PIPE-2958: align metric_name of missed-lines QA
-
Hajime Ezawa 7ba3a74fb6350031c5d362375000bca14e3c409d m
PIPE-2958: allign wordings to Theoretical/Observed sensitivity in pubished materials
- pipeline/hsd/tasks/imaging/imaging.py (version 7ba3a74fb6350031c5d362375000bca14e3c409d) (diffs)
- pipeline/hsd/tasks/imaging/imaging_params.py (version 7ba3a74fb6350031c5d362375000bca14e3c409d) (diffs)
- pipeline/hsd/tasks/imaging/qa.py (version 7ba3a74fb6350031c5d362375000bca14e3c409d) (diffs)
- pipeline/hsd/templates/hsd_imaging.mako (version 7ba3a74fb6350031c5d362375000bca14e3c409d) (diffs)
- pipeline/qa/scorecalculator.py (version 7ba3a74fb6350031c5d362375000bca14e3c409d) (diffs)
-
Rui Xue 1fa18ea485c9483ae9961384aedc3e0d4ce45e92 m
Pull request #1790: PIPE-3061/PIPE-3027
Merge in PIPE/pipeline from PIPE-3061-examining-the-possible-memory-leak-when-running-multiple-regression-tests-on-one-pytest to main
* commit 'ebb6ced6fd1f23d977a302abef49c5d461feca85':
PIPE-3061: Update `pixi_tasks.md`
PIPE-3061: drop Python 3.10/3.11, add CASA 6.7.4 env, update docs theme
PIPE-3061: add pixi tasks for regression/unit tests and developer docs
PIPE-3061: Add `pytest-xvfb` to development requirements for headless testing support without `xvfb-run`
PIPE-3061: Update documentation to clarify xdist worker scaling and per-test timing details
PIPE-3061: Add MPI worker scaling recommendations and per-test timing details to documentation
PIPE-3061: Add usage examples for `@pytest.mark.mpi` in documentation
PIPE-3061: Further improve memory management by triggering garbage collection after clearing caches and releasing context in `PipelineTester`
PIPE-3027: Simplify MPI marker handling in pytest configuration and update description for clarity
PIPE-3027: add `@pytest.mark.mpi` marker and `--ignore-mpi-markers` option
PIPE-2061: Add fixture to clear module-level caches between tests to reduce memory footprint
PIPE-3061: Implement unsubscribe functionality in eventbus and improve the memory management for `Context` in `PipelineTester` -
Shawn Booth 3ab57136e34a63bb34aa246c71478c4b38ae08f9 m
Merge remote-tracking branch 'origin/main' into PIPE-3002-importdata-weblog-might-not-render-correctly-if-a-polcal-field-selected-for-a-parallactic
-
Takeshi Nakazato 665baeaa8567472f7806e1aa22f0ff2d2cb5c680 m
CASA version requirement for getjyperkalma task
-
Takeshi Nakazato 9b6f98a19b6ed94b5194f1179604ed21ea3afb3f m
Merge branch 'main' into PIPE-3001-hsd_baseline-proper-use-of-virtual-real-spw-id-in-baseline-worker.py
-
Takeshi Nakazato 4ea0c6786241662b50f1467baab613b34249a4c4 m
Replace string format method with f-string
-
Rui Xue ebb6ced6fd1f23d977a302abef49c5d461feca85 m
PIPE-3061: Update `pixi_tasks.md`
-
Rui Xue d42126852c0284c766a041f46c36e47382b81001 m
PIPE-3061: drop Python 3.10/3.11, add CASA 6.7.4 env, update docs theme
- Bump requires-python to >=3.12; remove 3.10/3.11 classifiers
- Drop casa666-py310 and casa671-py310 pixi environments
- Add casa674-py312 feature/environment (CASA 6.7.4, Python 3.12)
- Enable casa675-py313 on osx-arm64; guard bdsf via sys_platform
marker in requirements.txt instead of platform list restriction
- Bump macos system-requirement to 15.0
- Simplify astropy pin: drop python_version conditional (3.10 gone)
- Add build-docs-fast (incremental) and build-pdf pixi tasks
- Add furo Sphinx theme to requirements_docs.txt and docs environment -
Rui Xue 87558d0b47039b0f84760dfe0c9eae09d184db12 m
PIPE-3061: add pixi tasks for regression/unit tests and developer docs
Add two pixi tasks to pyproject.toml:
- test-regression: runs tests/regression/fast/ with pytest-xdist
(worksteal, -m "not mpi") inside a plain Python session
- test-pltest1: runs a single small ALMA-IF regression test via `casashell`
Both tasks cd into PL_WORKDIR (default: ../working) before launching,
keeping CASA logs and pipeline output out of the source tree.
PL_WORKDIR is overridable at invocation time:
PL_WORKDIR=/zfs/scratch/myrun pixi run test-regression
Also add docs/source/develdocmd/pixi_tasks.md documenting all five
pixi tasks (test-unit, test-regression, test-pltest1,
fetch-casarundata, build-docs), the available CASA environments,
PL_WORKDIR usage, and per-environment invocation examples. -
Rui Xue 7a18fb44737e7411b666b4d547fec735bb9e78f6 m
PIPE-3061: Add `pytest-xvfb` to development requirements for headless testing support without `xvfb-run`
-
Rui Xue 1a2b9adcaa74775f5ed57162127b9132773f2488 m
PIPE-3061: Update documentation to clarify xdist worker scaling and per-test timing details
-
Rui Xue 2b588848a18322c774aaa1984949cd5c8105be68 m
PIPE-3061: Add MPI worker scaling recommendations and per-test timing details to documentation
-
Takeshi Nakazato 7dc7aaff480337556994edd8ab262731bd661c6a m
Replace "EB" with more generic word, "MS"
-
Takeshi Nakazato 205fec595f43e475ddc7936b24ba264a37d177aa m
Update spw-related variable names to indicate they store real spw id
-
Takeshi Nakazato f82c92fe5d2109a066c88967333c9607f0c95336 m
rename spw-related variables to distinguish virtual or real
-
Takeshi Nakazato dd6b61aebe504d8ffb1da0dff71191162b693197 m
Fix inconsistent order of the args
-
Takeshi Nakazato b9c676fa2d8c0223de535bb0c619f743f5bce723 m
Add docstrings to some methods
-
Shawn Booth 83fe0c033e0abcb3a7237aa8768681ca58a63e1a m
PIPE-2903: reverted super reference calls to the parent object; changed type of QuantityDict value
-
Ajay Vibhute 066a71cca960d6dab310199d95f057b0d7c8d10c m
Pull request #1796: PIPE-3070: changing doquantiles to boolean from string
Merge in PIPE/pipeline from PIPE-3070-change-the-value-of-doquantiles-argument-value-from-string-to-boolean to main
* commit '3225e28bc9fb8dd7374a28f163c05ba463fbe6ba':
PIPE-3070: changing doquantiles to boolean from string -
Rui Xue a03d8c4a4d73e15e284dcdf7c14ab036edd35d73 m
PIPE-3061: Add usage examples for `@pytest.mark.mpi` in documentation
-
Rui Xue 1fdf943c22c982a2fafa8be9827c6ac93cfbd1e0 m
Pull request #1789: PIPE-3060: Refactor masking logic in syspower to ensure proper boolean array handling
Merge in PIPE/pipeline from PIPE-3060-clone-typeerror-in-hifv_syspower to main
* commit '48578da916cdd083f98e572695bf6ed92de9d347':
PIPE-3060: Refactor masking logic in syspower to ensure proper boolean array handling