Build: #4 was successful Manual run by CASA adm account
Code commits
Pipeline
-
Rui Xue 70fef63292f843a271b70ff16aeaa857b9b00541
Merge remote-tracking branch 'origin/main' into PIPE-2952-hif_uvcontsub-runs-for-an-extremely-long-time-during-restore
-
Luke Maud 5cd6eaf410c6ed506ae8b0ba094de349868b61e7 m
Pull request #1784: PIPE-2858 correct cases where interpolation should use linearpd
Merge in PIPE/pipeline from PIPE-2858-correct-cases-where-interpolation-should-use-linearpd to main
* commit '2e3e82f9f64f7560373a1066f1265f9e750d0b23':
PIPE-2858 modify calapp for b2b offset in hifa_diffgaincal to use lienarPD when spws are combined
PIPE-2858 add linearPD to almaphcorbandpass apply for combine=spw -
Rui Xue 0d3105a4e1fbd578d9c878171bd95229d369d955 m
Pull request #1834: PIPE-3094 new mstransform based hanning smoothing breaks backward compatibility of vla restores
Merge in PIPE/pipeline from PIPE-3094-new-mstransform-based-hanning-smoothing-breaks-backward-compatibility-of-vla-restores to main
* commit '1353b188830624daeab9438c0396c7181c27aad9':
PIPE-3094: Fix hanning_test and remove duplicate test with broken mock setup
PIPE-3094: Minior codestyle updates
PIPE-3094: Replace field NAME with PHASE_DIR matching
PIPE-3094: Bump the minimum pytest version and configure tmp_path retention in `pyproject.toml`
PIPE-3094: fix VLA restore backward compat for mstools->mstransform Hanning switching
PIPE-3094: Add the `transfer_flagversion` utility function for post-mstransform flag version remapping -
Rui Xue 1353b188830624daeab9438c0396c7181c27aad9 m
PIPE-3094: Fix hanning_test and remove duplicate test with broken mock setup
Two tests (test_file_operation_failure_cleanup,
test_recovery_when_original_ms_removed) were near-identical and both
broken: MagicMock(spec=HanningInputs).keep_original returns a truthy
MagicMock, so the shutil.rmtree branch was never taken and the OSError
never raised, making task_successful always True.
Fix: merge into one test, set keep_original=False explicitly, replace
fragile ordered side_effect list with return_value=True. -
Rui Xue 812d0ab5a63ea8d0dc4fe1d163d3306714b3a06f m
PIPE-3094: Minior codestyle updates
-
Rui Xue 79db364d89faf811d35982cd6b4fd6a5c5e0fe86 m
PIPE-3094: Replace field NAME with PHASE_DIR matching
Use phase centre for field ID remapping to handles duplicate names. -
Rui Xue 6454b16ba3ed77cc45a3579c6384a2fdbd9ed46d m
PIPE-3094: Bump the minimum pytest version and configure tmp_path retention in `pyproject.toml`
-
Rui Xue e9057547575e1ffea5fc56ec9b643338fa1f2178 m
Pull request #1844: PIPE-3122: Reorder `hif_checkproductsize` stage in `procedure_hifv_calimage_cont_cube.xml`
Merge in PIPE/pipeline from PIPE-3122-wrong-order-of-the-checkproductsize-stages-in-procedure_hifv_calimage_cont_cube.xml to main
* commit 'c9bb23b77d37d0468a8889c5174bfa12512733d7':
PIPE-3122: Reorder `hif_checkproductsize` stage in `procedure_hifv_calimage_cont_cube.xml` -
Hajime Ezawa f6217dd637ac3b89ca49b7963c0740019dd712ac m
Pull request #1833: PIPE-3104 hsd imaging aggregate qa message from the same metric with the same score
Merge in PIPE/pipeline from PIPE-3104-hsd_imaging-aggregate-qa-message-from-the-same-metric-with-the-same-score to main
* commit '7bb7f1e37c4c35f941f2e200d0629f050a2fb5e6':
PIPE-3104: revert the longmsg-s to orignal
PIPE-3104: introduced groups of QAScores to improve the code
PIPE-3104: fix typos and careless errors
PIPE-3104: correct minor typo found by chance
PIPE-3104: accommodate NRO cases with both I, XXYY images
PIPE-3104: further modification to QA messages of SingleDishImageMaskedPixels
PIPE-3104: rephrase QA messages for ''SingleDishImageMaskedPixels' and add TargetDataSelection for aggregation
PIPE-3104: include 'score_rasterscan_correctness'
PIPE-3104: skip if non of the keys of 'keys_to_aggregate' exist in target_qascore
PIPE-3104: aggregate and format QA messages in hsd_imaging
PIPE-3104: inhibit formatting if everything associated with longmsg_keys are empty
PIPE-3104: show field-name instead of field-id to QA messages -
Rui Xue c9bb23b77d37d0468a8889c5174bfa12512733d7 m
PIPE-3122: Reorder `hif_checkproductsize` stage in `procedure_hifv_calimage_cont_cube.xml`
-
Hajime Ezawa 7bb7f1e37c4c35f941f2e200d0629f050a2fb5e6 m
Merge branch 'main' of https://open-bitbucket.nrao.edu/scm/pipe/pipeline into PIPE-3104-hsd_imaging-aggregate-qa-message-from-the-same-metric-with-the-same-score
-
Cristian Marocico cc8888b38f4f5c2a06605d8f17d0dd268353bbf8 m
Pull request #1817: PIPE-2956 add examineCrossPolSum boolean parameter
Merge in PIPE/pipeline from PIPE-2956-add-examineCrossPolSum-boolean-parameter to main
* commit '385dfd47b48a60eeab787055a5aaddcbc90c07b8':
Re-reverted changes to .readthedocs.yaml
Refactored some tests
Reverted changes to .readthedocs.yaml
PIPE-2956: add focused tests for cross-pol opt-in behavior
PIPE-2956: add examineCrossPolSum cross-pol flagging control -
Luke Maud 68b73a438d0491c28627dabe9f820cd1d1d611cc m
Pull request #1750: PIPE-2601 for alma data make hif lowgainflag use hifa bandpass almaphcorbandpass bandpass code
Merge in PIPE/pipeline from PIPE-2601-for-alma-data-make-hif_lowgainflag-use-hifa_bandpass-almaphcorbandpass-bandpass-code to main
* commit '0120946c52d52dd9fa40e464b154c73e44185cac':
PIPE-2601 minor format change in _get_prebandpass_params
PIPE-2601: Fix UnboundLocalError and style issues in lowgainflag
PIPE-2601: Use the domain object to identify the array name and avoid holding `MSMDReader` open while executing long CASA tasks
PIPE-2601: refactor the code to have methods for the bandpass, phaseup and ampcal
PIPE-2601 modification of the bandpass worker task to use SerialALMAPhcorBandpass and to propagate solint and combine into gaincal phaseup -
Takeshi Nakazato dda05e52fee65eae040dbe14e615799b70141b12 m
Pull request #1829: PIPE-3092 hsd imaging ignore edge channels and channels overlapped with atm lines while examining qa
Merge in PIPE/pipeline from PIPE-3092-hsd_imaging-ignore-edge-channels-and-channels-overlapped-with-atm-lines-while-examining-qa to main
* commit '8feba8c2c72a7838a2f2b661d86164b48bc98141':
add supplementary comment
PIPE-3092 define dedicated attributes instead of using results object
PIPE-3092 replace convert_range_list_to_ranges with equivalent manipulation using numpy
PIPE-3092 rename "edge_channels" as "extra_edge_channels"
PIPE-3092 documentation update
PIPE-3092 bug fix on trimming edge channels
PIPE-3092 exclude edge channels and ATM lines from line-free regions in missed line analysis -
Rui Xue 2213c13d4947ae79e03131c60010b9237f855dc4 m
Pull request #1839: PIPE-3115: Fix TsysData regex to support non-uid measurement set names
Merge in PIPE/pipeline from PIPE-3115-hifa_tsysflagcontamination-crashes-if-the-measurement-doesn-t-begin-with-uid to main
* commit '1d3b06ecff87c7b5667a5ddd3a3514bf0cd7d5e9':
PIPE-3115: Fix TsysData regex to support non-uid measurement set names -
Rui Xue 1d3b06ecff87c7b5667a5ddd3a3514bf0cd7d5e9 m
PIPE-3115: Fix TsysData regex to support non-uid measurement set names
-
Ajay Vibhute 3a9cb9c9ee040c44269dc3dfe2c660ef3cf5f22f m
Pull request #1694: PIPE-2512: Enable adaptive bandpass spectral solint for the VLA pipeline
Merge in PIPE/pipeline from PIPE-2512-enable-adaptive-bandpass-spectral-solint-for-the-vla-pipeline to main
* commit 'd6935f7a7f85827be7b35fdf765d9637f342708a':
PIPE-2512: renamed score3 to score4
PIPE-2512: update spw_solint initialization in result class
PIPE-2512: Changes suggested in PR review
PIPE-2512: adding QA score for solint per spw
PIPE-2512: update solint value
PIPE-2512: minor changes suggested in PR review
PIPE-2512: minor bugfix
PIPE-2512: minor changes suggested during PR review
PIPE-2512: using flags while computing the median SNR
PIPE-2512: minor changes suggested in PR review
PIPE-2512: code cleanup
PIPE-2512: backed up caltable with timestamp
PIPE-2512: re-run the low-SNR SPWs individually with smoothing -
Takeshi Nakazato 8feba8c2c72a7838a2f2b661d86164b48bc98141 m
Merge branch 'main' into PIPE-3092-hsd_imaging-ignore-edge-channels-and-channels-overlapped-with-atm-lines-while-examining-qa
-
Rui Xue 06b38a34285e2b7e3034285bab927aa85ac2015e m
Pull request #1830: PIPE-3099: Fix MPI race condition in flux.csv concurrent access
Merge in PIPE/pipeline from PIPE-3099-in-rare-condition-open_with_lock-might-release-lock-before-write-actually-flush-to-storage to main
* commit 'f6fa562cca9f96ad98d7e26ac407cdf48eb7eba5':
PIPE-3099: Fix MPI race condition in flux.csv concurrent access -
Hajime Ezawa bb057dfe28d03dcdc6c79737ff31c8bc057dd385 m
PIPE-3104: revert the longmsg-s to orignal
-
Takeshi Nakazato 50a50c8c90687678f5a4b1397f602cd636ab872c m
add supplementary comment
-
Ilsang Yoon c510e2e4f64fde66058cd33bf807f6d536420d11 m
Pull request #1828: PIPE-3037: Changed upper and lower threshold parameter for score
Merge in PIPE/pipeline from PIPE-3037-change-newly-flagged-qa-score-for-hif_applycal to main
* commit '9aa8f21df25a7564190c77409285e2ee6c8b9efa':
PIPE-3037: Changed upper and lower threshold parameter for score -
Ilsang Yoon 7080a548b83c58227dc8259cd9baf54326b86804 m
Pull request #1827: PIPE-3036: Changed upper and lower parameter for score
Merge in PIPE/pipeline from PIPE-3036-change-qa-score-for-flagged-in-hifa_flagdata to main
* commit '4fac328b35448605917831e1733529831f1f1504':
PIPE-3036: Changed upper and lower parameter for score -
Takeshi Nakazato 5990f5505187812a38f0d5417f4adb38f2c6bde8 m
PIPE-3092 define dedicated attributes instead of using results object
-
Takeshi Nakazato 6165d05cab0feb068f9aa9feac447b67754e524e m
PIPE-3092 replace convert_range_list_to_ranges with equivalent manipulation using numpy
-
Hajime Ezawa 882948299a8c14386dff6dcb49984fdc0125fa5a m
PIPE-3104: introduced groups of QAScores to improve the code
-
Hajime Ezawa b8695b2123d1523daeeb356e5b896b577f3766d6 m
PIPE-3104: fix typos and careless errors
-
Hajime Ezawa b86144db2438157591387d847d425f73119cd365 m
PIPE-3104: correct minor typo found by chance
-
Hajime Ezawa 55975d3aeb4ab9e5ade10d93958297fa873ed798 m
PIPE-3104: accommodate NRO cases with both I, XXYY images
-
Hajime Ezawa ff233ce0559eb171897a183c4183b625de3984c5 m
PIPE-3104: further modification to QA messages of SingleDishImageMaskedPixels
-
Hajime Ezawa f50f876d2a25bb62475159eebb5237704464ca43 m
PIPE-3104: rephrase QA messages for ''SingleDishImageMaskedPixels' and add TargetDataSelection for aggregation
-
Hajime Ezawa e01f53fc1278cbf6a4754eee145795fcf2d849f4 m
PIPE-3104: include 'score_rasterscan_correctness'
-
Hajime Ezawa b6ffba94b8be85dad3885a968e4a936293a5bf6a m
PIPE-3104: skip if non of the keys of 'keys_to_aggregate' exist in target_qascore
-
Rui Xue 8bc8738723a1cd03d60829faa6b0e458de0a8918 m
PIPE-3094: fix VLA restore backward compat for mstools->mstransform Hanning switching
Add keep_original parameter to hifv_hanning so the original MS is
preserved (as <vis>.original) when needed by the restore path.
In VLARestoreData.prepare(), detect MSes that were originally
Hanning-smoothed via the deprecated mstools (partial SPW subset,
pipeline ver < 2026.1.1.3); re-smooth them using the Hanning task
(mstransform-based) instead. In _do_restore_flags(), remap the
extracted flagversions from the original row order to the new
mstransform row order before restoring with flagmanager, then clean
up the .original MS and flagversions. -
Rui Xue eda5c66528cf0b0959bf8cc6a79f864b6730d25c m
PIPE-3094: Add the `transfer_flagversion` utility function for post-mstransform flag version remapping
Remaps every flags.* table in source_flagversions so it can be restored
onto a transformed MS (Hanning smoothing, spw/field selection, etc.).
Row matching uses structured-array sort + searchsorted on (DDID, FIELD_ID,
ANTENNA1, ANTENNA2, TIME). remap_ids=True additionally translates
post-mstransform DDIDs and field IDs back to source numbering via
REF_FREQUENCY and field NAME matching. Permutation is applied via
selectrows(perm).copy(), which preserves per-row FLAG shapes and avoids
TiledShapeStMan shape conflicts. Identity permutations use copytree.
Three tests covering identity, row-reorder (VLA Hanning), and
spw+field-selection (remap_ids=True) cases.- pipeline/infrastructure/utils/__init__.py (version eda5c66528cf0b0959bf8cc6a79f864b6730d25c) (diffs)
- pipeline/infrastructure/utils/flagversion_tools.py (version eda5c66528cf0b0959bf8cc6a79f864b6730d25c) (diffs)
- pipeline/infrastructure/utils/flagversion_tools_test.py (version eda5c66528cf0b0959bf8cc6a79f864b6730d25c) (diffs)
-
Hajime Ezawa 70df280fd1b8c720874059d88077e3235ce1fc53 m
PIPE-3104: aggregate and format QA messages in hsd_imaging
-
Hajime Ezawa 190961428d5832acc525cf816861c3c37870c08c m
PIPE-3104: inhibit formatting if everything associated with longmsg_keys are empty
-
Takeshi Nakazato 7b6ed11fe0d40a1e5a88cbbd8166bccb64cf8e3d m
Pull request #1808: PIPE-2959 / PIPE-2964 hsd_baseline use 75 quantile as a representative line width of the detected cluster & take into account flagged edge channels
Merge in PIPE/pipeline from PIPE-2964-hsd_baseline-use-75-quantile-as-a-representative-line-width-of-the-detected-cluster to main
* commit '2ac046968e0a967aa527bda4a46466a7a310ab24':
PIPE-2959/PIPE-2964 refactoring
PIPE-2959/PIPE-2964 avoid hard-coded value
Inherit function attributes (docstrings, type annotations, etc.) from wrapped function
Improve type annotation
PIPE-2959/PIPE-2964 tweak comments
PIPE-2959/PIPE-2964 remove context from the arg for score_sd_line_detection
PIPE-2959/PIPE-2964 use channelmap_range for wide line/edge line/deviation mask QA evaluation
PIPE-2959/PIPE-2964 Check *actual* flag status to obtain flagged edge channels
import numpy as np
adjust the formatting of log message
improvement of log message
fix bug in resolving conflict
Updated edge channel QA to take into account edge channels flagged by edgespw heuristic in hsd_flagdata stage
additional change for clustering property plot
import numpy as np
replace representative line with 75% percentile
remove set_data method
PIPE-2964 use 75% percentile value as representative line width -
Takeshi Nakazato c13a6419bdb0d5b463020eb1099258af447d3fdc m
Pull request #1785: PIPE-3047 update hsd_imaging weblog around diagnostic plots for missed-lines
Merge in PIPE/pipeline from PIPE-3047-has_imaging-add-explanation-on-the-diagnostic-plots-for-possible-missed-line-channels to main
* commit 'bb060d44cc1015483162882f44f0a83d7056bc97':
PIPE-3047 further refinement of html text
PIPE-3047 update of weblog paragraph
PIPE-3047 update hsd_imaging weblog around diagnostic plots for missed-lines -
Hajime Ezawa c6b952a30d1ee8f2a6f57ba5623399d2387d2224 m
PIPE-3104: show field-name instead of field-id to QA messages
-
Hajime Ezawa 94dfcbc32d7ca19acfa90b0c04ca9786ec6ab1d7 m
Pull request #1803: PIPE-3054 hsd baseline aggregate qa message from the same metric with the same score
Merge in PIPE/pipeline from PIPE-3054-hsd_baseline-aggregate-qa-message-from-the-same-metric-with-the-same-score to main
* commit '7510e8781a4261a2642b2d3a177b20a302e79429':
PIPE-3054: supplement missing type annotation, docstring fix
PIPE-3054: remove an unused line
PIPE-3054: turn off preserve_original for the default behavior of QAScoreAggregator
PIPE-3054: slight change to improve the readability of QAScoreAggregator._aggregate_qascores()
PIPE-3054: improve additional style issues
PIPE-3054: improve style issues
PIPE-3054: correct type annotations
PIPE-3054: typo corrections
PIPE-3054: resolve the conflicts with main
PIPE-3054: added condition (shortmsg) to match QA scores
PIPE-3054: improve aggregation logic to accommodate custom key_list, separate class for formatter, implemented registry to consolidate aggregator/formatter properties
PIPE-3054: limit the aggregation to weblog accordion only
PIPE-3054: added condition to match the metric_unit
PIPE-3054: aggregate QA scores of hsd_baseline stage
PIPE-3054: class to aggregate single-dish QA scores -
Takeshi Nakazato bb060d44cc1015483162882f44f0a83d7056bc97 m
Merge remote-tracking branch 'origin' into PIPE-3047-has_imaging-add-explanation-on-the-diagnostic-plots-for-possible-missed-line-channels
-
Takeshi Nakazato 21ac244ddbbb1f4427d2cabff49f59e4463582a6 m
PIPE-3047 further refinement of html text
-
Rui Xue 2c4f668882b0d30aa77124d4b3da68f29a0a6210 m
Pull request #1799: PIPE-2644: Add Python, library versions, platform tag, and GPU info to weblog environment page
Merge in PIPE/pipeline from PIPE-2644-add-python-3rd-part-library-version-info-into-weblog-environment-splash-page-optionally to main
* commit '00730539a125753e2f2d9fa8fb9419176c928f85':
PIPE-2644: Hide GPU info from weblog when unavailable and optimize GPU detection
PIPE-2644: Fix code review issues in `environment.py` and `htmlrenderer.py`
PIPE-2644: Add pull request template for PR submissions
PIPE-2644: Add GPU information to the weblog environment page
PIPE-2644: Add Python/library versions and platform tag to weblog env page -
Luke Maud 0120946c52d52dd9fa40e464b154c73e44185cac m
PIPE-2601 minor format change in _get_prebandpass_params
-
Takeshi Nakazato d57ba9fe2ca3cdc06f8bfa3c368c4b691dc4b058 m
PIPE-3047 update of weblog paragraph
Use the term "off-line-range emission" instead of "missed-line". -
Takeshi Nakazato 9abe6090cf80a1d9ab1adaea4b21d941843a5e90 m
Merge branch 'main' into PIPE-3047-has_imaging-add-explanation-on-the-diagnostic-plots-for-possible-missed-line-channels
-
Takeshi Nakazato bb5ad0f55ceccf59a71c502b1bb9986752e75ab6 m
Merge branch 'main' into PIPE-3092-hsd_imaging-ignore-edge-channels-and-channels-overlapped-with-atm-lines-while-examining-qa
-
Takeshi Nakazato 7abba55d5e8d8c9eb3b43ca826c1f043be86ada5 m
PIPE-3092 rename "edge_channels" as "extra_edge_channels"
-
Takeshi Nakazato b69296d978ee1ee78319ebf8fa4e4a084562d8aa m
PIPE-3092 documentation update
-
Luca Di Mascolo 9d8c39e305acfb512165e394656644d8ca19d1c5 m
Pull request #1780: PIPE-2274: avoid post-imaging exportdata flag version overwrite
Merge in PIPE/pipeline from PIPE-2274-hifa_exportdata-at-end-of-imaging-is-attempting-to-overwrite-pipeline_final-flag-version to main
* commit '1bc3d0edf3825efa22f11bcea0a500458d391c15':
PIPE-2274: fix missing check on flag dictionary
PIPE-2274: switch to flagmanager(mode='list') to check for pipeline_final flag
PIPE-2274: apply max-*-len formatting + general linting
PIPE-2274: fix linting errors (F841)
PIPE-2274: fix linting errors (E722)
PIPE-2274: fix linting errors (E702)
PIPE-2274: fix linting errors (D1xx/D2xx/D4xx)
PIPE-2274: fix linting errors (I001)
PIPE-2274: apply linting to edited function
PIPE-2274: minor fixes
PIPE-2274: save Pipeline_Final.old.timestamp without raising warning
PIPE-2274: revert inclusion of custom flag_version_name as exportdata argument
PIPE-2274: overwrite final flag and allow for custom flag name
PIPE-2274: add flag_version_name as input parameter
PIPE-2274: directly use any() generator expression
PIPE-2274: use Pipeline_Final_PreImaging for calibration only final flag version -
Ryan Loomis 09195b4d89d69bc366a67a4e26c3cf254711efd9 m
Pull request #1823: PIPE-3090: Fix duplicated virtual SPWs in imageprecheck
Merge in PIPE/pipeline from demo-PIPE-3090-fix-duplicated-spws-for-aggBW-in-imageprecheck to main
* commit 'cc68f2b88490a5b4f2233c45fddded067b7c8a46':
PIPE-3090: Filter intents in imageparams_base SPW checks
PIPE-3090: Precompute virtual spw lookup in imageprecheck
PIPE-3090: Filter target SPWs in imageprecheck
PIPE-3090: Fix duplicated virtual SPWs in imageprecheck -
Cristian Marocico 39725b55d666a5a0a87945f15db3bdec9cf098e3 m
Pull request #1795: PIPE-117 resolution columns
Merge in PIPE/pipeline from PIPE-117-resolution-columns to main
* commit '2c5f4f25862012ba0200152738b2e093dd285771':
Added a "Spectra" meta-column
Changed c from float to int to allow multiplication with Decimal
Removed ALMA-only restrictive condition
Replaced hard-coded value of c with constant
Fixed new column placement in HTML template
PIPE-117: Add C.M, L.D.M, and P.B. to the Contributors list
Make chan_resolutions a public attribute on SpectralWindow
Implemented resolution columns for Spectral Setup Details -
Rui Xue 8d5e59df9e4b3b0ddf0ac690ed8c455bf5c4693d m
Pull request #1825: PIPE-2947: Improve `_savgol_filter_nan` to pad NaNs around masked regions
Merge in PIPE/pipeline from PIPE-2947-savgol-filter-nan-version2 to main
* commit '1364dd8064999cc14aa5c71df7c178e6dc5928a2':
PIPE-2947: Make `_savgol_filter_nan` signature compatible with `savgol_filter`
PIPE-2947: Improve _savgol_filter_nan to pad NaNs around masked regions -
Ajay Vibhute d6935f7a7f85827be7b35fdf765d9637f342708a m
PIPE-2512: renamed score3 to score4
-
Ajay Vibhute a01bd6afb2e05fd3791c69ed29b3ee5425392a5f m
Solved merge conflicts
-
Rui Xue 357235e22099d79734b78442a6f6a47875775004 m
Merge remote-tracking branch 'origin/main' into PIPE-2601-for-alma-data-make-hif_lowgainflag-use-hifa_bandpass-almaphcorbandpass-bandpass-code
-
Ajay Vibhute 6235ca0d77b3c34848e163c039000f47fadf5451 m
PIPE-2512: update spw_solint initialization in result class
- pipeline/hifv/tasks/finalcals/finalcals.py (version 6235ca0d77b3c34848e163c039000f47fadf5451) (diffs)
- pipeline/hifv/tasks/semiFinalBPdcals/semiFinalBPdcals.py (version 6235ca0d77b3c34848e163c039000f47fadf5451) (diffs)
- pipeline/hifv/tasks/testBPdcals/testBPdcals.py (version 6235ca0d77b3c34848e163c039000f47fadf5451) (diffs)
-
Ajay Vibhute 747bb8a04857939de9920a68106747c212a783c8 m
Pull request #1664: PIPE-2582: updated QA criteria for semiFinalBPdcals
Merge in PIPE/pipeline from PIPE-2582-hifv_semifinalbpdcals-qa-criteria to main
* commit '9b09ac10aa3c105cbc95aee66acc0e6dfab10eaf':
PIPE-2582: Changes suggested in PR review
PIPE-2582: present warning if spw is fully flagged, or mulitple antennas are fully flagged
PIPE-2582: Skipping the spectral line SPW fully flagged warning if the offending antenna is fully flagged
PIPE-2582: passing set(vis) to applies_to
PIPE-2582: added flaggedSolnApplycaldelay dictionary is structured, and other minior changes suggested during the PR review
PIPE-2582: minor changes suggested during PR review
PIPE-2582: minor bugfix
PIPE-2582: updated QA messages
PIPE-2582: maxout delay to 0.3
PIPE-2582: removed join from warning
PIPE-2582: added applies_to to QAScore
PIPE-2582: showing warning for fully flagged spw if it is spectral line spw
PIPE-2582: updated QA criteria -
Rui Xue 00730539a125753e2f2d9fa8fb9419176c928f85 m
PIPE-2644: Hide GPU info from weblog when unavailable and optimize GPU detection
- Conditionally exclude GPU row from environment page when no GPU detected
- Check nvidia-smi availability before running to reduce startup time -
Rui Xue f6fa562cca9f96ad98d7e26ac407cdf48eb7eba5 m
PIPE-3099: Fix MPI race condition in flux.csv concurrent access
-
Rui Xue 058d8aeea34d9aecc6b4a6a172339e1e4dea7d4a m
Pull request #1810: PIPE-3085: reduce MS open calls
Merge in PIPE/pipeline from PIPE-3085-improve-the-freq-frame--chan-conversion-speed-within-fluxcalflag to main
* commit 'ad315c171f1d6b940df0fe50f4c392b23effb04e':
PIPE-3085: Replace singleton ms tool with MSReader; add type hints
PIPE-3085: Extend MS open optimization to `correctedampflag`, `rawflagchans`, `gcorfluxscale`
PIPE-3085: Reduce MS open overhead in `hif_applycal` weblog rendering
PIPE-3085: hifa_fluxcalflag - cache SPW freq grids and use searchsorted in _get_chanrange -
Rui Xue ad315c171f1d6b940df0fe50f4c392b23effb04e m
PIPE-3085: Replace singleton ms tool with MSReader; add type hints
-
Rui Xue 24b0bcf95c91a343ed9f2e6c319c2ffd0f72149f m
PIPE-2601: Fix UnboundLocalError and style issues in lowgainflag
-
Takeshi Nakazato 6e7908640e550fbd665ecb4934ca17efccf84b3e m
Merge branch 'main' into PIPE-3092-hsd_imaging-ignore-edge-channels-and-channels-overlapped-with-atm-lines-while-examining-qa
-
ryanaloomis <ryan.a.loomis@gmail.com> cc68f2b88490a5b4f2233c45fddded067b7c8a46 m
PIPE-3090: Filter intents in imageparams_base SPW checks
-
ryanaloomis <ryan.a.loomis@gmail.com> afb4719443fdb4dc52b919aac9f6021cb6490aca m
PIPE-3090: Precompute virtual spw lookup in imageprecheck
-
Rui Xue 1364dd8064999cc14aa5c71df7c178e6dc5928a2 m
PIPE-2947: Make `_savgol_filter_nan` signature compatible with `savgol_filter`
Promote `polyorder` from **kwargs to an explicit parameter, matching
savgol_filter's positional signature. Update call site keyword order
to reflect the new signature. -
Takeshi Nakazato 2ac046968e0a967aa527bda4a46466a7a310ab24 m
Merge branch 'main' into PIPE-2964-hsd_baseline-use-75-quantile-as-a-representative-line-width-of-the-detected-cluster
-
Takeshi Nakazato c1912973d1e9f87d675ce4ce53d931f250e6f56e m
-
Takeshi Nakazato 87dd224a30866997c5968a62a5601004fa093f43 m
PIPE-3092 bug fix on trimming edge channels
-
ilsangyoon <yoonilsang.umass@gmail.com> 9aa8f21df25a7564190c77409285e2ee6c8b9efa m
PIPE-3037: Changed upper and lower threshold parameter for score
-
Ajay Vibhute 9b09ac10aa3c105cbc95aee66acc0e6dfab10eaf m
PIPE-2582: Changes suggested in PR review
-
Ajay Vibhute 97bf2dec6083a339cba699c11222c226343d45ef m
PIPE-2512: Changes suggested in PR review
-
ilsangyoon <yoonilsang.umass@gmail.com> 4fac328b35448605917831e1733529831f1f1504 m
PIPE-3036: Changed upper and lower parameter for score
-
Luke Maud ef7a76e314f8eb8de28402720b4c4c5e1bdf82db m
Pull request #1775: PIPE-3016 polarization calibration recipes second hifa bandpass after hifa lockrefant should not
Merge in PIPE/pipeline from PIPE-3016-polarization-calibration-recipes-second-hifa_bandpass-after-hifa_lockrefant-should-not to main
* commit '73c2dd1ef7a52b32229634b19b59610cd75013d2':
PIPE-3016 edit to move table unregister to within merge to context at end of stage run
PIPE-3016: minor codestyle changes.
PIPE-3016 typos and better commentary
PIPE-3016 add parameter to unregister the spwphaseup offset table -
Takeshi Nakazato 83b30117a0034d964028bc6114af09c841930a29 m
-
Cristian A. Marocico <c.a.marocico@gmail.com> 2c5f4f25862012ba0200152738b2e093dd285771 m
Added a "Spectra" meta-column
-
Takeshi Nakazato dec10357d667dae9bbf852bf83c13a3fbe0637a0 m
Merge branch 'main' into PIPE-3092-hsd_imaging-ignore-edge-channels-and-channels-overlapped-with-atm-lines-while-examining-qa
-
Takeshi Nakazato 69128df0b50f555d2f17558bcae34eb6f8ff8a1b m
PIPE-3092 exclude edge channels and ATM lines from line-free regions in missed line analysis
-
Luca Di Mascolo 0e80eb93ef9ccb197fc019ce786fdd6d4558dfaf m
Pull request #1820: PIPE-2209: replace call to deprecated AntennaArray.baselines
Merge in PIPE/pipeline from PIPE-2209-eliminate-antennaarray.baselines-deprecation-warning to main
* commit '67aeda9c8e55a6aed96d04b61dc934e82d2960ed':
PIPE-2209: remove deprecated `baselines`/`_baselines` in AntennaArray
PIPE-2209: remove debugging leftover
PIPE-2209: replace call to deprecated AntennaArray.baselines -
ryanaloomis <ryan.a.loomis@gmail.com> 51e930b30e083ad2957fed005969abd8dfb29011 m
PIPE-3090: Filter target SPWs in imageprecheck
-
Rui Xue 143268e351aabe649c573e0ba88ad8ba3073b691 m
Pull request #1818: PIPE-3089: Fix caltable filename inconsistency between serial and parallel modes
Merge in PIPE/pipeline from PIPE-3089-caltable-filenames-might-differ-between-serial-and-parallel-modes-for-some-alma-cal-tasks to main
* commit '5a560095bf4b576af51b2d4a67e4fa35279b7235':
PIPE-3089: set NamingTemplate.task on MPI/Dask workers -
Takeshi Nakazato 7b4dd9b34aff97f0f8ef8f42d93e1d88e9e8ed71 m
Pull request #1802: PIPE-2967 hsd k2jycal backup url for jy k db query
Merge in PIPE/pipeline from PIPE-2967-hsd_k2jycal-backup-url-for-jy-k-db-query to main
* commit 'aedad0e415d5cf24d1920ae51eae2aabaad2c2b8':
revert max casa version
PIPE-2967 removed unnecessary rstrip operation
PIPE-2967 safely remove trailing newline
PIPE-2967 replace "hosts" in the variable names with "urls"
reverse the order of log records to be displayed with original order
improvement of the analysis of warning messages
Set default backup hosts
CASA version requirement for getjyperkalma task
remove unnecessary filter
take care of the case of dbservice=False
Define backup_hosts as vis-dependent property so that it is listed in the weblog
Emit warning message per endpoint URL (per EB)
refactoring
Migrate to getjyperkalma to access Jy/K DB -
Ajay Vibhute d7b197a8b71b63680f7058f858d4c508ab43b7b7 m
PIPE-2512: resolved merge conflicts
-
Hajime Ezawa 69c74d46d6e3ccce79f99b6281fc39816830ffda m
Pull request #1797: PIPE-2958 hsd imaging implement qa score based on the ratio of observed image rms to the theoretical
Merge in PIPE/pipeline from PIPE-2958-hsd_imaging-implement-qa-score-based-on-the-ratio-of-observed-image-rms-to-the-theoretical to main
* commit 'a0a1a262c66310e2ba8aa7111c0e10841400bb14':
PIPE-2958: improved the code for the thresholds of the relevant qascore
PIPE-2958: fix incomplete f-strings, remove unnecessary str() conversion
PIPE-2958: further correction
PIPE-2958: correct an accedental touch
PIPE-2958: align metric_name of missed-lines QA
PIPE-2958: allign wordings to Theoretical/Observed sensitivity in pubished materials
PIPE-2958: modified wordings in hsd_imaging weblog and QA messages
PIPE-2958: typo correction
PIPE-2958: skip processing QA score when result.sensitivity_info is None
PIPE-2958: minor docstring corrections
PIPE-2958: Implement a new QA score based on Observed/Theoretical sensitivities -
Cristian A. Marocico <c.a.marocico@gmail.com> 7de1c70611fd54f5886f214d20e595b8d2ccc212 m
Changed c from float to int to allow multiplication with Decimal
-
Ajay Vibhute b2dd2e80ceeaae6b07152dcec020904d8d2c301b m
PIPE-2512: adding QA score for solint per spw
- pipeline/hifv/heuristics/bandpass.py (version b2dd2e80ceeaae6b07152dcec020904d8d2c301b) (diffs)
- pipeline/hifv/tasks/finalcals/finalcals.py (version b2dd2e80ceeaae6b07152dcec020904d8d2c301b) (diffs)
- pipeline/hifv/tasks/finalcals/qa.py (version b2dd2e80ceeaae6b07152dcec020904d8d2c301b) (diffs)
- pipeline/hifv/tasks/semiFinalBPdcals/qa.py (version b2dd2e80ceeaae6b07152dcec020904d8d2c301b) (diffs)
- pipeline/hifv/tasks/semiFinalBPdcals/semiFinalBPdcals.py (version b2dd2e80ceeaae6b07152dcec020904d8d2c301b) (diffs)
- pipeline/hifv/tasks/testBPdcals/qa.py (version b2dd2e80ceeaae6b07152dcec020904d8d2c301b) (diffs)
- pipeline/hifv/tasks/testBPdcals/testBPdcals.py (version b2dd2e80ceeaae6b07152dcec020904d8d2c301b) (diffs)
- pipeline/qa/scorecalculator.py (version b2dd2e80ceeaae6b07152dcec020904d8d2c301b) (diffs)
-
Shawn Booth b76d81a947b8daec4cd1740cd3ae3b395c2fb45a m
Pull request #1764: PIPE-3002: importdata weblog might not render correctly if a polcal field selected for a parallactic angle plot is present in one EB but not the other, within a single session
Merge in PIPE/pipeline from PIPE-3002-importdata-weblog-might-not-render-correctly-if-a-polcal-field-selected-for-a-parallactic to main
* commit 'b3cdb891bd8c5b1b00db9fa2bc02d97a6e7f86c4':
PIPE-3002: renamed variable inside list comprehension to prevent shadowing and confusion with the outside variable of the same name; added same safeguard in hifv/importdata/renderer as in hifa
added conditional intent filtering to plotms call; changed intent mapping to a list and removed the early mapping to CASA intents
PIPE-3002: removed unnecessary context property added to InputContainer
PIPE-3002: updated parallactic-angle intent filtering; fixed bug in vdp.InputContainer to make sure the context object is present for multi-MS runs -
Shawn Booth 16312f0381af4aede3180215779f238fc48418e6 m
Pull request #1800: PIPE-3079: Documentation: Context Use Cases and Improvements
Merge in PIPE/pipeline from context-use-cases to main
* commit 'd996090a04c869254ff5b3b5974206bbf57692c6':
PIPE-3079: consolidated implementation notes to reduce redundancy
PIPE-3079: removed sections 3 and 4 and appendix A; added clarification stating these use cases are retrospective
PIPE-3079: removed hanging header section
PIPE-3079: updated document language to be more reflective of current behavior and removed suggestions related to an unrelated future design; migrated GAP scenarios to observations and limitations, removing specific implementation suggestions
Docs: update context_use_cases improvements -
Shawn Booth d996090a04c869254ff5b3b5974206bbf57692c6 m
PIPE-3079: consolidated implementation notes to reduce redundancy
-
Cristian A. Marocico <c.a.marocico@gmail.com> 506756f7568ff5cf48b3ddb5088a4151fb5352d4 m
Removed ALMA-only restrictive condition
-
Cristian A. Marocico <c.a.marocico@gmail.com> 1f48794c9a00272113abd1dc5ba215ec8325c88f m
Replaced hard-coded value of c with constant
-
Shawn Booth 71d45c1fc13cf6714da96d32e71add044d0d7c93 m
PIPE-3079: removed sections 3 and 4 and appendix A; added clarification stating these use cases are retrospective
-
Cristian A. Marocico <c.a.marocico@gmail.com> 5265546c06aa37fe4d7d08735f365f4d4f560ff1 m
Fixed new column placement in HTML template
-
Takeshi Nakazato aedad0e415d5cf24d1920ae51eae2aabaad2c2b8 m
Merge branch 'main' into PIPE-2967-hsd_k2jycal-backup-url-for-jy-k-db-query
-
Hajime Ezawa 7510e8781a4261a2642b2d3a177b20a302e79429 m
PIPE-3054: supplement missing type annotation, docstring fix
-
Hajime Ezawa a0a1a262c66310e2ba8aa7111c0e10841400bb14 m
PIPE-2958: improved the code for the thresholds of the relevant qascore
-
Luca Di Mascolo <lucadimascolo@gmail.com> 1bc3d0edf3825efa22f11bcea0a500458d391c15 m
PIPE-2274: fix missing check on flag dictionary
-
Takeshi Nakazato 674acc5fa300d5d875228aac3e96eba9a54934e1 m
revert max casa version
Casa6
-
Dirk Petry <dpetry@eso.org> 059b8ab234522648c1fb903cb3dd576ba9a4b65b
Pull request #1052: CAS-14694
Merge in CASA/casa6 from CAS-14694 to master
* commit '6f1345ac52f7252da24ccdc916163333039734d1':
For CAS-14694: In SolvableVisCal::setSimulate, for simulation of DJones only, short-circuit VI major and minor iteration over the MS so that instrumental polarization is neither time- nor (for mosaic sims) field-dependent. This breaks 'testSky_intLeak' in test_task_simobserve.py. NB: this is not yet a complete polarimetry simulation because parang is not yet treated.
CAS-14694: update simobserve task built-in help for leakage parameter
CAS-14694: make simobserve produce full-pol simulation when input model image is full-pol, independent of leakage
CAS-14694: activated test case testNZ_intMan_Leak and made sure leakage is only applied for obsmode='int'
Enable teardown for test_task_simobserve
CAS-14694: Made ANoise::nPar() in AMueller.h return 4 instead of 2.
CAS-14694: changes which seem to solve the remaining problems
CAS-14694: reverted use of sm.setfeed in simobserve task
CAS-14694 modified the task script to make appropriate sm tool calls when leakage>0 -
George Moellenbrock 6f1345ac52f7252da24ccdc916163333039734d1
For CAS-14694: In SolvableVisCal::setSimulate, for simulation of DJones only, short-circuit VI major and minor iteration over the MS so that instrumental polarization is neither time- nor (for mosaic sims) field-dependent. This breaks 'testSky_intLeak' in test_task_simobserve.py. NB: this is not yet a complete polarimetry simulation because parang is not yet treated.
-
Rui Xue 8f5e9581750c4df973277e17e3e5d999f318e7f7
Pull request #1030: CAS-14758: Optimize image processing by caching tool handles in fill_summary_minor to avoid redundant open/close operations
Merge in CASA/casa6 from CAS-14758 to master
* commit 'a46b0fa46be4985558dfe705e9f2eccf577b4aed':
CAS-14758: Handle missing residual images when constructing minor cycle summary
CAS-14758: Optimize image processing by caching tool handles in fill_summary_minor to avoid redundant open/close operations -
Rui Xue 44d9e7387ac3f53b7841b93a583e1354e361c06f
Pull request #1040: CAS-14622
Merge in CASA/casa6 from CAS-14622 to master
* commit '6599d358e517e2dc3f94260a0d8770dd0cc64964':
CAS-14622: Fix out-of-bounds MPI segmentation faults in BriggsCubeWeightor
CAS-14622: Fix ncores-dependent tclean segmentation fault in `BriggsCubeWeightor` -
Dirk Petry <dpetry@eso.org> e9db1cf9435f211d68a67190f8e650670a5de0eb
CAS-14694: update simobserve task built-in help for leakage parameter
-
Dirk Petry <dpetry@eso.org> 567c15cd27e8383205d3d8cee02a0be9b68ef5c4
Merge branch 'master' into CAS-14694
-
Akeem Wells 6599d358e517e2dc3f94260a0d8770dd0cc64964
Merge branch 'master' into CAS-14622
-
Bob Garwood 3f49f7503b516f27ec0cf65423a7a9e51c238085
Pull request #1049: CAS-14792
Merge in CASA/casa6 from CAS-14792 to master
* commit 'c4206a386e84fa23a749f5b95753860d36396bd7':
added more text to describe the measures data install step
readme.md uses www.astron.nl now for measures data -
Akeem Wells c4206a386e84fa23a749f5b95753860d36396bd7
Merge branch 'master' into CAS-14792
-
Bob Garwood 3493dc9839d8a596467d534b28d465c141bc35b2 m
added more text to describe the measures data install step
-
Bob Garwood 632ee87ba92665433856b99c66e70e590c1fff99 m
readme.md uses www.astron.nl now for measures data
-
Srikrishna Sekhar a46b0fa46be4985558dfe705e9f2eccf577b4aed m
Merge remote-tracking branch 'origin/master' into CAS-14758
-
Dirk Petry <dpetry@eso.org> 84f01fe0d32bbe506c9c0a9eb98a6b79da1636e8 m
CAS-14694: make simobserve produce full-pol simulation when input model image is full-pol, independent of leakage
-
Akeem Wells fe4695a054c35ccb9b648d91e0871c322bc267ad m
Merge branch 'master' into CAS-14622
-
Rui Xue ce691fac04de68e393cb9495357f72e9271ff892 m
CAS-14758: Handle missing residual images when constructing minor cycle summary
-
Rui Xue f03ba5194e6eaca95fb699a2fa3cccd2c7658cbf m
CAS-14758: Optimize image processing by caching tool handles in fill_summary_minor to avoid redundant open/close operations
-
Rui Xue 8fbe09615738a7a02c5c218a2ba4598911874183 m
CAS-14622: Fix out-of-bounds MPI segmentation faults in BriggsCubeWeightor
The SynthesisImager uses BriggsCubeWeightor to compute weights for the image cube. During distributed MPI execution (`ncores > 1`), the output cube's spectral axis is partitioned across multiple nodes into smaller local subcubes.
When evaluating MS data channels, `GridFT::channelMap` maps the visibilities' frequencies into the local subcube's channel indices. If an MS contains dataset frequencies (e.g., from large Doppler swings or wide ephemeris velocity changes) that land *above* the local MPI subcube's maximum channel, `channelMap` returns an index exceeding the local array bounds.
Previously, BriggsCubeWeightor only checked the lower bound (`chanMap(chn) > -1`) before using the calculated index to access `wgtDensity` and the `f2_p`/`d2_p` arrays. Accessing these arrays with indices belonging to higher MPI partitions resulted in unconstrained out-of-bounds memory accesses, causing silent heap corruption or explicit Segmentation Faults, heavily dependent on the MPI slicing and dataset footprint.
This commit resolves the issue by introducing strict upper-bound channel footprint constraints (`chanMap(chn) < shape()[3]`) in both `weightUniform` and `getWeightUniform`. Data channels that mathematically fall outside the upper bounds of the local MPI partition are now securely ignored. -
Rui Xue 3000ee0fe103736833f139a6c7dbdf3eca4abb82 m
CAS-14622: Fix ncores-dependent tclean segmentation fault in `BriggsCubeWeightor`
The SynthesisImager uses `BriggsCubeWeightor` to compute weights for the image cube. During this calculation, `BriggsCubeWeightor::cube2Matrix` extracts raw storage pointers `pcube` and `pflags` from `casacore::Array::getStorage(deleteIt)`.
Previously, the code incremented these exact pointers (`*pflags = *pcube++`) during array traversal. At the end of the function, the incremented pointers were passed directly to `freeStorage()` and `putStorage()`.
When array memory isn't contiguous (which occurs when data is sliced across MPI ranks), `getStorage()` allocates a temporary buffer and sets `deleteIt = true`. Passing the shifted pointer into `freeStorage()` invokes `delete[]` on an invalid memory address, causing heap corruption and a Segmentation fault randomly depending on the data slicing.
This commit resolves the issue by securely caching the original pointers returned from `getStorage` into distinct variables. The traversal loops now use separate copies of these pointers, enabling the original allocation addresses to be correctly evaluated and freed by `freeStorage()` and `putStorage()`. -
Dirk Petry <dpetry@eso.org> 4726346e973f5a3c785530a712e2479ff0ce3e3f m
Merge branch 'CAS-14694' of https://open-bitbucket.nrao.edu/scm/casa/casa6 into CAS-14694
-
Dirk Petry <dpetry@eso.org> c0d3e7a5d5bde00a576b1eda1affede4b660df06 m
CAS-14694: activated test case testNZ_intMan_Leak and made sure leakage is only applied for obsmode='int'
-
Dirk Petry <dpetry@eso.org> c896f72b5b4c031a88ef6ae292bd578e7fe5e4bd m
CAS-14694: Merge branch 'master' into CAS-14694
-
Akeem Wells 5531296aa4fcd61a0f096539e16d116479b6af64 m
Enable teardown for test_task_simobserve
-
Dirk Petry <dpetry@eso.org> 6656f07b8b82cc43b7f3bad62eaaf936fa1d4f2b m
CAS-14694: Made ANoise::nPar() in AMueller.h return 4 instead of 2.
-
Dirk Petry <dpetry@eso.org> b2d9854421f55b26faaec5b843a0d835ebc6feb7 m
CAS-14694: changes which seem to solve the remaining problems
- casatasks/src/private/task_simobserve.py (version b2d9854421f55b26faaec5b843a0d835ebc6feb7) (diffs)
- casatasks/tests/tasks/test_task_simobserve.py (version b2d9854421f55b26faaec5b843a0d835ebc6feb7) (diffs)
- casatools/src/code/synthesis/MeasurementComponents/AMueller.h (version b2d9854421f55b26faaec5b843a0d835ebc6feb7) (diffs)
- casatools/src/code/synthesis/MeasurementEquations/VisEquation.cc (version b2d9854421f55b26faaec5b843a0d835ebc6feb7) (diffs)
-
Dirk Petry <dpetry@eso.org> 99c16562c6617e2ce5918f6121fa25e7a1c18081 m
CAS-14694: Merge branch 'master' into CAS-14694
-
Dirk Petry <dpetry@eso.org> 557f387d9d81ba63eddc060f38e9fccd38d2f296 m
CAS-14694: reverted use of sm.setfeed in simobserve task
-
Dirk Petry <dpetry@eso.org> d502a409320f4c50964c3df924bde31e1737814f m
CAS-14694 modified the task script to make appropriate sm tool calls when leakage>0