Build: #3 was successful Changes by Rui Xue <rx.astro@gmail.com>
Code commits
Pipeline
-
Rui Xue <rx.astro@gmail.com> 5494b0144b4c425948f8e46843df37d645bd2e0b
Merge remote-tracking branch 'origin/main' into PIPE-1677-multiple-mses-in-vlass-imaging-pipeline
-
Rui Xue edfa947e20d405cd2555af1517b94c08b780b9b2 m
Pull request #1756: PIPE-2997/PIPE-2998: expand the gh action unit test matrix
Merge in PIPE/pipeline from PIPE-2997-expand-the-gh-action-unit-test-matrix to main
* commit '233566793c8887be813de84e5000119af22028a8':
PIPE-2997: Additional update/fixes to the workflow template
PIPE-2997: Add Pixi-based GitHub Actions workflows/templates for documentation building and unit testing
PIPE-2998: Recommend `memray` for memory use profiling as `memory_profiler` is no longer maintained.
PIPE-2998: Use `shutil.which` for improved compatibility with Python 3.13
PIPE-2997: Source `CASA6` packages explicitly from the NRAO PyPI index
PIPE-2997: Add new Pixi environments in `pyproject.toml` -
Rui Xue 56e111a89b8e0ad55239ece97777b1dcde7f4823 m
Pull request #1159: PIPE-2014: improve the timing stats in timetracker.json and weblog
Merge in PIPE/pipeline from PIPE-2014-improve-timetracker-and-weblog-timing-stats to main
* commit '38d910f91e9b3fbbf159b54bc4a0f0c17e2d4b60':
PIPE-2014: Use `datetime.timezone.utc` instead of `datetime.UTC` for Python 3.10 compatibility
PIPE-2014: Update task duration mapping to use integer keys for stage numbers in timetracker
PIPE-2014: Improve task duration formatting in HTML renderer and Mako template
PIPE-2014: Revert to the old approach of tracking weblog rendering duration - restricted to rendering of per-stage task result page
PIPE-2014: Switch to timezone-aware `datetime` in UTC for recording timestamps from `timetracker`
PIPE-2014: Switch to `shelve.open()` which returns a shelf object that is a context manager since Python 3.4
PIPE-2014: Ensure `TaskCompleteEvent` is sent for non-"multi_vis" tasks.
PIPE-2014: add back a code block accidentally removed from the last merge conflict resolution.
PIPE-2014: use the timetracker event stats for task duration calculations.
PIPE-2014: optimize the renderer execution order for more accurate timing stats displayed in the weblog.
PIPE-2014: start the weblog creation timer at the beginning of weblog rendering. -
Rui Xue <rx.astro@gmail.com> 233566793c8887be813de84e5000119af22028a8 m
PIPE-2997: Additional update/fixes to the workflow template
* `shell:` is evaluated at workflow parse time, not at runtime, so `${{ inputs.pixi_env }}` isn't available in the shell: field for reusable workflows.
* Disable caching for Pixi environment to ensure the latest dev snapshots are used
* Fix a bug in the CASA config stage of the Pixi template
* Remove unnecessary solve-group entries
* Update `protobuf` version specification in `environment.yml` to ensure compatibility with `casaplotms` -
Rui Xue <rx.astro@gmail.com> 789f75259e531c95b496974fb171ebf570a958f9 m
PIPE-2997: Add Pixi-based GitHub Actions workflows/templates for documentation building and unit testing
- .github/workflows/build-gh-pages-pixi.yml (version 789f75259e531c95b496974fb171ebf570a958f9) (diffs)
- .github/workflows/env-setup-run-template-pixi.yml (version 789f75259e531c95b496974fb171ebf570a958f9) (diffs)
- .github/workflows/test-unit-pixi.yml (version 789f75259e531c95b496974fb171ebf570a958f9) (diffs)
-
Rui Xue <rx.astro@gmail.com> 13dd7f53ef6b406f978e3563ef1817a607ba1218 m
PIPE-2998: Recommend `memray` for memory use profiling as `memory_profiler` is no longer maintained.
-
Rui Xue <rx.astro@gmail.com> 2ea804b5013f7e6ff2678b2f192bc5c5b39ea3db m
PIPE-2998: Use `shutil.which` for improved compatibility with Python 3.13
-
Rui Xue <rx.astro@gmail.com> 24162d0b26513b076ae54beee69386dfbf517700 m
PIPE-2997: Source `CASA6` packages explicitly from the NRAO PyPI index
To reduce the load on the NRAO PyPI index caused by uv's aggressive
parallel resolution strategy, we explicitly declare which packages must
be sourced from casa-pip.nrao.edu using per-dependency `index` fields:
- casashell, casatools, casatasks, casampi, casaplotms, casaplotserver
All other dependencies (numpy, scipy, grpcio, protobuf, etc.) resolve
from the default PyPI index only, avoiding unnecessary queries to the
NRAO server during environment solving. -
Rui Xue <rx.astro@gmail.com> 590fa43489a85d49ca3dbbd88730b31a682bcc45 m
PIPE-2997: Add new Pixi environments in `pyproject.toml`
-
Rui Xue 85cf92dc6fba8e3c2521cb90ad7c29f23121b5d2 m
Pull request #1755: PIPE-2996: Cache casarundata in env setup template
Merge in PIPE/pipeline from cache-casarundata-in-env-setup-template to main
* commit '32e17ed444ada97e6cbdf56cfc826e63d8473113':
Remove the redundant `casaconfig` setup again in `test-unit.yml`
Improve `casaconfig/casarundata` setup in the template with GH action caching -
Rui Xue <rx.astro@gmail.com> 38d910f91e9b3fbbf159b54bc4a0f0c17e2d4b60 m
Merge remote-tracking branch 'origin/main' into PIPE-2014-improve-timetracker-and-weblog-timing-stats
-
Rui Xue <rx.astro@gmail.com> 6c63b253b72e657075e037da7df85893b797f19e m
PIPE-2014: Use `datetime.timezone.utc` instead of `datetime.UTC` for Python 3.10 compatibility
-
Rui Xue 261c8820cf8063cfe2473d253e43767b132bbc31 m
Pull request #1754: PIPE-2995: Add GitHub workflow for syncing with On-Prem Bitbucket
Merge in PIPE/pipeline from sync-bitbucket-github-repos-with-workflow to main
* commit '598ae9e3096c183c8230659aa9a2caeb715ff123':
PIPE-2995: Add GitHub workflow for syncing with On-Prem Bitbucket -
Rui Xue <rx.astro@gmail.com> 598ae9e3096c183c8230659aa9a2caeb715ff123 m
PIPE-2995: Add GitHub workflow for syncing with On-Prem Bitbucket
-
Rui Xue 5dead9ae807422078ab5ba32dacd80165f625251 m
Pull request #1729: Update GitHub workflows and environment configuration to include unit tests for VLASS-PL2026.
Merge in PIPE/pipeline from unit-test-vlass2026 to main
* commit 'd9b21b61a46e2b5984530ce12c589bfcbf4805cd':
PIPE-2940: Improve test data checkout by adding a directory existence check before cloning the repository
PIPE-2940: Add Pixi environment definitions for the PL2026 development
PIPE-2940: add `nondefaults` to the channel configuration in `environment_vlass2026.yml`
PIPE-2940: Update coverage configuration to exclude additional lines and improve test reporting
Update GitHub workflows and environment configuration to include unit tests for VLASS-PL2026. -
Rui Xue d6e477200df51044506c241e43ea4e03b6b3d1c4 m
PIPE-2014: Update task duration mapping to use integer keys for stage numbers in timetracker
-
Rui Xue 70ccb2d1e69f7d5fe145c9d918a38f79efc90820 m
Pull request #1733: PIPE-2937/PIPE-2929/PIPE-2982
Merge in PIPE/pipeline from PIPE-2937-avoid-writing-large-tmp-large-images-from-the-nf-snr-calculation-function to main
* commit '29110a640d1e0d17d853a7b45d55d91bb63ffa27':
PIPE-2937: Bypass the `JobRequest` wrapper layer for direct `casaplotms` execution and enable debug tracebacks for mask checking errors
PIPE-2937: Reduce the number of used `iatool` instances and use in-place Boolean operators to further reduce peak memory usage
PIPE-2982: Replace vectorized contour plotting of clean mask islands with raster boundary plotting, to avoid excess memory usage for large images
PIPE-2937: Fix an issue with image removal before closing the associated `ia` instances
PIPE-2929: Fix a regression related to casa-compatble field name conversion, introduced when porting mosaic self-calibration (PIPE-2001)
PIPE-2937: Refactor `estimate_SNR` and `estimate_near_field_SNR` to avoid writing large temporary images, improving efficiency and memory usage
PIPE-2937: Refactor `create_noise_histogram` to improve efficiency and readability: -
Rui Xue 29110a640d1e0d17d853a7b45d55d91bb63ffa27 m
PIPE-2937: Bypass the `JobRequest` wrapper layer for direct `casaplotms` execution and enable debug tracebacks for mask checking errors
-
Rui Xue d779f7928d0ed958341c2bf25f3b20832f5592b4 m
PIPE-2014: Improve task duration formatting in HTML renderer and Mako template
-
Rui Xue 67e48129929d51db51793d7d086597c08294fbbe m
PIPE-2014: Revert to the old approach of tracking weblog rendering duration - restricted to rendering of per-stage task result page
-
Takeshi Nakazato 9e4f33aac7ac435cb3d1dc09b867f5e3c0f062d9 m
Pull request #1727: PIPE-2894 always display channel axis to sparse profile map
Merge in PIPE/pipeline from PIPE-2894-hsd_baseline-add-channel-axis-to-the-top-panel-of-sparse-profile-map to main
* commit '3a599a4f0e1e027e7f556d83ea70f672836bf335':
move channel axis label upward slightly
added note to docstring for add_channel_axis
PIPE-2894 always display channel axis to sparse profile map -
Rui Xue 8e265de65f7535cdf17b1192e9f4fafe325eb87b m
PIPE-2014: Switch to timezone-aware `datetime` in UTC for recording timestamps from `timetracker`
-
Rui Xue f69f39c3c4b99ea59d85e8b9da2438a2e56b7b18 m
PIPE-2014: Switch to `shelve.open()` which returns a shelf object that is a context manager since Python 3.4
-
Rui Xue 73998b946595418d179ed619454b197430a0ab5c m
Merge remote-tracking branch 'origin/main' into PIPE-2014-improve-timetracker-and-weblog-timing-stats
Resolved conflicts:
- `pipeline/infrastructure/basetask.py`
- `pipeline/infrastructure/renderer/htmlrenderer.py` -
Kristin Berry 465285e01b87f372c957660f993a467c2b67a264 m
Pull request #1745: PIPE-2985: Update docs to reflect change to use launcher to access the Pipeline and context objects
Merge in PIPE/pipeline from PIPE-2985-update-developer-documentation-to-reflect-change-in-how-to-fetch-the-context to main
* commit '84610e456d06c5b0b695a098e974859674ecf063':
PIPE-2985: Update bullet-points to use - rather than o
PIPE-2985: Remove outdated test instructions from docstrings. Similar test instructions exist in developer documentation markdown files.
PIPE-2985: Remove unneeded import statements and restore notebook execution count to 1
PIPE-2985: Update docs to reflect change to use launcher to access the Pipeline and context objects -
Kristin Berry 84610e456d06c5b0b695a098e974859674ecf063 m
PIPE-2985: Update bullet-points to use - rather than o
-
Kristin Berry bfbbaa0368fe900908701bbb32276f319be535df m
PIPE-2985: Remove outdated test instructions from docstrings. Similar test instructions exist in developer documentation markdown files.
- pipeline/h/tasks/exportdata/exportdata.py (version bfbbaa0368fe900908701bbb32276f319be535df) (diffs)
- pipeline/h/tasks/flagging/flagdeterbase.py (version bfbbaa0368fe900908701bbb32276f319be535df) (diffs)
- pipeline/h/tasks/restoredata/restoredata.py (version bfbbaa0368fe900908701bbb32276f319be535df) (diffs)
- pipeline/hifv/tasks/flagging/flagdetervla.py (version bfbbaa0368fe900908701bbb32276f319be535df) (diffs)
- pipeline/hsd/tasks/exportdata/exportdata.py (version bfbbaa0368fe900908701bbb32276f319be535df) (diffs)
-
Rui Xue d9b21b61a46e2b5984530ce12c589bfcbf4805cd m
PIPE-2940: Improve test data checkout by adding a directory existence check before cloning the repository
-
Kristin Berry 5129aacd04c5b51be9953b76172fb17ae994225f m
PIPE-2985: Remove unneeded import statements and restore notebook execution count to 1
-
Rui Xue 59c8188f469ff3a13e3cf13514ca275405dbc0e4 m
PIPE-2940: Add Pixi environment definitions for the PL2026 development
-
Kristin Berry 135cf97804064a13bedd459633f8849061b6ca0a m
PIPE-2985: Update docs to reflect change to use launcher to access the Pipeline and context objects
- docs/source/develdocmd/DeveloperDocumentation.md (version 135cf97804064a13bedd459633f8849061b6ca0a) (diffs)
- docs/source/develdocmd/ways_to_run_the_pipeline.md (version 135cf97804064a13bedd459633f8849061b6ca0a) (diffs)
- docs/source/notebooks/context.ipynb (version 135cf97804064a13bedd459633f8849061b6ca0a) (diffs)
- pipeline/h/tasks/exportdata/exportdata.py (version 135cf97804064a13bedd459633f8849061b6ca0a) (diffs)
- pipeline/h/tasks/flagging/flagdeterbase.py (version 135cf97804064a13bedd459633f8849061b6ca0a) (diffs)
- pipeline/h/tasks/restoredata/restoredata.py (version 135cf97804064a13bedd459633f8849061b6ca0a) (diffs)
- pipeline/hifv/tasks/flagging/flagdetervla.py (version 135cf97804064a13bedd459633f8849061b6ca0a) (diffs)
- pipeline/hsd/tasks/exportdata/exportdata.py (version 135cf97804064a13bedd459633f8849061b6ca0a) (diffs)
- pipeline/infrastructure/new_pipeline_task/verify_new_pipeline_task.py (version 135cf97804064a13bedd459633f8849061b6ca0a) (diffs)
-
Rui Xue ff81a69f88b19bde5acbec5fcc55c199e98bf8fc m
PIPE-2937: Reduce the number of used `iatool` instances and use in-place Boolean operators to further reduce peak memory usage
-
Rui Xue 5d766135d6e53fcaf6a8587471448022205f7311 m
PIPE-2982: Replace vectorized contour plotting of clean mask islands with raster boundary plotting, to avoid excess memory usage for large images
-
Rui Xue 31e86a1d7c971cb9da5283a724b8b03d673745d0 m
PIPE-2937: Fix an issue with image removal before closing the associated `ia` instances
-
Rui Xue 448be2c8aed741f59ac62b3667bc83c98fc73c5b m
PIPE-2929: Fix a regression related to casa-compatble field name conversion, introduced when porting mosaic self-calibration (PIPE-2001)
-
Rui Xue 8c358ee6dbe9eb8c1ea895f1b3e2c287ac16c3e6 m
PIPE-2937: Refactor `estimate_SNR` and `estimate_near_field_SNR` to avoid writing large temporary images, improving efficiency and memory usage
-
Rui Xue 8a0f7c3bf61b6a94b352e2d27b5db421574d7add m
PIPE-2937: Refactor `create_noise_histogram` to improve efficiency and readability:
- Remove temporary file handling.
- Use `ia.histograms` for histogram calculation. -
Rui Xue 32e17ed444ada97e6cbdf56cfc826e63d8473113 m
Remove the redundant `casaconfig` setup again in `test-unit.yml`
-
Rui Xue 7140e2bf7b9a2d1c71e9358681d5e1e1d531a5cf m
Improve `casaconfig/casarundata` setup in the template with GH action caching
-
Rui Xue 1021a12af545145f4d575bcc69f6f5c524c9318d m
Update GitHub workflows and environment configuration to include unit tests for VLASS-PL2026.
- Modify build-gh-pages.yml to trigger on changes to `environment*.yml`
- Update env-setup-run-template.yml to allow dynamic conda environment file input
- Adjust test-unit.yml to support multiple conda environments and update artifact naming logic
- Refine apisummary.rst to correct module references
- Add `new environment_vlass2026.yml` for specific pipeline dependencies of VLASS-PL2026- .github/workflows/build-gh-pages.yml (version 1021a12af545145f4d575bcc69f6f5c524c9318d) (diffs)
- .github/workflows/env-setup-run-template.yml (version 1021a12af545145f4d575bcc69f6f5c524c9318d) (diffs)
- .github/workflows/test-unit.yml (version 1021a12af545145f4d575bcc69f6f5c524c9318d) (diffs)
- docs/source/apisummary.rst (version 1021a12af545145f4d575bcc69f6f5c524c9318d) (diffs)
- environment_vlass2026.yml (version 1021a12af545145f4d575bcc69f6f5c524c9318d) (diffs)
-
Rui Xue 3aedaf8e0ccd58c7e15a27a43f638464f9cdcb43 m
PIPE-2940: add `nondefaults` to the channel configuration in `environment_vlass2026.yml`
-
Rui Xue 9ced50c5939dce02f1b0b571ef9ccdcae59533db m
PIPE-2940: Update coverage configuration to exclude additional lines and improve test reporting
-
Takeshi Nakazato 3a599a4f0e1e027e7f556d83ea70f672836bf335 m
move channel axis label upward slightly
-
Takeshi Nakazato f0f64bcdf812d8198d1c62dfe5fd779c245e73c9 m
added note to docstring for add_channel_axis
-
Takeshi Nakazato 9d26316d3f9be62fb3470dba9aac87e3cb77f95d m
Merge branch 'main' into PIPE-2894-hsd_baseline-add-channel-axis-to-the-top-panel-of-sparse-profile-map
-
Takeshi Nakazato 4d26e9806f4963cb529fa490af25b0bcab1be550 m
PIPE-2894 always display channel axis to sparse profile map
-
Rui Xue 9206657f8927dae5740597ce728df9eff45cc303 m
Merge remote-tracking branch 'origin/main' into PIPE-2014-improve-timetracker-and-weblog-timing-stats
Resolved conflicts in:
* pipeline/infrastructure/renderer/htmlrenderer.py -
Rui Xue ee9ca54e1b87e77b2b4dca876e1338a63ff5573e m
PIPE-2014: Ensure `TaskCompleteEvent` is sent for non-"multi_vis" tasks.
Fixes a regression from PIPE-2848 where a TaskCompleteEvent was not
being emitted when a subtask process was handled by
`StandardTasktemplate._handle_multiple_vis()`. -
Rui Xue a81be760ec06fdc34a98886d1a080d1f7c2be52d m
PIPE-2014: add back a code block accidentally removed from the last merge conflict resolution.
-
Rui Xue 95863486b219479860952afa4d3fd1bde70bb178 m
Merge remote-tracking branch 'origin/main' into PIPE-2014-improve-timetracker-and-weblog-timing-stats
Resolve conflicts in `pipeline/pipeline/infrastructure/renderer/htmlrenderer.py` -
Rui Xue 3a11df351b0532c2f19f36fb78d24d60f06aa6f5 m
Merge remote-tracking branch 'origin/main' into PIPE-2014-improve-timetracker-and-weblog-timing-stats
-
Rui Xue da6e2f2b8828a8704739407bf73e432c6567c25f m
PIPE-2014: use the timetracker event stats for task duration calculations.
-
Rui Xue 0e86fcfe576eaa0f381a989c5e308df272491b04 m
PIPE-2014: optimize the renderer execution order for more accurate timing stats displayed in the weblog.
-
Rui Xue 309ffbf553424d1531c32681a504e135faca09fd m
PIPE-2014: start the weblog creation timer at the beginning of weblog rendering.