Build: #5 was successful Changes by Shawn Booth

Build result summary

Details

Completed
Queue duration
< 1 second
Duration
14 minutes
Labels
None
Agent
cbt-pipeline-test.cv.nrao.edu
Revision
d985fec6bf51b70c6b92bf7d729b856940d3ed4e
Total tests
691
Successful since
#2 ()

Tests

Code commits

Author Commit Message Commit date
Shawn Booth Shawn Booth d985fec6bf51b70c6b92bf7d729b856940d3ed4e Merge remote-tracking branch 'origin/context-serializer' into context-serializer
Shawn Booth Shawn Booth bd92715f277279b406681e3368fb9aa06fc95ae4 m Implement Phase 2: weblog rendering optimization with per-renderer caching
Optimize pipeline weblog generation through strategic in-memory caching and
precomputation of expensive plot data. Move computation from render-time to
execution-time for applycal and importdata tasks.

Key improvements:
- Per-renderer timing instrumentation (htmlrenderer.py WebLogGenerator.render)
- T2_1DetailsRenderer: Cache expensive plot charts (IntentVsTime, FieldVsTime, etc)
- T2_2_3Renderer: Cache antenna setup plots (PlotAntsChart, UVChart)
- T2_2_4Renderer: Cache sky setup plots (AzElChart, SunTrackChart, etc)
- Applycal: Precompute calapps/caltypes during merge_with_context, store in metadata
- Importdata: Precompute summary rows and flux table data, store in metadata
- ALMA Importdata: Precompute parallactic angle plots for caching

Performance results:
- Phase 1 (metadata export): 120 min → 57 min (52.5% improvement)
- Phase 2 (caching optimizations): 57 min → 50 min (12% additional improvement)
- Total: 58.3% pipeline runtime reduction (70 minutes saved)

Modified files:
- pipeline/infrastructure/renderer/htmlrenderer.py: Timing and caching
- pipeline/h/tasks/applycal/applycal.py: _cache_weblog_data() precomputation
- pipeline/h/tasks/applycal/renderer.py: Cache-aware rendering
- pipeline/h/tasks/importdata/importdata.py: Summary data precomputation
- pipeline/h/tasks/importdata/renderer.py: Cache-aware rendering
- pipeline/h/templates/importdata.mako: Cache support
- pipeline/hifa/tasks/importdata/almaimportdata.py: Parang plot caching
- pipeline/hifa/tasks/importdata/renderer.py: Cache-aware rendering
- pipeline/infrastructure/launcher.py: Metadata export robustness
- pipeline/infrastructure/serializers.py: Added load_result_metadata()