Build: #538 failed Scheduled with changes by George Moellenbrock and Hendrik Muller
Code commits
Casa6
-
George Moellenbrock a8b6cc9b6ecadea9c95af1dc28341ed198a48da8
Pull request #1068: CAS-14763
Merge in CASA/casa6 from CAS-14763 to master
* commit 'f2e0db200c55d4b3c26e06bf80d6211944a47756': (74 commits)
Change type to G
Add standard errors for parameters
Update xml and tests
Change default clock order and update the test data origin
Remove debugging list type, return of task zenithdelaycal is dict
Updated test to use data with all VLBA antennas
Updated test header
Add test data procedence
Adds more tests
Add per polarization clock solution
Switch to CASA measures to compute antenna elevations
Updated tests for zenithdelaycal
Name based antenna selection from MS
Add implementation to ZenithDelayJones
Add tests
Add implementation to ZenithDelayJones
Add caltable output
For CAS-14763, added comments to aid development
For CAS-14763, in ZenithDelayJones::setApply(Record&), removed some unnecessary stuff.
For CAS-14763, added skeletal ZenithDelayJones class (ZenithDelayCal.{h,cc}) to eventually support application of zenith-angle dependent macroscopic delay calibration for VLBI, and make VisCalGlobals's SolvableVisCal factories aware of it. This requires further development for it to work.
... -
Hendrik Muller 48dba78833583f0737a0135c320a1b8bea09a03a
Pull request #1063: CAS-14797
Merge in CASA/casa6 from CAS-14797 to master
* commit 'd0332c4e3cda643f0a1b8a128c35f1a6ee2e973f':
Streamline the implementations of helper functions in just two helper functions now made available in MatrixCleaner.cc
Search only over inner square
Make MSMT-CLEAN compatible with previous runs again
repeat same accelerations for AspMatrixCleaner and for MultiTermMatrixCleaner
This code will increase the speed for MS-CLEAN. This is achieved by two main improvements: -> Subtract beam is faster than casacore functionality which creates temporary copies -> Run peak finding only over square determined by blcdirty and trcdirty rather than the full image -> again, the direct implementation is faster than casacore minMax functionality -
Hendrik Muller 7279dc159c47f7a3fc656e6e2c11d942cc0c383a
Pull request #1062: CAS-14814
Merge in CASA/casa6 from CAS-14814 to master
* commit 'cb65c9b710137b44a5624dadade29b814c6a2a3d':
Remove dead code
labeling, pruning and blob-size counting in memory, avoid temporary copies
Optimize Automasking region labelling and pruning
Main bottleneck is binaryDilation computation. This part of the code has been accelerated now, by: 1.) Remove IPosition constructions from inner loop 2.) Use direct storage pointers 3.) Precompute structure offsets 4.) Return changed from BinaryDilationCore 5.) Remove the full image check in BinaryDilation() 6.) Avoid extra full-plane scans -
Hendrik Muller b5ed254f61284e8d389fac82e4b21e8fe30e5194
Pull request #1061: CAS-14804
Merge in CASA/casa6 from CAS-14804 to master
* commit 'b21128a27d35f918b11d60dad2d1ff729e92d923':
Inflate epsilon for passing tests across platforms
Add tclean task for CAS-14804, depends on fasr dataset
Update comparison values in test_onefield_asp
Reverse to normalization with psf width again, this makes the unittest testonefieldasp performing superior to old version again
Solution by rescaling the objfunc, and amplitudes, for more details see description at Jira ticket -
Pedro Salas <salas.mz@gmail.com> f2e0db200c55d4b3c26e06bf80d6211944a47756 m
Change type to G
Polarization dependent clock corrections have
to be at the G position in the measurement equation. -
Pedro Salas <salas.mz@gmail.com> fb24dd1237d7fd51d73332ff3e909e5241ff7c68 m
Add standard errors for parameters
-
Pedro Salas <salas.mz@gmail.com> a9f69cae62f58a3db507572250368d3f622d20f4 m
Update xml and tests
- casatasks/src/private/task_zenithdelaycal.py (version a9f69cae62f58a3db507572250368d3f622d20f4) (diffs)
- casatasks/tests/tasks/test_task_zenithdelaycal.py (version a9f69cae62f58a3db507572250368d3f622d20f4) (diffs)
- casatasks/xml/zenithdelaycal.xml (version a9f69cae62f58a3db507572250368d3f622d20f4) (diffs)
-
Pedro Salas <salas.mz@gmail.com> 9e08fac4b625a44b976942466897d5d0118b68e9 m
Change default clock order and update the test data origin
-
Akeem Wells 0cd5ebaa0b28539a3611a43ec2c1aa497bdce994 m
Remove debugging list type, return of task zenithdelaycal is dict
-
Pedro Salas <salas.mz@gmail.com> c29bc1a95bacecfd32e7f2ba8bf59a003f4fad7c m
Updated test to use data with all VLBA antennas
-
Pedro Salas <salas.mz@gmail.com> 8efc298b1ee618a06ba5d6b24c4877a4812e2844 m
Updated test header
-
Pedro Salas <salas.mz@gmail.com> 6d0c27d99a4a0c6cc045e135f98b0bb32ad110c8 m
Add test data procedence
-
Pedro Salas <salas.mz@gmail.com> 461c72a18dde7781909a07f2e5f7632ac6382b31 m
Adds more tests
-
Pedro Salas <salas.mz@gmail.com> f02390b447f0071a83c0e76d32981ff9b14ecf13 m
Add per polarization clock solution
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> b21128a27d35f918b11d60dad2d1ff729e92d923 m
Merge branch 'master' into CAS-14804
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> bfb544eb61bfcb9abf7285412bb4235703c92594 m
Inflate epsilon for passing tests across platforms
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> cb65c9b710137b44a5624dadade29b814c6a2a3d m
Merge remote-tracking branch 'origin/CAS-13898' into CAS-14814
-
Pedro Salas <salas.mz@gmail.com> 02f46298c6e745fbd8b3e67318c60d6686bcc232 m
Switch to CASA measures to compute antenna elevations
-
Pedro Salas <salas.mz@gmail.com> 65b6f51bc9fa28f7c9e6ca65e182a60a0727b504 m
Updated tests for zenithdelaycal
Numerical comparisons of delays are now in ns.
Checks that after correction the delay is lower than the input. -
Pedro Salas <salas.mz@gmail.com> 2dcc51bf29adbbdded08744a793cfca5cd457f34 m
Name based antenna selection from MS
-
Pedro Salas <salas.mz@gmail.com> 6b625b706889644766dbb689f4c85a5b374fe3ae m
Add implementation to ZenithDelayJones
-
Pedro Salas <salas.mz@gmail.com> 33b2fbc415de56930be30570f5fa84d745a20169 m
Add tests
-
Pedro Salas <salas.mz@gmail.com> c3267a755113398c253955e4b5a35188798e4821 m
Add implementation to ZenithDelayJones
-
Pedro Salas <salas.mz@gmail.com> e6a6992396a62e7728b0d3857cf445a7eb3b21ba m
Add caltable output
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> cf29384c41f2f9a68d7cdb0dac981ca13e0ea475 m
Add tclean task for CAS-14804, depends on fasr dataset
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> e33d119e94c5dd69e1f570bf5fe8348c6bcc2383 m
Update comparison values in test_onefield_asp
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> 67ae240e9506db41372a659be98b4941a0653cab m
Remove dead code
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> 57dca645ed29516158c4292ea6b178920a2f6dd1 m
labeling, pruning and blob-size counting in memory, avoid temporary copies
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> 23f3834a2626e7f24feaa28f1caf8ac072cb53db m
Optimize Automasking region labelling and pruning
Replace IPosition based region traversal with flat-index DFS
rewrite blob-wise counting using direct image access
prune small regions in a single image pass -
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> fa59cb340f36c1709342a20a58e3790b4374cb78 m
Main bottleneck is binaryDilation computation. This part of the code has been accelerated now, by: 1.) Remove IPosition constructions from inner loop 2.) Use direct storage pointers 3.) Precompute structure offsets 4.) Return changed from BinaryDilationCore 5.) Remove the full image check in BinaryDilation() 6.) Avoid extra full-plane scans
This new version still needs to get validied solidly. -
George Moellenbrock 8e37292da807f31f380667d723ae1fec237c86da m
For CAS-14763, added comments to aid development
-
George Moellenbrock 86e433f3b0eafb5fa251d98ad45c15694eae424f m
For CAS-14763, in ZenithDelayJones::setApply(Record&), removed some unnecessary stuff.
-
George Moellenbrock 2326a6ec0adcef7d97c752e788ea1faf5e844343 m
Merge branch 'master' into CAS-14763
-
George Moellenbrock 12fa37f46cbda88ba49928e092c276ecfbea4c76 m
For CAS-14763, added skeletal ZenithDelayJones class (ZenithDelayCal.{h,cc}) to eventually support application of zenith-angle dependent macroscopic delay calibration for VLBI, and make VisCalGlobals's SolvableVisCal factories aware of it. This requires further development for it to work.
- casatools/src/code/synthesis/MeasurementComponents/VisCalGlobals.cc (version 12fa37f46cbda88ba49928e092c276ecfbea4c76) (diffs)
- casatools/src/code/synthesis/MeasurementComponents/ZenithDelayCal.cc (version 12fa37f46cbda88ba49928e092c276ecfbea4c76) (diffs)
- casatools/src/code/synthesis/MeasurementComponents/ZenithDelayCal.h (version 12fa37f46cbda88ba49928e092c276ecfbea4c76) (diffs)
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> 4cdb95ad5fe5d8cd4f3bfb41c27ffd0e1f78f5fe m
Reverse to normalization with psf width again, this makes the unittest testonefieldasp performing superior to old version again
-
Pedro Salas <salas.mz@gmail.com> 3ffc4428ba984d05ace5aae7b1f5145b7cad6e91 m
Updated zenithdelaycal.xml
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> 73c9e410f8f52d13449f3c3a3be79a8cd90ca12b m
Solution by rescaling the objfunc, and amplitudes, for more details see description at Jira ticket
- casatools/src/code/synthesis/MeasurementEquations/AspMatrixCleaner.cc (version 73c9e410f8f52d13449f3c3a3be79a8cd90ca12b) (diffs)
- casatools/src/code/synthesis/MeasurementEquations/AspMatrixCleaner.h (version 73c9e410f8f52d13449f3c3a3be79a8cd90ca12b) (diffs)
- casatools/src/code/synthesis/MeasurementEquations/objfunc_alglib.h (version 73c9e410f8f52d13449f3c3a3be79a8cd90ca12b) (diffs)
-
Pedro Salas <salas.mz@gmail.com> 090b5f82c32535a1d8a64498bbb6dca5ea101bfe m
Updated zenithdelaycal.xml
-
Victor de Souza Magalhaes 182e6340b4a6f93b3c4ff855af23e1499361bd2b m
Fixed a few typos
-
Pedro Salas <salas.mz@gmail.com> 04c70f9447a89f4edafe94c88005cc768247d3cc m
Adds docstrings, cleans up unused arguments and avoids computing axis limits for empty data in the visualization function
-
Victor de Souza Magalhaes e00682e32d2302adb3cd69c0d37d81a06148da2c m
Removed an unused import to pandas.
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> d0332c4e3cda643f0a1b8a128c35f1a6ee2e973f m
Streamline the implementations of helper functions in just two helper functions now made available in MatrixCleaner.cc
- casatools/src/code/synthesis/MeasurementEquations/AspMatrixCleaner.cc (version d0332c4e3cda643f0a1b8a128c35f1a6ee2e973f) (diffs)
- casatools/src/code/synthesis/MeasurementEquations/AspMatrixCleaner.h (version d0332c4e3cda643f0a1b8a128c35f1a6ee2e973f) (diffs)
- casatools/src/code/synthesis/MeasurementEquations/MatrixCleaner.cc (version d0332c4e3cda643f0a1b8a128c35f1a6ee2e973f) (diffs)
- casatools/src/code/synthesis/MeasurementEquations/MatrixCleaner.h (version d0332c4e3cda643f0a1b8a128c35f1a6ee2e973f) (diffs)
- casatools/src/code/synthesis/MeasurementEquations/MultiTermMatrixCleaner.cc (version d0332c4e3cda643f0a1b8a128c35f1a6ee2e973f) (diffs)
- casatools/src/code/synthesis/MeasurementEquations/MultiTermMatrixCleaner.h (version d0332c4e3cda643f0a1b8a128c35f1a6ee2e973f) (diffs)
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> 15708ce2bde3db2c99485f74edce8f23494746eb m
Search only over inner square
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> 1d6360878fa2290fc0ddbff3eb608e26724b7bfe m
Make MSMT-CLEAN compatible with previous runs again
-
Victor de Souza Magalhaes c40f06a27ef03eb7be7d8a956f2df103e2ea5155 m
Output dictionary now contain the phases for all SPW and pols at ref time.
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> 14caf11dcd1f721cba63370d77a62e5fbaff7ebe m
repeat same accelerations for AspMatrixCleaner and for MultiTermMatrixCleaner
- casatools/src/code/synthesis/MeasurementEquations/AspMatrixCleaner.cc (version 14caf11dcd1f721cba63370d77a62e5fbaff7ebe) (diffs)
- casatools/src/code/synthesis/MeasurementEquations/AspMatrixCleaner.h (version 14caf11dcd1f721cba63370d77a62e5fbaff7ebe) (diffs)
- casatools/src/code/synthesis/MeasurementEquations/MultiTermMatrixCleaner.cc (version 14caf11dcd1f721cba63370d77a62e5fbaff7ebe) (diffs)
- casatools/src/code/synthesis/MeasurementEquations/MultiTermMatrixCleaner.h (version 14caf11dcd1f721cba63370d77a62e5fbaff7ebe) (diffs)
-
Victor de Souza Magalhaes 86a3b0d97edd1e38d5a5a6eb0543892cea41bdc5 m
Factorized reading of fparam values to separated arrays in data dict and made it easier to include rate in the future (if desired).
-
Victor de Souza Magalhaes c2b47b0a47036191b581091092608d40889469da m
Finished removal of references to boolean describing if this is a dispersion fit; changed default value for dispersive delay polynomial order.
-
Victor de Souza Magalhaes 0d872edc35390c7468aa2915a91b460d7283e59e m
Merge remote-tracking branch 'origin/CAS-14763' into CAS-14763
-
Victor de Souza Magalhaes 234bd0f6d35e6065cd77d8e059d79765ab0605a6 m
Dispersive delays are now fetched and stored in the main data dictionary.
-
Hendrik Muller <hmuller@smurf.aoc.nrao.edu> 384f8b97911148293cf94675f03c60640cac8663 m
This code will increase the speed for MS-CLEAN. This is achieved by two main improvements: -> Subtract beam is faster than casacore functionality which creates temporary copies -> Run peak finding only over square determined by blcdirty and trcdirty rather than the full image -> again, the direct implementation is faster than casacore minMax functionality
For cygnus A, the upspeed in the minor loop was almost an order of magnitude.
TBD: Apply the same to other deconvolvers. -
Victor de Souza Magalhaes bedf82a5c31269c8d40d6f9d6a177758323547ae m
Added parsing for dispersive delay polynomial fit; added tooltips for most functions; refactored vis to ms_name where adequate.
-
Victor de Souza Magalhaes 10a4069a47a58ce9da4fdf699ab064c46781ce45 m
Added protection against refant being below elevation minimum, also it now raises runtime error if there are no line in the system.
-
Victor de Souza Magalhaes 181745c9c090f7e1140a0f9765d0bbf4f85e25be m
Added protection against refant being below elevation minimum, also it now raises runtime error if there are no line in the system.
-
Victor de Souza Magalhaes 41fef1ddbbe27e5d407a8e22172f907cd1c0392a m
Implemented data selection by scan.
-
Victor de Souza Magalhaes bb49241ae63f3f582b25d134f1f9215509e60bef m
Factorised the construction of the spw dict in its own function.
-
Victor de Souza Magalhaes 1f0717afe90721fbed3630273ea65879183c5943 m
Implemented data selection by field.
-
Victor de Souza Magalhaes 9b2f88481c33e17108bfa39ac21c47acf98200e6 m
Plot improvements: changed model markers so that they are more clearly visible; expanded and symmetrical about 0 the Y lims in the residuals plots for a clearer assessment; applied the same x lims for the residuals x-axis; forced a minimal margin for cases where data min and max are the same (to avoid runtime warnings).
-
Victor de Souza Magalhaes 8ef4345775a3bdb377455cc649612e673a1b2fce m
Calibration rows with antenna1 == antenna2 are now ignored in fit as they don't contribute to the solving (0*t = 0).
-
Victor de Souza Magalhaes 07d60d9da9e67ad2cb01730f431a961c60fa6701 m
Set maximal polynomial order to 2 based on experiments with higher order that show that results are degraded with higher polynomial orders.
-
Victor de Souza Magalhaes 68799b1fcdbb5c32f49aaaf0fed579a979f9c36c m
Implemented checks on polynomial orders, it is now possible to disable clock fits or atmospheric fits.
-
Victor de Souza Magalhaes acc6df6d133f9c54dc4605f5ab98738af0e5f6e5 m
System was slightly oversized due to antenna number being fetched from the wrong quantity.
-
Victor de Souza Magalhaes 207f6998d37192b6cc6578d1b7c97fdcb3edb54a m
Re-enabled delay solutions plots.
-
Victor de Souza Magalhaes 8cb6d4927c51b1d3996580949dd21eb3efa5359a m
Factorized data transformation and fitting to work with the input and data dictionaries.
-
Victor de Souza Magalhaes e1a7f5567b2bb711203ff2f8ae34f16e3f10ce51 m
Factorized parsing and data fetching into 2 routines producing 2 dictionaries.
-
Victor de Souza Magalhaes c2f3c5988f8ac2e10bc4f083c76d2a05243cded0 m
Some reordering, and comment out previous prototype.
-
Victor de Souza Magalhaes a3476720ab3ab0d8843cbea91c8d549333dd9388 m
Fully implemented spectral window selection and factorised antenna selection code to also be used by spw selection.
-
Victor de Souza Magalhaes 1583ba6b52d364dc57f4f5163e4ab8b1752e792e m
zennithdelaycal now separates delays by spectral window ID and infra structure to give space for dispersive delay implementation has been added.
-
Victor de Souza Magalhaes 7fd8fdc5ad8682d2c0d40be28faffe6044c3305f m
Y lims in delay plots are now compute used nanmin and nanmax, fully nan delay data (for example from the refant (if there is only one)) is now ignored for plots.
-
Victor de Souza Magalhaes 3416d44593a0d8c4c9ccc4b562332c6559a2a47d m
Added protection against missing data.
-
Victor de Souza Magalhaes 568a899a3d828d629556971e9d03e62177fff5cf m
Phases were being fetched as delays from fringe fit multi band delay table.
-
Victor de Souza Magalhaes 83cb16f4156ba3497906f58e3442810ea59c34b2 m
Implemented parsing and application of antenna selection.
-
Victor de Souza Magalhaes f125efb92cce51cc3ead68928939e16ce21478c9 m
Implemented parsing and application of timerange selection.
-
Victor de Souza Magalhaes 2647a93d151c177d8b672e88c902ba1eb2c6dc8d m
Implemented user decision on atmospheric delay model with elevation.
-
Victor de Souza Magalhaes 3d61748f24ccaea0219a1d5dd72164e577014e18 m
Implemented user decision on elevation limit.
-
Victor de Souza Magalhaes 1e719a2c00cf9438bf04d5a061c7cd1a2836919d m
Implemented toggle on the creation of delay solution plot.
-
Victor de Souza Magalhaes 9b91f9133d7619119d3a948bfa26c650016c23c4 m
Implemented polarization selection.
-
Victor de Souza Magalhaes ea5927af93d1d2fda0d5b6d2a1f71bd999dfab31 m
Added residuals plot at the bottom of the model vs data plots.
-
Victor de Souza Magalhaes 0e0e51b0dbff6a6fb9c0f43562670102be792ccf m
Added linear algebra least squares fitting using numpy, added plot of results against input delays.
-
Victor de Souza Magalhaes 37fe88b564777cd6f5f81f5c692992139ed347cc m
Reworked data dictionaries for efficient system matrix building and wrote system matrix builder that respects flags and polarization selection.
-
Victor de Souza Magalhaes ee77806b89f6e3249f4e9419b9f7ed0748f25050 m
Started work on flexible matrix builder for linear algebra solving.
-
Victor de Souza Magalhaes cebc14ceeaeab9438bf9ace5fc7836e1cd322797 m
Some reordering
-
Victor de Souza Magalhaes 79a55edfca6c1f01cf10df7fa76c31ef54273784 m
Implemented matching of elevations with delays, and fetching the polarization axis for the fringefit delays.
-
Victor de Souza Magalhaes 9a8db80a8bcef723dc8eb3de009591e1b87c4396 m
Isolated reading of antenna field and delay information.
-
Victor de Souza Magalhaes 848e2b4a520fb526cd88dec41d1e3ba681ac27cd m
caltable parameter type changed to "string" as "path" requires it to exist before execution.
-
Victor de Souza Magalhaes 1c27eb235234aa8940b444dd06a8acf324db8b26 m
Removed comments on astropy usage.
-
Victor de Souza Magalhaes 997bce9e11751f70f22d0b5e55ec212bcc5c023b m
zenithdelaycal can now be imported from the casatasks wheel without issues.
-
Victor de Souza Magalhaes c5923cd2ff5e7aa6bcce7138eb87612f42e9482e m
Added zenithdelaycal.xml to casatasks wheel compilation.
-
Victor de Souza Magalhaes e4653efcd2723bfbfd47636a979821eca6849429 m
progress on zenithdelaycal xml for proper inclusion in casatasks wheel.
-
Pedro Salas <salas.mz@gmail.com> c768e73d68bcd5b6d1a5de065ad9aaade9e635d5 m
Enable forcing clock1 to be offset from clock0
Adds force_clock_corr_offset option that forces clock1 to be
the same as clock0 with an offset. -
Pedro Salas <salas.mz@gmail.com> 44822b82999f868189936fd0f14004e3a41238a4 m
Fix missing key error for flagged data
The cost function would try to index data that was none
existent because of the crappy handling of the reference
antenna list. It now avoids things for which there's no
dictionary entry.
Also, run black on the code. -
Pedro Salas <salas.mz@gmail.com> fa4083d4c5d843b4a4ee47eeaae0a1face439117 m
Add output caltable
-
Pedro Salas <salas.mz@gmail.com> 42ecd44a8a4667fade84e91064b1197d74741296 m
Support for correlation products
Uses both correlation products when deriving solutions.
Each has its own clock term. It also removes antennas
that had all their solutions flagged. -
Pedro Salas <salas.mz@gmail.com> a1f674bc68f99233c61627f2a9ea5c7aebfc8e78 m
Use flags
Only unflagged data will be used to determine solutions -
Pedro Salas <salas.mz@gmail.com> 0262ff6aa3ecdb7f5cdd0eb663bc693580e1503e m
Preliminary zenithdelaycal implementation
tl/dr: this needs more work; polarization, flags, antenna parsing.
Implements the scipy-based least squares solver for the
zenith delay. It only handles one polarization, it will likely
raise an exception if the antenna parameter is a string, and it
ignores flags in the multi band delay solutions. -
Victor de Souza Magalhaes 203dd74d2a289accc5f00c53029758a365f1ac54 m
Created xml and python skeletons for zenith delay calibration