Build: #126 failed
Job: Test Linux failed
13a 537 calibration PPr regression: Test case result
The below summarizes the result of the test " 13a 537 calibration PPr regression" in build 126 of Pipeline - Pipeline Main with Casa Master Test - Test Linux.
- Description
- 13a 537 calibration PPr regression
- Test class
- pipeline.infrastructure.utils.regression-tester
- Method
- test_13A_537__calibration__PPR__regression
- Duration
- 65 mins
- Status
- Failed (Existing Failure)
Error Log
Failed: Failed to match 23 result values within tolerances :
s10.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 14950026.0
new: 14950000.0
s10.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_4.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 11363942.0
new: 11363922.0
s10.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_8.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 3586084.0
new: 3586078.0
s12.hifv_fluxboot.13A-537.sb24066356.eb24324502.56514.05971091435.flux_densities.spw_0
values differ by > a relative difference of 1e-07
expected: 0.71492089895766
new: 0.7149359173078492
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 48505318.0
new: 48505304.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.before
values differ by > a relative difference of 1e-07
expected: 48273914.0
new: 48273900.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_2.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 7355528.0
new: 7355498.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_2.num_rows_flagged.before
values differ by > a relative difference of 1e-07
expected: 7355528.0
new: 7355498.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_3.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 6989744.0
new: 6989786.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_3.num_rows_flagged.before
values differ by > a relative difference of 1e-07
expected: 6989744.0
new: 6989786.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_4.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 11363942.0
new: 11363922.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_4.num_rows_flagged.before
values differ by > a relative difference of 1e-07
expected: 11363942.0
new: 11363922.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_8.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 3586084.0
new: 3586078.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_8.num_rows_flagged.before
values differ by > a relative difference of 1e-07
expected: 3586084.0
new: 3586078.0
s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 29766945.0
new: 29767273.0
s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_5.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 9369909.0
new: 9369969.0
s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_6.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 7386853.0
new: 7387052.0
s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_7.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 7978247.0
new: 7978316.0
s16.hifv_statwt.13A-537.sb24066356.eb24324502.56514.05971091435.mean
values differ by > a relative difference of 1e-07
expected: 1.6478917775250932
new: 1.6478185552120468
s16.hifv_statwt.13A-537.sb24066356.eb24324502.56514.05971091435.variance
values differ by > a relative difference of 1e-07
expected: 0.4846718015378145
new: 0.48461324837153896
s8.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 14345272.0
new: 14345284.0
s8.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_2.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 7355528.0
new: 7355498.0
s8.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_3.num_rows_flagged.after
values differ by > a relative difference of 1e-07
expected: 6989744.0
new: 6989786.0
def test_13A_537__calibration__PPR__regression():
"""Run VLA calibration regression with a PPR file.
PPR name: PPR_13A-537.xml
Dataset: 13A-537/13A-537.sb24066356.eb24324502.56514.05971091435
Expected results version: casa-6.2.1.7-pipeline-2021.2.0.128
"""
input_dir = 'pl-regressiontest/13A-537'
pr = PipelineRegression(input_dir=input_dir,
visname='13A-537.sb24066356.eb24324502.56514.05971091435',
expectedoutput=(f'{input_dir}/' +
'13A-537.casa-6.2.1.7-pipeline-2021.2.0.128.results.txt'),
output_dir='13A_537__calibration__PPR__regression')
> pr.run(ppr=f'{input_dir}/PPR_13A-537.xml', telescope='vla', omp_num_threads=1)
pipeline/infrastructure/utils/regression-tester.py:437:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pipeline/infrastructure/utils/regression-tester.py:156: in run
self.__compare_results(new_file, default_relative_tolerance)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pipeline.infrastructure.utils.regression-tester.PipelineRegression object at 0x7faa05e067c0>
new_file = '13A-537.sb24066356.eb24324502.56514.05971091435.NEW.results.txt'
relative_tolerance = 1e-07
def __compare_results(self, new_file: str, relative_tolerance: float):
"""
Compare results between new one loaded from file and old one.
Args:
new_file : file path of new results
relative_tolerance : relative tolerance of output value
"""
expected = casa_tools.utils.resolve(self.expectedoutput)
with open(expected) as expected_fd, open(new_file) as new_fd:
expected_results = expected_fd.readlines()
new_results = new_fd.readlines()
errors = []
for old, new in zip(expected_results, new_results):
oldkey, oldval, tol = self.__sanitize_regression_string(old)
newkey, newval, _ = self.__sanitize_regression_string(new)
assert oldkey == newkey
tolerance = tol if tol else relative_tolerance
LOG.info(f'Comparing {oldval} to {newval} with a rel. tolerance of {tolerance}')
if oldval != pytest.approx(newval, rel=tolerance):
errorstr = f"{oldkey}\n\tvalues differ by > a relative difference of {tolerance}\n\texpected: {oldval}\n\tnew: {newval}"
errors.append(errorstr)
[LOG.warning(x) for x in errors]
n_errors = len(errors)
if n_errors > 0:
> pytest.fail("Failed to match {0} result value{1} within tolerance{1} :\n{2}".format(
n_errors, '' if n_errors == 1 else 's', '\n'.join(errors)), pytrace=True)
E Failed: Failed to match 23 result values within tolerances :
E s10.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 14950026.0
E new: 14950000.0
E s10.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_4.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 11363942.0
E new: 11363922.0
E s10.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_8.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 3586084.0
E new: 3586078.0
E s12.hifv_fluxboot.13A-537.sb24066356.eb24324502.56514.05971091435.flux_densities.spw_0
E values differ by > a relative difference of 1e-07
E expected: 0.71492089895766
E new: 0.7149359173078492
E s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 48505318.0
E new: 48505304.0
E s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.before
E values differ by > a relative difference of 1e-07
E expected: 48273914.0
E new: 48273900.0
E s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_2.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 7355528.0
E new: 7355498.0
E s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_2.num_rows_flagged.before
E values differ by > a relative difference of 1e-07
E expected: 7355528.0
E new: 7355498.0
E s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_3.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 6989744.0
E new: 6989786.0
E s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_3.num_rows_flagged.before
E values differ by > a relative difference of 1e-07
E expected: 6989744.0
E new: 6989786.0
E s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_4.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 11363942.0
E new: 11363922.0
E s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_4.num_rows_flagged.before
E values differ by > a relative difference of 1e-07
E expected: 11363942.0
E new: 11363922.0
E s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_8.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 3586084.0
E new: 3586078.0
E s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_8.num_rows_flagged.before
E values differ by > a relative difference of 1e-07
E expected: 3586084.0
E new: 3586078.0
E s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 29766945.0
E new: 29767273.0
E s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_5.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 9369909.0
E new: 9369969.0
E s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_6.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 7386853.0
E new: 7387052.0
E s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_7.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 7978247.0
E new: 7978316.0
E s16.hifv_statwt.13A-537.sb24066356.eb24324502.56514.05971091435.mean
E values differ by > a relative difference of 1e-07
E expected: 1.6478917775250932
E new: 1.6478185552120468
E s16.hifv_statwt.13A-537.sb24066356.eb24324502.56514.05971091435.variance
E values differ by > a relative difference of 1e-07
E expected: 0.4846718015378145
E new: 0.48461324837153896
E s8.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 14345272.0
E new: 14345284.0
E s8.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_2.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 7355528.0
E new: 7355498.0
E s8.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_3.num_rows_flagged.after
E values differ by > a relative difference of 1e-07
E expected: 6989744.0
E new: 6989786.0
pipeline/infrastructure/utils/regression-tester.py:198: Failed