13a 537 calibration PPr regression: Test case result

The below summarizes the result of the test " 13a 537 calibration PPr regression" in build 126 of Pipeline - Pipeline Main with Casa Master Test - Test Linux.

• Description: 13a 537 calibration PPr regression
• Test class: pipeline.infrastructure.utils.regression-tester
• Method: test_13A_537__calibration__PPR__regression

• Duration: 65 mins
• Status: Failed (Existing Failure)

Error Log

Failed: Failed to match 23 result values within tolerances :
s10.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 14950026.0
	new:      14950000.0
s10.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_4.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 11363942.0
	new:      11363922.0
s10.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_8.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 3586084.0
	new:      3586078.0
s12.hifv_fluxboot.13A-537.sb24066356.eb24324502.56514.05971091435.flux_densities.spw_0
	values differ by > a relative difference of 1e-07
	expected: 0.71492089895766
	new:      0.7149359173078492
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 48505318.0
	new:      48505304.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.before
	values differ by > a relative difference of 1e-07
	expected: 48273914.0
	new:      48273900.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_2.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 7355528.0
	new:      7355498.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_2.num_rows_flagged.before
	values differ by > a relative difference of 1e-07
	expected: 7355528.0
	new:      7355498.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_3.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 6989744.0
	new:      6989786.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_3.num_rows_flagged.before
	values differ by > a relative difference of 1e-07
	expected: 6989744.0
	new:      6989786.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_4.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 11363942.0
	new:      11363922.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_4.num_rows_flagged.before
	values differ by > a relative difference of 1e-07
	expected: 11363942.0
	new:      11363922.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_8.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 3586084.0
	new:      3586078.0
s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_8.num_rows_flagged.before
	values differ by > a relative difference of 1e-07
	expected: 3586084.0
	new:      3586078.0
s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 29766945.0
	new:      29767273.0
s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_5.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 9369909.0
	new:      9369969.0
s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_6.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 7386853.0
	new:      7387052.0
s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_7.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 7978247.0
	new:      7978316.0
s16.hifv_statwt.13A-537.sb24066356.eb24324502.56514.05971091435.mean
	values differ by > a relative difference of 1e-07
	expected: 1.6478917775250932
	new:      1.6478185552120468
s16.hifv_statwt.13A-537.sb24066356.eb24324502.56514.05971091435.variance
	values differ by > a relative difference of 1e-07
	expected: 0.4846718015378145
	new:      0.48461324837153896
s8.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 14345272.0
	new:      14345284.0
s8.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_2.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 7355528.0
	new:      7355498.0
s8.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_3.num_rows_flagged.after
	values differ by > a relative difference of 1e-07
	expected: 6989744.0
	new:      6989786.0
def test_13A_537__calibration__PPR__regression():
        """Run VLA calibration regression with a PPR file.
    
        PPR name:                   PPR_13A-537.xml
        Dataset:                    13A-537/13A-537.sb24066356.eb24324502.56514.05971091435
        Expected results version:   casa-6.2.1.7-pipeline-2021.2.0.128
        """
    
        input_dir = 'pl-regressiontest/13A-537'
    
        pr = PipelineRegression(input_dir=input_dir,
                                visname='13A-537.sb24066356.eb24324502.56514.05971091435',
                                expectedoutput=(f'{input_dir}/' +
                                                '13A-537.casa-6.2.1.7-pipeline-2021.2.0.128.results.txt'),
                                output_dir='13A_537__calibration__PPR__regression')
    
>       pr.run(ppr=f'{input_dir}/PPR_13A-537.xml', telescope='vla', omp_num_threads=1)

pipeline/infrastructure/utils/regression-tester.py:437: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pipeline/infrastructure/utils/regression-tester.py:156: in run
    self.__compare_results(new_file, default_relative_tolerance)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pipeline.infrastructure.utils.regression-tester.PipelineRegression object at 0x7faa05e067c0>
new_file = '13A-537.sb24066356.eb24324502.56514.05971091435.NEW.results.txt'
relative_tolerance = 1e-07

    def __compare_results(self, new_file: str, relative_tolerance: float):
        """
        Compare results between new one loaded from file and old one.
    
        Args:
            new_file : file path of new results
            relative_tolerance : relative tolerance of output value
        """
        expected = casa_tools.utils.resolve(self.expectedoutput)
        with open(expected) as expected_fd, open(new_file) as new_fd:
            expected_results = expected_fd.readlines()
            new_results = new_fd.readlines()
            errors = []
            for old, new in zip(expected_results, new_results):
                oldkey, oldval, tol = self.__sanitize_regression_string(old)
                newkey, newval, _ = self.__sanitize_regression_string(new)
                assert oldkey == newkey
                tolerance = tol if tol else relative_tolerance
                LOG.info(f'Comparing {oldval} to {newval} with a rel. tolerance of {tolerance}')
                if oldval != pytest.approx(newval, rel=tolerance):
                    errorstr = f"{oldkey}\n\tvalues differ by > a relative difference of {tolerance}\n\texpected: {oldval}\n\tnew:      {newval}"
                    errors.append(errorstr)
            [LOG.warning(x) for x in errors]
            n_errors = len(errors)
            if n_errors > 0:
>               pytest.fail("Failed to match {0} result value{1} within tolerance{1} :\n{2}".format(
                    n_errors, '' if n_errors == 1 else 's', '\n'.join(errors)), pytrace=True)
E               Failed: Failed to match 23 result values within tolerances :
E               s10.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 14950026.0
E               	new:      14950000.0
E               s10.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_4.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 11363942.0
E               	new:      11363922.0
E               s10.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_8.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 3586084.0
E               	new:      3586078.0
E               s12.hifv_fluxboot.13A-537.sb24066356.eb24324502.56514.05971091435.flux_densities.spw_0
E               	values differ by > a relative difference of 1e-07
E               	expected: 0.71492089895766
E               	new:      0.7149359173078492
E               s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 48505318.0
E               	new:      48505304.0
E               s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.before
E               	values differ by > a relative difference of 1e-07
E               	expected: 48273914.0
E               	new:      48273900.0
E               s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_2.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 7355528.0
E               	new:      7355498.0
E               s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_2.num_rows_flagged.before
E               	values differ by > a relative difference of 1e-07
E               	expected: 7355528.0
E               	new:      7355498.0
E               s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_3.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 6989744.0
E               	new:      6989786.0
E               s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_3.num_rows_flagged.before
E               	values differ by > a relative difference of 1e-07
E               	expected: 6989744.0
E               	new:      6989786.0
E               s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_4.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 11363942.0
E               	new:      11363922.0
E               s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_4.num_rows_flagged.before
E               	values differ by > a relative difference of 1e-07
E               	expected: 11363942.0
E               	new:      11363922.0
E               s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_8.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 3586084.0
E               	new:      3586078.0
E               s14.hifv_applycals.13A-537.sb24066356.eb24324502.56514.05971091435.scan_8.num_rows_flagged.before
E               	values differ by > a relative difference of 1e-07
E               	expected: 3586084.0
E               	new:      3586078.0
E               s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 29766945.0
E               	new:      29767273.0
E               s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_5.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 9369909.0
E               	new:      9369969.0
E               s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_6.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 7386853.0
E               	new:      7387052.0
E               s15.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_7.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 7978247.0
E               	new:      7978316.0
E               s16.hifv_statwt.13A-537.sb24066356.eb24324502.56514.05971091435.mean
E               	values differ by > a relative difference of 1e-07
E               	expected: 1.6478917775250932
E               	new:      1.6478185552120468
E               s16.hifv_statwt.13A-537.sb24066356.eb24324502.56514.05971091435.variance
E               	values differ by > a relative difference of 1e-07
E               	expected: 0.4846718015378145
E               	new:      0.48461324837153896
E               s8.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 14345272.0
E               	new:      14345284.0
E               s8.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_2.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 7355528.0
E               	new:      7355498.0
E               s8.hifv_checkflag.13A-537.sb24066356.eb24324502.56514.05971091435.scan_3.num_rows_flagged.after
E               	values differ by > a relative difference of 1e-07
E               	expected: 6989744.0
E               	new:      6989786.0

pipeline/infrastructure/utils/regression-tester.py:198: Failed