uid a002 xc46ab2 x15ae rep SPw spw16 17 small procedure hifa calimage regression: Test case result

The below summarizes the result of the test " uid a002 xc46ab2 x15ae rep SPw spw16 17 small procedure hifa calimage regression" in build 128 of Pipeline - Pipeline Main with Casa Master Test - Test Linux. View test case history

• Description: uid a002 xc46ab2 x15ae rep SPw spw16 17 small procedure hifa calimage regression
• Test class: pipeline.infrastructure.utils.regression-tester
• Method: test_uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small__procedure_hifa_calimage__regression

• Duration: 7 mins
• Status: Failed (Existing Failure)

Error Log

Failed: Failed to match 3 result values within tolerances :
s16.hifa_gfluxscale.uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small.field_0.spw_0.I
	values differ by > a relative difference of 1e-07
	expected: 2.2542400978094923
	new:      2.2542411194383027
s16.hifa_gfluxscale.uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small.field_2.spw_0.I
	values differ by > a relative difference of 1e-07
	expected: 0.7743004953241263
	new:      0.7742938056465042
s16.hifa_gfluxscale.uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small.qa.metric.score_derived_fluxes_snr
	values differ by > a relative difference of 1e-07
	expected: 412.94216427528795
	new:      412.9420931691852
def test_uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small__procedure_hifa_calimage__regression():
        """Run ALMA cal+image regression on a small test dataset.
    
        Recipe name:                procedure_hifa_calimage
        Dataset:                    uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small.ms
        Expected results version:   casa-6.1.1-15-pipeline-2020.1.0.40
        """
        pr = PipelineRegression(recipe='procedure_hifa_calimage.xml',
                                input_dir='pl-unittest',
                                visname='uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small.ms',
                                expectedoutput=('pl-regressiontest/uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small/' +
                                                'uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small.casa-6.1.1-15-pipeline-2020.1.0.40.results.txt'))
    
>       pr.run(ppr='pl-regressiontest/uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small/PPR.xml')

pipeline/infrastructure/utils/regression-tester.py:308: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pipeline/infrastructure/utils/regression-tester.py:156: in run
    self.__compare_results(new_file, default_relative_tolerance)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pipeline.infrastructure.utils.regression-tester.PipelineRegression object at 0x7f1d98137fd0>
new_file = 'uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small.ms.NEW.results.txt'
relative_tolerance = 1e-07

    def __compare_results(self, new_file: str, relative_tolerance: float):
        """
        Compare results between new one loaded from file and old one.
    
        Args:
            new_file : file path of new results
            relative_tolerance : relative tolerance of output value
        """
        expected = casa_tools.utils.resolve(self.expectedoutput)
        with open(expected) as expected_fd, open(new_file) as new_fd:
            expected_results = expected_fd.readlines()
            new_results = new_fd.readlines()
            errors = []
            for old, new in zip(expected_results, new_results):
                oldkey, oldval, tol = self.__sanitize_regression_string(old)
                newkey, newval, _ = self.__sanitize_regression_string(new)
                assert oldkey == newkey
                tolerance = tol if tol else relative_tolerance
                LOG.info(f'Comparing {oldval} to {newval} with a rel. tolerance of {tolerance}')
                if oldval != pytest.approx(newval, rel=tolerance):
                    errorstr = f"{oldkey}\n\tvalues differ by > a relative difference of {tolerance}\n\texpected: {oldval}\n\tnew:      {newval}"
                    errors.append(errorstr)
            [LOG.warning(x) for x in errors]
            n_errors = len(errors)
            if n_errors > 0:
>               pytest.fail("Failed to match {0} result value{1} within tolerance{1} :\n{2}".format(
                    n_errors, '' if n_errors == 1 else 's', '\n'.join(errors)), pytrace=True)
E               Failed: Failed to match 3 result values within tolerances :
E               s16.hifa_gfluxscale.uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small.field_0.spw_0.I
E               	values differ by > a relative difference of 1e-07
E               	expected: 2.2542400978094923
E               	new:      2.2542411194383027
E               s16.hifa_gfluxscale.uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small.field_2.spw_0.I
E               	values differ by > a relative difference of 1e-07
E               	expected: 0.7743004953241263
E               	new:      0.7742938056465042
E               s16.hifa_gfluxscale.uid___A002_Xc46ab2_X15ae_repSPW_spw16_17_small.qa.metric.score_derived_fluxes_snr
E               	values differ by > a relative difference of 1e-07
E               	expected: 412.94216427528795
E               	new:      412.9420931691852

pipeline/infrastructure/utils/regression-tester.py:198: Failed