Build: #65 failed
Job: Test ManyLinux 2.28 Python 3.12 failed
2022 1 00207 s uid a001 x2d20 x373d PPr regression: Test case result
The below summarizes the result of the test " 2022 1 00207 s uid a001 x2d20 x373d PPr regression" in build 65 of PIPESPECS - Test Pipeline main with Casa master - Test ManyLinux 2.28 Python 3.12.
- Description
- 2022 1 00207 s uid a001 x2d20 x373d PPr regression
- Test class
- tests.regression.fast.alma_if_fast_test
- Method
- test_2022_1_00207_S__uid___A001_X2d20_X373d__PPR__regression
- Duration
- 485 mins
- Status
- Failed (Existing Failure)
Error Log
Failed: Failed to match 29 result values within tolerances :
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_0.spw_16.qa.metric.score_gfluxscale_k_spw
values differ by > a relative difference of 1e-07
expected: 0.9998218986723444
new: 0.9998227851376125
diff: -8.864652680795615e-07
percent_diff: -8.866231768444877e-05%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_0.spw_18.I
values differ by > a relative difference of 1e-07
expected: 1.3653809717912258
new: 1.3653796551526987
diff: 1.3166385270935166e-06
percent_diff: 9.64301212844819e-05%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_0.spw_20.I
values differ by > a relative difference of 1e-07
expected: 1.313188383648403
new: 1.3131887675624525
diff: -3.8391404943993734e-07
percent_diff: -2.9235260852164803e-05%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_0.spw_20.qa.metric.score_gfluxscale_k_spw
values differ by > a relative difference of 1e-07
expected: 1.0019692397005648
new: 1.001970498830253
diff: -1.2591296880692937e-06
percent_diff: -0.00012566550330882217%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_0.spw_22.I
values differ by > a relative difference of 1e-07
expected: 1.3102862582551928
new: 1.3102863988307627
diff: -1.4057556985136443e-07
percent_diff: -1.072861513777593e-05%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_0.spw_22.qa.metric.score_gfluxscale_k_spw
values differ by > a relative difference of 1e-07
expected: 1.0054226609922603
new: 1.0054237383915183
diff: -1.0773992580315195e-06
percent_diff: -0.00010715883974290224%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_2.spw_16.I
values differ by > a relative difference of 1e-07
expected: 0.35676380919541717
new: 0.3567643335721676
diff: -5.243767504103936e-07
percent_diff: -0.00014698148660117216%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_2.spw_16.qa.metric.score_gfluxscale_k_spw
values differ by > a relative difference of 1e-07
expected: 1.0030947541027135
new: 1.0030954559681338
diff: -7.018654202273211e-07
percent_diff: -6.997000207175368e-05%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_2.spw_18.I
values differ by > a relative difference of 1e-07
expected: 0.3542420210295326
new: 0.354242293836381
diff: -2.7280684838926206e-07
percent_diff: -7.701143065873559e-05%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_2.spw_20.I
values differ by > a relative difference of 1e-07
expected: 0.3438184465624654
new: 0.34382060584345014
diff: -2.1592809847370553e-06
percent_diff: -0.0006280294167825443%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_2.spw_20.qa.metric.score_gfluxscale_k_spw
values differ by > a relative difference of 1e-07
expected: 0.9972084475163907
new: 0.997213942310064
diff: -5.494793673288534e-06
percent_diff: -0.0005510175617719302%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_2.spw_22.I
values differ by > a relative difference of 1e-07
expected: 0.3424346420160027
new: 0.34243473636467536
diff: -9.434867265234459e-08
percent_diff: -2.7552315413209704e-05%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_2.spw_22.qa.metric.score_gfluxscale_k_spw
values differ by > a relative difference of 1e-07
expected: 0.9969142110040273
new: 0.9969137179394586
diff: 4.93064568729551e-07
percent_diff: 4.945907714897236e-05%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.qa.metric.score_derived_fluxes_snr
values differ by > a relative difference of 1e-07
expected: 260.41168489992793
new: 260.41175375126136
diff: -6.885133342393601e-05
percent_diff: -2.643941782043862e-05%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_1.spw_10.I
values differ by > a relative difference of 1e-07
expected: 1.3100103840087651
new: 1.3100067493136076
diff: 3.6346951575172426e-06
percent_diff: 0.0002774554462991893%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_1.spw_10.qa.metric.score_gfluxscale_k_spw
values differ by > a relative difference of 1e-07
expected: 1.0076214074604941
new: 1.0076153277561768
diff: 6.079704317318857e-06
percent_diff: 0.0006033718887177598%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_1.spw_4.I
values differ by > a relative difference of 1e-07
expected: 1.3751936458839125
new: 1.3751907513694928
diff: 2.8945144197045636e-06
percent_diff: 0.00021048049693714953%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_1.spw_4.qa.metric.score_gfluxscale_k_spw
values differ by > a relative difference of 1e-07
expected: 1.0035071238173334
new: 1.0035017410336444
diff: 5.382783689000803e-06
percent_diff: 0.0005363971576529257%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_1.spw_6.I
values differ by > a relative difference of 1e-07
expected: 1.3621147057074792
new: 1.3621191450900558
diff: -4.439382576659767e-06
percent_diff: -0.00032591840893120397%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_1.spw_8.I
values differ by > a relative difference of 1e-07
expected: 1.3182131439092004
new: 1.3182179334732627
diff: -4.789564062246399e-06
percent_diff: -0.0003633376047247415%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_1.spw_8.qa.metric.score_gfluxscale_k_spw
values differ by > a relative difference of 1e-07
expected: 1.0082150098813702
new: 1.0082153871460893
diff: -3.7726471902033154e-07
percent_diff: -3.741907384068024e-05%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_10.I
values differ by > a relative difference of 1e-07
expected: 0.35352838264456693
new: 0.35353247318978354
diff: -4.090545216606234e-06
percent_diff: -0.0011570627472699464%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_10.qa.metric.score_gfluxscale_k_spw
values differ by > a relative difference of 1e-07
expected: 1.0129126775618893
new: 1.0129262614066323
diff: -1.3583844743036977e-05
percent_diff: -0.0013410676995112443%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_4.I
values differ by > a relative difference of 1e-07
expected: 0.3634560120039281
new: 0.3634562609375893
diff: -2.489336612399029e-07
percent_diff: -6.849072598012563e-05%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_4.qa.metric.score_gfluxscale_k_spw
values differ by > a relative difference of 1e-07
expected: 1.0057282529903182
new: 1.0057307923905467
diff: -2.5394002285406003e-06
percent_diff: -0.0002524936752040361%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_6.I
values differ by > a relative difference of 1e-07
expected: 0.3599419377377514
new: 0.3599412754356427
diff: 6.623021087115433e-07
percent_diff: 0.0001840024846435336%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_8.I
values differ by > a relative difference of 1e-07
expected: 0.352711869536587
new: 0.3527122673387697
diff: -3.978021826944733e-07
percent_diff: -0.0001127838944623918%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_8.qa.metric.score_gfluxscale_k_spw
values differ by > a relative difference of 1e-07
expected: 1.0068029434447139
new: 1.0068059315042126
diff: -2.9880594987208298e-06
percent_diff: -0.00029678692520478426%
s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.qa.metric.score_derived_fluxes_snr
values differ by > a relative difference of 1e-07
expected: 257.1270701945043
new: 257.1271593408154
diff: -8.914631109746551e-05
percent_diff: -3.4670138398897714e-05%
Worst absolute diff, s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.qa.metric.score_derived_fluxes_snr: -8.914631109746551e-05
Worst percentage diff, s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_10.qa.metric.score_gfluxscale_k_spw: -0.0013410676995112443%
@pytest.mark.seven
@pytest.mark.mpi
def test_2022_1_00207_S__uid___A001_X2d20_X373d__PPR__regression():
"""Run ALMA polcal+image regression on a multi-EB 7m test dataset with a PPR file.
PPR: pl-regressiontest/2022.1.00207.S/PPR.xml
Project: 2022.1.00207.S
MOUS: uid___A001_X2d20_X373d
EBs: uid___A002_X10b6f7c_X41d1
uid___A002_X10b6f7c_X46cc
"""
ref_directory = 'pl-regressiontest/2022.1.00207.S'
pt = PipelineTester(
visname=['uid___A002_X10b6f7c_X41d1',
'uid___A002_X10b6f7c_X46cc'],
ppr=f"{ref_directory}/PPR.xml",
input_dir=ref_directory,
expectedoutput_dir=ref_directory,
)
> pt.run()
tests/regression/fast/alma_if_fast_test.py:123:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/testing_utils.py:376: in run
self.__compare_results(new_file, default_relative_tolerance)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <tests.testing_utils.PipelineTester object at 0x7f22307e6510>
new_file = 'uid___A002_X10b6f7c_X41d1.NEW.results.txt'
relative_tolerance = 1e-07
def __compare_results(self, new_file: str, relative_tolerance: float) -> None:
"""
Compare results between new one loaded from file and old one.
Args:
new_file : file path of new results
relative_tolerance : relative tolerance of output value
"""
with open(self.expectedoutput_file) as expected_fd, open(new_file) as new_fd:
expected_results = expected_fd.readlines()
new_results = new_fd.readlines()
errors = []
worst_diff = (0, 0)
worst_percent_diff = (0, 0)
for old, new in zip(expected_results, new_results):
try:
oldkey, oldval, tol = self.__sanitize_results_string(old)
newkey, newval, _ = self.__sanitize_results_string(new)
except ValueError as e:
errorstr = "The results: {0} could not be parsed. Error: {1}".format(new, str(e))
errors.append(errorstr)
continue
assert oldkey == newkey, f"Expected key {oldkey} does not match new key {newkey}."
tolerance = tol if tol else relative_tolerance
if newval is not None:
LOG.info('Comparing %s to %s with a rel. tolerance of %s', oldval, newval, tolerance)
if oldval != pytest.approx(newval, rel=tolerance):
diff = oldval-newval
percent_diff = (oldval-newval)/oldval * 100 if oldval != 0 else 100
if abs(diff) > abs(worst_diff[0]):
worst_diff = diff, oldkey
if abs(percent_diff) > abs(worst_percent_diff[0]):
worst_percent_diff = percent_diff, oldkey
errorstr = f"{oldkey}\n\tvalues differ by > a relative difference of {tolerance}\n\texpected: {oldval}\n\tnew: {newval}\n\tdiff: {diff}\n\tpercent_diff: {percent_diff}%"
errors.append(errorstr)
elif oldval is not None:
# If only the new value is None, fail
errorstr = f"{oldkey}\n\tvalue is None\n\texpected: {oldval}\n\tnew: {newval}"
errors.append(errorstr)
else:
# If old and new values are both None, this is expected, so pass
LOG.info('Comparing %s and %s... both values are None.', oldval, newval)
[LOG.warning(x) for x in errors]
n_errors = len(errors)
if n_errors > 0:
summary_str = f"Worst absolute diff, {worst_diff[1]}: {worst_diff[0]}\nWorst percentage diff, {worst_percent_diff[1]}: {worst_percent_diff[0]}%"
errors.append(summary_str)
> pytest.fail("Failed to match {0} result value{1} within tolerance{1} :\n{2}".format(
n_errors, '' if n_errors == 1 else 's', '\n'.join(errors)), pytrace=True)
E Failed: Failed to match 29 result values within tolerances :
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_0.spw_16.qa.metric.score_gfluxscale_k_spw
E values differ by > a relative difference of 1e-07
E expected: 0.9998218986723444
E new: 0.9998227851376125
E diff: -8.864652680795615e-07
E percent_diff: -8.866231768444877e-05%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_0.spw_18.I
E values differ by > a relative difference of 1e-07
E expected: 1.3653809717912258
E new: 1.3653796551526987
E diff: 1.3166385270935166e-06
E percent_diff: 9.64301212844819e-05%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_0.spw_20.I
E values differ by > a relative difference of 1e-07
E expected: 1.313188383648403
E new: 1.3131887675624525
E diff: -3.8391404943993734e-07
E percent_diff: -2.9235260852164803e-05%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_0.spw_20.qa.metric.score_gfluxscale_k_spw
E values differ by > a relative difference of 1e-07
E expected: 1.0019692397005648
E new: 1.001970498830253
E diff: -1.2591296880692937e-06
E percent_diff: -0.00012566550330882217%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_0.spw_22.I
E values differ by > a relative difference of 1e-07
E expected: 1.3102862582551928
E new: 1.3102863988307627
E diff: -1.4057556985136443e-07
E percent_diff: -1.072861513777593e-05%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_0.spw_22.qa.metric.score_gfluxscale_k_spw
E values differ by > a relative difference of 1e-07
E expected: 1.0054226609922603
E new: 1.0054237383915183
E diff: -1.0773992580315195e-06
E percent_diff: -0.00010715883974290224%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_2.spw_16.I
E values differ by > a relative difference of 1e-07
E expected: 0.35676380919541717
E new: 0.3567643335721676
E diff: -5.243767504103936e-07
E percent_diff: -0.00014698148660117216%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_2.spw_16.qa.metric.score_gfluxscale_k_spw
E values differ by > a relative difference of 1e-07
E expected: 1.0030947541027135
E new: 1.0030954559681338
E diff: -7.018654202273211e-07
E percent_diff: -6.997000207175368e-05%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_2.spw_18.I
E values differ by > a relative difference of 1e-07
E expected: 0.3542420210295326
E new: 0.354242293836381
E diff: -2.7280684838926206e-07
E percent_diff: -7.701143065873559e-05%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_2.spw_20.I
E values differ by > a relative difference of 1e-07
E expected: 0.3438184465624654
E new: 0.34382060584345014
E diff: -2.1592809847370553e-06
E percent_diff: -0.0006280294167825443%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_2.spw_20.qa.metric.score_gfluxscale_k_spw
E values differ by > a relative difference of 1e-07
E expected: 0.9972084475163907
E new: 0.997213942310064
E diff: -5.494793673288534e-06
E percent_diff: -0.0005510175617719302%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_2.spw_22.I
E values differ by > a relative difference of 1e-07
E expected: 0.3424346420160027
E new: 0.34243473636467536
E diff: -9.434867265234459e-08
E percent_diff: -2.7552315413209704e-05%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.field_2.spw_22.qa.metric.score_gfluxscale_k_spw
E values differ by > a relative difference of 1e-07
E expected: 0.9969142110040273
E new: 0.9969137179394586
E diff: 4.93064568729551e-07
E percent_diff: 4.945907714897236e-05%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X41d1.qa.metric.score_derived_fluxes_snr
E values differ by > a relative difference of 1e-07
E expected: 260.41168489992793
E new: 260.41175375126136
E diff: -6.885133342393601e-05
E percent_diff: -2.643941782043862e-05%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_1.spw_10.I
E values differ by > a relative difference of 1e-07
E expected: 1.3100103840087651
E new: 1.3100067493136076
E diff: 3.6346951575172426e-06
E percent_diff: 0.0002774554462991893%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_1.spw_10.qa.metric.score_gfluxscale_k_spw
E values differ by > a relative difference of 1e-07
E expected: 1.0076214074604941
E new: 1.0076153277561768
E diff: 6.079704317318857e-06
E percent_diff: 0.0006033718887177598%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_1.spw_4.I
E values differ by > a relative difference of 1e-07
E expected: 1.3751936458839125
E new: 1.3751907513694928
E diff: 2.8945144197045636e-06
E percent_diff: 0.00021048049693714953%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_1.spw_4.qa.metric.score_gfluxscale_k_spw
E values differ by > a relative difference of 1e-07
E expected: 1.0035071238173334
E new: 1.0035017410336444
E diff: 5.382783689000803e-06
E percent_diff: 0.0005363971576529257%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_1.spw_6.I
E values differ by > a relative difference of 1e-07
E expected: 1.3621147057074792
E new: 1.3621191450900558
E diff: -4.439382576659767e-06
E percent_diff: -0.00032591840893120397%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_1.spw_8.I
E values differ by > a relative difference of 1e-07
E expected: 1.3182131439092004
E new: 1.3182179334732627
E diff: -4.789564062246399e-06
E percent_diff: -0.0003633376047247415%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_1.spw_8.qa.metric.score_gfluxscale_k_spw
E values differ by > a relative difference of 1e-07
E expected: 1.0082150098813702
E new: 1.0082153871460893
E diff: -3.7726471902033154e-07
E percent_diff: -3.741907384068024e-05%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_10.I
E values differ by > a relative difference of 1e-07
E expected: 0.35352838264456693
E new: 0.35353247318978354
E diff: -4.090545216606234e-06
E percent_diff: -0.0011570627472699464%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_10.qa.metric.score_gfluxscale_k_spw
E values differ by > a relative difference of 1e-07
E expected: 1.0129126775618893
E new: 1.0129262614066323
E diff: -1.3583844743036977e-05
E percent_diff: -0.0013410676995112443%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_4.I
E values differ by > a relative difference of 1e-07
E expected: 0.3634560120039281
E new: 0.3634562609375893
E diff: -2.489336612399029e-07
E percent_diff: -6.849072598012563e-05%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_4.qa.metric.score_gfluxscale_k_spw
E values differ by > a relative difference of 1e-07
E expected: 1.0057282529903182
E new: 1.0057307923905467
E diff: -2.5394002285406003e-06
E percent_diff: -0.0002524936752040361%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_6.I
E values differ by > a relative difference of 1e-07
E expected: 0.3599419377377514
E new: 0.3599412754356427
E diff: 6.623021087115433e-07
E percent_diff: 0.0001840024846435336%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_8.I
E values differ by > a relative difference of 1e-07
E expected: 0.352711869536587
E new: 0.3527122673387697
E diff: -3.978021826944733e-07
E percent_diff: -0.0001127838944623918%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_8.qa.metric.score_gfluxscale_k_spw
E values differ by > a relative difference of 1e-07
E expected: 1.0068029434447139
E new: 1.0068059315042126
E diff: -2.9880594987208298e-06
E percent_diff: -0.00029678692520478426%
E s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.qa.metric.score_derived_fluxes_snr
E values differ by > a relative difference of 1e-07
E expected: 257.1270701945043
E new: 257.1271593408154
E diff: -8.914631109746551e-05
E percent_diff: -3.4670138398897714e-05%
E Worst absolute diff, s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.qa.metric.score_derived_fluxes_snr: -8.914631109746551e-05
E Worst percentage diff, s21.hifa_gfluxscale.uid___A002_X10b6f7c_X46cc.field_3.spw_10.qa.metric.score_gfluxscale_k_spw: -0.0013410676995112443%
tests/testing_utils.py:435: Failed