Build #47

Plan branch:

Build: #47 failed Child of CASA-CMTC-82

Logs

The following logs have been generated by the jobs in this plan.
  • Expand all
  • Collapse all
Job Logs
Collapse ALMA Stakeholder Tests MPI EL8 Default Stage Download or View
15-May-2024 16:53:35   File "/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/pkg/casa-6.6.4-27-py3.8.el8/lib/py/lib/python3.8/site-packages/casatestutils/__init__.py", line 280, in function_timer
15-May-2024 16:53:35     result = function(*args, **kwargs)
15-May-2024 16:53:35   File "/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/pkg/casa-6.6.4-27-py3.8.el8/lib/py/lib/python3.8/site-packages/casatestutils/__init__.py", line 378, in function_mem
15-May-2024 16:53:35     result = function(*args, **kwargs)
15-May-2024 16:53:35   File "/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/pkg/casa-6.6.4-27-py3.8.el8/lib/py/lib/python3.8/site-packages/casatestutils/__init__.py", line 395, in all_wrapped
15-May-2024 16:53:35     return function(*args, **kwargs)
15-May-2024 16:53:35   File "/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/casasources//casa6/casatests/stakeholder/test_stk_alma_pipeline_imaging.py", line 2465, in test_standard_mtmfs_eph
15-May-2024 16:53:35     self.assertTrue(th.check_final(pstr = report), \
15-May-2024 16:53:35 AssertionError: False is not true :
15-May-2024 16:53:50 [ check_ims ] Image made : ['/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.alpha', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.alpha.error', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.alpha.pbcor', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.psf.tt0', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.psf.tt1', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.psf.tt2', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.residual.tt0', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.residual.tt1', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.image.tt0', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.image.tt1', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.image.tt0.pbcor', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.image.tt1.pbcor', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.mask', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.pb.tt0', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.model.tt0', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.model.tt1', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.sumwt.tt0', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.sumwt.tt1', '/export/cvpost135-nvme/casa/bamboohome/xml-data/build-dir/CASA-CMTS-ASTME/work/test_stk_alma_pipeline_imaging/standard_mtmfs_eph.iter1.sumwt.tt2'] =  [True, True, False, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True, True] ( Fail : should all be True )
15-May-2024 16:53:50 1 individual test failure(s)
15-May-2024 16:53:50
15-May-2024 16:53:50 ----------------------------------------------------------------------
15-May-2024 16:53:50 Ran 22 tests in 4295.136s
15-May-2024 16:53:50
15-May-2024 16:53:50 FAILED (failures=4)
15-May-2024 16:53:50 CASA exits with a non-zero status : system exit called
15-May-2024 16:53:50 Executor result: 1
15-May-2024 16:53:50 Failed tests:['test_mosaic_mtmfs', 'test_mosaic_mtmfs_eph', 'test_standard_mtmfs', 'test_standard_mtmfs_eph', 'Executor returned a non-zero exit code']
15-May-2024 16:53:50
15-May-2024 16:53:50 Results: [{'testname': 'test_stk_alma_pipeline_imaging', 'runtime': '1:11:51.959247', 'returncode': 5, 'testerr': ['test_mosaic_mtmfs', 'test_mosaic_mtmfs_eph', 'test_standard_mtmfs', 'test_standard_mtmfs_eph', 'Executor returned a non-zero exit code']}]
15-May-2024 16:53:50 Stopping Xvfb.
15-May-2024 16:53:50 Stopping framebuffer <subprocess.Popen object at 0x7f4872897c88>
15-May-2024 16:53:50 Virtual frame buffer deployed at :2971361 with pid 2971422 successfully shutdown
15-May-2024 16:53:50 Xvfb stopped.
Collapse VLA Stakeholder Tests MPI EL8 Docker Default Stage Download or View
15-May-2024 17:31:25
15-May-2024 17:31:25    Process name: [prterun-697bcc76c55d-211@1,4] Exit code:    1
15-May-2024 17:31:25 --------------------------------------------------------------------------
15-May-2024 17:31:25
15-May-2024 17:31:25 Executor result: 1
15-May-2024 17:31:25 Failed tests:['Executor returned a non-zero exit code']
15-May-2024 17:31:25
15-May-2024 17:31:25 Results: [{'testname': 'test_stk_vla_users_continuum_from_SDM', 'runtime': '0:00:00.149171', 'returncode': 1, 'testerr': ['Executor returned a non-zero exit code']}]
15-May-2024 17:31:25 Stopping Xvfb.
15-May-2024 17:31:25 Stopping framebuffer <Popen: returncode: None args: ['Xvfb', ':87', '-screen', '0', '2048x2048x24...>
15-May-2024 17:31:25 Virtual frame buffer deployed at :87 with pid 100 successfully shutdown
15-May-2024 17:31:25 Xvfb stopped.
15-May-2024 17:31:25 cp: cannot stat '/home/casatest/work/test_stk_vla_users_continuum_from_SDM/test_vla_users_continuum_from_sdm_weblog.html': No such file or directory
15-May-2024 17:31:25 cp: cannot stat '/home/casatest/work/test_stk_vla_users_continuum_from_SDM/*.png': No such file or directory
15-May-2024 17:31:26 total 1011804
15-May-2024 17:31:26 -rw-r--r-- 1 casaci    15001 1036080100 May 15 17:29 casa-6.6.4-27-py3.8.el8.tar.xz
15-May-2024 17:31:26 drwxr-xr-x 2 casaci root              6 May 15 17:31 html
15-May-2024 17:31:26 -rw-r--r-- 1 casaci casatest        411 May 15 17:31 xUnit-suite-mpi-8.xml
15-May-2024 17:31:26 total 0
15-May-2024 17:31:26 Starting cleanup
15-May-2024 17:31:36 test-container
15-May-2024 17:31:41 test-container
15-May-2024 17:31:41 Cleanup complete
15-May-2024 17:31:41 Failing as no matching files has been found and empty artifacts are not allowed.
15-May-2024 17:31:41 Unable to publish artifact [VLA Stakeholder MPI]:
Collapse ALMA Stakeholder Tests MPI EL7 Default Stage Download or View
15-May-2024 14:41:32 2024-05-15 20:41:27        INFO        tclean::::casa        setupCluster, Setting up 7 engines.
15-May-2024 14:41:32 2024-05-15 20:41:28        INFO        SynthesisUtilMethods::continuumDataPartition         
15-May-2024 14:41:32 2024-05-15 20:41:28        INFO        SynthesisUtilMethods::continuumDataPartition +        Rows = 37069 43240 [P][M]: 0:6 ROWNUMBER() >= 37069 && ROWNUMBER() <= 43240
15-May-2024 14:41:32 2024-05-15 20:41:28        INFO        tclean::::casa        Partitioned Selection : {'0': {'ms0': {'antenna': '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46', 'datacolumn': 'data', 'field': '0', 'freqbeg': '', 'freqend': '', 'freqframe': 'LSRK', 'incrmodel': False, 'msname': '/lustre/cv/sw/casa/casaci/data/casatestdata-large/stakeholder/alma/2018.1.00879.S_tclean.ms', 'obs': '', 'outframe': 'LSRK', 'readonly': True, 'scan': '7,11', 'spw': '0:245.220516619~245.273983416GHz,1:261.752937691~261.774177925GHz;261.783699409~261.837898628GHz;261.958504097~261.984871284GHz', 'state': 'OBSERVE_TARGET#ON_SOURCE', 'taql': 'ROWNUMBER() >= 1 && ROWNUMBER() <= 6178', 'timestr': '', 'usescratch': False, 'uvdist': ''}}, '1': {'ms0': {'antenna': '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46', 'datacolumn': 'data', 'field': '0', 'freqbeg': '', 'freqend': '', 'freqframe': 'LSRK', 'incrmodel': False, 'msname': '/lustre/cv/sw/casa/casaci/data/casatestdata-large/stakeholder/alma/2018.1.00879.S_tclean.ms', 'obs': '', 'outframe': 'LSRK', 'readonly': True, 'scan': '7,11', 'spw': '0:245.220516619~245.273983416GHz,1:261.752937691~261.774177925GHz;261.783699409~261.837898628GHz;261.958504097~261.984871284GHz', 'state': 'OBSERVE_TARGET#ON_SOURCE', 'taql': 'ROWNUMBER() >= 6179 && ROWNUMBER() <= 12356', 'timestr': '', 'usescratch': False, 'uvdist': ''}}, '2': {'ms0': {'antenna': '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46', 'datacolumn': 'data', 'field': '0', 'freqbeg': '', 'freqend': '', 'freqframe': 'LSRK', 'incrmodel': False, 'msname': '/lustre/cv/sw/casa/casaci/data/casatestdata-large/stakeholder/alma/2018.1.00879.S_tclean.ms', 'obs': '', 'outframe': 'LSRK', 'readonly': True, 'scan': '7,11', 'spw': '0:245.220516619~245.273983416GHz,1:261.752937691~261.774177925GHz;261.783699409~261.837898628GHz;261.958504097~261.984871284GHz', 'state': 'OBSERVE_TARGET#ON_SOURCE', 'taql': 'ROWNUMBER() >= 12357 && ROWNUMBER() <= 18534', 'timestr': '', 'usescratch': False, 'uvdist': ''}}, '3': {'ms0': {'antenna': '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46', 'datacolumn': 'data', 'field': '0', 'freqbeg': '', 'freqend': '', 'freqframe': 'LSRK', 'incrmodel': False, 'msname': '/lustre/cv/sw/casa/casaci/data/casatestdata-large/stakeholder/alma/2018.1.00879.S_tclean.ms', 'obs': '', 'outframe': 'LSRK', 'readonly': True, 'scan': '7,11', 'spw': '0:245.220516619~245.273983416GHz,1:261.752937691~261.774177925GHz;261.783699409~261.837898628GHz;261.958504097~261.984871284GHz', 'state': 'OBSERVE_TARGET#ON_SOURCE', 'taql': 'ROWNUMBER() >= 18535 && ROWNUMBER() <= 24712', 'timestr': '', 'usescratch': False, 'uvdist': ''}}, '4': {'ms0': {'antenna': '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46', 'datacolumn': 'data', 'field': '0', 'freqbeg': '', 'freqend': '', 'freqframe': 'LSRK', 'incrmodel': False, 'msname': '/lustre/cv/sw/casa/casaci/data/casatestdata-large/stakeholder/alma/2018.1.00879.S_tclean.ms', 'obs': '', 'outframe': 'LSRK', 'readonly': True, 'scan': '7,11', 'spw': '0:245.220516619~245.273983416GHz,1:261.752937691~261.774177925GHz;261.783699409~261.837898628GHz;261.958504097~261.984871284GHz', 'state': 'OBSERVE_TARGET#ON_SOURCE', 'taql': 'ROWNUMBER() >= 24713 && ROWNUMBER() <= 30890', 'timestr': '', 'usescratch': False, 'uvdist': ''}}, '5': {'ms0': {'antenna': '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46', 'datacolumn': 'data', 'field': '0', 'freqbeg': '', 'freqend': '', 'freqframe': 'LSRK', 'incrmodel': False, 'msname': '/lustre/cv/sw/casa/casaci/data/casatestdata-large/stakeholder/alma/2018.1.00879.S_tclean.ms', 'obs': '', 'outframe': 'LSRK', 'readonly': True, 'scan': '7,11', 'spw': '0:245.220516619~245.273983416GHz,1:261.752937691~261.774177925GHz;261.783699409~261.837898628GHz;261.958504097~261.984871284GHz', 'state': 'OBSERVE_TARGET#ON_SOURCE', 'taql': 'ROWNUMBER() >= 30891 && ROWNUMBER() <= 37068', 'timestr': '', 'usescratch': False, 'uvdist': ''}}, '6': {'ms0': {'antenna': '0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46', 'datacolumn': 'data', 'field': '0', 'freqbeg': '', 'freqend': '', 'freqframe': 'LSRK', 'incrmodel': False, 'msname': '/lustre/cv/sw/casa/casaci/data/casatestdata-large/stakeholder/alma/2018.1.00879.S_tclean.ms', 'obs': '', 'outframe': 'LSRK', 'readonly': True, 'scan': '7,11', 'spw': '0:245.220516619~245.273983416GHz,1:261.752937691~261.774177925GHz;261.783699409~261.837898628GHz;261.958504097~261.984871284GHz', 'state': 'OBSERVE_TARGET#ON_SOURCE', 'taql': 'ROWNUMBER() >= 37069 && ROWNUMBER() <= 43240', 'timestr': '', 'usescratch': False, 'uvdist': ''}}}
15-May-2024 14:41:32 2024-05-15 20:41:28        INFO        SynthesisImagerVi2::selectData         MS : /lustre/cv/sw/casa/casaci/data/casatestdata-large/stakeholder/alma/2018.1.00879.S_tclean.ms | Selecting on fields : 0 | Selecting on spw :0:245.220516619~245.273983416GHz,1:261.752937691~261.774177925GHz;261.783699409~261.837898628GHz;261.958504097~261.984871284GHz | Selecting on antenna names : 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46 | Selecting on scan : 7,11 | Selecting on Scan Intent/State : OBSERVE_TARGET#ON_SOURCE | [Opened in readonly mode]
15-May-2024 14:41:32 2024-05-15 20:41:28        INFO        SynthesisImagerVi2::selectData           NRows selected : 43240
15-May-2024 14:41:32 2024-05-15 20:41:29        INFO        synthesisimager::defineimage        Detected tracking of moving source
15-May-2024 14:41:32 2024-05-15 20:41:29        INFO        SynthesisImagerVi2::defineImage         Define image coordinates for [standard_mfs_eph.iter0] :
15-May-2024 14:41:32 2024-05-15 20:41:29        INFO        SynthesisImagerVi2::defineImage         Define image coordinates for [standard_mfs_eph.iter0] :
15-May-2024 14:41:32 2024-05-15 20:41:29        INFO        SynthesisImagerVi2::defineImage         Impars: start
15-May-2024 14:41:32 2024-05-15 20:41:29        INFO        SynthesisImagerVi2::defineImage         Shape: [288, 288, 1, 1] Spectral: [2.53574e+11] at [0] with increment [1.67625e+10]
15-May-2024 14:41:32 2024-05-15 20:41:29        INFO        SynthesisImagerVi2::defineImage         Set Gridding options for [standard_mfs_eph.iter0] with ftmachine: gridft
15-May-2024 14:41:32 2024-05-15 20:41:29        INFO        SynthesisImagerVi2::nSubCubeFitInMemory        Required memory: 0.03476 GB. Available mem.: 183.6 GB (rc, mem. fraction: 80%, memory: -) => Subcubes: 1. Processes on node: 8.
15-May-2024 14:41:32 2024-05-15 20:41:29        INFO        tclean::::casa        checkJobs. Blocking for nodes to finish
15-May-2024 14:41:32 2024-05-15 20:41:30        INFO        tclean::::casa        ...done
15-May-2024 14:41:32 2024-05-15 20:41:30        INFO        tclean::::casa        checkJobs. Blocking for nodes to finish
15-May-2024 14:41:32 2024-05-15 20:41:31        INFO        tclean::::casa        ...done
15-May-2024 14:41:32
15-May-2024 14:41:32 Failed tests:['Caught exception during test execution.']
15-May-2024 14:41:32
15-May-2024 14:41:32 Results: [{'testname': 'test_stk_alma_pipeline_imaging', 'runtime': '5:00:00.003847', 'returncode': 1, 'testerr': ['Caught exception during test execution.']}]
15-May-2024 14:41:32 Stopping Xvfb.
15-May-2024 14:41:32 Stopping framebuffer <subprocess.Popen object at 0x7f374b1df2b0>
15-May-2024 14:41:32 Virtual frame buffer deployed at :27656 with pid 5809 successfully shutdown
15-May-2024 14:41:32 Xvfb stopped.
Collapse VLA Stakeholder Tests EL7 Docker Default Stage Download or View
15-May-2024 16:08:53 2024-05-15 20:08:54        SEVERE        ::casa        The expected casa data was not found at measurespath. CASA may still work if the data can be found in datapath.
15-May-2024 16:08:53 2024-05-15 20:08:54        SEVERE        ::casa        The expected measures data was not found at measurespath. CASA may still work if the data can be found in datapath.
15-May-2024 16:08:53 2024-05-15 20:08:54        SEVERE        ::casa        visit https://casadocs.readthedocs.io/en/stable/notebooks/external-data.html for more information
15-May-2024 16:08:53 2024-05-15 20:08:54        SEVERE        ::casa        
15-May-2024 16:08:53 Executor result: 1
15-May-2024 16:08:53 Failed tests:['Executor returned a non-zero exit code']
15-May-2024 16:08:53
15-May-2024 16:08:53 Results: [{'testname': 'test_stk_vla_users_continuum_from_SDM', 'runtime': '0:00:03.497291', 'returncode': 1, 'testerr': ['Executor returned a non-zero exit code']}]
15-May-2024 16:08:53 Stopping Xvfb.
15-May-2024 16:08:53 Stopping framebuffer <Popen: returncode: None args: ['Xvfb', ':150', '-screen', '0', '2048x2048x2...>
15-May-2024 16:08:53 Virtual frame buffer deployed at :150 with pid 162 successfully shutdown
15-May-2024 16:08:53 Xvfb stopped.
15-May-2024 16:23:10 cp: cannot stat '/home/casatest/work/test_stk_vla_users_continuum_from_SDM/test_vla_users_continuum_from_sdm_weblog.html': No such file or directory
15-May-2024 16:23:11 cp: cannot stat '/home/casatest/work/test_stk_vla_users_continuum_from_SDM/*.png': No such file or directory
15-May-2024 16:23:13 total 949920
15-May-2024 16:23:13 -rw-r--r-- 1 casaci    15001 972702072 May 15 20:03 casa-6.6.4-27-py3.8.el7.tar.xz
15-May-2024 16:23:13 drwxr-xr-x 2 casaci root          4096 May 15 20:23 html
15-May-2024 16:23:13 -rw-r--r-- 1 casaci casatest       411 May 15 20:08 xUnit-suite-mpi-4.xml
15-May-2024 16:23:13 total 0
15-May-2024 16:23:13 Starting cleanup
15-May-2024 16:23:27 test-container
15-May-2024 16:23:29 test-container
15-May-2024 16:23:29 Cleanup complete
15-May-2024 16:23:29 Failing as no matching files has been found and empty artifacts are not allowed.
15-May-2024 16:23:29 Unable to publish artifact [VLA Stakeholder]: