Build: #5 failed
Job: Test Tasks MPI Many Linux 2.28 Rocky 8.10 Py3.12 failed
Job result summary
- Completed
- Duration
- 80 minutes
- Agent
- cbt-el7-11.cv.nrao.edu
- Revision
-
873c61fbb37c8dc6af128afc90c1fdd0d3972b95
- Total tests
- 171
- Fixed in
- #31 (Child of CASASPECS-MASTERBUILDCASAMPI-31)
- Number of retries
- 1
Tests
- 171 tests in total
- 1 test failed
- 1 failure is new
- 10 tests were quarantined / skipped
- 70 minutes taken in total.
| Status | Test | Duration | |
|---|---|---|---|
| Collapse |
test_0_MPIInterface
test_PyParallelImagerHelper_interface
|
< 1 sec | |
TypeError: 'NoneType' object is not subscriptable
self = <casampi.tests.test_casampi.test_0_MPIInterface testMethod=test_PyParallelImagerHelper_interface>
def test_PyParallelImagerHelper_interface(self):
# Get cluster (getCluster should automatically initialize it)
self.sc = MPIInterface.getCluster()
self.CL = self.sc._cluster
(21 more lines...)
|
|||
Error summary
The build generated some errors. See the full build log for more details.
From https://open-bitbucket.nrao.edu/scm/casa/casa-data
faa54e9..ca094d9 master -> origin/master
Downloading geodetic/IERSpredict/table.dat (3.3 KB)
Downloading geodetic/IERSpredict/table.f0 (230 KB)
Downloading geodetic/IERSpredict/table.lock (325 B)
fatal: Not a git repository (or any parent up to mount point /export/home/cbt-el7-11)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Error response from daemon: No such container: wheel-container-test
Error response from daemon: No such container: wheel-container-test
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 6002k 100 6002k 0 0 21.7M 0 --:--:-- --:--:-- --:--:-- 22.1M
[16134] Failed to execute script 'atlutil' due to unhandled exception!
Traceback (most recent call last):
File "atlutil.py", line 200, in <module>
File "atlutil.py", line 165, in has_fix_version
File "json/__init__.py", line 354, in loads
File "json/decoder.py", line 339, in decode
File "json/decoder.py", line 357, in raw_decode
json.decoder.JSONDecodeError: Expecting value: line 12 column 1 (char 11)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
Already on 'master'
Already on 'master'
2025-05-20 18:40:33 SEVERE MPICommandClient::stop_services::MPICommandClient::stop_services::casa Aborting command request with id# 70: {'command': 'pow(a,b)', 'parameters': {'a': 10, 'b': 100000000000000000}, 'mode': 'eval', 'id': 70, 'server': 1, 'status': 'request sent'}
2025-05-20 18:40:36 SEVERE MPICommandClient::stop_services::MPICommandClient::stop_services::casa MPIServers with rank [1] are in timeout condition, skipping MPI_Finalize()
2025-05-20 18:40:36 SEVERE MPICommandClient::stop_services::MPICommandClient::stop_services::casa Not possible to finalize gracefully... calling Aborting MPI environment
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
Proc: [[53661,1],0]
Errorcode: 0
NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
prterun has exited due to process rank 0 with PID 5489 on node 9f37a20f6bd4 exiting
improperly. There are three reasons this could occur:
1. this process did not call "init" before exiting, but others in the
job did. This can cause a job to hang indefinitely while it waits for
all processes to call "init". By rule, if one process calls "init",
then ALL processes must call "init" prior to termination.
2. this process called "init", but exited without calling "finalize".
By rule, all processes that call "init" MUST call "finalize" prior to
exiting or it will be considered an "abnormal termination"
3. this process called "MPI_Abort" or "prte_abort" and the mca
parameter prte_create_session_dirs is set to false. In this case, the
run-time cannot detect that the abort call was an abnormal
termination. Hence, the only error message you will receive is this
one.
This may have caused other processes in the application to be
terminated by signals sent by prterun (as reported here).
You can avoid this message by specifying -quiet on the prterun command
line.
--------------------------------------------------------------------------
faa54e9..ca094d9 master -> origin/master
Downloading geodetic/IERSpredict/table.dat (3.3 KB)
Downloading geodetic/IERSpredict/table.f0 (230 KB)
Downloading geodetic/IERSpredict/table.lock (325 B)
fatal: Not a git repository (or any parent up to mount point /export/home/cbt-el7-11)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Error response from daemon: No such container: wheel-container-test
Error response from daemon: No such container: wheel-container-test
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 6002k 100 6002k 0 0 21.7M 0 --:--:-- --:--:-- --:--:-- 22.1M
[16134] Failed to execute script 'atlutil' due to unhandled exception!
Traceback (most recent call last):
File "atlutil.py", line 200, in <module>
File "atlutil.py", line 165, in has_fix_version
File "json/__init__.py", line 354, in loads
File "json/decoder.py", line 339, in decode
File "json/decoder.py", line 357, in raw_decode
json.decoder.JSONDecodeError: Expecting value: line 12 column 1 (char 11)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
Already on 'master'
Already on 'master'
2025-05-20 18:40:33 SEVERE MPICommandClient::stop_services::MPICommandClient::stop_services::casa Aborting command request with id# 70: {'command': 'pow(a,b)', 'parameters': {'a': 10, 'b': 100000000000000000}, 'mode': 'eval', 'id': 70, 'server': 1, 'status': 'request sent'}
2025-05-20 18:40:36 SEVERE MPICommandClient::stop_services::MPICommandClient::stop_services::casa MPIServers with rank [1] are in timeout condition, skipping MPI_Finalize()
2025-05-20 18:40:36 SEVERE MPICommandClient::stop_services::MPICommandClient::stop_services::casa Not possible to finalize gracefully... calling Aborting MPI environment
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
Proc: [[53661,1],0]
Errorcode: 0
NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
prterun has exited due to process rank 0 with PID 5489 on node 9f37a20f6bd4 exiting
improperly. There are three reasons this could occur:
1. this process did not call "init" before exiting, but others in the
job did. This can cause a job to hang indefinitely while it waits for
all processes to call "init". By rule, if one process calls "init",
then ALL processes must call "init" prior to termination.
2. this process called "init", but exited without calling "finalize".
By rule, all processes that call "init" MUST call "finalize" prior to
exiting or it will be considered an "abnormal termination"
3. this process called "MPI_Abort" or "prte_abort" and the mca
parameter prte_create_session_dirs is set to false. In this case, the
run-time cannot detect that the abort call was an abnormal
termination. Hence, the only error message you will receive is this
one.
This may have caused other processes in the application to be
terminated by signals sent by prterun (as reported here).
You can avoid this message by specifying -quiet on the prterun command
line.
--------------------------------------------------------------------------