Crashes during opening process

dgrist · May 6, 2016, 2:19pm

Hi,

I’m having trouble running Mantid on a completely fresh Ubuntu 14.04 VM instance. After installing it from the repo package and attempting to run, it crashes before fully loaded with the following message:

Floating point exception (core dumped)

Sample entries recorded in the syslog at the same time are:

May 6 15:10:20 davidros-VirtualBox kernel: [ 1274.580087] traps: MantidPlot_exe[2191] trap divide error ip:7f6445851a84 sp:7fff2e3defe0 error:0 in libpgm-5.1.so.0.0.118[7f6445847000+47000]

May 6 15:17:48 davidros-VirtualBox kernel: [ 1722.184088] traps: MantidPlot_exe[2243] trap divide error ip:7ff24a63aa84 sp:7fff8c765e60 error:0 in libpgm-5.1.so.0.0.118[7ff24a630000+47000]

Mantid has previously worked in a similar VM on the same host system, and intermittently crashed in the same way when there was no free HDD space available. This time I have made sure that there is ~20GB free, but the problem persists.

Would anyone be able to help isolate the problem?

Thanks.

martyngigg · May 9, 2016, 8:58am

Hi,

Can I first ask what version you are running, is it the nightly or official release?

Would you then also be able to start MantidPlot with:

gdb -ex "run" /opt/Mantid/bin/MantidPlot_exe

and when it crashes type bt then press return and past the output here?

Thanks.

dgrist · May 9, 2016, 12:16pm

Hi,

Thanks for the quick response!

I’m not sure what release it is, although it was installed from the command line in the following manner:

sudo apt-add-repository  "deb http://apt.isis.rl.ac.uk trusty main"
sudo apt-add-repository ppa:mantid/mantid
sudo apt-get update
sudo apt-get install mantid

The suggested GDB command seemed to spend less time loading before it failed than when executing it normally. It gave the following output:

$ gdb -ex "run" /opt/Mantid/bin/MantidPlot_exe
GNU gdb (Ubuntu 7.7-0ubuntu3.1) 7.7
This GDB was configured as "x86_64-linux-gnu".
Reading symbols from /opt/Mantid/bin/MantidPlot_exe...(no debugging symbols found)...done.
Starting program: /opt/Mantid/bin/MantidPlot_exe 
/opt/Mantid/bin/MantidPlot_exe: error while loading shared libraries: libvtkPVServerManagerRendering-pv5.0.so.1: cannot open shared object file: No such file or directory
[Inferior 1 (process 2618) exited with code 0177]

Bizarrely, after a week of failing, Mantid seemed to open fine on the first attempt, and then reverted again to crashing in the above way on subsequent attempts.

dgrist · May 9, 2016, 12:20pm

Aha, I tried adding the location of that file (/opt/Mantid/lib/paraview-5.0/) to LD_LIBRARY_PATH, and it got a bit further this time:

$ gdb -ex "run" /opt/Mantid/bin/MantidPlot_exe
GNU gdb (Ubuntu 7.7-0ubuntu3.1) 7.7
Starting program: /opt/Mantid/bin/MantidPlot_exe 
warning: File "/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19-gdb.py" auto-loading has been declined by your `auto-load safe-path' set to "$debugdir:$datadir/auto-load".
To enable execution of this file, add
    add-auto-load-safe-path /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19-gdb.py
line to your configuration file "/home/davidros/.gdbinit".
To completely disable this security protection, add
    set auto-load safe-path /
line to your configuration file "/home/davidros/.gdbinit".
For more information about this security protection, see the
"Auto-loading safe path" section in the GDB manual.  E.g., run from the shell:
    info "(gdb)Auto-loading safe path"
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
FrameworkManager-[Notice] Welcome to Mantid 3.6.0
FrameworkManager-[Notice] Please cite: http://dx.doi.org/10.1016/j.nima.2014.07.029 and this release: http://dx.doi.org/10.5286/Software/Mantid3.6
[New Thread 0x7fffc5ae1700 (LWP 2674)]
[New Thread 0x7fffc52e0700 (LWP 2675)]
[New Thread 0x7fffb8931700 (LWP 2676)]
[New Thread 0x7fffb37fe700 (LWP 2678)]
[New Thread 0x7fffb3fff700 (LWP 2677)]
[New Thread 0x7fff972b6700 (LWP 2685)]
Warn: Linux kernel reports non-constant Time Stamp Counter (TSC).

Program received signal SIGFPE, Arithmetic exception.
0x00007fff98e12a84 in ?? () from /usr/lib/x86_64-linux-gnu/libpgm-5.1.so.0

martyngigg · May 12, 2016, 10:32am

Hi,

My apologies for the delay in response. It turns out our email notifications weren’t working and I missed your replies.

The fault seems to occur in libpgm, which I’m not familiar with. Could you run the same gdb call and this time when it stops type bt and see what it prints?

Thanks

dgrist · May 13, 2016, 8:07am

Ok, here’s the result of that:

(gdb) bt
#0  0x00007fff98e9da84 in ?? () from /usr/lib/x86_64-linux-gnu/libpgm-5.1.so.0
#1  0x00007fff98eb9f5f in pgm_init ()
   from /usr/lib/x86_64-linux-gnu/libpgm-5.1.so.0
#2  0x00007fff991201f9 in zmq_ctx_new ()
   from /usr/lib/x86_64-linux-gnu/libzmq.so.3
#3  0x00007fff98a7ef16 in ?? ()
   from /usr/lib/python2.7/dist-packages/zmq/backend/cython/context.x86_64-linux-gnu.so
#4  0x00007ffff245c643 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#5  0x00007ffff2459d43 in PyObject_Call ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#6  0x00007ffff24b4816 in PyEval_EvalFrameEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#7  0x00007ffff24b7059 in PyEval_EvalFrameEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#8  0x00007ffff24b854d in PyEval_EvalCodeEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#9  0x00007ffff24ed6d0 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#10 0x00007ffff2459d43 in PyObject_Call ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#11 0x00007ffff23e57bd in ?? ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#12 0x00007fff98c8ab63 in ?? ()
   from /usr/lib/python2.7/dist-packages/zmq/backend/cython/message.x86_64-linux-gnu.so
#13 0x00007ffff245c643 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#14 0x00007ffff2459d43 in PyObject_Call ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#15 0x00007ffff24d2577 in PyEval_CallObjectWithKeywords ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#16 0x00007ffff23ad1d1 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#17 0x00007ffff24b70d4 in PyEval_EvalFrameEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#18 0x00007ffff24b854d in PyEval_EvalCodeEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#19 0x00007ffff24b6dd8 in PyEval_EvalFrameEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#20 0x00007ffff24b854d in PyEval_EvalCodeEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#21 0x00007ffff24b6dd8 in PyEval_EvalFrameEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#22 0x00007ffff24b7059 in PyEval_EvalFrameEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#23 0x00007ffff24b854d in PyEval_EvalCodeEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#24 0x00007ffff24b6dd8 in PyEval_EvalFrameEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#25 0x00007ffff24b854d in PyEval_EvalCodeEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#26 0x00007ffff24b6dd8 in PyEval_EvalFrameEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#27 0x00007ffff24b854d in PyEval_EvalCodeEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#28 0x00007ffff24b6dd8 in PyEval_EvalFrameEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#29 0x00007ffff24b854d in PyEval_EvalCodeEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#30 0x00007ffff24b6dd8 in PyEval_EvalFrameEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#31 0x00007ffff24b7059 in PyEval_EvalFrameEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#32 0x00007ffff24b854d in PyEval_EvalCodeEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#33 0x00007ffff24ed6d0 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#34 0x00007ffff2459d43 in PyObject_Call ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#35 0x00007ffff24147eb in PyObject_CallFunction ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#36 0x00007ffff2414a2d in ?? ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#37 0x00007ffff2404627 in _PyObject_GenericSetAttrWithDict ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#38 0x00007ffff2470bbf in PyObject_SetAttr ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#39 0x00007ffff24b41da in PyEval_EvalFrameEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#40 0x00007ffff24b854d in PyEval_EvalCodeEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#41 0x00007ffff24ed6d0 in ?? ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#42 0x00007ffff2459d43 in PyObject_Call ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#43 0x00007ffff23e57bd in ?? ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#44 0x00007ffff2459d43 in PyObject_Call ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#45 0x00007ffff245e67f in ?? ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#46 0x00007ffff245c68f in ?? ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#47 0x00007ffff2459d43 in PyObject_Call ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#48 0x00007ffff24b4816 in PyEval_EvalFrameEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#49 0x00007ffff24b854d in PyEval_EvalCodeEx ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#50 0x00007ffff24b8682 in PyEval_EvalCode ()
   from /usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0
#51 0x00000000007774b5 in PythonScript::executeCompiledCode(_object*) ()
#52 0x000000000077a3fb in PythonScript::executeString() ()
#53 0x000000000077a5e5 in PythonScript::executeImpl() ()
#54 0x000000000056abcf in ApplicationWindow::runPythonScript(QString const&, bool, bool, bool) [clone .constprop.712] ()
#55 0x00000000005d7a7e in ApplicationWindow::init(bool, QStringList const&) ()
#56 0x00000000005dd6f3 in ApplicationWindow::ApplicationWindow(bool, QStringList const&) ()
#57 0x0000000000539019 in main ()

martyngigg · May 13, 2016, 9:21am

Hi,

This is certainly an odd problem. I’ve run Mantid inside a VirtualBox VM myself many times and not had a problem. The syslog errors and the gdb backtrace seem to point to a problem with loading one of the IPython dependencies, I have found a similar bug report elsewhere but nothing much else. Are you ale to run ipython on its own?

The next thing I can think to try is to upgrade the ipython dependency locally:

pip install --user --upgrade pyzmq
echo "import os, sys; sys.path.insert(0, os.path.dirname(__file__))" > $HOME/.local/lib/python2.7/site-packages/usercustomize.py

Will Mantid or IPython start after this?

dgrist · May 13, 2016, 10:08am

Aha! Yes, ipython worked ok before, but now both Mantid and ipython open fine.

Thanks very much for the help.