PDA

View Full Version : Why TotalView Express Student generate 3 thread for every process?



uwyocs
10-14-2009, 02:22 PM
We installed TotalView Express Student 8.7.0-3 for Linux/x86-64

When we run MPI program (http://en.wikipedia.org/wiki/Message_Passing_Interface#Example_program).
We set Open MPI, tasks:1, Nodes:1
Totalview shows that each process contains 3 threads. Actually, the code does not create threads at all. Thus, we cannot use Totalview to debug any MPI program which has two or more processes, since TotalView Student version will terminate by itself if there are more than 4 threads(student's edition can only has less than 4 processes or threads).

What's the problem with it? Thank you very much.



Here are some information about our computer.
uname -a
Linux seismicmstm.cluster 2.6.18-128.1.10.el5 #1 SMP Thu May 7 10:35:59 EDT 2009 x86_64 x86_64 x86_64 GNU/Linux

cat /etc/*release
CentOS release 5.3 (Final)

gcc -v
Using built-in specs.
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-libgcj-multifile --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --enable-plugin --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre --with-cpu=generic --host=x86_64-redhat-linux
Thread model: posix
gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)

openmpi-1.3.2




Here are the information displayed by TotalView
$ /home/lwang7/TotalViewIndividual/toolworks/totalview.8.7.0-3/bin/totalview
Linux x86_64 TotalView 8.7.0-3
Copyright 2007-2009 by TotalView Technologies, LLC. ALL RIGHTS RESERVED.
Copyright 1999-2007 by Etnus, LLC.
Copyright 1999 by Etnus, Inc.
Copyright 1996-1998 by Dolphin Interconnect Solutions, Inc.
Copyright 1989-1996 by BBN Inc.
TotalView Technologies ReplayEngine
Copyright 2009 TotalView Technologies
ReplayEngine uses the UndoDB Reverse Execution Engine
Copyright 2005-2009 Undo Limited
Reading symbols for process 1, executing "/home/lwang7/hhuang1/hpc/assignment3/HPCA3MV2/a.out"
Library /home/lwang7/hhuang1/hpc/assignment3/HPCA3MV2/a.out, with 2 asects, was linked at 0x00400000, and initially loaded at 0xff00000090000000
Mapping 12229 bytes of ELF string data from '/home/lwang7/hhuang1/hpc/assignment3/HPCA3MV2/a.out'...done
Indexing 8088 bytes of DWARF '.debug_frame' symbols from '/home/lwang7/hhuang1/hpc/assignment3/HPCA3MV2/a.out'...done
Indexing 7820 bytes of DWARF '.eh_frame' symbols from '/home/lwang7/hhuang1/hpc/assignment3/HPCA3MV2/a.out'...done
Skimming 78570 bytes of DWARF '.debug_info' symbols from '/home/lwang7/hhuang1/hpc/assignment3/HPCA3MV2/a.out'...done
Library /usr/local/openmpi-1.3.2/lib/libmpi_cxx.so.0, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff0000009002dc00
Mapping 16735 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/libmpi_cxx.so.0'...done
Indexing 8988 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/libmpi_cxx.so.0'...done
Library /usr/local/openmpi-1.3.2/lib/libmpi.so.0, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff00000090047a00
Mapping 61783 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/libmpi.so.0'...done
Indexing 216 bytes of DWARF '.debug_frame' symbols from '/usr/local/openmpi-1.3.2/lib/libmpi.so.0'...done
Indexing 39500 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/libmpi.so.0'...done
Skimming 28920 bytes of DWARF '.debug_info' symbols from '/usr/local/openmpi-1.3.2/lib/libmpi.so.0'...done
Library /usr/local/openmpi-1.3.2/lib/libopen-rte.so.0, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000900ea000
Mapping 22299 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/libopen-rte.so.0'...done
Indexing 15204 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/libopen-rte.so.0'...done
Library /usr/local/openmpi-1.3.2/lib/libopen-pal.so.0, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff00000090130000
Mapping 28603 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/libopen-pal.so.0'...done
Indexing 23404 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/libopen-pal.so.0'...done
Library /lib64/libdl.so.2, with 2 asects, was linked at 0x355ac00000, and initially loaded at 0xff000000901a0900
Mapping 1585 bytes of ELF string data from '/lib64/libdl.so.2'...done
Indexing 596 bytes of DWARF '.eh_frame' symbols from '/lib64/libdl.so.2'...done
Library /lib64/libnsl.so.1, with 2 asects, was linked at 0x3561000000, and initially loaded at 0xff000000901a2d00
Mapping 8336 bytes of ELF string data from '/lib64/libnsl.so.1'...done
Indexing 5380 bytes of DWARF '.eh_frame' symbols from '/lib64/libnsl.so.1'...done
Library /lib64/libutil.so.1, with 2 asects, was linked at 0x3569000000, and initially loaded at 0xff000000901b9c00
Mapping 1378 bytes of ELF string data from '/lib64/libutil.so.1'...done
Indexing 276 bytes of DWARF '.eh_frame' symbols from '/lib64/libutil.so.1'...done
Library /usr/lib64/libstdc++.so.6, with 2 asects, was linked at 0x3560000000, and initially loaded at 0xff000000901bbb00
Mapping 138981 bytes of ELF string data from '/usr/lib64/libstdc++.so.6'...done
Indexing 78412 bytes of DWARF '.eh_frame' symbols from '/usr/lib64/libstdc++.so.6'...done
Library /lib64/libm.so.6, with 2 asects, was linked at 0x355a800000, and initially loaded at 0xff000000902bb700
Mapping 15664 bytes of ELF string data from '/lib64/libm.so.6'...done
Indexing 12668 bytes of DWARF '.eh_frame' symbols from '/lib64/libm.so.6'...done
Library /lib64/libgcc_s.so.1, with 2 asects, was linked at 0x355dc00000, and initially loaded at 0xff0000009033ce00
Mapping 1602 bytes of ELF string data from '/lib64/libgcc_s.so.1'...done
Indexing 3932 bytes of DWARF '.eh_frame' symbols from '/lib64/libgcc_s.so.1'...done
Library /lib64/libpthread.so.0, with 2 asects, was linked at 0x355b000000, and initially loaded at 0xff00000090349f00
Mapping 19029 bytes of ELF string data from '/lib64/libpthread.so.0'...done
Indexing 14276 bytes of DWARF '.eh_frame' symbols from '/lib64/libpthread.so.0'...done
Library /lib64/libc.so.6, with 2 asects, was linked at 0x355a400000, and initially loaded at 0xff00000090363d00
Mapping 93348 bytes of ELF string data from '/lib64/libc.so.6'...done
Indexing 131444 bytes of DWARF '.eh_frame' symbols from '/lib64/libc.so.6'...done
Library /lib64/ld-linux-x86-64.so.2, with 2 asects, was linked at 0x355a000000, and initially loaded at 0xff00000090011000
Mapping 5510 bytes of ELF string data from '/lib64/ld-linux-x86-64.so.2'...done
Indexing 5736 bytes of DWARF '.eh_frame' symbols from '/lib64/ld-linux-x86-64.so.2'...done
MPI launch string is: orterun -np 2 %B/tvdbootstrap
Testing MPI configuration with 'ompi_info -c'
Exit value was 0 (expected 0), status: execute_command_t::exited
Launching MPI job with command: orterun -np 2 %B/tvdbootstrap
Server args: -callback 192.168.1.100:4142 -set_pw 7c13e90c:7b9706b8
Environment:
Launching TotalView Debugger Server with command:
orterun -np 2 /home/lwang7/TotalViewIndividual/toolworks/totalview.8.7.0-3/linux-x86-64/bin/tvdbootstrap
Reading symbols for process 2, executing "/home/lwang7/hhuang1/hpc/assignment3/HPCA3MV2/a.out"
Reading symbols for process 3, executing "/home/lwang7/hhuang1/hpc/assignment3/HPCA3MV2/a.out"
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_paffinity_linux.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904b8100
Mapping 2560 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_paffinity_linux.so'...done
Indexing 1052 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_paffinity_linux.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_carto_auto_detect.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904bc400
Mapping 920 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_carto_auto_detect.so'...done
Indexing 132 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_carto_auto_detect.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_ess_env.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904bd400
Mapping 1304 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_ess_env.so'...done
Indexing 332 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_ess_env.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_rml_oob.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904bf500
Mapping 2104 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_rml_oob.so'...done
Indexing 1228 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_rml_oob.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_oob_tcp.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904c4300
Mapping 4152 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_oob_tcp.so'...done
Indexing 2308 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_oob_tcp.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_routed_binomial.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904d1e00
Mapping 1963 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_routed_binomial.so'...done
Indexing 652 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_routed_binomial.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_grpcomm_bad.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904d5b00
Mapping 1297 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_grpcomm_bad.so'...done
Indexing 444 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_grpcomm_bad.so'...done
Library /lib64/libnss_files.so.2, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904d8400
Mapping 3447 bytes of ELF string data from '/lib64/libnss_files.so.2'...done
Indexing 3076 bytes of DWARF '.eh_frame' symbols from '/lib64/libnss_files.so.2'...done
Library /lib64/libnss_nis.so.2, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904e2300
Mapping 3370 bytes of ELF string data from '/lib64/libnss_nis.so.2'...done
Indexing 2444 bytes of DWARF '.eh_frame' symbols from '/lib64/libnss_nis.so.2'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_allocator_basic.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904ebb00
Mapping 974 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_allocator_basic.so'...done
Indexing 316 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_allocator_basic.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_allocator_bucket.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904ed500
Mapping 938 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_allocator_bucket.so'...done
Indexing 380 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_allocator_bucket.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_rcache_vma.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904eec00
Mapping 1262 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_rcache_vma.so'...done
Indexing 596 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_rcache_vma.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_mpool_fake.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904f1500
Mapping 582 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_mpool_fake.so'...done
Indexing 172 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_mpool_fake.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_mpool_rdma.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904f2200
Mapping 1306 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_mpool_rdma.so'...done
Indexing 548 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_mpool_rdma.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_mpool_sm.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904f4e00
Mapping 1051 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_mpool_sm.so'...done
Indexing 292 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_mpool_sm.so'...done
Library /usr/local/openmpi-1.3.2/lib/libmca_common_sm.so.0, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904f6900
Mapping 779 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/libmca_common_sm.so.0'...done
Indexing 172 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/libmca_common_sm.so.0'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_pml_ob1.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000904f7f00
Mapping 4558 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_pml_ob1.so'...done
Indexing 3260 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_pml_ob1.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_bml_r2.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff0000009050bf00
Mapping 1597 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_bml_r2.so'...done
Indexing 564 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_bml_r2.so'...done
Library /usr/lib64/librdmacm.so.1, with 2 asects, was linked at 0x355b400000, and initially loaded at 0xff0000009050f900
Mapping 1163 bytes of ELF string data from '/usr/lib64/librdmacm.so.1'...done
Indexing 1484 bytes of DWARF '.eh_frame' symbols from '/usr/lib64/librdmacm.so.1'...done
Library /usr/lib64/libibverbs.so.1, with 2 asects, was linked at 0x355a800000, and initially loaded at 0xff00000090514600
Mapping 2125 bytes of ELF string data from '/usr/lib64/libibverbs.so.1'...done
Indexing 3972 bytes of DWARF '.eh_frame' symbols from '/usr/lib64/libibverbs.so.1'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_btl_openib.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff00000090520400
Mapping 11292 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_btl_openib.so'...done
Indexing 5436 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_btl_openib.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_btl_self.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff00000090555e00
Mapping 1364 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_btl_self.so'...done
Indexing 508 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_btl_self.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_btl_sm.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff00000090558f00
Mapping 2136 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_btl_sm.so'...done
Indexing 620 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_btl_sm.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_btl_tcp.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff0000009055e100
Mapping 4040 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_btl_tcp.so'...done
Indexing 1684 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_btl_tcp.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_basic.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000905e8a00
Mapping 2520 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_basic.so'...done
Indexing 1844 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_basic.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_hierarch.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000905f0300
Mapping 1923 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_hierarch.so'...done
Indexing 820 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_hierarch.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_inter.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000905f4100
Mapping 1365 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_inter.so'...done
Indexing 532 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_inter.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_self.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000905f6b00
Mapping 1586 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_self.so'...done
Indexing 532 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_self.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_sm.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000905f8200
Mapping 1584 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_sm.so'...done
Indexing 460 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_sm.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_sync.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000905fca00
Mapping 1221 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_sync.so'...done
Indexing 588 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_sync.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_tuned.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff000000905ff400
Mapping 7286 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_tuned.so'...done
Indexing 5380 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_coll_tuned.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_osc_pt2pt.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff00000090616700
Mapping 3451 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_osc_pt2pt.so'...done
Indexing 1756 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_osc_pt2pt.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_osc_rdma.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff00000090620500
Mapping 3496 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_osc_rdma.so'...done
Indexing 1860 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_osc_rdma.so'...done
Library /usr/lib64/libmthca-rdmav2.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff0000009062c600
Mapping 859 bytes of ELF string data from '/usr/lib64/libmthca-rdmav2.so'...done
Indexing 2068 bytes of DWARF '.eh_frame' symbols from '/usr/lib64/libmthca-rdmav2.so'...done
Library /usr/lib64/libipathverbs-rdmav2.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff00000090633700
Mapping 876 bytes of ELF string data from '/usr/lib64/libipathverbs-rdmav2.so'...done
Indexing 1076 bytes of DWARF '.eh_frame' symbols from '/usr/lib64/libipathverbs-rdmav2.so'...done
Library /usr/lib64/libnes-rdmav2.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff00000090636c00
Mapping 606 bytes of ELF string data from '/usr/lib64/libnes-rdmav2.so'...done
Indexing 916 bytes of DWARF '.eh_frame' symbols from '/usr/lib64/libnes-rdmav2.so'...done
Library /usr/lib64/libcxgb3-rdmav2.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff0000009063a300
Mapping 690 bytes of ELF string data from '/usr/lib64/libcxgb3-rdmav2.so'...done
Indexing 1172 bytes of DWARF '.eh_frame' symbols from '/usr/lib64/libcxgb3-rdmav2.so'...done
Library /usr/lib64/libmlx4-rdmav2.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff0000009063e800
Mapping 847 bytes of ELF string data from '/usr/lib64/libmlx4-rdmav2.so'...done
Indexing 1804 bytes of DWARF '.eh_frame' symbols from '/usr/lib64/libmlx4-rdmav2.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_pubsub_orte.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff00000090643f00
Mapping 928 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_pubsub_orte.so'...done
Indexing 364 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_pubsub_orte.so'...done
Library /usr/local/openmpi-1.3.2/lib/openmpi/mca_dpm_orte.so, with 2 asects, was linked at 0x00000000, and initially loaded at 0xff00000090646300
Mapping 1966 bytes of ELF string data from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_dpm_orte.so'...done
Indexing 540 bytes of DWARF '.eh_frame' symbols from '/usr/local/openmpi-1.3.2/lib/openmpi/mca_dpm_orte.so'...done

PeterT-RogueWave
10-15-2009, 06:07 AM
Okay, I've tested this a few ways and one method I get threads, and another method I get no thread, or rather one per process. The difference appears to be the fact that on the system where I'm getting multiple threads, I'm using OpenMPI on a cluster with Infiniband support, and OpenMPI was built with that support. On a system with no Infiniband interconnect, I see only one thread per process.

Now, while I can't tell you what to do with the free Student Express license, the general intent here was that this would be used by individual students on their personal laptop or workstation. It appears in this case that you've installed this on a cluster, seismicmstm.cluster, and are running into some of the limits we've put into place to prevent this. I suppose you could always re-build OpenMPI without infiniband support (there must be an option to disable that). and it would work better for you. Or move the license to a different machine without the Infiniband support. Let me know if you would like to do that, and I will grant a one-time exception.

Of course there may be a way to tell OpenMPI not to go multi-threaded for this case, but I don't know if that is possible.

Let me know if this makes sense.

Regards,

uwyocs
10-15-2009, 10:53 AM
Thank you very much. I will try it.

uwyocs
10-15-2009, 05:09 PM
Under "Parallel" tab, in the "Additional Starter Arguments", please use "--mca btl ^openib" as additional argument.

This approach works!