Hi
We have solved by applying the latest SAP Note 172747. You're right, we were running out of shared memory segments. We have shmmax to 1024mb and have set it up to 4096mb. SAP ECC has started with no problem.
Our Swap space is 20Gb as per SAP recomendation:
bm-sap90:/usr/sap/QAS/DVEBMGS00/work> swapinfo -tam
Mb Mb Mb PCT START/ Mb
TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME
dev 20480 0 20480 0% 0 - 1 /dev/vg00/lvol2
reserve - 16418 -16418
memory 15536 2238 13298 14%
total 36016 18656 17360 52% - 0 -
With 15Gb RAM installed. And no, we didn't do any modification to server's profiles. Just ran a reorg of some tables.
Thanks for your answer
| | | ---------------Original Message---------------
From: cdelgadop
Sent: Monday, October 25, 2010 9:07 AM
Subject: Problems with ECC when starting dispatcher
Hi
Our SAP team worked on weekend in a reorganization of our SAP QAS ECC system. After following the procedure we have now a problem when starting SAP. Dispatcher process died and dispatcher server is stopped.
SAP is running on top of a HP-UX 11.31 vPar and we think is a problem with the shared memory segments. Following is a copy of the dev-disp file. Could you please help me on this issue ?
trc file: "dev_disp.new", trc level: 1, release: "700"
sysno 00
sid QAS
systemid 274 (HP (IA-64) with HP-UX)
relno 7000
patchlevel 0
patchno 185
intno 20050900
make: single threaded, ASCII, 64 bit, optimized
pid 19833
Mon Oct 25 04:15:33 2010
kernel runs with dp version 241(ext=110) (@(#) DPLIB-INT-VERSION-241)
length of sys_adm_ext is 364 bytes
*** SWITCH TRC-HIDE on ***
***LOG Q00=> DpSapEnvInit, DPStart (00 19833) [dpxxdisp.c 1281]
shared lib "dw_xml.sl" version 185 successfully loaded
shared lib "dw_xtc.sl" version 185 successfully loaded
shared lib "dw_stl.sl" version 185 successfully loaded
shared lib "dw_gui.sl" version 185 successfully loaded
shared lib "dw_mdm.sl" version 185 successfully loaded
rdisp/softcancel_sequence : -> 0,5,-1
use internal message server connection to port 3900
MtxInit: 30000 0 0
DpSysAdmExtInit: ABAP is active
DpSysAdmExtInit: VMC (JAVA VM in WP) is not active
DpIPCInit2: start server >bm-sap9_QAS_00 <
DpShMCreate: sizeof(wp_adm) 34272 (1224)
DpShMCreate: sizeof(tm_adm) 133146624 (26624)
DpShMCreate: sizeof(wp_ca_adm) 22912 (76)
DpShMCreate: sizeof(appc_ca_adm) 156256 (76)
DpCommTableSize: max/headSize/ftSize/tableSize
00/16/2208048/2208064
DpShMCreate: sizeof(comm_adm) 2208064 (1088)
DpSlockTableSize: max/headSize/ftSize/fiSize/tableSize=0/0/0/0/0
DpShMCreate: sizeof(slock_adm) 0 (104)
DpFileTableSize: max/headSize/ftSize/tableSize=0/0/0/0
DpShMCreate: sizeof(file_adm) 0 (72)
DpShMCreate: sizeof(vmc_adm) 0 (1760)
DpShMCreate: sizeof(wall_adm) (640048/862336/80/104)
DpShMCreate: sizeof(gw_adm) 48
DpShMCreate: SHM_DP_ADM_KEY (addr: c000000140000000, size: 137078336)
DpShMCreate: allocated sys_adm at c000000140000000
DpShMCreate: allocated wp_adm at c000000140001d20
DpShMCreate: allocated tm_adm_list at c00000014000a300
DpShMCreate: allocated tm_adm at c00000014000a360
DpShMCreate: allocated wp_ca_adm at c000000147f04b60
DpShMCreate: allocated appc_ca_adm at c000000147f0a4e0
DpShMCreate: allocated comm_adm at c000000147f30740
DpShMCreate: system runs without slock table
DpShMCreate: system runs without file table
DpShMCreate: allocated vmc_adm_list at c00000014814b880
DpShMCreate: allocated gw_adm at c00000014814b900
DpShMCreate: system runs without vmc_adm
DpShMCreate: allocated ca_info at c00000014814b930
DpShMCreate: allocated wall_adm at c00000014814b940
MBUF state OFF
DpCommInitTable: init table for 2000 entries
rdisp/queue_size_check_value : -> off
ThTaskStatus: rdisp/reset_online_during_debug 0
EmInit: MmSetImplementation( 2 ).
MM global diagnostic options set: 0
<ES> client 0 initializing ....
<ES> InitFreeList
<ES> block size is 4096 kByte.
Using implementation std
<ES> Info: use normal pages (no huge table support available)
EsStdUnamFileMapInit: shmget() of 5368709120 bytes failed.
errno = 22(Invalid argument)
EsStdInit: unable to allocate 5120 MB
EsStdInit: try to allocate 4096 MB
EsStdUnamFileMapInit: shmget() of 4294967296 bytes failed.
errno = 22(Invalid argument)
EsStdInit: unable to allocate 4096 MB
EsStdInit: try to allocate 3276 MB
EsStdUnamFileMapInit: shmget() of 3435134976 bytes failed.
errno = 22(Invalid argument)
EsStdInit: unable to allocate 3276 MB
EsStdInit: try to allocate 2620 MB
EsStdUnamFileMapInit: shmget() of 2747269120 bytes failed.
errno = 22(Invalid argument)
EsStdInit: unable to allocate 2620 MB
EsStdInit: try to allocate 2096 MB
EsStdUnamFileMapInit: shmget() of 2197815296 bytes failed.
errno = 22(Invalid argument)
EsStdInit: unable to allocate 2096 MB
EsStdInit: try to allocate 2048 MB
EsStdUnamFileMapInit: shmget() of 2147483648 bytes failed.
errno = 22(Invalid argument)
EsStdInit: unable to allocate 2048 MB
EsStdInit: try to allocate 1636 MB
EsStdUnamFileMapInit: shmget() of 1715470336 bytes failed.
errno = 22(Invalid argument)
EsStdInit: unable to allocate 1636 MB
EsStdInit: try to allocate 1308 MB
EsStdUnamFileMapInit: shmget() of 1371537408 bytes failed.
errno = 22(Invalid argument)
EsStdInit: unable to allocate 1308 MB
EsStdInit: try to allocate 1044 MB
EsStdUnamFileMapInit: shmget() of 1094713344 bytes failed.
errno = 22(Invalid argument)
EsStdInit: unable to allocate 1044 MB
EsStdInit: try to allocate 1024 MB
EsStdUnamFileMapInit: ES base = 0xc000000180000000
EsStdInit: 1024 MB successfully allocated
EsStdUnamFileMapInit: ES base = 0xc0000001c0000000
EsStdInit: 1024 MB successfully allocated
EsStdUnamFileMapInit: ES base = 0xc000000200000000
EsStdInit: 1024 MB successfully allocated
EsStdUnamFileMapInit: ES base = 0xc000000240000000
EsStdInit: 1024 MB successfully allocated
EsStdUnamFileMapInit: ES base = 0xc000000280000000
EsStdInit: 1024 MB successfully allocated
EsStdInit: Extended Memory 5120 MB allocated
<ES> 1279 blocks reserved for free list.
ES initialized.
mm.dump: set maximum dump mem to 96 MB
Mon Oct 25 04:15:35 2010
rdisp/http_min_wait_dia_wp : 1 -> 1
***LOG Q0K=> DpMsAttach, mscon ( bm-sap9) [dpxxdisp.c 12429]
use SAPLOCALHOST=<bm-sap9> as internal hostname
DpStartStopMsg: send start message (myname is >bm-sap9_QAS_00 <)
DpStartStopMsg: start msg sent
CCMS: AlInitGlobals : alert/use_sema_lock = TRUE.
CCMS: start to initalize 3.X shared alert area (first segment).
*** ERROR => DpWPCheck: W1 (pid 19846) died (severity=0, status=65280) [dpxxdisp.c 15551]
child (pid=19846) exited with exit code 255
DpMsgAdmin: Set release to 7000, patchlevel 0
MBUF state PREPARED
MBUF component UP
DpMBufHwIdSet: set Hardware-ID
***LOG Q1C=> DpMBufHwIdSet [dpxxmbuf.c 1050]
DpMsgAdmin: Set patchno for this platform to 185
Release check o.K.
Mon Oct 25 04:16:15 2010
*** ERROR => DpWPCheck: W0 (pid 19845) died (severity=0, status=0) [dpxxdisp.c 15551]
*** ERROR => DpWPCheck: W1 (pid 19846) died (severity=0, status=0) [dpxxdisp.c 15551]
*** ERROR => DpWPCheck: W2 (pid 19847) died (severity=0, status=0) [dpxxdisp.c 15551]
*** ERROR => DpWPCheck: W3 (pid 19849) died (severity=0, status=0) [dpxxdisp.c 15551]
*** ERROR => DpWPCheck: W4 (pid 19851) died (severity=0, status=0) [dpxxdisp.c 15551]
*** ERROR => DpWPCheck: W5 (pid 19854) died (severity=0, status=0) [dpxxdisp.c 15551]
*** ERROR => DpWPCheck: W6 (pid 19855) died (severity=0, status=0) [dpxxdisp.c 15551]
*** ERROR => DpWPCheck: W7 (pid 19856) died (severity=0, status=0) [dpxxdisp.c 15551]
*** ERROR => DpWPCheck: W8 (pid 19857) died (severity=0, status=0) [dpxxdisp.c 15551]
*** ERROR => DpWPCheck: W9 (pid 19860) died (severity=0, status=0) [dpxxdisp.c 15551]
*** ERROR => DpWPCheck: W11 (pid 19862) died (severity=0, status=0) [dpxxdisp.c 15551]
my types changed after wp death/restart 0xbf --> 0xbe
my types changed after wp death/restart 0xbe --> 0xbc
my types changed after wp death/restart 0xbc --> 0xb8
my types changed after wp death/restart 0xb8 --> 0xb0
my types changed after wp death/restart 0xb0 --> 0xa0
my types changed after wp death/restart 0xa0 --> 0x80
*** DP_FATAL_ERROR => DpWPCheck: no more work processes
*** DISPATCHER EMERGENCY SHUTDOWN ***
increase tracelevel of WPs
NiWait: sleep (10000ms) ...
NiISelect: timeout 10000ms
NiISelect: maximum fd=15
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Mon Oct 25 04:16:25 2010
NiISelect: TIMEOUT occured (10000ms)
dump system status
Workprocess Table (long) Mon Oct 25 08:46:25 2010
========================
No Ty. Pid Status Cause Start Err Sem CPU Time Program Cl User Action Table
-
0 DIA 19845 Ended no 2 0 0
1 DIA 19846 Ended no 2 0 0
2 DIA 19847 Ended no 2 0 0
3 DIA 19849 Ended no 2 0 0
4 DIA 19851 Ended no 2 0 0
5 DIA 19854 Ended no 2 0 0
6 DIA 19855 Ended no 2 0 0
7 DIA 19856 Ended no 2 0 0
8 DIA 19857 Ended no 2 0 0
9 DIA 19860 Ended no 2 0 0
10 DIA 19861 Ended no 1 0 0
11 DIA 19862 Ended no 2 0 0
12 DIA 19865 Ended no 1 0 0
13 DIA 19868 Ended no 1 0 0
14 UPD 19871 Ended no 1 0 0
15 UPD 19872 Ended no 1 0 0
16 UPD 19873 Ended no 1 0 0
17 UPD 19876 Ended no 1 0 0
18 ENQ 19877 Ended no 1 0 0
19 BTC 19880 Ended no 1 0 0
20 BTC 19881 Ended no 1 0 0
21 BTC 19884 Ended no 1 0 0
22 BTC 19885 Ended no 1 0 0
23 BTC 19893 Ended no 1 0 0
24 BTC 19896 Ended no 1 0 0
25 SPO 19899 Ended no 1 0 0
26 UP2 19901 Ended no 1 0 0
27 UP2 19906 Ended no 1 0 0
Dispatcher Queue Statistics Mon Oct 25 08:46:25 2010
===========================
+ + --+ --+ --+ --+ --+
| Typ | now | high | max | writes | reads |
+ + --+ --+ --+ --+ --+
| NOWP | 0 | 2 | 2000 | 6 | 6 |
+ + --+ --+ --+ --+ --+
| DIA | 5 | 5 | 2000 | 5 | 0 |
+ + --+ --+ --+ --+ --+
| UPD | 0 | 0 | 2000 | 0 | 0 |
+ + --+ --+ --+ --+ --+
| ENQ | 0 | 0 | 2000 | 0 | 0 |
+ + --+ --+ --+ --+ --+
| BTC | 0 | 0 | 2000 | 0 | 0 |
+ + --+ --+ --+ --+ --+
| SPO | 0 | 0 | 2000 | 0 | 0 |
+ + --+ --+ --+ --+ --+
| UP2 | 0 | 0 | 2000 | 0 | 0 |
+ + --+ --+ --+ --+ --+
max_rq_id 12
wake_evt_udp_now 0
wake events total 8, udp 6 ( 75%), shm 2 ( 25%)
since last update total 8, udp 6 ( 75%), shm 2 ( 25%)
Dump of tm_adm structure: Mon Oct 25 08:46:25 2010
=========================
Term uid man user term lastop mod wp ta a/i (modes)
Workprocess Comm. Area Blocks Mon Oct 25 08:46:25 2010
=============================
Slots: 300, Used: 1, Max: 0
+ + --+ -+ -+
| id | owner | pid | eyecatcher |
+ + --+ -+ -+
| 0 | DISPATCHER | -1 | *WPCAAD000* |
NiWait: sleep (5000ms) ...
NiISelect: timeout 5000ms
NiISelect: maximum fd=15
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Mon Oct 25 04:16:30 2010
NiISelect: TIMEOUT occured (5000ms)
DpHalt: shutdown server >bm-sap9_QAS_00 < (normal)
DpJ2eeDisableRestart
DpModState: buffer in state MBUF_PREPARED
NiBufSend starting
NiIWrite: hdl 2 sent data (wrt=110,pac=1,MESG_IO)
MsINiWrite: sent 110 bytes
MsIModState: change state to SHUTDOWN
DpModState: change server state from STARTING to SHUTDOWN
Switch off Shared memory profiling
ShmProtect( 57, 3 )
ShmProtect( key 57 valid )
ShmProtect( slot Index 56 )
ShmProtect( Mode: 0 )
ShmProtect( before shmdt )
ShmProtect( after shmdt )
ShmProtect( before shmat )
ShmProtect( after shmat )
ShmProtect: shmat key 57 prot 3/0 done
ShmProtect(SHM_PROFILE, SHM_PROT_RW
ShmProtect( 57, 1 )
ShmProtect( key 57 valid )
ShmProtect( slot Index 56 )
ShmProtect( Mode: 0 )
ShmProtect( before shmdt )
ShmProtect( after shmdt )
ShmProtect( before shmat )
ShmProtect( after shmat )
ShmProtect: shmat key 57 prot 1/4096 done
ShmProtect(SHM_PROFILE, SHM_PROT_RD
DpWakeUpWps: wake up all wp's
Stop work processes
Stop gateway
killing proc (19843) (SOFT_KILL)
Stop icman
killing proc (19844) (SOFT_KILL)
Terminate gui connections
wait for end of work processes
wait for end of gateway
kill(19843,0) successful -> process alive
waiting for termination of gateway ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=15
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Mon Oct 25 04:16:31 2010
NiISelect: TIMEOUT occured (1000ms)
child zombie with pid 19843 died
kill(19843,0) -> ESRCH: process died
wait for end of icman
kill(19844,0) successful -> process alive
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=15
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Mon Oct 25 04:16:32 2010
NiISelect: TIMEOUT occured (1000ms)
kill(19844,0) successful -> process alive
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=15
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Mon Oct 25 04:16:33 2010
NiISelect: TIMEOUT occured (1000ms)
kill(19844,0) successful -> process alive
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=15
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Mon Oct 25 04:16:34 2010
NiISelect: TIMEOUT occured (1000ms)
kill(19844,0) successful -> process alive
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=15
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Mon Oct 25 04:16:35 2010
NiISelect: TIMEOUT occured (1000ms)
kill(19844,0) successful -> process alive
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=15
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Mon Oct 25 04:16:36 2010
NiISelect: TIMEOUT occured (1000ms)
kill(19844,0) successful -> process alive
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=15
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Mon Oct 25 04:16:37 2010
NiISelect: TIMEOUT occured (1000ms)
kill(19844,0) successful -> process alive
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=15
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Mon Oct 25 04:16:38 2010
NiISelect: TIMEOUT occured (1000ms)
kill(19844,0) successful -> process alive
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=15
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Mon Oct 25 04:16:39 2010
NiISelect: TIMEOUT occured (1000ms)
kill(19844,0) successful -> process alive
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=15
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Mon Oct 25 04:16:40 2010
NiISelect: TIMEOUT occured (1000ms)
kill(19844,0) successful -> process alive
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=15
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Mon Oct 25 04:16:41 2010
NiISelect: TIMEOUT occured (1000ms)
kill(19844,0) successful -> process alive
waiting for termination of icman ...
NiWait: sleep (1000ms) ...
NiISelect: timeout 1000ms
NiISelect: maximum fd=15
NiISelect: read-mask is NULL
NiISelect: write-mask is NULL
Mon Oct 25 04:16:42 2010
NiISelect: TIMEOUT occured (1000ms)
child zombie with pid 19844 died
kill(19844,0) -> ESRCH: process died
DpStartStopMsg: send stop message (myname is >bm-sap9_QAS_00 <)
AdGetSelfIdentRecord: > <
AdCvtRecToExt: opcode 60 (AD_SELFIDENT), ser 0, ex 0, errno 0
AdCvtRecToExt: opcode 4 (AD_STARTSTOP), ser 0, ex 0, errno 0
DpConvertRequest: net size = 189 bytes
NiBufSend starting
NiIWrite: hdl 2 sent data (wrt=562,pac=1,MESG_IO)
MsINiWrite: sent 562 bytes
send msg (len 110+452) to name -, type 4, key -
DpStartStopMsg: stop msg sent
NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key -
DpHalt: received 164 bytes from message server
NiIRead: hdl 2 received data (rcd=274,pac=1,MESG_IO)
NiBufIIn: NIBUF len=274
NiBufIIn: packet complete for hdl 2
NiBufReceive starting
MsINiRead: received 274 bytes
MSG received, len 110+164, flag 1, from MSG_SERVER , typ 0, key - |
| __.____._ Copyright © 2010 Toolbox.com and message author.
Toolbox.com 4343 N. Scottsdale Road Suite 280, Scottsdale, AZ 85251
| | Related Content Most Popular White Papers
In the Spotlight  _.____.__ |
Mark as helpful
