PASE Debug

(click to open)

Quick Page Table of Contents

Scanning…

PASE Debugging

This section of the wiki is intended to provide you access to frequently asked information about PASE debugging.

Sections

locate data after job death (autopsy)

PASE process died with coredump (VLOGS 4700 0012)

When PASE is guilty of dying then often times you will find static data in the VLOGS.

STRSST/STRDST
1. Start a service tool
5. Licensed Internal Code log
1. Select entries from the Licensed Internal Code (LIC) log

Just press enter ...

 Type choices, press Enter.

   Entry ID:
     Starting  . . . . . . . . . . .   FFFFFFFF   00000000-FFFFFFFF

… default options until you see list of vlogs …

    > 0100642A  OS/400 PASE              4700  0100  07/21/11  14:55:15      9
    > 0100642A  OS/400 PASE              4700  0012  07/21/11  14:55:15      9
    > 0100642B  OS/400 PASE              4700  0013  07/21/11  14:55:17      9

    Looking for ...
    4700 0100 - PASE panic (all of pase dies)
    4700 0012 - PASE core dump (pase process dies)

These VLOGs will have PASE tia stack dump for state up to death.

VLOG content top to bottom:
- SLIC stack
- general process info
- exec program name and argv[] data
- signal information / status
- user files (php-cgi or user open files)
- kernel files (loader open files)
- IBM i SLIC stack (IBM i kernel)
- PASE application registers active (php-cgi or tiastate ... tags inactive state)
- PASE application stack (pgp-cgi call stack)
- Last significant kernel boundary state (TiaTDE program fault, etc.)
  • exec program name and argv[] data
 Program: "/usr/local/ZendSvr/bin/php-cgi.bin"    TDE: B00240000952A000
   argv at 2FF22E10
     argv[0] "/usr/local/ZendSvr/bin/php-cgi.bin"
  • signal information / status
   Signal status flags.......0x0000             Suspended Threads.........0
  • user files (php-cgi or user open files)
User File Descriptors    rlim_cur=2000  open_max=65534  fd_limit=25  fd_ile_ma
 |--------|--------|----------|------------------|
 | PASE # | IFS #  |   Flags  |  TiaDescriptor   |
 |--------|--------|----------|------------------|
 |     0  |     0  | 81000000 | D096D10FDF000480 |  VALID SOCKET
 |     1  |     1  | 81000000 | D096D10FDF000500 |  VALID SOCKET
 |     2  |     2  | 81000000 | D096D10FDF000580 |  VALID SOCKET
 |     3  |    23  | 80800000 | D096D10FDF000600 |  VALID FILE /www/zendsvr/log
 |     4  |   100  | 80800000 | D096D10FDF000700 |  VALID FILE /usr/local/zends
 |     5  |   103  | 80800000 | D096D10FDF000800 |  VALID FILE /usr/local/zends
 |     6  |   104  | A0800000 | D096D10FDF000900 |  VALID CLOEXEC FILE /usr/loc
 |     7  |   105  | 80800000 | D096D10FDF000A00 |  VALID FILE /tmp/zshm_ZShmSt
  • kernel files (loader open files)
 Kernel File Descriptors    rlim_cur=4096  open_max=4096  fd_limit=84  fd_ile_m
 |--------|--------|----------|------------------|
 | PASE # | IFS #  |   Flags  |  TiaDescriptor   |
 |--------|--------|----------|------------------|
 |     0  |    24  | 80C00000 | D096D10FDF001900 |  VALID FILE KERNEL /usr/lib/
 |     1  |    25  | 80C00000 | D096D10FDF001A00 |  VALID FILE KERNEL /usr/lib/
  • IBM i SLIC stack (IBM i kernel)
 ISF=FFB4FA1F62FFE8B0 NIA=FFFFFFFFB806D2A8 module LoSocketApiHighUse+0x2A18
 ISF=FFB4FA1F62FFEBF0 NIA=FFFFFFFFC2C3F864 module TiaSyscallSocket+0xCB4
 ISF=FFB4FA1F62FFEDD0 NIA=FFFFFFFF810D1424 module tiaasm+0x5A4
 ISF=FFB4FA1F62FFF160 NIA=FFFFFFFF810D0F4C module tiaasm+0xCC
 ISF=FFB4FA1F62FFF300 NIA=FFFFFFFFC31F7164 module PpPaseMIInterface+0x32E4
  • PASE application registers active (php-cgi or tiastate … tags inactive state)
 -------------------------------------------------
 Interrupt save state (from TiaTde)...
 TiaSaveState: FFB4FA1F62FFEE10    TDE: B00240000952A000
   errno=0/0x0    errnopp=20093744    errnop=2FF22FF8
   Flags=8804  SYSCALL_STATE NEED_UNWIND FPEU
     MSR=000000000000F032  32_BIT TAGS_INACTIVE PROBLEM_MODE
     IAR=0000000000003608 <syscall32>:naccept  saved GPR2=20074E64
      LR=000000001046AA48 fcgi_accept_request+0x194
     CTR=0000000000003600  CR=22200442  FPSCR=A2804000  FPSCRX=00000000  MQ=000
     XER=0000000008000000  FVA=0000000000000000  si_code=8/0x8

   GPR00 not saved         GPR01=000000002FF107A8  GPR02=0000000000000250  GPR0
   GPR04=000000002FF107E0  GPR05=000000002FF20860  GPR06=000000002007FCC4  GPR0
  • PASE application stack (pgp-cgi call stack)
 Tia/Ta ISF=2FF107A8 95FEAE803FF107A8  NIA=1046A924 fcgi_accept_request+0x70
 Tia/Ta ISF=2FF208E0 95FEAE803FF208E0  NIA=10002090 main+0x8B0
 Tia/Ta ISF=2FF22D90 95FEAE803FF22D90  NIA=10000204 __start+0x8C
 Stack unwind complete
  • Last significant kernel boundary state (TiaTDE program fault, etc.)
 ----------------------------
 TiaTde: B00240000952AF30     TDE: B00240000952A000
   Flags=00000000
   SLIC  MSR=C0000000000099B2   GPR1=FFB4FA1F62FFF160 (Anchor stack frame)
   Tia   MSR=000000000000F032   HST=B0002000002A1220   ASR=0000000255AC7001
   Tia   IAR=0000000000003608 <syscall32>
   Tia GPR13=000000002FF22FF8   CR=22200442   FPSCRX=00000000   MQ=00000000
   GPR save1=000000002FF107A8 (Tia LR on fault path)
   GPR save2=0000000000000250 (Tia GPR1 on fault path)
   GPR save3=000000002FF20860 (Tia FVA on fault path)
   thread_userdata=20093680   PASE pthread_self=0x00000001 (assumed 32-bit proc
      Kernel errno=00000000   System time=00000000006B52C2   Last time=00000000
 ----------------------------

For Unix/Linux people debug core file after process death (dbx -W) …

When a PASE VLOG 4700 0012 appears, there will also be a traditional process “core” file dump in the file system. A core file is essentially a capture of process image you may examine state of program as it died (stack, registers, etc.). To find your core file check IFS file system starting location of the main program, which, often times for daemon jobs will be root (ls -l /core).

> ls -l /core
-rw-rw-rw-    1 qtmhhttp 0         102061515 Feb  7 17:35 /core

You can send/debug this core file on another service IBM i as long as you have same binaries installed on the machine (ie. Zend Server 5.6.3 matching core file). To start debugger use dbx -d 100 -W /core , where -W is designed to read core file on multiple machines, and -d 100 allows deep nesting (big programs).

See dbx manual for commands dbx, but a few useful commands:

> help            -- help (help display)
> where           -- stack dump time of die (same as 4700 0012 vlog)
> registers       -- machine registers time of die (same as 4700 0012 vlog)
> map             -- map of all objects loaded by main program (not in 4700 0012 vlog)
> quit            -- done
> 

> dbx -d 100 -W /core
Type 'help' for help.
[using memory image in /core]
reading symbolic information ...warning: sha1.c is newer than /usr/local/ZendSvr/lib/libmysqlclient.a
IOT/Abort trap in pthread_kill at 0xd1cf4a14
0xd1cf4a14 (pthread_kill+0x84) 480008c9          bl   0xd1cf52dc (thread_kill)  

------------------
what program stack time of death -- ZendDBi thread died on PASE SQL connect (this case) 
------------------

(dbx) where
pthdb_session.c, 833: 0 PTHDB_INTERNAL (internal error)
pthreaded.c, 1976: PTHDB_INTERNAL (internal error)
pthread_kill(??, ??) at 0xd1cf4a14
_p_raise(??) at 0xd1cf44b0
raise.raise(??) at 0xd9971888
abort.abort() at 0xd99d0478
SQLSetConnectAttr(??, ??, ??, ??) at 0xd53ff34c
:
_Z20zend_monitor_executeP14_zend_op_array(??) at 0xd21cd498
debugger_execute(??) at 0xdfb49e50
zend_execute_scripts(??, ??, ??) at 0x1000f54c
php_execute_script(??) at 0x10104234
main(??, ??) at 0x1000379c
(dbx) 


------------------
what registers time of death
------------------

(dbx) registers
  $r0:0x0000003a  $stkp:0x2ff1d828   $toc:0x000000b8    $r3:0x00000000  
  $r4:0x00000006    $r5:0x00000000    $r6:0x00000000    $r7:0x0000027a  
  $r8:0x805e1000    $r9:0x22200482   $r10:0x085c8000   $r11:0x00000000  
 $r12:0xd004a040   $r13:0xdeadbeef   $r14:0x00000001   $r15:0x2ff22e04  
 $r16:0x2ff22e0c   $r17:0x00000000   $r18:0x300b8964   $r19:0x2ff1f2b4  
 $r20:0x3009b530   $r21:0x00000000   $r22:0x3009afc8   $r23:0x00000000  
 $r24:0xf1bf8088   $r25:0x00000001   $r26:0x2ff1d9e0   $r27:0xfffffffd  
 $r28:0x30821178   $r29:0xf04ca308   $r30:0x300afe28   $r31:0x00000006  
 $iar:0xd1cf4a14   $msr:0x0000f032    $cr:0x22204482  $link:0xd1cf4a18  
 $ctr:0x00003600   $xer:0x0802d4b6    $mq:0x00000000  
          Condition status = 0:e 1:e 2:e 4:g 5:g 6:l 7:e 
        [unset $noflregs to view floating point registers]
        [unset $novregs to view vector registers]
in pthread_kill at 0xd1cf4a14
0xd1cf4a14 (pthread_kill+0x84) 480008c9          bl   0xd1cf52dc (thread_kill) 

------------------
display some memory in hex or in string format
------------------

(dbx) 0x300b8964 / 10 X
0x300b8964:  2f6d7961 7370322f 7777772f 7a656e64
0x300b8974:  322f6874 646f6373 000b886c 0000003d
0x300b8984:  00000025 1db85847

(dbx) 0x300b8964 / s
0x300b8964: "/myasp2/www/zend2/htdocs"

------------------
disassemble code memory to power instructions 
------------------

(dbx) 0xd1cf52d0 / 4 i
0xd1cf52d0 (fsig+0x24) 7ca30034      cntlzw   r3,r5
0xd1cf52d4 (fsig+0x28) 20630040      subfic   r3,r3,0x40
0xd1cf52d8 (fsig+0x2c) 4e800020         blr
0xd1cf52dc (thread_kill)     818201c4         lwz   r12,0x1c4(r2)
(dbx) 


> quit   

capture data while job running (real time)

When you have no idea (dumping many processes)?

Some times you just have no idea what is going on, here is a handy macro to dump a lot of stacks. You should do this activity when your PASE program is behaving badly in effort to “catch it in the act”.

STRSST/STRDST
1. Start a service tool
4. Display/Alter/Dump
1. Display/Alter storage
.... or dump to printer ...
2. Licensed Internal Code (LIC) data
14. Advanced analysis
 Option    Command                                                             
    1      processinfo

In this case dumping all process dealing with keyword "ZEND" appearing in job ...

                       Specify Advanced Analysis Options                       
 Output device  . . . . . . :   Display
 Type options, press Enter.
   Command . . . . :   PROCESSINFO
   Options . . . . .   -NAMES ZEND                                             

Note:
Information dumped printer/display is same as paseps/vlog macro (below).
If not using standard Zend Server installation, but using custom Apache set-up, then change -NAMES ZEND to -NAMES something-else

Reduced information from wrkactjob …

wrkactjob

 Type options, press Enter.                                         
   2=Change   3=Hold   4=End   5=Work with   6=Release   7=Display m
   8=Work with spooled files   13=Disconnect ...                    
                     Current                                        
 Opt  Subsystem/Job  User        Type  CPU %  Function        Status
      QUSRWRK        QSYS        SBS      .0                   DEQW 
 5      QP0ZSPWP     QSECOFR     BCI      .0  PGM-sshd         SELW 


     11. Display call stack, if active                                                 
 Selection or command                            
 ===> 11                                         

In this case sshd daemon is waiting in unix kernel system call select (waiting for work) ...

       QP2USER2   QSYS          6                 runpase_main__FPi   
       QP2USER2   QSYS          5                 runpase_common__FiPv
  P    sshd                             0000006C  __start             
  P    sshd                     1906    00001940  main                
  P    sshd                     1138    00000278  server_accept_loop  
  P    sshd                     232     00000038  select              
  P    libc.a(shr.o)                    000000CC  __fd_select         
  P    unix                             00000008  <syscall32>:_select    

PASE parent job, child job, zombie job

Sometimes useful to understand PASE job hierarchy parent job, child job, zombie job (child job ended, not reaped exit status).

STRSST/STRDST
1. Start a service tool
4. Display/Alter/Dump
1. Display/Alter storage
.... or dump to printer ...
2. Licensed Internal Code (LIC) data
14. Advanced analysis
 Option    Command                                                             
    1      ps

Output:

 Running macro: PS                                                            
 Pxpm (anchor) address: DC4C54263B0CFF80      DECIMAL pid/pgrp/sid values     
    Active: 64    Zombie: 1    Total: 672    PID generator: 2924              
:
 pid: 41   ppid: 3   pgrp: 3   sid: 0   flags: 10000000 NOSIGCLD_STOPPED      
    pcs.............F8F43C1196000000   initial tde.....B00020000AAE8000    QRM
    entry address...E741C1BDF01FFDC0   exit_status.....00000000   session_coun
    tty_info........0000000000000000   zombie_info.....0000000000000000       
    children:                                                                 
       pid: 50   ppid: 41   pgrp: 3   sid: 0   flags: 00000000                
       pid: 47   ppid: 41   pgrp: 3   sid: 0   flags: 04000000 PASE_ACTIVE    
       pid: 46   ppid: 41   pgrp: 3   sid: 0   flags: 00000000                
       pid: 43   ppid: 41   pgrp: 3   sid: 0   flags: 00000000                

Leaking memory, want to snoop PASE memory?

Some times a look into running PASE process memory can be very insightful for understanding issues (like leaking memory).

STRSST/STRDST
1. Start a service tool
4. Display/Alter/Dump
1. Display/Alter storage
.... or dump to printer ...
2. Licensed Internal Code (LIC) data
14. Advanced analysis
 Option    Command                                                             
    1      dspmem

                       Specify Advanced Analysis Options                       
 Output device  . . . . . . :   Display
 Type options, press Enter.
   Command . . . . :   DSPMEM
   Options . . . . .   20000000 -t B00240000952AF30 -tia  

Where:
20000000 - hex location of PASE address 32bit or 64bit addresses (see address map bottom this page)
-t TDE   - TDE of the pase process as seen paseps or processinfo macro 
-tia     - tags inactive addresses (PASE addresses 32 bit or 64 bit)                                       

A big hint ... these PASE kernel addresses MAY change, but ...
0xF00000002FF3BEC8 U_minbreak
0xF00000002FF3BED0 U_breakhiwater       <-- how much heap has leaked so far ???

How to find PASE shared libraries (tiale)

tiale is a special PASE provided macro tool available from STRSST only. It will dump all the shared libraries loaded on the machine since PASE IPL (libc.a, etc.) and can tell you where all the shared program text is loaded by address (very handy). You can do everything from the 5250 screen (green screen), but you MUST have the password to STRSST to access the paseps tool (macro). The tiale macro/tool runs from D/A/D service tool in STRSST.

On the customer machine you may be able to see ownership by looking at the PASE shared library load map (PASE loader IPL data) ...
STRSST/STRDST
1. Start a service tool
4. Display/Alter/Dump
1. Display/Alter storage
2. Licensed Internal Code (LIC) data
14. Advanced analysis
 Option    Command                                                             
    1      tiale                           
 Type options, press Enter.
   Command . . . . :   TIALE
   Options . . . . . 

Example output (64bit 0900000xxxxxxx):
                             Display Formatted Data                            
                                                 Page/Line. . .      40 /  42
                                                 Columns. . . :  1 - 78
 Find . . . . . . . . . . .  090000000                       
 ....+....1....+....2....+....3....+....4....+....5....+....6....+....7....+...
     le: F100009D80049700        "shr_64.o"  "/usr/lib/libcrypt.a"
       le_flags: LE_DATA 000800C0
       le_usecount: 1   le_loadcount: 0   le_ndepend: 2   le_maxdepend: 2   f_c
       le_fp:            0000000001064160  le_filename:      F100009D80049498
       _le_file_union:   09000000002D9280  le_filesize:      0000000000000A2B
===> functions from 09000000002D9280 (+A28) to 9000000002D9CAB

     le: F100009D80049200        "shr_64.o"  "/usr/lib/libc.a"
       le_flags: LE_TEXT 00080882
       le_usecount: 1   le_loadcount: 0   le_ndepend: 0   le_maxdepend: 1   f_c
       le_fp:            0000000001064180  le_filename:      F100009D80049298
       _le_file_union:   0900000000003F00  le_filesize:      00000000002D47F2
       _le_data_union:   090000000022A208  le_datasize:      0000000000067C90
===> functions from 0900000000003F00 (+2D47F2) to 9000000002D86F2
Note:
a) _le_file_union may also have label xxx_text 
b)  _le_file_union + le_filesize = all functions in this module
    for example so you will know if GETPLATFORMCPUTIME__FPV  is an IBM module
    NIA=090000000181961C GETPLATFORMCPUTIME__FPV+0X1F0
    because /usr/lib is IBM shipped (just example)

How to find active PASE job stack (paseps)

paseps is a special PASE provided macro tool available from STRSST only. It is essentially UNIX ps on steroids complete with open files, current call stack, etc. You can do everything from the 5250 screen (green screen), but you MUST have the password to STRSST to access the paseps tool (macro). The paseps macro/tool runs from D/A/D service tool in STRSST.

Macros processinfo and paseps are every day PASE toolkit items for PASE experts, most of the time active PASE processes that are misbehaving can be caught in the act right here.

On the customer machine you may be able to see ownership by looking at the PASE shared library load map (PASE loader IPL data) ...
call qp2term
 > ps -ef
        UID   PID  PPID   C    STIME    TTY  TIME CMD
    qsecofr    10     7   0   May 07      -  1:41 /QOpenSys/QIBM/ProdData/Java
   qtmhhttp    46     1   0   May 07      -  0:00 /usr/local/zendsvr/bin/watch
   qtmhhttp    49    46   0   May 07      -  0:00 /usr/local/ZendSvr/bin/jqd -
   qibmhelp    54    53   0   May 07      -  5:05 /QOpenSys/QIBM/ProdData/Java

You need the PID for next step (PID 54)


STRSST/STRDST
1. Start a service tool
4. Display/Alter/Dump
1. Display/Alter storage
2. Licensed Internal Code (LIC) data
14. Advanced analysis
 Option    Command                                                             
    1      paseps                           
 Type options, press Enter.
   Command . . . . :   PASEPS
   Options . . . . . 54
Output (scroll information)
top to bottom:
- general process info
- exec program name and argv[] data
- signal information / status
- user files (php-cgi or user open files)
- kernel files (loader open files)
- IBM i SLIC stack (IBM i kernel)
- PASE application registers active (php-cgi or tiastate ... tags inactive state)
- PASE application stack (pgp-cgi call stack)
- Last significant kernel boundary state (TiaTDE program fault, etc.)

How to read the paseps stack dump …

  • general process info
                             Display Formatted Data                            
                                                 Page/Line. . .       1 /   1
                                                 Columns. . . :  1 - 78
 Find . . . . . . . . . . .                                  
 ....+....1....+....2....+....3....+....4....+....5....+....6....+....7....+...
      DISPLAY/ALTER/DUMP
 Running macro: PASEPS                           19214
 TDE=B00240000952A000
  • exec program name and argv[] data
 Program: "/usr/local/ZendSvr/bin/php-cgi.bin"    TDE: B00240000952A000
   argv at 2FF22E10
     argv[0] "/usr/local/ZendSvr/bin/php-cgi.bin"
  • signal information / status
   Signal status flags.......0x0000             Suspended Threads.........0
  • user files (php-cgi or user open files)
User File Descriptors    rlim_cur=2000  open_max=65534  fd_limit=25  fd_ile_ma
 |--------|--------|----------|------------------|
 | PASE # | IFS #  |   Flags  |  TiaDescriptor   |
 |--------|--------|----------|------------------|
 |     0  |     0  | 81000000 | D096D10FDF000480 |  VALID SOCKET
 |     1  |     1  | 81000000 | D096D10FDF000500 |  VALID SOCKET
 |     2  |     2  | 81000000 | D096D10FDF000580 |  VALID SOCKET
 |     3  |    23  | 80800000 | D096D10FDF000600 |  VALID FILE /www/zendsvr/log
 |     4  |   100  | 80800000 | D096D10FDF000700 |  VALID FILE /usr/local/zends
 |     5  |   103  | 80800000 | D096D10FDF000800 |  VALID FILE /usr/local/zends
 |     6  |   104  | A0800000 | D096D10FDF000900 |  VALID CLOEXEC FILE /usr/loc
 |     7  |   105  | 80800000 | D096D10FDF000A00 |  VALID FILE /tmp/zshm_ZShmSt
  • kernel files (loader open files)
 Kernel File Descriptors    rlim_cur=4096  open_max=4096  fd_limit=84  fd_ile_m
 |--------|--------|----------|------------------|
 | PASE # | IFS #  |   Flags  |  TiaDescriptor   |
 |--------|--------|----------|------------------|
 |     0  |    24  | 80C00000 | D096D10FDF001900 |  VALID FILE KERNEL /usr/lib/
 |     1  |    25  | 80C00000 | D096D10FDF001A00 |  VALID FILE KERNEL /usr/lib/
  • IBM i SLIC stack (IBM i kernel)
 ISF=FFB4FA1F62FFE8B0 NIA=FFFFFFFFB806D2A8 module LoSocketApiHighUse+0x2A18
 ISF=FFB4FA1F62FFEBF0 NIA=FFFFFFFFC2C3F864 module TiaSyscallSocket+0xCB4
 ISF=FFB4FA1F62FFEDD0 NIA=FFFFFFFF810D1424 module tiaasm+0x5A4
 ISF=FFB4FA1F62FFF160 NIA=FFFFFFFF810D0F4C module tiaasm+0xCC
 ISF=FFB4FA1F62FFF300 NIA=FFFFFFFFC31F7164 module PpPaseMIInterface+0x32E4
  • PASE application registers active (php-cgi or tiastate … tags inactive state)
 -------------------------------------------------
 Interrupt save state (from TiaTde)...
 TiaSaveState: FFB4FA1F62FFEE10    TDE: B00240000952A000
   errno=0/0x0    errnopp=20093744    errnop=2FF22FF8
   Flags=8804  SYSCALL_STATE NEED_UNWIND FPEU
     MSR=000000000000F032  32_BIT TAGS_INACTIVE PROBLEM_MODE
     IAR=0000000000003608 <syscall32>:naccept  saved GPR2=20074E64
      LR=000000001046AA48 fcgi_accept_request+0x194
     CTR=0000000000003600  CR=22200442  FPSCR=A2804000  FPSCRX=00000000  MQ=000
     XER=0000000008000000  FVA=0000000000000000  si_code=8/0x8

   GPR00 not saved         GPR01=000000002FF107A8  GPR02=0000000000000250  GPR0
   GPR04=000000002FF107E0  GPR05=000000002FF20860  GPR06=000000002007FCC4  GPR0
  • PASE application stack (pgp-cgi call stack)
 Tia/Ta ISF=2FF107A8 95FEAE803FF107A8  NIA=1046A924 fcgi_accept_request+0x70
 Tia/Ta ISF=2FF208E0 95FEAE803FF208E0  NIA=10002090 main+0x8B0
 Tia/Ta ISF=2FF22D90 95FEAE803FF22D90  NIA=10000204 __start+0x8C
 Stack unwind complete
  • Last significant kernel boundary state (TiaTDE program fault, etc.)
 ----------------------------
 TiaTde: B00240000952AF30     TDE: B00240000952A000
   Flags=00000000
   SLIC  MSR=C0000000000099B2   GPR1=FFB4FA1F62FFF160 (Anchor stack frame)
   Tia   MSR=000000000000F032   HST=B0002000002A1220   ASR=0000000255AC7001
   Tia   IAR=0000000000003608 <syscall32>
   Tia GPR13=000000002FF22FF8   CR=22200442   FPSCRX=00000000   MQ=00000000
   GPR save1=000000002FF107A8 (Tia LR on fault path)
   GPR save2=0000000000000250 (Tia GPR1 on fault path)
   GPR save3=000000002FF20860 (Tia FVA on fault path)
   thread_userdata=20093680   PASE pthread_self=0x00000001 (assumed 32-bit proc
      Kernel errno=00000000   System time=00000000006B52C2   Last time=00000000
 ----------------------------

How to run the PASE loader flight recorder (tiafly)

tiafly is a special PASE provided macro tool available from STRSST only. This macro is used to show the PASE kernel loader flight recorder (off/on/dump). You can do everything from the 5250 screen (green screen), but you MUST have the password to STRSST to access the tiafly tool (macro). The tiafly macro/tool runs from D/A/D service tool in STRSST.


tiafly - V6 and beyond (V5R4 has no on/off, can only be done by D/A/D address manipluation)


   1. Start a service tool
   4. Display/Alter/Dump
   1. Display/Alter storage
   …. or dump to printer …   
   2. Licensed Internal Code (LIC) data
   14. Advanced analysis
       Option  Command                                                             
       1      tiafly                     
       Command . . . . :  tiafly
       Options . . . . .   -h  

Output:

 Running macro: TIAFLY                           -H
 This macro dumps global flight recorder information
 Syntax: tiafly [-h, -c, -on, -off]
            -h   == help
            -c   == clear
            -on  == on flight
            -off == off flight

====== scenario ====== 1) Turn on flight

       Option  Command                                                             
       1      tiafly                     
       Command . . . . :  tiafly
       Options . . . . .   -on  

2) run your PASE bad problem (start Zend Server, etc.)

3) Turn off flight (don’t forget Dude)

       Option  Command                                                             
       1      tiafly                     
       Command . . . . :  tiafly
       Options . . . . .   -off

4) Dump the information (no parameters)

       Option  Command                                                             
       1      tiafly                     
       Command . . . . :  tiafly
       Options . . . . . 

Keys to data (do_open all you need 99% time): - pid=nnn - which job (maybe 0 until job far enough alive) - execvex start - start of loader attempt to init program - do_open - most useful data where did the loader look (err=0 good, err=!0 is PASE errno)

 [0] "pid=0 ===== slic_execvex start ===== "
[11] "pid=66129 do_open /QOpenSys/usr/bin/sleep fp=1064020 err=0"

=================== OLD V5R4 machines only ===================

 1. Start a service tool
 4. Display/Alter/Dump
 1. Display/Alter storage
 5. Starting address
 Address . . . . . .  000000000 01E0000
 TDE address . . . .  0000000000 000000
 F10=Tags inactive (need F10 to enter tags inactive D/A/D mode)
 →press enter

1) Turn flight off (default)

                                Display Storage         
 Control . . . . . . .                       nnnnn, Pnnnn, Lcccccc, .cccccc, >
 Address . . . . . . .   000000000 01E0000   Tags inactive
 0000    00000000 00000000      00000000 00000000       * ................ *
             |
             flight recorder is oFF

2) Turn flight on (remember to turn off again)

                                Display Storage         
 Control . . . . . . .                       nnnnn, Pnnnn, Lcccccc, .cccccc, >
 Address . . . . . . .   000000000 01E0000   Tags inactive
 0000    11111111 00000000      00000000 00000000       * ................ *
             |
             flight recorder is on

Reminder: don’t forget F11=Alter storage (default is F11=Display storage)

Tracing PASE syscalls (watching PASE program progress)

At times you want to watch the progress of a running PASE program. One of the easier ways is to watch syscalls. That is, for PASE programs to get anything substantial done, it will make calls into IBM i kernel, so we can watch the progress.

This is range of syscall trace entires.

 Trace Data Format 50000 - 54999 syscall entry (in):
 Data Name               Length      Description
 --------------------    ------      --------------------------------------
 syscallname            varying      System call name
 arguments                   64      Argument registers
 tiacaller                    8      TIA caller address
 syscall#                     8      System call number

 Trace Data Format 55000 - 59999 syscall normal exit (kernel errno zero -- out normal):
 Data Name               Length      Description
 --------------------    ------      --------------------------------------
 syscallname            varying      System call name
 arguments                   64      Argument registers
 rc                           8      Return code #1
 rc2                          8      Return code #2

 Trace Data Format 60000 - 64999 syscall error exit (nonzero errno -- out error):
 Data Name               Length      Description
 --------------------    ------      --------------------------------------
 syscallname            varying      System call name
 arguments                   64      Argument registers
 rc                           8      Return code
 errno                        8      Error number

To watch traces, you must have authority to STRSST.

Start the trace …

STRSST
1. Start a service tool
2. Trace Licensed Internal Code

  F6=Create
               Create Internal Trace
  Trace name . . . . . . .  Where is Waldo
  Buffer size  . . . . . .      30   megabytes   1- 35624
  Wrap allowed . . . . . .  Y        Y=Yes, N=No

 Opt  Trace Name (3=Work with enable details)
  3   WHERE IS WALDO

 Opt   Description  (3=Work with enable details)    
  3    Component traces 

 Opt   Description (1=Enable)
  1    Portable application solutions environment

 Opt  Trace Name  (7=Activate)
  7   WHERE IS WALDO

Run your application …

====== run application (trace) ==========

Stop the trace, look for syscall entries …

1. Start a service tool
2. Trace Licensed Internal Code

 Opt  Trace Name  (8=Deactivate)
  8   WHERE IS WALDO

 Opt  Trace Name  (6=Dump)
  6   WHERE IS WALDO

  F9=Dump all

  Specify time . . . . . . .   N
  Dump device  . . . . . . .   1   (1=Printer)
  Title  . . . . . . . . . .   Where is Waldo
  TDE cross reference  . . .   1

wrksplf
   Opt  File
    5    QPCSMPRT         1  FROG        ADC         875896

Job died, no error job log, no PASE vlog, nothing to see, but, is it PASE?

Part 1 — Obvious answer, job ended normally

Obvious answer, supported by your data (no error in job log)? PASE did not fail, instead, PASE application simply decided to normal exit. One way to verify is to use a PASE trace, and simply look in resulting trace data for _exit, which occurs when PASE application chooses to self terminate. Of course if you find _exit and is correct job (see TDE# example), you will have to contact the owning application programmer and explain this application is choosing to exit (well … best of luck).

STRSST
1. Start a service tool
2. Trace Licensed Internal Code

  F6=Create
               Create Internal Trace
  Trace name . . . . . . .  Where is Waldo
  Buffer size  . . . . . .      30   megabytes   1- 35624
  Wrap allowed . . . . . .  Y        Y=Yes, N=No

 Opt  Trace Name (3=Work with enable details)
  3   WHERE IS WALDO

 Opt   Description  (3=Work with enable details)    
  3    Component traces 

 Opt   Description (1=Enable)
  1    Portable application solutions environment

 Opt  Trace Name  (7=Activate)
  7   WHERE IS WALDO

====== run to fail ==========

1. Start a service tool
2. Trace Licensed Internal Code

 Opt  Trace Name  (8=Deactivate)
  8   WHERE IS WALDO

 Opt  Trace Name  (6=Dump)
  6   WHERE IS WALDO

  F9=Dump all

  Specify time . . . . . . .   N
  Dump device  . . . . . . .   1   (1=Printer)
  Title  . . . . . . . . . .   Where is Waldo
  TDE cross reference  . . .   1

=== data analysis did Mr Waldo simply _exit? ===
wrksplf
   Opt  File
    5    QPCSMPRT         1  FROG        ADC         875896

We are looking for normal exit ...
                                                        Display Spooled File                                                        
 File  . . . . . :   QPCSMPRT                                                                             Page/Line   1743/58       
 Control . . . . .                                                                                        Columns     1 - 130       
 Find  . . . . . .   *_exit                                                                                                               
   AS/400 PASE          IDENTIFIER : PE#50118                         TIME 10/08/14  12:19:20.475 ;:   TDE# 00000000DEEF            
    #1    (    5)    +0000   6D85A789A3                                                              *_exit                                          


We find job matching TDE# 00000000DEEF, found bottom of trace TDE NUMBER.
      00000000DEEF is MI THREAD APACHEDFT QTMHHTTP  875936

                                                        Display Spooled File                                                        
 File  . . . . . :   QPCSMPRT                                                                             Page/Line   1743/58       
 Control . . . . .                                                                                        Columns     1 - 130       
 Find  . . . . . .                                                                                                                  
 *...+....1....+....2....+....3....+....4....+....5....+....6....+....7....+....8....+....9....+....0....+....1....+....2....+....3 
  #3    (   32)    +0000   0000010000000000  0000000000000001   0000000000000000  0000000000000000 *............................... 
      *** LIC TRACE DUMP OUTPUT ***                     WHERE IS WALDO                              10/08/14  12:25:00   PAGE   174 
  TDE NUMBER     TDE ID       TDE ADDRESS        TYPE      NAME                             THREAD ID      PRIMARY TDE NUMBER       
 00000000DC68   00066F9A   B000600006 F9A000   MI THREAD   APACHEDFT QTMHHTTP  875808       0000000001CA                            
 00000000DC7C   00066FFA   B000600006 FFA000   MI THREAD   APACHEDFT QTMHHTTP  875822       000000000485                            
 00000000DEEF   0006780C   B000600007 80C000   MI THREAD   APACHEDFT QTMHHTTP  875936       00000000001A

Part 2 - Alternative answer, unhanded signal

If you cannot find simple PASE syscall _exit, a more sinister scenario does exist, some AIX/Unix/Linux/PASE signals default action may involve silent termination (no error in job log).

=== data analysis did Mr Waldo miss a signal? ===
wrksplf
   Opt  File
    5    QPCSMPRT         1  FROG        ADC         875896

We are looking for signal exit ...
 File  . . . . . :   QPCSMPRT                                                                             Page/Line   1743/58       
 Control . . . . .                                                                                        Columns     1 - 130       
 Find  . . . . . .   PE#00506                                                                                                               

AS/400 PASE          IDENTIFIER : PE#00506                         TIME 10/08/14  15:28:23.443 !    TDE# 00000000DC7C
 #1    (    8)    +0000   0000000000007E3A                                                        *......=.          
 #2    (    8)    +0000   000000000000000F                                                        *........          

where 
  0000000000007E3A is pid (pid 32314 decimal)
  000000000000000F is term signal (SIGTERM 15 decimal)

same trace TDE to JOB match rules as _exit
    TDE NUMBER     TDE ID       TDE ADDRESS        TYPE      NAME                             THREAD ID      PRIMARY TDE NUMBER       
  00000000DC7C   00066FFA   B000600006 FFA000   MI THREAD   APACHEDFT QTMHHTTP  875822       000000000485

general information

CCSID issues

Have CCSID issues? Junk characters? Nothing works?

  • IBM i actions, see CCSID

Locks

Checking your locks.

  • CALL PGM(QP0FPTOS) PARM(*LSTOBJREF ‘/path/to/locked_file’ *FORMAT2)
  • WRKOBJLCK OBJ(ZENDSVR) OBJTYPE(*LIB)

PASE debug address layout

The following layout of any given 32bit (starts 0×10000000 - 7 zeros ) or 64bit (starts 0×100000000 - 8 zeros), is intended to be a quick reference for you to “guess” what the meaning of any Tags Inactive address may mean. It is NOT to taken as completely accurate in that PASE loader controls can change the meaning of some of the address layout (and/or AIX design changes), but at least you will have a general reference for common addresses (most of the time).

Historical information (some details may be inaccurate)...

AIXLoader in priviledged mode:

  AIXLoader compiled              PASE SLIC and
  PASE.ruhx                       XPF running
  running in PASE tia             in TA 64-bit
  priviledged mode

                          | S |  ----------------
                          | Y |  | XPF (LIBC.A) |
                          | S |  ================ MI
  |----------|---------|->| C |->| SLIC PASE    |
  | aix       pase     |<-| A |<-| ------------ |
  | program   libc.a   |  | L |  | | XLoader: | |
  |----------|---------|  | L |  | | static:  | |
  ===============================| | -isok    | |
  priviledged mode boundary      | | -syscalls| |
  ===============================| ------------ |
  |----------|---------|  | S |  | | pasesc_  | |
  |           kernel   |  | Y |  | |  brk     | |
  | aix       interface|  | S |  | |  sbrk    | |
  | ported    wrapper  |->| C |->| |  load    | |
  | loader    to SLIC  |<-| A |<-| |  etc.    | |
  | (/PASE)   PASE     |  | L |  | ------------ |
  |----------|---------|  | L |  ----------------

*********************************************************************************
*********************************************************************************
*********************************************************************************
Entire address space key elements
                           ======USER SHADOW REGION 32ALL START
      0x0000000000000000LL ------------------------------------------------------ 
                           |PASE.ruhx (low.s start at zero)
                           | zeros
      0x0000000000001000LL |----------------------------------------------------|
                           | unused
      0x0000000000003000LL |----------------------------------------------------|
                           |mulh_addr,	0x3100
                           |mulh_size,	0x0080
                           |mull_addr,	0x3180
                           |mull_size,	0x0080
                           |divss_addr,	0x3200
                           |divss_size,	0x0080
                           |divus_addr,	0x3280
                           |divus_size,	0x0080
                           |quoss_addr,	0x3300
                           |quoss_size,	0x0080
                           |quous_addr,	0x3380
                           |quous_size,	0x0080
                           |_clear_lock_addr,	0x3400
                           |_clear_lock_size,	0x0010
                           |_clear_lock_mem_addr,	0x3410
                           |_clear_lock_mem_size,	0x0010
                           |_check_lock_addr,	0x3420
                           |_check_lock_size,	0x0040
                           |cs_addr,	        0x3460
                           |cs_size,	        0x0040
                           |----------------------------------------------------|
                           |Trconflag_addr,         0x34C4 (Trconflag)
                           |----------------------------------------------------|
                           |_syscall32_override_addr, 0x3500 (obsolete)
                           |----------------------------------------------------|
                           |__extension_status_addr, 0x35D0 (__extension_status)
                           |----------------------------------------------------|
                           |_syscall32_addr,        0x3600 (svc_instr)
                           |----------------------------------------------------|
                           |_syscall64_addr,        0x3610 (svc64_instr)
                           |----------------------------------------------------|
                           |_syscall32_fast_addr,   0x3620 (fast_sc)
                           |----------------------------------------------------|
                           |_sig_redrive_addr,      0x3680  (sigredrive)
                           |_ldrexit_addr,          0x36C0  (ldrexit)
                           |_kernelrtn_addr,        0x36d0  (kernelrtn)
                           |_callback_rtn_addr,     0x36F0  (callbackrtn)
      0x0000000000004000LL |----------------------------------------------------|
                           |_system_configuration_addr, 0x4000 (_system_configuration)
                           |_system_TB_config_addr, 0x4100  (_system_TB_config)
      0x0000000000005000LL |----------------------------------------------------|
                           |priv_syscall_addr, 0x5000
      0x0000000000007000LL |----------------------------------------------------|
                           |fwnmi_reserv_addr,	0x7000
                           |fwnmi_reserv_size,	0x1000
      0x0000000000008000LL |----------------------------------------------------|
                           |nonpriv_reserv_addr,	0x8000
                           |nonpriv_reserv_size,	0x1000
      0x0000000000009000LL |----------------------------------------------------|
                           |fetch_and_add_addr,	0x9B00
                           |fetch_and_add_size,	0x0060
                           |set_cpu_priv_addr,	0x9b80
                           |set_cpu_priv_size,	0x0080
                           |update_tuserdata_addr,	0x9c00
                           |update_tuserdata_size,	0x0080
      0x000000000000A000LL |----------------------------------------------------|
                           |compare_and_swap_addr,	0xA4C0
                           |compare_and_swap_size,	0x0100
                           |test_and_set_addr,	0xA5C0
                           |test_and_set_size,	0x0100
                           |fetch_and_or_addr,	0xAEC0
                           |fetch_and_or_size,	0x0060
                           |fetch_and_nop_addr,	0xAF20
                           |fetch_and_nop_size,	0x0060
                           |fetch_and_and_addr,	0xAF80
                           |fetch_and_and_size,	0x0060
      0x000000000000B000LL |----------------------------------------------------|
                           |fetch_and_add_h_addr,	0xB000
                           |fetch_and_add_h_size,	0x0100
                           |fetch_and_addlp_addr,	0xb580
                           |fetch_and_addlp_size,	0x0080
                           |fetch_and_andlp_addr,	0xb800
                           |fetch_and_andlp_size,	0x0080
                           |fetch_and_orlp_addr,	0xb880
                           |fetch_and_orlp_size,	0x0080
                           |compare_and_swaplp_addr, 0xb900
                           |compare_and_swaplp_size, 0x0080
                           |test_and_setlp_addr,	0xb980
                           |test_and_setlp_size,	0x0080
      0x000000000000C000LL |----------------------------------------------------|
                           |thread_data_sprg3_addr, 0xC000 (XL_THREAD_DATA_SPRG3)
                           |svc_table_addr,         0xC010
                           |svc_table_entries_addr, 0xC018
                           |svc64_table_addr,       0xC020
                           |svc64_table_entries_addr, 0xC028
      0x000000000000D000LL |----------------------------------------------------|
                           |memcmp_addr,	0xD000
                           |memcmp_size,	0x200
                           |memcmp64_addr,	0xD200
                           |memcmp64_size,	0x200
                           |strstr_addr,	0xD400
                           |strstr_size,	0x200
                           |strstr64_addr,	0xD600
                           |strstr64_size,	0x200
                           |memccpy_addr,	0xD800
                           |memccpy_size,	0x200
                           |memccpy64_addr,	0xDA00
                           |memccpy64_size,	0x200
      0x000000000000E000LL |----------------------------------------------------|
                           |bzero_addr,	0xE000
                           |bzero_size,	0x8
                           |memset_addr,	0xE008
                           |memset_size,	0x7F8
                           |bzero64_addr,	0xE800
                           |bzero64_size,	0x8
                           |memset64_addr,	0xE808
                           |memset64_size,	0x7F8
      0x000000000000F000LL |----------------------------------------------------|
                           |memmove_addr,	0xF000
                           |memmove_size,	0x400
                           |memmove64_addr,	0xF400
                           |memmove64_size,	0x400
                           |fill_addr,	0xF800
                           |fill_size,	0x200
                           |fill64_addr,	0xFA00
                           |fill64_size,	0x200
      0x0000000000010000LL |----------------------------------------------------|
                           |callback_table_addr,          0x21000
                           |genesis_callback_table_size,  0x1000
      0x0000000000011000LL |----------------------------------------------------|
                           |region_table_addr,            0x20000
                           |region_table_size,            0x1000
      0x0000000000012000LL |----------------------------------------------------|
                           |kernel_header_addr,  0x12000
                           |kernel_header_size,  0x400
                           |-----------
                           |kernel_shlapheader_addr,  0x12400
                           |kernel_shlapheader_size,  0x400
      0x0000000000013000LL |----------------------------------------------------|
                           |ppda_addr,               0x13000
      0x0000000000014000LL |----------------------------------------------------|
                           |unused
      0x0000000000015000LL |----------------------------------------------------|
                           |pnda_addr,               0x15000
      0x0000000000016000LL |----------------------------------------------------|
                           |unused
      0x0000000000017000LL |----------------------------------------------------|
                           |vmker_addr,              0x17000
      0x0000000000018000LL |----------------------------------------------------|
                           |unused
      0x0000000000019000LL |----------------------------------------------------|
                           |v                        0x19000 struct kernvars
      0x000000000001a000LL |----------------------------------------------------|
                           |unused thru 1ffff
      0x0000000000020000LL |----------------------------------------------------|
                           |Remainder of PASE.ruhx image (other than low.s)
                           |text, string table, etc.
      0x00000000000C0000LL |----------------------------------------------------|
                           |Data for PASE.ruhx
      0x0000000000100000LL |----------------------------------------------------|
                           |shlap.ruhx
                           |-----------
                           |text, data, and bss
      0x0000000000120000LL |----------------------------------------------------|
                           |unused (growth for shlap.ruhx)
      0x00000000001E0000LL |--------
                           | flight_on          0x00000000001E0000 (inline check)
                           | flight_priority    0x00000000001E0004
                           |   flight_critical  0x10
                           |   flight_important 0x40
                           |   flight_verbose   0x100
                           | (see kernel/sys/syspest.h and kernel/db/vdbprf.c) 
      0x0000000000FFF000LL |----------------------------------------------------|
                           | start 2nd segment (AIX heap start = 1e00000)
                           | (16mb-4k)
                           | PASE heap start
                           | endcomm (XL_ld.imp)
                           |----------------------------------------------------|
      =====PASE_DEBUG ONLY==================
                           |----xmalloc() variables PASE_DEBUG
                           |XMDBG_recs	          0x000000000C000000 (ipl64.imp)
                           | Note: This area is inside the heap for the kernel
                           | so if enough xmallocs occur data corruption will
                           | result. This is only the case when xmdbg is on
                           | as a compile option PASE_DEBUG (see xmpin.c).
      =====PASE_DEBUG ONLY==================
      ---------------------------------------------------------------------------
      0x0000000010000000LL |----------------------------------------------------|
                           | text 32
      0x0000000020000000LL |----------------------------------------------------|
                           | data 32
                           |environ         0x2FF22FF4 (ipl64.imp)
                           |_environ        0x2FF22FF4 (ipl64.imp)
                           |errno           0x2FF22FF8 (ipl64.imp)
                           |_errno          0x2FF22FF8 (ipl64.imp)
                           |errnop          0x2FF22FFC (ipl64.imp)
                           |_errnop         0x2FF22FFC (ipl64.imp)
      0x0000000030000000LL |----------------------------------------------------|
                           | heap/shmat 32 3-A
      0x00000000D0000000LL |----------------------------------------------------|
                           | shared lib text 32
      0x00000000F0000000LL |----------------------------------------------------|
                           | shared lib data 32
                           |----------------------------------------------------|
                           ======USER SHADOW REGION 32ALL END
      ---------------------------------------------------------------------------
                           ======USER SHADOW REGION 64TEXT START
      0x0000000100000000LL |----------------------------------------------------|
                           | text 64
                           | struct loader_anchor64 {
                           |   LOADER_ANCHOR anchor64
                           |   vmid_t text_sids[LDR_MAX_EXEC_SEGMENTS]
                           | }
                           | TEXTORG64 (0x0000000100000000LL)
                           | region LDR_MAX_EXEC_SEGMENTS 8 (0x0000000180000000)
                           ======USER SHADOW REGION 64TEXT END   
                           ======REGION START
      0x0000000180000000LL |----------------------------------------------------|
                           | data/heap 64
                           | -----------
                           | XL_DATAORG64 START   ((ptr64)0x0000000180000000LL)
                           | limit calculation
                           | U.U_datamax = (0x100000000LL)
                           |               - (unsigned long long) start
                           | region ADC_MIN_REGION_TEXT_DATA_SEGMENTS 16  (0x0000000280000000LL)
                           | region java low                              (0x0000000300000000LL)
                           | region java high - 1                         (0x0000000800000000LL)
                           | region ADC_MAX_REGION_TEXT_DATA_SEGMENTS 256 (0x0000001180000000LL)
                           ======REGION END   
                           ======REGION START
      0x0000001180000000LL |----------------------------------------------------|
                           | XL_HEAPORG64 START   ((ptr64)0x0000001180000000LL)
                           | normal unused (aix limit)    
                           | U.U_datamax = (0x70000000LL << SEGSHIFT)
                           |               - (unsigned long long) start
                           |   default region is 256 segments   (0x0000001180000000LL)
                           |   LDR64_NUM_SHDATA_SEGS 16 (0x00000012E0000000LL))
                           | env var LDR_CNTRL=MAXDATA= can set higher value
                           ======REGION END   
                          ======REGION START
      0x0700000000000000LL |----------------------------------------------------|
                           | U.U_datamax   = (0x70000000ULL << SEGSHIFT) - U.U_datastart;
                           | shared memory region start
                           |   default region is 256 segments  
                           |   ADC_MAX_REGION_SHRM_DATA_SEGMENTS 256
      0x0700001000000000LL |----------------------------------------------------|
                           | env var XL_PASE_MAXSHR64_STRING cca set the nbr
                           | of segments to higher value
                           ======REGION END   
                           ======USER SHADOW REGION 64PLOADTEXT START
      0x0800000000000000LL |----------------------------------------------------|
                           | priv text 64
                           | -----------
                           | PLOADTEXTORG64 START ((cptr64)0x0800000000000000LL)
                           | PLOADTEXTORG64 END   ((cptr64)0x0800000100000000LL)
                           | region PLOADTEXT_NUM_SEGS 16
                           ======USER SHADOW REGION 64PLOADTEXT END   
                           ======REGION START
      0x08001000A0000000LL |----------------------------------------------------|
                           | priv data 64
                           | -----------
                           | PLOADDATAORG64 START ((cptr64)0x08001000A0000000LL)
                           | PLOADDATAORG64 END   ((cptr64)0x08001001A0000000LL)
                           | region PLOADDATA_NUM_SEGS 16
                           ======REGION END   
                           ======REGION START
      0x0800200140000000LL |----------------------------------------------------|
                           | usla data heap
                           | -----------
                           | USLA_HEAPORG64 START ((ptr64)0x0800200140000000LL)
                           | USLA_HEAPORG64 END   ((ptr64)0x0800200240000000LL)
                           | region USLA_HEAP_NUM_SEGS 16
                           ======REGION END   
      0x0800200240000000LL |----------------------------------------------------|
                           | unused
                           ======USER REGION SHTEXTDSA START
      0x08FFFFFFD0000000LL |----------------------------------------------------|
                           | SHLIB32_SHTEXTDSA 0x08FFFFFFD
                           ======USER SHADOW REGION SHTEXTDSA  END   
                           ======USER REGION 32SHRLE START
      0x08FFFFFFE0000000LL |----------------------------------------------------|
                           | SHLIB_LE_SEG 0x08FFFFFFE
                           ======USER SHADOW REGION 32SHRLE  END   
                           ======USER SHADOW REGION 64PRIVLE START
      0x08FFFFFFF0000000LL |----------------------------------------------------|
                           |UPAGESORG64     0x08FFFFFFF0000000
                           ======USER SHADOW REGION 64PRIVLE END   
                           ======USER SHADOW REGION 64SHTEXT START
      0x0900000000000000LL |----------------------------------------------------|
                           | shlib 64
                           | -----------
                           | SHLIB_REGION_START   ((ptr64)0x0900000000000000LL)
                           | define LDR64_NUM_SHTEXT_SEGS 16 
                           | extern struct library_64 {
                           |  :
                           |  vmid_t shlib_text_sid[LDR64_NUM_SHTEXT_SEGS]
                           |  vmid_t shlib_data_sid[LDR64_NUM_SHDATA_SEGS]
                           |  vmid_t shlap_heap_sid[LDR64_NUM_SHLAP_HEAP_SEGS]
                           | :
                           | } ldr64
                           |
                           | shlib text 64
                           | -----------
                           | SHTEXTORG64 START    ((ptr64)0x0900000000000000LL)
                           | SHTEXTORG64 END      ((ptr64)0x0900000160000000LL)
                           ======USER SHADOW REGION 64SHTEXT END   
      0x0900000170000000LL |----------------------------------------------------|
                           | unused
                           ======USER SHADOW REGION 64SHDATA START
      0x09001000A0000000LL |----------------------------------------------------|
                           | shlib data 64
                           | -----------
                           | SHDATAORG64 START    ((ptr64)0x09001000A0000000LL)
                           | SHDATAORG64 END      ((ptr64)0x0900100200000000LL)
                           | define LDR64_NUM_SHDATA_SEGS 16
                           ======USER SHADOW REGION 64SHDATA END   
      0x0900100210000000LL |----------------------------------------------------|
                           |unused
                           ======USER SHADOW REGION REGION 64SHLE START
      0x09FFFFFFE0000000LL |----------------------------------------------------|
                           |SHLIB_LE_ORG64 0x09FFFFFFE0000000LL
                           |SHLIB_REGION_END = SHLIB_LE_ORG64
                           ======USER SHADOW REGION REGION 64SHLE END   
      0x09FFFFFFF0000000LL |----------------------------------------------------|
                           |USLAORG64 0x09FFFFFFF0000000LL
      0x0A00000000000000LL |----------------------------------------------------|
                           |unused
      0x0F00000000000000LL |----------------------------------------------------|
                           | STKFLOOR64 (0x0F00000000000000ull)
                           |unused
                           ======REGION START
      0x0FFFFFFFF0000000LL |----------------------------------------------------|
                           | user/usla stacks 64
                           | -----------
                           |environ64	0x0FFFFFFFFFFFFFD8 (ipl64.imp)
                           |_environ64	0x0FFFFFFFFFFFFFD8 (ipl64.imp)
                           |errno64	0x0FFFFFFFFFFFFFE0 (ipl64.imp)
                           |_errno64	0x0FFFFFFFFFFFFFE0 (ipl64.imp)
                           |errnop64	0x0FFFFFFFFFFFFFE8 (ipl64.imp)
                           |_errnop64	0x0FFFFFFFFFFFFFE8 (ipl64.imp)
                           |XL_USER64_STACK_TOP (0x1000000000000000ULL)
                           |VM_MAXADDR ((unsigned)0x0fffffff)
                           |STKTOP64 ((unsigned long long)(VM_MAXADDR + 1) << 32)
                           |INITSTACK64 (STKTOP64)
                           |MAXSTACK64 (SEGSIZE * 16ll)
                           |U.U_stkmax = STKTOP64 - STKFLOOR64 (exec_mkdatastk64)
                           | pase has 1 segment stack user (not 16 segments)
                           ======REGION END   
      0x0FFFFFFFFFFFFFFFLL |----------------------------------------------------|

*********************************************************************************
*********************************************************************************
*********************************************************************************
             =========================================================
             32 bit only call to user program
             =========================================================
             Figure 3.3: DATAORG/PRIVORG
                      DATAORG/PRIVORG(2)
             20000000 |------------------|---
                      | program data     |  | private load
                      | defined in       |  | and data (sbreak)
                      | xcoff header     |  |
                      |------------------|---
                      | user heap        |  | sbreak storage
                      | (libc.a malloc)  |  | non-huge model
                      |  huge use        |  | 
                      |  -bmaxdata:bytes |  |
   (default) 2FF0A000 |------------------|--- stack bottom 2
                      | user stack       |  | user stack part 2
                      |  normal default  |  | with default
                      |    0x18000       |  | or aouthdr.o_maxstack
                      | -bmaxstack:bytes |  |
                      |                  |  |
             2FF22F28 |------------------|---
                      |                  |
             2ff22ff4 |------------------|
                      | environ          |
             2ff22ff8 |------------------|
                      | errno            |
             2ff22ffc |------------------|
                      | errnop           |
             2ff23000 |------------------|--- STACKTOP32
                      | unused           |   
             2ff45000 |------------------|---
                      | kernel heap      |  | kernel/loader heap
                      | (xmalloc)        |  | setup in ld_memory.c
                      |                  |  | ld_uinitheap()
                      |                  |   
             2fffffff |------------------|


             struct tos32 {
             2FF22F70 |-----------------------------|
                      |ptr32_aix main_reg[NGPRS_AIX]| (NGPRS == 32)
             2FF22FF0 |-----------------------------|
                      |ptr64 lr                     |
             2FF22FF4 |-----------------------------|
                      |ptr64 environ                |
             2ff22ff8 |-----------------------------|
                      |int errno                    | (assume errno is anchor)
             2FF22FFC |-----------------------------|
                      |ptr64 errnop                 |
             200FEFF0 |-----------------------------|
             }


             ========================================================
             64 bit only call to usla
             =========================================================
             struct tos64 {
             0FFFFFFFFFFFFED0 |---------------------|
                              |ptr64 main_reg[NGPRS]| (NGPRS == 32) 0x100
             0FFFFFFFFFFFFFD0 |---------------------|
                              |ptr64 lr             |
             0FFFFFFFFFFFFFD8 |---------------------|
                              |ptr64 environ        |
             0FFFFFFFFFFFFFE0 |---------------------|
                              |int errno            | (assume errno is anchor)
             0FFFFFFFFFFFFFE4 |---------------------|
                              |uint reserved1       |
             0FFFFFFFFFFFFFE8 |---------------------|
                              |ptr64 errnop         |
             0FFFFFFFFFFFFFF0 |---------------------|
                              |ptr64 usla_cblk      | 0800200140000000
             0FFFFFFFFFFFFFF8 |---------------------|
                              |ulonglong reserved3  |  
             1000000000000000 |---------------------|
             }

*********************************************************************************
*********************************************************************************
*********************************************************************************

Author(s)

Tony “Ranger” Cairns - IBM i PHP / PASE