Service

under construction

  • download tools
    • pigstack - PASE shell utility to dump stack of high CPU PASE job
    • cli - PASE DB2 CLI trace library
  • on-line tools
    • vlog - on-line PASE VLOG analysis tool (cut/paste)
  • Common
    • CCSID 65535 - PASE convert issues system default CCSID 65535
    • broken - PASE destroyed by SAV/RST
    • geek - PASE debug tips form the pros
  • more great tools and techniques

Download tools

CPU pig process

Sometimes a PASE process is eating CPU (option 1). Sometime want to see what a PASE job is doing (option 2).

  • Attach:pigstack103c.zip 2015–09–11 (version 103c)
     pigstack [pid]
       pid - monitor pid (ps -ef)
      !pid - monitor high CPU (ps -eo pcpu,pid,user,args)
     ENV VARS:
       PIGS_CPU - greater percent CPU (export PIGS_CPU=10.0)
       PIGS_SEC - check idle sleep interval (export PIGS_SEC=10)
    
  • version 103c (2015–09–11)
    • NEW loop check based on env vars PIGS_CPU and PIGS_SEC
    • NEW loop over time PIGS_SEC job CPU actual multiplier (closer to wrkactjob F5, not F10)
  • version 102a (2015–06–26)
    • NEW pigstack dumps ALL thread stacks (Ruby/Java guys)
    • NEW pigstack allows pid attach

Installation

> unzip pigstacknnn.zip
> ftp pigstack to /QOpenSys/usr/bin/pigstack (binary ftp)

on IBM i
call qp2term
> chmod +x /QOpenSys/usr/bin/pigstack
> pigstack

As of version 103, pigstack loops looking for a job greater than PIGS_CPU (either option below). Once %CPU > PIGS_CPU found, dbx attach may take some time, possibly minutes, so do not overreact and kill everything before attach/dump has a chance.

Option 1) pigstack find a CPU using jobs

$ pigstack
pigstack for high CPU, PIGS_CPU>10.0 at PIGS_SEC=10 second interval (version 103)

Option 2) attach to a PASE pid/job

find a job you like ...
$ ps -eo pcpu,pid,user,args
 %CPU   PID     USER COMMAND
  0.1  2469 qtmhhttp /usr/local/zendsvr6/bin/zsd /usr/local/zendsvr6/etc/zsd.ini -e

run pigstack to see what is going on ...
$ pigstack 2469
pigstack for pid=2469, PIGS_CPU>0.0 at PIGS_SEC=5 second interval (version 103)
**************************************
  0.1  2469 qtmhhttp /usr/local/zendsvr6/bin/zsd /usr/local/zendsvr6/etc/zsd.ini -e
**************************************
print stack   0.1  2469 qtmhhttp /usr/local/zendsvr6/bin/zsd /usr/local/zendsvr6/etc/zsd.ini -e
dbx -d 100 -a 2469 -c /tmp/pigs_cmd1_20574.dbx > /tmp/pigs_out1_20574.dbx 2>/dev/null
dbx -d 100 -a 2469 -c /tmp/pigs_cmd2_20574.dbx 2>/dev/null
Waiting to attach to process 2469 ...

PASE DB2 CLI trace library

At times a PASE side DB2 CLI trace is helpful.

Introduction

This module is designed to intercept and trace PASE DB2 CLI calls to libdb400.a. 
1) /mypath/libdb400.a - PASE DB2 CLI intercept and trace results
2) /QOpenSys/usr/lib/libdb400.a - PASE DB2 CLI module (*)

(*) IMPORTANT 
/mypath/libdb400.a is NOT a replacement driver, never delete/move PASE /QOpenSys/usr/lib/libdb400.a.

configure

To configure /mypath/libdb400.a, /mypath must first in LIBPATH environment variable. 

unzip trace-1.0.6-xlc.zip

PHP example (zend server old):
1) copy trace module into zend look first library ...
> cp libdb400.a /usr/local/zendsvr/lib/libdb400.a
2) when done tracing remove ...
> rm /usr/local/zendsvr/lib/libdb400.a

PHP example (zend server new):
1) copy trace module into zend look first library ...
> cp libdb400.a /usr/local/zendsvr6/lib/libdb400.a
2) when done tracing remove ...
> rm /usr/local/zendsvr6/lib/libdb400.a

Output (file default):

> ls /tmp/libdb400_trace_*
/tmp/libdb400_trace_43206

> cat /tmp/libdb400_trace_43206
SQLExecDirect.43206.23922409.tbeg ---error---
:

trace control (see paseclitrace.ini)

Search order paseclitrace.ini file
1) environment variable ( export PASETRACE=/mypath/paseclitrace.ini )
2) current directory ( ./pasetrace.ini )
3) /tmp directory ( /tmp/pasetrace.ini )
*) search fail, defaults compiled ( PaseCliIni_gen.c )

Control trace behaviour (see paseclitrace.ini)

> cat paseclitrace.ini 
; device_type=[tty,file]
;   0  - console
;   1  - file (/tmp/libdb400_trace)
; device_pid (device_type=1-file)
;   0 - /tmp/libdb400_trace
;   1 - /tmp/libdb400_trace[_pid]
; trace_type
;   0 - trace all
;   1 - trace errors only
; trace_stack
;   0 - no PASE stack dump
;   1 - PASE stack dump errors only
;   2 - PASE stack dump all
; trace_break (use with trace_stack)
;   0 - PASE internal stack (very fast, internal, recommended)
;   1 - PASE dspjob stack (slower, external, not recommended)
;   2 - PASE dbx attach stack (much slower, external, not recommended)
; cli api=[0,1]
;   0    - API ignore
;   1    - API trace
device_type=1
device_pid=1
trace_type=1
trace_stack=1
trace_break=0
sqloverrideccsid400=1
sqlallocenv=1
sqlallocconnect=1
:
switch on/off each CLI API individually
:

changelog

  • Version 1.0.6 (2015–09–29)
    • added trace_script for merged script and cli trace
    • note: Your language driver needs to be enabled
      paseclitrace.ini
      trace_script=1
      
      Example latest ruby ibm_db:
      -- any old ruby ibm_db script --
      class WelcomeController < ApplicationController
        def index
           IBM_DB.trace_exclude(["type_cast","simplified_type"])
           IBM_DB.trace("hey adc i am in welcome controller")
           @frogs = FrogParty.find_each
        end
      end
      -- ibm_db_adapter.rb --
        trace = TracePoint.new(:call) do |tp|
          if tp.path.include? 'ibm_db_adapter.rb'
            IBM_DB.trace(sprintf("%s %s %-2d", tp.defined_class, tp.method_id, tp.lineno))
          end
        end
        trace.enable
      
  • Version 1.0.5 (2015–09–08)
    • change default for SQLFree to ‘on’ (was off)
  • Version 1.0.4 (2015–09–03)
    • fixed multiple coredumps running ruby threaded tests
    • key - expanded time seconds in report
    • key - added thread id to key in report
    • data - added text for SQL_SUCCESS, SQL_ERROR to sqlrc in report
  • Version 1.0.3 (2015–08–19)
    • very fast internal stack dump (trace_break=0), no longer recommend other options
  • Version 1.0.2 (2015–06–26)
    • xlc compile removed all dependencies on gcc environment (should work PASE V7+)
    • DB2 CLI trace now dumps ALL threads when using paseclitrace.ini trace_break=1 (dbx method). However, only ONE thread is dumped default trace_break=0 (dspjob *PGMSTK).

On-line tools

On-line PASE VLOG tool (cut/paste)

Common

CCSID issues

Have CCSID issues? Junk characters? Nothing works?

  • IBM i actions, see CCSID

PASE broken, let me count the ways …

   1) Customer authorization IFS "lock down" until PASE runs fail authorization (IFS "lock down" until nothing works)
   2) Customer destroys PASE product symbolic links /usr/bin, /usr/lib, ../bin, ../lib, etc. (installing AIX binaries)
   3) Customer restores "back-up" which clobbers PASE with ancient version (SAV/RST error)
   4) Customer creates IFS install for "other machines" which clobbers PASE version (INSTALL error)
   5) Customer journaling IFS bytestream files part of PASE product (journaling /QOpenSys/* IFS files)

Debug tips from PASE pros