NT Debugging Overview
Last updated on April 12, 1999
  1. Debug Setup
    1. Debug cabling
    2. Boot.ini settings
    3. Environment variables
    4. kd vs. windbg vs. ntsd
    5. Crash dump
    6. Remote debugging
  2. Debugging Kernel Mode
    1. Common debugger commands
    2. How to read blue screens and what's useful.
    3. Hardware failures
    4. Other common failures
    5. Identifying bad symbols
  3. Debugging User Mode
    1. Symbols on target machine
    2. Debugging an app
    3. Debugging csrss
    4. Attaching a debugger to a running process
    5. Sending user mode debug output to kernel debugger
    6. Application error debug
    7. Types of user mode failures
Appendix A. Debugger Quick Reference
Appendix B. Debugger Commands
Appendix C. Debugger Extensions
Appendix D. Debugger Command Line Options

Section I. Debug Setup

I.a. Debug Cabling

To be able to debug your machine, you will need a second machine running NT. They should be connected with a null modem cable. To obtain a null-mode cable, use your normal procurement process, as you would for any other equipment.

When you have the cable in hand, connect one end to the test machine's (debuggee's) serial port (by default NT uses com2 on x86 if there is one, otherwise com1), and attach the other end of the debug cable to any of the debugger machine's serial ports.

I.b. Boot.Ini Settings

On Intel machines, the Boot.Ini file describes how a machine should boot NT, including a few debug options. All of these options go on the end of the entry for the appropriate build in the [operating systems] section. In most cases, you will simply want to place /DEBUG on the end of the build that you want to be able to debug (on the test machine/debuggee). (On your debugger machine, you will usually not want /DEBUG so that all COM ports are available for debugging other machines..) Any build with /DEBUG should show up on the boot menu with "[debugger enabled]" at the end of the description.

Here is a list of the Boot.ini options and what they mean:

/DEBUG  = send debug information always
/CRASHDEBUG  = send debug info only on a fatal system error (FSE)
/NODEBUG  = don't send debug info, even for FSEs
/DEBUGPORT=comx  = use serial port x instead of default
/BAUDRATE=x  = use different baud rate than default (19200)

On RISC platforms, the BootOptions field in the Arc setup can be set to DEBUG, CRASHDEBUG or NODEBUG.

I.c. Environment variables

On the debugger machine, there are a number of environment variables which tell kd (i386kd, mipskd & alphakd) how to run. The most important of these are _NT_DEBUG_PORT (serial port on debugger machine) and _NT_SYMBOL_PATH (which points to the symbols directory). Here is a list of all the environment variables used by kd:

_NT_DEBUG_PORT=com[1|2|...]serial port on debugger, com1 default
_NT_DEBUG_BAUD_RATE=baud ratedefault 19200
_NT_DEBUG_LOG_FILE_APPEND=filenameappend to existing log
_NT_DEBUG_LOG_FILE_OPEN=filenamenew log created
_NT_DEBUG_CACHE_SIZE=x
_NT_SYMBOL_PATH=[Drive:][Path]points to "symbols" dir with *.dbg tree
_NT_ALT_SYMBOL_PATH=[Drive:][Path]alternate symbol path, used first

Note: The symbol paths must point to a directory that is called "symbols", off of which should be directories for each extension's .dbg files (exe, dll, sys, etc.).

The above environment variables are used by kd. Windbg has menu options to set info or all info can be specified on the command line with the following options:

-y <symbol path>

-k <platform> <com_port> <baudrate>

-s <remotename>

I.d. Kd vs. WinDbg vs. Ntsd

There are 3 primary debuggers used with NT:

Ntsd (and windbg when debugging user mode processes) use symbols in the %windir%\symbols directory. The dll and exe symbols are needed for debugging most user mode problems. See the User-Mode Debugging section later in the document for more information.

For any of these debuggers, it is important to use a debugger, extension dlls and symbols that match the build being debugged. Internal system structures may change which affect the way debuggers and extensions work.

Windbg comes with a help file, so most of this document will cover the interface from kd. All of the commands covered here can be entered in the command window of windbg.

I.e. Crash Dump

The crash dump feature allows a machine that hit a fatal system error to be debugged after it is rebooted. Turn on the crash dump option in the Systems Control Panel applet in the Recovery dialog. To debug this after the machine reboots, start windbg or kd with the following options: -z <PathToDumpFile> -y <sympath>. Prior to NT 3.5 (807), there was no Crash Dump support and kd Crash Dump support was added shortly after the 3.5 release.

This is useful for debugging failures after a crash, but should not be used in place of kd. In case there is a problem writing the dump file (e.g. out of disk space), it is important to have the debugger running to catch a failure. There are also some problems which aren't debuggable without kd (e.g. resource timeouts). Thus, it is a good idea to have crash dump enabled at all times, but definitely have a debugger when trying to repro a problem.

If you have a machine which has broken into the debugger, you can dump IopFinalCrashDumpStatus (dd nt!IopFinalCrashDumpStatus L1). If the result is 0, the process was successful. If it is -1 (FFFFFFFF), the dump process has not started. Any other value is a status code indicating the error during the dump process.

I.f. Remote Debugging

Remote is a tool that allows redirection of input and output so that info can be viewed or entered from remote sites. It can be used with any console app, but its most common use is with the kernel debugger.

To setup a remote debugging session, set the environment variables listed above and execute "remote /s i386kd <testmachname>" on your debugger machine. This will start i386kd for the machine name you specified. Someone can connect to this machine by doing a "remote /c <dbgmachname> <testmachname>". From the remote session, you can issue any command as if you were at the debugger.

Windbg also supports remote debugging. To enable remote debugging, start windbg with the -s <testmachname> option. Again, you connect with "remote /c <dbgmachname> <testmachname>". You can also start remote on an active windbg by typing "remote <testmachname>" in the command window.

Section II. Debugging Kernel Mode

Now that the debugger is setup, this section will describe some of the most common debugger commands and types of failures in kernel mode. (Kernel mode includes the kernel, hal and drivers and is debugged using a kernel debugger on a separate machine.) Appendix G contains additional useful information for configuring a machine to be debugged using global flags.

II.a. Common Debugger Commands

There are many different commands that are accepted by all of the NT debuggers. Unfortunately, the huge list of commands (available with a "?" at the debugger prompt) makes it difficult for the new user to identify the most useful commands. The simple commands which are most useful are:

k    Stack trace (by far the most useful, do this first!); kb for parameters
r     Register dump
dd   Dump double (starting from specified address)
db   Dump bytes [other: dc (combined), dl (list), da/du (ascii/unicode)]
u     Unassemble specified address

In addition to the basic commands, there are some kernel extensions which can be especially useful, including:

!process    Identifies the current process; "!process 0 7" for all processes
!thread    Identifies the current (or specified) thread
!reload    Reload symbols
!devnode    Lists all Device Nodes.
!sympath    List or change symbol path
!drivers    List loaded drivers (similar to "x *!")

II.b. How to Read Blue Screens and What's Useful.

The problem many people have with the fatal system error blue screens is that there is just too much information. There is only a small amount that is actually useful in identifying most of the problems. The first thing to note is the error code and the four parameters (hex numbers) at the top of the screen. Secondly, note what modules are on the stack at the bottom of the screen. Finally, see if the serial monitors in the upper right are attempting to send data to the kernel debugger. See the following page for an example of a blue screen.

-------------------------------------------------------------------------------

DSR CTS

*** STOP: 0x0000000A (0x00000000,0x0000001a,0x00000000,0x00000000)

IRQL_NOT_LESS_OR_EQUAL
p4-0300 irql:1f SYSVER:0xf000030e
Dll Base DateStmp - Name Dll Base DateStmp - Name
80100000 2e53fe55 - ntoskrnl.exe 80400000 2e53eba6 - hal.dll
80010000 2e41884b - Aha154x.sys 80013000 2e4bc29a - SCSIPORT.SYS
8001b000 2e4e7b6b - Scsidisk.sys 80220000 2e53f238 - Ntfs.sys
fe420000 2e406607 - Floppy.SYS fe430000 2e406618 - Scsicdrm.SYS
fe440000 2e406659 - Fs_Rec.SYS fe450000 2e40660f - Null.SYS
fe460000 2e4065f4 - Beep.SYS fe470000 2e406634 - Sermouse.SYS
fe480000 2e42a4a4 - i8042prt.SYS fe490000 2e40660d - Mouclass.SYS
fe4a0000 2e40660c - Kbdclass.SYS fe4c0000 2e4065e2 - VIDEOPRT.SYS
fe4b0000 2e53d49d - ati.SYS fe4d0000 2e4065e8 - vga.sys
fe4e0000 2e406655 - Msfs.SYS fe4f0000 2e414f30 - Npfs.SYS
fe510000 2e53f222 - NDIS.SYS fe500000 2e40719b - lance.sys
fe550000 2e406697 - TDI.SYS fe530000 2e47c740 - nbf.sys
fe560000 2e5279d9 - nwlnkipx.sys fe570000 2e53a89e - nwlnknb.sys
fe580000 2e494973 - tcpip.sys fe5a0000 2e5256b8 - afd.sys
fe5b0000 2e5279d3 - netbt.sys fe5d0000 2e4167f7 - netbios.sys
fe5e0000 2e4066b3 - mup.sys fe5f0000 2e4f9f51 - rdr.sys
fe630000 2e53f24a - srv.sys fe660000 2e516062 - nwlnkspx.sys

Address dword dump Build [782] - Name
ff541e4c fe5105df fe5105df 00000001 ff640128 fe4a8228 000002fe - NDIS.SYS
ff541e60 fe501368 fe501368 00000246 00004002 00000000 00000000 - lance.sys
ff541eb4 fe481509 fe481509 ff6688c8 ff668288 00000000 ff668138 - i8042prt.SYS
ff541ee0 fe481ea8 fe481ea8 fe482078 00000000 ff541f04 8013c58a - i8042prt.SYS
ff541ee4 fe482078 fe482078 00000000 ff541f04 8013c58a ff6688c8 - i8042prt.sys
ff541ef0 8013c58a 8013c58a ff6688c8 ff668040 80405900 00000031 - ntoskrnl.exe
ff541efc 80405900 80405900 00000031 06060606 06060606 06060606 - hal.dll

Restart and set the recovery options in the system control panel
or the /CRASHDEBUG system start option if this message reappears,
contact your system administrator or technical support group.
CRASHDUMP: Initializing miniport driver
CRASHDUMP: Dumping physical memory to disk: 2000
CRASHDUMP: Physical memory dump complete

-------------------------------------------------------------------------------

In the above blue screen, you would report a bugcheck A in ndis and lance on build 782. The parameters for the bugcheck were 0,1a,0 and 0. It would be worth mentioning that a crashdump file was created and a debugger is attached (as shown by the indicators in the upper right corner). The dll bases and time stamps are NOT necessary.

II.c. Hardware Failures

The most common blue screens are hardware failures. These can manifest themselves as Non-Maskable Interrupts (NMIs), or a variety of Fatal System Errors (FSEs or Bugchecks). The bugchecks that are in the hardware failure category are 2e (data_bus_error), 80 (nmi_hardware_failure), 77 (kernel_stack_inpage_error), 7A (kernel_data_inpage_error), and FACEFEED. One other way to identify hardware failures on Mips is if any of the registers contain 0xBFC00304.

II.d. Other Common Failures

Besides the hardware failures, there are several other bugchecks which occur with some frequency. The most common is bugcheck A (irql_not_less_or_equal), followed by 1e (kmode_exception_not_handled), 23 (fat_file_system), and 24 (ntfs_file_system). All bugchecks are documented in bugcodes.txt (see Appendix E). For any bugchecks, please do a "dd KiBugCheckData L5" and a "k" and send the results to the NtGroup.

Similar to bugchecks are "hard errors" which occur when a necessary service encounters a fatal error. These generally have a status code similar to "C000021A" and say something like "The <x> subsystem terminated unexpectedly." These are rarely debuggable, but the information should be sent to the NT group, nonetheless.

Besides bugchecks and hard errors, it is also possible to get resource timeouts (possible deadlocks), asserts (checked builds only), data misalignments, and access violations. Each of these will break into the debuggers and a stack trace should be sent to the NT group.

It is also possible to accidentally break into the debugger by pressing SysReq or F12 (if ntsd is running on csrss) on the test machine or by hitting ctrl+c in the debugger window. These can be identified by stacks like the following (all x86 examples):

SysReq stack:

ntoskrnl!_DbgBreakPoint
i8042prt!_I8042KeyboardInterruptService+0x2c3
ntoskrnl!_KiInterruptDispatch+0x2a
Ntfs!_NtfsNonCachedIo+0x296
Ntfs!_NtfsCommonRead_0803+0x381
Ntfs!_NtfsFsdRead+0x2ec
ntoskrnl!_IoAllocateIrp+0x6d
ntoskrnl!IofCallDriver+0x38
ntoskrnl!_IoPageRead+0xb5
ntoskrnl!_MiDispatchFault+0x10d
ntoskrnl!_MmAccessFault+0x68b
ntoskrnl!_KiTrap0E+0xab
winsrv!_KeyboardApcProcedure
ntoskrnl!_KeWaitForSingleObject+0x220

Ctrl+c stack:

ntoskrnl!_KeUpdateSystemTime+0x131
hal!_HalProcessorIdle+0x2

F12 stack:

ntdll!DbgBreakPoint+0x4
kernel32!DebugBreak+0x8
winsrv!ActivateDebugger+0x108
winsrv!DoHotKeyStuff+0x168
winsrv!_KeyEvent+0x1a4
winsrv!ProcessKeyEvent+0x114
winsrv!KeyboardApcProcedure+0x1cc
winsrv!RawInputThread+0x27c

Finally, sometimes machines will simply hang (no mouse or keyboard response, no video update, etc.) without breaking into the debugger. Ctrl+c on the debugger should cause a break to occur.

II.e. Identifying Bad Symbols

One of the most common problems in debugging failures on a machine that is often updated is mismatched symbols. There are 3 main causes for this problem: pointing at symbols for wrong build, using a privately built binary without the corresponding binary, and using the uni-processor hal and kernel symbols on a multi-processor machine. The first two are simply a matter of matching your binaries and symbols; the third can be corrected by either renaming your hal*.dbg and ntkrnlmp.dbg to hal.dbg and ntoskrnl.dbg, or by using the -mp and -h: options for kd or windbg.

To check symbol integrity, the first thing to do is dump out the build number with the following command: "dd NtBuildNumber L1". The result should be a hex number where the first digit is either 'f' for free or 'c' for checked. The rest of the number should be the hex representation of the build number. Do a "? <rest of number>" to see the decimal representation.

If the kernel symbols are correct, but you aren't getting a complete stack, do an "x *!". This will list the modules that currently have symbols loaded. "!reload" will attempt to load all kernel mode symbols. Try this and see if the stack then looks correct.

One other useful technique for verifying symbols is unassembling code. Most functions begin with an add, sub, or push operation using either the base pointer (ebp) or the stack pointer (esp or sp). Try unassembling ("u <funct>") some of the functions on the stack (from offset 0) to verify the symbols. See Appendix F for examples of good and bad symbols.

Section III. Debugging User Mode

User mode debugging uses essentially the same commands and methods as the kernel debugger. One additional command that becomes useful here is the "~ns" which allows you to switch to debugging thread #n (and less commonly used is "|ns" which allows you to switch to process n). This section discusses the various methods and requirements for user mode debugging, as well as describing most of the failures that occur in the course of debugging user mode. Appendix G contains additional useful information for configuring a machine to be debugged using global flags.

III.a. Symbols on Target Machine

The most common mistake in debugging user mode code is assuming that you don't need symbols because the kernel debugger has them. The symbols for user mode debugging must be on the target (test) machine, not the kernel debugger machine.

The symbols should be placed in a "symbols" directory off of the windows directory. Symbol files can be found on the freebins shares (in the nt\symbols directory) from the NT release points or on the CDs in the support directory for whichever platform. There should be a subdirectory under symbols for each binary extension. Dll and exe are most important types of symbols for user mode debugging.

Some (non-system) binaries contain symbolic information within them. In that case, no extra symbol files are necessary.

III.b. Debugging an App

For debugging a 32-bit application or test, you can start the program in ntsd or windbg with "ntsd <exe>" or "windbg <exe>". The program will break into the debugger before starting normal execution. This allows you a chance to set breakpoints and such. "-g" on the command line will make it go past the initial break or "g" at the debugger prompt will continue execution. The app will break into the debugger whenever it encounters a new thread start or stop, an access violation, a status is raise or ctrl-c is pressed while the app is active.

If you have the source code for what you are debugging, windbg can be much easier to use than ntsd. After starting a debug session, you can open the source and set breakpoints within the C code and look at actual variables instead of always dumping registers.

To build your own code for debugging with either ntsd or windbg, set the following environment variables before doing a build:

MSC_OPTIMIZATION=/Od

NTDEBUG=ntsd

ntdebugtype=both

III.c. Debugging Csrss

Csrss is the exe which controls the underlying layer for the windows environment. It is often necessary to attach ntsd to csrss to find certain types of problems. When running workstation stress, debugging csrss is necessary to find many of the problems encountered there, and the stress manager automatically copies symbols and starts ntsd for you. For other testing, it is often similarly useful. In particular, to collect more information when the windows subsystem terminates unexpectedly with a hard error c000021a, debugging csrss will catch the failure before it gets to the "unexpected" point.

To start ntsd on csrss, you must first use RegEdt32 to edit the GlobalFlags registry value in HKEY_LOCAL_MACHINE\CurrentControlSet\Control\SessionManager. For NT 3.1 and 3.5, the 0x00080000 bit needs to be cleared. For example, 211a0000 would become 21120000 and 20180000 would be 20100000. For current (post-NT 3.5) builds, the global flags have changed. Now, Csr debugging is enabled by setting the 0x00020000 bit in the GlobalFlags. For either case, you can also run munge.bat from \\ntstress\stress to set the global flags appropriately. For optimal debugging configuration, run munge.bat and choose preferred stress settings, which will set CSR debugging, timeout values and various other options for debugging. The registry change requires a reboot to take effect.

After the registry has been properly configured, it is a simple matter of starting ntsd as follows: "ntsd --" (ntsd minus minus is equivalent to "ntsd -p -1 -g -G"). To break into ntsd by hand in this case, press F12. The debugger will break for the same cases as listed in the previous section. In addition, you may see an "in page io error" message that is another manifestation of a hardware failure.

III.d. Attaching a Debugger to a Running Process

Both ntsd and windbg support attaching to a running process using the process id, which can be identified using tlist.exe or pview.exe. When you start the debugger with "-p <pid>", it will attach and break in to allow debugging. Windbg also has a Run->Attach menu option that allows you to select which process to attach.

To attach a debugger to winlogon, you must go through the registry so that the process is debugged from the time it starts up. To set up winlogon debugging, set HKEY_SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\WinLogon.EXE\Debugger to "ntsd -d -g" (-d sends to kernel debugger and -g causes it to go initially). In addition to the debugger entry, you should set the "GLOBAL_FLAG" value under the winlogon.exe key to REG_SZ "0x08A00008" for 3.1 and 3.5 or "0x000400f0" for current builds. These changes will take effect on the next reboot.

III.e. Sending User Mode Debug Output to Kernel Debugger

Ntsd has an additional feature which allows the debug input and output for any ntsd session to go to the kernel debug session for the machine ntsd starts on. This enables remote debugging of user mode failures (assuming the kernel debugger is setup for remote debugging). To send ntsd output to the kernel debugger, start ntsd with the "-d" option. (Csrss debugging always goes to the kernel debugger, with or without -d.)

III.f. Application Error Debugging

By default, a retail system does not enable application error debugging. An application error occurs when a user-mode process accesses memory it shouldn&145;t or when an exception is raised. They appear as popups which tell which process raised what exception or accessed what memory. The most common application errors are "Possible deadlocks" and access violations (The instruction at X referenced memory at Y).

Application errors can be debugged using a number of methods. When the popup appears on a retail system, it will only have an OK button, but by attaching ntsd (or windbg) to the process which caused the popup, the error can be fully debugged.

To make application error debugging simpler, there are some options in the registry for AeDebug for "debugger" and "auto". The debugger option specifies which debugger to run when an application error occurs, and auto specifies whether the error will break directly into the debugger of if it will display the popup first. To set these options, use the following commands (INI is a IDW tool provided with each build):

INI AeDebug . Debugger = "ntsd -p %ld -e %ld -g" [-d for output to kd]

INI AeDebug . Auto = 1

Without INI, look in HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\AeDebug. This key will not exist on a new retail system. Also note that the debugger command line can be used for any other debugger (windbg, drwatson, cdb, etc.). If you run workstation stress, application error debugging will be turned on for you.

As with any user mode debugging, symbols for the appropriate EXEs and DLLs must exist on the local machine. You can copy them at the time of the popup, but not after the debugger is started.

III.g. Types of User Mode Failures

User mode failures include access violations, datatype misalignments, exceptions, critical section timeouts (deadlocks), and in page io errors.

Access violations are the most common. They usually occur when an invalid pointer is dereferenced. The culprit is usually the function that faulted or, if the offending value is a parameter, one of it's callers. This is also true of datatype misalignments.

Exceptions occur for a variety of reasons and the exception code should be looked up in ntstatus.h or winerror.h before attempting to find the cause. In general, if the first hex digit is set, check in ntstatus.h, otherwise look for the decimal equivalent in winerror.h. Many statuses are not interesting (e.g. out of resources).

Critical section timeouts (or "possible deadlocks") occur when one thread is waiting for a critical section for a long time. These are more difficult to debug and identify and should be sent to the NtGroup with a stack trace. More information on debugging timeouts is given in the Advanced Debugging Topics section.

In page io errors are almost always hardware failures. You can double-check the status code in ntstatus.h to verify.

Debugger Quick Reference

Debugger setup:

Serial cable (null modem)
Boot.ini on test machine: /DEBUG, /DEBUGPORT
Environment vars: _NT_SYMBOL_PATH, _NT_DEBUG_PORT

Debugger commands:

Stack trace: k, kb (kv)
View memory: dd [double], db [byte] (dw [word], da [ascii], du [unicode], dc [combined, dl [list])
Registers : r
View code : u
Breakpoints: bp (br, bw, ba)
Debugger extensions: !process, !thread, !reload, !sympath

Symbol verification

dd NtBuildNumber -> f0000xxx (where xxx is the build # in hex)
Unassemble functions in stack (most start with push ebp or add/sub esp)

Hardware failures

Non-Maskable Interrupt (NMI)
Bugcheck 2e (DATA_BUS_ERROR)
Bugcheck 77 (KERNEL_STACK_INPAGE_ERROR), 7A (KERNEL_DATA_INPAGE_ERROR),
Bugcheck 80 (NMI_HARDWARE_FAILURE)
Bugcheck FACEFEED
Mips value BFC00304
in page io errors

Types of failures:

Bugchecks (A, 1e, 23, 24) - dd KiBugCheckData L5
Timeouts (resource, critical section) -> k
Access violations -> k
Datatype misalignments -> k
Hung machines -> !process

Ntsd:

Debug csrss: global flags & ntsd -p -1 -g -G
Debug existing: tlist/pview -> ntsd -p <pid>
Output to kd: ntsd -d

Debugger Commands

This is the list of debugger commands from the WinDbg help file. For more information about specific commands, check there. Starred items indicate commonly used commands

CommandDefinition
; Command Separator
? Evaluate Expression
! User Extension DLL
| Display Process State
~ Display Thread State
# Search for Disassembly Pattern
% Change Context
.ATTACH Attach to Process
.CACHE Cache Size* ".cache decodeptes" to load transition data
.LIST Display Source/Assembly Listing
.LOGAPPEND Append Log File
.LOGCLOSE Close Log File
.LOGOPEN Open Log File
.REBOOT Reboot Target Machine
.RELOAD Reload Symbols* !reload used often for kernel debugging
BC Breakpoint Clear
BD Breakpoint Disable
BE Breakpoint Enable
BL Breakpoint List
BP Set Breakpoint* all breakpoints useful, also BA (address)
C Compare Memory
DA, DB, DC, DD, DI, DS, DT, DU, DW Display Memory* dd used most often
EA, EB, ED, EI, ES, ET, EU, EW Enter Values
F Freeze Thread
FIA, FIB, FID, FII, FIS, FIT, FIU, FIW Fill Memory
FR Floating-Point Registers
G Go* "g" to continue the machine
GH Go-- Exception Handled
GN Go-- Exception not Handled
K,KB,KN,KS,KV Display Stack Backtrace* "k": do this first! stack trace
L Restart Debuggee
LM List Loaded Modules
LN List Nearest Symbols* "ln" quick way of viewing symbols
M Move Memory
N Set Number Base
P Program Step* "p" used for stepping (skip functions)
Q Quit WinDbg* "q" quit debugger
R Registers* "r" check (or set) registers
REMOTE Start Remote Server* REMOTE to dynamically set remote name
RT Register Display Toggle
SA, SB, SD, SI, SS, ST, SU, SW Search Memory
S+, S- Set Source/Assembly Mode
SEB, SEW Set Error Break, Set Error Warning
SX,SXD,SXE,SXN Set Exceptions
T Trace* "t" used for stepping (enter functions)
U Unassemble* "u" unassemble to see more of the function
X Examine Symbols
Z Unfreeze Thread

Debugger Extensions

This appendix simply lists the description for the extensions in the listed extension dlls. There are also some special extension dlls (e.g. splexts.dll) which are usually called *exts.dll.

1. Kernel Debugger Extensions:

KD has a number of built-in extension commands that are useful in debugging drivers. The command-line syntax for a built-in extension is:

!extension_name [arguments]

The following table lists the built-in extensions. You can also enter !? at the KD prompt for a list of available extensions. See also Creating Extensions.

Syntax Description

!cxr address Displays the context record at the specified address.

!db [PhysicalAddress] Displays a hexadecimal and ASCII dump of 128 bytes from the specified physical memory address on the target machine. If the PhysicalAddress argument is omitted, the command dumps beginning at the byte following the previous !db dump.

!dd [PhysicalAddress] Displays a hexadecimal dump of 32 ULONGs from the specified physical memory address on the target machine. If the PhysicalAddress argument is omitted, the command dumps beginning at the byte following the previous !dd dump.

!default DllName Change the default extension DLL.

!devobj address Displays a list containing information about a specified device object. The address argument is a pointer to the device object, and can be obtained using the !drvobj command. The information displayed includes the device name of the object, information about the device's current IRP, and a list of addresses of any pending IRPs in the device's queue.

!drvobj address Displays a list of all device objects created by a specified driver. The address argument is a pointer to the driver object, and can be obtained by examining the arguments passed to the driver's DriverEntry routine.

!drivers Displays information about all loaded drivers.

!ed PhysAddr ulong0 [ul1] Writes one or more ULONG values to the specified physical memory address on the target machine.

!errlog Displays information about any pending events in the I/O system's error log. These are events queued by calls to the IoWriteErrorLogEntry function to be written to the system's event log for subsequent viewing by the Event Viewer. This command only displays pending entries that have not been written to the event log. If the system crashes, this function is useful for viewing events that are not written to the event log because of the crash.

!exr address Displays the formatted contents of the EXCEPTION_RECORD pointed to by address.

!frag [flags] Displays information about fragmentation in the nonpaged pool. The two low-order bits of the flags argument specify the level of detail displayed.

!filecache Displays information about the file system cache.

!handle [h [flags [proc]]] Displays data about handle, the index of the handle in the process's handle table. If handle is 0 (or unspecified), the command dumps data for all handles. The two low-order bits of the flags argument specify the level of detail displayed. The process argument identifies the process to dump handles for, and is either the process ID or the address of a process object. If not specified, the current process is used. 0 dumps information for all active processes.

!heap [addr [flags [proc]]] Dumps the heap for a process.

!irp address Displays formatted information about the IRP specified by address. Note that the function does not verify that address is an IRP. This command can dump IRPs after they have been freed or completed, or in a post-mortem situation.

!irpzone [F] Displays information about all IRPs allocated from zoned pool. If the 'F' argument is omitted, the command prints only the address of the IRP, the address of the thread object associated with the IRP, and a pointer to the IRP's current device object (for example, the device associated with the last stack).

!kb Displays a stack trace from the last frame dumped by !cxr or !trap.

!load DllName Load the named extension DLL.

!locks [-v] [address] Displays information about a kernel-mode resource lock specified by address. If address is omitted, the command dumps information on all kernel-mode resource locks. Use the -v option for verbose mode.

!lpc Displays information about all LPC port objects.

!memusage Displays summary statistics on physical memory usage, collected from the page frame database.

!noversion Disable version checking for extension DLLs.

!object address Displays information about the object pointed to by address.

!pcr Displays the formatted contents of the PCR.

!pfn PageFrameNumber Dumps the page frame database entry for the physical page identified by PageFrameNumber, which is the page's index in the array of records of the page frame database.

!pool Dumps the kernel-mode heap.

!process [process [flags]] Dumps information about the process specified by process, which is either the process ID or the address of the process object. If not specified, the current process is used. 0 dumps information for all active processes. The three low-order bits of the flags argument determine the level of detail displayed.

!pte VirtualAddr Displays information about the physical address that corresponds to the specified virtual address. This is useful if a driver does a lot of translations back and forth from virtual to physical. The PFN address is the page frame number of the virtual address. Use this as the first 20 bits of the physical address, and append the last 12 bits of the virtual address to come up with the physical address.

!reload [imagename] Causes WinDbg KD to load the symbol files, discarding any previously loaded symbols. You can specify the name of an image file whose symbols you want WinDbg KD to reload.

!sel [selector] Displays selector values.

!srb address Displays the formatted contents of the SCSI_REQUEST_BLOCK pointed to by address.

!sympath SymbolPath Resets the path used by WinDbg KD to find debugging symbols.

!thread [thread [flags]] Dumps information about a specified thread, where address points to a thread object. If the thread argument is omitted, the command dumps data for the current thread. The three low-order bits of the flags argument determine the level of detail displayed.

!time Displays the PerformanceCounterRate and TimerDifference.

!trap TrapFrameAddr Dumps the machine state when the trap frame occurred. On x86-based platforms, a trap frame is generated whenever there is an interrupt or a system call. Identify a trap frame by the value of its first argument (badb0d00). Use the kv command to find a trap frame. This is useful if you want to see the state of the machine when an access fault occurred.

!tss Displays the contents of the Task State Segment.

!unload Unloads the default extension DLL.

!vad Dumps VADs.

!vm Displays summary statistics about virtual memory usage.

2. User Extensions

!Userexts.Help in ntsd gives:

!help [cmd] Displays this list or gives details on command
!kbd [pq] Displays key state for queue
!dc <pclass> Dump class info
!dde [v] [r] [conv|window|xact] Dump DDE tracking information
!ddeftbp <pxstate> Find transaction back pointers
!ddeml [v] [i<inst>] [t<type>] [hObj|pObj] Dump DDEML state information
!dcolors Dump system color values and brushes.
!ddbDump Debug Buffer - for function traceing.
!ddesk <pdesk> Displays objects allocated in desktop
!dfs Dump current focus state variables.
!dhe <pointer|handle> Dump handle entry
!dhot Dump registered hotkeys
!dhk [l] [g] [pti] Dump hooks
!dhs [v] [[p|t] id] Dump handle table statistics
!di Displays USER globals related to input processing
!dll [[*]addr [l#] [o#] [c#]] Dump linked list
!dlr <pointer|handle> Displays assignment locks for object
!dm <pmenu> Dumps a menu
!dmq <pq> Lists messages in queue specified
!dq [t] <pq> Displays Q structure specified
!dsi Displays server info struct
!dsbs <pSBState> Displays Scroll Bar State structure
!dsbwnd <psbwnd> Dump extra fields of Scrollbar windows
!dsms [v] [l] Displays SMS (SendMessage structure) specified
!dt [v][id] Displays simple thread information
!dti [pti] Displays THREADINFO structure
!dtl <pointer|handle> Displays thread locks for object
!dtmr Dumps all timer structures.
!du <pointer|handle> Generic object dumping routine
!dumphmgr [s] Dumps object allocation counts (dbg only)
!dumpPED <ped> Dump PEDitControl structure
!dw [g][v][s|p][r][w] [pwnd] Displays information on windows in system
!dws Dump windowstations
!dpi <ppi|*> Displays PROCESSINFO structure specified
!dwpi <pwpi> Displays WOWPROCESSINFO structure specified
!oem Displays global OEMINFO struct (oeminfo)
!options [c][f] Displays current debugger options settings

Debugger Command Line Options

1. KD Command Line: (same for mipskd, alphakd, ppckd)

Usage: i386kd [-?] [-v] [-m] [-r] [-n] [-b] [-x] [[-l SymbolFile] [KernelName] where:
-v Verbose mode
-? Display this help
-n No Lazy symbol loading
-m Use modem controls
-b Break into kernel

Environment Variables:

_NT_DEBUG_PORT=com[1|2|...] Specifiy which com port to use. (Default = com1)
_NT_SYMBOL_PATH=[Drive:][Path] Specifiy symbol image path. (Default = x:)
_NT_DEBUG_BAUD_RATE=baud rate Specifiy the baud used by debug port. (Def = 19200)
_NT_DEBUG_LOG_FILE_APPEND=file If specified, output will be appended to this file.
_NT_DEBUG_LOG_FILE_OPEN=file If specified, output is written to file from offset 0.
_NT_DEBUG_CACHE_SIZE=x

Control Keys:

<Ctrl-C> Break into kernel
<Ctrl-B><Enter> Quit debugger
<Ctrl-R><Enter> Resynchronize target and host
<Ctrl-V><Enter> Toggle Verbose mode
<Ctrl-D><Enter> Display debugger debugging information

2. Windbg Command Line Options:

windbg [-a] [-g] [-h] [-i] [-k [platform port speed]] [-l[text]] [-m] [-p id [-e event]] [-s[pipe]] [-v] [-w name] [-y path] [-z crashfile] [filename[.ext] [arguments]]

Syntax Explanation

-a Ignore all bad symbols (but still print warning message).
-g Go now; start executing the process.
-h Causes child processes to inherit access to WinDbg's handles.
-i Ignore workspace; like running without any registry data.
-k [platform port speed] Run as a kernel debugger with the specified options:
    platform is the target machine type (i386, mips, alpha)
    port is the com port (com1 ... comn)
    speed is the com port speed (9600, 19200, 57600, ...)
-l [text] Sets the window title for WinDbg.
-m Start WinDbg minimized.
-p id Attach to the process with the given id.
-e event Signal an event after process is attached. Used only for post-mortem debugging (AeDebug).
-s [pipe] Start a remote.exe server, using the named pipe.
-v Verbose option; WinDbg prints module load and unload messages.
-w name Load the named workspace.
-y path Search for symbols along the specified path. You can specify multiple paths by separating them with semicolons.
-z crashfile Debug the specified crash dump file.

3. Ntsd Command Line

Usage: ntsd [--] [-s] [-x] [-a ExtensionDll] [-o] [-2] [-d] [-g] [-G] [-v] [-w] [-p pid] [-r RipLevel] [-t RipLevel] [-e Event] [-z CrashFile] [filename[.ext]] [arguments]

Syntax Explanation

-- Starts ntsd on csrss if global flags set correctly. Same as ntsd -p -1 -g -G
-s Disable lazy symbol loading
-x Don"t break on access violations
-a ExtensionDll Set default extension dll
-o Debug child processes
-2 Use second console
-d Send output to kernel debugger
-g Go now; start executing the process
-G Don"t break on process terminate
-v Verbose option; ntsd prints module load and unload messages
-w Use a separate vdm for WOW.
-p pid Attach to the process with the given id
-r RipLevel Level to break Rips
-t RipLevel Level to print Rips
-e Event Signal an event after process is attached. Used only for AeDebug
-z CrashFile Debug the specified crash dump file