Sunday, December 11, 2011

Troubleshooting 0xC9 DRIVER_VERIFIER_IOMANAGER_VIOLATION

The Debugging Tools for Windows are required to analyze crash dump files. If you do not have the Debugging Tools for Windows installed or dump files are not being generated on system crash, see this post for installation/configuration instructions:
http://mikemstech.blogspot.com/2011/11/windows-crash-dump-analysis.html

In cases of memory corruption, the driver verifier can be an invaluable tool in helping to identify the root cause of a kernel mode crash, or at the very least working towards identifying a definitive cause. In this case, the system was initially throwing 0x0000001A (MEMORY_MANAGEMENT) and 0x00000019 (BAD_POOL_HEADER). Typically, you won't see this bug check unless you have enabled the driver verifier and it has caught a specific type of error when the driver is performing a task with the Windows IO manager. Troubleshooting 0x000000C9 begins with a !analyze -v

4: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

DRIVER_VERIFIER_IOMANAGER_VIOLATION (c9)
The IO manager has caught a misbehaving driver.
Arguments:
Arg1: 0000000000000220, IRP_MJ_SYSTEM_CONTROL has been completed by someone other than the ProviderId.
 This IRP should either have been completed earlier or should have been passed
 down.
Arg2: fffff880057de710, The address in the driver's code where the error was detected.
Arg3: fffff980134b8dc0, IRP address.
Arg4: fffffa800cafb060, ProviderId.

Debugging Details:
------------------


BUGCHECK_STR:  0xc9_220

DRIVER_VERIFIER_IO_VIOLATION_TYPE:  220

FAULTING_IP: 
HIDCLASS!HidpMajorHandler+0
fffff880`057de710 48895c2410      mov     qword ptr [rsp+10h],rbx

FOLLOWUP_IP: 
HIDCLASS!HidpMajorHandler+0
fffff880`057de710 48895c2410      mov     qword ptr [rsp+10h],rbx

IRP_ADDRESS:  fffff980134b8dc0

DEVICE_OBJECT: fffffa800fe96b80

CUSTOMER_CRASH_COUNT:  1

DEFAULT_BUCKET_ID:  VERIFIER_ENABLED_VISTA_MINIDUMP

PROCESS_NAME:  System

CURRENT_IRQL:  2

LOCK_ADDRESS:  fffff800036c5b80 -- (!locks fffff800036c5b80)

Resource @ nt!PiEngineLock (0xfffff800036c5b80)    Available

WARNING: SystemResourcesList->Flink chain invalid. Resource may be corrupted, or already deleted.


WARNING: SystemResourcesList->Blink chain invalid. Resource may be corrupted, or already deleted.

1 total locks

PNP_TRIAGE: 
 Lock address  : 0xfffff800036c5b80
 Thread Count  : 0
 Thread address: 0x0000000000000000
 Thread wait   : 0x0

LAST_CONTROL_TRANSFER:  from fffff800039503dc to fffff800034c6c40

STACK_TEXT: 
... : nt!KeBugCheckEx
... : nt!VerifierBugCheckIfAppropriate+0x3c
... : nt!ViErrorFinishReport+0xda
... : nt!VfErrorReport10+0x6f
... : nt!VfWmiVerifyIrpStackUpward+0x67
... : nt!VfMajorVerifyIrpStackUpward+0x6e
... : nt!IovpCompleteRequest2+0xad
... : nt!IovpLocalCompletionRoutine+0x9d
... : nt!IopfCompleteRequest+0x341
... : nt!IovCompleteRequest+0x19f
... : nt!IopInvalidDeviceRequest+0x16
... : HIDCLASS!HidpIrpMajorDefault+0x8b
... : HIDCLASS!HidpMajorHandler+0xeb
... : nt!IovCallDriver+0x566
... : nt!ViFilterDispatchGeneric+0x62
... : nt!IovCallDriver+0x566
... : nt!VfIrpSendSynchronousIrp+0xe8
... : nt!VfWmiTestStartedPdoStack+0x72
... : nt!VfMajorTestStartedPdoStack+0x5f
... : nt!PpvUtilTestStartedPdoStack+0x12
... : nt!PipProcessStartPhase3+0x55c
... : nt!PipProcessDevNodeTree+0x264
... : nt!PiProcessStartSystemDevices+0x7c
... : nt!PnpDeviceActionWorker+0x302
... : nt!ExpWorkerThread+0x111
... : nt!PspSystemThreadStartup+0x5a
... : nt!KxStartSystemThread+0x16


STACK_COMMAND:  .bugcheck ; kb

SYMBOL_NAME:  HIDCLASS!HidpMajorHandler+0

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: HIDCLASS

IMAGE_NAME:  HIDCLASS.SYS

DEBUG_FLR_IMAGE_TIMESTAMP:  4ce7a665

FAILURE_BUCKET_ID:  X64_0xc9_220_VRF_HIDCLASS!HidpMajorHandler+0

BUCKET_ID:  X64_0xc9_220_VRF_HIDCLASS!HidpMajorHandler+0

Followup: MachineOwner
---------
 
 
This particular crash (Arg1 = 0x220 indicates that the suspected driver completed a WMI request that it should have passed on to another driver). A complete list of parameter 1 values is available on MSDN. The error is suspected to have occurred in HIDCLASS!HidpMajorHandler, indicating that this problem was due to a human interface device (keyboard, mouse, etc) or a driver related to one of these devices. This is determined using the ln debugger command (not for creating links in linux filesystems in this case, but the Windows Debuggers "list nearest symbols" command).

4: kd> ln fffff880057de710
(fffff880`057de710)   HIDCLASS!HidpMajorHandler   |  (fffff880`057de984)   HIDCLASS!HidpIrpMajorDefault
Exact matches:
    HIDCLASS!HidpMajorHandler = 
 
 
In this case the device object is not present in the minidump, but the IRP can be examined with the !irp extension command.

4: kd> !irp fffff980134b8dc0
Irp is active with 5 stacks 4 is current (= 0xfffff980134b8f68)
 No Mdl: No System Buffer: Thread 00000000:  Irp stack trace.  
     cmd  flg cl Device   File     Completion-Context
 [  0, 0]   0  2 00000000 00000000 00000000-00000000    

   Args: 00000000 00000000 00000000 ffffffffc0000010
 [  0, 0]   0  0 00000000 00000000 00000000-00000000    

   Args: 00000000 00000000 00000000 00000000
 [ 17,ff]   0  2 fffffa800fe96b80 00000000 00000000-00000000    
       fffffa800fe96b80: Could not read device object or _DEVICE_OBJECT not 
                                found

   Args: fffffa800cafb060 00000000 00000000 00000000
>[ 17,ff]   0 e0 fffffa800fe96b80 00000000 fffff80003952da0-fffff980134b8fb0 Success Error Cancel 
       fffffa800fe96b80: Could not read device object or _DEVICE_OBJECT not 
                                found
 nt!IovpInternalCompletionTrap
   Args: fffffa800cafb060 00000000 00000000 00000000
 [ 17,ff]   0 e0 fffffa800fe96920 00000000 fffff8000395c240-fffff8800318b720 Success Error Cancel 
       fffffa800fe96920: Could not read device object or _DEVICE_OBJECT not 
                                found
 nt!ViIrpSynchronousCompletionRoutine
   Args: fffffa800cafb060 00000000 00000000 00000000
 

Since the device object was not dumped, information about the     problem IRP and the related device object is not readily available.     In this case, troubleshooting with the driver verifier would need to     continue and it should be checked that all of the correct device     drivers for the system are installed and at their latest version     (unless known version issues exist) and that the latest system BIOS is installed.

See Also
Windows Crash Dump Analysis


No comments:

Post a Comment