Microsoft Open Debugger Extension for WinDbg

NEW POST: Project architecture & coding guidelines

jeffda — Mon, 12 May 2008 22:45:12 GMT

Nicolas, this looks good. I think setting some standards will be very helpful as long as we don’t over complicate simple tasks.

Jeff-

NEW POST: Project architecture & coding guidelines

nicd_msft — Fri, 09 May 2008 21:43:33 GMT

Hello

For such opened project, I think we need to establish some general guidance to have a coding coherence between the various contributors.
Those are just propositions and thought to start establishing rules, feel free to comment/correct/add each point.

1) General source code layout

I think the first thing would be to agree a physical source layout, maybe like the basic one bellow:

--odbgext
---- core
------ IO
------ execengine
------ sessionstate
-------- sqlstate
-------- travelstate
--------- xml/soap
--------- propertybag
---- helpers
------ search
------ conversions
---- analysis
------ hang
------ crash
------ leaks
---- disasm

2) Platform specific code layout

As code for debugging extensions can be very specific to target, a physical layout for each feature could be like bellow:

-- x86
---- user
---- kernel
---- both
-- x64

The building physical from WDK might be good to keep (like objchkwnetx86), but maybe under a \release folder.

3) C/C++/STL/TR1/Boost/QT/...

This project is currently built on top of exts dbgsdk sample, relying on dbgeng extensions API, thus COM interfaces.
This requires to have C entry points exported from common def file (with HRESULT CALLBACK prototypes) and using C++ for engine.
Shared core engines would benefit from C++ OO design & RAII in order to share code easily when possible, maybe with templates usage when dealing with multiple possible types (core functions behaving differently for dword, qword, strings, slist, dlists, etc...), what do you think or it?

Regarding the libraries, I like to use STL even if that may sometimes make “debugging the extension” less easy, it would bring lots of built in container capabilities, string usage, etc…
I would even think STL strings and containers might be to use as much as possible, i.e. string, list<> and vectors< >
I could imagine some features by implementing inproc COM objects within the extension like inter debuggers communication, that might benefit ATL, but that's for a far future.
Using MFC is prohibited, even if I plan to think about a GUI mechanism for example for navigating inside disasm code or the proposed perf treeviews (yeahhh that's possible from a debugger extension :) )

TR1 is very attractive especially thinking about possible regexp additions to odbgext extension. This might however make the build environment stricter regarding VC++ libraries.
Since this extension is IMO primary intended to be build under VC++, Boost/QT external libraries might be for sure useful but inappropriate in this project unless very specific needs.

4) Naming conventions

I think we have for source coherence to agree source file naming conventions, like:

- Only 8 chars source file & folder names (preferable for WDK build env)
- Conventions like avoiding “dbg” in the title as all code there will be dbg related? (:-p)
- Using coding conventions? I like lowerCamelCase (http://en.wikipedia.org/wiki/CamelCase)
- Interfaces names starting with ‘I’, classes starting with ‘C’, globals with ‘g’, members with ‘m_’, and accessors like getMember()?
- Functions prototypes & documentation

I will in the next days check in to contribute to the I/O & execution engine to try to provide a common easy way to recursively execute commands, parse output and get specific output tokens.
This might provide a good startup for easy features like calling & parsing already built in commands to create things like "!findoldestmodules", search features, automate managed debugging with sos, etc...

TIA for your comments and additions
Nicolas

CREATED ISSUE: Paramater Dbg Info Usage

Infro — Sat, 03 May 2008 00:15:16 GMT

It would be nice to have a reversed paramater check system.
following code is probably going to be mixed between assembly, c, c++, c#, and possibly others, sorry for any confusion (consider it pseudo code)

for(;ebp valid, ebp = *ebp)
SymFromAddr(process, [ebp+4], 0, &SymInfo); //ebp+4 is return eip
OurFunction = FunctionList[SymInfo.Name];
DisplayFunction(OurFunction.Name);
for(int x = OurFunction.ParamCount-1;x >=0;x--)
{0
DisplayVariable(OurFunction.Param[x].Type , OurFunction.Param[x].Name, [ebp+8+4*x]); //ebp+8 is param 1
}
this is 32bit code only of course

would output the following
"
Function1(TCHAR* OurString, int OurLock)
Unicode String OurString, "This would be in memmory",0
Integer 32bit OurLock, 0x0000-0000
Function2(pOurStruct Data, ...)
pOurStruct Data+ (Expandable for more specific information)
? 0xFFFF-FFFA, -6, *=Invalid
? 0xC000-0000, +3221225472, *=00000000
"

Source code checked in

jeffda — Fri, 02 May 2008 23:03:46 GMT

Fix to setpaths to include debugger sdk path values

UPDATED WIKI: Home

jeffda — Fri, 02 May 2008 20:00:54 GMT

Project Description
ODbgExt is an open source debugger extension for WinDbg that is intended to be developed by the debugging community. It is based on the Windows Debugger SDK Sample ‘exts’ and uses the COM Debugger interface IDebugControl. The primary objective of the project is to provide the debugging community a central location to share their debugger extensions that help to isolate common problems in the community and make debugging both live systems and dumps easier. The initial release will be the basic framework that we expect other developers to contribute to. There will be very basic functionality in the initial version. We, GES (Global Escalation Services) intend on doing a series of blogs to talk about writing debugger extensions over the coming year. As we blog about writing extension we will include the code in ODbgExt (This project). This is the same group that runs the http://blogs.msdn.com/ntdebugging blog. We encourage you to sign up and contribute your debugger extension ideas and or code to the project. Simply create a codeplex account and request access.

For more details see the discussions section for ideas!

If you are just getting started please make sure that you donwload the latest version of the changes.
There were some problems with the early build environment that have been fix.

Build Instructions

Before we begin, please only check in code that builds with no warnings. (Do not break the build!)

The project requires you install the Windows DDK http://www.microsoft.com/whdc/devtools/ddk/default.mspx
The latest debugger and debuger SDK. http://www.microsoft.com/whdc/devtools/debugging/default.mspx
The latest Platform SDK

You will need to use the DDK Build environment to build the project.

Cmd.exe /k C:\WinDDK\6000\bin\setenv.bat C:\WinDDK\6000\ chk WNET

There is a setpaths.cmd file that is part of the project that sets these paths up for you.

set include=C:\debuggers\sdk\inc;%include%
set lib=C:\debuggers\sdk\lib\i386;%lib%;
set include=C:\winsdk\include;%include%
set lib=C:\winsdk\lib;%lib%;
set path=C:\codeplex;%path%
set DBGSDKLIBPATH=C:\debuggers\sdk\lib
set PSDKLIBPATH=C:\winsdk\lib

To build the project from your DDK Build environment change to the ODbgExt directory and type bcz
This will produce the following output and will build the binary based on your DDK Build environment settings. (ie Free, Checked, x86, or x64)

C:\Debuggers\sdk\samples\ODbgExt>bcz
BUILD: Compile and Link for x86
BUILD: Start time: Tue Apr 08 20:48:21 2008
BUILD: Examining c:\debuggers\sdk\samples\odbgext directory for files to compile.
BUILD: Compiling c:\debuggers\sdk\samples\odbgext directory
NTTARGET_VERSION SET TO WINXP
Compiling - odbgex.rc
Compiling - odbgext.cpp
Compiling - odbg.cpp
Compiling - generating code...
Building Library - objfrewnetx86\i386\odbgext.lib
BUILD: Linking c:\debuggers\sdk\samples\odbgext directory
NTTARGET_VERSION SET TO WINXP
Linking Executable - objfrewnetx86\i386\odbgext.dll
BUILD: Finish time: Tue Apr 08 20:48:23 2008
BUILD: Done

5 files compiled
1 library built
1 executable built

NEW POST: Project Overview Page?

jeffda — Fri, 02 May 2008 18:02:27 GMT

Great Idea, I'll spend some time answering these questions and expanding on this. I really want this to be very open ended. As long as we are benefiting debuggers in the community I'm happy to see code added to the project.

I think if we have major functionality to add we should add another module CPP file. This will reduce the amount of collations in revision control.

I'll work on refining these ideas over the next few days.

NEW POST: Project Overview Page?

Infro — Fri, 02 May 2008 17:48:51 GMT

Could there be a page a bit more like the following? (Something with structure and encouriging questions and answers?)

Project Description

ODbgExt is an open source debugger extension for WinDbg that is intended to be developed by the debugging community. It is based on the Windows Debugger SDK Sample ‘exts’ and uses the COM Debugger interface IDebugControl. The primary objective of the project is to provide the debugging community a central location to share their debugger extensions that help to isolate common problems in the community and make debugging both live systems and dumps easier. The initial release will be the basic framework that we expect other developers to contribute to. There will be very basic functionality in the initial version. We, GES (Global Escalation Services) intend on doing a series of blogs to talk about writing debugger extensions over the coming year. As we blog about writing extension we will include the code in ODbgExt (This project). This is the same group that runs the http://blogs.msdn.com/ntdebugging blog. We encourage you to sign up and contribute your debugger extension ideas and or code to the project. Simply create a codeplex account and request access.

Things we would like to include

A Graphical representation of Kernel, and User mode execution time by process and thread, while correlating to idle / non-idle time
A Graphical representation of idle time for kernel and user mode.
A Graphical representation of pool resources used on a per process bases along with handle table counts.
Hang detection, scanning for various conditions that could cause hangs in a system or user mode process.
Detailed system information such as oldest and newest binaries.
Binary info based on vendor name
A better dissembler that colorizes calls, jumps and indents to show code flow
Extensive use of DML (Debugger Mark-up Language) to enable more point and click debugging within windbg.
Support for a SQL Database backend to allow storing information about debugging sessions or binaries in a SQL Database.
Warnings when critical thresholds are exceeded such as handle counts over 10,000, Low PTE Conditions, etc.
Simplified searching for pool tags in binaries.
Support for VBA for Windbg
Dump annotation, via dump streaming; the ability to embed data into a dump via the debugger extension and later retrieve it. (Imagine embedded debug notes)
Embedding a snapshot of performance data in the dump at the time the dump is taken, ie. CPU, IO etc.

Questions About Project

How will the source be controlled?

How will "Developers" for the project be choosen?
Will there be a ticket based system for "Logged-In" people to add their code/suggestions or should it be done on the discussion board?

How will changes be handled?

Developers will add what they want, or by ticket system?
What additions are allowed? (Macros, Libraries, etc.?)
What is the scope of the project? Debugging, Reversing, Programming

Basically something to organize the flow?
Discuss what is a good idea in discussion (Allows repeat additions to be filtered and simple project modifications to be found easier)
Issue Tracker should only be used for source code issues?

NEW POST: DML (Debugger Markup Language)

jeffda — Fri, 02 May 2008 00:53:21 GMT

Ok, I added a simple function called DoDml that takes a command and text and prints the DML in WinDBG. This is very handy if you want to have point and click debugging. We can build lots of functionality around this initial function.

Source code checked in

jeffda — Fri, 02 May 2008 00:51:03 GMT

Adding code to show simple DML (Debugger Markup) For !help command.

UPDATED WIKI: Home

jeffda — Thu, 01 May 2008 21:45:40 GMT

Project Description
ODbgExt is an open source debugger extension for WinDbg that is intended to be developed by the debugging community. It is based on the Windows Debugger SDK Sample ‘exts’ and uses the COM Debugger interface IDebugControl. The primary objective of the project is to provide the debugging community a central location to share their debugger extensions that help to isolate common problems in the community and make debugging both live systems and dumps easier. The initial release will be the basic framework that we expect other developers to contribute to. There will be very basic functionality in the initial version. We, GES (Global Escalation Services) intend on doing a series of blogs to talk about writing debugger extensions over the coming year. As we blog about writing extension we will include the code in ODbgExt (This project). This is the same group that runs the http://blogs.msdn.com/ntdebugging blog. We encourage you to sign up and contribute your debugger extension ideas and or code to the project. Simply create a codeplex account and request access.

For more details see the discussions section for ideas!

If you are just getting started please make sure that you donwload the latest version of the changes.
There were some problems with the early build environment that have been fix.

Build Instructions

Before we being, please only check in code that builds with no warnings. (Do not break the build!)

The project requires you install the Windows DDK http://www.microsoft.com/whdc/devtools/ddk/default.mspx
The latest debugger and debuger SDK. http://www.microsoft.com/whdc/devtools/debugging/default.mspx
The latest Platform SDK

You will need to use the DDK Build environment to build the project.

Cmd.exe /k C:\WinDDK\6000\bin\setenv.bat C:\WinDDK\6000\ chk WNET

There is a setpaths.cmd file that is part of the project that sets these paths up for you.

set include=C:\debuggers\sdk\inc;%include%
set lib=C:\debuggers\sdk\lib\i386;%lib%;
set include=C:\winsdk\include;%include%
set lib=C:\winsdk\lib;%lib%;
set path=C:\codeplex;%path%
set DBGSDKLIBPATH=C:\debuggers\sdk\lib
set PSDKLIBPATH=C:\winsdk\lib

To build the project from your DDK Build environment change to the ODbgExt directory and type bcz
This will produce the following output and will build the binary based on your DDK Build environment settings. (ie Free, Checked, x86, or x64)

C:\Debuggers\sdk\samples\ODbgExt>bcz
BUILD: Compile and Link for x86
BUILD: Start time: Tue Apr 08 20:48:21 2008
BUILD: Examining c:\debuggers\sdk\samples\odbgext directory for files to compile.
BUILD: Compiling c:\debuggers\sdk\samples\odbgext directory
NTTARGET_VERSION SET TO WINXP
Compiling - odbgex.rc
Compiling - odbgext.cpp
Compiling - odbg.cpp
Compiling - generating code...
Building Library - objfrewnetx86\i386\odbgext.lib
BUILD: Linking c:\debuggers\sdk\samples\odbgext directory
NTTARGET_VERSION SET TO WINXP
Linking Executable - objfrewnetx86\i386\odbgext.dll
BUILD: Finish time: Tue Apr 08 20:48:23 2008
BUILD: Done

5 files compiled
1 library built
1 executable built

NEW POST: !hangcheck and !memcheck

jeffda — Thu, 01 May 2008 14:24:17 GMT

Hi All, The first thing I’m spending some time adding code to is !hangcheck and !memcheck. If you have ideas for things to check please add to this post or you can add to the code. I believe your going to need to request access before you can sync and check in.

Thank you
Jeff Dailey

Source code checked in

jeffda — Thu, 01 May 2008 13:08:00 GMT

Added code for !memcheck Checks SysPtes, Pool and VM

COMMENTED ISSUE: source and makefile

jeffda — Thu, 01 May 2008 12:14:33 GMT

Where is subj?
Comments: ** Comment from web user: jeffda **

Ok, there were some problems with the initial sync up. The latest change set has the full project. Thank you for bringing this to my attention.

Source code checked in

jeffda — Thu, 01 May 2008 12:13:18 GMT

Resyncing project, there were some critical files that did not sync

CREATED ISSUE: source and makefile

TarasCo — Wed, 30 Apr 2008 13:41:17 GMT

Where is subj?

NEW POST: discussions section for ODbgExt ideas!

jeffda — Tue, 29 Apr 2008 17:01:35 GMT

Things we would like to include:

• A Graphical representation of Kernel, and User mode execution time by process and thread, while correlating to idle / non ideal time
• A Graphical representation of idle time for kernel and user mode.
• A Graphical representation of pool resources used on a per process bases along with handle table counts.
• Hang detection, scanning for various conditions that could cause hangs in a system or user mode process.
• Detailed system information such as oldest, and newest binaries.
• Binary info based on vendor name
• A better dissembler that colorizes calls, jumps and indents to show code flow
• Extensive use of DML (Debugger Mark-up Language), To enable more point and click debugging within windbg.
• Support for a SQL Database backend to allow storing information about debugging sessions or binaries in a SQL Database.
• Warnings when critical thresholds are exceeded such as handle counts over 10,000, Low PTE Conditions, etc.
• Simplified searching for pool tags in binaries.
• Support for VBA for Windbg
• Dump annotation, via dump streaming, The ability to embed data into a dump via the debugger extension and later retrieve it. (Imagine embedded debug notes)
• Embedding a snapshot of performance data in the dump at the time the dump is taken, ie. CPU, IO etc.

Please feel free to suggest more items for this list.

UPDATED WIKI: Home

jeffda — Tue, 29 Apr 2008 16:57:10 GMT

Project Description
ODbgExt is an open source debugger extension for WinDbg that is indented to be developed by the debugging community. It is based on the Windows Debugger SDK Sample ‘exts’ and uses the COM Debugger interface IDebugControl. The primary objective of the project is to provide the debugging community a central location to share their debugger extensions that help to isolate common problems in the community and make debugging both live systems and dumps easier. The initial release will be the basic framework that we expect other developers to contribute to. There will be very basic functionality in the initial version. We, GES (Global Escalation Services) intend on doing a series of blogs to talk about writing debugger extensions over the coming year. As we blog about writing extension we will include the code in ODbgExt (This project). This is the same group that runs the http://blogs.msdn.com/ntdebugging blog. We encourage you to sign up and contribute your debugger extension ideas and or code to the project. Simply create a codeplex account and request access.

Things we would like to include:

• A Graphical representation of Kernel, and User mode execution time by process and thread, while correlating to idle / non ideal time
• A Graphical representation of idle time for kernel and user mode.
• A Graphical representation of pool resources used on a per process bases along with handle table counts.
• Hang detection, scanning for various conditions that could cause hangs in a system or user mode process.
• Detailed system information such as oldest, and newest binaries.
• Binary info based on vendor name
• A better dissembler that colorizes calls, jumps and indents to show code flow
• Extensive use of DML (Debugger Mark-up Language), To enable more point and click debugging within windbg.
• Support for a SQL Database backend to allow storing information about debugging sessions or binaries in a SQL Database.
• Warnings when critical thresholds are exceeded such as han

Build Instructions

Before we being, please only check in code that builds with no warnings. (Do not break the build!)

The project requires you install the Windows DDK http://www.microsoft.com/whdc/devtools/ddk/default.mspx
The latest debugger and debuger SDK. http://www.microsoft.com/whdc/devtools/debugging/default.mspx
The latest Platform SDK

You will need to use the DDK Build environment to build the project.

Cmd.exe /k C:\WinDDK\6000\bin\setenv.bat C:\WinDDK\6000\ chk WNET

There is a setpaths.cmd file that is part of the project that sets these paths up for you.

set include=C:\debuggers\sdk\inc;%include%
set lib=C:\debuggers\sdk\lib\i386;%lib%;
set include=C:\winsdk\include;%include%
set lib=C:\winsdk\lib;%lib%;
set path=C:\codeplex;%path%
set DBGSDKLIBPATH=C:\debuggers\sdk\lib
set PSDKLIBPATH=C:\winsdk\lib

To build the project from your DDK Build environment change to the ODbgExt directory and type bcz
This will produce the following output and will build the binary based on your DDK Build environment settings. (ie Free, Checked, x86, or x64)

C:\Debuggers\sdk\samples\ODbgExt>bcz
BUILD: Compile and Link for x86
BUILD: Start time: Tue Apr 08 20:48:21 2008
BUILD: Examining c:\debuggers\sdk\samples\odbgext directory for files to compile.
BUILD: Compiling c:\debuggers\sdk\samples\odbgext directory
NTTARGET_VERSION SET TO WINXP
Compiling - odbgex.rc
Compiling - odbgext.cpp
Compiling - odbg.cpp
Compiling - generating code...
Building Library - objfrewnetx86\i386\odbgext.lib
BUILD: Linking c:\debuggers\sdk\samples\odbgext directory
NTTARGET_VERSION SET TO WINXP
Linking Executable - objfrewnetx86\i386\odbgext.dll
BUILD: Finish time: Tue Apr 08 20:48:23 2008
BUILD: Done

5 files compiled
1 library built
1 executable built

Source code checked in

jeffda — Tue, 29 Apr 2008 16:55:14 GMT

Sample for setting up your build env.

Source code checked in

jeffda — Tue, 29 Apr 2008 16:48:22 GMT

This is the initial code for the ODbgExt open source debugger extension project. We will build from here. Jeffda