Chris's Wiki :: blog/solaris/DTraceWhyNotII Commentshttps://utcc.utoronto.ca/~cks/space/blog/solaris/DTraceWhyNotII?atomcommentsDWiki2013-01-03T08:55:19ZRecent comments in Chris's Wiki :: blog/solaris/DTraceWhyNotII.From 67.188.160.90 on /blog/solaris/DTraceWhyNotIItag:CSpace:blog/solaris/DTraceWhyNotII:b1325ea5b397034f09154282b547bd37d16e9455From 67.188.160.90<div class="wikitext"><p>G'Day,</p>
<p>Thanks again - the "why not" issue for DTrace continues to be perplexing, so opinions like this shed light.</p>
<p>In your case, I think you ran out of runway with the stable, documented providers, and had to use the unstable and undocumented fbt provider to answer the questions you had. Using fbt well usually requires reading kernel source code, which isn't something most sysadmins of today can be expected to do (not system engineers). It's certainly a task that can be served by vendor support, similar to providing kernel patches.</p>
<p>In terms of difficulty, using the DTrace providers is a little like:</p>
<ul><li>fbt provider: writing a simple kernel patch</li>
<li>stable providers: writing a shell script</li>
</ul>
<p>Sysadmins should be able to handle stable providers (eg, io, proc, sched, vminfo). They are documented - you don't need to reach for kernel code. Programming them may be no more difficult than shell scripting.</p>
<p>It's fbt scripting, like writing a simple kernel patch, that may not be for everyone. It's great if you have the staff who can, and it's great that it's possible at all, but there aren't many out there who can (system engineers). I'm hoping we can train more, and increase the pool of DTrace skilled people out there (I've started running classes again). It should be a skill that all performance engineers, at least, have. In cases where you can't hire or train, you can still ask your support vendor to provide the scripts, like they provide kernel patches.</p>
<p>What about basic observability, like iosnoop, execsnoop, opensnoop, etc (DTraceToolkit). These aren't any more complicated (system engineering-y) than truss or snoop. But you don't want to open a support ticket to run snoop, nor do you want to have to write it yourself - I think you'd want basic tools like that provided with the OS, along with man pages and docs. Is this another practical avenue for sysadmins to use DTrace, but without needing to program DTrace themselves?</p>
<p>I think this may be what you are saying with "A mechanism to provide diagnostic tools is not a selling feature; at best, the diagnostic tools themselves are."</p>
<p>Next time I give a DTrace talk, should I teach how to develop DTrace scripts, or, how to use my DTrace scripts? :-)</p>
<p>- Brendan</p>
</div>2013-01-03T08:55:19ZFrom 86.135.244.55 on /blog/solaris/DTraceWhyNotIItag:CSpace:blog/solaris/DTraceWhyNotII:64c85b33be637d74c49e8b55888df1d3f31bd1d4From 86.135.244.55<div class="wikitext"><p>I only partly agree - most sysadmins these days do not even know what a syscall is and they are unlikely to be able to use DTrace directly. But there are still lots of sysadmins who do have required skills to use it and they are not necessarily kernel developers. I've been using DTrace since the very beginning and have been able to fixed lots of issues thanks to it, sometimes with spectacular outcomes. And I'm not an OS developer.</p>
<p>I also don't agree one has to understand kernel code or be able to read it to use DTrace - at least it depends on what kind of problems one is trying to solve. Obviously if a problem is in a kernel driver, etc. then it is unlikely it could be solved or even identified without elementary understanding of C and kernels... but then I've been able to root-cause and fix lots of issue with applications and the way they interact with OS without having to look into how kernel works or checking Solaris code. And in many cases, these problems would be very, very hard (and time consuming) to identify without DTrace.</p>
</div>2012-11-16T10:54:57Z