More about my issues with DTrace's language

June 29, 2012

In his comment on my entry about why we haven't taken to DTrace, Brendan Gregg wrote in part:

It's been mentioned a few times, but I suspect it would be possible to create a higher level language ("D++") that speaks to libdtrace. An advantage of D being low level is that the user is conscious of how the system is actually getting traced, in the same way that C is low level. [...]

I don't think that this is the case. In fact I think it works the other way around; I doubt very much that people can go from D's limitations to any real understanding how the system is traced, but if you know how DTrace is implemented you can see the bones of this implementation underneath some of D's oddities.

To start with, I will agree that making some things clear is useful and even important. For example, I think that access to kernel data and variables should look different than access to user level data (and in a way that makes access to user level data look more expensive). What I object to is things that D makes pointlessly difficult, things where it doesn't support the obvious simple way of doing whatever and forces you to be indirect. The shining example of this is conditionals. D does not have any form of an if statement that you can use in the action that fires for a particular probe; however, probes themselves can be conditional, based on an expression. So you're left to fake an if by writing your entire probe action twice (and yes, I've done this in DTrace code).

The story I remember hearing about why this limitation exists is that the DTrace implementation doesn't want to be dynamically allocating output buffer space as a probe action executes; it wants to allocate the space once, before the probe's action starts. Well, fine, but if this is the reason you can deal with it in if-using D code by allocating the maximum amount of space the code might need if it followed the most pessimistic, space consuming path through the conditionals. Alternately, you could transform ifs by automatically creating multiple specific probes with probe conditions. Forcing DTrace users to duplicate their code in order to do this by hand is perverse, or at least an excessive focus on being literal about how DTrace's internals behave.

(You can argue that it saves users from themselves under some circumstances, for example if a rare condition requires a bunch more buffer space than the common ones. But this is an optimization and generally a premature one.)

Now, this story is clearly not the complete explanation given that DTrace has plenty of things that certainly look like they create variable sized output (including an outright ternary ?: operator). This pretty much illustrates my point, in that running into this D constraint hasn't made me any better informed than before about how the system is actually traced. It's still a black box, it's just a more frustrating black box.


Comments on this page:

From 67.188.160.90 at 2012-06-29 06:22:43:

I'm glad you brought up "if", since that may help explain this further.

The internals of DTrace execute using probe/predicate/action. With the probe specified in the ECB.

The syntax we write is also probe/predicate/action. That matches the implementation.

Given the DTrace architecture, and a desire for if/then/else, an implementation may (at compile time) map the if/then/else into two separate probe/predicate/action blocks. Now, the syntax does not match how it is actually implemented. If we do add if/then/else, then how about adding loops too? This could become out of hand, where the mapped implementation vastly differs from the written syntax, and has performance implications that are no longer obvious, and potentially dangerous. (I never fully understood this myself until I heard Adam describe it recently; and he can probably describe it better than I can.)

It may be possible to change the DTrace internals so that the implementation can resemble the if/then/else syntax (extend ternary?). This may be a significant change, for what some may describe as unnecessary syntactic sugar. But is this sugar what's stopping DTrace adoption?

- Brendan

By cks at 2012-06-29 11:15:22:

I think that there's a straightforward line that can be drawn between if (which I would put in) and loops (which I would not). The difference between the two is that right now D doesn't have loops in any form but it does have conditionals in some form, so you can implement if by hand yourself (it's just very awkward).

In short: given that D has conditionals I think that it should make them convenient to use. Given that it doesn't have loops at all, I don't think it needs to add them (in any form).

But is this sugar what's stopping DTrace adoption?

My answer is that it makes D scripts harder and more annoying to write, which discourages people from doing it, which doesn't help DTrace. It doesn't stop adoption by itself but it adds friction, and friction adds up.

Written on 29 June 2012.
« Why I don't like implicit string interpolation
The magnitude of the migration to Python 3, illustrated »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Jun 29 01:52:20 2012
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.