Wandering Thoughts archives


Python's extra-clever help() function

I recently used 'help()' on one of my own internal modules, mostly as a quick way to browse the entire function/class collection in one go. Now, I'm kind of a slacker when it comes to using docstrings (for some reason I prefer comments before the function), so I didn't expect to see very much in the help() output.

Much to my surprise I saw things like:

  # Extract the host from the
  # REQUEST_URI, if it is present.

The amount of introspective magic required to do this turns out to be pretty impressive; for extra points, it's pretty much all done in Python.

The help() function is a shim created in site.py as an instance of a class. The function wraps pydoc.help(), which winds up calling on inspect.getcomments() for the heavy lifting. This works by finding and scanning the source file itself, starting from the co_filename and co_firstlineno attributes that the bytecode compiler glues on things.

(I should probably get over my reluctance to use docstrings, especially if I'm putting the exact same information in the comments. At the moment I like the visual look of comments better; docstrings make me feel that the documentation is quietly disappearing into the rest of the code.)

python/CleverHelp written at 22:08:22; Add Comment

'In place' filesystem defragmentation with Disksuite

While the Berkeley FFS and derivatives like Solaris UFS are much, much better at dealing with fragmentation than the original V7 filesystem (which continued on into System V before SysVR4), they can actually get fragmented over time to a degree that matters.

Traditional Unix, Solaris included, really doesn't have any tools for defragmenting filesystems; instead, you get to to do it the brute force way, by copying everything into a clear filesystem. With Solaris Disksuite and mirrored disks, it is possible to do this 'in place', so that you don't have to copy twice or remount the filesystem on any NFS clients.

The procedure is slightly less nerve-wracking if you have a three way mirror, but should work OK even for a two-way mirror. Here's how it goes:

  1. bring the machine into single-user mode, but do not unmount the filesystem you want to defragment.
  2. insure that all the submirrors are in sync.
  3. metadetach all but one submirror.
  4. rm everything in the filesystem except the lost+found directory.
  5. ufsdump 0f - /dev/md/rdsk/<detached submirror> | ufsrestore rf -
  6. check and make sure the data is there. No, really, do it twice.
  7. metattach the detached submirrors.
  8. bring the machine back up multi-user. (Optionally, first wait for the resync to finish.)

You're done. NFS mounts are even still intact, although anyone with an open file or a current directory in a subdirectory of the filesystem will see a few small problems.

(In our case it was a mail spool, so there weren't any subdirectories to worry about.)

There should be a similar procedure with Linux software RAID, although it'll probably be slightly more troublesome since the Linux equivalent of metadetach is somewhat more abrupt. (I'd probably unmount the filesystem before ripping the mirror off.)

PS: I am not sure if I am grumpy that Sun didn't make 'metattach' be called 'metaattach' instead, for complete consistency among the Disksuite command names.

solaris/InplaceDefragmentation written at 17:26:57; Add Comment

Page tools: See As Normal.
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.