Monday, December 18, 2006

shell extensions gotcha

Jeff Shrager posed this question back in September regarding his desire to create a Windows Explorer shell extension using Lisp (not specifically CL). I replied thusly and he later indicated that he'd had success.

It didn't occur to me at the time, but I've just now discovered that implementing shell extensions via certain implementations of Lisp, or any other single-instance runtime environment for that matter, might not be such a good idea. This thread has discussion of a similar idea, in this case involving the CLR. As a couple knowledgeable people point out in the ensuing discussion, the problem is that every application that does a mundane thing like invoking the common file dialog suddenly gets the runtime you used for your extension (Lisp or CLR or whatever) injected into it. Chaos could ensue for several reasons, for example if your runtime needs to reserve large chunks of virtual address space for itself that the host application isn't expecting and/or can't satisfy. Not to mention the potential inability of two such runtimes to co-exist in the same process, such as when you develop a GUI app that uses the same (or different!) version of said runtime.

There are easy-to-find examples and shareware floating around the Net for doing this. Nonetheless, I'd take Jesse Kaplan's and Raymond Chen's opinions at face value and think really hard about which Lisp I was using before venturing down such a path.


Anonymous said...

If i'm not mistaken, you're saying that the memory usage of lisp prevent s it to be used in a lot of cases.

This is so true.

(format nil "Hello world~%") will generate a several megabyte executable.

This restricts lisp and other image based programming language (smalltalk) to a small niche, mainly server side programming.

Jack Unrue said...

I wasn't referring to memory usage in the sense of a hello-world app being a 10M or whatever executable. I had SBCL in mind as an example, since I know from looking at the code that (without the relocation patch that's been floating around) SBCL wants to *reserve* certain virtual memory addresses for its use and if it can't, this is considered a fatal error -- but how much of that address space is actually committed and used depends on your code.

But yeah, I think you've got a valid point in that multi-megabyte shell extensions are probably less than ideal.

ken said...

The way Java (yeah, I used the j-word) solved this is by having a big Java Runtime which you download once, and then compiled Java apps just call into this. I wonder if it would be possible/easy to come up with an sbcl.dll that would let you compile and run small apps.

If you disassemble "hello, world" in your favorite Lisp compiler, it's actually pretty small. It's the rest of the CL universe that's big (and in Lisp it's basically impossible to figure out what exactly your app needs). But if all apps got to share a sbcl.dll, hello.exe could be quite small.

Then you might need to get into tricky areas like simultaneous access to Lisp images, which sounds hard. But way cool! :-)

Jack Unrue said...

Yes indeed, folks on #lisp mention the sbcl.dll project idea once in a while, sometimes in jest but I'm sure there are more than a few who would use such a thing. As you say, it seems like a hard project, especially with other infrastructural work going on at the same time (Win32 threading and Mach-style exception handling, for two examples).

Last time I looked at the TODO/Wishlist pages for CLISP, I don't think I saw any mention of a similar idea proposed there.

Other than knowing that the 'E' in ECL stands for 'embeddable', I don't have much experience with ECL. It would be interesting to hear whether juanjo has thought about image sharing.