splittist's Journal

6:23pm: Emacs newbies - don't swap Ctrl and Caps Lock

Newcomers to Common Lisp are encouraged to use Emacs (see also Finseth) so that they can use the fabulous Slime (and one should not forget paredit) as a development environment. Those that don't insist they'll stick with Vim (and then unaccountably fail to develop slim-vim), or try Cusp, will at some early point be advised to swap their Control and Caps Lock keys to save wear and tear on the pinkie. Even EmacsWiki points to typically forthright Steve Yegge piece which leads off with this advice.

I say - No. The proper place for the Control key is not "on the home row" but on each side of the Space Bar, where Tom Knight and the mothership intended them to be. And outside the Control keys should be the Meta keys, and outside them should be the Super keys, and finally the Hyper keys. This allows for what I think is a very natural way of typing key-chords - the modifiers with one hand, and the letter/symbol with the other. These are, after all, the sorts of keyboards on which Emacs was first developed.

How you do this will depend on your OS and your keyboard. For Windows, you might like to start here or here.^* In OSX you can look in System Preferences > Keyboard & Mouse > Modifier Keys. For Linux, there are a thousand xmodmap and xkb tutorials...

The other keyboard reconfiguration I recommend is, where possible, to put the parentheses on unshifted keys (swapping with the [square] brackets works on some keyboards). Of course, if, like me, you live/work in a locale with all manner of chicken scratchings on top of proper latin letters this may not be possible or convenient. (Eric Naggum's more radical approach can be seen here.)

So, my advice for Emacs (Climacs, VBacs) newbies would be: don't swap Caps-Lock and Control, swap Alt, Ctrl, Windows, Application, AltGr...

---

* For example, if my Windows XP notebook had the following key configuration to the left of the spacebar (from right to left, inside to outside) Alt Windows Ctrl and I wanted it to be Ctrl Alt Windows, I would do the following:

1. Fire up regedt32.exe and navigate to HKEY_CURRENT_USER\Keyboard Layout.
2. Create a new entry Scancode Map (using Edit > New > Binary Value).
3. Edit the new entry (using Edit > Modify Binary Data) to contain the following:

       0000   00 00 00 00 00 00 00 00
       0008   04 00 00 00 1D 00 38 00
       0010   38 00 5B E0 5B E0 1D 00
       0018   00 00 00 00
   

   The table is in little-endian form. The first 8 bytes are header (all zeroes). The next four bytes contain the count of entries (4, including the sentinel). The three following four byte entries swap Left Alt (0x38) with Left Ctrl (0x1D), Left Win (0xE058) with Left Alt, and Left Ctrl with Left Win. The last 4 bytes are a null terminator/sentinel.
4. Close up, log off, log on, and enjoy.

If you did this, of course, it would be entirely at your own risk.

6:39pm: Climacs and Java

Hot on the heels of the rudiments of a C syntax for Climacs, I'm about to commit a Java(TM) syntax. This one is slightly better, and should lead to improvements in the C syntax, too. (And, eventually, some refactoring of the - many - common elements.) Like the C syntax, this one does not yet try to parse too much Java grammar, but in the absence of a preprocessing stage, I have less excuse. On the other hand, the progressive layering of things like annotations (which look to this non-Java person like an afternoon's work with a thin macro layer over symbol plists) and generics make the Java grammar distinctly grody. It's a pity about infix, too...

"Java" is a trademark or registered trademark of Sun Microsystems, Inc. in the United States and other countries. They have an awful lot.

8:29pm: C and Lisp and Climacs and stuff

As a diversion from all the things I promised myself I'd be doing after ilc07, I've ended up playing with a C language syntax (mode) for Climacs.

When previewing on #lisp, the first question everyone had was "how accurately does this parse C?" Well, the answer is: at the moment, not at all. And I thought that, in the presence of C's textual preprocessing facility*, that's the way it would have to be. But looking briefly at emacs' cc-mode, it looks like it ends up doing the closest thing to a full parse anyway, so I have some way to go...

More seriously, the C syntax uses the same LR parsing infrastructure that the existing Lisp syntax uses, so I've begun to abstract that out into its own LR-parsing mixin syntax. (I love mixins.) But it's early days yet.

(This posted to shame me into actually delivering a patch to McClim reasonably soon rather than waiting for things to bit-rot. Which is the other thing -- the wonder that is Drei means that this syntax is available to all text gadgets in McClim. Think of the possibilities...)

* "Many other changes occurred around 1972-3, but the most important was the introduction of the preprocessor. ... Its original version was exceedingly simple, and provided only included files and simple string replacements: #include and #define of parameterless macros. Soon thereafter, it was extended ... to incorporate macros with arguments and conditional compilation. The preprocessor was originally considered an optional adjunct to the language itself. ... This attitude persisted, and explains both the incomplete integration of the syntax of the preprocessor with the rest of the language and the imprecision of its description in early reference manuals." Ritchie, The Development of the C Language, 1993, in History of Programming Languages-II ed. Thomas J. Bergin, Jr. and Richard G. Gibson, Jr. ACM Press (New York) and Addison-Wesley (Reading, Mass), 1996

4:39pm: ilc07 report
Well, ilc07 was pretty fabulous. The setting and the weather was unfeasibly beautiful. (I found/find it interesting that a not insignificant number of people were compelled to invoke Harry Potter in response.) The organisation was great, and the papers and presentations of a pretty consistently high standard. My notes follow. They will make most sense to those who were there, I guess.

Day 1

MPS: Disappointingly too much reminiscing about dead projects. I didn't mind the business anecdotes (although the frogs were unnecessary). The MPS system itself seems very interesting, but I felt we got only the merest glimpse.

Debugging: OK. Interesting that there seems little difference between a commercial lisp and something like SBCL. ll (low-level language, as opposed to LAP) seems an ideal target to which to compile C...

Day 2

Lisp: Themes and History: A presentation in two parts. The first a fascinating piece of lisp archeology. (LISP looked real ugly pre-parens.) The second a quick overview of semantically-preserving rewrite rules to compile lisp to a machine independent LAP. But why?

Constraint Propagation: Probably very good, but I didn't understand why I should care. As Christophe Rhodes has already mentioned, a number of the talks lacked the slide on the motivation for the research/approach/product/technique.

Binary DSL in Dylan: Engaging presenters. Two takeaways: there is life in Dylan yet, and procedural macros rule.

FREEDIUS: Looked cool, but what is it for? (I wonder if it can track a ball in real time?)

ESA: I enjoyed it (: Very useful for when you need that instant gui gratification.

Ltk: Like ECLM'06 but with Tetris!

Gene Expressions: Very dynamic, but didn't understand a word. Cyrus gets the prize for blatantly ignoring the timekeeper.

Teaching CS: Great presentation. I'll never read LiSP the same way again.

R6RS: Nicely presented, but it really did make Scheme seem the amateur little brother of Common Lisp. Square brackets? Case sensitivity?

Next 700 Libraries: I found this fascinating, but it seems some people have missed the point: it is a message of great optimism to demonstrate that automated assistance in the conversion of existing, substantial, documented and test-enhanced Java libraries is feasible.

Day 3

CL-HTTP: A report from a piece of the lisp world which seems to have been becalmed for the last 10 to 15 years... Very odd.

SC: S-expression-based C. Excellent slides. Who knew the future of C was sexps?

Crosscuts: A nice presentation, not least for introducing me to "the tyranny of the dominant decomposition".

Inter-domain Network Management: Key point: rewriting everything in Lisp was the only way of reducing the errors introduced by the C and Perl modules.

CL-SNA: Next year's ECLM in Istanbul should be a blast.

DAUTI in Scheme: Using such standard Scheme facilities as structs and macros, it took the author three evenings to get this going. Why so long?

Classification: Bayes meets Shannon. Probability meets belief. Umm, OK.

LispWorks Demo: And here's one I made earlier. Frankly, it didn't make me jump out of my seat (or some other appropriate Simon Cowellism...)

Dynamic Memory Management: A highly accomplished presentation from someone who really understands both the subject and the state of the art. Impressive.

ALU meeting: I will pass over this except to say (i) Europe and the ALU seem to be going their separate ways, and (ii) Rusty, I voted for you!

Day 4

HOP: Highly professional presentation backed up by 2-300k lines of scheme code. Something to explore one day.

Racer OWL: Description Logics are different from Object Orientation. Or Ontological Orientation. Or something.

List Comprehensions: LOOP!

CLforJava: Might have been better to take his own advice and not explain CLOS and MOP to the group. (Geoff -- what's wrong with "close"?)

Google SoC Lunch: Everyone hates Kenny...

DADTs: I missed the beginning of this, so it rather passed me by.

XMLisp: AgentCubes is very pretty. Which can't be a bad thing...

Extensible Sequences: Five implementations and counting. Christophe dealt nicely with some heckling from the pensioners' seats.

Liskell: Sexps are good for what ails you; Haskell means not having to write tests; and some very cute examples.

Scalable Lisp Applications: Some of this was interesting, but I came away with an impression much like Luke Gorrie's, I'm afraid. (Now there's a smart guy.)

Of course, the really cool stuff happened outside the conference room. But that's another story.

12:05pm: Looking forward to ilc07
I am very much looking forward to ilc07 after a few almost lisp-free months (too much day-job). My only contact has been the invaluable Planet Lisp and the odd quick browse of the #lisp logs. Certainly seems to have been lots of McClim activity in my absence (perhaps I should stay away?) - I'm sure my half-finished apps will need some serious forward-porting. Or perhaps I'll be converted to CAPI...

Predicted highlight: obviously ESA.

Possible break-out hit: CL-HTTP.

Most heat least light: ...

See you there!

1:09pm: FTD projectified
Following overwhelming^[*] demand, FTD, the Flexi Trivial Directory editor, is now a project on common-lisp.net.

FTD is now available via darcs get http://common-lisp.net/project/ftd/darcs/ftd or here.

Thanks to common-lisp.net mailing lists are at:

• ftd-devel
  for developers
• ftd-cvs
  CVS log feed.
• ftd-announce
  for announcements.

While FTD has a lot of dependencies, they are pretty common libraries^[&dagger] and it would be nice to know if it built and was able to get a directory listing on your system. Feedback to the devel list, please (:

---

* Not really.
&dagger I know, it's that old ‘Common Lisp has too many libraries’ complaint...

12:23pm: ... on a stick

What do you get when you put SBCL's windows port (helpfully packaged by Nikodemus Siivola), EmacsW32, Slime, McClim and Xming on a USB stick?

A nice little portable development environment with cross-platform GUI, of course!

6:48pm: FTD 0.0.3

I resurrected FTD, the Flexi Trivial Dired, after some months' hiatus. When I started the project last year it was to provide another ESA (Emacs Style Application) example, and to provide a test-bed for McCLIM application integration. Well, ESA was split out of Climacs anyway, and Clim Desktop made its appearance without a directory editor.

Still, it's fun to see how much of the Emacs Dired functionality can be obtained by putting together a bunch of common lisp libraries. Documentation and downloading instructions can be found here.

Being an ESA, FTD inherits all of the cool help functionality built into the ESA framework. Unfortunately, ESA won't write the documentation for you yet...

A couple of things makes FTD actually usable (from a UI point of view) in this revision. First, I remembered to specify :allow for the :end-of-line- and :end-of-page-actions. And, second, Troels Henriksen pointed out, after a few seconds of looking at the code, that I was using the default #'eql cache test, even though I was comparing cache-values of freshly consed lists each time through the display loop. Correcting this test (to :cache-test #'equal) prevented McClim from redrawing everything each time I moved the cursor...

Comments and patches welcome (splittist at yahoo com). If there is any real interest I'll set up a proper project.

8:23pm: Indenting LOOP

In working on indenting LOOP forms for Climacs (see the latest update here) I've come across a couple of competing ideas on how they should be formatted.

From cl-indent-patches.el ("Original version by rst" with patches by "salem" and "bromley" according to the header) we have the following "validation" (line numbers added by me):

 1 (loop for i from 0 below 2
 2       for j from 0 below 2
 3       when foo
 4         do (fubar)
 5            (bar)
 6            (moo)
 7         and collect cash
 8               into honduras
 9       else do ;; this is the body of the first else
10               ;; the body is ...
11               (indented to the above comment)
12               (ZMACS gets this wrong)
13            and do this
14            and do that
15            and when foo
16                  do the-other
17                  and cry
18       when this-is-a-short-condition do
19         (body code of the when)
20       when here's something I used to botch do (here is a body)
21                                                (rest of body indented same)
22       do
23    (exdented loop body)
24    (I'm not sure I like this but it's compatible)
25       when funny-predicate do ;; Here's a comment
26                               (body filled to comment))

The forms at issue are the DOs at 9-12 (and 25-6; compare with 20-21), 18-19 and 22-24. I would indent these like this:

 9       else do ;; this is the body of the first else
10              ;; the body is ...
11              (indented to the above comment)
12              (ZMACS gets this wrong)

(I see no reason for conforming to a dubious commenting style. But I'm interested to know why not doing so is "wrong".)

18       when this-is-a-short-condition do
19                                        (body code of the when)

(I see no reason to special-case a "short-condition", whatever that is.)

22       do
23         (exdented loop body)
24         (I'm not sure I like this but it's compatible)

(Exdenting is just gross - but "compatible" with what?)

Of course, I'm now doubting my own taste, so I'd be interested in what the Planet Lisp community thinks. Answers on a postcard to splittist at-sign yahoo period com for summarising in a later edition...

12:46pm: Climacs Search

Climacs has a bunch of search commands, but it needs more!

What it has at the moment are many of the standard ones:

• Incremental Search (forward and back) - C-s and C-r
• String Search (and Reverse String Search)
• Word Search (and Reverse Word Search)
• Regex Search Forward and Regex Search Backward
• How Many (lines after point match some regex)
• Replace String (unconditionally)
• Query Replace - M-%

and a few less common:

• Multiple Query Replace
• Query Exchange
• Multiple Query Replace From Buffer

Incremental Search and Query Replace have a fair subset of the GNU emacs functionality in their loops. While in an incremental search you can switch direction, add a newline to the search string (C-j), snarf the next word into the search string (C-w), or the remainder of the line (C-y), or the top of the kill ring (M-y) etc. While in a query replace loop you can replace, skip, exit, replace and exit, or just replace the rest of the matches.

So far so good for the normal stuff. You'll see that some functionality isn't there yet. Partially this is just that no-one has gotten around to it (and if your favourite functionality is missing patches are appreciated!). But partially this is because climacs isn't emacs, and we're still feeling our way towards what is the best way to use the command-loop, in which a lot of CLIMy magic happens.

As an example of the issues, take the emacsly-traditional way of invoking a string search - C-s immediately followed by the command to exit the search (RET in GNU emacs, Altmode in TWENEX EMACS, etc). The Incremental Search command works by setting up a special command table (a CLIM object imaginatively named isearch-climacs-table) that maps gestures to commands (so that, for instance, C-j is mapped to com-isearch-append-newline). The gesture #\Newline invokes com-isearch-exit. But when the search string is empty, #\Newline should run the command String Search (or Reverse String Search, if we're isearching backward). String Search prompts the user for a string to search for (!), and one of the beauties of CLIM is that the system handles this automatically. But not here, where we are inside another command (and, indeed, inside another command loop - but that's another story), so for the moment I have to reach behind the curtain and pull a few levers myself:

(execute-frame-command *application-frame*
                       (funcall
                        *partial-command-parser*
                        (frame-command-table *application-frame*)
                        (frame-standard-input *application-frame*)
                        (if search-forward-p
                            `(com-string-search ,*unsupplied-argument-marker*)
                            `(com-reverse-string-search ,*unsupplied-argument-marker*))
                        0))

Obviously this is uglyrific, but the moral of the story isn't that CLIM makes normal things hard (although there are some things about command tables that the CLIM 2 spec leaves rather underspecified), but that I don't yet know the best way to get this (admittedly rather special) behaviour out of the existing command loop. (One possibility: have a pending-commands slot that the command-loop checks, analogous to the remaining-keys slot used by the keyboard macro machinery.)

Another thing I don't understand is why GNU emacs doesn't have a command like Multiple Query Replace. This prompts for pairs of strings and then goes through the buffer asking if you want to replace any of the first strings of the pairs with the corresponding second string. Since this happens essentially in parallel, you can replace FOO with BAR, BAR with BAZ, BAZ with QUUX and QUUX with FOO in one invocation, without any messing around with temporary strings or abbrevs (as suggested as workarounds in the GNU emacs documentation). (The command Query Exchange is specialised for the case when you want to swap each FOO with a BAR and vice versa.) If you think entering the search and replacement strings in the minibuffer is prone to mistake, you can put them in a buffer and use Multiple Query Replace From Buffer. An essential refactoring tool, I would have thought.

Well, essential once climacs gets Tag Files (the Zmacs name for a collection of buffers or files, whether named in a TAG file, or all the open buffers, or all the files in a system etc.). Then various searching commands can be run over all the files in a Tag File. Which means the ability to suspend and resume a given search, and to visualise pending operations, becomes important.

And then there are the syntax specific searches that climacs should really have - things that allow you to ask for all lists of length 3 with CL:IF as the first element...

But all that is for another day.

10:04pm: Climacs Help

Climacs has quite a few help commands now. The picture above shows Apropos Command (in the lower panel) and Describe Command (in the upper panel).

The interesting thing is that the upper panel was obtained by clicking on the "Kill Line" heading below. And the magic involved?

(present command
         `(command-name :command-table ,command-table)
         :stream out-stream)

when printing and

(define-presentation-to-command-translator describe-command
    (command-name com-describe-command help-table
                  :gesture :select
                  :documentation "Describe command")
    (object)
    (list object))

to make it clickable.

Which just goes to show some of the cool things you get for free if you use CLIM.

The full list of help commands currently available is:

• Apropos Command (C-h a)
• Describe Bindings (C-h b)
• Describe Key Briefly (C-h c)
• Describe Command (C-h f)
• Describe Key (C-h k)
• Where Is (C-h w)

In fact, all these commands are part of ESA, and should therefore be available to any Emacs Style Application written using that framework.

12:45pm: Belated ECLM impressions

ECLM 2006 was my first Lisp event. In fact, it was my first programming event. I had a great time.

As the other blogging attendees have noted, the presentations were of varying interest. The AllegroGraph presentation was a bit once-over lightly, and never touched upon telecom fraud detection (unless that's what Osama Bin Laden was wanted for). In contrast to many others I found James Anderson's talk quite interesting, if only for the "here's something interesting and perhaps counter-intuitive we found when implementing the system" moments. Arthur Lemmens, Martin Cracauer and David McClain all went for a low key presentation style, which worked with their material and the crowd. But David McClain's late-night-DJ voice, impressive pictures and fascinating background stole the show.

The most common question in the breaks, asked with the expectation of a negative answer, was "Do you use lisp in your work?". The negative answer was, of course, forthcoming in the vast majority of cases.

It was fun putting faces to nicks. Andreas Fuchs managed to look nothing at all like antifuchs, for some reason I can't articulate. Whereas Juho Snellman did look like jsnell. (Christophe Rhodes is never Xophian, because the fangs barely show IRL...).

Many thanks to Edi and Arthur!