The PLT Scheme advantage

I was doing a lot of builds at work today. While waiting for it to complete, I decided to spend some time coding in Lisp. Clozure CL host irc logs for #ccl, #scheme and #lisp on their website. I thought it would be cool idea to download the logs for a particular date by writing a CL program. 

What it turned out to be was a huge waste of time. I decided to try the trivial-http library. I downloaded the tarball. Then I didn't quite know how to install it. Some amount of googling gave me links to asdf-install. I then spent the next couple hours trying to get it work. It doesn't work. Checked on #lisp, and #ccl but no responses. 

In frustration, I turned on DrScheme and grokked the documentation. There is a net/url library that is shipped with PLT Scheme. Two minutes later my code looked like so

(require net/url)

(display-pure-port (get-pure-port (string->url "http://ccl.clozure.com/irc-logs/lisp/2009-12/lisp-2009.12.28.txt")))

I'm not sure if any other scheme (other than maybe chicken) would have allow me to get going so quickly.  I doubt whether any further example is required for a batteries included distro of CL. 

Update: Finally got something in CL - a TCP stream descriptor. So, we aren't there yet. But here is how it goes. 

1. Don't bother about asdf-install - it takes care of downloading asdf packages in order and compiling them. To use ASDF packages, we don't need asdf-install.
2. Download trivial-http and usocket asdf packages and untar them. I put them in my libcl installation. My libcl is in d:/programs/libcl. It doesn't need to be here, but having it here makes my directory structure clean.
3. Update ccl-init.lisp for asdf:*central-registry* to see the new asdf locations. Eg. for usocket we do - (pushnew #P"d:/programs/libcl/usocket/" asdf:*central-registry* :test #'equal) 
4. At the REPL compile the new asdf files. E.g for usocket we do - (asdf:operate 'asdf:compile-op 'usocket)
5. Next at the REPL load the compiled asdf. We need to load only trivial-http. ASDF automatically loads usocket - (asdf:operate 'asdf:load-op 'trivial-http)
6. We can now see that that trivial-http is available by doing (list-all-packages).
7. Finally, (require 'trivial-http) and (trivial-http:http-get "http://ccl.clozure.com/irc-logs/lisp/2009-12/lisp-2009.12.28.txt")

What we have now is just a stream. We now need to read the stream. That's for later.

1. 
   
   

Size of a file

Rosetta Code is an interesting project where you can see code in different languages for the same problem. Its a wiki project and you are encouraged to put in your solution if one does not exist in your language of interest.

Here's a small bit contribution from me - Get the size of a file in scheme.

(define (file-size filename)
 (call-with-input-file filename (lambda (port)
   (let loop ((c (read-char port))
              (count 0))
     (if (eof-object? c)
         count
         (loop (read-char port) (+ 1 count)))))))

(file-size "input.txt")
(file-size "/input.txt")

Lisp Epiphany - Native Code vs Byte Code

What does "Native code" Compiler mean in Lisp terms? If we look at SBCL which is a Native Code compiler and compare it against CLISP which is a Byte Code compiler, then we can get a better idea.

Coming from a C and Scripting background, the first thought that came to my mind was that a Native Code compiler generates executables whereas a Byte Code compiler does not. Once I dug in a bit more, I realized that is not the case. To assume SBCL is a sort of GCC that reads in code and spits outs an ELF executable is completely wrong. It does nothing of the sort.

Both SBCL and CLISP can save environments as a standalone executable. So, there is no real difference from an application delivery perspective. The real difference comes when we look at how SBCL and CLISP compile functions that we define.

Common Lisp has an environment function - disassemble. If we define a simple function and pass that function to disassemble, then we can see how different SBCL and CLISP are.

Consider the following function -

 (defun square (x) (* x x)) 

Its pretty straight-forward. Just a function that squares its input. Now we call disassemble on this function.

 (disassemble #'square) 

In CLISP we see the following output

Disassembly of function SQUARE
1 required argument
0 optional arguments
No rest parameter
No keyword parameters
4 byte-code instructions:
0     (LOAD&PUSH 1)
1     (LOAD&PUSH 2)
2     (CALLSR 2 57)                       ; *
5     (SKIP&RET 2)
NIL

whereas in SBCL we see something completely different -

; disassembly for SQUARE
; 23B6C274:       8BD3             MOV EDX, EBX               ; no-arg-parsing entry point
;       76:       8BFB             MOV EDI, EBX
;       78:       E8304049FE       CALL #x220002AD            ; GENERIC-*
;       7D:       7302             JNB L0
;       7F:       8BE3             MOV ESP, EBX
;       81: L0:   8B5DFC           MOV EBX, [EBP-4]
;       84:       8BE5             MOV ESP, EBP
;       86:       F8               CLC
;       87:       5D               POP EBP
;       88:       C3               RET
;       89:       CC0A             BREAK 10                   ; error trap
;       8B:       02               BYTE #X02
;       8C:       18               BYTE #X18                  ; INVALID-ARG-COUNT-ERROR
;       8D:       4D               BYTE #X4D                  ; ECX
NIL

What we have in SBCL is real machine code whereas in CLISP we see its Virtual Machine byte code. That's the real difference. SBCL actually compiled our square function into machine code but CLISP didn't.

Both approaches have their advantages and disadvantages but I won't be putting that in this post.

CLisp application delivery

Here's the simplest way to deliver an Hello World program on Windows. I'm going to use clisp for this, which does not compile to native code, but only bytecode.

The program that prints hello world is simple - 

(defun hello-world ()
 (format t "Hello, World~%"))

(progn
 (hello-world)
 (format t "Enter any key to exit...")
 (read)
 (exit))

Save the code in as hw.lisp and now fire up clisp. At the repl do the following

[1]> (compile-file "hw.lisp")

This will now generate a bytecompiled file - hw.fas. 

Now, save an image of the clisp

[2]> (saveinitmem "hello.exe" :executable t :quiet t
                 :init-function #'(lambda () (load "hw.fas")))

Now, this will generate an executable - hello.exe. 

Copy hello.exe, hw.fas into a new directory that you want to use as the installation target directory. Copy readline5.dll, libintl-8.dll and libiconv-2.dll from \base to this new directory.

Now you can zip this directory or create an installer using InstallJammer or NSIS. 

That's it!

One caveat though. You may need to ship readline as well. That's available in \readline.

GUI with Gambit

There doesn't seem to be any real GUI libraries for Scheme. I spent
some time looking for a GUI framework that I can interface with
Gambit. WxWidgets was the first place I looked. I wasn't fully sure
that I could interface a C++ GUI framework with Gambit. The docs
mentioned it, but I didn't want to spent time exploring and debugging
C++ issues this time around.

Tk is a GUI library that is cross platform, works almost everywhere
without a hitch. I had spent a lot of time coding in Tcl/Tk a few
years back, and so getting back on the Tk bandwagon looked
appealing. The latest "Tile" updates to Tk also improves the
look-and-feel considerably. So, what options did I have for Scheme +
Tk?

On obvious way is to look at STKlos. The only reason, I didn't want to
do that, was because I wanted to stick to Gambit for the
moment. Gambit allows me to create a standalone application. Having a
standalone exe simplifies application delivery, esp. on Windows.

My first thought was to have a native interface to Tk. I thought about
a FFI interface to the Tk dll itself. Again no luck here. There was a
Snow package for Tk available but Snow itself seems to be dead with no
development happening on it.

I came across PS/Tk. PS/Tk uses Tcl as a bridge to Tk. You need to
have TCL installed though. We can get past this, if we use tclkit and
so, it looked like a good starting point.

The whole PS/Tk is just one file - pstk.scm that you download. Just
load the file and it should be good to go.

A-ha one Gotcha!

That doesn't work. What you need is to modify the file for specific
scheme support. There is example code written up for each scheme
implementation that PS/Tk supports and its just a question of
uncommenting the right one. For Gambit the code looks like so -

; GAMBIT
;   (run-program
;     (lambda (program)
;       (let ((port
;               (open-process
;                 (list path: "/bin/sh"
;                 arguments: (list "-c" (string-append "exec " program))
;                 stderr-redirection: #t))))
;         (list port port))))
;
;   (flush-output-port force-output)

Uncommenting this won't do as its Linux specific. Some small changes
and we should be able to use it on Windows. The changes are -

; GAMBIT / Windows
(run-program
(lambda (program)
 (let ((port
   (open-process
    (list path: "c:/tcl/bin/tclsh85.exe"
   stderr-redirection: #t))))
   (list port port))))

(flush-output-port force-output)

Once that's done, we are good to go. We can create a simple frame with
a button like so -

(define f (tk 'create-widget 'frame))
(define b (f 'create-widget 'button))
(b 'configure 'text: "Hello World")
(tk/pack b)
(tk/pack f)

Next stop - change from a full fledged Tcl/Tk installation to Tclkit.

Building an hello world program using gambit

We know that gambit-c allows you to compile scheme code into native executables by generating C files and then running them through a C compiler such as gcc. There are real advantages of doing so. You can even compile scheme to run an app on the iPhone.

On Windows, follow these steps to write your first hello world program that gambit then compiles to an executable.

1. Download and install Gambit.
2. Download and install MinGW
3. Set your path to allow gsc to view gcc
4. Write your hello world program
5. run gsc -exe on the code and that's it.

You now have an exe that prints hello world.

Here's a small scheme example that prints hello world.

(define (hw)
 (begin
   (display "Hello World")
   (newline)))

(hw)

The simplest syntax-case

Have spent lots of time trying to understand syntax-case, yet there are no simple examples. At least until I came across this post.

Small change for Iron Scheme -

(define-syntax is-nil?
 (lambda (stx)
   (syntax-case stx ()
     ((_ ()) #t)
     ((_ stx) #f))))

The importance of quasiquote

If you were like me and skipped through most of the chapter on quote, quasiquote and splicedquote, then you made a bad mistake. This part is particularily important as the whole macro business is dependent on this. So here's a refresher.

Very simply put quote just quotes its arguments. So, when we execute the following, we get the output as test.

(import (rnrs))

(define foo 'test)
(display foo)

Quasiquote allows us to do more. It allows us have a template which we can fill in with values. Consider the following.

(import (rnrs))

(define x 10)
(display `(list x ,x))

The output will be (list x 10). However, the advantage of quasiquote goes much beyond just templates. We can use it to debug our code better. 

(import (rnrs))

(define (fact n)
(cond
((or (= n 0) (= n 1)) n)
(else (* n (fact (- n 1))))))

(display (fact 5))

Now, we want to understand how fact works. So, let us re-write fact not to compute the value but to just print out the expanded s-expressions.

(import (rnrs))

(define (fact n)
(cond
((or (= n 0) (= n 1)) n)
(else `(* ,n ,(fact (- n 1))))))

(display (fact 5))

Now the output is more interesting - (* 5 (* 4 (* 3 (* 2 1)))). What quasiquote and unquote ended up showing us is how the expression is calculated. Now, the fact example is trivial but it does give us an insight into the computation involved. 

Related to quote is spliced-quote. If we use spliced-quote then the output of a list is expanded. So, 

(import (rnrs))

(define foo `(1 ,@(list 2 3)))
(define bar `(1 (list 2 3)))

(display foo)
(newline)
(display bar)
(newline)

 will give us the output as (1 2 3) and (1 (list 2 3)).

Scheme - let-syntax

It took me some time to understand what let-syntax does. After
spending much time on the net search for information, I finally
figured it out. let-syntax is the let counterpart of define-syntax.

What does that mean? It means you create a macro that is lexically
bound. That means, let-syntax acts just like let, but it works on
syntax rather than variables. This means we can use let-syntax to
create localized macros. Consider this example..

(import (rnrs))

(define (hw n)
(display n)
(newline))

(let-syntax ((hw
(syntax-rules ()
((_ 0) (begin
(display "Hello World")
(newline)))
((_ n) (begin
(display "Goodbye World")
(newline))))))
(begin
(hw 0)
(hw 1)))

(hw 0)
(hw 1)

The (hw 0) and (hw 1) inside let-syntax end up printing "Hello World"
and "Goodbye World", whereas the (hw 0) and (hw 1) outside the scope,
print 0 and 1.

Windows Forms in IronScheme

The Discussion thread on Iron Scheme's project is here. I spent some time, playing with the code, and changing the style. Here's what I got.

(import
 (rnrs)
 (ironscheme clr)
 (ironscheme clr shorthand))

(clr-reference System.Windows.Forms)
(clr-reference System.Drawing)

(clr-using System.Windows.Forms)
(clr-using System.Drawing)

;; Macros

;; Macro to set a property
(define-syntax set-property!
  (syntax-rules (:button :form)
    ((set-property! :button button text x y)
     (let ((b button))
       (with-clr-type ((b Button))
        (b : Text = text)
        (b : Location = (clr-new Point x y)))))
    ((set-property! :form form text)
     (let ((f form))
       (with-clr-type ((f Form))
        (f : Text = text))))))
  
;; Main Function that runs the form
(define (run form controls b1 b2)
  (let ((mc controls))
    (begin
      (with-clr-type ((mc Form+ControlCollection))
       (mc : Add (b1))
       (mc : Add (b2))))
    
    ;; SHOW FORM AND RUN PUMP
    (clr-static-call System.Console WriteLine "Start")
    (clr-static-call Application (Run Form) form)
    (clr-static-call System.Console WriteLine "Stop")
    
    ;; REMOVE CONTROLS
    (with-clr-type ((mc Form+ControlCollection))
     (mc : Remove (b1))
     (mc : Remove (b2)))))
  


;; SETUP EVENTS
(define (make-event-handler text)
  (lambda (s e)
    (display text)
    (newline)))

(define mainForm_MouseEnter (make-event-handler "Enter"))
(define mainForm_MouseLeave (make-event-handler "Leave"))
(define btnGo_Click (make-event-handler "Go"))
(define btnStop_Click (make-event-handler "Stop"))

(begin
  ;; INITIALIZE
  (define mainForm (clr-new Form))
  (define btnGo (clr-new Button))
  (define btnStop (clr-new Button))
  (define mainControls (clr-prop-get Form Controls mainForm))
  
  ;; APPLY EVENTS
  (clr-event-add! Form MouseEnter mainForm mainForm_MouseEnter)
  (clr-event-add! Form MouseLeave mainForm mainForm_MouseLeave)
  (clr-event-add! Button Click btnGo btnGo_Click)
  (clr-event-add! Button Click btnStop btnStop_Click)
  
  (set-property! :form mainForm "Hello World")
  (set-property! :button btnGo "GO" 10 20)
  (set-property! :button btnStop "STOP" 100 20)

  (run mainForm mainControls btnGo btnStop)
  
  ;; REMOVE EVENTS
  (clr-event-remove! Form MouseEnter mainForm mainForm_MouseEnter)
  (clr-event-remove! Form MouseLeave mainForm mainForm_MouseLeave)
  (clr-event-remove! Button Click btnGo btnGo_Click)
  (clr-event-remove! Button Click btnStop btnStop_Click))

Emacs tramp mode vs. angeftp

Looks like I had it completely wrong!

Emacs tramp mode is really awesome. From windows if we use plink then everything just works, include things like vc-mode, make etc..

git-changelog

I like to use git tag to keep track of revision versions in my code. When I do deliver my code, its usually as a prebuilt binary with code, generated documentation and most importantly a CHANGELOG file. What does happen is that the entire .git tree is not shipped. (Hey, if you need the git tree, just pull off it. You don't need a release).

I tried git-log to generate a CHANGELOG but it does not give me a mapping to tags. So, here's a bash script that does that.


# Generate a nice changelog from git
PREV=
for I in `git tag`; do
echo " ";
git log --pretty=format:" %s" $PREV..$I;
echo " ";
echo $I;
PREV=$I;
done | tac > CHANGELOG


Works like a charm!

The difference between FILE* and FD

Obviously using fopen, fwrite etc... is a little bit easier especially since you don't have to keep track of the location in the stream, using plain old open, write etc.. also have their uses.

Consider this.. I want to write a test case which writes to a file over and over again. The idea is to write x bytes to the file, seek to 0, and repeat.

Problem is that this will not work when you use FILE pointers. It will work most times, except when the file itself has to be created. In which case, using seek to 0 does nothing, because every fwrite will automatically seek to the end of the file. So, fopen a new file will always open it in append mode.

Its my baby

Started implementing my own stack based language. I have played a bit with factor and liked it, so I decided to base some things from it. I also have started to read a little more on forth and have now understood how some basic constructs can be created. Especially useful is jonesforth. The code is a must read!

So, the design goals I have in mind are as follows.

1. Must be in embeddable in C. My main goal here is to be like Lua. A small library that you can call from C. The main users of this language would be hardcode C programmers esp. on embedded linux and embedded linux system testers. 
2. Must be written in ANSI C. Factor implementation has moved to C++ and also has a lot work going on in creating a fast and optimizing VM. I don't have that goal, as I want to be able to port my language very quickly to different hardware. The easiest way to port? Just recompile. Obviously, you lose out on some performance, but that's ok. The target audience will code most of the fast bits in C anyways.
3. Must be simple. System testers should be able to use this language very quickly to write small tests. The main part of the test application itself would be written in C, but the scripting part would be in this language.
4. Support for quotations, sequences. Ideas from Factor (and lisp) and good ones too!
5. Reader macros would primarily be from C. 

So far, I have the basic language working. It supports numbers, strings and quotations. You can define your own "words" and run them. There is a "if" control structure and a "loop" control structure. The implementation is still very buggy though. 

Will keep posting as I progress.

Factor and Unicode

I'm just starting with Factor. More for fun rather than anything else. Selling it at work will be a real task!

Anyways, I was curious about unicode support.

So, I tried to reverse a kannada word! Here's what I got.

( scratchpad ) "ಕನ್ನಧ"

--- Data stack:

"ಕನ್ನಧ"

( scratchpad ) dup

--- Data stack:

"ಕನ್ನಧ"

"ಕನ್ನಧ"

( scratchpad ) reverse dup reverse

--- Data stack:

"ಕನ್ನಧ"

"ಧನ್ನಕ"

"ಕನ್ನಧ"

( scratchpad ) drop

--- Data stack:

"ಕನ್ನಧ"

"ಧನ್ನಕ"

So far so good. But lets change the vowels!

( scratchpad ) "ಸಿದ್ದಾರ್ಥ"

--- Data stack:

"ಸಿದ್ದಾರ್ಥ"

( scratchpad ) dup reverse

--- Data stack:

"ಸಿದ್ದಾರ್ಥ"

"ಥ್ರಾದ್ದಿಸ"

( scratchpad )

Now, we have a problem! This is part of the FAQ.

Quoting -

Does Factor support Unicode?

There is no one meaning to the phrase "Unicode support", but there are a few things that a modern programming language is expected to support in its library: UTF-8/UTF-16 input and output, Unicode collation, Unicode-appropriate casing operations, normalization, strings can hold any Unicode code point, and support for Unicode text rendering in the UI. Of these, Factor supports all but Unicode font rendering, which should be finished before 1.0 comes out.

How do I convert a character to upper or lower case in Unicode?

This isn't a well-defined operation. For example, the ß character becomes SS in upper case. Some letters have context-dependent case mappings. So if you need to change the case of something, use strings, not individual characters. The Factor Unicode library doesn't implement character mapping, because the behavior could only be incorrect. If what you're converting is just ASCII, then there are character conversion routines defined just for that. For case-insensitive comparison, partial collation keys might be appropriate.

Lua is a god send

I have been spending some time integrating Lua with some code. Its really good. Adding Lua scriptability is almost trivial and the payoffs are just amazing.

Worth checking out. Lua homepage.

Perl Internal Variables

http://www.perl.com/pub/a/2004/06/18/variables.html . Must read perlvar sometime.

YASnippet mode - The biggest time saver

If you haven't tried YASnippet with Emacs yet, then do it ASAP. Its awesome.

Its a clone of textmate's snippets feature, but I don't have Mac and haven't seen/used it. YASnippet by itself is awesome!