ActiveBlog

Concurrency in Tcl: Events without getting Twisted
by Andreas Kupries

Andreas Kupries, May 6, 2010

So you wish to perform several things in parallel, or at least concurrently enough that the user can believe they are happening in parallel? Tcl supports this in multiple ways, actually too many for a single post to cover, so this post is the start of a small series about concurrency in Tcl. Here we'll talk about events and their management, followed by a post about threads, and finally a post about how we can make the parts of a parallel application talk to each other.

History

Historically speaking, Tcl's support for events and their handling started in the Tk package, written the year after Tcl's introduction at the 1990 Usenix conference. From there it moved into Tcl proper in 1996, around Tcl 7.5, more than a decade ago. Ever since then we've enjoyed event support as part of the core language, without a need for a separate package (like Python's Twisted or Ruby's EventMachine).

Example - Event-based Network Server

Here's an example of a network server, a common case for the use of events:

  1. socket -server NewConnection 7777
  2.  
  3. proc NewConnection {channel peerhost peerport} {
  4.     fconfigure $channel -blocking 0 -buffering none
  5.     fileevent $channel readable [list NewData $channel]
  6.     return
  7. }
  8.  
  9. proc NewData {channel} {
  10.     if {[eof $channel]} {
  11.         close $channel
  12.         return
  13.     }
  14.     if {[gets $channel line] < 0} {
  15.         return
  16.     }
  17.     puts $channel $line
  18.     return
  19. }
  20.  
  21. vwait _dummy_variable_

An echo server in 21 lines. First (line 1) we create and initialize a listening socket and have it call a Tcl procedure whenever a new connection is made. This procedure (lines 3-7) arranges for the channel to call our main procedure whenever new characters arrive on the socket. It handles 'read'able events. The data handler (lines 9-19) closes the connection if the end of the input has arrived (lines 10-13) and ignores the call if there is no complete line in the input (lines 14-16). Otherwise, the input is echoed back to the other side (line 17). At last, in line 21, we start the event loop. This will run until either the variable is written to, or no events are possible anymore. If I know that the loop should not stop I often write 'vwait forever', using the variable name argument to convey a bit more meaning. Other commands for event handling are 'after', arranging for the delayed execution of a bit of code, and 'update', for a one-shot run through accumulated events. Tk of course, still uses events as well, which you can 'bind' to. For more information go to the Tcler's Wiki, where 'Event programming and why it is relevant to Tcl/Tk programming' is a good starting point.

Limitations

It is not all roses though. The event handlers are not truly run in parallel, which means that a long-running handler, like a complex calculation, will block the processing of any other events. One solution to this is to sprinkle 'update' in the code of such handlers, but that has its own problems. Another is to slice the handler into lots of small pieces, each of which runs quickly, and which call each other indirectly via events ('after'). This is called continuation-passing-style. The problem with this is that we cannot use local variables to contain our state, but have to schlep them around in either (semi-)global variables, or in arguments of the various handler procedures.

Outlook

This is part of the reason why Tcl 8.6 got coroutines, allowing us to write the handler in a normal linear fashion and the slicing and state transfer (aka conversion to continuation-passing-style) is done for us by Tcl, under the hood. A form of green threads, so to speak. The use of threads (and sub-processes) is another solution to the problem of long-running calculations, one which is out of scope here. The next post in the series will delve into that.

Subscribe to ActiveState Blogs by Email

Share this post:

Category: tcl
About the Author: RSS

Andreas Kupries is a senior Tcl developer at ActiveState where his focus is the ActiveTcl distribution. He also works on the Tcl Dev Kit component of ASPN Tcl. Andreas is a member of the Tcl Core Team. He's the Tcllib release manager and has authored more than 25 Tcl modules and extensions! He has a Master in Computer Science from the RWTH Aachen.

Comments

2 comments for Concurrency in Tcl: Events without getting Twisted
Permalink

Hello Andreas,
thank you for the post and the example.

I have a question on the code.
I experienced, that eof is only updated, when there was a read operation before.
Thus I do the eof check after the gets.

Is this different in the context of this example?
I would have made the handler as follows:


proc NewData {channel} {
set res [gets $channel line]
if {$res >= 0} {
puts $channel $line
}
if {[eof $channel]} {
close $channel
}
return
}

This is a sketch. I experience this on windows and normally use "read" instead "gets".

Regards,
Harald

Permalink

> I experienced, that eof is only updated, when there was a read operation before.
> Thus I do the eof check after the gets.

True. However, please remember that the fileevent handler is called not only when new data appears, but also when the channel gets to EOF. In my implementation the EOF check for the current gets/read happens in the _next_ invocation of the handler.

> I would have made the handler as follows:
> [... code elided...]

Your code works as well.

From a higher level perspective, i.e. looking at the semi-infinite series of read/processing and eof-check we have simply chosen different groupings of the pairs into the handler. Mine is

(eof read) (eof read) ...

and yours

(read eof) (read eof) ...

The overall sequence is identical.

> This is a sketch. I experience this on windows and normally use "read" instead "gets".

When using 'read' there is actually a small difference between the two approaches. In your approach you have to deal with the fact that the channel may not be eof after your read, but recognized later, invoking the handler with eof before you are reading. As such you have to be prepared for a short read returning nothing, forcing you to skip over the processing before you come to your eof check.

In my approach of generally testing eof in the next round this additional check is not necessary, I can be sure that there is data by the time I came to my read.

In the case of 'gets' both approaches need the additional check because 'gets' may return a short read even if there is data present, i.e. when the data does not contain a complete line.

I hope that explains the more intricate details.