Case Study: Lua Debugging in Komodo

I had thoughts about trying something crazy with Komodo...so when ActiveState held its annual hackathon I saw it as a great opportunity for me to try out one of these ideas. Before I jump into what I did, I want to touch on something I said in an earlier blog post about Komodo's impressive debugging functionality:

The absolute coolest part about Komodo's use of the DBGP protocol for its debugging functionality is that once you understand the protocol, you can develop your own debug client completely independently of Komodo and then, when you are ready to integrate it with Komodo, it will slip on like a glove and just work.

My hackathon project was to get Komodo to debug Lua code.

The Challenge

Komodo already has some support for the Lua programming language: syntax highlighting and syntax checking. I wanted to see if I could throw together some basic Lua debugging support in less than 24 hours (the length of our Hackathon). I was going to take an off-the-shelf Lua debugger script that was written in Lua, and hack it up in order to turn it into a debug client that can send and receive DBGP messages. If all went well, I'd be able to plug-and-play this into a default Komodo installation with absolutely no configuration.

Plan of Attack

First things first, I needed a refresher on the DBGP protocol. After studying the Lua debugger script I downloaded, I noticed it had most of the main features the DBGP protocol needs, like the ability to set breakpoints, start execution, step into and over functions, and request a stack trace. However, some features were missing, like the ability to retrieve local variables in a stack frame -- I would have to write those myself. After figuring out what was missing and what was not, I determined my plan of attack:

  • Figure out the entry point at which debugging starts in Lua, and wherever that is, reach out to Komodo for a DBGP connection. Once the connection is established, construct and send the DBGP initialization packet to Komodo. Only then can the main Lua debugger loop be run.
  • Once debugging has started, the main debugger loop has an input loop that waits for instructions from the user (via the console) and executes them as given (e.g. set breakpoints, run, step, etc.). Instead of waiting for input from the user via console, wait for input over the socket connection that the debugger established with Komodo.
  • For each instruction received by Komodo, parse it and carry out the instruction using existing Lua routines or routines I needed to write.
  • Write a test script that initiates a remote debugging session with Komodo. This last step was important because if I wanted to have Komodo begin a debug session, I would need to edit Komodo in order to have it spawn the Lua debug client for Lua files and so on. I wanted to be able plug-and-play the Lua debug client with absolutely no modification to Komodo, so remote debugging was the way to go here.

Development

I will not go into details about what exactly I did in carrying out my plan of attack (they are boring details), but I will share that I did not have to fire up Komodo once during my initial development of the Lua debug client. Due to the fact that DBGP clients send and receive messages over sockets, I was able to emulate a Komodo connection with a Lua prompt and test and analyze debug output from a terminal. This really sped up development since I did not have to manually interact with Komodo in order to start debugging, step through functions, etc. It was also very easy to see the instances where I sent badly formed XML. It would have been very difficult to debug this if I was using Komodo to test the Lua debug client.

Only when I was happy with my Lua debug client's test output did I actually fire up Komodo and try to establish a remote debugging session. For the most part it fit like a glove. Komodo immediately started debugging my Lua code. Sure, there were a few kinks to iron out (mostly due to my lack of completely understanding the DBGP protocol, so I had to change some of the information in the messages I was sending to the server), but I was quite pleased that in less than 24 hours of work I had a working Lua debugger that could be used with Komodo.

Conclusion

Komodo's debugging facilities are truly outstanding. Once you learn the DBGP protocol, you can create your own debug client (completely independently of Komodo) and have Komodo start using it right away, out of the box, with no configuration necessary when you are finished. It is true you will not be able to press the "play" button in the Komodo toolbar in order to start using your debug client, but initiating a remote debug session is a fair compromise for this killer feature.

Lua Debugging

Note: There are currently no plans to include this Lua debugger in a future release of Komodo. It was primarily a proof of concept and is very fragile in its current state (e.g. there is no error handling).