Under the Hood: Komodo X and Chrome Debugging

It is easy to take for granted Komodo IDE's superb debugging support. Komodo excels at debugging common scripting languages like Perl, Python, Tcl, and Ruby. It can also debug PHP, Node.js, and XSLT. With Komodo X, the debugger supports an all new target: Google Chrome. What is the secret to Komodo's debugging prowess, and how does Komodo communicate with such an unorthodox target as Chrome? Read on to find out!

Under the Hood

Under the hood, Komodo's debugger makes use of a debugging protocol called DBGP, which was co-designed by our developers here at ActiveState. This protocol is based on bi-directional socket communications between a debug server (typically Komodo or another IDE with DBGP support) and a debug client (typically the code being debugged).

A typical debug session begins when Komodo kicks off the debug client in a separate process that initializes the script to debug. Komodo then sits and waits for a connection. The client initiates the connection, sends an initialization packet, and waits for instructions from the server. At this point Komodo can issue DBGP commands to set breakpoints, run/pause/step, inspect variables, and so on.

What exactly is the debug client? More often than not it is just a script (or set of scripts) written in the language to debug that performs the debugging, and then communicates with the debug server. For example, the Python debug client is a Python module composed of a number of Python scripts. This module makes use of Python's built-in debugging facilities in order to actually perform the debugging, and it uses the socket routines provided by Python's standard library in order to communicate with the debug server.

Chrome and Komodo

Google Chrome provides a fantastic set of built-in developer tools for debugging websites. In particular, we at ActiveState are interested in harnessing Chrome's JavaScript debugging capabilities from Komodo. After all, since you are editing the code for your website with Komodo, why should you have to context-switch to Chrome and its developer tools in order to debug your site? The remainder of this blog post is a high-level overview of how we developed our "Remote Chrome Debugging" support for Komodo X.

The Problem

Based on the client-server architecture described in an earlier section of this post, it is only natural to assume that Chrome would be the debug client and that Komodo would communicate with Chrome directly when debugging JavaScript code. The caveat here is that Chrome has its own debugging protocol for socket communications with debug clients. (Not only that, but the "socket communications" are actually over websockets instead of your typical TCP sockets!) Chrome's debug protocol is completely different from the DBGP protocol Komodo understands, so the problem becomes much more challenging.

A Solution

Here is where we stand:

  • Komodo's debugging facilities send and receive messages via the DBGP Protocol.
  • Chrome's debugging facilities send and receive messages via the Chrome Remote Debugging Protocol (henceforth referred to as "CRDP").
  • These protocols are completely different.
  • What to do?

In our minds, the most reasonable thing to do was create an "interpreter", but not in the sense of an interpreter for a programming language. No, this interpreter would act sort of like a foreign language translator. It understands both protocols and can act as the "middle man" in-between Komodo and Chrome. Any DBGP messages sent by Komodo would be interpreted and translated into CRDP messages before being forwarded on to Chrome. Any CRDP messages received by Chrome would be interpreted and translated into DBGP messages before being forwarded to Komodo.

We decided that this interpreter should be a stand-alone program spawned by Komodo. Why not build it into Komodo and run it in-process? Well by nature the web is asynchronous. If the interpreter was built into Komodo's main process, any time Komodo sent a debug command, its interface would "block" (or wait) until a response was received. We know in the web world this is simply bad practice. Okay, then why not create a custom web-page in Chrome and have it do the debugging and communication work behind the scenes? Chrome's tabs are separate processes, no? That is a good point, but a very important aspect of developing this interpreter is the ability to easily unit test it. At the time, there was/is no viable way to do this from Chrome itself.

Implementation

Our interpreter is a Python program composed of two classes: a class that handles websocket CRDP communications with Chrome, and a class that handles standard socket DBGP communications with Komodo. Both classes listen and communicate with their respective clients on separate threads in order to allow for asynchronous actions. (For example, suppose Komodo issued an instruction to step over a JavaScript function that runs a time-consuming loop. If the loop is taking too long and Komodo wanted to break within that loop to see what was taking so long, the interpreter should not be "blocked" waiting for that function to finish and return a result to Komodo's original "over" instruction. By running two threads asynchronously, the interpreter can read the "break" instruction from Komodo and issue it to Chrome, even if a result from the previous "over" instruction has not yet been received from Chrome.)

For our unit testing framework, we designed a "ThreadedTestClient" that spawns an instance of Chrome in a separate process and also starts the interpreter in a separate thread. After initializing the interpreter, the client waits a few seconds for a connection from a unit test. Each unit test contains a suite of test cases designed to test a particular group of debugging functionality. For example, there is a unit test that tests breakpoint setting, updating, removing, and listing. There is another unit test that tests run, step into, step over, and stop actions. Another unit test is responsible for breaking out of an infinite loop. Still other tests interacting with the current JavaScript state while the debugger is paused. You get the idea. After a unit test finishes running its particular suite of tests, it kills the "ThreadedTestClient" and interpreter along with Chrome itself so the next test can begin with a "clean slate" so to speak.

When it comes to executing those unit tests, prior to each unit test body being run, there is a "setUp" function that instantiates the "ThreadedTestClient" (which starts Chrome and the interpreter). When the main unit test body is executed, it makes the connection with the interpreter and then proceeds with its particular suite of tests. Afterwards, a "tearDown" function kills the "ThreadedTestClient" and Chrome. All of this happens automatically for each unit test when we execute our python test.py command. This is far more convenient than manually starting Chrome and working through a scripted set of instructions that re-tests all of our debugging functionality after making a small change.

Conclusion

Komodo X adds JavaScript debugging via Google Chrome to the IDE's impressive debugging resumé. In order to handle Komodo's DBGP debug message protocol and Chrome's CRDP protocol, we created a stand-alone, asynchronous interpreter that acts as a "middle-man" between Komodo and Chrome. This interpreter is backed by an impressive test suite that eliminates the need for time-consuming, manual QA testing for every little change -- this was particularly essential in the rapid development of the interpreter itself.

The absolute coolest part about Komodo's use of the DBGP protocol for its debugging functionality is that once you understand the protocol, you can develop your own debug client completely independently of Komodo and then, when you are ready to integrate it with Komodo, it will slip on like a glove and just work. That is exactly how we developed this Chrome debug client, and how we developed the all new Ruby 2.1+ debug client, which was also introduced in Komodo X. In an upcoming blog post, we will illustrate this process at a deeper level with a completely new programming language. Stay tuned!