As there are some notable features (namely networking) that are only available to the C++ API, I've decided to start developing that way instead. I'll post my findings thus so far on the debugging protocol, and I may return to the project on a future date.
Firstly, there was absolutely no need to develop such a program using Love2D, and I was only doing so because I had some libraries built up in it. It is far more practical to simply use a Lua program, parsed by some simple C++.
The protocol is completely accessible to standard UDP libraries (such as LuaSocket, which I used), and is bound to "127.0.0.1:4630".
Upon startup, a Polycode program begins to broadcast pings every second. These pings differ only by a single byte, which counts what number ping it is (not the number of seconds elapsed).
When a print statement is executed, or an error occurs, it is added to a list of packets sent every cycle (one second). These packets are all sent simultaneously, but in order.
print("This is the first print statement.")
print("This is the second print statement.")
After the first three or so pings (the time it takes to initialize the program, at least on my computer), this sequence will be sent every second:
Print Statement 1
Print Statement 2
When an uncaught error occurs, a similar process transpires. Packets will continue being sent, even though program execution has halted, until the application is closed.
I have not yet analysed the print statement packets, but my analysis of the error packets follows:
Errors queue two packets: The error and the backtrace.
The error packet contains at least three relevant data points: The line the error occurred on, the message the error returned, and the file the error occurred in. It may also contain the length of the backtrace, but this is unconfirmed.
The line and message are a fixed length from the start of the datagram:
The line number is the twenty-first (21st) byte in the datagram.
The error message starts on the twenty-fifth (25th) byte, and is null-terminated (is immediately followed by a zero byte, which can be used to detect the length of the message).
The file path is a bit trickier. The position of this data within the datagram is variable, so it must be detected by string operations. I used the following line to extract it:
local file = data:match("%g*%.lua")
Where 'data' is the datagram string.
I have not yet analysed any of the rest of the protocol, but I have some additional insights.
As the datagram contains unprintable characters, it cannot be printed. It should instead be passed to a function that translates all unprintable characters to a readable, unique format. This is the function I used:
local function fixstring(str)
local ret = ""
for i=1,str:len() do
local c = str:sub(i,i)
if not c:find("[%w%p]") then
ret = ret.."_"..c:byte().."_"
ret = ret..c
Some errors (such as a 'require' error) will not pass the same text to the debugging protocol as they would to a console output. As a result these errors are entirely uninformative, so I advise finding some way to capture the true error using 'pcall' and/or 'xpcall' (There is also a character limit on the print messages sent to the protocol, so you may need to separate the error into smaller strings.).