Updates on Fenius and life :: Elmord's Magic Valley

Updates on Fenius and life

2024-03-26 15:19 +0000. Tags: comp, prog, pldesign, fenius, lisp, life, in-english

Fenius

Over the last couple of months (but mainly over the last four weeks or so), I’ve been working on the Fenius interpreter, refactoring it and adding features. The latest significant feature was the ability to import Common Lisp packages, and support for keyword arguments in a Common-Lisp-compatible way, i.e., f(x, y=z) ends up invoking (f x :y z), i.e., f with three arguments, x, the keyword :y, and z. Although this can lead to weird results if keyword arguments are passed where positional arguments are expected or vice-versa (a keyword like :y may end up being interpreted as a regular positional value rather than as the key of the next argument), the semantics is exactly the same as in Common Lisp, which means we can call Common Lisp functions from Fenius (and vice-versa) transparently. Coupled with the ability to import Common Lisp packages, this means that we can write some useful pieces of code even though Fenius still doesn’t have much in its standard library. For example, this little script accepts HTTP requests and responds with a message and the parsed data from the request headers (yes, I know that it’s not even close to fully supporting the HTTP standard, but this is just a demonstration of what can be done):

# Import the Common Lisp standard functions, as well as SBCL's socket library.
let lisp = importLispPackage("COMMON-LISP")
let sockets = importLispPackage("SB-BSD-SOCKETS")

# We need a few Common Lisp keywords (think of it as constants)
# to pass to the socket library.
let STREAM = getLispValue("KEYWORD", "STREAM")
let TCP = getLispValue("KEYWORD", "TCP")

# Import an internal function from the Fenius interpreter.
# This should be exposed in the Fenius standard library, but we don't have much
# of a standard library yet.
let makePort = getLispFunction("FENIUS", "MAKE-PORT")

# Add a `split` method to the builtin `Str` class.
# This syntax is provisional (as is most of the language anyway).
# `@key start=0` defines a keyword argument `start` with default value 0.
method (self: Str).split(separator, @key start=0) = {
    if start > self.charCount() {
        []
    } else {
        let position = lisp.search(separator, self, start2=start)
        let end = (if position == [] then self.charCount() else position)
        lisp.cons(
            lisp.subseq(self, start, end),
            self.split(separator, start=end+separator.charCount()),
        )
    }
}

# Listen to TCP port 8000 and wait for requests.
let main() = {
    let socket = sockets.makeInetSocket(STREAM, TCP)
    sockets.socketBind(socket, (0,0,0,0), 8000)
    sockets.socketListen(socket, 10)

    serveRequests(socket)
}

# Process one request and call itself recursively to loop.
let serveRequests(socket) = {
    print("Accepting connections...")

    let client = sockets.socketAccept(socket)
    print("Client: ", client)
    let clientStream = sockets.socketMakeStream(client, input=true, output=true)
    let clientPort = makePort(stream=clientStream, path="<client>")
    let request = parseRequest(clientPort)

    clientPort.print("HTTP/1.0 200 OK")
    clientPort.print("")
    clientPort.print("Hello from Fenius!")
    clientPort.print(request.repr())

    lisp.close(clientStream)
    sockets.socketClose(client)

    serveRequests(socket)
}

# Remove the "\r" from HTTP headers. We don't have "\r" syntax yet, so we call
# Common Lisp's `(code-char 13)` to get us a \r character (ASCII value 13).
let strip(text) = lisp.remove(lisp.codeChar(13), text)

# Define a structure to contain data about an HTTP request.
# `@key` defines the constructor as taking keyword (rather than positional) arguments.
record HttpRequest(@key method, path, headers)

# Read an HTTP request from the client socket and return an HttpRequest value.
let parseRequest(port) = {
    let firstLine = strip(port.readLine()).split(" ")
    let method = firstLine[0]
    let path = firstLine[1]
    let protocolVersion = firstLine[2]

    let headers = parseHeaders(port)

    HttpRequest(method=method, path=path, headers=headers)
}

# Parse the headers of an HTTP request.
let parseHeaders(port) = {
    let line = strip(port.readLine())
    if line == "" {
        []
    } else {
        let items = line.split(": ") # todo: split only once
        let key = items[0]
        let value = items[1]
        lisp.cons((key, value), parseHeaders(port))
    }
}

main()

Having reached this stage, it’s easier for me to just start trying to use the language to write small programs and get an idea of what is missing, what works well and what doesn’t, and so on.

One open question going forward is how much I should lean on Common Lisp compatibility. In one direction, I might go all-in into compatibility and integration into the Common Lisp ecosystem. This would give Fenius easy access to a whole lot of existing libraries, but on the other hand would limit how much we can deviate from Common Lisp semantics, and the language might end up being not much more than a skin over Common Lisp, albeit with a cleaner standard library. That might actually be a useful thing in itself, considering the success of ReasonML (which is basically a skin over OCaml).

In the opposite direction, I might try to not rely on Common Lisp too much, which means having to write more libraries instead of using existing ones, but also opens up the way for a future standalone Fenius implementation.

Life

I quit my job about 6 months ago. My plan was to relax a bit and work on Fenius (among other things), but I’ve only been able to really start working on it regularly over the last month. I’ve been mostly recovering from burnout, and only recently have started to get back my motivation to sit down and code things. I’ve also been reading stuff on Old Chinese (and watching a lot of great videos from Nathan Hill’s channel), and re-reading some Le Guin books, as well as visiting and hosting friends and family.

I would like to go on with this sabbatical of sorts, but unfortunately money is finite, my apartment rental contract ends by the end of July, and the feudal lord wants to raise the rent by over 40%, which means I will have to (1) get a job in the upcoming months, and (2) probably move out of Lisbon. I’m thinking of trying to find some kind of part-time job, or go freelancing, so I have extra time and braincells to work on my personal projects. We will see how this plays out.

EOF

That’s all for now, folks! See you next time with more thoughts on Fenius and other shenanigans.

Elmord's Magic Valley

Computers, languages, and computer languages. Às vezes em Português, sometimes in English.