whaack

My previous article detaling TheFleet's pseudocode was poorly worded and contained extraneous implementation details. This revision aims to give a concise summary of how TheFleet works.

In order to log the irc space TheFleet uses fleetbot, a class similar to logbot. Each instance of fleetbot connects to 1 network and joins the max channels allowed per nick on that network.¹ Once connected to its network, a fleetbot logs messages in a local postgres db.² Certain messages - join, part, kick, and privmsg³ - are logged to a table named irclog. Events that are specific to a fleetbot (disconnection from the server, being kicked from a channel, joining a channel, and failing to reconnect) are logged to a separate table named fleetlog.⁴

Fleetbots have an aggressive reconnection strategy. The code that receives and sends messages to the irc network is surrounded in a handler-case.⁵ If a runtime error is signaled a fleetbot attempts to reconnect. A fleetbot also attempts to reconnect if it hasn't received a pong from the server in *max-lag* (60) seconds.⁶

I still need a way to orchestrate the large number of fleetbots connected to various networks. Previously I ran all the fleetbots for 1 network together in their own sbcl process. This design needs to be discarded since each process takes over 40MB of RAM and my VMs only have 1GB available. Other constraints I have to deal with are: A VM costs $5 / month, most networks allow only 3 connections per ip, and I can only run 1,000 concurrent threads on each VM.⁷

1. If a network has 130 channels and a max 50chans/nick, TheFleet creates 3 fleetbots - 2 join 50 channels and 1 joins 30 channels. [↩]
2. This postgres db is shared amongst all fleetbots running on the same VM. [↩]
3. Privmsg is the name for a standard irc message you see in a direct message or in a channel. A privmsg is sent to a target. A target is either a nick or a channel. [↩]
4. I also currently print some debugging statements and unhandled messages from networks to standard-output, which I redirect to a log file when I run the program. I plan to remove this extra logging to save disk space once TheFleet is running in full gear. [↩]
5. I.e. a try/catch block. The robustness of this design is dubious since it allows for unknown errors to persist. [↩]
6. This behavior is inherited from fleetbot's superclass, ircbot. [↩]
7. The thread per VM limit is possibly adjustable. My understanding is that the limit comes from each thread allocating ~ 1MB for its own stack. [↩]

This entry was posted on Friday, February 7th, 2020 at 5:39 a.m. and is filed under tmsr. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.