Jungle Coder
eggbug ate all of the spiders, excuse the flies
Just added post themes! Witness The Colors, and Exult!

Nim: Scripting ease in a compiled language

Edit from November 2023

The main Nim developer has gone rather mask-off fascist. Given that, I can no longer recommend Nim as a language. Below is the original article, for posterity


Original Article

Nim went 1.0 around the beginning of October, 2019. I’ve taken the chance to dig into it for the past three weeks, and while I can’t call myself an Nim expert as of yet, I think I can offer a decently informed opinion.

I suppose I should add a bit of context: I’ve used Go as a side-project language since 2014, mostly for Gills, PISC, and the blog you’re reading right now. It’s definitely served me well, but I’ve been growing rather tired of some of it’s… choices. It’s not been terrible, but I’ve grown to miss generics and exceptions from C#, and Go just tends to be verbose in general.

That being said, there are some things I like about Go: It compiles on both Windows and Linux, it makes executable files that are easy to deploy (at least for my amateur sysadmin sensibilities), and it gets web servers spun up relatively quickly, all things considered.

For a while, I got around Go’s verbosity by using PISC and then Lua as scripting languages for the projects I had going in Go. I see myself using Lua as a scripting layer for Gills for quite some time to come, just due to how nice a search/tag driven CMS is.

But, when Nim announced their 1.0 status, I decided to do a few small projects to see if it was the language I wanted to make my side-project language of choice for the next 5-10 years, the way that Go has been my side-project language of choice (outside of games, where I’ve mostly used Lua) for the past 4 years.

Project Zero: Porting Jumplist

So, a project that I’ve landed on as a decent way to exercise a programming language without requiring the language to have a full web stack (which is a very heavy requirement) is to implement a directory jumplist manager. I first wrote one in bash that was an abomination of sed, awk, and bash that had slightly self-modifying code. The concept was to avoid calling into windows processes in git-bash, since that seemed to bring a non-trivial amount of lag.

When I wanted to bring the same jumping experience to cmd.exe as a result of using that a lot more at my new job, I opted to write a jumplist manager in Go and some wrapper batch scripts.

Porting this program to Nim was mostly a pleasant experience. I was able to remove about half my code, because Nim had a pre-existing insertion-ordered table, so I didn’t have to write my own insertion-ordered key-value store like I did in Go. The only grumble I had from this project that I had some difficulties figuring out some of the type errors I got at first. But, seeing half of the code I wrote disappear, and the code that I wrote being much smaller was really nice.

Project One: Porting kb_wiki

After I ported the Jumplist script, I had a pretty solid idea of my next target: The various instances of the kilobyte_wiki software. This was software I’d written in Erlang a while back. It was still running in tmux panes because I’d not figured out how to do proper erlang releases while using erlang.mk as my build system. (Side note: I recommend trying out rebar3 over erlang.mk, as with erlang.mk, you have debug a complicated makefile on top of the erlang tools. From what I’ve heard, rebar3 is more user-friendly than erlang.mk, which was difficult for this erlang and make newb to use). I also wanted to stand up an instance for tracking various articles and the best comments I’d found on Lobsters and Hacker News in a given month. In the process of porting kb_wiki from Erlang to Nim, I ended up finding a few things that kinda confused me at first glance, but that make sense in retrospect.

First, the standard library database modules only return database results as seq[string], roughly a List<string> if you’re from C#, or an ArrayList<string> if you’re from Java. This is a deliberate decision to optimize for the likely case where most of the data is going back into other strings in other parts of the application most of the time, and to try to help keep the database code from having to expose different sets of types for different databases, (as I gathered from talking Araq, the creator of Nim on IRC). While it’s not the choice I’d go with for a database application where I was heavily concerned with I/O performance, it’s certainly not a terrible choice for my more standard Sqlite+Web Server type stuff I’ve done a lot of lately.

Project One and a half: Understanding Jester

Also, though it currently (in October of 2019) not look the most kept up to date, Jester is definitely very nice to use as a web framework. Here are a few things that I ran across when porting wiki from Erlang to Nim:

Cool things learned from two and a half projects

So, I had already learned from my first project in Nim that it was a lot more concise than Go, but it was in this second project where that truly became evident. See, Nim is the language I’ve had the easiest experience creating executables in. If you look at the code for kb_wiki, you’ll notice the main app, kbwiki.nim, and that it requires database.nim, kb_config.nim and views.nim. Unrelated to the main kbwiki app are mdtest.nim, todo.nim, and scraper.nim. They are all separate applications that can be compiled into executables using nim c mdtest, nim c todo or nim c scraper, and they can use other nim files.

But, because files are that easy to re-use, and executables are the easy to make, it makes code re-use much easier, at least in the small, than I’ve ever had it be until now. In Go, creating new executables involves a lot more ceremony, in terms of making sure that a new folder structure is created and such. In C#, unless you use LinqPad, you have either MSBuild/VS projects, or you have to create a whole folder and use then dotnet tool to create projects and explain to the C# compiler how to turn a set of source files into an executable. But, in Nim, you just write your file, install the right things via nimble, and then you can compile it into an executable.

Project Three: Scripts and utilities at work

This nimbleness of executable creation makes Nim the first complied language I’d use for general purpose command-line tools, where the value of writing 10 lines of code isn’t dominated by the process of setting up a folder structure and project files to get everything set up.

So, I’ve made some small utilities for my day job. An program for spitting out “banners”, aka the “-“ with a user-specified color in the background. A little command-line punch-clock for helping me keep track of what time I’m working (when working flex hours, it’s helpful to be able to punch in and out for personal tracking). A tool for copying these little tools from the directory they get developed in to the designated “My command-line tools live here to be on the $PATH” directory.

Project the Fourth: Scraping kb_wiki

Most recently, like I mentioned above, I wrote a scraper that can be pointed at instances of kbwiki and download their content. The motivation for the project was pretty simple: I had two instances of the Erlang version of kbwiki floating around, and I wanted to get the content hosted on them in Sqlite files.

The Erlang versions of the wiki use dets tables, because that’s what was easiest to get access to when I was writing the wiki in the first place. So, at first, I looked into trying to connect my Erlang code to SQLite. But, after digging into that for about 90 minutes, I was running into issues around gcc compiler versions on my linux box. Not wanting to spend until the wee hours of the morning trying to fight both Erlang, GCC, and Sqlite, I decided to switch from trying to dump the content into Sqlite files from Erlang, to writing a Nim scraper that basically gloms over the HTML, logs in as an admin, and then downloads all of the markdown for the articles. That took me just under two and a half hours (give or take) to finish, and then I was able to scrap down the wikis in a matter of seconds.

After that, I moved https://idea.junglecoder.com and https://drift.junglecoder.com from running on Erlang to running on Nim. And I didn’t have to fuss with certificates or the like, due to them running behind nginx. I also put up https://feed.junglecoder.com, where I’ve been keeping a feed of what seem to me to be notable comments, articles, and such.

Conclusions

So, having done 4.314 things in Nim in the space of just under a month, I’ve got to say that I quite like Nim. I don’t think it’s quite ready to be a programmer’s first language, a la Python or Javascript, mostly due to a lack of beginner friendly documenation. I also think it still has a lot of room to grow. But I’d like to try to help it grow. Jester is super handy for making websites with. I’ve discovered a whole new level of code-reusability, and the language is just so darn handy to build small/medium things with. I’d like to try to build something bigger with it, probably something off my idea backlog.

Comments

boris
"some of it’s… choices" -> "some of its... choices
Previously: What 8 years of side projects has taught me
Next: Tabula Scripta: A spreadsheet for small data-driven apps