Finding Clojure By Playing On The Seaside

The Technitai blog has been far too quiet for far too long, but it’s been because I’ve been burning the midnight oil learning a brand new, for me, way of thinking.

Late last year, I began compiling a list of programming languages and frameworks that I thought that I should at least know a little about. As part of this process, I  sat down with a project idea – a player/event management app for Firefight! That way, I could explore the problem domain and I knew where I could get a subject matter expert if I needed one. Early this past spring, I started digging in.

I started out with Opa, but didn’t get very far. Adam Koprowski, over at Opa HQ was very helpful with answering my early questions. But a few things made it difficult for me to gain traction with the language. The Opa project was transitioning from a Caml based syntax to one based on JavaScript. At the time, the documentation was split between the two which made the going quite rough as I have zero experience with Caml. Compound that with an inexperience in functional programming – well, sit back and imagine the soothing sound of grinding gears.

Opa, I shall be back. Oh, yes. I shall be.

I moved on to Seaside on the Pharo Smalltalk implementation. Six or seven years ago I attempted to dabble with Seaside on Squeak. I wouldn’t call it a crash-and-burn experience but I got nowhere fast. This time around things were totally different. I made some serious headway on the project and I fell in love with the expressiveness of Smalltalk. Years ago, the intrinsic coupling of the language and its environment threw me for a loop. Now I totally appreciate it for its elegance and empowerment. Seaside as a web framework is a stroke of genius. Couple Magritte with Seaside and you’ve got an extremely potent one-two punch for capturing and validating form data on the web. At the conclusion of the prototyping sprint, I felt like I developed an actual application rather than a collection of web oriented components. I thought that I had found my way of developing web apps.

Okay, two more languages were on the short list to check out: Scala and Clojure.

Scala looked the less exotic of the two. The biggest difference being not-LISP. Its syntax didn’t look all that far off from Python and Ruby where I felt quite comfortable. The functional bits could be ignored and used when desired. Kinda like how I was using JavaScript. Besides, it was Lift that put Scala on my RADAR to begin with, so I started heading off in that direction. But something made me pause and reexamine where I was headed.

Each week, I receive The Pragmatic Bookshelf monthly email filled with info on releases, news, and their PragPub magazine. That month they just released Programming Clojure (2nd Edition).

I remembered stumbling onto Clojure when it was new and noting to myself that I should learn-it-someday. I was of course intrigued by what I perceived as its arcane LISPness that verged on the mythical. Back then it was a someday-language. A language to learn someday when I’m suddenly smart enough to learn it.

I bought a copy of Programming Clojure and examined the Clojure web development field but didn’t find a counterpart to Seaside or Lift. Instead, I found stuff called RingHiccup, and Compojure. All of the moving parts had me a bit confused. What was I thinking? Then I found Noir:

Noir is a micro-framework that allows you to rapidly develop websites in Clojure.

And it couldn’t be any simpler.

Sold!

In less than a day, I put my eyes on every character of every word of Programming Clojure. After the once-through, I didn’t intimately understand everything that I read but I got it. On day two I began coding. There were some fits and starts and sudden stops, but soon, I gained some real velocity. After prototyping the Clojure project to be as close as possible to the one done in Smalltalk, I took a step back and thought that while it looked the same in the browser, it was relatively primitive underneath. At first, I thought that I went backwards. But I soon learned that this was a instead a strength.

I did a quick survey of what JavaScript frameworks were out there for building modern web apps. I went with AngularJS. It turned out to be an excellent fit with Noir.

Using AngularJS, I “modernized” the prototype and soon its functionality surpassed the one done in Smalltalk and Seaside. But even more importantly, I understood what Halloway, Bedra, and Hickey were talking about. The lingering doubts that I had about OOP modeling, all the while that I thought that I was good at it, were gone. To top it off, my code base was smaller and just as legible. Don’t get me wrong. My code was quite unpretty here and there and organization in places left much to be desired but those were things that I knew how to fix. And I could fix them without the fixes rippling out across the rest of the system. I freaking learned Clojure!

Okay, back to the mission. I didn’t want the exuberance of learning Clojure to limit my appreciation of Scala so I jumped into it headlong.

There are a lot of good Scala tutorials on the ‘net but I homed in on Programming Scala at O’Reilly. The book itself is a good value too, by the way.

The first thing I did was create a new Scala project to parse the Firefight! ePub in order to pre-populate the database with stuff. I had a heck of a time getting sbt, the Scala Built Tool, to download the project dependencies. Eventually, I gave up and just downloaded them manually. But I did get Scala to parse the XHTML and to insert data into MongoDB.

I’m totally sold on Scala as a scripting language. Having access to all of the libraries that run on the JVM is a huge boon.

As I worked through Programming Scala, I couldn’t manage to gain velocity on the prototype project. A ball of frustration grew inside of me because, as a language, Scala looked familiar. The functional parts were empowering. Traits looked, to me, lightyears ahead of Java Interfaces. After coming off Clojure, there seemed to be so much code but I could have gotten over that if I had a better experience with the build tools, and hadn’t encountered binary incompatibility with compiled libraries, all the while trying to learn both a language and a framework at the same time. It simply felt like too much, and I ran back to Lein.

Scala, Lift, and the Play! framework are all obviously powerful and I’ll be revisiting them before too long, but I’ve decided to build my app prototype into a full blown product using Clojure.

So, yep. I found Clojure by playing on the Seaside.

From Scrivener to EPUB

Outside of the technical arena I’m smack dab in the middle of writing a game book. Up to now it’s been distributed as a PDF but the workflow and distribution has been anything but agile. So, the decision was made to turn it into an HTML5 mobile app that we could iterate over and push the updates to customers. On the way to HTML5 we decided to turn the whole thing into an EPUB. We’d have the chapters already broken up into separate HTML files and we’d have a new product. Talk about limiting waste.

Scrivener is an excellent app that we use for outlining, editing, storyboarding, and writing. It exports to multiple formats; PDF, RTF, Word, EPUB, HTML, etc. Scrivener’s EPUB export works really well for quickly getting your book packaged up. But for us there were a couple of issues. The conversion to HTML is handled by libraries from the OS and it does enough funky stuff to make future reuse of the exported markup difficult. Looking at the resulting markup, one time a header is a header and the next it’s a font whose size is set to 180%. That kind of thing makes it really tough to apply sensible custom CSS. But all was not lost.

Scrivener also compiles books to MultiMarkdown which can be transformed into very clean HTML. To leverage the Scrivener-MultiMarkdown combo I wrote a set of scripts called XHTML2EPUB. The scripts convert the Scrivener generated MultiMarkdown file into a standards compliant EPUB file ready for distribution.

What does MultiMarkdown look like?

Here’s a very basic but straight forward example.

## Playing The Game ##

The Firefight! Tactical Roleplay Engine requires at least one OPFOR Commander and a small group of players; some playing as the Operators and the rest helping the commander by playing as OPFOR.

The text surrounded by double hash marks are denoted to be a second level header.

When rendered as HTML it looks like this:

Playing The Game

The Firefight! Tactical Roleplay Engine requires at least one OPFOR Commander and a small group of players; some playing as the Operators and the rest helping the commander by playing as OPFOR.

Piece of cake and very easy to work with. We converted the entire project to MultiMarkdown by hand instead of relying on the built-in WYSIWYG tools. You tell MultiMarkdown what you mean the text to look like instead of worrying about font selection and what-not. Beyond making it easier to apply custom CSS, I’ve found that this approach makes it easier to focus on the writing itself. WYSIWYG can be distracting.

Here’s more information on creating a MultiMarkdown document that’s straight from the source.

Setup

Before you can run the scripts there are some things that you have to make sure are installed:

  • MultiMarkdown Select the installer for your operating system and have a go.
  • Ruby If you’re running OS X or Linux then odds are you already have it installed. If you’re on Windows then have no fear. It’s a straight forward install.

Installing the necessary Ruby bits

From a terminal/command line run the following lines:

gem install nokogiri
gem install uuidtools

Depending on your setup you may have to use the sudo command. For example:

sudo gem install nokogiri

And so on.

The Workflow

1. Compile your work to MultiMarkdown

In Scrivener select Compile… from the File menu.

Note the document titled Meta-Data. Here is where you can place bits of information useful to Multimarkdown. Make sure this bit is in there:

css: ../Styles/epub_style.css

Under Formatting, at top of the representative text area, add:

# Title #

This tells Scrivener to insert the chapter title as a header element into each chapter’s text. You could do this step by hand but you lose the ability to freely rearrange chapters within Scrivener via drag and drop.

XHTML2EPUB uses this bit of information to do its thing.

2. Configure XHTML2EPUB

3. Run mmd2xhtml.rb

mmd2xhtml.rb is included as a convenience script to convert the file denoted by multimarkdown_file_name into plain ole XHTML. After this script is run there will be a monolithic XHTML file in the master_directory. It is split into individual chapter files in the next step.

4. Run xhtml2epub.rb

This is where most of the work is done. After you run xhtml2epub.rb you will have a complete working directory for your EPUB with all of the assets; text, and images all in the right place. The script also massages the generated HTML to comply with EPUB standards.

5. Tweak the CSS

Since EPUB is styled with a subset of CSS 2 you can use your favorite editor to tweak epub_directory/OEBPS/Styles/epub_style.css.

Once everything looks good simply run xhtml2epub.rb again and it will scoop everything up and recreate the EPUB file with your CSS changes.

Hip Pocket AAR Writer on Android?

I’ve got Eclipse with the Android development plugin installed.  I’ve gone through some tutorials and am now looking at porting the Hip Pocket AAR Writer to Android.

If I give myself the green light my intention is not to make it a straight port but instead to play to the strengths of the Android platform.  But boy do I wish Core Data was available.