I’ve been packaging dependencies for quite some time now and I regret not doing this earlier. When I started writing clojurehelper I guess I had a different idea of what I really needed in order to easily package libraries. As of now I have some 13 packages on mentors (some of them have already been accepted into NEW) plus 4 or 5 more in my hard drive almost ready to be uploaded. Apart from learning that you should never estimate the amount of libraries you need in order to satisfy dependencies (because one leads to another which also leads to another which happens to depend on some JSR that has not been included into java yet and then some other library compiles but is not compatible…), I also learned that some of the features I was including in clojurehelper were not very useful and that I was also ignoring some basic functionality that I seemed to be constantly needing. I used clojurehelper in “javahelper” mode to package most of the clojure libraries, and maven-debian-helper for the java ones. After talking to my mentor and we decided to go back to clojurehelper and finish integrating lein2 into the build system.
Sane(r) naming scheme and modularization
Simple schematic of clojurehelper
The first thing that came to my mind when I started going through the code was “WHAT IS THIS?”. It was a bad sign that it took me some time to figure out again how everything worked. I guess it made sense back then when I knew what “tools”, “debtemplates”, “project” really were. So I spent some more time reading the code and came to the conclusion that some parts had to be refactored and renamed.
I explained in earlier posts that clojurehelper is only about parsing project.clj and creating debian/* files based on the information that was retrieved. The only special logic that is important but “hidden” is the fact that the package variables (such as dependencies, package name, version..) can also be retrieved from different sources and they all must be merged somewhere.
So what used to be tools is now called parsers (since it only contains the pom parser and the project parser). I realized there was no real advantage of having the template logic encapsulated in an object so all those methods are now functions and they are part of core, debtemplates (which ironically didn’t have the template logic) is also part of core.
Flexible specification of dependencies
Leiningen uses a project.clj file in order to know what it needs to fetch and configure before a build. When I started using maven-debian-helper I truly saw how related Leiningen was to Maven and made me think we could use the same approaches to solve the same problems. For maven-debian-helper to work it ‘tweaks’ the original pom.xml file so the maintainer can build the package without packaging every single version of the libraries that are already in the archives.
An idea borrowed from maven-debian-helper is to create a rules file where the maintainer can indicate what libraries and which versions to use. Although it isn’t truly necessary to have the dependencies installed in the system, Leiningen 2 likes to complain if it cannot find them plus javahelper needs to have the libraries installed to figure out the name of the libraries by looking at the jars.
This is what a rules file looks like for now:
But it doesn’t really work like a rules file, it’s more of a list of dependencies that will be included in the project. For that reason you can delete a line if you don’t need it or just change a version manually.
Integration with Leiningen 2
The initial version of clojurehelper didn’t even rely on Leiningen. Our super sophisticated build system was a jar command which essentially just packaged the source files and included a manifest file as well. Leiningen 2 is much more robust than Leiningen 1 (thank you technomancy) and for that reason when can almost forget about how clojure libraries are built and just ask Leiningen to do the hard work.
Ideally we would like to change project.clj during the build, just like maven-debian-helper does with pom.xml, but the problem is that project.clj is much more harder to parse and edit (unless you have a Leiningen plugin which I don’t discard to be using in the future). Fortunately Leiningen 2 let’s you create profiles and specify dependencies there, if you take this into account and also use the update-in task you can pretty much do anything you want without editing project.clj.
I wrote a new script called lein_create_profile which reads debian/leiningen.rules and creates a profiles.clj file with the necessaries dependencies. Leiningen 2 can then be called this way in order to build the jar file (this is in fact the content of lein_build):
lein -o update-in :dependencies empty — with-profile debian-build jar
lein_configure (which comes after dh_auto_configure) makes sure that the profile is available before the build my calling lein_create_profile on the debian/leiningen.rules file created by lein_makepkg.
I like this profiles approach but I don’t know if we’ll find problems with it in the future, this is why the build system might change later on but still rely on Leiningen 2 to do the work of course.
More interaction if we need it
maven-debian-helper is very interactive. I like the fact that if it doesn’t know what to do it will ask you to avoid surprises. Clojurehelper does this now when it comes to resolving dependencies.
During the execution of lein_makepkg, clojurehelper will try to look for unmet dependencies and ask the user what to do about it. For now it only prompts for a version if the version is not installed in /usr/share/maven-repo but I will implement automatic Y.X -> Y.{1.2.3}.. type of matching shortly
Still missing
Since the major refactor and renaming all the tests break because of change in the interface. I will be fixing this soon as well as improving what already has been done on clojurehelper. Like I said previously smart version guessing is not being done at all and lein_makepkg should ask the user if he wants to export a “debian” version or not. Interoperability with the maven-repo-helper has not been tested yet and this is a MUST but at least Leiningen is creating jars and the pom files properly and according to the rules that we set.
eugecm 