ruby
Migrating Drupal comments to Disqus
March 5, 2010 - 22:07 — bhugaAfter learning about Disqus this week, I became quite enamored of it. Drupal's comment system is swell and all, but I know that I don't have time to set it up as to be as nifty as it could be. So I think it's swell if someone wants to take it off my hands for free. The downside is that if I upgrade to D7, I wouldn't get those swell forthcoming RDF-a comments. But there's a Disqus API, so I can solve that later if it's important to me.
The problem is that having both Disqus and comments enabled makes things confusing. Each post has a link at the bottom with '13 comments 13 comments and 0 Reactions' or something similar. Not swell. I don't want to abandon my existing comments--in particular, my Puppet vs Chef and SPARQL posts got good feedback from their respective communities. The Puppet post is #1 on Google for 'puppet vs chef' largely because the authors of both projects commented there. Hopefully, Disqus would let people track those authors to comments on my site in the future.
So the obvious solution is to just import my comments into Disqus! I'll just download some module or converter and...and...nuts. Wordpress and Blogger have the goods, but not Drupal. So there went another afternoon: I wrote a basic importer in Ruby. Ruby is where the good Disqus library was (even if I did have to fix a bug).
Drupal comments have features that do not map to cleanly to Disqus, so it has weaknesses. All the comments are anonymous, and I really miss markdown. But it worked for me. Hopefully it helps someone else, too.
Rapid and Incremental Infrastructure Development with Puppet
March 4, 2010 - 17:33 — bhugaSo pretty much everyone has at least heard of Puppet by now. And yes, it's awesome. But it can be daunting to get started--generally speaking, configuring something with Puppet takes longer than just doing it, and that means that setting something up requires, if not Big Design Up Front, some Big Work Up Front. You need a puppetmasterd, and some servers, and some config, and it gets a complicated quickly.
What I want is a simple testbed, a dry run setup, where I can run my code repeatedly, just like during normal development. I want to develop incrementally and flexibly. Can it be easier? Of course it can, that's why we have clouds. I've been working on some infrastructure for a new project, and the workflow I'm using is easy and effective. I just boot a community EC2 AMI, get git, and pull down my puppet repo. That repo has a handy script that installs ruby, gem installs puppet (every distro is using a dated version--forget them), and then I'm ready to go.
The directory structure, like most Puppet installations, looks like this:
- etc
- puppet
- setup_puppet.sh
- modules
- (the goods)
- manifests
- nodes.pp
- site.pp
From here, it's easy. puppet manifests/site.pp will run the config locally, without a server or any other trouble. In my development, I have a branch for the actual puppetmaster, which replaces nodes.pp from a default node that includes everything into something more meaty. Everything else is the same. From here, I can hack away, testing as I go. Add a --noop to do dry runs. Add -d to enable debug mode and see exactly what commands are run.
The setup script is dirt simple:
#!/usr/bin/env bash
# libopenssl-ruby1.8 isn't necessarily required with this,
# but you do need it for the puppetmaster server.
sudo apt-get install -y rubygems libopenssl-ruby1.8
sudo gem sources --remove http://gems.rubyforge.org/
sudo gem sources --add http://rubygems.org
sudo gem install puppet --no-rdoc --no-ri
# Debian weird path
export PATH=$PATH:/var/lib/gems/1.8/bin
4 hours of very productive infrastructure work cost me about 35 cents. No puppetmasterd, no existing servers, and no temptation to store meaningful config on the local disk (since I shut these instances down after a few hours, as I'm easily distracted). No messing around. I really like this workflow.
As an aside, this workflow is fast enough that two Ubuntu gotchas are now actually a problem for me. Firstly, the official AMIs from Cannonical now require the initial login to be via the 'ubuntu' user, which is a pain, because now root can't effectively git clone anything without more hoops. Secondly, Rubygems is broken on Debian. It flatly refuses to run gem update --system, and when you force it to with the rubygems-update gem, it manages to lose track of all installed gems, including Puppet. Since any Puppet gathers all information before doing anything, it will read that Puppet is installed, and any code that runs the gem update won't understand that Puppet now needs to be reinstalled. I'm not sure this can be worked around in Puppet at all; it might have to be out of band.
This is a well-known issue, it's more than 2 years old, and I can't find why this is the way it is. Google results are too full of people working around the problem to find an actual discussion of the original issue. Rubygems.org has a faq on the issue, but I did find a the Debian issue in which this appears to have been done instead of simply fixing the problem. I'm not sure if this has to do with a wonky directory setup, or if Debian just assumes I'd put an eye out with that much Power. Either way, tons of people have problems with it, and it seems curious to me that Debian has decided they should suffer. And it is a decision: Rubygems has been updated a number of times since 2007, and the disabling code had to be explicitly upgraded at least once that I found.
Anyways, the next bit of agile infrastructure work I do will be on a RightScale Centos 5.4 AMI. What's the point of being agile if you don't try new things?
Updated: Centos 5.4 is running Ruby 1.8.5. I guess being agile is about trying hopelessly outdated things, too.
Hacking on RDF in Ruby
February 28, 2010 - 08:20 — bhugaRDF.rb is easily the most fun RDF library I've used. It uses Ruby's dynamic system of mixins to create a library that's very easy to use.
If you're new at Ruby, you might know about mixins in other languages--Scala traits, for example, are almost exactly functionally equivalent. They're distinctly more powerful than Java interfaces or abstract classes. A mixin is basically an interface and an abstract class rolled into one. Rather than extend an abstract class, one includes a mixin into your own class. A mixin will usually require that a given class implement a particular method. Ruby's own Enumerable class, for example, requires that implementing classes implement #each. For that tiny bit of trouble, you get a ton of methods (listed here), including iterators, mapping, partitions, conversion to arrays, and more. (If you're new to Ruby, it might also help you to know that #method_name means 'an instance method named method_name').
RDF.rb uses the principle extensively. RDF::Repository is, in fact, little more than an in-memory reference implementation for 4 traits: RDF::Enumerable, RDF::Mutable, RDF::Queryable, and RDF::Durable. RDF::Sesame::Repository has the exact same interface as the in-memory representation, but is based entirely on a Sesame server. In order to work as a repository, RDF::Sesame::Repository only had to extend the reference implementation and implement #each, #insert_statement, and #delete_statement. Nice! Of course, implementing those took some doing, but it's still exceedingly easy.
RDF::Enumerable is the key here. For implementing an #each that yields RDF::Statement objects, one gains a ton of functionality: #each_subject, #each_predicate, #each_object, #each_context, #has_subject?, #has_triple?, and more. It's a key abstraction that provides huge amounts of functionality.
But the module system goes the other way--not only is it easy to implement new RDF models, existing ones are easily extended. I recently wrote RDF::Isomorphic, which extends RDF::Enumerable with #bijection_to and #isomorphic_with? methods. The module-based system provided by RDF.rb means that my isomorphic methods are now available on RDF::Sesame::Repositories, and indeed anything which includes RDF::Enumerable. This is everything from Repositories to Graphs to query results! In fact, query results themselves implement RDF::Enumerable, and thus implement RDF::Queryable and can be checked for isomorphism, or whatever else you want to add. This is functionality that Sesame does not have natively, and which I wrote for a completely different purpose (testing parsers). Every RDF::Enumerable gets it for free because I wanted to compare 2 textual formats. Neat!
For example, here's what it takes to extend any RDF collection, from RDF::Isomorphic:
require 'rdf'
module RDF
##
# Isomorphism for RDF::Enumerables
module Isomorphic
def isomorphic_with(other)
# code that uses #each, or any other method from RDF::Enumerable goes here
...
end
def bijection_to(other)
# code that uses #each, or any other method from RDF::Enumerable goes here
...
end
end
# re-open RDF::Enumerable and add the isomorphic methods
module Enumerable
include RDF::Isomorphic
end
end
Of course, this just can't be done without monkey patching. Mixins and monkey patching together make for a powerful toolkit. To my knowledge, this is the first RDF library that takes advantage of these features.
It's possible to provide powerful features to a wide range of implementations with this. RDF.rb does not yet have a inference layer, but any such layer would instantly work for any store which implements RDF::Enumerable. Want to prototype some custom business logic that operates over existing RDF data? Copy it into a local repository and hack away. No need for the production RDF store to be the same at all, but you can still apply the same code.
As a counter-example, compare this to the Java RDF ecosystem. There are some excellent implementations (RDF::Isomorphic is heavily in debt to Jena), but they're all incompatible. Jena's check for isomorphism is not really translatable to Sesame, or anything else. RDF.rb, in addition to providing a reference implementation, acts as an abstraction layer for underlying RDF implementations. The difference is night and day--with RDF.rb, you only need to implement a feature once, at the API layer, to have it apply to any implementation. This is not a knock at the very talented people behind those Java implementations; making this happen is a lot of work in a language without monkey patching, and RDF.rb is only as good as it is because of the significant influences those projects have been on Arto's design.
The end result of the mixin-based approach is a system that is incredibly easy to extend, and just downright fun. It would be a fairly simple task to extend a Ruby class completely unrelated to RDF with an #each method that yields statements, allowing it to work in RDF::Enumerable. Voila, your existing classes now have an RDF representation. Along the same lines, if one is bothered by the statement-oriented nature of RDF.rb, building a system which took a resource-oriented view would not require one to 'break away' from the RDF.rb ecosystem. Just build your resource-oriented model objects and implement #each, and away you go--you can now run RDF queries and test isomorphism on your model. Build it to accept an RDF::Enumerable in the constructor and you can use any existing repository or query to initialize your model.
RDF.rb is not yet ready for production use, but it's under heavy development and already quite useful. Give it a shot. You can post any issues in the GitHub issue queue.
The Promising Future of the Unlicense
January 13, 2010 - 18:35 — bhugaSo the public launch of the unlicense movement on January 1st has gone better than expected. Arto's post hit top of the list for 'most controversial' on Reddit for a while, and unlicense.org itself is seeing decent traffic. Since the target audience of the site is developers, even a few thousand eyeballs is a good number, and that number is being handily beaten.
There are three main concerns that seem to keep appearing in discussions in regards to releasing software into the public domain. I'd like to briefly offer a response to them, and then provide an example of why public domain might be what you're looking for for your software.
Concern 1: Only the GPL preserves Freedom!
This is amazing to me. There is an incredible amount of worry, it seems, that faceless corporate Cthulhu-associated entities are lurking in the shadows, just waiting to pounce on vulnerable code unprotected by the GPL's armor. These faceless horrors have crushed promising startup projects with vulnerable licenses like Apache (the world's most-installed web server), SQLite (the world's most-installed SQL server) (and public domain, not actually licensed at all, by the way), BIND (the world's most-installed DNS server), and other sad stories. I find this concern so unrealistic it boggles the mind.
But just for fun, let's start a real flame war. Numerous folks out there claim, with a certain sort of correctness, that the GPL keeps software free from the lockdown of derivative works. This is called Freedom (capital F). However, that software's Freedom is enforced by a copyright, which restricts the actions of people by proscribing certain kinds of copy and use. These proscriptions are enforced via a system with far reaching effects, which prevent me, for example, from purchasing a DVD player that disregards region codes.
The inevitable conclusion is that the GPL is about valuing the 'Freedom' of bits over the freedom of humans.
You, dear reader, are far more important than my code, regardless of your choice of license. I have no time for a moral system that makes such claims on your autonomy. I will avoid that system as much as possible: by using the unlicense.
Concern 2: Statutory Law vs Common Law
Statutory law, such as that of many European states, often fails to specify a process for explicit public domain donations, leading many to wonder if such a thing is even possible. Folks better educated than I seem to have differing opinions. But I'll note that if you're concerned about it, there is precedent. The original w3 server, which was owned by CERN (guess what the 'E' stands for?), was placed into the public domain in 1993.
Apache and Netscape both trace their heritage back to European public-domain software. You'd be foolish to accept this post as legal advice, of course, but in my mind, those are perfectly acceptable European counter-examples.
Concern 3: Moral rights cannot be relinquished
A true concern, many jurisdictions specifically prohibit the relinquishment of 'moral rights' of authorship, namely, the right to be the named author of a work, and the right to not have works you did not author attributed to you. To my mind, this a not a problem of copyright, it's basically statutory encoding of the fact of authorship. The issue is muddied by several states conflating copyright enforcement with moral rights enforcement. While again not legal advice, I'd say that simple attribution covers you.
Concerns 2 and 3 really bother me, because legal arguments against them are complicated and require specialized knowledge. I'm not qualified to argue them in a bulletproof manner. But most software will never become big enough to have a license (or unlicense) issue, and you can issue someone an explicit license if it's ever a problem (as SQLite does).
How public domain lets your software grow
"CERN's decision to make the Web foundations and protocols available on a royalty free basis, and without additional impediments, was crucial to the Web's existence. Without this commitment, the enormous individual and corporate investment in Web technology simply would never have happened, and we wouldn't have the Web today."
Tim Berners-Lee, Director, WWW Consortium
The public domain is the best way that others can take your ideas and run with them. CERN's public-domain dedication is probably the best example of that, but if you want your software to change the world, you need to allow others to use it as freely as possible. I'll give you a little personal example.
A few days ago, I published promising future, a Ruby gem that adds Scheme-style promises and futures to Ruby. I did this because I happen to love promises and futures, and it drives me absolutely nuts whenever they are not available.
If my goal is (and it is) to always have promises and futures available, the ideal would be that it were in the Ruby core library, or even a language future. I could start a rallying cry on a mailing list somewhere, and may yet, but my odds are slim. But what if a much better known author, with a much more popular library, wants to use these lovely little things in his code? Well, they could add a gem dependency, but that's not a popular option for various reasons. If the licenses work out, they could incorporate the code, usually requiring a note of attribution.
But my promises and futures code has the maximum possible flexibility. Anyone, with or without any license, can copy/paste my promises and futures into their code, without attribution, and be done with it. Problem solved. My code lives on, or at least inspires the creation of equivalent functionality implemented in a better way. And maybe, one day in a promising future, every Rubyist everywhere will have promises and futures available. That SQLite can be embedded in other software is a huge factor in its unparalleled adoption.
How could my software be more free? How could I be more free? How could you be more free? What could be better? Public domain promises a very promising future indeed.
Quantity.rb: first-class quantities for Ruby
January 5, 2010 - 11:07 — bhugaI've just put out a first release of Quantity.rb, which scratches an itch I had and much more.
Quantity.rb provides first-class Quantity objects, like '12 meters', '1 liter', or '1 dozen'. More significantly, it supports things like '12 meters * 1 kilogram / 2 seconds**2'. It was an outgrowth of an attempt to do some automated unit conversions of a project I am working on involving some monitoring, and I wasn't happy with what was out there. In particular, I wanted to eventually provide the ability to divide one time series of data points by another, regardless of units. It needed to be something more than 'meters to feet'. Maybe it didn't need to be this involved, but it's the right way to do it: anything can be built on top of this.
It's not the first attempt, and perhaps not even the first success. Quanty is the earliest one I can find, and it does most of what I want. Unfortunately, it uses yacc, which I have no intention of learning, and the English docs are sparse. There's something called the Quantity Management Framework, but I can't find much info about it.
Besides, I figured it would be fun. I would learn something, and sometimes it's good to have a project with a well-defined scope so that you can Finish It. Especially when you have a handful of muddy projects mixed with a handful of very long term ones. So it was the charge of the light brigade. And I did learn something. Earlier versions used some class inheritance features that made me learn far more about Ruby's metaobject system than I had ever hoped to. That was kind of like this for me.
Anyways, there's more to do, but I'm pleased with the results so far. Some of the things you can do, from the README:
require 'quantity/all'
1.meter #=> 1 meter
1.meter.to_feet #=> 3.28083... foot
c = 299792458.meters / 1.second #=> 299792458 meter/second
newton = 1.meter * 1.kilogram / 1.second**2 #=> 1 meter*kilogram/second^2
newton.to_feet #=> 3.28083989501312 foot*kilogram/second^2
newton.convert(:feet) #=> 3.28083989501312 foot*kilogram/second^2
jerk_newton = newton / 1.second #=> 1 meter*kilogram/second^3
jerk_newton * 1.second == newton #=> true
mmcubed = 1.mm.cubed #=> 1 millimeter^3
mmcubed * 1000 == 1.milliliter #=> true
[1.meter, 1.foot, 1.inch].sort #=> [1 inch, 1 foot, 1 meter]
m_to_f = Quantity::Unit.for(:meter).convert_proc(:feet)
m_to_f.call(1) #=> 3.28083... (or a Rational)
It's made my IRB shell quite the handy calculator. Try it out for that, if you're CLI-inclined.
This whole affair was also an excuse to release something meaningful via the unlicense (I also did a growl-amqp thingee but it hardly counts). The unlicense is a framework for releasing code not with a license, but as public domain. Public domain is something that old timers remember: what used to older copyrighted works. Originally some pithy few years, copyright these days now lasts for an author's lifetime + 70 years, and it's been several years since anything entered the public domain in the US due to numerous extensions. Some countries have gone so far down the rabbit hole that one cannot dedicate things to the public domain.
This is all the more ridiculous when one considers that most people now believe copyright is bunk. Eventually, legal frameworks will respect how the world is, and not how it was. A lot of people won't release software under the public domain because of the spotty legal status. A few years ago, people were equally afraid of the GPL until some court cases affirmed the common-sense interpretation of the license. Let's release some public domain software and push the issue of what happens when you don't have a license at all. I was hoping to release this on the first of January for public domain day, but it needed more work. I guess it's not much of a holiday since nothing enters the public domain anymore anyway.
Anyways, 'gem install quantity' and have fun.
