Daniel Morrison

I’m really excited that we launched the site for Sessions by Collective Idea this afternoon.

Sessions is a new take on training. Not only are we putting together amazing training courses (starting with Ruby on Rails) but we’re making it fun.

We’re starting by bringing people to Holland, Michigan. Most training takes place in an out-of the way hotel or conference center with no character. Holland in the summer simply feels like a vacation, and using the brand new CityFlats Hotel as our venue adds to the unique flavor of Sessions.

Then, we’re not ending the day and sending you back to your room. We’re giving you opportunities to see the area and meet other Rubyists, as well as helping you explore on your own.

The content of the Rails Session has already been praised by students around the world. We’re not focusing just on the fun—we’re simply adding it on.

If you want to learn Ruby on Rails, this Session is for you. Sign up today and you can save 20%.

In response to my previous article, Ryan McGeary made some good comments that I think warrant a followup.

but isn’t there a small advantage to a piston or braid-like approach where all the files are in your own repository?

Definitely in some cases, but I value keeping my repository small over having everything inside. Especially during development, when I want to keep up-to-date with my submodules (especially Rails).

Once a project launches (and/or slows down), storing external dependencies in the repository is more desirable, and my opinion may change. Same goes for gems. I don’t like storing them in the project unless or until I need to.

Also, adding Piston/Braid is one more dependency.

What happens during deployments? Doesn’t the deployment now rely on a possible many remote repositories to succeed? Consider both vendor/rails and a possible many vendor/plugins each with their own submodule.

This is an important thought, and about the only thing that makes me hesitant about submodules, but I still like ‘em.

If you go with a typical Capistrano deployment, you’ll have to download each submodule each time you deploy (same as with svn:externals). This is one of the biggest problems I have with svn:externals, because if a single plugin wasn’t available (I’m looking at you, acts_as_taggable_on_steroids), it kills the checkout.

The same is true with git submodules but there is a very important difference: When you hit the network.

When I do a git pull, it doesn’t look at the submodules at all. Even if I changed them. Even when I do a git submodule update, it will only try to pull new files if I explicitly updated the submodule (see previous post).

Contrast that with svn:externals, which will have to hit the remote repository just to see if anything’s changed.

If you switch your deployment strategy to set :deploy_via, :remote_cache, your average deploy won’t look for submodule updates at all. You can safely deploy if a submodule isn’t available, as it will be already on the server, when you need to update a submodule, you can have a cap task to do that.

Obviously there are other ideas. Maybe I’ll post on that later too.

I think I also missed an important point. How does Matt’s git submodule update avoid pulling a later version of edge rails than the original revision that you intended? The .gitmodules file only seems to specify the public clone url without any commit revision.

The .gitmodules file only stores the paths, but the commit version does get stored with the main project. I created a project on GitHub to demonstrate what happens. Specifically look at this commit, which adds the submodule, to see what’s going on:

+Subproject commit 60be4b09f51d2560802ebd744893bb6f737ef57c

Clone the repository and see how it works locally too. Even though it includes all of Rails, the .git directory (and my github usage) is tiny.

Now that everyone’s on Git, people are starting to play with submodules.

Here’s why you should too.

Just like svn:externals

• They aren’t stored in your repository.
• Only a reference to the location is stored (can be local or remote)

Just like Piston

• A reference to the commit is stored. It will be tied to that revision until you say otherwise.

This sounds kinda confusing, and at least one aspect is probably not what you expect.

Example

For MicroRevie.ws, I wanted to add Rails as a submodule, as I like living on the edge. I used Koz’s version on GitHub (since the official Rails git repo isn’t yet available), so in my git project I typed:

git submodule add git://github.com/NZKoz/koz-rails.git vendor/rails

This tells git where the code is from (remember, this could be local too) and where I want it to live.

If I then run git status I’ll see a new file called .gitmodules. The file looks like:

[submodule "vendor/rails"]
  path = vendor/rails
  url = git://github.com/NZKoz/koz-rails.git

Fantastic! Once I git add and git commit the files, they’re locked into my repository. They are not stored in my repository, however, just references. When I push the main MicroRevie.ws project up to GitHub, it small and fast (better than Piston!).

When Matt pulls down his copy of the project, he needs to run these two commands to use the submodules:

git submodule init
git submodule update

Updating a submodule

Here’s where submodules are more like Piston. You must explicitly update submodules. Here are the steps:

cd vendor/rails
git pull
cd -
git commit -a

Or in English: go into the submodule, pull updates, then get out and commit.

When you commit, you are again only committing a change in reference. You’re not storing tons of new updates, just a reference to the new revision. You can even go backwards, but that’s left as an exercise for the reader.

When Matt wants to get my changes, he can’t just do a git pull. He has to also type:

git submodule update

And he’ll be updated to the version I committed.

Is it worth it?

Yes. However, I’ve defended (and simultaneously cursed) svn:externals, so I’m not the final judge.

In the end though, I really like git submodules better than either externals or Piston. For me it comes down to:

• Don’t need any extra software (such as Piston, and yes I’m aware of Braid)
• Doesn’t clog up my repository with tons of extra files (Piston, again.)
• Doesn’t force me to the newest version (like externals do Update: Mark points out in the comments that svn does allow you to specify a revision. Very good catch!, such as when Rails or rspec make incompatible changes… which has happened)

Try them out. At worst, you’ll remove the references and be out a few minutes of your time.

Once again, its coming up on CSS Naked Day! Sadly, they changed the date by a few days this year, so if you’re using my css_naked_day Rails plugin, your site doesn’t have CSS right now.

I updated the git and svn versions, so you can update. The actual change is tiny, so do it yourself.

In light of this, I am explicitly setting the year. We’ll all have to update next year too.

A number of people have commented that this is a completely worthless plugin. While it is pretty silly, i like that I can, in the words of Ron Popeil, set it, and forget it.

I’m pleased to announce MicroRevie.ws

It is a site that aggregates small reviews from Twitter and marks them up with all kinds of microformats and other goodies.

If you don’t understand yet, go read the About page. If you still don’t understand, let me know, and I’ll rewrite it.

The site is a small playground that Matt Slack and I whipped up in the last few weeks (mainly during SXSW).

Love it? Hate it? Have feature suggestions? Let me know!

P.S. Don’t forget to follow @hreview on Twitter!

I’m headed to SXSW and sticking around for the Music festival. I thought it would be fun to load up my iPod with the bands so I have better ideas who to see.

The huge list of bands notes which ones have free MP3s to download, so I decided to grab them all, with Ruby.

I whipped together a script last weekend to do just that. It will look over all the bands and grab all the songs. Grab the file and run it. It will create a directory called ‘downloads’ and put all the songs in there.

If you run it more than once, it will only grab new songs. At minimum, it only makes requests for the list pages (6 currently), so its pretty low-overhead.

Feel free to improve it and let me know. The code is pretty simple, but did what I needed.

Oh, and be careful: the list is currently just under 4 GB.

I think I’ve written code to do this at least 3 times, so today I tried to do it right. Let’s say you have a Rails model with a couple dates (any event with beginning and end will do). You generally want to make sure it ends after it begins.

validates_chronological :begin_time, :end_time

If you try to set end_time before begin_time, you’ll get an error like:

1 errors prohibited this event from being saved

There were problems with the following fields:

• End time is not in chronological order

It will handle more than 2 fields and is configurable as you’d expect.

Grab the code, which is now part of a plugin we’ll talk about more in the next few weeks.

Update: Since I wrote this, I found out the git-svnimport is deprecated. The good news is that the process is now much easier!

I started importing a bunch of our Rails plugins into GitHub this morning.

A couple of issues proved problematic:

1. Our plugins were in a single svn repository and not the typical trunk and branches setup:

   rails/plugins/acts_as_audited
   rails/plugins/acts_as_geocodable
   …
   rails/plugins/with_action
   

   To get this to work, you use the root of the repository as the path and pass -T with the path you want to fetch.

   git svnimport -T plugins/acts_as_audited http://source.collectiveidea.com/public/rails

   This is totally fine with git svn. Don’t worry about it. I strongly recommend putting each plugin (or project) into its own git repository, as it will make your git life easier.
2. Usernames in svn are stored just as our unix usernames (brandon, daniel). For git, I want to use our email addresses.

   For that, you make a file of all the authors in the format

   username = User's Full Name <email@addr.es>

   and pass -A path_to_author_file

   git svnimport -T plugins/acts_as_audited -A ./authors http://source.collectiveidea.com/public/rails

   git svn clone http://source.collectiveidea.com/public/rails/acts_as_audited -A ./authors

3. Finally, to save trouble, I want to create a new git repository. Trust me, its just easier. Pass in the path you want. -C some_path

   git svnimport -C acts_as_audited -T plugins/acts_as_audited -A ./authors http://source.collectiveidea.com/public/rails

That’s it! After that, you have a git copy of your repository and can push it to GitHub, or do whatever you want.

Here’s the final command for reference (I added -v for verbose):

<del>git svnimport -v -C acts_as_audited -T plugins/acts_as_audited -A ./authors http://source.collectiveidea.com/public/rails</del>
<ins>git svn clone http://source.collectiveidea.com/public/rails/acts_as_audited -A ./authors </ins>

git svnimport -v -C acts_as_audited -T plugins/acts_as_audited -A ./authors http://source.collectiveidea.com/public/rails

git svn clone http://source.collectiveidea.com/public/rails/acts_as_audited -A ./authors 

Check out our projects on GitHub.

My least favorite science fiction story type is space aliens act friendly but in a shocking twist they’re out to screw us over.

-Leonard Richardson

I, for one, welcome our new X-UA-Compatible overlords.

-Jonathan Snook

Background.

We got a Christmas card today addressed to Collective Idea, Corp. Yuck!

The company is Collective Idea.

For legal purposes, the business is Collective Idea, Inc. (and before May 2006, Collective Idea LLC) however that is not what we go by.

Personally, I think that Inc. (and LLC, etc.) sound pretentious. It sounds like Look at us, we’re fancy and a huge company! Whereas I see it as We can fill out forms, but have no sense of aesthetics.

Therefore, we have registered Collective Idea as an Assumed Name. That’s what we go by, and it’s all good, legal and beautiful.

In addition, we have CollectiveIdea and (at the brilliant suggestion of our Lawyer) collectiveidea.com.

So call us Collective Idea. I had to sign a lot of forms for that name.

I was a member of the ACM in college, which was great for papers and research. They want me to join again, but i can’t find a reason to.

As a member of ACM, the Association for Computing Machinery, you will have access to a host of career-enhancing benefits including:

• Over 2,200 free online course titles from SkillSoft

Not interested.

• More than 1,100 free online books, including a large selection from Safari® featuring O’Reilly and many others

Slightly interested, but reading books online is tiresome, and I buy ones I really want.

• Access to ACM’s new highly targeted Career & Job Center

Not interested.

• ACM’s flagship magazine “Communications of the ACM”

Would be cool, but I probably wouldn’t take the time to read it.

• Full access to “Queue” magazine online, plus access to Queuecasts, downloadable discussions with technology experts

This is already free, and I already get the print edition (also free).

• “TechNews” and “CareerNews” email digests

Not interested.

• ACM’s monthly electronic newsletter “MemberNet”

Not interested.

• The ACM Guide to Computing Literature with over one million bibliographic citations

Not interested.

• The option to subscribe to the full ACM Digital Library with unlimited access to all ACM journals, magazines, proceedings, and newsletters

Yes! This is actually appealing. There are a ton of papers both new and old that interest me. Unfortunately, it is an additional $100. Not interested.

• The opportunity to share your knowledge by participating in ACM’s MentorNet One-on-One partnership program

Not interested.

• A free “acm.org” email forwarding address with high-quality Postini spam filtering

Not interested.

Sorry ACM, I think its time to simplify your offerings and try to add value instead of just adding benifits.

Join ACM today and you will receive a Free ACM Flip-Top World Calculator!

Well in that case, sign me up!

Tired of Rails’ truncate() method cutting off mid-word? Me neither, but my clients are.

Enter awesome_truncate:

# Awesome truncate
# First regex truncates to the length, plus the rest of that word, if any.
# Second regex removes any trailing whitespace or punctuation (except ;).
# Unlike the regular truncate method, this avoids the problem with cutting
# in the middle of an entity ex.: truncate("this & that",9)  => "this &am..."
# though it will not be the exact length.
def awesome_truncate(text, length = 30, truncate_string = "...")
  return if text.nil?
  l = length - truncate_string.chars.length
  text.chars.length > length ? text[/\A.{#{l}}\w*\;?/m][/.*[\w\;]/m] + truncate_string : text
end

The comments say it all, as the code is pretty much voodoo. The bulk of it is based on the standard truncate, so the magic is in the regular expressions.

The resulting string will always end with a full word, not punctuation or whitespace or the middle of an entity. It means the you don’t get the exact number of characters, but usually very close. I prefer this to truncating to a number or words; you get closer to the same number of characters each time.

I’ve seen top apps (Basecamp) get bit by truncating in the middle of an entity. This is my favorite bit.

I’m curious if anyone can think of a more efficient method.

We work with a number of people who are addicted to planning. They will devote weeks, months, or even years to planning every piece of their application, before getting any developers involved. Planning is certainly valuable, in moderation, but too much will doom your project before it starts.

We are uncovering better ways of developing software…Through this work we have come to value: …Responding to change over following a plan

The signers of the Agile Manifesto understood this well; they had learned firsthand the problems with planning.

A software project cannot be fully defined before you begin. Most clients will think their project is the exception, but they are frankly wrong. Inevitably, some new idea will come up in development that will make change desirable, if not necessary. 37signals have written about this extensively, but they use a different term: Getting Real.

With a large, detailed plan (did somebody say functional spec?), you will spend too much time up front working on things that don’t matter. Don’t start with logins, don’t build fields for every possible piece of data you might someday collect. Start building the parts that matter. You’ll be able to try them out, refine them, and get feedback before you waste time implementing features you won’t need for 6 months (also, chances are, you’ll have a better way to do it in 6 months).

Quit planning. Find some agile developers and start building. You’ll live a longer, happier and more productive life.

Goal: Get 3 random records from a database table.

There are a number or ways to pick random records in Rails, but none that are perfect.

1. Do it in Ruby

things = Thing.find(:all)
random_things = []
3.times do 
  random_things << things[rand(things.size)]
end

Two problems with this approach:

• Have to fetch the entire set of things
• random_things might have duplicates

The second problem is easy to fix (unless there are fewer than 3 things total, or our rand() takes too long to get 3 distinct numbers)

things = Thing.find(:all)
random_things = []
while random_things.size < 3 
  random_things |= things[rand(things.size)]
end

This solution isn’t great. And we didn’t solve the problem with fetching the whole set.

2. All Ruby with a 2nd query

thing_ids = Thing.find(:all, :select => 'id').map(&:id)
random_ids = []
while random_ids.size < 3
  random_things |= things[rand(things.size)]
end

random_things = Thing.find(:all, :conditions => ['id IN (?)', random_ids])

Ok, now we only select ids, so we don’t have to load the whole set. We could make the first query faster if we wrote SQL instead of fetching them as Things. (Also, does SQLite support the ‘IN’ operator?)

3. Do it in the database

I stumbled across this idea, and thought it was fantastic.

random_things = Thing.find(:all, :limit => 3, :order => 'random()')

Take a look at this a minute. We can do it all simply, and in one line! This seems to be a standard SQL construct, as it works similarly across a few of them (silly MySQL uses rand() instead) including SQLite.

This has the benefit of being simple, and would seem to be faster as it is happening in the database.

Caveats

Unfortunately, its not that clear. Each database does this differently (generally adds a column to each row in your table, in memory), and it doesn’t scale well (remember that :limit doesn’t can’t get applied until the result set is fully built).

After I put together a plugin and a patch to make this transparent, I came across a Rails ticket that got flagged ‘wontfix’. Jeremy Kemper’s response to this idea was:

order by rand() limit 1 is very taxing on the database. It shouldn’t be a standard nor encouraged feature.

Jeremy certainly isn’t wrong, but I’m not sure he’s fully right either. We have a solution that won’t be usable with large tables (or a high-traffic site), but technically works well.

The Plugin:

Here’s the working plugin. which lets you pass :random to any finder.

Thing.find :all, :order => :random

./script/plugin install -x http://source.collectiveidea.com/public/rails/plugins/random_finders

It works as with MySQL, SQLite and PostgreSQL. Do listen to Jeremy’s warning. I’m using it on a an app with a dataset that will stay relatively small. If you run into scaling issues, don’t say we didn’t warn you.

Anyone have a better solution?

After last year’s great time in Chicago, Brandon and I are traveling to Portland, Oregon for RailsConf 2007.

Personally, I’m looking forward to:

• Portland. I’ve never been, but have wanted to. I just hope there is some time in the packed 9am-11pm schedule to see the city.
• People. I’ve met a lot of great Rails developers, so I’m glad to see them and meet more.
• Listening. We’re not speaking, presenting, searching, or have any other agenda. I’m going to enjoy hearing people talk about new ideas, different approaches, and all the edge rails bits that we’ve been using for a while.

If you’re at the conference, stop and say hi.