I love fragment caching in Rails
Phil Karlton said the only difficult things in software development were cache expiration and naming things.
While this statement can be debated, nobody can argue that these two things are extremely difficult (and for the record, I tend to agree with Phil). This post is about the first one: caching and the expiration of that cache.
While caching can become a headache as the data model gets more complex, at a rudimentary level, a little caching in Rails can not only be done extremely easily, but it can go a LONG way.
FourthSegment had a dashboard action that was performing less than ideal, so I decided to spend some time seeing if I could speed things up.
I sat down for about 20 minutes, made a couple changes, and pushed them live. The results — recorded over 7 days by New Relic — speak for themselves:

Yeah.
All I did here was implement some simple fragment caching around the view of the conversation table. The caching was extremely simple:
And the expiration was also simple:
Note that the cache fragments are keyed by a prefix and the site ID. If I left the site ID out, then all users would see the dashboard from whoever was the last one to refresh the fragment. Not ideal :)
FourthSegment runs on Heroku, so I used their free memcached add-on to facilitate the actual caching. It was super easy to setup.
There’s something very interesting in this graph that might not be obvious right away. Notice the dark green on the left side. That’s request queuing time. It’s the amount of time Heroku’s nginx layer had to hold onto the request before my application was ready to process it. It doesn’t show up nearly as much after the fix was put in place. In this case, the dashboard was taking so long that, despite having several concurrent processing threads, they were piling up and blocking other requests.
The bottom line here is that you shouldn’t let the fact that caching is hard stop you from spending a little bit of time on your worst endpoints. It can go a long way towards speeding up your entire application.
Scoped mass-assignment in Rails3
I’ve fought this beast a few times — I think this is a perfect solution. Via @dpickett
Stinky Magic
This post was written in response to http://modernsage.wordpress.com/2011/02/05/rails-3-0-arel-acrobatics/. The original post has code examples that are clearly for illustrative purposes and not from the codebase (code where AR models are not descending from AR::Base, for example), so it’s possible that some important details have been left out. However, for the sake of conversation (and because sometimes it’s just fun to write a nerdy blog post), I wanted to respond.
In that blog post, my buddy Chris is doing some stuff with Arel/AR that, in my opinion, is backwards. It’s causing some confusion that led him to extend ActiveRecord::Base in a way that I feel is unnecessary, and potentially dangerous down the road.
Chris wants to do stuff like:
Forum.mine.posts.comments
The intent is that he wants to access all the comments owned by a post owned by a forum owned by a particular user. Working off the Forum model, he wants to scope things through the ‘mine’ scope, which scopes the Forums to an authenticated user.
Two problems
First, Forum.mine is a backwards way of accessing records belonging to an authenticated user. Normally, a Rails app would authenticate a user and provide helpers at the view and controller level to access that user. You’ll see this in a lot of apps as the method ‘current_user’.
The problem with Forums.mine is that in order for it to work, there has to be code inside the model that goes back out to the authentication system. Authentication is the controller’s domain. It’s an awkward coupling of concerns that causes your code to get all tangled up.
A much more straightforward way to retrieve objects is to have the user be at the root when going left to right. The common way to get Forums owned by a person at the controller level would be:
current_user.forums
The second problem is that:
Forum.mine.posts.comments
just doesn’t make sense.
Posts, as a collection, don’t have comments. A single Post has comments. The concept of Comments that belong to many Posts doesn’t exist here. It’s a confusion I’ve seen in new and experienced ActiveRecord developers alike.
This is much more straightforward:
current_user.forum_comments
In a github repo, I’ve put an example of how I would write this. User#forum_comments, in this case, uses a has_many :through relationship to get at the posts, and then does a map and a second query to get the comments.
class User < ActiveRecord::Base
has_many :forums
has_many :posts, :through => :forums
def forum_comments
Comment.where(:post_id => posts.map(&:id))
end
end
Because Chris didn’t want to do this, he implemented an extension of ActiveRecord::Base that allows it to work:
Forum.mine.posts.comments
It goes through and creates all the necessary scoping to allow that query to return comments. Even though I don’t have any examples, I’m pretty this will break down as more complicated relationships are introduced.
Stinky Magic
The last paragraph of Chris’ post says:
You can use it with a single association or with a monster hierarchy of the form A -> B -> C -> D -> E -> F ->DRY, recursive, scalable. Now that’s what I call magic.
In my opinion, this is horribly dangerous. If you start doing this stuff, and A, B, C, D, and E grow to be a couple million rows (which is not very uncommon at all nowadays) you’re gonna have a very non-scalable join to deal with.
I think a developer should err on the side of verbosity and explicitness when defining relationships. If you really need to do A -> B -> C -> D -> E -> F a lot, that’s a sign that some de-normalization may be justified. Assuming that -> denotes has_many, F would have an e_id on it. In this case, adding an a_id would allow you to go A -> F directly and much faster.
A little bit of CS (read: Kool-Aid)
Another reason to go the “user.forum_comments” route instead of the “Forum.mine.photos.comments” route is that it invites developers to constantly violate the Law of Demeter. I didn’t go to school for CS, but I’ve come to respect this rule over my years of Ruby development. It’ll help to keep your code readable, and easier to mock behavior for testing.
An open letter to recruiters from developers
I get InMail, tweets, and direct emails all the time from recruiters. I think I make it pretty clear that I’m not interested in finding a new job right now, but the emails come anyway. They almost always have the same format:
[generic greeting]. I came across your (twitter profile|profile on LinkedIn|blog). My organization [generic recruitment firm name] has a [arbitrary list of ‘excitement’ related adjectives] position in Ruby/Rails that I think you’d be perfect for [even though there’s a very good chance I don’t have a clue what your skill set is even after reading your resume/profile because I don’t have a technical background]. If you are interested, please get back to me and I will send you [the job description that for some inane reason I didn’t include in the first place]. If you are not, perhaps there is someone in your network who would be interested.
[generic sign off / apology for wasting time]
* rolls eyes *
Here’s my open letter response to all recruiters:
Dear Recruiters,
Us developers have been getting pretty tired of the “you might not be interested, but maybe you know someone who is” line. ALL of you recruiters use this line. I understand you probably think that you are ”networking”, but it really just seems like you’re trying to get us to do your job. What ends up happening is that us developers all get together in a bar, have a few beers, and talk about how incompetent tech recruiters are. Lord help you if you happen to reach out generically with the SAME email on the SAME day to 3 or 4 developers who are friends. It’s probably not the case that you are incompetent, but it’s definitely a place where we lose some respect for you.
So, cool it. If you have a good idea, go find people that are actually looking for a job and write them a personalized email telling them about it. If you’re new and not technical, admit it. Tell us “hey man I have this Ruby/Rails gig and I saw those keywords on your profile. It might not be a perfect fit, but would you ever want to chat about it? I’m trying to learn more about this space.”
Most developers (that I know anyway) react better to requests for help than generic sales pitches.
Be honest. Be personalized. Be respectful. Everybody wins.
$0.02
Love,Developers
EDIT: Shame on me. The recruiter who reminded me to finally publish this draft responded to me directly about this blog post. He informed me (very respectfully) that I had not removed ‘looking for new ventures’ or ‘looking for career opportunities’ from my LinkedIn profile. So apparently I was wrong; I had not made it clear to recruiters that I’m not looking for work. The rest of the rant stands anyway, though. :) Happy New Year!
How can we accelerate a Ruby developer’s development?
My slides from Mongo Boston
Going from Web Services to Party Planning
A few months ago, MyPunchbowl launched what we’ve been calling the Vendor Portal. If you go to www.punchbowl.com/vendors, you can search for vendors by category or keyword in a specific location. In Rails-speak, www.mypunchbowl.com/vendors is part of the monolithic MyPunchbowl.com application. We have a namespaced controller that deals with the requests, layouts and views for the main, state, city, state category, city category, and search results pages. But none of the vendor data exists in the MyPunchbowl.com database. We have another Rails app, creatively referred to as “The Vendors App”, that exposes a RESTful API around the several resources:
- Cities
- States
- Vendors
- Categories
We’ve extend the standard REST actions to include search on each of those resources, and that’s where all the magic happens. Whatever a user types in for a search gets passed to the Vendors REST controller. Some basic parameter filtering goes on in the controller, which then hands off to a wrapper that sits on top of ThinkingSphinx for full-text geosearching.
Next, we launched http://vendors.punchbowl.com, which provides a way for vendors to add their business to our database. The app is fairly straightforward:
- Some CRUD for adding a profile, a photo, selecting some themes
- A payment gateway so that vendors can buy better search ranking, photo galleries, hide ads on their listing page, etc…
- An admin where entries can be reviewed be our Vendor Quality team
I’ve had my share of nerdstorms on this project:
- Hacking up ThinkingSphinx because GoogleBot decided to start getting page 492 of search results when there were only 8 pages of search results, which TS did not particularly appreciate
- Playing with MongoDB’s mapreduce to allow us to give real time information on how many users are searching for a particular vendor category in a particular location
- Moving from an XML API to a JSON one when I realized that JSON was oodles more efficient (on both the generation and parsing ends)
- Improving caching mechanisms on the MyPunchbowl.com side of things to reduce the load on our Vendors API application
And so on….
But it’s cool to pop up from the technology trenches every now and then and actually take a look at what we’re doing. And what we’re doing is allowing people do this:
http://www.punchbowl.com/vendors/nm-new-mexico/taos/2401264/fun-peak
http://www.punchbowl.com/vendors/ma-massachusetts/lexington/2176091/sweet-beads—-birthday-parties-at-home
http://www.punchbowl.com/vendors/me-maine/bangor/2504232/dana-lavertu-dj-entertainment
http://www.punchbowl.com/vendors/il-illinois/maple-park/1494610/wild-orchid-custom-floral-design
http://www.punchbowl.com/vendors/al-alabama/birmingham/2360520/amerson-events-dj-service
And that’s pretty damn cool if you ask me. Yes, we have a long way to go. But seeing how many vendors are actually using this system to get in touch with people planning parties is awesome. Seeing how much effort they put into their listings and how creative they can be is exciting. I’m really proud of what we’ve done with the Vendor system at MyPunchbowl. And I’m looking forward to see where it goes from here.
Take a look around at what you’re doing. Have you popped up lately to see how users are using what you are building? It can be really motivating. If you have any cool stories for stuff that users are doing with your applications, share them in the comments. I’d love to see it.
Easier debugging when refactoring Rails controllers
When refactoring a big controller with lots of before filters, it’s common (for me anyway) to get into a situation where tests start failing due to before filters redirecting. Today I made it a lot easier to debug these failing tests by adding this to my ApplicationController:
def redirect_to_with_logging(*args)
logger.debug "Redirect: #{args.inspect} from #{caller[0]}"
redirect_to_without_logging *args
end
alias_method_chain :redirect_to, :logging
This will log where the new redirect is coming from, and make it a lot easier to figure out what is breaking your tests. Enjoy.
Named routes on rake tasks
To be able to use named routes ( modelName_path, edit_modelName_path, etc ) on a rake task, you have to include ActionController::UrlWriter.
