Imagine you have a Rails model that does not have an integer primary key. Also imagine that you have a massive number of these models and not a whole lot of RAM. “No problem,” you’d say, “I’ll just use find_each,” which as we all know is a convenient wrapper around find_in_batches. However, if you’re using PostgreSQL and not sqlite, you’ll soon learn that it does not allow you to use the >= operator on strings thus rendering find_in_batches inoperable.

Your options here are to either try and fit them all into RAM or come up with some scheme (perhaps a jiggery-hackery one) to only load a few at a time but iterate over them all. Well hack something together no longer. Introducing Batchelor, an extension for your ActiveRecord classes that have non-integer primary keys.

It is designed to work similar to the default find_in_batches that is a part of ActiveRecord::Relation but allow the use of non-integer primary keys by avoiding the use of >=. I have been told that the reason this is not in Rails is because find_in_batches and find_each are expected to process any new records that are created while the block is running. This will happen for incrementing integer primary keys but is not guaranteed for strings. This is just to give you a convenient and query efficient way to iterate over all records of a model with a non integer primary key.

Check it out and let me know what you think.


—Aaron Rosenberg