In MPDX, the Google Contacts sync job takes a long time and the google accounts loop to sync each job could benefit from find_each(batch_size: 1)
. Basically it seems likes find_each
pulls in the records in batches and then saves them in an array to enumerate through. Here's a comparison of the memory results with different batch_sizes
that I did using a similar, but contrived MemHungry
model. To setup, first do 6.times { MemHungry.create }
.
Using the default batch_size
of 1000 the memory at the end reflecs all 6 objects and the RAM they hold onto:
[1] pry(main)> MemHungry.all.find_each { |m| puts m.id; m.eat_memory; }
MemHungry Load (0.5ms) SELECT "mem_hungries".* FROM "mem_hungries" ORDER BY "mem_hungries"."id" ASC LIMIT 1000
1
Memory before GC: 112.63671875
Memory before allocation: 159.90234375
Memory after allocation: 1759.90234375