Building sql insert strings on the fly in Ruby on Rails to improve performance -
i use following code sql insert string build insert statement bulk insertion of large numbers of records. need because saving them singly via activerecord slow.
adding them in via activerecord-import slow -- slower current method. e.g., method takes 15 seconds activerecord-import takes 4 minutes 10 seconds on sample bulk insert of 100k records (which typical me).
class myclass < activerecord::base def get_sql_insert_string "('#{entry_date.strftime('%y-%m-%d')}', #{field1.nil? ? "null" : field1}, #{field2.nil? ? "null" : field2}, #{field3.nil? ? "null" : field3})" end end the problem fragile. every time add field, need remember include in method. can use dynamic programming (define_method) build once when class loads?
there few tools exist make life lot easier. rather write own sql insert statements recommend using gem such activerecord-import handle doing bulk inserts you.
to squeeze last bit of performance out of activerecord-import can use columns property of active record classes. instance
columns = myclass.columns.map(&:name) values = my_instances.map { |x| myclass.columns.map { |y| x.send(y.name) } } myclass.import columns, values, :validate => false
Comments
Post a Comment