Performance Optimization Techniques

This article is about few performance improvements points which I have been using in my current project and they can be applied generally by you as well. I have written this article in a generic way since the tech stack is different than Rails which is commonly used.

1. Eager Loading & Avoiding n+1 problem

Eager loading is a way to solve the classic N + 1 query performance problem caused by inefficient use of child objects.

For Example, In the following code, it will fetch 10 users.

users = User.all(:limit => 10)
users.each do |user|
  puts user.address.pincode
end

Hence, 11 queries will be executed, 1 for the top and 10. The solution is to rewrite it to eager load address:

users = User.includes(:address).limit(10)

users.each do |user|
  puts user.address.pincode
end

2. Memoization

A basic example to demonstrate is:

def current_user
  User.find(session[:user_id])
end

It will make the query multiple times in a single request whenever we require current user.

The better approach will be:

def current_user
  @current_user ||= User.find(session[:user_id])
end

3. DRY(Don’t Repeat Yourself)

Here is very basic example of DRY in terms of optimization which we sometimes ignore:

if(User.find_by_id(user_id).admin == true)
  return User.find_by_id(user_id)
else
  return nil
end

Here User.find_by_id(user_id) should have been taken in a variable.

4. Use SQL when necessary

Using SQL at the right places can boost your app performance immensely. The main tradeoff of using SQL is, it makes debugging difficult. So, the clear, readable and indented code should be written. Following are the points related to usage of SQL:

a) Query Interface
Using the query interface of the ORM to make a single query with multi-table data is generally used rather than using multiple queries in ORM way which also takes account of the ORM overhead.

b) Stored Procedures
Highly intensive method(s) where a large amount of data is manipulated with complicated conditions can be moved at the DB level by using stored procedures which can really improve your app performance.

c) Data Aggregation
Data aggregation (sum, avg, etc.) should mainly be done at database level especially in case of multiple and/or complicated aggregations which might also need sorting of data. It will always be faster than the in-memory language level aggregations.

d) Query Optimizations
It is one thing to write direct SQL queries/procedures and other to make optimizations to it. Somethings like using joins instead of subqueries generally work better. But, bad performance is only noticeable when data increases. So, keep looking for certain small changes which might make a huge difference specific to a query when the problem arises.

5. Bulk Updation

In cases of plenty of insertions/updates at once, it is better to take your code as close to the database as possible to do bulk insertions/updates instead of looping each data object and do insertion/update.
For this, we use our own gem ‘dm-bulk’ which is just a wrapper to create SQL queries for bulk tasks efficiently. Similarly, in case of Rails ‘update_all’ can be used. For insertion, there is no inbuilt functionality but there might be gems which do so.

6. Avoiding Dynamism

Although dynamic methods like find_by and find_all_by are really cool, they are also kind of slow because each one needs to run through method_missing and parse the filename against the list of columns in a database table.

7. Database Optimization

Some of the things related to optimally using the database have been discussed in above points. But, this point is related to optimizations on DB level. It is a vast area and specific from project to project. So, I would like to just brush on it.

a) Indexing
Database indexing is one of the simplest ways to improve database performance in terms of fetching the data. The tradeoff is that the insert operation will become slower but will boost up fetching data which is more frequently used in web application. But, be mindful of Unnecessary or Bad indexing where indexes aren’t being used can adversely affect the DB performance.

b)Tuning of database
In this various parameters can be tuned permanently or temporarily(may for the particular query) according to specific needs. For e.g. work_mem can be increased to speed up sorts as it facilitates in-memory sorting. But, every tuning has its tradeoff, that’s why it should be done after careful analyzation.

8. Profiling

Last but not the least is Profiling. Sometimes nothing obvious seems to be apparent. The only way to go is thoroughly profiling your bad performing code to figure out what’s wrong yourself and apply the above-mentioned techniques or some other. That’s when you use the profiler. Ruby-Prof is what everybody uses in Ruby world.

Even if you don’t want to use any profiling tool, we can just have the time capturing points to see which part of the code is taking longer.

p Time.now
... code goes here
p Time.now
... code goes here
p Time.now

Conclusion

Most recently, we have been working on using direct SQL queries and DB optimizations to speed up some parts of our application.
For example:

1. There was a background worker which earlier for around 2 min for completing a specific task took ~2 sec after using sql queries and bulk update optimizations.
2. Another example is of SQL optimization where valid_busines_day/date
   was calculated for each invoice but actually, it is independent of
   an invoice and only dependent on the program to which invoices belong. So
   changed SQL to calculate valid_busines_day/date first and then
   joined it with invoices.

select … from programs
join invoices on (..)
join (calculation for valid_busines_day for the program) on (..)

was changed to

select … from programs
join (calculation for valid_busines_day for the program) on (..)
join invoices on (..)

which gave us a huge (~20x) improve improvement on a particular report.

These were some of the obvious but important things regarding the backend optimizations which we have been doing constantly. I hope you like the article and feel free to comment and feedback.

Inspirational Resources:
https://www.airpair.com/ruby-on-rails/performance
http://www.nascenia.com/10-tips-to-boost-up-performance-of-your-ruby-on-rails-application/