Tuesday, November 13, 2012

Redis

Tonight I was testing the recommendation engine and I noticed that it was recording the questions that the user missed but was not generating any recommendations.

I found out that I need to have a server install that would process the "jobs" of calculating the recommendations. Apparently this is something that is very math intensive and its better to do it this way. Plus its completely compatible with heroku.

I did the following

  1. I installed redis-server and configured it to listen to incoming connections.
  2. Installed Resque to manage the queue of jobs and send them to redis.

After that I just ran

<pre><code>
QUEUE=* rake environment resque:work
</code></pre>

And now everything works like magic.

Thursday, November 8, 2012

Recommendation Engine

Building the recommendation engine was easy, it took me around 20 minutes.
I installed and configured the recommendable gem (https://github.com/davidcelis/recommendable). After that I just had to add one line of code in the stats generation logic that makes the user like a question if its marked as wrong.

Now I can recommend questions to a user based on what other users have missed.

>> user.recommended_questions
=> [#<question_id: 88="88">, #<question_id: 200="200">, #<question_id: 50="50">...]



The next step is to make it work for concept, concept groups, and categories.

Wednesday, November 7, 2012

Stats retrieval

Assignments can be done more than once. Every time a user submits an assignment new stats are generated. This becomes an issue when pulling stats for an assignment because there will be duplicate stats.

To fix this issue I wrote a method that pulls the latest set of stats for an assignment.

  

def self.latest_for_assignment assignment
    categories = []
    concept_groups = []
    concepts = []
    Category.all.each do |c|
      categories << Stat.where(:statable_id => c.id, :statable_type => c.class.name ,:assignment_id => assignment.id  ).last
    end
    ConceptGroup.all.each do |c|
      concept_groups << Stat.where(:statable_id => c.id, :statable_type => c.class.name, :assignment_id => assignment.id).last
    end
    Concept.all.each do |c|
      concepts << Stat.where(:statable_id => c.id, :statable_type => c.class.name, :assignment_id => assignment.id ).last
    end
    [categories,concept_groups,concepts]
  end


I also change the way the show method (in the controller) works.




def show
  @categories,@concept_groups,@concepts = Stat.latest_for_assignment @assignment
  @categories.compact!
  @concept_groups.compact!
  @concepts.compact!
end

The compact! clears nils from the arrays 

Tuesday, November 6, 2012

Thursday, November 1, 2012

Stats generation refactoring

Most of the stats generation methods had unnecessary db calls and overall were very messy. Last week I began rewriting these methods.

Last week I was also able to compile a small group of questions with all their categorization models so this week i'm beginning to do the recommendation engine.

I saw this project and I think I might be able to use it if I modify the rating system to fit questions.
https://github.com/davidcelis/recommendable


Next week I will work on this, continue to make small refactoring, and clean up the UI.