Productification

Intersection between Technology, User Experience and Product Innovation


01 Jul

How to influence the query plan in Lucene / Solr?


I was looking through Luence’s source code today (okay - night) to find whether you could provide hints to Lucene to change the clause precedence during query execution. Unfortunately, I found that Lucene does not support users to supply any such hint (I was looking at ConjunctionScorer).

At work, we have a use case, where we have knowledge about the data that gets indexed into the system. Post query classification and pre-processing, we can utilize this knowledge to inform lucene about what we think the execution order of each clause in the query should be. This could drastically improve the performance of the query, mostly for AND queries where, when run independently, one clause would return a handful of results and the other clause would return thousands or even tens of thousands of results.

In the past, I have worked on IR systems that would maintain cardinalities (statistics) of the index, that would help optimize the query and produce a best (in given time and resources) plan.

Does anyone know whether Lucene maintains these cardinalities internally? If so, how does it impact the query execution plan?


2 Responses Filed under: Development, Search Tags: ,
02 Mar

django-reviews - Generic ratings and review application for django projects


django-reviews - A generic review application for Django projects, which allows association of a number of reviews with any Model instance and makes retrieval of reviews simple.

django reviews

I dug around the django space and couldn’t find a generic review application that would allow users to add a review (including ratings) to an entity. So, I decided to write one myself. The current application is modeled similar to django comments. The screenshot above is what you can use the application for.


20 Feb

Django - Login with email or username


This snippet allows you to login with either username or email in your Django applications. All you have to do is add the middleware and combine it in the authentication backends.


1 Response Filed under: Development, Django
09 Jan

Running Hadoop on Mac


For those of you who want to run map reduce jobs using Hadoop on your Mac - here is a great article that guides you through a step by step process.


No Response Filed under: Development Tags:
04 Nov

Scriptify your Django deployments using Fabric


Anyone who has worked on a decent size product where Django is serving as the web front, understands how monotonous the update / deployment procedure gets. Especially, when you have more than a couple of servers serving your user base. And if you are one of the companies who have embraced the cloud computing platform (Amazon Web Services), then you already have an appreciation for the problem.

Fabric is one such tool that helps in solving that problem.

Fabric is a tool that, at its core, logs into a number of hosts with SSH, and executes a set of commands, and possibly uploads or downloads files.

Check out Will Larsen’s post on how you can use Fabric. He has done a great job of explaining it.