Job queue and search engine
Hi,
I want to implement a search engine with lucene.
To be scalable, I would like to execute search jobs asynchronously (with a job queuing system).
But i don't know if it is a good design... Why ?
Search results can be large ! (eg: 100+ pages with 25 documents per page)
With asynchronous sytem, I need to store results for each search job.
I can set a short expiration time (~5 min) for each search result, but it's still large.
What do you think about it ?
Which design would you use for that ?
Thanks
Mat