Multiple queries in single request

[Brian Knoth]

Apologies up front if this is a often-asked newbie question, but I've searched the solrhandler docs and I can't find what I'm interested in.

I'd like to be able to present multiple queries to SOLR, for example:

[(+a +b) ]
[(+a +c)]
[(+a+ d)]

And get back a multi-result set (one for each query). I don't see that any of the default, included search handlers provide this capability?

Is this possible, or would a custom handler need to be created.

Thanks very much for any help!

Regards,
Brian

[Shalin Shekhar Mangar]

Post by Brian Knoth
Apologies up front if this is a often-asked newbie question, but I've
searched the solrhandler docs and I can't find what I'm interested in.
[(+a +b) ]
[(+a +c)]
[(+a+ d)]
And get back a multi-result set (one for each query). I don't see that any
of the default, included search handlers provide this capability?
Is this possible, or would a custom handler need to be created.

No but you can always fire three requests. Writing your own handler which
prints data in a custom format means that you can no longer use existing
solr clients for java/ruby/python etc.

[Brian Knoth]

Thanks for your comment. Yes, it would be simple if it was just 3 queries, I really have about 50-60 I'd like to consolidate as a single request. I'm simply getting a count of records for each query, and right now, it takes about 40mS for each request as a single query, but multiplied by about 50 or 60, brings me up into the 1.5/2.0 second response time (to handle them all)

I've got to believe that it would be much more efficient for the search engine to know about all of these things that I want in one request and generate a multi-response set. I'm kind of surprised that this is not a more commonly requested capability.

If anyone has any more suggestions, I'd appreciate hearing them.

Thanks,
Brian

-----Original Message-----
From: Shalin Shekhar Mangar [mailto:***@gmail.com]
Sent: Friday, March 06, 2009 2:55 PM
To: solr-***@lucene.apache.org
Subject: Re: Multiple queries in single request

Post by Brian Knoth
Apologies up front if this is a often-asked newbie question, but I've
searched the solrhandler docs and I can't find what I'm interested in.
[(+a +b) ]
[(+a +c)]
[(+a+ d)]
And get back a multi-result set (one for each query). I don't see that any
of the default, included search handlers provide this capability?
Is this possible, or would a custom handler need to be created.

No but you can always fire three requests. Writing your own handler which
prints data in a custom format means that you can no longer use existing
solr clients for java/ruby/python etc.

[Shalin Shekhar Mangar]

Post by Brian Knoth
Thanks for your comment. Yes, it would be simple if it was just 3 queries,
I really have about 50-60 I'd like to consolidate as a single request. I'm
simply getting a count of records for each query, and right now, it takes
about 40mS for each request as a single query, but multiplied by about 50 or
60, brings me up into the 1.5/2.0 second response time (to handle them all)

Perhaps if you tell us more about what you are trying to do, we can suggest
an alternative? What do those queries (for which you need count) look like?
Are you familiar with faceting? Looks like facet.field or facet.query may be
helpful here.

The 40ms for a single query may not be the overhead of an http request (that
is more like 2-5ms) and even if you executed all of them through one http
request, they may still take approximately the same amount of time.

[Erik Hatcher]

If you're only after the count for those queries, use a standard
request with a bunch of facet.query's.

Erik

Post by Brian Knoth
Thanks for your comment. Yes, it would be simple if it was just 3
queries, I really have about 50-60 I'd like to consolidate as a
single request. I'm simply getting a count of records for each
query, and right now, it takes about 40mS for each request as a
single query, but multiplied by about 50 or 60, brings me up into
the 1.5/2.0 second response time (to handle them all)
I've got to believe that it would be much more efficient for the
search engine to know about all of these things that I want in one
request and generate a multi-response set. I'm kind of surprised
that this is not a more commonly requested capability.
If anyone has any more suggestions, I'd appreciate hearing them.
Thanks,
Brian
-----Original Message-----
Sent: Friday, March 06, 2009 2:55 PM
Subject: Re: Multiple queries in single request
On Sat, Mar 7, 2009 at 12:27 AM, Brian Knoth

Post by Brian Knoth
Apologies up front if this is a often-asked newbie question, but I've
searched the solrhandler docs and I can't find what I'm interested in.
[(+a +b) ]
[(+a +c)]
[(+a+ d)]
And get back a multi-result set (one for each query). I don't see that any
of the default, included search handlers provide this capability?
Is this possible, or would a custom handler need to be created.

No but you can always fire three requests. Writing your own handler which
prints data in a custom format means that you can no longer use existing
solr clients for java/ruby/python etc.
--
Regards,
Shalin Shekhar Mangar.

[Brian Knoth]

Shalin, Erik:

Faceting is exactly what I was looking for (and didn't even know it). This is giving me precisely what I wanted.

Thanks for your help!

brian

-----Original Message-----
From: Erik Hatcher [mailto:***@ehatchersolutions.com]
Sent: Friday, March 06, 2009 3:39 PM
To: solr-***@lucene.apache.org
Subject: Re: Multiple queries in single request

If you're only after the count for those queries, use a standard
request with a bunch of facet.query's.

Erik

Post by Brian Knoth
Thanks for your comment. Yes, it would be simple if it was just 3
queries, I really have about 50-60 I'd like to consolidate as a
single request. I'm simply getting a count of records for each
query, and right now, it takes about 40mS for each request as a
single query, but multiplied by about 50 or 60, brings me up into
the 1.5/2.0 second response time (to handle them all)
I've got to believe that it would be much more efficient for the
search engine to know about all of these things that I want in one
request and generate a multi-response set. I'm kind of surprised
that this is not a more commonly requested capability.
If anyone has any more suggestions, I'd appreciate hearing them.
Thanks,
Brian
-----Original Message-----
Sent: Friday, March 06, 2009 2:55 PM
Subject: Re: Multiple queries in single request
On Sat, Mar 7, 2009 at 12:27 AM, Brian Knoth

Post by Brian Knoth
Apologies up front if this is a often-asked newbie question, but I've
searched the solrhandler docs and I can't find what I'm interested in.
[(+a +b) ]
[(+a +c)]
[(+a+ d)]
And get back a multi-result set (one for each query). I don't see that any
of the default, included search handlers provide this capability?
Is this possible, or would a custom handler need to be created.

No but you can always fire three requests. Writing your own handler which
prints data in a custom format means that you can no longer use existing
solr clients for java/ruby/python etc.
--
Regards,
Shalin Shekhar Mangar.

[Erik Hatcher]

On Sat, Mar 7, 2009 at 12:27 AM, Brian Knoth

Post by Brian Knoth
Apologies up front if this is a often-asked newbie question, but I've
searched the solrhandler docs and I can't find what I'm interested in.
[(+a +b) ]
[(+a +c)]
[(+a+ d)]
And get back a multi-result set (one for each query). I don't see that any
of the default, included search handlers provide this capability?
Is this possible, or would a custom handler need to be created.

No but you can always fire three requests. Writing your own handler which
prints data in a custom format means that you can no longer use existing
solr clients for java/ruby/python etc.

That's not a fair characterization of at least the Ruby client. The
NamedList (err, Hash in Ruby) is simply a data structure returned that
can be easily navigated. Sure, any client API that expects a certain
structure will have issues, but in general no problem in just Hashing
it out.

Erik

[Ryan McKinley]

Post by Erik Hatcher

Post by Shalin Shekhar Mangar
No but you can always fire three requests. Writing your own handler which
prints data in a custom format means that you can no longer use existing
solr clients for java/ruby/python etc.

That's not a fair characterization of at least the Ruby client. The
NamedList (err, Hash in Ruby) is simply a data structure returned
that can be easily navigated. Sure, any client API that expects a
certain structure will have issues, but in general no problem in
just Hashing it out.

Also not entirely true for the solrj client. Assuming the response
includes the standard solr data structures (NamedList, DocList, Doc,
etc...) the solrj client will parse the response.

ryan

[Shalin Shekhar Mangar]

Post by Ryan McKinley

Post by Erik Hatcher

Post by Shalin Shekhar Mangar
No but you can always fire three requests. Writing your own handler which
prints data in a custom format means that you can no longer use existing
solr clients for java/ruby/python etc.

That's not a fair characterization of at least the Ruby client. The
NamedList (err, Hash in Ruby) is simply a data structure returned that can
be easily navigated. Sure, any client API that expects a certain structure
will have issues, but in general no problem in just Hashing it out.

Also not entirely true for the solrj client. Assuming the response
includes the standard solr data structures (NamedList, DocList, Doc, etc...)
the solrj client will parse the response.

Yes, apologies for the wrong characterization. I meant to say that one
cannot use the nice API methods to navigate the results.

[Noble Paul നോബിള്‍ नोब्ळ्]

But it is still possible to have a RequestHandler (Say
BatchRequestHandler ) which can take in multiple queries .
say
bc.q.count=3
bc.1.q=[(+a +b) ]
bc.2.q=[(+a +c)]
bc.3.q=[(+a+ d)]

The response may contain several sections

<response>
<lst name="bc.1.q">
...output one 1.q goes here
</lst>
<lst name="bc.2.q">
...output of 2.q goes here
</lst>
<lst name="bc.3.q">
.....output of 3.q goes here
</lst>
</response>


The standard SolrJ API's may not be useful but it can be enhanced.

--Noble

On Sat, Mar 7, 2009 at 1:51 PM, Shalin Shekhar Mangar

Post by Shalin Shekhar Mangar

Post by Shalin Shekhar Mangar
No but you can always fire three requests. Writing your own handler which
prints data in a custom format means that you can no longer use existing
solr clients for java/ruby/python etc.

That's not a fair characterization of at least the Ruby client.  The
NamedList (err, Hash in Ruby) is simply a data structure returned that can
be easily navigated.  Sure, any client API that expects a certain structure
will have issues, but in general no problem in just Hashing it out.

Also not entirely true for the solrj client.  Assuming the response
includes the standard solr data structures (NamedList, DocList, Doc, etc...)
the solrj client will parse the response.

Yes, apologies for the wrong characterization. I meant to say that one
cannot use the nice API methods to navigate the results.
--
Regards,
Shalin Shekhar Mangar.