URI is too long

[Salman Ansari]

Hi,

I am building a long query containing multiple ORs between query terms. I
started to receive the following exception:

The remote server returned an error: (414) Request-URI Too Long. Any idea
what is the limit of the URL in Solr? Moreover, as a solution I was
thinking of chunking the query into multiple requests but I was wondering
if anyone has a better approach?

Regards,
Salman

[Paul Libbrecht]

How about using POST?

paul

31 January 2016 at 15:20
Hi,
I am building a long query containing multiple ORs between query terms. I
The remote server returned an error: (414) Request-URI Too Long. Any idea
what is the limit of the URL in Solr? Moreover, as a solution I was
thinking of chunking the query into multiple requests but I was wondering
if anyone has a better approach?
Regards,
Salman

[Jack Krupansky]

Or try the terms query parser that lets you eliminate all the OR operators:
https://cwiki.apache.org/confluence/display/solr/Other+Parsers#OtherParsers-TermsQueryParser


-- Jack Krupansky

Post by Paul Libbrecht
How about using POST?
paul

31 January 2016 at 15:20
Hi,
I am building a long query containing multiple ORs between query terms. I
The remote server returned an error: (414) Request-URI Too Long. Any idea
what is the limit of the URL in Solr? Moreover, as a solution I was
thinking of chunking the query into multiple requests but I was wondering
if anyone has a better approach?
Regards,
Salman

[Shawn Heisey]

Post by Salman Ansari
I am building a long query containing multiple ORs between query terms. I
The remote server returned an error: (414) Request-URI Too Long. Any idea
what is the limit of the URL in Solr? Moreover, as a solution I was
thinking of chunking the query into multiple requests but I was wondering
if anyone has a better approach?

The default HTTP header size limit on most webservers and containers
(including the Jetty that ships with Solr) is 8192 bytes. A typical
request like this will start with "GET " and end with " HTTP/1.1", which
count against that 8192 bytes. The max header size can be increased.

If you place the parameters into a POST request instead of on the URL,
then the default size limit of that POST request in Solr is 2MB. This
can also be increased.

Thanks,
Shawn

[Salman Ansari]

Cool. I would give POST a try. Any samples of using Post while passing the
query string values (such as ORing between Solr field values) using
Solr.NET?

Regards,
Salman

Post by Shawn Heisey

Post by Salman Ansari
I am building a long query containing multiple ORs between query terms. I
The remote server returned an error: (414) Request-URI Too Long. Any idea
what is the limit of the URL in Solr? Moreover, as a solution I was
thinking of chunking the query into multiple requests but I was wondering
if anyone has a better approach?

The default HTTP header size limit on most webservers and containers
(including the Jetty that ships with Solr) is 8192 bytes. A typical
request like this will start with "GET " and end with " HTTP/1.1", which
count against that 8192 bytes. The max header size can be increased.
If you place the parameters into a POST request instead of on the URL,
then the default size limit of that POST request in Solr is 2MB. This
can also be increased.
Thanks,
Shawn

[Midas A]

Is there any drawback of POST request and why we prefer GET.

Post by Salman Ansari
Cool. I would give POST a try. Any samples of using Post while passing the
query string values (such as ORing between Solr field values) using
Solr.NET?
Regards,
Salman

Post by Shawn Heisey

Post by Salman Ansari
I am building a long query containing multiple ORs between query

terms. I

Post by Shawn Heisey

Post by Salman Ansari
The remote server returned an error: (414) Request-URI Too Long. Any

idea

Post by Shawn Heisey

Post by Salman Ansari
what is the limit of the URL in Solr? Moreover, as a solution I was
thinking of chunking the query into multiple requests but I was

wondering

Post by Shawn Heisey

Post by Salman Ansari
if anyone has a better approach?

The default HTTP header size limit on most webservers and containers
(including the Jetty that ships with Solr) is 8192 bytes. A typical
request like this will start with "GET " and end with " HTTP/1.1", which
count against that 8192 bytes. The max header size can be increased.
If you place the parameters into a POST request instead of on the URL,
then the default size limit of that POST request in Solr is 2MB. This
can also be increased.
Thanks,
Shawn

[Upayavira]

POST is supposed (as defined by REST) to imply a request with
side-effects. A query as such does not have side effects, so
conceptually, it should be a GET. In practice, whilst it might cause
some developers to grumble, using a POST for a request should make no
difference to Solr (other than accepting a larger query).

Upayavira

Post by Midas A
Is there any drawback of POST request and why we prefer GET.

Post by Salman Ansari
Cool. I would give POST a try. Any samples of using Post while passing the
query string values (such as ORing between Solr field values) using
Solr.NET?
Regards,
Salman

Post by Shawn Heisey

Post by Salman Ansari
I am building a long query containing multiple ORs between query

terms. I

Post by Shawn Heisey

Post by Salman Ansari
The remote server returned an error: (414) Request-URI Too Long. Any

idea

Post by Shawn Heisey

Post by Salman Ansari
what is the limit of the URL in Solr? Moreover, as a solution I was
thinking of chunking the query into multiple requests but I was

wondering

Post by Shawn Heisey

Post by Salman Ansari
if anyone has a better approach?

The default HTTP header size limit on most webservers and containers
(including the Jetty that ships with Solr) is 8192 bytes. A typical
request like this will start with "GET " and end with " HTTP/1.1", which
count against that 8192 bytes. The max header size can be increased.
If you place the parameters into a POST request instead of on the URL,
then the default size limit of that POST request in Solr is 2MB. This
can also be increased.
Thanks,
Shawn

[Salman Ansari]

I tried using POST but faced an issue where I am still not able to send
long data. When I send data in the body that exceeds 35KB I get the
following exception:

"An exception of type 'SolrNet.Exceptions.SolrConnectionException' occurred
in [Myproject] but was not handled in user code

Additional information: The request was aborted: The connection was closed
unexpectedly."

Any ideas why this is happening and how to resolve this?

Regards,
Salman

Post by Upayavira
POST is supposed (as defined by REST) to imply a request with
side-effects. A query as such does not have side effects, so
conceptually, it should be a GET. In practice, whilst it might cause
some developers to grumble, using a POST for a request should make no
difference to Solr (other than accepting a larger query).
Upayavira

Post by Midas A
Is there any drawback of POST request and why we prefer GET.

Post by Salman Ansari
Cool. I would give POST a try. Any samples of using Post while passing

the

Post by Midas A

Post by Salman Ansari
query string values (such as ORing between Solr field values) using
Solr.NET?
Regards,
Salman

Post by Shawn Heisey

Post by Salman Ansari
I am building a long query containing multiple ORs between query

terms. I

Post by Shawn Heisey

Post by Salman Ansari
The remote server returned an error: (414) Request-URI Too Long.

Any

Post by Midas A

Post by Salman Ansari
idea

Post by Shawn Heisey

Post by Salman Ansari
what is the limit of the URL in Solr? Moreover, as a solution I was
thinking of chunking the query into multiple requests but I was

wondering

Post by Shawn Heisey

Post by Salman Ansari
if anyone has a better approach?

The default HTTP header size limit on most webservers and containers
(including the Jetty that ships with Solr) is 8192 bytes. A typical
request like this will start with "GET " and end with " HTTP/1.1",

which

Post by Midas A

Post by Salman Ansari

Post by Shawn Heisey
count against that 8192 bytes. The max header size can be increased.
If you place the parameters into a POST request instead of on the

URL,

Post by Midas A

Post by Salman Ansari

Post by Shawn Heisey
then the default size limit of that POST request in Solr is 2MB.

This

Post by Midas A

Post by Salman Ansari

Post by Shawn Heisey
can also be increased.
Thanks,
Shawn

[Susheel Kumar]

Post is pretty much similar to GET. You can use any REST Client to try.
Same select URL & pass below header and put the queries parameters into body

POST: http://localhost:8983/solr/techproducts/select

Header
==
Content-Type:application/x-www-form-urlencoded

payload/body:
==
q=*:*&rows=2


Thanks,
Susheel

Post by Salman Ansari
Cool. I would give POST a try. Any samples of using Post while passing the
query string values (such as ORing between Solr field values) using
Solr.NET?
Regards,
Salman

Post by Shawn Heisey

Post by Salman Ansari
I am building a long query containing multiple ORs between query

terms. I

Post by Shawn Heisey

Post by Salman Ansari
The remote server returned an error: (414) Request-URI Too Long. Any

idea

Post by Shawn Heisey

Post by Salman Ansari
what is the limit of the URL in Solr? Moreover, as a solution I was
thinking of chunking the query into multiple requests but I was

wondering

Post by Shawn Heisey

Post by Salman Ansari
if anyone has a better approach?

The default HTTP header size limit on most webservers and containers
(including the Jetty that ships with Solr) is 8192 bytes. A typical
request like this will start with "GET " and end with " HTTP/1.1", which
count against that 8192 bytes. The max header size can be increased.
If you place the parameters into a POST request instead of on the URL,
then the default size limit of that POST request in Solr is 2MB. This
can also be increased.
Thanks,
Shawn

[Salman Ansari]

That is what I have tried. I tried using POST with
application/x-www-form-urlencoded and I got the exception I mentioned. Is
there a way I can get around this exception?

Regards,
Salman

Post by Susheel Kumar
Post is pretty much similar to GET. You can use any REST Client to try.
Same select URL & pass below header and put the queries parameters into body
POST: http://localhost:8983/solr/techproducts/select
Header
==
Content-Type:application/x-www-form-urlencoded
==
q=*:*&rows=2
Thanks,
Susheel

Post by Salman Ansari
Cool. I would give POST a try. Any samples of using Post while passing

the

Post by Salman Ansari
query string values (such as ORing between Solr field values) using
Solr.NET?
Regards,
Salman

Post by Shawn Heisey

Post by Salman Ansari
I am building a long query containing multiple ORs between query

terms. I

Post by Shawn Heisey

Post by Salman Ansari
The remote server returned an error: (414) Request-URI Too Long. Any

idea

Post by Shawn Heisey

Post by Salman Ansari
what is the limit of the URL in Solr? Moreover, as a solution I was
thinking of chunking the query into multiple requests but I was

wondering

Post by Shawn Heisey

Post by Salman Ansari
if anyone has a better approach?

The default HTTP header size limit on most webservers and containers
(including the Jetty that ships with Solr) is 8192 bytes. A typical
request like this will start with "GET " and end with " HTTP/1.1",

which

Post by Salman Ansari

Post by Shawn Heisey
count against that 8192 bytes. The max header size can be increased.
If you place the parameters into a POST request instead of on the URL,
then the default size limit of that POST request in Solr is 2MB. This
can also be increased.
Thanks,
Shawn

[Salman Ansari]

OK then, if there is no way around this problem, can someone tell me the
maximum size a POST body can handle in Solr?

Regards,
Salman

Post by Salman Ansari
That is what I have tried. I tried using POST with
application/x-www-form-urlencoded and I got the exception I mentioned. Is
there a way I can get around this exception?
Regards,
Salman

Post by Susheel Kumar
Post is pretty much similar to GET. You can use any REST Client to try.
Same select URL & pass below header and put the queries parameters into body
POST: http://localhost:8983/solr/techproducts/select
Header
==
Content-Type:application/x-www-form-urlencoded
==
q=*:*&rows=2
Thanks,
Susheel

Post by Salman Ansari
Cool. I would give POST a try. Any samples of using Post while passing

the

Post by Salman Ansari
query string values (such as ORing between Solr field values) using
Solr.NET?
Regards,
Salman

Post by Shawn Heisey

Post by Salman Ansari
I am building a long query containing multiple ORs between query

terms. I

Post by Shawn Heisey

Post by Salman Ansari
The remote server returned an error: (414) Request-URI Too Long. Any

idea

Post by Shawn Heisey

Post by Salman Ansari
what is the limit of the URL in Solr? Moreover, as a solution I was
thinking of chunking the query into multiple requests but I was

wondering

Post by Shawn Heisey

Post by Salman Ansari
if anyone has a better approach?

The default HTTP header size limit on most webservers and containers
(including the Jetty that ships with Solr) is 8192 bytes. A typical
request like this will start with "GET " and end with " HTTP/1.1",

which

Post by Salman Ansari

Post by Shawn Heisey
count against that 8192 bytes. The max header size can be increased.
If you place the parameters into a POST request instead of on the URL,
then the default size limit of that POST request in Solr is 2MB. This
can also be increased.
Thanks,
Shawn

[Shawn Heisey]

Post by Salman Ansari
OK then, if there is no way around this problem, can someone tell me the
maximum size a POST body can handle in Solr?

It is configurable in solrconfig.xml. Look for the
formdataUploadLimitInKB setting in the 5.x configsets. This setting
defaults to 2048, which means 2 megabytes.

Thanks,
Shawn

[Salman Ansari]

It looked like there was another issue with my query. I had too many
boolean operators (I believe maxBooleanClause property in SolrConfig.xml).
I just looped in batch of 1000 to get all the docs. Not sure if there is a
better way of handling this.

Regards,
Salman

Post by Shawn Heisey

Post by Salman Ansari
OK then, if there is no way around this problem, can someone tell me the
maximum size a POST body can handle in Solr?

It is configurable in solrconfig.xml. Look for the
formdataUploadLimitInKB setting in the 5.x configsets. This setting
defaults to 2048, which means 2 megabytes.
Thanks,
Shawn

[Jack Krupansky]

And you're sure that you can't use the terms query parser, which was
explicitly designed for handling a very long list of terms to be implicitly
ORed?

-- Jack Krupansky

Post by Salman Ansari
It looked like there was another issue with my query. I had too many
boolean operators (I believe maxBooleanClause property in SolrConfig.xml).
I just looped in batch of 1000 to get all the docs. Not sure if there is a
better way of handling this.
Regards,
Salman

Post by Shawn Heisey

Post by Salman Ansari
OK then, if there is no way around this problem, can someone tell me

the

Post by Shawn Heisey

Post by Salman Ansari
maximum size a POST body can handle in Solr?

It is configurable in solrconfig.xml. Look for the
formdataUploadLimitInKB setting in the 5.x configsets. This setting
defaults to 2048, which means 2 megabytes.
Thanks,
Shawn

[Maciej Lisiewski]

GET request length is limited by URL length. RFCs defining HTTP recommend
you keep it under 8KB but put no hard limit in place.
Because of no hard limit it boils down to the implementation - both on the
server side and client side:
- On client side Internet Explorer is limited to about 2KB for example
- On server side Jetty (used by Solr) defaults to a 4KB limit

POST should be used for larger requests, but it can sometimes be limited
too - for example Jetty (default container for Solr) defaults to 200.000
bytes max.
To increase that change this:
http://www.eclipse.org/jetty/documentation/current/setting-form-size.html
--
Maciej Lisiewski
[phone]

Post by Jack Krupansky
And you're sure that you can't use the terms query parser, which was
explicitly designed for handling a very long list of terms to be implicitly
ORed?
-- Jack Krupansky

Post by Salman Ansari
It looked like there was another issue with my query. I had too many
boolean operators (I believe maxBooleanClause property in

SolrConfig.xml).

Post by Salman Ansari
I just looped in batch of 1000 to get all the docs. Not sure if there is

a

Post by Salman Ansari
better way of handling this.
Regards,
Salman

Post by Shawn Heisey

Post by Salman Ansari
OK then, if there is no way around this problem, can someone tell me

the

Post by Shawn Heisey

Post by Salman Ansari
maximum size a POST body can handle in Solr?

It is configurable in solrconfig.xml. Look for the
formdataUploadLimitInKB setting in the 5.x configsets. This setting
defaults to 2048, which means 2 megabytes.
Thanks,
Shawn

[Shawn Heisey]

Post by Maciej Lisiewski
GET request length is limited by URL length. RFCs defining HTTP recommend
you keep it under 8KB but put no hard limit in place.
Because of no hard limit it boils down to the implementation - both on the
- On client side Internet Explorer is limited to about 2KB for example
- On server side Jetty (used by Solr) defaults to a 4KB limit
POST should be used for larger requests, but it can sometimes be limited
too - for example Jetty (default container for Solr) defaults to 200.000
bytes max.
http://www.eclipse.org/jetty/documentation/current/setting-form-size.html

Solr controls the max size of the POST request. It defaults to 2MB. It
is changed with the formdataUploadLimitInKB setting in solrconfig.xml.
Changing the size limit in the container config will likely have no effect.

Thanks,
Shawn

[vlspavan]

Hi Salman,

I have the same issue that URI is too long. Did you try implementing the
post method ? If so, can you please post the code from your end on post
requests. This will help greatly.

Thank you
Pavan



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html