Stemming not working with wildcard search

[Geepalem]

Hi,

I have added SnowballPorterFilterFactory filter to field type to make
singular and plural search terms return same results.

So below queries (double quotes around search term) returning similar
results which is fine.

http://localhost:8080/solr/master/select?q=page_title_t:"product*"
http://localhost:8080/solr/master/select?q=page_title_t:"products*"

But when I have analyzed results, in both result sets, documents which dont
start with words "Product" or "products" didnt come though there are few
documents available.

So I have added * as prefix and suffix to search term without double quotes
to do wildcard search.

http://localhost:8080/solr/master/select?q=page_title_t:*product*
http://localhost:8080/solr/master/select?q=page_title_t:*products*

Now, stemming is not working as above second query is not returning similar
results as query 1.

If double quotes are added around search term then its returning similar
results but results are not as expected. With double quotes it wont return
results like "Old products", "New products", "Cool Product".
It will only return results with the values like "Product 1", "Product
2","Products of USA".

Please suggest or guide how to make stemming work with wildcard search.


Appreciate immediate response!!

Thanks,
G. Naresh Kumar





--
View this message in context: http://lucene.472066.n3.nabble.com/Stemming-not-working-with-wildcard-search-tp4133382.html
Sent from the Solr - User mailing list archive at Nabble.com.

[Geepalem]

Can some one please help me with this as I am struck with this issue..



--
View this message in context: http://lucene.472066.n3.nabble.com/Stemming-not-working-with-wildcard-search-tp4133382p4133477.html
Sent from the Solr - User mailing list archive at Nabble.com.

[Jack Krupansky]

Wildcards and stemming are incompatible at query time - you need to manually
stem the term before applying your wildcard.

Wildcards are not supported in quoted phrases. They will be treated as
punctuation, and ignored by the standard tokenizer or the word delimiter
filter.

-- Jack Krupansky

-----Original Message-----
From: Geepalem
Sent: Sunday, April 27, 2014 3:13 PM
To: solr-***@lucene.apache.org
Subject: Stemming not working with wildcard search

Hi,

I have added SnowballPorterFilterFactory filter to field type to make
singular and plural search terms return same results.

So below queries (double quotes around search term) returning similar
results which is fine.

http://localhost:8080/solr/master/select?q=page_title_t:"product*"
http://localhost:8080/solr/master/select?q=page_title_t:"products*"

But when I have analyzed results, in both result sets, documents which dont
start with words "Product" or "products" didnt come though there are few
documents available.

So I have added * as prefix and suffix to search term without double quotes
to do wildcard search.

http://localhost:8080/solr/master/select?q=page_title_t:*product*
http://localhost:8080/solr/master/select?q=page_title_t:*products*

Now, stemming is not working as above second query is not returning similar
results as query 1.

If double quotes are added around search term then its returning similar
results but results are not as expected. With double quotes it wont return
results like "Old products", "New products", "Cool Product".
It will only return results with the values like "Product 1", "Product
2","Products of USA".

Please suggest or guide how to make stemming work with wildcard search.


Appreciate immediate response!!

Thanks,
G. Naresh Kumar





--
View this message in context:
http://lucene.472066.n3.nabble.com/Stemming-not-working-with-wildcard-search-tp4133382.html
Sent from the Solr - User mailing list archive at Nabble.com.

[Ahmet Arslan]

Hi Naresh,

quotes are only meaningful when there are two or more terms. don't use quotes for products* and product*.

As regarding stemming and wildcards, use following chain, and your wildcard searches will be happier.

<filter class="solr.KeywordRepeatFilterFactory"/>
<filter class="solr.PorterStemFilterFactory"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>

Ahmet


On Monday, April 28, 2014 5:41 PM, Jack Krupansky <***@basetechnology.com> wrote:
Wildcards and stemming are incompatible at query time - you need to manually
stem the term before applying your wildcard.

Wildcards are not supported in quoted phrases. They will be treated as
punctuation, and ignored by the standard tokenizer or the word delimiter
filter.

-- Jack Krupansky

-----Original Message-----
From: Geepalem
Sent: Sunday, April 27, 2014 3:13 PM
To: solr-***@lucene.apache.org
Subject: Stemming not working with wildcard search

Hi,

I have added  SnowballPorterFilterFactory filter to field type to make
singular and plural search terms return same results.

So below queries (double quotes around search term) returning similar
results which is fine.

http://localhost:8080/solr/master/select?q=page_title_t:"product*"
http://localhost:8080/solr/master/select?q=page_title_t:"products*"

But when I have analyzed results, in both result sets, documents which dont
start with words "Product" or "products" didnt come though there are few
documents available.

So I have added * as prefix and suffix to search term without double quotes
to do wildcard search.

http://localhost:8080/solr/master/select?q=page_title_t:*product*
http://localhost:8080/solr/master/select?q=page_title_t:*products*

Now, stemming is not working as above second query is not returning similar
results as query 1.

If double quotes are added around search term then its returning similar
results but results are not as expected. With double quotes it wont return
results like "Old products", "New products", "Cool Product".
It will only return results with the values like "Product 1", "Product
2","Products of USA".

Please suggest or guide how to make stemming work with wildcard search.


Appreciate immediate response!!

Thanks,
G. Naresh Kumar





--
View this message in context:
http://lucene.472066.n3.nabble.com/Stemming-not-working-with-wildcard-search-tp4133382.html
Sent from the Solr - User mailing list archive at Nabble.com.

[Geepalem]

Hi Ahmet,

Thanks for your prompt response!

I have added filters which you have specified but still its not working.
Below is field Query Analyzer

<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />

<filter class="solr.KeywordRepeatFilterFactory"/>
<filter class="solr.PorterStemFilterFactory"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>

http://localhost:8080/solr/master/select?q=page_title_t:*products*
http://localhost:8080/solr/master/select?q=page_title_t:*product*


Please let me know if I am doing anything wrong.

Thanks,
G. Naresh Kumar



--
View this message in context: http://lucene.472066.n3.nabble.com/Stemming-not-working-with-wildcard-search-tp4133382p4133556.html
Sent from the Solr - User mailing list archive at Nabble.com.

[Geepalem]

Can someone help me out with this issue?



--
View this message in context: http://lucene.472066.n3.nabble.com/Stemming-not-working-with-wildcard-search-tp4133382p4133769.html
Sent from the Solr - User mailing list archive at Nabble.com.

[Erick Erickson]

Did you re-index? And what do you get when adding &debug=query? That
should show you the parsed query. Have you looked at the results of
the admin/analysis page? That tool is invaluable for seeing what the
actual transformations are.

Best,
Erick

Post by Geepalem
Hi Ahmet,
Thanks for your prompt response!
I have added filters which you have specified but still its not working.
Below is field Query Analyzer
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.KeywordRepeatFilterFactory"/>
<filter class="solr.PorterStemFilterFactory"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
http://localhost:8080/solr/master/select?q=page_title_t:*products*
http://localhost:8080/solr/master/select?q=page_title_t:*product*
Please let me know if I am doing anything wrong.
Thanks,
G. Naresh Kumar
--
View this message in context: http://lucene.472066.n3.nabble.com/Stemming-not-working-with-wildcard-search-tp4133382p4133556.html
Sent from the Solr - User mailing list archive at Nabble.com.