在 lucene 中使用命中荧光笔

本文介绍了在 lucene 中使用命中荧光笔的处理方法,对大家解决问题具有一定的参考价值

问题描述

我有两个关于 apache lucene 提供的命中荧光笔的问题:

I have two questions regarding hit highlighter provided with apache lucene:

  1. 参见 这个函数你能解释一下令牌流参数的使用吗?

  1. see this function could you explain the use of token stream parameter.

我有几个包含许多字段的大型 lucene 文档,每个字段中都有一些字符串.现在我找到了与特定查询最相关的文档.现在找到了这个文档,因为查询中的几个单词可能与文档中的单词匹配.我想找出查询中的哪些单词导致了这种情况.所以为此我打算使用 Lucene Hit Highlighter.示例:如果查询是skin doctor delhi"并且标题为dermatologist"的文档包含skin"和doctor"这两个词,那么在点击突出显示后,我应该能够从查询中分离出skin"和doctor".几个星期以来,我一直在尝试为此编写代码.无法得到我想要的.你能帮帮我吗?

I have several large lucene document containing many fields and each field has some strings in it. Now I have found the most relevant document for a particular query. Now this document was found because several words in the query might have matched with the words in the document. I want to find out what words in the query caused this. So for this I plan to use Lucene Hit Highlighter. Example: if the query is "skin doctor delhi" and the document titled "dermatologist" contains the words "skin" and "doctor" then after hit highlighting i should be able to separate out "skin" and "doctor" from the query. I have been trying to write the code for this for several weeks now. Not able to get what i want. Could you help me please?

提前致谢.

更新:

当前方法:我创建了一个包含文档中所有单词的查询.

Current Approach: I create a query containing all the words in the document.

Field[] field = doc.getFields("description");
String desc = "";
for (int j = 0; j < field.length; ++j) {
     desc += field[j].stringValue() + " ";
}

Query q = qp.parse(desc);
QueryScorer scorer = new QueryScorer(q, reader, "description");
Highlighter highlighter = new Highlighter(scorer);

String fragment = highlighter.getBestFragment(analyzer, "description", text);

它适用于小文档,但不适用于大文档.得到如下stacktrace.

It works for small documents but does not work for large documents. The following stacktrace is obtained.

    org.apache.lucene.search.BooleanQuery$TooManyClauses: maxClauseCount is set to 1024
    at org.apache.lucene.search.BooleanQuery.add(BooleanQuery.java:152)
    at org.apache.lucene.queryParser.QueryParser.getBooleanQuery(QueryParser.java:891)
    at org.apache.lucene.queryParser.QueryParser.getBooleanQuery(QueryParser.java:866)
    at org.apache.lucene.queryParser.QueryParser.Query(QueryParser.java:1213)
    at org.apache.lucene.queryParser.QueryParser.TopLevelQuery(QueryParser.java:1167)
    at org.apache.lucene.queryParser.QueryParser.parse(QueryParser.java:182)

很明显,这种方法对于大文档是不合理的.应该怎么做才能纠正这个问题?

It is obvious that the approach is unreasonable for large documents. What should be done to correct this?

顺便说一句,我正在使用 FuzzyQuery 匹配.

BTW I am using FuzzyQuery matching.

推荐答案

添加了一些关于 explain() 的细节.

added some details about explain().

一些一般介绍:Lucene Highlighter 旨在从命中文档中查找文本片段,并突出显示与查询匹配的标记.

Some general introduction: The Lucene Highlighter is meant to find text snippets from a hit document, and to highlight tokens matching the query.

  1. 因此,TokenStream 参数用于将命中文本分解为标记.然后,荧光笔的评分器对每个标记进行评分,以便对片段进行评分并选择要突出显示的片段和标记.
  2. 我相信你做错了.如果您只想了解文档中匹配了哪些查询词,则应使用 explain() 方法.基本上,在您实例化搜索器后,使用:
  1. Therefore, The TokenStream parameter is used to break the hit text into tokens. The highlighter's scorer then scores each token, in order to score fragments and choose snippets and tokens to be highlighted.
  2. I believe you are doing it wrong. If all you want to do is understand which query terms were matched in the document, you should use the explain() method. Basically, after you have instantiated a searcher, use:

解释 expl = searcher.explain(query, docId);

String asText = expl.toString();

String asHtml = expl.toHtml();

docId 是搜索结果中的原始文档 ID.

docId is the raw document id from the search results.

仅当您确实需要片段和/或亮点时,才应使用荧光笔.如果您仍想使用荧光笔,请遵循 Nicholas Hrychan 的建议.不过要当心,因为他描述了 Lucene 2.4.1 API - 如果您使用更高级的版本,您应该在他说SpanScorer"的地方使用QueryScorer".

Only if you do need the snippets and/or highlights, you should use the Highlighter. If you still want to use the highlighter, follow Nicholas Hrychan's advice. Beware, though, as he describes the Lucene 2.4.1 API - If you use a more advanced version, you should use "QueryScorer" where he says "SpanScorer" .

这篇关于在 lucene 中使用命中荧光笔的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,WP2

admin_action_{$_REQUEST[‘action’]}

do_action( "admin_action_{$_REQUEST[‘action’]}" )动作钩子::在发送“Action”请求变量时激发。Action Hook: Fires when an ‘action’ request variable is sent.目录锚点:#说明#源码说明(Description)钩子名称的动态部分$_REQUEST['action']引用从GET或POST请求派生的操作。源码(Source)更新版本源码位置使用被使用2.6.0 wp-admin/admin.php:...

日期:2020-09-02 17:44:16 浏览:1127

admin_footer-{$GLOBALS[‘hook_suffix’]}

do_action( "admin_footer-{$GLOBALS[‘hook_suffix’]}", string $hook_suffix )操作挂钩:在默认页脚脚本之后打印脚本或数据。Action Hook: Print scripts or data after the default footer scripts.目录锚点:#说明#参数#源码说明(Description)钩子名的动态部分,$GLOBALS['hook_suffix']引用当前页的全局钩子后缀。参数(Parameters)参数类...

日期:2020-09-02 17:44:20 浏览:1032

customize_save_{$this->id_data[‘base’]}

do_action( "customize_save_{$this-&gt;id_data[‘base’]}", WP_Customize_Setting $this )动作钩子::在调用WP_Customize_Setting::save()方法时激发。Action Hook: Fires when the WP_Customize_Setting::save() method is called.目录锚点:#说明#参数#源码说明(Description)钩子名称的动态部分,$this->id_data...

日期:2020-08-15 15:47:24 浏览:775

customize_value_{$this->id_data[‘base’]}

apply_filters( "customize_value_{$this-&gt;id_data[‘base’]}", mixed $default )过滤器::过滤未作为主题模式或选项处理的自定义设置值。Filter Hook: Filter a Customize setting value not handled as a theme_mod or option.目录锚点:#说明#参数#源码说明(Description)钩子名称的动态部分,$this->id_date['base'],指的是设置...

日期:2020-08-15 15:47:24 浏览:866

get_comment_author_url

过滤钩子:过滤评论作者的URL。Filter Hook: Filters the comment author’s URL.目录锚点:#源码源码(Source)更新版本源码位置使用被使用 wp-includes/comment-template.php:32610...

日期:2020-08-10 23:06:14 浏览:903

network_admin_edit_{$_GET[‘action’]}

do_action( "network_admin_edit_{$_GET[‘action’]}" )操作挂钩:启动请求的处理程序操作。Action Hook: Fires the requested handler action.目录锚点:#说明#源码说明(Description)钩子名称的动态部分$u GET['action']引用请求的操作的名称。源码(Source)更新版本源码位置使用被使用3.1.0 wp-admin/network/edit.php:3600...

日期:2020-08-02 09:56:09 浏览:848

network_sites_updated_message_{$_GET[‘updated’]}

apply_filters( "network_sites_updated_message_{$_GET[‘updated’]}", string $msg )筛选器挂钩:在网络管理中筛选特定的非默认站点更新消息。Filter Hook: Filters a specific, non-default site-updated message in the Network admin.目录锚点:#说明#参数#源码说明(Description)钩子名称的动态部分$_GET['updated']引用了非默认的...

日期:2020-08-02 09:56:03 浏览:834

pre_wp_is_site_initialized

过滤器::过滤在访问数据库之前是否初始化站点的检查。Filter Hook: Filters the check for whether a site is initialized before the database is accessed.目录锚点:#源码源码(Source)更新版本源码位置使用被使用 wp-includes/ms-site.php:93910...

日期:2020-07-29 10:15:38 浏览:809

WordPress 的SEO 教学:如何在网站中加入关键字(Meta Keywords)与Meta 描述(Meta Description)?

你想在WordPress 中添加关键字和meta 描述吗?关键字和meta 描述使你能够提高网站的SEO。在本文中,我们将向你展示如何在WordPress 中正确添加关键字和meta 描述。为什么要在WordPress 中添加关键字和Meta 描述?关键字和说明让搜寻引擎更了解您的帖子和页面的内容。关键词是人们寻找您发布的内容时,可能会搜索的重要词语或片语。而Meta Description则是对你的页面和文章的简要描述。如果你想要了解更多关于中继标签的资讯,可以参考Google的说明。Meta 关键字和描...

日期:2020-10-03 21:18:25 浏览:1619

谷歌的SEO是什么

SEO (Search Engine Optimization)中文是搜寻引擎最佳化,意思近于「关键字自然排序」、「网站排名优化」。简言之,SEO是以搜索引擎(如Google、Bing)为曝光媒体的行销手法。例如搜寻「wordpress教学」,会看到本站的「WordPress教学:12个课程…」排行Google第一:关键字:wordpress教学、wordpress课程…若搜寻「网站架设」,则会看到另一个网页排名第1:关键字:网站架设、架站…以上两个网页,每月从搜寻引擎导入自然流量,达2万4千:每月「有机搜...

日期:2020-10-30 17:23:57 浏览:1262