Package org.apache.lucene.search.similarities

Class StableTflSimilarity

java.lang.Object
org.apache.lucene.search.similarities.Similarity
org.apache.lucene.search.similarities.StableTflSimilarity
public class StableTflSimilarity extends Similarity
StableTfl Similarity. A similarity algorithm that estimates term rarity based on term length and document length, rather than relying on corpus-level statistics like document frequency.

Inspired by BM25, this similarity model replaces IDF with a synthetic function that increases with term length (as a proxy for rarity) and adjusts based on document length.

Scoring formula: 

 tf = freq / (freq + k1)
 p  = 1 - (1 - m * 2 ^ (-c * |t|)) ^ |d|
 tr = log(1 + (1 - p + 0.05) / (p + 0.05))
 score = boost * tr * tf

This similarity is parameterized by:

  • k1 - saturation parameter for term frequency
  • c - decay constant controlling how term length impacts rarity

The constant m is fixed internally and controls the base term-match probability.

WARNING: This API is experimental and might change in incompatible ways in the next release.

  • Field Details

    • DEFAULT_K1

      public static final float DEFAULT_K1
      Default k1 value used in Lucene's implementation of BM25.
      See Also:

    • DEFAULT_C

      public static final float DEFAULT_C
      Empirical value of c derived from modeling term frequency in English corpuses.
      See Also:

  • Constructor Details

    • StableTflSimilarity

      public StableTflSimilarity(float k1, float c)
      StableTFL with the supplied parameter values.
      Parameters:
      k1 - Controls non-linear term frequency normalization (saturation).
      c - Controls how much term length affects term rarity (decay constant).
      Throws:
      IllegalArgumentException - if k1 is infinite or negative
      IllegalArgumentException - if c is infinite or negative

    • StableTflSimilarity

      public StableTflSimilarity()
      StableTFL with these default values:
      • k1 = 1.2
      • c = 0.917

  • Method Details

    • scorer

      public final Similarity.SimScorer scorer(float boost, CollectionStatistics fieldStats, TermStatistics... termStats)
      Description copied from class: Similarity
      Compute any collection-level weight (e.g. IDF, average document length, etc) needed for scoring a query.
      Specified by:
      scorer in class Similarity
      Parameters:
      boost - a multiplicative factor to apply to the produces scores
      fieldStats - collection-level statistics, such as the number of tokens in the collection.
      termStats - term-level statistics, such as the document frequency of a term across the collection.
      Returns:
      SimWeight object with the information this Similarity needs to score a query.

    • getK1

      public final float getK1()
      Returns the k1 parameter.
      See Also:

    • getC

      public final float getC()
      Returns the c parameter.
      See Also:

    • toString

      public String toString()
      Overrides:
      toString in class Object