Package org.apache.lucene.codecs.lucene104

Class Lucene104ScalarQuantizedVectorsWriter

java.lang.Object

org.apache.lucene.codecs.KnnVectorsWriter

org.apache.lucene.codecs.hnsw.FlatVectorsWriter

org.apache.lucene.codecs.lucene104.Lucene104ScalarQuantizedVectorsWriter

All Implemented Interfaces:

Closeable, AutoCloseable, Accountable

public class Lucene104ScalarQuantizedVectorsWriter
extends FlatVectorsWriter

Writes quantized vector values and metadata to index segments in the format for Lucene 10.4.

WARNING: This API is experimental and might change in incompatible ways in the next release.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.codecs.hnsw.FlatVectorsWriter

FlatVectorsWriter.MergeScorerData

Nested classes/interfaces inherited from class org.apache.lucene.codecs.KnnVectorsWriter

KnnVectorsWriter.MergedVectorValues

Field Summary

Fields inherited from class org.apache.lucene.codecs.hnsw.FlatVectorsWriter

vectorsScorer

Fields inherited from interface org.apache.lucene.util.Accountable

NULL_ACCOUNTABLE

Constructor Summary

Constructors

Constructor	Description
Lucene104ScalarQuantizedVectorsWriter(SegmentWriteState state, QuantizedByteVectorValues.ScalarEncoding encoding, FlatVectorsWriter rawVectorDelegate, Lucene104ScalarQuantizedVectorScorer vectorsScorer)	Sole constructor

Method Summary

Modifier and Type	Method	Description
FlatFieldVectorsWriter<?>	addField(FieldInfo fieldInfo)	Add a new field for indexing
void	close()
void	finish()	Called once at the end before close
void	flush(int maxDoc, Sorter.DocMap sortMap)	Flush all buffered data on disk *
void	mergeOneFlatVectorField(FieldInfo fieldInfo, MergeState mergeState)
FlatVectorsWriter.MergeScorerData	mergeOneFlatVectorFieldForMergeScorer(FieldInfo fieldInfo, MergeState mergeState, IntPredicate needsMergeScorer)	Merges one field and, while writing the merged vectors, optionally prepares what a merge scorer needs, so that the merged float vectors need not be read back.
long	ramBytesUsed()	Returns an estimate of the JVM heap memory used by this object in bytes.

Methods inherited from class org.apache.lucene.codecs.hnsw.FlatVectorsWriter

getFlatVectorScorer, mergeOneField

Methods inherited from class org.apache.lucene.codecs.KnnVectorsWriter

mapOldOrdToNewOrd, merge

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.lucene.util.Accountable

getChildResources

Constructor Details

Lucene104ScalarQuantizedVectorsWriter

public Lucene104ScalarQuantizedVectorsWriter(SegmentWriteState state,
 QuantizedByteVectorValues.ScalarEncoding encoding,
 FlatVectorsWriter rawVectorDelegate,
 Lucene104ScalarQuantizedVectorScorer vectorsScorer)
                                      throws IOException

Sole constructor

Throws:

IOException

Method Details

addField

public FlatFieldVectorsWriter<?> addField(FieldInfo fieldInfo)
                                   throws IOException

Description copied from class: FlatVectorsWriter

Add a new field for indexing

Specified by:

addField in class FlatVectorsWriter

Parameters:

fieldInfo - fieldInfo of the field to add

Returns:

a writer for the field

Throws:

IOException - if an I/O error occurs when adding the field

flush

public void flush(int maxDoc,
 Sorter.DocMap sortMap)
           throws IOException

Description copied from class: KnnVectorsWriter

Flush all buffered data on disk *

Specified by:

flush in class KnnVectorsWriter

Throws:

IOException

finish

public void finish()
            throws IOException

Description copied from class: KnnVectorsWriter

Called once at the end before close

Specified by:

finish in class KnnVectorsWriter

Throws:

IOException

mergeOneFlatVectorField

public void mergeOneFlatVectorField(FieldInfo fieldInfo,
 MergeState mergeState)
                             throws IOException

Specified by:

mergeOneFlatVectorField in class FlatVectorsWriter

Throws:

IOException

mergeOneFlatVectorFieldForMergeScorer

public FlatVectorsWriter.MergeScorerData mergeOneFlatVectorFieldForMergeScorer(FieldInfo fieldInfo,
 MergeState mergeState,
 IntPredicate needsMergeScorer)
                                                                        throws IOException

Merges one field and, while writing the merged vectors, optionally prepares what a merge scorer needs, so that the merged float vectors need not be read back.

A wrapping writer should forward this method along with FlatVectorsWriter.mergeOneFlatVectorField(org.apache.lucene.index.FieldInfo, org.apache.lucene.index.MergeState). Forwarding only the latter stays correct, but the wrapped writer then prepares nothing. Forwarding helps only when the wrapping format's reader over the merged segment is the wrapped writer's own reader. Otherwise FlatVectorsWriter.MergeScorerData.scorerSupplier(org.apache.lucene.codecs.hnsw.FlatVectorsReader) rejects the prepared data and the merge uses its fallback.

HNSW merges call this overload rather than mergeOneFlatVectorField(FieldInfo, MergeState), so subclasses that customize merging must override this method too.

Overrides:

mergeOneFlatVectorFieldForMergeScorer in class FlatVectorsWriter

Parameters:

fieldInfo - field to merge

mergeState - merge state

needsMergeScorer - whether to prepare scorer data for a vector count. The writer applies this predicate to the number of live vectors written after deletions, so it does not prepare data for a merged field that is too small to build a graph.

Returns:

prepared data that the caller must close, or null if none was prepared, which is what the default implementation returns and what any writer returns for a field it cannot prepare for. The caller then builds the scorer itself.

Throws:

IOException - if an I/O error occurs

close

public void close()
           throws IOException

Throws:

IOException

ramBytesUsed

public long ramBytesUsed()

Description copied from interface: Accountable

Returns an estimate of the JVM heap memory used by this object in bytes. The method name uses "ram" for historical reasons; only JVM heap memory should be reported. Off-heap resources such as memory-mapped files or native direct buffers should not be included. Negative values are illegal.