Package org.languagetool

Class AnalyzedSentence

java.lang.Object

org.languagetool.AnalyzedSentence

public final class AnalyzedSentence
extends Object

A sentence that has been tokenized and analyzed.

Field Summary

Fields

Modifier and Type	Field	Description
private final Map<String,List<Integer>>	lemmaOffsets
private final AnalyzedTokenReadings[]	nonBlankPreDisambigTokens
private final AnalyzedTokenReadings[]	nonBlankTokens
private final AnalyzedTokenReadings[]	preDisambigTokens
private String	text
private final Map<String,List<Integer>>	tokenOffsets
private final AnalyzedTokenReadings[]	tokens
private final int[]	whPositions

Constructor Summary

Constructors

Modifier	Constructor	Description
	AnalyzedSentence(AnalyzedTokenReadings[] tokens)	Creates an AnalyzedSentence from the given AnalyzedTokenReadings.
private	AnalyzedSentence(AnalyzedTokenReadings[] tokens, int[] mapping, AnalyzedTokenReadings[] nonBlankTokens, AnalyzedTokenReadings[] nonBlankPreDisambigTokens)
	AnalyzedSentence(AnalyzedTokenReadings[] tokens, AnalyzedTokenReadings[] preDisambigTokens)

Method Summary

Modifier and Type	Method	Description
private String	calcText()
AnalyzedSentence	copy(AnalyzedSentence sentence)	The method copies AnalyzedSentence and returns the copy.
boolean	equals(Object o)
String	getAnnotations()	Get disambiguator actions log.
int	getCorrectedTextLength()	Text length taking position fixes (for removed soft hyphens etc.) into account, so this is _not_ always equal to getText().
List<Integer>	getLemmaOffsets(String token)
Set<String>	getLemmaSet()	Get the lowercase lemmas of this sentence in a set.
private List<AnalyzedTokenReadings>	getNonBlankReadings(AnalyzedTokenReadings[] tokens, int whCounter, int nonWhCounter, int[] mapping)
int	getNonWhitespaceTokenCount()	Get the length of the array returned by getTokensWithoutWhitespace() without additional allocations.
int	getOriginalPosition(int nonWhPosition)	Get a position of a non-whitespace token in the original sentence with whitespace.
AnalyzedTokenReadings[]	getPreDisambigTokens()
AnalyzedTokenReadings[]	getPreDisambigTokensWithoutWhitespace()
String	getText()	Return the original text.
List<Integer>	getTokenOffsets(String token)
AnalyzedTokenReadings[]	getTokens()	Returns the AnalyzedTokenReadings of the analyzed text.
Set<String>	getTokenSet()	Get the lowercase tokens of this sentence in a set.
AnalyzedTokenReadings[]	getTokensWithoutWhitespace()	Returns the AnalyzedTokenReadings of the analyzed text, with whitespace tokens removed but with the artificial SENT_START token included.
int	hashCode()
private static Map<String,List<Integer>>	indexLemmas(AnalyzedTokenReadings[] tokens)
private static Map<String,List<Integer>>	indexTokens(AnalyzedTokenReadings[] tokens)
private static Map<String,List<Integer>>	makeUnmodifiable(Map<String,List<Integer>> result)
String	toShortString(String readingDelimiter)	Return string representation without chunk information.
String	toString()
String	toString(String readingDelimiter)	Return string representation with chunk information.
private String	toString(String readingDelimiter, boolean includeChunks)
(package private) String	toTextString()	Return string representation without any analysis information, just the original text.

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Details

tokens

private final AnalyzedTokenReadings[] tokens

preDisambigTokens

private final AnalyzedTokenReadings[] preDisambigTokens

nonBlankTokens

private final AnalyzedTokenReadings[] nonBlankTokens

nonBlankPreDisambigTokens

private final AnalyzedTokenReadings[] nonBlankPreDisambigTokens

whPositions

private final int[] whPositions

tokenOffsets

private final Map<String,List<Integer>> tokenOffsets

lemmaOffsets

private final Map<String,List<Integer>> lemmaOffsets

text

private volatile String text

Constructor Details

AnalyzedSentence

public AnalyzedSentence(AnalyzedTokenReadings[] tokens)

Creates an AnalyzedSentence from the given AnalyzedTokenReadings. Whitespace is also a token.

AnalyzedSentence

public AnalyzedSentence(AnalyzedTokenReadings[] tokens,
 AnalyzedTokenReadings[] preDisambigTokens)

AnalyzedSentence

private AnalyzedSentence(AnalyzedTokenReadings[] tokens,
 int[] mapping,
 AnalyzedTokenReadings[] nonBlankTokens,
 AnalyzedTokenReadings[] nonBlankPreDisambigTokens)

Method Details

getNonBlankReadings

@NotNull
private List<AnalyzedTokenReadings> getNonBlankReadings(AnalyzedTokenReadings[] tokens,
 int whCounter,
 int nonWhCounter,
 int[] mapping)

indexTokens

private static Map<String,List<Integer>> indexTokens(AnalyzedTokenReadings[] tokens)

indexLemmas

private static Map<String,List<Integer>> indexLemmas(AnalyzedTokenReadings[] tokens)

makeUnmodifiable

private static Map<String,List<Integer>> makeUnmodifiable(Map<String,List<Integer>> result)

copy

public AnalyzedSentence copy(AnalyzedSentence sentence)

The method copies AnalyzedSentence and returns the copy. Useful for performing local immunization (for example).

Parameters:

sentence - AnalyzedSentence to be copied

Returns:

a new object which is a copy

Since:

2.5

getTokens

public AnalyzedTokenReadings[] getTokens()

Returns the AnalyzedTokenReadings of the analyzed text. Whitespace is also a token.

getPreDisambigTokens

public AnalyzedTokenReadings[] getPreDisambigTokens()

Since:

4.5

getTokensWithoutWhitespace

public AnalyzedTokenReadings[] getTokensWithoutWhitespace()

Returns the AnalyzedTokenReadings of the analyzed text, with whitespace tokens removed but with the artificial SENT_START token included.

getNonWhitespaceTokenCount

@Internal
public int getNonWhitespaceTokenCount()

Get the length of the array returned by getTokensWithoutWhitespace() without additional allocations.

getPreDisambigTokensWithoutWhitespace

public AnalyzedTokenReadings[] getPreDisambigTokensWithoutWhitespace()

Since:

4.5

getOriginalPosition

public int getOriginalPosition(int nonWhPosition)

Get a position of a non-whitespace token in the original sentence with whitespace.

Parameters:

nonWhPosition - position of a non-whitespace token

Returns:

position in the original sentence.

toString

public String toString()

Overrides:

toString in class Object

toShortString

public String toShortString(String readingDelimiter)

Return string representation without chunk information.

Since:

2.3

getText

public String getText()

Return the original text.

Since:

2.7

calcText

private String calcText()

getCorrectedTextLength

public int getCorrectedTextLength()

Text length taking position fixes (for removed soft hyphens etc.) into account, so this is _not_ always equal to getText().

Since:

5.1

toTextString

String toTextString()

Return string representation without any analysis information, just the original text.

Since:

2.6

toString

public String toString(String readingDelimiter)

Return string representation with chunk information.

toString

private String toString(String readingDelimiter,
 boolean includeChunks)

getAnnotations

public String getAnnotations()

Get disambiguator actions log.

getTokenSet

public Set<String> getTokenSet()

Get the lowercase tokens of this sentence in a set. Used internally for performance optimization.

Since:

2.4

getLemmaSet

public Set<String> getLemmaSet()

Get the lowercase lemmas of this sentence in a set. Used internally for performance optimization.

Since:

2.5

getTokenOffsets

@Nullable
@Internal
public List<Integer> getTokenOffsets(String token)

Returns:

all offsets in getTokensWithoutWhitespace() where tokens with the given text occur (case-insensitive), or null if there are no such occurrences

Since:

5.3

getLemmaOffsets

@Nullable
@Internal
public List<Integer> getLemmaOffsets(String token)

Returns:

all offsets in getTokensWithoutWhitespace() where tokens with the given lemma occur (case-insensitive), or null if there are no such occurrences

Since:

5.3

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object