|
Open Chinese Convert 1.4.2+g71621a0e1.dirty
A project for conversion between Traditional and Simplified Chinese
|
Streaming variant of ConvertWithAmbiguities() with bounded memory. More...
#include <src/ConversionAmbiguities.hpp>
Classes | |
| struct | Chunk |
Public Member Functions | |
| AmbiguityStream (ConverterPtr converter, size_t maxKeepChars=internal::kDefaultStreamKeepChars) | |
| Chunk | ConvertChunk (std::string_view input) |
Appends input and flushes everything but the kept tail. | |
| Chunk | Finish (std::string_view input) |
Appends input and flushes everything, ending the stream. | |
| Chunk | Finish () |
| Flushes everything, ending the stream. | |
| size_t | SourceCount () const |
| Number of distinct sources defined so far. | |
Streaming variant of ConvertWithAmbiguities() with bounded memory.
Mirrors ConverterStream's windowing (a tail of maxKeepChars code points is withheld from each flush so matches never straddle a window boundary) but reports ambiguous spans per flushed chunk. Producer-side state is the pending tail plus one global source-deduplication table whose size is bounded by the dictionaries' one-to-many entries, not by the input length, enabling define-on-first-use record emission: Chunk::newSources lists the sources first seen in this chunk (in global index order), and span sourceIndex values are global stream-wide indexes. Offsets in Chunk::ambiguities are relative to Chunk::output.