Skip to content

v0.3.0

Compare
Choose a tag to compare
@dwhieb dwhieb released this 07 Jul 19:22

This release now treats non-whitespace characters that are adjacent to brackets as part of the same word. For example, [word], will be tokenized as [word], rather than [word] ,, and [word word]=enclitic will be tokenized as [word word]=enclitic rather than [word word] =enclitic.