StandardTokenizer Fields |
The StandardTokenizer type exposes the following members.
Fields| Name | Description | |
|---|---|---|
  | ACRONYM | |
| ACRONYM_DEP | Obsolete. |
| ALPHANUM | |
| APOSTROPHE | |
| CJ | |
| COMPANY | |
| ||
| HOST | |
| NUM | |
| TOKEN_TYPES | String token types that correspond to token type int constants |
See Also