Create a lexer by defining the regex that matches tokens and the regex that matches delimiter.
Create a lexer by defining the regex that matches tokens and the regex that matches delimiter. Note: singleCharTokenRegex and singleCharDelimRegex must be mutually exclusive. If they match the same characters the results of the lexer are undefined.
a regex that matches a single char of a token
a regex that matches a single char of delimiter
a lexer based on the supplied token and delimiter regexes
Create a lexer by defining one or more delimiters
Create a lexer by defining one or more delimiters
a sequence of delimiter characters
a lexer that separates token and delimiter as defined by the supplied delimiter chars