We will soon support additional blocks beyond `feature`,
and keeping this refactoring separate from new functionality
makes it easier for code reviewers to follow the changes.
The OpenType Feature File specification does not specify hex numbers
as explicit token type, but they appear several times in the examples
given by the spec.
In Python 3, the method unittest.TestCase.assertRaisesRegexp
has been renamed to "assertRaisesRegex", and the frameworks spits
out deprecation warnings when using the method name from Python 2.
When the lexer encounters the "include" keyword, it now enters
a special mode for scanning file names. After having scanned over
the file name, the lexer goes back to normal. The exact format
of file name strings is not defined by the OpenType feature file
specification, so we accept any character that is not a closing
parenthesis.
This simplifies the implementation of the parser for
OpenType feature files, since it can now just keep
token locations returned by the lexer. Before this
change, the parser had to un-pack the location tuples
and build new tuples that included the file path.