I've moved tokenization of bare words into the external scanner. This
way we can keep the grammar simple, but support some fancy rules that
I've inferred from experimenting with bash:
- Only allow '}' inside of a bare word if '}' isn't a valid lookahead
token (i.e. we're not inside of a variable expansion).
- Only allow ']' at the *start* of a bare word if neither ']' nor ']]'
are valid lookahead tokens (i.e. we're not inside of a square bracket
command or an array subscript).
- Parentheses seem to never be allowed in bare words. You have to quote
them.
For alphanumeric words, I fall through to the normal scanner so that it
can continue to distinguish reserved words from other words.
Fixes#5
675a581839
