Commit Graph

153 Commits

Author SHA1 Message Date
Max Brunsfeld 6f81608535 Always lex braces and brackets as separate tokens
The lexer needs to always return braces and brackets separately so that
the parser can decide if they are part of some construct like an array
subscript or a variable expansion. This means that there was no point
in moving bare-word tokenization into the external scanner. I've moved
it back into the normal scanner.

The tricky part is how to deal with the separate '}' and ']' tokens
in the case where they are *not* part of a subscript or an expansion.
For example, in code like `echo {hi}`, the syntax tree should still
clearly indicate that only *one* argument is being passed to echo.
For now, we achieve this by grouping the '{', hi, and '}' tokens into
a single `concatenation` node, which is a bit odd, but it's the best
we can do.
2018-02-28 14:33:50 -08:00
Max Brunsfeld c34619a1c4 Don't parse square bracket commands as special syntax 2018-02-28 14:32:58 -08:00
Mads Hartmann c410548caf Handle escape sequences in strings (#7)
This should fix #6
2018-02-28 12:15:05 -08:00
Max Brunsfeld 675a581839 Allow special characters in bare word arguments
I've moved tokenization of bare words into the external scanner. This
way we can keep the grammar simple, but support some fancy rules that
I've inferred from experimenting with bash:

- Only allow '}' inside of a bare word if '}' isn't a valid lookahead
token (i.e. we're not inside of a variable expansion).
- Only allow ']' at the *start* of a bare word if neither ']' nor ']]'
are valid lookahead tokens (i.e. we're not inside of a square bracket
command or an array subscript).
- Parentheses seem to never be allowed in bare words. You have to quote
them.

For alphanumeric words, I fall through to the normal scanner so that it
can continue to distinguish reserved words from other words.

Fixes #5
2018-02-27 16:55:20 -08:00
Max Brunsfeld 8a0a4a8501 Parse for loop identifier as a variable name
Also remove some duplication in expression and concatenation
2018-02-27 13:46:36 -08:00
Max Brunsfeld 7e73a575e9 Parse function definitions more strictly 2018-02-27 13:34:45 -08:00
Mads Hartmann 8ca2ecd52d Add support for exporting environment variables (#4)
* Add support for exporting environment variables

* Support multiple exports in one command

Also improves the structure of the AST so the variable name and the
assigned expression aren't free-floating

* Allow for multiple declarations in local declarations as well

Simplify the generated nodes.

* Commit generated code

* Consolidate declarations into a single declaration_command

This also adds support for declare, typeset, and readonly

* Rename environment_variable_assignment to variable_assignment
2018-02-27 13:22:28 -08:00
Max Brunsfeld 1efe2e6f7a
Merge pull request #2 from mads-hartmann/support-local-variable-declarations
Support local variable declarations
2018-02-24 20:25:56 -08:00
Mads Hartmann 0ed39445d3 Support local variable declarations 2018-02-25 00:02:24 +01:00
Mads Hartmann 741cfce4b4 Handle empty default value during variable expansion (#1)
* Handle empty default value during variable expansion

* Make expression after colon optional

As suggested by @maxbrunsfeld
2018-02-24 13:40:43 -08:00
Max Brunsfeld bda693990e 0.5.0 2018-02-12 16:54:53 -08:00
Max Brunsfeld 31c17c066b Regenerate with latest tree-sitter 2018-02-12 16:53:11 -08:00
Max Brunsfeld 0259c5bce2 Guard against buffer overflow when serializing scanner
/cc philipturnbull
2018-01-19 09:52:45 -08:00
Max Brunsfeld 700d95cc95 Add tree-sitter's test script as an example file 2017-12-26 14:56:51 -08:00
Max Brunsfeld dd247cad99 0.4.2 2017-12-26 14:55:44 -08:00
Max Brunsfeld 145d4e86a4 Add missing array syntax 2017-12-26 14:55:37 -08:00
Max Brunsfeld 2a034b5252 0.4.1 2017-12-26 14:27:16 -08:00
Max Brunsfeld 52d54bb173 Add missing function definition forms 2017-12-26 14:27:05 -08:00
Max Brunsfeld 75833b90ea 0.4.0 2017-12-21 17:13:18 -08:00
Max Brunsfeld d85b6287e6 Upgrade tree-sitter for new lexer API 2017-12-21 17:13:14 -08:00
Max Brunsfeld 811de9534d 0.3.0 2017-09-18 09:32:06 -07:00
Max Brunsfeld e51dfb661a Regenerate with latest tree-sitter 2017-09-18 09:32:01 -07:00
Max Brunsfeld d05776b86d 0.2.2 2017-08-02 16:56:54 -07:00
Max Brunsfeld 8f67869b16 Use string::copy rather than memcpy 2017-08-01 10:26:42 -07:00
Max Brunsfeld ccc07eab4b Use new alias API 2017-08-01 10:17:28 -07:00
Max Brunsfeld 4340002e38 Implement external scanner serialization 2017-08-01 10:12:16 -07:00
Max Brunsfeld f49479deca ⬆️ tree-sitter-cli 2017-07-17 11:18:15 -07:00
Max Brunsfeld dce4ee6937 Handle words concatenated with strings and expansions 2017-07-17 10:47:34 -07:00
Max Brunsfeld 403361626d Add length operator in variable expansions 2017-07-15 23:12:22 -07:00
Max Brunsfeld 3abfbbd7bd Handle strings containing command substitutions 2017-07-15 22:41:56 -07:00
Max Brunsfeld e9afbb4381 Name inlined rules as if they were hidden 2017-07-15 22:23:38 -07:00
Max Brunsfeld 5081293f65 Add process substitutions 2017-07-15 22:22:38 -07:00
Max Brunsfeld 1b5e1557f2 Reorganize, rename a bunch of things 2017-07-15 22:13:55 -07:00
Max Brunsfeld 7424d9fc9e Use a newer clang on travis 2017-07-14 17:53:04 -07:00
Max Brunsfeld f73de068e7 Add for statements 2017-07-14 17:51:06 -07:00
Max Brunsfeld 38d22aa8c3 Add examples directory w/ one real shell script 2017-07-14 17:41:51 -07:00
Max Brunsfeld 66693d7575 Add empty environment variable values 2017-07-14 17:41:14 -07:00
Max Brunsfeld 6d341b8314 Add function definitions 2017-07-14 17:35:51 -07:00
Max Brunsfeld 5446533c20 Add quoted command names, subshells 2017-07-14 17:32:55 -07:00
Max Brunsfeld a0406c8906 Fix distinction between numeric args and redirect file descriptors 2017-07-14 17:14:23 -07:00
Max Brunsfeld 86e67f9403 Add quoted arguments, bracket expressions 2017-07-14 16:47:25 -07:00
Max Brunsfeld 5b26947ca8 Add case statements 2017-07-14 16:29:28 -07:00
Max Brunsfeld 9f38e36bc3 Add if statements 2017-07-14 16:18:46 -07:00
Max Brunsfeld cce4a65d33 Add while statements 2017-07-14 16:11:35 -07:00
Max Brunsfeld dc84753599 Add travis config file 2017-07-14 14:48:51 -07:00
Max Brunsfeld 6861ab03bf Add license and readme 2017-07-14 14:46:24 -07:00
Max Brunsfeld d68e65d8f9 Add escaped newlines 2017-07-14 14:39:28 -07:00
Max Brunsfeld 5a9bfe1571 Add comments 2017-07-14 14:34:49 -07:00
Max Brunsfeld a46748839e Add variable expansion inside of heredocs 2017-07-14 14:27:13 -07:00
Max Brunsfeld 6be8857926 Start work on heredocs 2017-07-14 13:54:05 -07:00