上下文相关词法分析器

Question

我在bash的parse.y中看到了以下内容。这意味着词法分析将依赖于上下文。如何使用flex来做这样的上下文依赖分析？这种依赖于上下文的需求会让flex代码变得太混乱吗？谢谢。

http://git.savannah.gnu.org/cgit/bash.git/tree/parse.y#n3006

/* Handle special cases of token recognition:
  IN is recognized if the last token was WORD and the token
  before that was FOR or CASE or SELECT.

  DO is recognized if the last token was WORD and the token
  before that was FOR or SELECT.

  ESAC is recognized if the last token caused `esacs_needed_count'
  to be set

  `{' is recognized if the last token as WORD and the token
  before that was FUNCTION, or if we just parsed an arithmetic
  `for' command.

  `}' is recognized if there is an unclosed `{' present.

  `-p' is returned as TIMEOPT if the last read token was TIME.
  `--' is returned as TIMEIGN if the last read token was TIMEOPT.

  ']]' is returned as COND_END if the parser is currently parsing
  a conditional expression ((parser_state & PST_CONDEXPR) != 0)

  `time' is returned as TIME if and only if it is immediately
  preceded by one of `;', `\n', `||', `&&', or `&'.
*/

Answer 1

(F)lex提供start conditions以允许上下文相关的词法分析。

如果您避免在词法扫描器中将解析逻辑复制为手写状态机，那么启动条件肯定可以简化上下文相关扫描器的实现。

对于条件识别关键字的特定应用--通常称为“半保留字”--依赖于上下文的词法分析通常不是最佳解决方案。相反，请考虑编写扫描器以始终识别关键字，然后在语法中添加规则，以便在关键字不可能的上下文中将单词视为标识符。有关示例，请参阅this answer。