我的简单ANTLR语法在解析时忽略某些无效的标记。

Question

几周前，我问了一个关于我的反ago语法(My simple ANTLR grammar is not working as expected)的问题。自从问了这个问题之后，我做了更多的调查和调试，并且把大部分的问题都解决了。不过，我只剩下一个问题。

我生成的解析器代码不会在处理的文本的一个特定部分中拾取无效的标记。lexer正确地将事物分解为令牌，但解析器在某些情况下不会将无效令牌排除在外。特别是，当无效令牌位于短语"A“和"B”的末尾时，解析器会忽略它--就像令牌甚至不在那里一样。

一些具体的例子：

• "A和B“-完全有效
• "A@和B“-解析器正确地提取无效的@令牌
• "A和@B“-解析器正确地选择无效的@令牌
• "A和B@“--这是一个谜--词法器找到@令牌，解析器忽略它(!)
• "(A和B@)或C“--进一步的神秘--词法器找到@令牌，解析器忽略它(!)

这是我的语法：

grammar QvidianPlaybooks; 
options{ language=CSharp3; output=AST; ASTLabelType = CommonTree; } 

public parse
  :  expression
  ;

LPAREN : '(' ;
RPAREN : ')' ;
ANDOR : 'AND'|'and'|'OR'|'or';
NAME : ('A'..'Z');
WS : ' ' { $channel = Hidden; };
THEREST : .;

// ***************** parser rules:
expression : anexpression EOF!;
anexpression : atom (ANDOR^ atom)*;
atom : NAME | LPAREN! anexpression RPAREN!;

然后处理结果树的代码如下所示：

... from the main program
QvidianPlaybooksLexer lexer = new QvidianPlaybooksLexer(new ANTLRStringStream(src));
QvidianPlaybooksParser parser = new QvidianPlaybooksParser(new CommonTokenStream(lexer));
parser.TreeAdaptor = new CommonTreeAdaptor();

CommonTree tree = (CommonTree)parser.parse().Tree;
ValidateTree(tree, 0, iValidIdentifierCount);

// recursive code that walks the tree
public static RuleLogicValidationResult ValidateTree(ITree Tree, int depth, int conditionCount)
{
    RuleLogicValidationResult rlvr = null;

    if (Tree != null)
    {
        CommonErrorNode commonErrorNode = Tree as CommonErrorNode;
        if (null != commonErrorNode)
        {
            rlvr = new RuleLogicValidationResult();
            rlvr.IsValid = false;
            rlvr.ErrorType = LogicValidationErrorType.Other;
            Console.WriteLine(rlvr.ToString());
        }
        else
        {
            string strTree = Tree.ToString();
            strTree = strTree.Trim();
            strTree = strTree.ToUpper();
            if ((Tree.ChildCount != 0) && (Tree.ChildCount != 2))
            {
                rlvr = new RuleLogicValidationResult();
                rlvr.IsValid = false;
                rlvr.ErrorType = LogicValidationErrorType.Other;
                rlvr.InvalidIdentifier = strTree;
                rlvr.ErrorPosition = 0;
                Console.WriteLine(String.Format("CHILD COUNT of {0} = {1}", strTree, tree.ChildCount));
            }
            // if the current node is valid, then validate the two child nodes
            if (null == rlvr || rlvr.IsValid)
            {
                // output the tree node
                for (int i = 0; i < depth; i++)
                {
                    Console.Write("  ");
                }
                Console.WriteLine(Tree);
                rlvr = ValidateTree(Tree.GetChild(0), depth + 1, conditionCount);
                if (rlvr.IsValid)
                {
                    rlvr = ValidateTree(Tree.GetChild(1), depth + 1, conditionCount);
                }
            }
            else
            {
                Console.WriteLine(rlvr.ToString());
            }
        }
    }
    else
    {
        // this tree is null, return a "it's valid" result
        rlvr = new RuleLogicValidationResult();
        rlvr.ErrorType = LogicValidationErrorType.None;
        rlvr.IsValid = true;
    }

    return rlvr;
}

Answer 1

将EOF添加到开始规则的末尾。:)