如何为shlex.split使用特定的停止字符？

Question

如何告诉shlex，如果找到了;字符，那么就不要再分割任何东西了？

示例：

shlex.split("""hello "column number 2" foo ; bar baz""")  

应给予

["hello", "column number 2", "foo", "; bar baz"]

而不是["hello", "column number 2", "foo", ";", "bar", "baz"]。

更普遍地说，是否有一种方法可以用shlex定义“注释”分隔符？即

shlex.split("""hello "column number 2" foo ;this is a comment; "last one" bye """)  

应给予

["hello", "column number 2", "foo", ";this is a comment;", "last one", "bye"]

Answer 1

shlex解析器提供了一个指定注释字符的选项，但它不能从简化的shlex.split接口中获得。示例：

import shlex

a = 'hello "bla bla" ; this is a comment'

lex = shlex.shlex(a, posix=True)
lex.commenters = ';'
print(list(lex))  # ['hello', 'bla bla']

下面是一个稍微扩展的split函数，主要是从Python库复制的，对comments参数做了一些修改，允许指定注释字符：

import shlex
def shlex_split(s, comments='', posix=True):
    """Split the string *s* using shell-like syntax."""
    if s is None:
        import warnings
        warnings.warn("Passing None for 's' to shlex.split() is deprecated.",
                      DeprecationWarning, stacklevel=2)
    lex = shlex.shlex(s, posix=posix)
    lex.whitespace_split = True
    if isinstance(comments, str):
        lex.commenters = comments
    elif not comments:
        lex.commenters = ''
    return list(lex)

您可能希望在上面的代码中更改comments的默认值；正如所编写的那样，它与shlex.split具有相同的默认值，后者根本不识别注释。( shlex.shlex创建的解析器对象默认为#作为注释字符，如果指定comments=True，就会得到注释字符。我保持这种行为是为了兼容。)

注意，注释被忽略了；它们根本没有出现在结果向量中。当解析器命中注释字符时，它就会停止解析。(因此永远不会有两条评论。)comments字符串是一个可能的注释字符列表，而不是注释序列。因此，如果要同时识别#和;为注释字符，请指定comments='#:'。

下面是一个示例运行：

>>> # Default behaviour is the same as shlex.split
>>> shlex_split("""hello "column number 2" foo ; bar baz""") 
['hello', 'column number 2', 'foo', ';', 'bar', 'baz']
>>> # Supply a comments parameter to specify a comment character 
>>> shlex_split("""hello "column number 2" foo ; bar baz""", comments=';') 
['hello', 'column number 2', 'foo']
>>> shlex_split("""hello "column number 2" foo ;this is a comment; "last one" bye """, comments=';')
['hello', 'column number 2', 'foo']
>>> # The ; is recognised as a comment even if it is not preceded by whitespace.
>>> shlex_split("""hello "column number 2" foo;this is a comment; "last one" bye """, comments=';')
['hello', 'column number 2', 'foo']