Python的shelve.open可以以嵌套的方式调用吗？

Question

我正在尝试编写一个回忆录库，它使用架架来持久地存储返回值。如果我有回忆录函数调用其他回忆录函数，我想知道如何正确打开书架文件。

import shelve
import functools


def cache(filename):
    def decorating_function(user_function):
        def wrapper(*args, **kwds):
            key = str(hash(functools._make_key(args, kwds, typed=False)))
            with shelve.open(filename, writeback=True) as cache:
                if key in cache:
                    return cache[key]
                else:
                    result = user_function(*args, **kwds)
                    cache[key] = result
                    return result

        return functools.update_wrapper(wrapper, user_function)

    return decorating_function


@cache(filename='cache')
def expensive_calculation():
    print('inside function')
    return


@cache(filename='cache')
def other_expensive_calculation():
    print('outside function')
    return expensive_calculation()

other_expensive_calculation()

但这不起作用

$ python3 shelve_test.py
outside function
Traceback (most recent call last):
  File "shelve_test.py", line 33, in <module>
    other_expensive_calculation()
  File "shelve_test.py", line 13, in wrapper
    result = user_function(*args, **kwds)
  File "shelve_test.py", line 31, in other_expensive_calculation
    return expensive_calculation()
  File "shelve_test.py", line 9, in wrapper
    with shelve.open(filename, writeback=True) as cache:
  File "/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/shelve.py", line 239, in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
  File "/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/shelve.py", line 223, in __init__
    Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
  File "/usr/local/Cellar/python3/3.4.1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/dbm/__init__.py", line 94, in open
    return mod.open(file, flag, mode)
_gdbm.error: [Errno 35] Resource temporarily unavailable

您对此类问题的解决方案的建议。

Answer 1

与其尝试嵌套要打开的调用(正如您发现的那样，它不起作用)，不如让您的装饰器维护对shelve.open返回的句柄的引用，然后如果它存在且仍然处于打开状态，则将其重用到后续调用中：

import shelve
import functools

def _check_cache(cache_, key, func, args, kwargs):
    if key in cache_:
        print("Using cached results")
        return cache_[key]
    else:
        print("No cached results, calling function")
        result = func(*args, **kwargs)
        cache_[key] = result
        return result

def cache(filename):
    def decorating_function(user_function):
        def wrapper(*args, **kwds):
            args_key = str(hash(functools._make_key(args, kwds, typed=False)))
            func_key = '.'.join([user_function.__module__, user_function.__name__])
            key = func_key + args_key
            handle_name = "{}_handle".format(filename)
            if (hasattr(cache, handle_name) and
                not hasattr(getattr(cache, handle_name).dict, "closed")
               ):
                print("Using open handle")
                return _check_cache(getattr(cache, handle_name), key, 
                                    user_function, args, kwds)
            else:
                print("Opening handle")
                with shelve.open(filename, writeback=True) as c:
                    setattr(cache, handle_name, c)  # Save a reference to the open handle
                    return _check_cache(c, key, user_function, args, kwds)

        return functools.update_wrapper(wrapper, user_function)
    return decorating_function


@cache(filename='cache')
def expensive_calculation():
    print('inside function')
    return


@cache(filename='cache')
def other_expensive_calculation():
    print('outside function')
    return expensive_calculation()

other_expensive_calculation()
print("Again")
other_expensive_calculation()

输出：

Opening handle
No cached results, calling function
outside function
Using open handle
No cached results, calling function
inside function
Again
Opening handle
Using cached results

编辑：

您还可以使用WeakValueDictionary实现装饰器，它看起来更易读：

from weakref import WeakValueDictionary

_handle_dict = WeakValueDictionary()
def cache(filename):
    def decorating_function(user_function):
        def wrapper(*args, **kwds):
            args_key = str(hash(functools._make_key(args, kwds, typed=False)))
            func_key = '.'.join([user_function.__module__, user_function.__name__])
            key = func_key + args_key
            handle_name = "{}_handle".format(filename)
            if handle_name in _handle_dict:
                print("Using open handle")
                return _check_cache(_handle_dict[handle_name], key, 
                                    user_function, args, kwds)
            else:
                print("Opening handle")
                with shelve.open(filename, writeback=True) as c:
                    _handle_dict[handle_name] = c
                    return _check_cache(c, key, user_function, args, kwds)

        return functools.update_wrapper(wrapper, user_function)
    return decorating_function

一旦没有其他对句柄的引用，它就会从字典中删除。由于我们的句柄只有在外部对修饰函数的大部分调用结束时才会超出作用域，所以在句柄打开时，我们总是在dict中有一个条目，并且在句柄关闭后就没有条目了。

Answer 2

不，您可能没有具有相同文件名的嵌套shelve实例。

搁置模块不支持对搁置对象的并发读/写访问。(多个同时读取访问是安全的。)当一个程序有一个可以写的架子时，任何其他程序都不应该打开它来读或写。Unix文件锁定可以用来解决这一问题，但这在Unix版本中有所不同，需要了解所使用的数据库实现。

https://docs.python.org/3/library/shelve.html#restrictions

Answer 3

您将打开该文件两次，但从未实际关闭它以更新该文件以供任何用途。最后使用f.close ()。