[Zope3-checkins] SVN: Zope3/branches/jim-adapter/src/zope/deferredimport/ Changed to use a proxy-based approach. This is needed to get around

Jim Fulton jim at zope.com
Sun Apr 2 12:59:09 EDT 2006


Log message for revision 66296:
  Changed to use a proxy-based approach.  This is needed to get around
  the the inspect module's use of isinstance to test whether something
  is a module.  We can't subclass ModuleType because it doesn't let us
  replace a module's dictionary.
  
  Added a defineFrom convenience function.
  
  Improved the doctest by defining example modules in-line
  

Changed:
  U   Zope3/branches/jim-adapter/src/zope/deferredimport/README.txt
  U   Zope3/branches/jim-adapter/src/zope/deferredimport/__init__.py
  U   Zope3/branches/jim-adapter/src/zope/deferredimport/deferredmodule.py
  D   Zope3/branches/jim-adapter/src/zope/deferredimport/sample1.py.in
  D   Zope3/branches/jim-adapter/src/zope/deferredimport/sample2.py.in
  U   Zope3/branches/jim-adapter/src/zope/deferredimport/tests.py

-=-
Modified: Zope3/branches/jim-adapter/src/zope/deferredimport/README.txt
===================================================================
--- Zope3/branches/jim-adapter/src/zope/deferredimport/README.txt	2006-04-02 16:59:07 UTC (rev 66295)
+++ Zope3/branches/jim-adapter/src/zope/deferredimport/README.txt	2006-04-02 16:59:09 UTC (rev 66296)
@@ -3,109 +3,283 @@
 
 Often, especially for package modules, you want to import names for
 convenience, but not actually perform the imports until necessary.
+The zope.deferredimport package provided facilities for defining names
+in modules that will be imported from somewhere else when used.  You
+can also cause deprecation warnings to be issued when a variable is
+used, but we'll get to that later.
 
-The zope.deferredimport.define function supports this.  You can use
-the function to define one or more names to be imported when they are
-accessed.  Simply provide names as keyword arguments with import
-specifiers as values.  The import specifiers are given as atrings of
-the form "module:name", where module is the dotted name of the 
-module and name is a, possibly dotted, name of an object within the
-module. 
+The zope.deferredimport.define function can be used to define one or
+more names to be imported when they are accessed.  Simply provide
+names as keyword arguments with import specifiers as values.  The
+import specifiers are given as strings of the form "module:name",
+where module is the dotted name of the module and name is a, possibly
+dotted, name of an object within the module.
 
-To see how this works, see the sample modules, sample1 and sample2.
-The sample1 module defines several values as deferred imports of and
-from sample 2 using the define function::
+To see how this works, we'll create some sample modules within the
+zope.deferredimport package.  We'll actually use a helper function 
+specific to this document to define the modules inline so we can
+easily see what's in them.  Let's start by defining a module, sample1,
+that defined some things to be imported:
 
- zope.deferredimport.define(
-    sample2 = 'zope.deferredimport.sample2',
-    one = 'zope.deferredimport.sample2:x',
-    two = 'zope.deferredimport.sample2:C.y',
-    )
+    >>> create_module(sample1 = '''
+    ...
+    ... print "Sampe 1 imported!"
+    ... 
+    ... x = 1
+    ... 
+    ... class C:
+    ...     y = 2
+    ... 
+    ... z = 3
+    ... q = 4
+    ...
+    ... ''')
+    
+Note that the module starts by printing a message.  This allows us to
+see when the module is actually imported.  Now, let's define a module
+that imports some names from this module:
 
-The define function defines names that will be satisfied by later
-importing modules and importing names from them.  In this example, we
-defined the name 'sample2' and the module
-zope.deferredimport.sample2. The module isn't imported immediately,
+
+    >>> create_module(sample2 = '''
+    ...
+    ... import zope.deferredimport
+    ... 
+    ... zope.deferredimport.define(
+    ...     sample1 = 'zope.deferredimport.sample1',
+    ...     one = 'zope.deferredimport.sample1:x',
+    ...     two = 'zope.deferredimport.sample1:C.y',
+    ...     )
+    ... 
+    ... three = 3
+    ... x = 4
+    ... def getx():
+    ...     return x
+    ...
+    ... ''')
+
+
+In this example, we defined the name 'sample1' as the module
+zope.deferredimport.sample1. The module isn't imported immediately,
 but will be imported when needed.  Similarly, the name 'one' is
-defined as the 'x' attribute of sample2.
+defined as the 'x' attribute of sample1.
 
-The sample2 module prints a message when it is
-imported.  When we import sample1, we don't see a message until we
+The sample1 module prints a message when it is
+imported.  When we import sample2, we don't see a message until we
 access a variable:
 
-    >>> import zope.deferredimport.sample1
-    >>> print zope.deferredimport.sample1.one
-    Sampe 2 imported!
+    >>> import zope.deferredimport.sample2
+    >>> print zope.deferredimport.sample2.one
+    Sampe 1 imported!
     1
 
-    >>> import zope.deferredimport.sample2
+    >>> import zope.deferredimport.sample1
 
-    >>> zope.deferredimport.sample1.sample2 is zope.deferredimport.sample2
+    >>> zope.deferredimport.sample2.sample1 is zope.deferredimport.sample1
     True
 
 Note that a deferred attribute appears in a module's dictionary *after* 
 it is accessed the first time:
 
-    >>> 'two' in zope.deferredimport.sample1.__dict__
+    >>> 'two' in zope.deferredimport.sample2.__dict__
     False
 
-    >>> zope.deferredimport.sample1.two
+    >>> zope.deferredimport.sample2.two
     2
 
-    >>> 'two' in zope.deferredimport.sample1.__dict__
+    >>> 'two' in zope.deferredimport.sample2.__dict__
     True
 
+When deferred imports are used, the original module is replaced with a
+proxy. 
+
+    >>> type(zope.deferredimport.sample2)
+    <class 'zope.deferredimport.deferredmodule.ModuleProxy'>
+
+But we can use the proxy just like the original.  We can even update
+it.
+
+    >>> zope.deferredimport.sample2.x=5
+    >>> zope.deferredimport.sample2.getx()
+    5
+
+And the inspect module thinks it's a module:
+
+   >>> import inspect
+   >>> inspect.ismodule(zope.deferredimport.sample2)
+   True
+
+
+In the example above, the modules were fairly simple.  Let's look at a
+more complicated example.
+
+    >>> create_module(sample3 = '''
+    ... 
+    ... import zope.deferredimport
+    ... import zope.deferredimport.sample4
+    ... 
+    ... zope.deferredimport.define(
+    ...     sample1 = 'zope.deferredimport.sample1',
+    ...     one = 'zope.deferredimport.sample1:x',
+    ...     two = 'zope.deferredimport.sample1:C.y',
+    ...     )
+    ... 
+    ... x = 1
+    ...
+    ... ''')
+
+    >>> create_module(sample4 = '''
+    ... 
+    ... import sample3
+    ...
+    ... def getone():
+    ...     return sample3.one
+    ...
+    ... ''')
+
+Here, we have a circular import between sample3 and sample4.  When
+sample3 is imported, it imports sample 4, which then imports sample3.
+Let's see what happens when we use these modules in an unfortunate
+order:
+
+    >>> import zope.deferredimport.sample3
+    >>> import zope.deferredimport.sample4
+
+    >>> zope.deferredimport.sample4.getone()
+    Traceback (most recent call last):
+    ...
+    AttributeError: 'module' object has no attribute 'one'
+
+Hm.  Let's try accessing one through sample3:
+
+    >>> zope.deferredimport.sample3.one
+    1
+
+Funny, let's try getone again:
+
+    >>> zope.deferredimport.sample4.getone()
+    1
+
+The problem is that sample4 obtained sample3 before sample4 was
+replaced by a proxy.  This example is slightly pathalogical because it
+requires a circular import and a relative import, but the bug
+introduced is very subtle.  To guard against this, you should define
+defered imports ebfore importing any other modules.  Alternatively,
+you can call the initialize fuction before importing any other
+modules, as in:
+
+
+    >>> create_module(sample5 = '''
+    ... 
+    ... import zope.deferredimport
+    ... zope.deferredimport.initialize()
+    ...
+    ... import zope.deferredimport.sample6
+    ... 
+    ... zope.deferredimport.define(
+    ...     sample1 = 'zope.deferredimport.sample1',
+    ...     one = 'zope.deferredimport.sample1:x',
+    ...     two = 'zope.deferredimport.sample1:C.y',
+    ...     )
+    ... 
+    ... x = 1
+    ...
+    ... ''')
+
+    >>> create_module(sample6 = '''
+    ... 
+    ... import sample5
+    ...
+    ... def getone():
+    ...     return sample5.one
+    ...
+    ... ''')
+
+    >>> import zope.deferredimport.sample5
+    >>> import zope.deferredimport.sample6
+
+    >>> zope.deferredimport.sample6.getone()
+    1
+
+
+Deprecation
+-----------
+
 Deferred attributes can also be marked as deprecated, in which case, a
 message will be printed the first time they are accessed.
 
-The sample1 module defines deprecated attribute using two
-functions. The deprecated function is used like the define function::
+Lets define a module that has deprecated attributes defined as
+deferred imports:
 
-  zope.deferredimport.deprecated(
-    "Will go away in 2007.",
-    three = 'zope.deferredimport.sample2:C',
-    )
+    >>> create_module(sample7 = '''
+    ... 
+    ... import zope.deferredimport
+    ... zope.deferredimport.initialize()
+    ...
+    ... zope.deferredimport.deprecated(
+    ...     "Import from sample1 instead",
+    ...     x = 'zope.deferredimport.sample1:x',
+    ...     y = 'zope.deferredimport.sample1:C.y',
+    ...     z = 'zope.deferredimport.sample1:z',
+    ...     )
+    ...
+    ... ''')
 
-except that the first argument is a depecation message.  The
-deprecated function causes deprecation warnings to be printed when the
-defined variable is accessed the first time:
+Now, if we use one of these variables, we'll get a deprecation
+warning:
 
-    >>> zope.deferredimport.sample1.three is zope.deferredimport.sample2.C
-    README.txt:1: 
-    DeprecationWarning: three is deprecated. Will go away in 2007.
+    >>> import zope.deferredimport.sample7
+    >>> zope.deferredimport.sample7.x
+    ... doctest: +NORMALIZE_WHITESPACE
+    zope/deferredimport/README.txt:1: DeprecationWarning: 
+                x is deprecated. Import from sample1 instead
       Deferred Import
-    True
+    1
 
-The deprecatedFrom function handles the common case that we want to
-depecate multiple variables that we import from another module.  We pass
-the deprecation message, the name of the module that we want to import
-from, and one or more names to be imported::
+but only the first time:
 
-  zope.deferredimport.deprecatedFrom(
-    "Will go away in 2007.",
-    'zope.deferredimport.sample2',
-    'z', 'q',
-    )
+    >>> zope.deferredimport.sample7.x
+    1
 
-As with the deprecated function, warnings will be generated when the
-variables are accessed the first time.
+Importing multiple names from the same module
+---------------------------------------------
 
-    >>> zope.deferredimport.sample1.z is zope.deferredimport.sample2.z
-    README.txt:1: 
-    DeprecationWarning: z is deprecated. Will go away in 2007.
-      Deferred Import
-    True
+Sometimes, you want to get multiple things from the same module.  You
+can use defineFrom or deprecatedFrom to do that:
 
-    >>> zope.deferredimport.sample1.q is zope.deferredimport.sample2.q
-    README.txt:1: 
-    DeprecationWarning: q is deprecated. Will go away in 2007.
-      Deferred Import
-    True
 
-Of course, non-deferred variables are accessible as usuall:
+    >>> create_module(sample8 = '''
+    ... 
+    ... import zope.deferredimport
+    ...
+    ... zope.deferredimport.deprecatedFrom(
+    ...     "Import from sample1 instead",
+    ...     'zope.deferredimport.sample1',
+    ...     'x', 'z', 'q',
+    ...     )
+    ...
+    ... zope.deferredimport.defineFrom(
+    ...     'zope.deferredimport.sample9',
+    ...     'a', 'b', 'c',
+    ...     )
+    ...
+    ... ''')
 
-    >>> print zope.deferredimport.sample1.four
+    >>> create_module(sample9 = '''
+    ... print 'Imported sample 9'
+    ... a, b, c = range(10,13)
+    ... ''')
+
+    >>> import zope.deferredimport.sample8
+    >>> zope.deferredimport.sample8.q
+    ... doctest: +NORMALIZE_WHITESPACE
+    zope/deferredimport/README.txt:1: DeprecationWarning:
+            q is deprecated. Import from sample1 instead
+      Deferred Import
     4
 
-    >>> print zope.deferredimport.sample1.five
-    5
+    >>> zope.deferredimport.sample8.c
+    Imported sample 9
+    12
+
+Note, as in the example above, that you can make multiple
+deferred-import calls in a module.

Modified: Zope3/branches/jim-adapter/src/zope/deferredimport/__init__.py
===================================================================
--- Zope3/branches/jim-adapter/src/zope/deferredimport/__init__.py	2006-04-02 16:59:07 UTC (rev 66295)
+++ Zope3/branches/jim-adapter/src/zope/deferredimport/__init__.py	2006-04-02 16:59:09 UTC (rev 66296)
@@ -1,3 +1,3 @@
-from zope.deferredimport.deferredmodule import (
-    define, deprecated, deprecatedFrom,
-    )
+from zope.deferredimport.deferredmodule import initialize
+from zope.deferredimport.deferredmodule import define, defineFrom
+from zope.deferredimport.deferredmodule import deprecated, deprecatedFrom

Modified: Zope3/branches/jim-adapter/src/zope/deferredimport/deferredmodule.py
===================================================================
--- Zope3/branches/jim-adapter/src/zope/deferredimport/deferredmodule.py	2006-04-02 16:59:07 UTC (rev 66295)
+++ Zope3/branches/jim-adapter/src/zope/deferredimport/deferredmodule.py	2006-04-02 16:59:09 UTC (rev 66296)
@@ -18,16 +18,9 @@
 import types
 import sys
 import warnings
+import zope.proxy
 
-class Module(object):
 
-    def __init__(self, old):
-        self.__dict__ = old.__dict__
-        self.__original_module__ = old
-
-    def __repr__(self):
-        return `self.__original_module__`
-
 class Deferred(object):
 
     def __init__(self, name, specifier):
@@ -36,9 +29,7 @@
 
     _import_chicken = {}, {}, ['*']
 
-    def __get__(self, inst, class_):
-        if inst is None:
-            return self
+    def get(self):
 
         specifier = self.specifier
         if ':' in specifier:
@@ -50,50 +41,74 @@
         if name:
             for n in name.split('.'):
                 v = getattr(v, n)
-        setattr(inst, self.__name__, v)
         return v
 
 class DeferredAndDeprecated(Deferred):
 
     def __init__(self, name, specifier, message):
-        self.__name__ = name
-        self.specifier = specifier
+        super(DeferredAndDeprecated, self).__init__(name, specifier)
         self.message = message
-        
 
-    def __get__(self, inst, class_):
-        if inst is None:
-            return self
-
+    def get(self):
         warnings.warn(
             self.__name__ + " is deprecated. " + self.message,
-            DeprecationWarning, stacklevel=2)
-        return Deferred.__get__(self, inst, class_)
+            DeprecationWarning, stacklevel=3)
+        
+        return super(DeferredAndDeprecated, self).get()
 
 
-def getClass():
-    __name__ = sys._getframe(2).f_globals['__name__']
+class ModuleProxy(zope.proxy.ProxyBase):
+    __slots__ = ('__deferred_definitions__', )
+
+    def __init__(self, module):
+        super(ModuleProxy, self).__init__(module)
+        self.__deferred_definitions__ = {}
+
+    def __getattr__(self, name):
+        try:
+            get = self.__deferred_definitions__.pop(name)
+        except KeyError:
+            raise AttributeError, name
+        v = get.get()
+        setattr(self, name, v)
+        return v
+
+def initialize(level=1):
+    __name__ = sys._getframe(level).f_globals['__name__']
     module = sys.modules[__name__]
-    cls = module.__class__
-    if not issubclass(cls, Module):
-        cls = type('Module', (Module, ), {})
-        module = cls(module)
+    if not (type(module) is ModuleProxy):
+        module = ModuleProxy(module)
         sys.modules[__name__] = module
-    return cls
 
+    if level == 1:
+        return
+    return module
+
 def define(**names):
-    cls = getClass()
+    module = initialize(2)
+    __deferred_definitions__ = module.__deferred_definitions__
     for name, specifier in names.iteritems():
-        setattr(cls, name, Deferred(name, specifier))
+        __deferred_definitions__[name] = Deferred(name, specifier)
 
+def defineFrom(from_name, *names):
+    module = initialize(2)
+    __deferred_definitions__ = module.__deferred_definitions__
+    for name in names:
+        specifier = from_name + ':' + name
+        __deferred_definitions__[name] = Deferred(name, specifier)
+
 def deprecated(message, **names):
-    cls = getClass()
+    module = initialize(2)
+    __deferred_definitions__ = module.__deferred_definitions__
     for name, specifier in names.iteritems():
-        setattr(cls, name, DeferredAndDeprecated(name, specifier, message))
+        __deferred_definitions__[name] = DeferredAndDeprecated(
+            name, specifier, message)
 
-def deprecatedFrom(message, module, *names):
-    cls = getClass()
+def deprecatedFrom(message, from_name, *names):
+    module = initialize(2)
+    __deferred_definitions__ = module.__deferred_definitions__
     for name in names:
-        specifier = module + ':' + name
-        setattr(cls, name, DeferredAndDeprecated(name, specifier, message))
+        specifier = from_name + ':' + name
+        __deferred_definitions__[name] = DeferredAndDeprecated(
+            name, specifier, message)
     

Deleted: Zope3/branches/jim-adapter/src/zope/deferredimport/sample1.py.in
===================================================================
--- Zope3/branches/jim-adapter/src/zope/deferredimport/sample1.py.in	2006-04-02 16:59:07 UTC (rev 66295)
+++ Zope3/branches/jim-adapter/src/zope/deferredimport/sample1.py.in	2006-04-02 16:59:09 UTC (rev 66296)
@@ -1,22 +0,0 @@
-import zope.deferredimport
-
-four = 4
-
-zope.deferredimport.define(
-    sample2 = 'zope.deferredimport.sample2',
-    one = 'zope.deferredimport.sample2:x',
-    two = 'zope.deferredimport.sample2:C.y',
-    )
-
-zope.deferredimport.deprecated(
-    "Will go away in 2007.",
-    three = 'zope.deferredimport.sample2:C',
-    )
-
-zope.deferredimport.deprecatedFrom(
-    "Will go away in 2007.",
-    'zope.deferredimport.sample2',
-    'z', 'q',
-    )
-
-five = 5

Deleted: Zope3/branches/jim-adapter/src/zope/deferredimport/sample2.py.in
===================================================================
--- Zope3/branches/jim-adapter/src/zope/deferredimport/sample2.py.in	2006-04-02 16:59:07 UTC (rev 66295)
+++ Zope3/branches/jim-adapter/src/zope/deferredimport/sample2.py.in	2006-04-02 16:59:09 UTC (rev 66296)
@@ -1,9 +0,0 @@
-print "Sampe 2 imported!"
-
-x = 1
-
-class C:
-    y = 2
-
-z = 3
-q = 4

Modified: Zope3/branches/jim-adapter/src/zope/deferredimport/tests.py
===================================================================
--- Zope3/branches/jim-adapter/src/zope/deferredimport/tests.py	2006-04-02 16:59:07 UTC (rev 66295)
+++ Zope3/branches/jim-adapter/src/zope/deferredimport/tests.py	2006-04-02 16:59:09 UTC (rev 66296)
@@ -23,17 +23,18 @@
 
 def setUp(test):
     d = test.globs['tmp_d'] = tempfile.mkdtemp('deferredimport')
-    shutil.copy(
-        os.path.join(os.path.dirname(__file__), 'sample1.py.in'),
-        os.path.join(d, 'sample1.py'),
-        )
-    shutil.copy(
-        os.path.join(os.path.dirname(__file__), 'sample2.py.in'),
-        os.path.join(d, 'sample2.py'),
-        )
+
+    def create_module(**modules):
+        for name, src in modules.iteritems():
+            f = open(os.path.join(d, name+'.py'), 'w')
+            f.write(src)
+            f.close()
+            test.globs['created_modules'].append(name)
+
+    test.globs['created_modules'] = []
+    test.globs['create_module'] = create_module
+
     zope.deferredimport.__path__.append(d)
-    sys.modules.pop('zope.deferredimport.sample1', None)
-    sys.modules.pop('zope.deferredimport.sample2', None)
 
     test.globs['oldstderr'] = sys.stderr
     sys.stderr = OutErr
@@ -43,8 +44,8 @@
 
     zope.deferredimport.__path__.pop()
     shutil.rmtree(test.globs['tmp_d'])
-    sys.modules.pop('zope.deferredimport.sample1', None)
-    sys.modules.pop('zope.deferredimport.sample2', None)
+    for name in test.globs['created_modules']:
+        sys.modules.pop(name, None)
 
 def test_suite():
     checker = renormalizing.RENormalizing((



More information about the Zope3-Checkins mailing list