[Zope3-checkins] SVN: Zope3/branches/jim-index/ Checking in progress:

Jim Fulton jim at zope.com
Mon Aug 16 09:40:02 EDT 2004


Log message for revision 27144:
  Checking in progress:
  
  - Removed zope.app.index (again)
  
  - Moved catalog-specific indexing code to zope.app.catalog
  
  - Added various bits of configuration to tie things together.
  
  See the todo file.
  
  


Changed:
  D   Zope3/branches/jim-index/package-includes/index-configure.zcml
  U   Zope3/branches/jim-index/src/zope/app/catalog/README.txt
  A   Zope3/branches/jim-index/src/zope/app/catalog/attribute.py
  U   Zope3/branches/jim-index/src/zope/app/catalog/browser/__init__.py
  U   Zope3/branches/jim-index/src/zope/app/catalog/browser/advanced.pt
  U   Zope3/branches/jim-index/src/zope/app/catalog/browser/catalog.py
  U   Zope3/branches/jim-index/src/zope/app/catalog/browser/configure.zcml
  U   Zope3/branches/jim-index/src/zope/app/catalog/catalog.py
  U   Zope3/branches/jim-index/src/zope/app/catalog/configure.zcml
  A   Zope3/branches/jim-index/src/zope/app/catalog/field.py
  U   Zope3/branches/jim-index/src/zope/app/catalog/interfaces.py
  A   Zope3/branches/jim-index/src/zope/app/catalog/keyword.py
  U   Zope3/branches/jim-index/src/zope/app/catalog/tests.py
  A   Zope3/branches/jim-index/src/zope/app/catalog/text.py
  U   Zope3/branches/jim-index/src/zope/app/container/constraints.py
  D   Zope3/branches/jim-index/src/zope/app/index/
  U   Zope3/branches/jim-index/src/zope/app/uniqueid/__init__.py
  U   Zope3/branches/jim-index/src/zope/app/zptpage/textindex/configure.zcml
  U   Zope3/branches/jim-index/src/zope/app/zptpage/textindex/tests.py
  U   Zope3/branches/jim-index/src/zope/app/zptpage/textindex/zptpage.py
  U   Zope3/branches/jim-index/src/zope/index/field/__init__.py
  U   Zope3/branches/jim-index/src/zope/index/interfaces/__init__.py
  A   Zope3/branches/jim-index/src/zope/index/interfaces/searchabletext.py
  U   Zope3/branches/jim-index/src/zope/index/keyword/__init__.py
  D   Zope3/branches/jim-index/src/zope/index/keyword/tests/
  A   Zope3/branches/jim-index/src/zope/index/keyword/tests.py
  U   Zope3/branches/jim-index/src/zope/index/text/__init__.py
  A   Zope3/branches/jim-index/todo


-=-
Deleted: Zope3/branches/jim-index/package-includes/index-configure.zcml
===================================================================
--- Zope3/branches/jim-index/package-includes/index-configure.zcml	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/package-includes/index-configure.zcml	2004-08-16 13:40:02 UTC (rev 27144)
@@ -1 +0,0 @@
-<include package="zope.app.index"/>

Modified: Zope3/branches/jim-index/src/zope/app/catalog/README.txt
===================================================================
--- Zope3/branches/jim-index/src/zope/app/catalog/README.txt	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/app/catalog/README.txt	2004-08-16 13:40:02 UTC (rev 27144)
@@ -1,9 +1,5 @@
 Adding a site catalog:
 
-The following presupposes that the ObjectHub is installed and the
-Registration object for the object hub is installed and active, so the
-ObjectHub can pass events on to the catalogs/indexes.
-
 This also presupposes a product called "blog", which allows you to create
 content objects - this should apply equally to any other content objects
 in your own zope install, presuming they have attributes with values ;)

Copied: Zope3/branches/jim-index/src/zope/app/catalog/attribute.py (from rev 26898, Zope3/branches/jim-index/src/zope/app/index/ifaceindex.py)
===================================================================
--- Zope3/branches/jim-index/src/zope/app/index/ifaceindex.py	2004-08-04 09:38:12 UTC (rev 26898)
+++ Zope3/branches/jim-index/src/zope/app/catalog/attribute.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -0,0 +1,141 @@
+#############################################################################
+#
+# Copyright (c) 2004 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Index interface-defined attributes
+
+$Id$
+"""
+
+import zope.interface
+from zope.app.catalog.interfaces import IAttributeIndex
+
+class AttributeIndex(object):
+    """Index interface-defined attributes
+
+       Mixin for indexing a particular attribute of an object after
+       first adapting the object to be indexed to an interface.
+
+       The class is meant to be mixed with a base class that defines an
+       index_doc method:
+       
+         >>> class BaseIndex(object):
+         ...     def __init__(self):
+         ...         self.data = []
+         ...     def index_doc(self, id, value):
+         ...         self.data.append((id, value))
+
+       The class does two things. The first is to get a named field
+       from an object:
+
+         >>> class Data:
+         ...     def __init__(self, v):
+         ...         self.x = v
+
+         >>> class Index(AttributeIndex, BaseIndex):
+         ...     pass
+
+         >>> index = Index('x')
+         >>> index.index_doc(11, Data(1))
+         >>> index.index_doc(22, Data(2))
+         >>> index.data
+         [(11, 1), (22, 2)]
+
+       A method can be indexed:
+
+         >>> Data.z = lambda self: self.x + 20
+         >>> index = Index('z', field_callable=True)
+         >>> index.index_doc(11, Data(1))
+         >>> index.index_doc(22, Data(2))
+         >>> index.data
+         [(11, 21), (22, 22)]
+
+       But you'll get an error if you try to index a method without
+       setting field_callable:
+
+         >>> index = Index('z')
+         >>> index.index_doc(11, Data(1))
+         
+       The class can also adapt an object to an interface:
+
+         >>> from zope.interface import Interface
+         >>> class I(Interface):
+         ...     pass
+
+         >>> class Data:
+         ...     def __init__(self, v):
+         ...         self.x = v
+         ...     def __conform__(self, iface):
+         ...         if iface is I:
+         ...             return Data2(self.x)
+
+         >>> class Data2:
+         ...     def __init__(self, v):
+         ...         self.y = v*v
+         
+         >>> index = Index('y', I)
+         >>> index.index_doc(11, Data(3))
+         >>> index.index_doc(22, Data(2))
+         >>> index.data
+         [(11, 9), (22, 4)]
+
+       When you define an index class, you can define a default
+       interface and/or a default interface:
+
+         >>> class Index(AttributeIndex, BaseIndex):
+         ...     default_interface = I
+         ...     default_field_name = 'y'
+        
+         >>> index = Index()
+         >>> index.index_doc(11, Data(3))
+         >>> index.index_doc(22, Data(2))
+         >>> index.data
+         [(11, 9), (22, 4)]
+
+       """
+
+    zope.interface.implements(IAttributeIndex)
+    
+    default_field_name = None
+    default_interface = None
+
+    def __init__(self, field_name=None, interface=None, field_callable=False):
+        super(AttributeIndex, self).__init__()
+        if field_name is None and self.default_field_name is None:
+            raise ValueError, "Must pass a field_name"
+        if field_name is None:
+            self.field_name = self.default_field_name
+        else:
+            self.field_name = field_name
+        if interface is None:
+            self.interface = self.default_interface
+        else:
+            self.interface = interface
+        self.field_callable = field_callable
+
+    def index_doc(self, docid, object):
+        if self.interface is not None:
+            object = self.interface(object, None)
+            if object is None:
+                return None
+
+        value = getattr(object, self.field_name, None)
+        if value is None:
+            return None
+
+        if self.field_callable:
+            try:
+                value = value()
+            except:
+                return None
+
+        return super(AttributeIndex, self).index_doc(docid, value)

Modified: Zope3/branches/jim-index/src/zope/app/catalog/browser/__init__.py
===================================================================
--- Zope3/branches/jim-index/src/zope/app/catalog/browser/__init__.py	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/app/catalog/browser/__init__.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -1 +1 @@
-# make this directory a package
+#

Modified: Zope3/branches/jim-index/src/zope/app/catalog/browser/advanced.pt
===================================================================
--- Zope3/branches/jim-index/src/zope/app/catalog/browser/advanced.pt	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/app/catalog/browser/advanced.pt	2004-08-16 13:40:02 UTC (rev 27144)
@@ -2,61 +2,26 @@
 <body>
 <div metal:fill-slot="body">
 
-  <table border="0">
-    <caption i18n:translate="">Advanced Catalog Thingies</caption>
-    <tbody>
-      <tr>
-        <span tal:condition="context/getSubscribed" tal:omit-tag="">
-          <td i18n:translate="">
-            Catalog is currently <strong>subscribed</strong> to the 
-            object hub.
-          </td>
-          <td>
-            <form method="post" action="unsubscribe.html">
-              <input type="submit" name="unsubscribe" value="Unsubscribe"
-                  i18n:attributes="value unsubscribe-button" />
-            </form>
-          </td>
-        </span>
-        <span tal:condition="not:context/getSubscribed" tal:omit-tag="">
-          <td>
-            Catalog is currently <strong>unsubscribed</strong> from the 
-            object hub.
-          </td>
-          <td>
-            <form method="post" action="subscribe.html">
-              <input type="submit" name="subscribe" value="Subscribe"
-                     i18n:attributes="value subscribe-button"/>
-              <input type="checkbox" value="true" name="update" checked=""/> 
-              <span tal:omit-tag="" i18n:translate="">
-                (and reindex all objects, if checked)
-              </span>
-            </form>
-          </td>
-        </span>
-      </tr>
-      <tr>
-        <form method="POST" action="clear.html">
-          <td align="right">
-            Clear all indexes<br><i>not hooked up yet</i>
-          </td>
-          <td><input type="submit" name="submit" value="Clear"></td>
-        </form>
-      </tr>
-      <tr>
-        <form method="POST" action="reindex.html">
-          <td align="right">
-            Refresh all indexes<br><i>not hooked up yet</i>
-          </td>
-          <td>
-            <input type="submit" name="submit" value="Refresh"
-                   i18n:attributes="value refresh-button"/>
-          </td>
-        </form>
-      </tr>
-    </tbody>
-  </table>
+<h2 i18n:translate="">Catalog statistics</h2>
 
+<table border="0">
+   <tr><th i18n:translate="">Index</th>
+       <th i18n:translate="">Document Count</th>
+       <th i18n:translate="">Word Count</th>
+   </tr>
+   <tr tal:repeat="indexname context"
+       >
+       <td tal:content="indexname">foo</td>
+       <td tal:content="index/?indexname/documentCount|nothing">42</td>
+       <td tal:content="index/?indexname/wordCount|nothing">99</td>
+    </tr>
+</table>
+
+<form method="POST" action="reindex.html">
+    <input type="submit" value="Reindex"
+           i18n:attributes="value reindex-button"/>
+</form>
+
 </div>
 </body>
 </html>

Modified: Zope3/branches/jim-index/src/zope/app/catalog/browser/catalog.py
===================================================================
--- Zope3/branches/jim-index/src/zope/app/catalog/browser/catalog.py	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/app/catalog/browser/catalog.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -18,7 +18,7 @@
 from zope.app.container.browser.adding import Adding
 from zope.app.catalog.interfaces import ICatalog
 
-class CatalogEditView:
+class Advanced:
     "Provides a user interface for configuring a catalog"
 
     __used_for__ = ICatalog
@@ -27,23 +27,7 @@
         self.context = context
         self.request = request
 
-    def subscribe(self, update=False):
-        self.context.subscribeEvents(update)
-        self.request.response.redirect(".")
-
-    def unsubscribe(self):
-        self.context.unsubscribeEvents()
-        self.request.response.redirect(".")
-
-    def clear(self):
-        self.context.clearIndexes()
-        self.request.response.redirect(".")
-
     def reindex(self):
+        self.context.clear()
         self.context.updateIndexes()
         self.request.response.redirect(".")
-
-
-class IndexAdding(Adding):
-    menu_id = 'catalog_index_menu'
-

Modified: Zope3/branches/jim-index/src/zope/app/catalog/browser/configure.zcml
===================================================================
--- Zope3/branches/jim-index/src/zope/app/catalog/browser/configure.zcml	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/app/catalog/browser/configure.zcml	2004-08-16 13:40:02 UTC (rev 27144)
@@ -4,101 +4,117 @@
    i18n_domain="zope"
    >
 
-  <!-- Allow a catalog to be added to content space -->
-  <addMenuItem
-      title="Catalog"
-      description="A Catalog allows indexing and searching of objects"
-      class="zope.app.catalog.catalog.Catalog"
-      permission="zope.ManageContent"
-      />
+<!-- Allow a catalog to be added to content space -->
+<addMenuItem
+    title="Catalog"
+    description="A Catalog allows indexing and searching of objects"
+    class="zope.app.catalog.catalog.Catalog"
+    permission="zope.ManageContent"
+    />
 
-  <icon
-      name="zmi_icon"
-      for="zope.app.catalog.interfaces.ICatalog"
-      file="catalog_icon.gif"
-      />
+<icon
+    name="zmi_icon"
+    for="zope.app.catalog.interfaces.ICatalog"
+    file="catalog_icon.gif"
+    />
 
-  <addMenuItem
-      title="Site Catalog"
-      description="A Catalog allows indexing and searching of objects"
-      class="zope.app.catalog.catalog.CatalogUtility"
-      permission="zope.ManageServices"
+<containerViews
+     for="..interfaces.ICatalog"
+     contents="zope.ManageServices"
+     index="zope.ManageServices"
+     add="zope.ManageServices"
      />
 
-  <!-- Standard container 'contents' tab -->
-  <page
-      for="zope.app.catalog.interfaces.ICatalog"
-      name="contents.html"
-      menu="zmi_views"
-      title="Indexes"
-      attribute="contents"
-      class="zope.app.container.browser.contents.Contents"
-      permission="zope.ManageContent"
-      />
+<pages
+    for="..interfaces.ICatalog"
+    class=".catalog.Advanced"
+    permission="zope.ManageContent">
 
-  <!-- This hooks up a custom add menu.  -->
-  <view
-      for="zope.app.catalog.interfaces.ICatalog"
-      name="+"
-      menu="zmi_actions" title="Add"
-      class="zope.app.catalog.browser.catalog.IndexAdding"
-      permission="zope.ManageContent">
-    <page name="index.html"  attribute="index" />
-    <page name="action.html" attribute="action" />
-  </view>
+  <page name="advances.html" template="advanced.pt"
+        menu="zmi_views" title="Advanced"/>
+  <page name="reindex.html" attribute="reindex"/>
+</pages>
 
-  <!-- the add menu is browser.IndexAdding -->
-  <menu id="catalog_index_menu" title="Add Index" />
+<!-- Indexes -->
 
-  <!-- Add a couple of items to the add menu -->
-  <!-- TextIndex -->
-  <addform
-      name="AddTextIndexToCatalog"
-      menu="catalog_index_menu" title="Text Index"
-      schema="zope.app.index.interfaces.text.IUITextCatalogIndex"
-      permission="zope.ManageServices"
-      content_factory="zope.app.index.text.index.TextCatalogIndex"
-      arguments="field_name"
-      keyword_arguments="interface"
-      description="A full text index"
-      />
+<addform
+    name="AddFieldIndex"
+    label="Add a field index"
+    schema="..interfaces.IAttributeIndex"
+    permission="zope.ManageServices"
+    content_factory="..field.FieldIndex"
+    arguments="field_name"
+    keyword_arguments="interface"
+    />
 
-  <!-- FieldIndex -->
-  <addform
-      name="AddFieldIndexToCatalog"
-      menu="catalog_index_menu" title="Field Index"
-      schema="zope.app.index.interfaces.field.IUIFieldCatalogIndex"
-      permission="zope.ManageServices"
-      content_factory="zope.app.index.field.index.FieldCatalogIndex"
-      arguments="field_name"
-      keyword_arguments="interface"
-      description="An index of a specific field"
-      />
+<addMenuItem
+    title="Field Index"
+    description="Index items based on an orderable field value"
+    class="..field.FieldIndex"
+    permission="zope.ManageServices"
+    view="AddFieldIndex"
+   />
 
-  <!-- KeywordIndex -->
-  <addform
-      name="AddKeywordIndexToCatalog"
-      menu="catalog_index_menu" title="Keyword Index"
-      schema="zope.app.index.interfaces.keyword.IUIKeywordCatalogIndex"
-      permission="zope.ManageServices"
-      content_factory="zope.app.index.keyword.index.KeywordCatalogIndex"
-      arguments="field_name"
-      keyword_arguments="interface"
-      description="A keyword index of a specific field"
-      />
+<schemadisplay
+    name="index.html"
+    schema="..field.IFieldIndex"
+    label="Field Index"
+    permission="zope.ManageServices"
+    menu="zmi_views" title="Configuration"
+    />
 
-  <!-- the Advanced tab of the Catalog -->
-  <pages
-      for="zope.app.catalog.interfaces.ICatalog"
-      class="zope.app.catalog.browser.catalog.CatalogEditView"
-      permission="zope.ManageContent">
+<addform
+    name="AddKeywordIndex"
+    label="Add a keyword index"
+    schema="..interfaces.IAttributeIndex"
+    permission="zope.ManageServices"
+    content_factory="..keyword.KeywordIndex"
+    arguments="field_name"
+    keyword_arguments="interface"
+    />
 
-    <page name="index.html" template="advanced.pt"
-          menu="zmi_views" title="Advanced"/>
-    <page name="subscribe.html" attribute="subscribe"/>
-    <page name="unsubscribe.html" attribute="unsubscribe"/>
-    <page name="clear.html" attribute="clear"/>
-    <page name="reindex.html" attribute="reindex"/>
-  </pages>
+<addMenuItem
+    title="Keyword Index"
+    description="Index items based on multi-value fields with
+                 orderable values"
+    class="..keyword.KeywordIndex"
+    permission="zope.ManageServices"
+    view="AddKeywordIndex"
+   />
 
+<schemadisplay
+    name="index.html"
+    schema="..keyword.IKeywordIndex"
+    label="Keyword Index"
+    permission="zope.ManageServices"
+    menu="zmi_views" title="Configuration"
+    />
+
+<addform
+    name="AddTextIndex"
+    label="Add a text index"
+    schema="..interfaces.IAttributeIndex"
+    permission="zope.ManageServices"
+    content_factory="..text.TextIndex"
+    arguments="field_name"
+    keyword_arguments="interface"
+    />
+
+<addMenuItem
+    title="Text Index"
+    description="Index items based on multi-value fields with
+                 orderable values"
+    class="..text.TextIndex"
+    permission="zope.ManageServices"
+    view="AddTextIndex"
+   />
+
+<schemadisplay
+    name="index.html"
+    schema="..text.ITextIndex"
+    label="Text Index"
+    permission="zope.ManageServices"
+    menu="zmi_views" title="Configuration"
+    />
+
 </zope:configure>

Modified: Zope3/branches/jim-index/src/zope/app/catalog/catalog.py
===================================================================
--- Zope3/branches/jim-index/src/zope/app/catalog/catalog.py	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/app/catalog/catalog.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -46,11 +46,11 @@
             yield obj
 
 
-class CatalogBase(Persistent, SampleContainer):
+class Catalog(Persistent, SampleContainer):
 
-    implements(ICatalog, IContainer, IAttributeAnnotatable)
+    implements(ICatalog, IContainer, IAttributeAnnotatable, ILocalUtility)
 
-    def clearIndexes(self):
+    def clear(self):
         for index in self.values():
             index.clear()
 
@@ -64,6 +64,12 @@
         for index in self.values():
             index.unindex_doc(docid)
 
+    def updateIndex(self, index):
+        uidutil = zapi.getUtility(IUniqueIdUtility)
+        for uid, ref in uidutil.items():
+            obj = ref()
+            index.index_doc(uid, obj)
+
     def updateIndexes(self):
         uidutil = zapi.getUtility(IUniqueIdUtility)
         for uid, ref in uidutil.items():
@@ -95,20 +101,27 @@
         results = ResultSet(pendingResults, uidutil)
         return results
 
+def indexAdded(index, event):
+    """When an index is added to a catalog, we have to index existing objects
 
-class CatalogUtility(CatalogBase):
-    """A catalog in service-space."""
-    # Utilities will default to implementing the most specific
-    # interface. This piece of delightfulness is needed because
-    # the interface resolution order machinery implements (no
-    # pun intended) the old-style Python resolution order machinery.
-    implements(ILocalUtility)
+       When an index is added, we tell it's parent to index it:
 
+         >>> class FauxCatalog:
+         ...     def updateIndex(self, index):
+         ...         self.updated = index
 
-class Catalog(CatalogBase):
-    """A catalog in content-space."""
+         >>> class FauxIndex:
+         ...     pass
 
+         >>> index = FauxIndex()
+         >>> index.__parent__ = FauxCatalog()
 
+         >>> indexAdded(index, None)
+         >>> index.__parent__.updated is index
+         True
+       """
+    index.__parent__.updateIndex(index)
+    
 def indexDocSubscriber(event):
     """A subscriber to UniqueIdAddedEvent"""
     for cat in zapi.getAllUtilitiesRegisteredFor(ICatalog):

Modified: Zope3/branches/jim-index/src/zope/app/catalog/configure.zcml
===================================================================
--- Zope3/branches/jim-index/src/zope/app/catalog/configure.zcml	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/app/catalog/configure.zcml	2004-08-16 13:40:02 UTC (rev 27144)
@@ -4,9 +4,6 @@
     >
 
 <content class=".catalog.Catalog">
-  <implements
-      interface="zope.app.annotation.interfaces.IAttributeAnnotatable"
-      />
   <factory
       id="zope.app.catalog"
       />
@@ -16,55 +13,68 @@
       />
   <require
       interface="zope.app.catalog.interfaces.ICatalogEdit"
-      permission="zope.ManageContent"
+      permission="zope.ManageServices"
       />
   <require
       interface="zope.app.container.interfaces.IContainer"
-      permission="zope.ManageContent"
+      permission="zope.ManageServices"
       />
 </content>
 
-<content class=".catalog.CatalogUtility">
-  <factory
-      id="zope.app.catalogutility"
-      />
-  <require
-      interface="zope.app.catalog.interfaces.ICatalogQuery"
-      permission="zope.View"
-      />
-  <require
-      interface="zope.app.catalog.interfaces.ICatalogEdit"
-      permission="zope.ManageContent"
-      />
-  <require
-      interface="zope.app.container.interfaces.IContainer"
-      permission="zope.ManageContent"
-      />
-</content>
-
 <content class=".catalog.ResultSet">
   <require
       attributes="__iter__ __len__"
       permission="zope.Public"/>
 </content>
 
-  <subscriber
-      factory=".catalog.indexDocSubscriber"
-      for="zope.app.uniqueid.interfaces.UniqueIdAddedEvent"
+<subscriber
+    factory=".catalog.indexAdded"
+    for=".interfaces.ICatalogIndex
+         zope.app.container.interfaces.IObjectAddedEvent"
+    />
+
+<subscriber
+    factory=".catalog.indexDocSubscriber"
+    for="zope.app.uniqueid.interfaces.IUniqueIdAddedEvent"
+    />
+
+<subscriber
+    factory=".catalog.reindexDocSubscriber"
+    for="zope.app.event.objectevent.IObjectModifiedEvent"
+    />
+
+<subscriber
+    factory=".catalog.unindexDocSubscriber"
+    for="zope.app.uniqueid.interfaces.IUniqueIdRemovedEvent"
+    />
+
+
+<content class=".field.FieldIndex">
+  <require
+      permission="zope.ManageServices"
+      interface=".interfaces.IAttributeIndex"
+      set_schema=".interfaces.IAttributeIndex"
       />
+</content>
 
-  <subscriber
-      factory=".catalog.reindexDocSubscriber"
-      for="zope.app.event.objectevent.ObjectModifiedEvent"
+<content class=".keyword.KeywordIndex">
+  <require
+      permission="zope.ManageServices"
+      interface=".interfaces.IAttributeIndex"
+      set_schema=".interfaces.IAttributeIndex"
       />
+</content>
 
-  <subscriber
-      factory=".catalog.unindexDocSubscriber"
-      for="zope.app.uniqueid.interfaces.UniqueIdRemovedEvent"
+<content class=".text.TextIndex">
+  <require
+      permission="zope.ManageServices"
+      interface=".interfaces.IAttributeIndex"
+      set_schema=".interfaces.IAttributeIndex"
       />
+</content>
 
-<!-- Include browser package -->
 
+
 <include package=".browser" />
 
 </configure>

Copied: Zope3/branches/jim-index/src/zope/app/catalog/field.py (from rev 26898, Zope3/branches/jim-index/src/zope/app/index/field/index.py)
===================================================================
--- Zope3/branches/jim-index/src/zope/app/index/field/index.py	2004-08-04 09:38:12 UTC (rev 26898)
+++ Zope3/branches/jim-index/src/zope/app/catalog/field.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -0,0 +1,36 @@
+##############################################################################
+#
+# Copyright (c) 2002 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Field catalog indexes
+
+$Id$
+"""
+import zope.index.field
+import zope.interface
+
+import zope.app.container.contained
+import zope.app.catalog.attribute
+import zope.app.catalog.interfaces
+
+class IFieldIndex(zope.app.catalog.interfaces.IAttributeIndex,
+                  zope.app.catalog.interfaces.ICatalogIndex):
+    """Interface-based catalog field index
+    """
+
+class FieldIndex(zope.app.catalog.attribute.AttributeIndex,
+                 zope.index.field.FieldIndex,
+                 zope.app.container.contained.Contained):
+
+    zope.interface.implements(IFieldIndex)
+
+

Modified: Zope3/branches/jim-index/src/zope/app/catalog/interfaces.py
===================================================================
--- Zope3/branches/jim-index/src/zope/app/catalog/interfaces.py	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/app/catalog/interfaces.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -15,26 +15,64 @@
 
 $Id$
 """
-from zope.interface import Interface
-from zope.index.interfaces import IInjection
 
-class ICatalogQuery(Interface):
+import zope.index.interfaces
+import zope.interface
+import zope.schema
+import zope.app.container.interfaces
+import zope.app.container.constraints
+
+from zope.app.i18n import ZopeMessageIDFactory as _
+
+class ICatalogQuery(zope.interface.Interface):
     """Provides Catalog Queries."""
 
     def searchResults(**kw):
         """Search on the given indexes."""
 
 
-class ICatalogEdit(IInjection):
+class ICatalogEdit(zope.index.interfaces.IInjection):
     """Allows one to manipulate the Catalog information."""
 
-    def clearIndexes():
-        """Remove all index data."""
-
     def updateIndexes():
         """Reindex all objects."""
 
 
-class ICatalog(ICatalogQuery, ICatalogEdit): 
+class ICatalogIndex(zope.index.interfaces.IInjection):
+    """An index to be used in a catalog
+    """
+    __parent__ = zope.schema.Field()
+    
+
+class ICatalog(ICatalogQuery, ICatalogEdit,
+               zope.app.container.interfaces.IContainer): 
     """Marker to describe a catalog in content space."""
 
+    zope.app.container.constraints.contains(ICatalogIndex)
+
+ICatalogIndex['__parent__'].constraint = (
+    zope.app.container.constraints.ContainerTypesConstraint(ICatalog))
+
+class IAttributeIndex(zope.interface.Interface):
+    """I index objects by first adapting them to an interface, then
+       retrieving a field on the adapted object.
+    """
+
+    interface = zope.schema.Choice(
+        title=_(u"Interface"),
+        description=_(u"Objects will be adapted to this interface"),
+        vocabulary=_("Interfaces"),
+        required=False,
+        )
+
+    field_name = zope.schema.BytesLine(
+        title=_(u"Field Name"),
+        description=_(u"Name of the field to index"),
+        )
+
+    field_callable = zope.schema.Bool(
+        title=_(u"Field Callable"),
+        description=_(u"If true, then the field should be called to get the "
+                      u"value to be indexed"),
+        )
+        

Copied: Zope3/branches/jim-index/src/zope/app/catalog/keyword.py (from rev 26898, Zope3/branches/jim-index/src/zope/app/index/keyword/index.py)
===================================================================
--- Zope3/branches/jim-index/src/zope/app/index/keyword/index.py	2004-08-04 09:38:12 UTC (rev 26898)
+++ Zope3/branches/jim-index/src/zope/app/catalog/keyword.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -0,0 +1,37 @@
+##############################################################################
+#
+# Copyright (c) 2003 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Keyword catalog indexes
+
+$Id$
+"""
+
+import zope.index.keyword
+import zope.interface
+
+import zope.app.container.contained
+import zope.app.catalog.attribute
+import zope.app.catalog.interfaces
+
+class IKeywordIndex(zope.app.catalog.interfaces.IAttributeIndex,
+                    zope.app.catalog.interfaces.ICatalogIndex,
+                    ):
+    """Interface-based catalog keyword index
+    """
+
+class KeywordIndex(zope.app.catalog.attribute.AttributeIndex,
+                   zope.index.keyword.KeywordIndex,
+                   zope.app.container.contained.Contained):
+
+    zope.interface.implements(IKeywordIndex)
+

Modified: Zope3/branches/jim-index/src/zope/app/catalog/tests.py
===================================================================
--- Zope3/branches/jim-index/src/zope/app/catalog/tests.py	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/app/catalog/tests.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -156,12 +156,19 @@
         for index in catalog.values():
             checkNotifies = index.doc
             self.assertEqual(len(checkNotifies), 18)
-            ### notifLocs = [ x.location for x in checkNotifies ]
-            ### notifLocs.sort()
-            ### expected = [ "/%s"%(i+1) for i in range(18) ]
-            ### expected.sort()
-            ### self.assertEqual(notifLocs, expected)
 
+    def test_updateindex(self):
+        """Test a full refresh."""
+        self._frob_uniqueidutil()
+        catalog = Catalog()
+        catalog['author'] = StubIndex('author', None)
+        catalog['title'] = StubIndex('author', None)
+        catalog.updateIndexe(catalog['author'])
+        checkNotifies = catalog['author'].doc
+        self.assertEqual(len(checkNotifies), 18)
+        checkNotifies = catalog['title'].doc
+        self.assertEqual(len(checkNotifies), 0)
+
     def test_basicsearch(self):
         """Test the simple search results interface."""
         self._frob_uniqueidutil(ints=0)
@@ -269,9 +276,11 @@
         self.assertEqual(self.cat.regs, [])
 
 def test_suite():
+    from zope.testing.doctestunit import DocTestSuite
     suite = unittest.TestSuite()
     suite.addTest(unittest.makeSuite(Test))
     suite.addTest(unittest.makeSuite(TestEventSubscribers))
+    suite.addTest(DocTestSuite('zope.app.catalog.attribute'))
     return suite
 
 

Copied: Zope3/branches/jim-index/src/zope/app/catalog/text.py (from rev 26898, Zope3/branches/jim-index/src/zope/app/index/text/index.py)
===================================================================
--- Zope3/branches/jim-index/src/zope/app/index/text/index.py	2004-08-04 09:38:12 UTC (rev 26898)
+++ Zope3/branches/jim-index/src/zope/app/catalog/text.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -0,0 +1,34 @@
+##############################################################################
+#
+# Copyright (c) 2002 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Text catalog indexes
+
+$Id$
+"""
+import zope.index.text
+import zope.interface
+
+import zope.app.container.contained
+import zope.app.catalog.attribute
+import zope.app.catalog.interfaces
+
+class ITextIndex(zope.app.catalog.interfaces.IAttributeIndex,
+                 zope.app.catalog.interfaces.ICatalogIndex):
+    """Interface-based catalog text index
+    """
+
+class TextIndex(zope.app.catalog.attribute.AttributeIndex,
+                zope.index.text.TextIndex,
+                zope.app.container.contained.Contained):
+
+    zope.interface.implements(ITextIndex)

Modified: Zope3/branches/jim-index/src/zope/app/container/constraints.py
===================================================================
--- Zope3/branches/jim-index/src/zope/app/container/constraints.py	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/app/container/constraints.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -150,7 +150,11 @@
    $Id$
    """
 
-import zope.interface
+import sys
+
+import zope.interface.advice
+import zope.schema
+
 from zope.app.container.interfaces import InvalidItemType, InvalidContainerType
 from zope.app.i18n import ZopeMessageIDFactory as _
 from zope.app.container.interfaces import IContainer
@@ -303,6 +307,49 @@
         raise InvalidItemType(container, factory, self.types)
 
 
+def contains(*types):
+    """Declare that a container type contains only the given types
+
+    This is used within a class suite defining an interface to create
+    a __setitem__ specification with a precondition allowing only the
+    given types:
+
+      >>> class IFoo(zope.interface.Interface):
+      ...     pass
+      >>> class IBar(zope.interface.Interface):
+      ...     pass
+      >>> class IFooBarContainer(IContainer):
+      ...     contains(IFoo, IBar)
+
+      >>> __setitem__ = IFooBarContainer['__setitem__']
+      >>> __setitem__.getTaggedValue('precondition').types == (IFoo, IBar)
+      True
+
+    It is invalid to call contains outside a class suite:
+
+      >>> contains(IFoo, IBar)
+      Traceback (most recent call last):
+      ...
+      TypeError: contains not called from suite
+    """
+
+    frame = sys._getframe(1)
+    f_locals = frame.f_locals
+    f_globals = frame.f_globals
+    
+    if not (f_locals is not f_globals
+            and f_locals.get('__module__')
+            and f_locals.get('__module__') == f_globals.get('__name__')
+            ):
+        raise TypeError("contains not called from suite")
+
+    def __setitem__(key, value):
+        pass
+    __setitem__.__doc__ = IContainer['__setitem__'].__doc__
+    __setitem__.precondition = ItemTypePrecondition(*types)
+    f_locals['__setitem__'] = __setitem__
+
+
 class ContainerTypesConstraint:
     """Constrain a container to be one of a number of types
 
@@ -337,3 +384,48 @@
                return True
        else:
            raise InvalidContainerType(object, self.types)
+
+
+def containers(*types):
+    """Declare the container types a type can be contained in
+
+    This is used within a class suite defining an interface to create
+    a __parent__ specification with a constraint allowing only the
+    given types:
+
+      >>> class IFoo(IContainer):
+      ...     pass
+      >>> class IBar(IContainer):
+      ...     pass
+      
+      >>> from zope.app.container.interfaces import IContained
+      >>> class IFooBarContained(IContained):
+      ...     containers(IFoo, IBar)
+
+      >>> __parent__ = IFooBarContained['__parent__']
+      >>> __parent__.constraint.types == (IFoo, IBar)
+      True
+
+    It is invalid to call containers outside a class suite:
+
+      >>> containers(IFoo, IBar)
+      Traceback (most recent call last):
+      ...
+      TypeError: containers not called from suite
+    """
+
+    frame = sys._getframe(1)
+    f_locals = frame.f_locals
+    f_globals = frame.f_globals
+    
+    if not (f_locals is not f_globals
+            and f_locals.get('__module__')
+            and f_locals.get('__module__') == f_globals.get('__name__')
+            ):
+        raise TypeError("containers not called from suite")
+
+    __parent__ = zope.schema.Field(
+        constraint = ContainerTypesConstraint(*types)
+        )
+    f_locals['__parent__'] = __parent__
+

Modified: Zope3/branches/jim-index/src/zope/app/uniqueid/__init__.py
===================================================================
--- Zope3/branches/jim-index/src/zope/app/uniqueid/__init__.py	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/app/uniqueid/__init__.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -113,13 +113,13 @@
         return self.object
 
     def __hash__(self):
-        return self.object._p_oid
+        return hash(self.object._p_oid)
 
     def __cmp__(self, other):
         if not isinstance(other, ReferenceToPersistent):
             raise TypeError("Cannot compare ReferenceToPersistent with %r" %
                             (other,))
-        return cmp(self.__hash__(), other.__hash__())
+        return cmp(self.object._p_oid, other.object._p_oid)
 
 
 def connectionOfPersistent(ob):

Modified: Zope3/branches/jim-index/src/zope/app/zptpage/textindex/configure.zcml
===================================================================
--- Zope3/branches/jim-index/src/zope/app/zptpage/textindex/configure.zcml	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/app/zptpage/textindex/configure.zcml	2004-08-16 13:40:02 UTC (rev 27144)
@@ -2,7 +2,7 @@
 
   <adapter
       for="..interfaces.IZPTPage"
-      provides="zope.app.index.interfaces.text.ISearchableText"
+      provides="zope.index.interfaces.searchabletext.ISearchableText"
       factory=".zptpage.SearchableText"
       />
 

Modified: Zope3/branches/jim-index/src/zope/app/zptpage/textindex/tests.py
===================================================================
--- Zope3/branches/jim-index/src/zope/app/zptpage/textindex/tests.py	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/app/zptpage/textindex/tests.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -16,7 +16,7 @@
 $Id$
 """
 
-from zope.app.index.interfaces.text import ISearchableText
+from zope.index.interfaces.searchabletext import ISearchableText
 from zope.app.tests import ztapi
 from zope.app.tests.placelesssetup import PlacelessSetup
 from zope.app.zptpage.interfaces import IZPTPage

Modified: Zope3/branches/jim-index/src/zope/app/zptpage/textindex/zptpage.py
===================================================================
--- Zope3/branches/jim-index/src/zope/app/zptpage/textindex/zptpage.py	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/app/zptpage/textindex/zptpage.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -17,7 +17,7 @@
 
 from zope.interface import implements
 from zope.app.zptpage.interfaces import IZPTPage
-from zope.app.index.interfaces.text import ISearchableText
+from zope.index.interfaces.searchabletext import ISearchableText
 import re
 
 tag = re.compile(r"<[^>]+>")

Modified: Zope3/branches/jim-index/src/zope/index/field/__init__.py
===================================================================
--- Zope3/branches/jim-index/src/zope/index/field/__init__.py	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/index/field/__init__.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -1,2 +1 @@
-#
-# This file is necessary to make this directory a package.
+from zope.index.field.index import FieldIndex

Modified: Zope3/branches/jim-index/src/zope/index/interfaces/__init__.py
===================================================================
--- Zope3/branches/jim-index/src/zope/index/interfaces/__init__.py	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/index/interfaces/__init__.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -40,6 +40,10 @@
         If docid does not exist, KeyError is raised.
         """
 
+    def clear():
+        """Unindex all documents indexed by the index
+        """
+
 class IQuerying(Interface):
     """An index that can be queried by some text and returns a result set."""
 

Added: Zope3/branches/jim-index/src/zope/index/interfaces/searchabletext.py
===================================================================
--- Zope3/branches/jim-index/src/zope/index/interfaces/searchabletext.py	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/index/interfaces/searchabletext.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -0,0 +1,31 @@
+##############################################################################
+#
+# Copyright (c) 2002 Zope Corporation and Contributors.
+# All Rights Reserved.
+#
+# This software is subject to the provisions of the Zope Public License,
+# Version 2.0 (ZPL).  A copy of the ZPL should accompany this distribution.
+# THIS SOFTWARE IS PROVIDED "AS IS" AND ANY AND ALL EXPRESS OR IMPLIED
+# WARRANTIES ARE DISCLAIMED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF TITLE, MERCHANTABILITY, AGAINST INFRINGEMENT, AND FITNESS
+# FOR A PARTICULAR PURPOSE.
+#
+##############################################################################
+"""Interfaces related to text indexing and searching.
+
+$Id: interfaces.py 25353 2004-06-11 15:22:11Z gintautasm $
+"""
+from zope.interface import Interface
+
+class ISearchableText(Interface):
+    """Interface that text-indexable objects should implement."""
+
+    def getSearchableText():
+        """Return a sequence of unicode strings to be indexed.
+
+        Each unicode string in the returned sequence will be run
+        through the splitter pipeline; the combined stream of words
+        coming out of the pipeline will be indexed.
+
+        returning None indicates the object should not be indexed
+        """

Modified: Zope3/branches/jim-index/src/zope/index/keyword/__init__.py
===================================================================
--- Zope3/branches/jim-index/src/zope/index/keyword/__init__.py	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/index/keyword/__init__.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -1,2 +1 @@
-#
-# This file is necessary to make this directory a package.
+from zope.index.keyword.index import KeywordIndex

Copied: Zope3/branches/jim-index/src/zope/index/keyword/tests.py (from rev 25353, Zope3/branches/jim-index/src/zope/index/keyword/tests/test_keywordindex.py)

Modified: Zope3/branches/jim-index/src/zope/index/text/__init__.py
===================================================================
--- Zope3/branches/jim-index/src/zope/index/text/__init__.py	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/src/zope/index/text/__init__.py	2004-08-16 13:40:02 UTC (rev 27144)
@@ -1,2 +1 @@
-#
-# This file is necessary to make this directory a package.
+from zope.index.text.textindexwrapper import TextIndexWrapper as TextIndex

Added: Zope3/branches/jim-index/todo
===================================================================
--- Zope3/branches/jim-index/todo	2004-08-16 09:30:09 UTC (rev 27143)
+++ Zope3/branches/jim-index/todo	2004-08-16 13:40:02 UTC (rev 27144)
@@ -0,0 +1,17 @@
+This is a to-do list for this branch
+
+DON'T MERGE IT TO THE TRUNK!
+
+- zope.app.uniqueid needs to be restructured:
+
+  - persistent.py, persistent uid adapter
+    persistent.txt doctest (incl hash)
+
+  - registry.py, the unique id registry
+    registry.txt doctest
+
+  Rename UniqueIdUtility to UniqueIds
+
+- ftest for catalog
+
+- Define catalog tool



More information about the Zope3-Checkins mailing list