[Zope] DESPERATION: ZCatalog search madness

[Darran Edmundson]

I've been tottering on the edge of sanity for a couple of
days now, the launch of my site delayed indefinitely by
the most bizarre problems with ZCatalog searching.  Nearly
all of the content I generate results from searches of
catalogued data and without this I am sunk.  Imagine a
list of recent publications or a notice for forthcoming
seminars that returns null or incorrect records.

As an example, consider the following catalog of Journal
instances.  (Journal is a catalogAware ZClass.) I have a 
catalog 'publicationCatalog' with 3 catalogued objects.
Excerpts from my index status tab:

     3 object are indexed in Year (integer, FieldIndex) 
     3 object are indexed in Journal (string, FieldIndex) 
     3 object are indexed in Authors (string, TextIndex) 

Metatable data includes the above indexes as columns plus others.


The following baseline Author search returns all 3 records.

<dtml-in "publicationCatalog(Authors='Kivshar')">
   <dtml-var Authors>, <dtml-var Journal>, (<dtml-var Year>).<br>
</dtml-in>

Ostrovskaya Kivshar Chen Segev, Optics Letters, (2000).
Ostrovskaya Kivshar, Journal of Optics B: Quantum Semiclassical Optics, (1990).
Ostrovskaya Kivshar Skryabin Firth, Physical Review Letters, (1995).


Fine.  However, if I now try and order the Journals by either the Year 
or Journal FieldIndices, the result is an empty search result!  E.g., 

<dtml-in "publicationCatalog(Authors='Kivshar',sort_on='Year')">
   <dtml-var Authors>, <dtml-var Journal>, (<dtml-var Year>).<br>
</dtml-in>


Next I set all of the Journal instances to have the same year, 2000, so
that the baseline search returns:

<dtml-in "publicationCatalog(Authors='Kivshar')">
Ostrovskaya Kivshar, Journal of Optics B: Quantum Semiclassical Optics, (2000).
Ostrovskaya Kivshar Skryabin Firth, Physical Review Letters, (2000).
Ostrovskaya Kivshar Chen Segev, Optics Letters, (2000).

This search works:

<dtml-in "publicationCatalog(Year=2000,sort_on='Journal')">
Ostrovskaya Kivshar, Journal of Optics B: Quantum Semiclassical Optics, (2000).
Ostrovskaya Kivshar Chen Segev, Optics Letters, (2000).
Ostrovskaya Kivshar Skryabin Firth, Physical Review Letters, (2000).

Whereas this one returns an empty result:

<dtml-in "publicationCatalog(Authors='Kivshar',sort_on='Journal')">

and this one causes a key error:

<dtml-in "publicationCatalog(Authors='Kivshar',sort_on='Year')">

Traceback (innermost last):
  File /tmp_mnt/users/optics/www124/Zope-2.1.4-src/lib/python/ZPublisher/Publish.py, line
214, in publish_module
  File /tmp_mnt/users/optics/www124/Zope-2.1.4-src/lib/python/ZPublisher/Publish.py, line
179, in publish
  File /tmp_mnt/users/optics/www124/Zope-2.1.4-src/lib/python/Zope/__init__.py, line 202,
in zpublisher_exception_hook
    (Object: ElementWithAttributes)
  File /tmp_mnt/users/optics/www124/Zope-2.1.4-src/lib/python/ZPublisher/Publish.py, line
165, in publish
  File /tmp_mnt/users/optics/www124/Zope-2.1.4-src/lib/python/ZPublisher/mapply.py, line
160, in mapply
    (Object: recentPublications)
  File /tmp_mnt/users/optics/www124/Zope-2.1.4-src/lib/python/ZPublisher/Publish.py, line
102, in call_object
    (Object: recentPublications)
  File /tmp_mnt/users/optics/www124/Zope-2.1.4-src/lib/python/OFS/DTMLMethod.py, line 145,
in __call__
    (Object: recentPublications)
  File
/tmp_mnt/users/optics/www124/Zope-2.1.4-src/lib/python/DocumentTemplate/DT_String.py, line
502, in __call__
    (Object: recentPublications)
  File /tmp_mnt/users/optics/www124/Zope-2.1.4-src/lib/python/DocumentTemplate/DT_In.py,
line 672, in renderwob
    (Object: publicationCatalog(Authors='Kivshar',sort_on='Year'))
  File /tmp_mnt/users/optics/www124/Zope-2.1.4-src/lib/python/Products/ZCatalog/Lazy.py,
line 190, in __getitem__
KeyError: (see above)


I upgraded my Zope to 2.1.4 today hoping this would solve my problems but 
no such luck.  I've packed my database, changed types of my Journal instance data,
recreated my catalog, pored over the source, searched the archives, even cried a
little.  To paraphrase Dr. Evil, can someone PLEASE tell me what's wrong with my 
frickin "ZCatalog"?

In desperation,
Darran.