[Zope] Cataloging/ignoring objects that generate errors on in
dexing
Chris McDonough
chrism at zope.com
Wed Oct 1 11:41:22 EDT 2003
Bare except clauses cause other problems, like hiding conflict errors.
But if this works for you, I'd say run with it. The other option is to
fix the security problem which is throwing the KeyError.
- C
On Wed, 2003-10-01 at 11:32, Jean-Francois.Doyon at CCRS.NRCan.gc.ca wrote:
> OK, I've found the code that is causing this to happen:
>
> Zope 2.6.1, in Products/PluginIndexes/TextIndex/TextIndex.py, Line 291:
>
> except (AttributeError, TypeError):
>
> For some reason, the index_object method catches specific exceptions, not
> ALL exceptions. If I change this to a simple "except:", then my problem
> goes away.
>
> Conceptually, I would think this is better ... Just ignore ANY error that
> might occur and don't index the object ... Why only handle specific
> exceptions ?
>
> Is this a bug ?
>
> Thanks,
>
> J.F.
>
> -----Original Message-----
> From: Jean-Francois.Doyon at CCRS.NRCan.gc.ca
> [mailto:Jean-Francois.Doyon at CCRS.NRCan.gc.ca]
> Sent: Monday, September 29, 2003 1:15 PM
> To: chrism at zope.com
> Cc: zope at zope.org
> Subject: RE: [Zope] Cataloging/ignoring objects that generate errors on
> in dexing
>
>
> Hello,
>
> Here is the traceback, but in case that's confusing, find the code that
> modifies the CMF Document below as well:
>
> Traceback (innermost last):
>
> Module ZPublisher.Publish, line 49, in publish
> Module ZPublisher.mapply, line 32, in mapply
> Module ZPublisher.Publish, line 38, in call_object
> Module Products.ZCatalog.ZCatalog, line 292, in manage_catalogFoundItems
> Module Products.ZCatalog.ZCatalog, line 656, in ZopeFindAndApply
> Module Products.ZCatalog.ZCatalog, line 656, in ZopeFindAndApply
> Module Products.ZCatalog.ZCatalog, line 656, in ZopeFindAndApply
> Module Products.CMFCore.CatalogTool, line 226, in catalog_object
> Module Products.ZCatalog.ZCatalog, line 486, in catalog_object
> Module Products.ZCatalog.Catalog, line 299, in catalogObject
> Module Products.PluginIndexes.TextIndex.TextIndex, line 271, in index_object
>
> Module Products.Atlas, line 47, in SearchableText
> Module Products.Atlas, line 34, in CookedBody
> Module Shared.DC.Scripts.Bindings, line 250, in __call__
> Module Shared.DC.Scripts.Bindings, line 270, in _bindAndExec
> Module Products.PageTemplates.ZopePageTemplate, line 198, in _exec
> Module Products.PageTemplates.PageTemplate, line 79, in pt_render
> <ZopePageTemplate at /english/learningresources/quizzes/questions.html/>
> Module TAL.TALInterpreter, line 196, in __call__
> Module TAL.TALInterpreter, line 229, in interpret
> Module TAL.TALInterpreter, line 633, in do_loop_tal
> Module Products.PageTemplates.TALES, line 201, in setRepeat
> Module Products.PageTemplates.TALES, line 211, in evaluate
> URL:
> Line 5, Column 0
> Expression: <PythonExpr
> here.quizquery(mode='getquestions',cat=int(request.form['category']),dif=int
> (request.form['difficulty']),limit=int(request.form['num']))>
> Names:
> {'container': <Document at
> /english/learningresources/quizzes/questions.html>,
> 'default': <Products.PageTemplates.TALES.Default instance at 0x8657204>,
> 'here': <Document at /english/learningresources/quizzes/questions.html>,
> 'loop': <SafeMapping instance at 94b8a60>,
> 'modules': <Products.PageTemplates.ZRPythonExpr._SecureModuleImporter
> instance at 0x8656fe4>,
> 'nothing': None,
> 'options': {'args': ()},
> 'repeat': <SafeMapping instance at 94b8a60>,
> 'request': <HTTPRequest,
> URL=http://atlas.gc.ca/newsite/english/portal_catalog/manage_catalogFoundIte
> ms>,
> 'root': <Application instance at 8d0a9e8>,
> 'template': <ZopePageTemplate at
> /english/learningresources/quizzes/questions.html/>,
> 'traverse_subpath': [],
> 'user': admin}
> Module Products.PageTemplates.ZRPythonExpr, line 43, in __call__
> __traceback_info__:
> here.quizquery(mode='getquestions',cat=int(request.form['category']),dif=int
> (request.form['difficulty']),limit=int(request.form['num']))
> Module Python expression
> "here.quizquery(mode='getquestions',cat=int(request.form['category']),dif=in
> t(request.form['difficulty']),limit=int(request.form['num']))", line 2, in f
>
> Module AccessControl.ZopeGuards, line 75, in guarded_getitem
> KeyError: category
>
> And, in my product's __init__ , I have :
>
> # Monkey Patch to turn CMF Documents into ZPT's, give them Properties, and
> fix up the indexing to strip HTML
>
> originalCookedBody = Document.CookedBody
> originalManageOptions = Document.manage_options
>
> def CookedBody(self, stx_level=None, setlevel=0):
> # First, run through the original logic
> cookedbody = originalCookedBody(self, stx_level, setlevel)
> # Then if document is HTML, run it through the ZPT rendering engine
> if ( self.text_format == 'html' ):
> pt =
> ZopePageTemplate('',cookedbody,'text/html').__of__(self)
> cookedbody = pt()
> return cookedbody
>
> Document.CookedBody = CookedBody
>
> Document.manage_options = ( originalManageOptions +
> PropertyManager.manage_options )
>
> def SearchableText(self):
> return "%s %s %s %s" %
> (self.Title(),self.Description(),stripper(self.CookedBody()),self.Keywords()
> )
>
> Document.SearchableText = SearchableText
>
> ...
>
> The thought had occured to me that maybe the catalog only ignores certain
> types of exceptions, but I mean ... a KeyError is pretty standard, and
> precisely the kind of exception I'd expect the catalog to ignore.
>
> Maybe I can trap the erro and raise some other exception ? Or maybe there's
> a bug that exception handling isn't done right ?
>
> Thanks in advance for any help!
>
> J.F.
>
> -----Original Message-----
> From: Chris McDonough [mailto:chrism at zope.com]
> Sent: Friday, September 26, 2003 2:01 PM
> To: Jean-Francois.Doyon at CCRS.NRCan.gc.ca
> Cc: zope at zope.org
> Subject: Re: [Zope] Cataloging/ignoring objects that generate errors on
> indexing
>
>
> Can you supply a traceback? I've always been under the same impression
> as you....
>
> On Fri, 2003-09-26 at 12:15, Jean-Francois.Doyon at CCRS.NRCan.gc.ca wrote:
> > Hello,
> >
> > I'm trying to update my catalog, and much to my surprise, it ain't working
> !
> >
> > Some objects require form parameters and so on, raise errors when none are
> > provided. So now when I ask the catalog (In this case a CMF one, but the
> > fact it's CMF shouldn't really matter, right?) to index everything it can
> > find, it raises an error when it lands on such objects.
> >
> > Now this has never happened ot me before, and I'd always assumed that
> > objects that raised errors while being index simply didn't get indexed,
> > hence my suprise. I was actually counting on such doucments NOT getting
> > indexed BECAUSE they raise errors.
> >
> > Admitedly, I'm doing something somewhat out of the ordinary. I have a CMF
> > Document that actually renders/interprets TAL (Basically a CMF Document
> > ZPT). SearchableText() runs a CookedBody that does the ZPT rendering ...
> Of
> > course NOT using CookedBody at indexing might be an option, but I'm not
> sure
> > of the consequences ... Some documents that have simple
> > tal:replace/tal:content type instructions in them I DO want to have Cooked
> > before indexing ...
> >
> > Anybody have any good ways or suggestions on how to get around this ?
> >
> > Any help would be much appreciated!
> >
> > Thanks,
> >
> > Jean-François Doyon
> > Internet Service Development and Systems Support
> > GeoAccess Division
> > Canadian Center for Remote Sensing
> > Natural Resources Canada
> > http://atlas.gc.ca
> > Phone: (613) 992-4902
> > Fax: (613) 947-2410
> >
> >
> > _______________________________________________
> > Zope maillist - Zope at zope.org
> > http://mail.zope.org/mailman/listinfo/zope
> > ** No cross posts or HTML encoding! **
> > (Related lists -
> > http://mail.zope.org/mailman/listinfo/zope-announce
> > http://mail.zope.org/mailman/listinfo/zope-dev )
--
Chris McDonough <chrism at zope.com>
Zope Corporation
More information about the Zope
mailing list