[Zope-dev] Re: [Repoze-dev] repoze.bfg

[Ian Bicking]

Chris McDonough wrote:
>> There's been a tension in the opencore stuff with the catalog, mostly 
>> that it's easy to setup and use for things, but it doesn't really work 
>> for things outside of the ZODB.  Or, I guess theoretically you could 
>> catalog things not in the ZODB, but it's never happened.
> 
> IMO, mostly it's a matter of deciding what "index" and "catalog" means 
> for searching.  If you only need full-text search, some indexing systems 
> are totally inappropriate.  Likewise, if you only need "field" indexes 
> that match just one particular value, it's sometimes vastly simpler just 
> to come up with your own system based on, e.g. a relational table, than 
> it is to try to use somebody else's "indexer" or "catalog".

They are similar in that they both need to get information about 
updates, and a way to reindex.  Full text search can be a little lazier, 
as being 100% up-to-date isn't such a big issue.

>> That said, there's a real need for cross-system indexing of different 
>> kinds.  There's text search indexes, but other kinds of more strict 
>> indexes also make sense.  It would be very handy to have an index that 
>> wasn't tied to the ZODB, a database, or anything else.  (It could be 
>> implemented using the ZODB or a database, of course.)
> 
> Xapian seems like a good choice too.  It has its own persistence 
> mechanism and reasonable Python bindings (although from experience maybe 
> slightly memory-leaky).

Yes, once you get the documents into it.  Actually, it's really a 
many-to-many notification system that's needed.  We have one that needs 
documenting and review 
(http://www.openplans.org/projects/cabochon/project-home) but while it 
handles notifications and events (as do several other systems), that 
doesn't cover reindexing the site.  And then you also need a set of 
useful endpoints, but those can grow over time.

-- 
Ian Bicking : ianb at colorstudy.com : http://blog.ianbicking.org