[Zope] Log aggregator?

[Dennis Allison]

When you get a solution let me know.  I am in the middle of log hell 
at the moment (including Zopes logs) trying to get things to sync up
properly across multiple machines.

When my stuff works, I'll share. 

	-d


On Fri, 23 Sep 2005, Paul Winkler wrote:

> Anybody know of any software that will aggregate logs in a variety
> of formats and allow me to correlate events in one log with another?
> 
> I'm getting really REALLY tired of eyeballing and grepping a multitude
> of logs on different servers, mentally translating incorrect timestamps,
> and trying to work out what the heck from one log corresponds to something
> in another.  Anytime something goes wrong, i waste a huge amount of time
> on this.
> 
> Well, duh, this should be automated, right?
> 
> I googled a bit and came up with nothing useful;
> then I started hacking up a python script, which I'm sure I could
> do in an hour or three, but - blah, it would be a huge timesaver if
> somebody had written something like this already.
> 
> I need to handle:
> 
> * Different log formats.  I'm mainly interested in the zope access log
> format (aka apache format) and the zope event log format.
> Must tolerate malformed lines like the #!@)%* ftp lines in the zope access
> log.
> 
> * System clocks being incorrect.
> On some of the servers I work with, I am not the primary admin and can't do
> much about the %@W#% clock being wrong. (*cough* firewall blocking
> ntp for no good reason *cough*)
> So I'd like the software to let me specify how far wrong
> the various logs are, and compensate.
> 
> * Servers in different time zones.  Rare for me, but it does come up.
> 
> * Filtering based on arbitrary strings, or (better) regular expressions,
> on a per-log basis.
> 
> * Multi-megabyte logs. Sometimes i go to check a log and sysadmin has
> forgotten to rotate ever...
> 
> -PW
> 
> 
> 
> _______________________________________________
> Zope maillist  -  Zope at zope.org
> http://mail.zope.org/mailman/listinfo/zope
> **   No cross posts or HTML encoding!  **
> (Related lists - 
>  http://mail.zope.org/mailman/listinfo/zope-announce
>  http://mail.zope.org/mailman/listinfo/zope-dev )
> 

--