How slow should dtml-var be? (was: RE: [Zope] Problems with *BIG* SQL queries)

[Mayers, Philip J]

As mentioned, I'm doing really big SQL queries which return a HTML report
that's about 1Mb. The ZopeTime trick suggested by Alan helps a lot - the
time spent really is on the Zope side, and happens even with the most
minimal file - just a:

<dtml-var "ZopeTime()">
<dtml-in QUERY>
<dtml-var column1>,<dtml-var column2>,<dtml-var column3>,<dtml-var column4>
</dtml-in>
<dtml-var "ZopeTime()">

This takes about 4 minutes *on the server* - file return is quick. *Sigh*.
This however:

<dtml-var "ZopeTime()">
<dtml-in QUERY>
</dtml-in>
<dtml-var "ZopeTime()">

Takes about 30 seconds (which is reasonable):

HTTP/1.1 200 OK
Date: Wed, 21 Mar 2001 14:02:07 GMT
Server: Apache/1.3.14 (Unix)  (Red-Hat/Linux)
X-Powered-By: Zope (www.zope.org), Python (www.python.org)
Vary: Authorization
Content-Length: 94
Connection: close
Content-Type: text/plain

2001/03/21 14:02:07.2783 GMT
2001/03/21 14:02:46.6733 GMT

I can only assume it's the namespace lookup in <dtml-var variable> - in
fact, my calculations (9575 rows, 4 <dtml-var>s per row) make that 6ms per
dtml-var, which isn't unreasonable. I guess I really do just have too much
data...

Using a PythonScript:

print context.ZopeTime()
for result in container.QUERY():
    print "%s,%s,%s,%s" % (r.column1,r.column2,r.column3,r.column4)
print context.ZopeTime()
return printed

...doesn't seem to help

Regards,
Phil

+----------------------------------+
| Phil Mayers, Network Support     |
| Centre for Computing Services    |
| Imperial College                 |
+----------------------------------+