[Zope] More on understanding conflicts

[Chris McDonough]

> The structure of the method is simple enough: there is a large  
> <dtml-let>
> block which populates local variables with data from the session
> variables
>
> 	<dtml-let foobar="getSessionVariable('foobar')"
>          ...
>>
> 	
> with the body of the <dtml-let> containing 300 lines or so of dtml,
> control code, sql queries, and occasional dtml-let blocks to hold
> query results and computed values.

If this is the same getSessionVariable as you reported before (a  
Python Script), note that each call that you make to it has  
overhead.  It might be better to create a single Python script to  
return all of the values that you need from the session rather than  
calling this over and over again.  If that doesn't work, it might be  
even better to just do SESSION.get('foobar') instead of calling a  
Python Script for every value.

> So, this navigation method is clearly a good candidate for  
> refactoring,
> but the question is "What's the best way to refactor it?".

Making it do its job faster will likely help.  Likewise,

> I still don't have an intuitve understanding of the conflict  
> mechanism in
> practice.  For example, Why is the conflict error reported for the
> navigation dtml method rather than for the getSessionVariable python
> script which actually references the SESSION object where the conflict
> occurs?

Write conflicts happen for a transaction.  In Zope, there is one  
transaction per request.  So it reports the URL of the method that  
was called as the thing that caused the conflict (because, basically,  
it was).  A conflict is not "noticed" until a transaction is  
committed; ZODB has no idea what caused the data structure to change  
in a conflicting way within "application code".

> A conflict error gets raised whenever another thread references the  
> same
> session's SESSION object.

Are you using frames by any chance?  It would be understandable that  
you're seeing an increased number conflicts if you have an  
application in which many threads try to access the same session  
object at once due to a browser making multiple writes to the same  
session object because of frames.

>   Apparently the MVCC read-read conflict
> resolution fails for session variables although I am not sure why.

Can we see one of the conflict tracebacks?  You should be seeing only  
write conflicts.  Read conflict errors are reported in a traceback  
with a ReadConflictError message (conflicts reported as only  
ConflictError are write conflicts).

>   The
> DEBUG logs indicate that read-read conflicts found.  It would be  
> helpful
> to understand why.

Yes.  A traceback would help.

> When the conflict is resolved, one is backed out (everything is
> transactional) and restarted.  After three retries, it is considered
> an error.   This processing is handled automatically by Zope.

That's right.

> At one point, I had considered writing a custom conflict resolution
> function where resolution assumed independence of individual session
> variables.  After some thought and experimentation, I concluded  
> that was a
> bad idea because it could break synchronization assertions between  
> session
> variables (e.g., X is twice Y).

That's a good conclusion.

> Absent a magical conflict resolution method, the best approach  
> would seem
> to be to minimize the potential for conflicts.  Intuition says that  
> the
> simplest way to do this would be to have the navigation dtml method  
> make a
> single call to a python script, getSessionVariables, which would  
> return a
> copy of the session variables. The copy could then be unpacked and
> processed without any potential conflict errors.

Heh.  Good thinking! ;-)   Really, anything that makes your  
transaction finish faster is the right thing (and good coding  
practice to boot).  Maybe not a silver bullet especially if you're  
using frames but it's a start.

- C