My 4 debugging rules

1. Check what is actually sent over the network and what is actually stored on file/in a database

This is your data. Everything else is just crunching that data.

Check so that there isn't something wrong here, because if there is you fall victim to the GIGO (Garbage In, Garbage Out) principle, and it does not matter how well the code itself is written. I once had a problem with Internet Explorer (IE7 I think) not displaying stuff the same way as the other browsers. Turned out after quite some debugging, that it was sent a subtly different version of the web page.

2. If you're looking at something for more than 40 minutes, you're looking in the wrong place

3. There is no such thing as rollback - switch back instead

When rolling back an upgrade: Replace with previous binary structure instead.

Back when I installed and upgraded Plone based systems, there was of course the version control system, but also the ZODB appending database, hundreds of python eggs, some files and the operating system libraries. All changes in these can be rolled back, but did you miss anything? Is something somewhere not pinned properly? If so, how long would it take you to find it? If you just make a binary copy of the old system (to the extent that you can), you can just switch back to that system.

4: Testing must be done as close to the user as possible

It does not matter how good the system is under the hood, unless the last mile to the user is also in perfect shape.