In recent releases Oracle has silently changed the behavior of ITL waits. The aim of this post it to describe what has changed and why. But, first of all, let’s review some essential concepts about ITLs and ITL waits.
Interested Transaction List
The Oracle database engine locks the data modified by a transaction at the row level. To implement this feature every data block contains a list of all transactions that are modifying it. This list is commonly called interested transaction list (ITL). Its purpose is twofold. First, it is used to store information to identify a transaction as well as a reference to access the undo data associated to it. Second, it is referenced by every modified or locked row to indicate which transaction it is involved.
INITRANS
The initial number of slots composing the ITL is set through the INITRANS parameter. Even though it can be set to 1, which is the default value as well, as of 9i at least 2 slots are always created. Note that the data dictionary lies to us on this matter. In fact, as shown in the following example, the data dictionary shows the value specified when the object was created and not the actual number of slots.
SQL> CREATE TABLE t (n NUMBER) INITRANS 1; SQL> SELECT ini_trans FROM user_tables WHERE table_name = 'T'; INI_TRANS ---------- 1
MAXTRANS
There is a maximum number of slots an ITL can contain. The actual maximum number depends on the blocks size. For example, an 8KB block can have up to 169 slots. Up to 9i the maximum is limited by the MAXTRANS parameter as well. As of 10g, however, this parameter is deprecated and, therefore, no longer honored. In the same way as for INITRANS, the data dictionary shows the value specified when the object was created and not the actual maximum number of slots.
Also note that while creating an object the database engine checks whether the MAXTRANS value is not greater than 255. And, if it is greater, it raises an ORA-02209 (invalid MAXTRANS option value).
ITL Waits
When a session requires a slot but all the available ones are in use by other active transactions, the database engine tries to dynamically create a new slot. This is of course only possible when a) the maximum number of slots was not already allocated b) enough free space (one slot occupies 24 bytes) is available in the block itself. If a new slot cannot be created, the session requiring it hits a so-called ITL wait. Note that the name of the actual wait event is called “enq: TX – allocate ITL entry”.
It is essential to point out that a session does not wait on the first slot becoming free. Instead, it probes, round-robin, the available slots to find out one that becomes free. And, while doing so, it waits few seconds on every one it probes. When during this short wait the slot becomes free, it uses it. Otherwise, it tries with another slot.
The actual implementation for finding a free slot is what Oracle changed in recent releases. So, let’s describe what the behavior in recent releases is.
ITL Waits in 11gR1
In 11.1.0.6 and 11.1.0.7 a session waits at most one time on every slot. For all slots but one it waits up to 5 seconds. For the other one it might wait indefinitely. The following pseudo code illustrates this (you should consider the variable called “itl” as an array referencing/containing all ITL slots).
FOR i IN itl.FIRST..itl.LAST LOOP EXIT WHEN itl(i) IS FREE IF i <> itl.LAST THEN WAIT ON itl(i) FOR 5 SECONDS ELSE WAIT ON itl(i) FOREVER END IF END LOOP
The problem of this algorithm is that an “unlucky” session might wait much longer than necessary. In fact, once it enters the WAIT FOREVER status, it no longer considers the other slots.
ITL Waits in 11gR2
In 11.2.0.1 and 11.2.0.2 a session might wait several times for the same slot. Initially the wait is short. As the time passes, the wait time increases exponentially based on the formula “wait time = power(2,iteration-1)”. For all slots but one there is a maximum wait time of 5 seconds, though. For the other one, and for the first 10 iterations only, the wait time is computed with the very same formula. Then, during the 11th iteration, the session waits indefinitely. The following pseudo code illustrates this.
iteration = 0 LOOP iteration++ FOR i IN itl.FIRST..itl.LAST LOOP EXIT WHEN itl(i) IS FREE IF i <> itl.LAST THEN WAIT ON itl(i) FOR min(power(2,iteration-1),5) SECONDS ELSIF iteration <= 10 THEN WAIT ON itl(i) FOR power(2,iteration-1) SECONDS ELSE WAIT ON itl(i) FOREVER END IF END LOOP EXIT WHEN free_itl_found END LOOP
The advantage of this algorithm is that a session might probe several time all the available slots and, as a result, enters the WAIT FOREVER status after about 20 minutes only.
ITL Waits in 9i/10g
Up to 10.2.0.4 the behavior is similar to 11gR1. The only noticeable difference is that the wait time is not always 5 seconds. Instead, it is either 3 or 6 seconds. I was not able to spot a rule behind the choice between the two durations. So, there might be some randomness involved.
In 10.2.0.5 the behavior is similar to 11gR2. Also in this case the only noticeable difference is that the maximum wait time is not always 5 seconds. Instead, as in releases up to 10.2.0.4, it is either 3 or 6 seconds.
Great article.
[…] Antognini and Oracle Internals are the perfect match. This time ITL Waits are in rendezvous with […]
Hi Christian,
Great article. I am always puzzled at:
1) How did you come across this change in 11gR1/R2 etc? i.e. what clue directed you to this discovery as such change will hardly be visible while trying to debug waits unless we went into deep tracing of each wait and found the route TX slot number for each wait.
2) How did you ascertain your logic without having access to Oracle code?
I would love to do simillar debugging of my own and always wonder how can I get inside the code. Do you use something like dtrace, gdb etc?
Thanks in advance,
Paresh
Hi Peresh
> How did you come across this change in 11gR1/R2 etc?
I have a demo about ITL waits/deadlocks that I use from time to time while teaching. When I used it with old releases it always worked. With newer one, it was almost never the case… Hence I investigated while it was failing.
> How did you ascertain your logic without having access to Oracle code?
There is no magic. I wrote a test case that reproduce ITL waits and a piece of code that monitors them. Since it is a bit too long to be posted in a comment, I wrote a blog entry about it.
HTH
Chris
Hi Christian,
Thanks again for your reply and the new blog entry with the shell script. From the output of the script I can see what you wrote about the waits above and “no magic” part (there is magic there, your well crafted shell script ;)). Thanks for sharing your expertise.
Regards,
Paresh
[…] Antognini and Oracle Internals are the perfect match. This time ITL Waits are in rendezvous with […]