Feb 17 2009

Virtual Column-Based Partitioning Might Lead to Wrong Results

Tag: 11gR1, Bug, PartitioningChristian Antognini @ 7:53 am

As of Oracle Database 11g it is possible to use a virtual column as partition key. In this post I do not want to discuss how it works and whether this is good or not… Instead, I would like to show you that the feature might lead to wrong results.

First of all, I would like to show you a test where everything works fine. For that purpose, let’s create a table (notice the virtual column n2), insert one row into it, and gather the object statistics:

SQL> CREATE TABLE t (
  2    n1 NUMBER,
  3    n2 AS (CASE n1 WHEN 1 THEN 1 WHEN 2 THEN 2 ELSE 0 END) VIRTUAL
  4  )
  5  PARTITION BY LIST (n2) (
  6    PARTITION zero VALUES (0),
  7    PARTITION one VALUES (1),
  8    PARTITION two VALUES (2)
  9  )
 10  ENABLE ROW MOVEMENT;

SQL> INSERT INTO t (n1) VALUES (1);

SQL> COMMIT;

SQL> execute dbms_stats.gather_table_stats(user,'t')

The aim of the following test is to check whether row movement works correctly. Hence, I update the column n1 to cause such a movement. To check whether row movement is performed or not, I display the content of the two involved partitions before and after the update statement. In addition, I also display the rowids (because of the movement the row should get a new rowid).

SQL> SELECT rowid, n1, n2 FROM t PARTITION (zero);

no rows selected

SQL> SELECT rowid, n1, n2 FROM t PARTITION (one);

ROWID                      N1         N2
------------------ ---------- ----------
AAAE89AAEAAAAGNAAA          1          1

SQL> UPDATE t SET n1 = 3;

SQL> COMMIT;

SQL> SELECT rowid, n1, n2 FROM t PARTITION (zero);

ROWID                      N1         N2
------------------ ---------- ----------
AAAE88AAEAAAAF9AAA          3          0

SQL> SELECT rowid, n1, n2 FROM t PARTITION (one);

no rows selected

The previous test was successful. Now, let me show you a situation that leads to wrong results :-(

To reproduce the bug I basically execute the same operations as before. The only difference is that seven columns are added before the columns n1 and n2 in the table. Hence, the test table is recreated with the following statements:

SQL> DROP TABLE t PURGE;

SQL> CREATE TABLE t (
  2    d1 NUMBER,
  3    d2 NUMBER,
  4    d3 NUMBER,
  5    d4 NUMBER,
  6    d5 NUMBER,
  7    d6 NUMBER,
  8    d7 NUMBER,
  9    n1 NUMBER,
 10    n2 AS (CASE n1 WHEN 1 THEN 1 WHEN 2 THEN 2 ELSE 0 END) VIRTUAL
 11  )
 12  PARTITION BY LIST (n2) (
 13    PARTITION zero VALUES (0),
 14    PARTITION one VALUES (1),
 15    PARTITION two VALUES (2)
 16  )
 17  ENABLE ROW MOVEMENT;

SQL> INSERT INTO t (n1) VALUES (1);

SQL> COMMIT;

SQL> execute dbms_stats.gather_table_stats(user,'t')

As before, I update the row to cause the movement and display the content of the two involved partitions before and after doing it.

SQL> SELECT rowid, n1, n2 FROM t PARTITION (zero);

no rows selected

SQL> SELECT rowid, n1, n2 FROM t PARTITION (one);

ROWID                      N1         N2
------------------ ---------- ----------
AAAE9BAAEAAAAGNAAA          1          1

SQL> UPDATE t SET n1 = 3;

SQL> COMMIT;

SQL> SELECT rowid, n1, n2 FROM t PARTITION (zero);

no rows selected

SQL> SELECT rowid, n1, n2 FROM t PARTITION (one);

ROWID                      N1         N2
------------------ ---------- ----------
AAAE9BAAEAAAAGNAAA          3          0

As you can see, the two queries after the update statement return wrong results. Also the rowid is the same. Hence, row movement was not performed. It goes without saying that also other queries might return wrong results. An example is the following:

SQL> SELECT rowid, n1, n2 FROM t WHERE n2 = 1;

ROWID                      N1         N2
------------------ ---------- ----------
AAAE9BAAEAAAAGNAAA          3          0

By playing around with the number of columns and position of the columns n1 and n2, I found out that depending on the situation you might have correct results or wrong results.

Since I was able to reproduce the problem with several databases (both 11.1.0.6 and 11.1.0.7), last Friday I opened a service request. Now the issue is tracked as bug# 8258501.


Feb 14 2009

Upcoming Speaking Engagements

Tag: SpeakingChristian Antognini @ 3:16 pm

In March I’ll be speaking at the following two events.

When Where What
2009-03-10 15:30-16:30 Hotsos Symposium, Dallas (USA) Bloom Filters
2009-03-18 16:00-16:45 SOUG-Tagung, Baden-Dättwil (CH) SQL Plan Baselines

The abstracts of the presentations are the following:

  • Bloom Filters: A bloom filter is a data structure used to support membership queries. Simply put, a bloom filter is used to test whether an element is a member of a given set or not. Since Oracle Database 10g Release 2, bloom filters are used in various situations. Unfortunately, no information about their usage is available in Oracle documentation.
    The aim of this presentation is to explain not only what bloom filters are, but also, and foremost, to describe how the database engine makes use of them. Specifically, it explains how the database engine uses bloom filters to reduce data communication between slave processes in parallel joins and to implement join-filter pruning.
  • SQL Plan Baselines: A SQL plan baseline is an object associated with a SQL statement that is designed to influence the query optimizer while it generates execution plans. As of Oracle Database 11g, SQL plan baselines substitute stored outlines. Actually, they can be considered an enhanced version of stored outlines.
    The aim of this presentation is to explain what SQL plan baselines are, how to create and manage them, and when to use them.

Feb 13 2009

Oracle AD4J Installation on Linux

Tag: Bug, Java, Oracle AD4JChristian Antognini @ 11:37 am

Today I tried to install Oracle AD4J on a Linux server that I have at home. The installation procedure is really simple and fully described here. Unfortunately, when I tried to access the console for the first time (that access is one of the installation steps), the HTTP server returned an internal server error (500). In the mod_jserv.log logfile I found the following error messages:

[13/02/2009 10:06:38:079] (EMERGENCY) ajp12: can not connect to host 127.0.0.1:3501
[13/02/2009 10:06:38:181] (EMERGENCY) ajp12: connection fail
[13/02/2009 10:06:38:181] (ERROR) an error returned handling request via protocol "ajpv12"

Mhmm… a listener should be available on port 3501. But, no such listener was available on my system (note that port 3500 is used for the HTTP listener):

oracle@helicon:/u00/app/oracle/product/ad4j/ [rdbms11107] netstat -l --numeric-ports | grep 350[01]
tcp        0      0 *:3500                      *:*                         LISTEN

A quick search in Metalink revealed that at least another person has hit the same issue few days ago (see bug# 8235076). Since OSS is still working on it, I’ll wait to see what the findings are. In the mean time, I was able to successfully install it on my Windows laptop.

Anyway, if somebody of you managed to successfully install AD4J on Linux, please, let me know!

ADDENDA (February 16th, 2009): Because of the comments of Charles and Michael I spent a bit more time looking at the problem. In fact, the first time I stopped immediately after seeing the bug in Metalink… I was lazy ;-). Hey, I try to optimize my worktime as well. Anyway, both suggested to manually start jserv. When I tried to do so, I received an error (at last). Based on it the problem was self explanatory! The java environment was causing the problem… In fact, with the default installation of CentOS 4.4 only the package java-1.4.2-gcj-compat was installed. After downloading and installing the most recent version of HotSpot (build 1.6.0_12-b04) the problem was solved.


Feb 09 2009

New Infrastructure

Tag: UncategorizedChristian Antognini @ 6:08 pm

This is just a short note to point out that I just moved this site to a new infrastructure (new hosting provider, new Wordpress version and new MySQL version). If the migration went fine, nobody should notice the difference ;-) If you have any problem, please, let me know!

For a couple of days (i.e. until all DNS are updated) I expect hits on both sites. Anyway, if you see this post, you are on the new one.


Feb 03 2009

11g New Feature in DBMS_MONITOR

Tag: 11gR1, SQL TraceChristian Antognini @ 1:41 am

As of 11g the package DBMS_MONITOR provides an important new feature. The aim of this post is to describe not only what this feature is, but also why it is important.

To illustrate how the new feature works, two things are necessary. First, a small table:

SQL> SELECT * FROM t;

         N
----------
         1
         2
         3
         4
         5

Second, an anonymous PL/SQL block. About it notice that:

  • Three queries using the very same cursor are executed.
  • The number of fetched rows is different for each execution.
  • Array processing is used to fetch all data in a single call.
  • The package DBMS_MONITOR is used to enable and disable SQL trace.

DECLARE
  l_cursor INTEGER;
  l_n dbms_sql.number_table;
  l_retval INTEGER;
BEGIN
  dbms_monitor.session_trace_enable;
  l_cursor := dbms_sql.open_cursor;
  dbms_sql.parse(l_cursor, 'SELECT n FROM t WHERE n <= :1', 1);
  dbms_sql.define_array(l_cursor, 1, l_n, 5, 1);
  FOR i IN 1..3
  LOOP
    dbms_sql.bind_variable(l_cursor, ':1', i);
    l_retval := dbms_sql.execute(l_cursor);
    l_retval := dbms_sql.fetch_rows(l_cursor);
  END LOOP;
  dbms_sql.close_cursor(l_cursor);
  dbms_monitor.session_trace_disable;
END;

Now, let’s start by executing the anonymous PL/SQL block in 10.2.0.4. In that version, the content of the generated trace file is the following. Notice that:

  • One single parse (PARSE line) was performed.
  • Three executions (EXEC lines) were performed.
  • Three fetches (FETCH lines) were performed. The first one fetched 1 row (value “r”), the second one fetched 2 rows, and the third one fetched 3 rows.
  • The information about the execution plan (STAT line; notice that I manually removed the runtime statistics and query optimizer estimations to keep the output more readable) was written in the trace file only when the cursor was closed. According to it, 6 rows were fetched (value “cnt”). In other words, 1+2+3.

PARSING IN CURSOR #18 len=29 dep=1 uid=28 oct=3 lid=28 tim=1204698401458764 hv=2067044879 ad='72de8330'
SELECT n FROM t WHERE n <= :1
END OF STMT
PARSE #18:c=0,e=427,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=1,tim=1204698401458760
EXEC #18:c=1000,e=912,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=1,tim=1204698401509992
FETCH #18:c=0,e=68,p=0,cr=3,cu=0,mis=0,r=1,dep=1,og=1,tim=1204698401510109
EXEC #18:c=0,e=12,p=0,cr=0,cu=0,mis=0,r=0,dep=1,og=1,tim=1204698401510184
FETCH #18:c=0,e=26,p=0,cr=3,cu=0,mis=0,r=2,dep=1,og=1,tim=1204698401510234
EXEC #18:c=0,e=11,p=0,cr=0,cu=0,mis=0,r=0,dep=1,og=1,tim=1204698401510290
FETCH #18:c=0,e=19,p=0,cr=3,cu=0,mis=0,r=3,dep=1,og=1,tim=1204698401510333
STAT #18 id=1 cnt=6 pid=0 pos=1 obj=15665 op='TABLE ACCESS FULL T'

And now, let’s execute it in 11.1.0.6. In this case notice that:

  • The same number of parse, execute and fetch calls as in 10.2.0.4 were performed.
  • The information about the execution plan was written in the trace file just after the first execution and not when the statement was closed. For this reason, according to it, only 1 row was fetched.

PARSING IN CURSOR #4 len=29 dep=1 uid=30 oct=3 lid=30 tim=1233611735664091 hv=2067044879 ad='6ae30880' sqlid='cxa35s1xm96hg'
SELECT n FROM t WHERE n <= :1
END OF STMT
PARSE #4:c=4000,e=118923,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=1,tim=1233611735664086
EXEC #4:c=1999,e=28275,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=1,tim=1233611735779067
FETCH #4:c=0,e=88,p=0,cr=3,cu=0,mis=0,r=1,dep=1,og=1,tim=1233611735779278
STAT #4 id=1 cnt=1 pid=0 pos=1 obj=17363 op='TABLE ACCESS FULL T'
EXEC #4:c=0,e=43,p=0,cr=0,cu=0,mis=0,r=0,dep=1,og=1,tim=1233611735795372
FETCH #4:c=0,e=45,p=0,cr=3,cu=0,mis=0,r=2,dep=1,og=1,tim=1233611735795449
EXEC #4:c=0,e=12,p=0,cr=0,cu=0,mis=0,r=0,dep=1,og=1,tim=1233611735795509
FETCH #4:c=0,e=18,p=0,cr=3,cu=0,mis=0,r=3,dep=1,og=1,tim=1233611735795552

Why this difference?

It is because, as of 11g, the procedures in the package dbms_monitor that are used to enable SQL trace accept an additional parameter: PLAN_STAT. As written in the documentation, this parameter is used to specify the frequency at which the row source statistics (i.e. information about execution plans) are written in trace files. The accepted values are the following (the default value is NULL):

  • NEVER: no information about the execution plan is written in trace files.
  • FIRST_EXECUTION (equivalent to NULL): information about the execution plan is written just after the first execution.
  • ALL_EXECUTIONS: information about the execution plan is written for every execution.

Therefore, when the parameter PLAN_STAT is set to ALL_EXECUTIONS, the content of the trace file is the following:

PARSING IN CURSOR #9 len=29 dep=1 uid=30 oct=3 lid=30 tim=1233613010415243 hv=2067044879 ad='6ae30880' sqlid='cxa35s1xm96hg'
SELECT n FROM t WHERE n <= :1
END OF STMT
PARSE #9:c=2000,e=28550,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=1,tim=1233613010415239
EXEC #9:c=2000,e=21361,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=1,tim=1233613010500308
FETCH #9:c=0,e=70,p=0,cr=3,cu=0,mis=0,r=1,dep=1,og=1,tim=1233613010500878
STAT #9 id=1 cnt=1 pid=0 pos=1 obj=17364 op='TABLE ACCESS FULL T'
EXEC #9:c=999,e=24,p=0,cr=0,cu=0,mis=0,r=0,dep=1,og=1,tim=1233613010506855
FETCH #9:c=0,e=24,p=0,cr=3,cu=0,mis=0,r=2,dep=1,og=1,tim=1233613010506906
STAT #9 id=1 cnt=2 pid=0 pos=1 obj=17364 op='TABLE ACCESS FULL T'
EXEC #9:c=0,e=15,p=0,cr=0,cu=0,mis=0,r=0,dep=1,og=1,tim=1233613010507425
FETCH #9:c=0,e=27,p=0,cr=3,cu=0,mis=0,r=3,dep=1,og=1,tim=1233613010507479
STAT #9 id=1 cnt=3 pid=0 pos=1 obj=17364 op='TABLE ACCESS FULL T'

Why is this feature important?

It is because up to 10g, especially for application keeping cursors open for a “long time”, it is not unusual to see trace files not containing information about execution plans for every cursor. Since that information is critical to diagnose performance problems, it might be a major issue. As of 11g, however, the trace files should always contain this critical information (except if the value NEVER is used, of course).