Dec 18 2011

Ad: The Oracle Query Optimizer 2-Day Seminar

Tag: Query Optimizer, Speaking, TOPChristian Antognini @ 12:42 am

The 31st of January and 1st of February 2012 I will present a 2-day seminar about the Oracle query optimizer in Ballerup (DK). The event is organized by Miracle A/S. The content, which is based on the chapters 2, 4, 5, 6, 9 and 10 of my book, is the following:

  • Chapter 1 describes the life cycle of SQL statements and when the database engine can share cursors.
  • Chapter 2 describes the aim and architecture of the query optimizer.
  • Chapter 3 and 4 discuss the statistics used by the query optimizer to carry out its work.
  • Chapter 5 describes the initialization parameters influencing the behavior of the query optimizer and how to set them.
  • Chapter 6 outlines different methods of obtaining execution plans, as well as how to read them and recognize inefficient ones.
  • Chapter 7 describes how to take advantage of available access structures in order to access data stored in a single table efficiently.
  • Chapter 8 goes beyond accessing a single table, by describing how to join data from several tables together.

The flyer and this page provide detailed information about the seminar.


Dec 12 2011

Challenges and Chances of the 11g Query Optimizer

Tag: 11gR1, 11gR2, Bug, Indexes, Object Statistics, Query Optimizer, Speaking, System StatisticsChristian Antognini @ 10:59 am

Challenges and Chances of the 11g Query Optimizer is the name of a presentation I gave at several events (e.g. Trivadis Performance Days, Oracle OpenWorld, DOAG Konferenz, UKOUG Conference) throughout 2011. Its abstract is the following:

With every new release, the query optimizer is enhanced. Oracle Database 11g Release 1 and Release 2 are no exception to the rule. Specifically, they introduce key improvements in the following areas: indexing, optimization techniques, object statistics and plan stability. The aim of this presentation is to review the new features from a practical point of view as well as to point out challenges related to them. In other words, to let you know what you can expect from the query optimizer when you upgrade to Oracle Database 11g.

The aim of this short post is to point out that I made available the current version of the slides and all the scripts that go with them here.

The structure of the presentation (incl. a reference to the available scripts) is the following:

  • Observations
    • Number of Query Optimizer Parameters by Release
    • Number of Query Optimizer Bugs Fixed by Patchset
  • Indexing
    • Invisible Indexes (ex_invisible_index.sql)
    • Index Support for Linguistic LIKE (ex_linguistic_like.sql)
    • INDEX REBUILD and Statistics History (ex_index_rebuild.sql)
  • Optimization Techniques
    • Full Outer Join (ex_full_outer_join.sql)
    • Join-Filter Pruning (ex_join_filter_pruning.sql)
    • Table Expansion (ex_table_expansion.sql)
    • Join Factorization (ex_join_factorization.sql)
    • OR Expansion (ex_or_expansion.sql)
    • Join Elimination (ex_join_elimination.sql)
    • Subquery Unnesting (ex_subquery_unnesting.sql)
  • System and Object Statistics (DBMS_STATS)
    • Workload System Statistics
    • Object Statistics – Default Preferences
    • Object Statistics – Auto Sample Size
    • Object Statistics – Pending Statistics (ex_pending_object_statistics.sql)
    • Object Statistics – Incremental Statistics (ex_incremental_stats.sql)
    • Object Statistics – Extended Statistics on Expressions (ex_extended_statistics1.sql)
    • Object Statistics – Extended Statistics on Column Groups (ex_extended_statistics2.sql)
    • Object Statistics – Seeding Column Groups
    • Object Statistics – Comparing Statistics (ex_comparing_statistics.sql)
    • Object Statistics – Locks not Exported
    • JOB_QUEUE_PROCESSES
  • Plan Stability
    • CURSOR_SHARING
    • SQL Plan Baselines (ex_execution_plan_stability.sql, ex_execution_plan_stability_10g.sql, ex_execution_plan_stability_11g.sql)
    • Stored Outlines
    • Adaptive Cursor Sharing (ex_bind_peeking.sql, ex_bind_peeking_bind_aware.sql)
    • Cardinality Feedback (ex_cardinality_feedback.sql)

Sep 19 2011

Impact of STATISTICS_LEVEL on Cardinality Feedback and Adaptive Cursor Sharing

Tag: 11gR1, 11gR2, Query OptimizerChristian Antognini @ 6:20 pm

The STATISTICS_LEVEL parameter controls a bunch of features. In addition to the documentation, also the V$STATISTICS_LEVEL view provides a list of the ones it controls.

SQL> SELECT statistics_name, description, activation_level
  2  FROM v$statistics_level
  3  ORDER BY 3 DESC, 1;

STATISTICS_NAME                        DESCRIPTION                                                  ACTIVATION_LEVEL
-------------------------------------- ------------------------------------------------------------ ----------------
Active Session History                 Monitors active session activity using MMNL                  TYPICAL
Adaptive Thresholds Enabled            Controls if Adaptive Thresholds should be enabled            TYPICAL
Automated Maintenance Tasks            Controls if Automated Maintenance should be enabled          TYPICAL
Bind Data Capture                      Enables capture of bind values used by SQL statements        TYPICAL
Buffer Cache Advice                    Predicts the impact of different cache sizes on number of    TYPICAL
                                       physical reads
Global Cache Statistics                RAC Buffer Cache statistics                                  TYPICAL
Longops Statistics                     Enables Longops Statistics                                   TYPICAL
MTTR Advice                            Predicts the impact of different MTTR settings on number of  TYPICAL
                                       physical I/Os
Modification Monitoring                Enables modification monitoring                              TYPICAL
PGA Advice                             Predicts the impact of different values of pga_aggregate_tar TYPICAL
                                       get on the performance of memory intensive SQL operators
Plan Execution Sampling                Enables plan lines sampling                                  TYPICAL
SQL Monitoring                         Controls if SQL Monitoring should be enabled                 TYPICAL
Segment Level Statistics               Enables gathering of segment access statistics               TYPICAL
Shared Pool Advice                     Predicts the impact of different values of shared_pool_size  TYPICAL
                                       on elapsed parse time saved
Streams Pool Advice                    Predicts impact on Streams perfomance of different  Streams  TYPICAL
                                       pool sizes
Threshold-based Alerts                 Controls if Threshold-based Alerts should be enabled         TYPICAL
Time Model Events                      Enables Statics collection for time events                   TYPICAL
Timed Statistics                       Enables gathering of timed statistics                        TYPICAL
Ultrafast Latch Statistics             Maintains statistics for ultrafast latches in the fast path  TYPICAL
Undo Advisor, Alerts and Fast Ramp up  Transaction layer manageability features                     TYPICAL
V$IOSTAT_* statistics                  Controls if I/O stats in v$iostat_ should be enabled         TYPICAL
Plan Execution Statistics              Enables collection of plan execution statistics              ALL
Timed OS Statistics                    Enables gathering of timed operating system statistics       ALL

Something that I learned only recently is that STATISTICS_LEVEL also controls cardinality feedback and adaptive cursor sharing. This fact, according to me, is neither (clearly) documented nor pointed out by the information provided by V$STATISTICS_LEVEL. In any case, when STATISTICS_LEVEL is set to BASIC at the system level both features are disabled. Interestingly, an ALTER SESSION SET STATISTICS_LEVEL = TYPICAL it is not enough to enable them… For adaptive cursor sharing it is possible to use the BIND_AWARE hint, though.

Note that I never advise to set STATISTICS_LEVEL at the system level to a value that is different from the default (TYPICAL). Probably for this reason I didn’t notice its impact for such a long time…

In any case I find it a bit disappointing that this information is not clearly stated somewhere. Or I’m the only one that was not aware?


Sep 11 2011

optimizer_secure_view_merging and VPD

Tag: 10gR2, 11gR1, 11gR2, Query Optimizer, TOPChristian Antognini @ 10:21 am

At page 189 of TOP I wrote the following piece of text:

In summary, with the initialization parameter optimizer_secure_view_merging set to TRUE, the query optimizer checks whether view merging could lead to security issues. If this is the case, no view merging will be performed, and performance could be suboptimal as a result. For this reason, if you are not using views for security purposes, it is better to set this initialization parameter to FALSE.

What I didn’t consider when I wrote it, it is the implication of predicate move-around related to Virtual Private Database (VPD). In fact, as described in the documentation, that parameter controls view merging as well as predicate move-around.

To point out what the impact is, let’s have a look to an example based on the description provided in TOP:

  • Say you have a very simple table with one primary key and two more columns.

CREATE TABLE t (
  id NUMBER(10) PRIMARY KEY,
  class NUMBER(10),
  pad VARCHAR2(10)
);

  • For security reasons, you define the following policy. Notice the filter that is applied with the function to partially show the content of the table. How this function is implemented and what it does exactly is not important.

CREATE OR REPLACE FUNCTION s (schema IN VARCHAR2, tab IN VARCHAR2) RETURN VARCHAR2 AS
BEGIN
  RETURN 'f(class) = 1';
END;
/

BEGIN
  dbms_rls.add_policy(object_schema   => 'U1',
                      object_name     => 'T',
                      policy_name     => 'T_SEC',
                      function_schema => 'U1',
                      policy_function => 'S');
END;
/

  • Now let’s say that a user who has access to the table creates the following PL/SQL function. As you can see, it will just display the value of the input parameters through a call to the package dbms_output.

CREATE OR REPLACE FUNCTION spy (id IN NUMBER, pad IN VARCHAR2) RETURN NUMBER AS
BEGIN
  dbms_output.put_line('id=' || id || ' pad=' || pad);
  RETURN 1;
END;
/

  • With the initialization parameter optimizer_secure_view_merging set to FALSE, you can run two test queries. Both return only the values that the user is allowed to see. In the second one, however, you are able to see data that you should not be able to access.

SQL> SELECT id, pad
  2  FROM t
  3  WHERE id BETWEEN 1 AND 5;

        ID PAD
---------- ----------
         1 DrMLTDXxxq
         4 AszBGEUGEL

SQL> SELECT id, pad
  2  FROM t
  3  WHERE id BETWEEN 1 AND 5
  4  AND spy(id, pad) = 1;

        ID PAD
---------- ----------
         1 DrMLTDXxxq
         4 AszBGEUGEL
id=1 pad=DrMLTDXxxq
id=2 pad=XOZnqYRJwI
id=3 pad=nlGfGBTxNk
id=4 pad=AszBGEUGEL
id=5 pad=qTSRnFjRGb

  • With the initialization parameter optimizer_secure_view_merging set to TRUE, the second query returns the following output. As you can see, the function and the query display the same data.

SQL> SELECT id, pad
  2  FROM t
  3  WHERE id BETWEEN 1 AND 5
  4  AND spy(id, pad) = 1;

        ID PAD
---------- ----------
         1 DrMLTDXxxq
         4 AszBGEUGEL
id=1 pad=DrMLTDXxxq
id=4 pad=AszBGEUGEL

The execution plans that are used in the two situations are the following. As you can see only the second one guarantee that the policy defined via VPD is applied before the predicate based on the SPY function. Interestingly enough the other predicate based on the ID column is applied before the one of the policy. Hence, the query optimizer can choose an access path that takes advantage of the primary key.

  • optimizer_secure_view_merging = FALSE

---------------------------------------------------
| Id  | Operation                   | Name        |
---------------------------------------------------
|   0 | SELECT STATEMENT            |             |
|*  1 |  TABLE ACCESS BY INDEX ROWID| T           |
|*  2 |   INDEX RANGE SCAN          | SYS_C009970 |
---------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter(("SPY"("ID","PAD")=1 AND "F"("CLASS")=1))
   2 - access("ID">=1 AND "ID"<=5)

  • optimizer_secure_view_merging = TRUE

----------------------------------------------------
| Id  | Operation                    | Name        |
----------------------------------------------------
|   0 | SELECT STATEMENT             |             |
|*  1 |  VIEW                        | T           |
|*  2 |   TABLE ACCESS BY INDEX ROWID| T           |
|*  3 |    INDEX RANGE SCAN          | SYS_C009971 |
----------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   1 - filter("SPY"("ID","PAD")=1)
   2 - filter("F"("CLASS")=1)
   3 - access("ID">=1 AND "ID"<=5)

Based on these observations, the summary that is provided by TOP at page 189 should be amended as follows:

In summary, with the initialization parameter optimizer_secure_view_merging set to TRUE, the query optimizer checks whether view merging or predicate move-around could lead to security issues. If this is the case, they will not be performed, and performance could be suboptimal as a result. For this reason, if you are not using views or VPD for security purposes, it is better to set this initialization parameter to FALSE.


Feb 17 2011

IS NULL Conditions and B-tree Indexes

Tag: 10gR1, 10gR2, 11gR1, 11gR2, 9iR2, Indexes, Query Optimizer, TOPChristian Antognini @ 11:01 am

At page 383 of my book I wrote the following sentence (BTW, the same information is also provided by Table 9-3 at page 381):

With B-tree indexes, IS NULL conditions can be applied only through composite B-tree indexes when several SQL conditions are applied and at least one of them is not based on IS NULL or an inequality.

The text continues by showing the following examples (notice that in both cases the IS NULL predicate is applied through an access predicate):

SELECT /*+ index(t) */ * FROM t WHERE n1 = 6 AND n2 IS NULL

Plan hash value: 780655320

----------------------------------------------
| Id  | Operation                   | Name   |
----------------------------------------------
|   0 | SELECT STATEMENT            |        |
|   1 |  TABLE ACCESS BY INDEX ROWID| T      |
|*  2 |   INDEX RANGE SCAN          | I_N123 |
----------------------------------------------

   2 - access("N1"=6 AND "N2" IS NULL)

SELECT /*+ index(t) */ * FROM t WHERE n1 IS NULL AND n2 = 8

Plan hash value: 780655320

----------------------------------------------
| Id  | Operation                   | Name   |
----------------------------------------------
|   0 | SELECT STATEMENT            |        |
|   1 |  TABLE ACCESS BY INDEX ROWID| T      |
|*  2 |   INDEX RANGE SCAN          | I_N123 |
----------------------------------------------

   2 - access("N1" IS NULL AND "N2"=8)
       filter("N2"=8)

When I wrote that sentence I didn’t think about one case that, according to it, specifically the part “is not based on IS NULL or an inequality”, is not covered. In fact, as the following examples show, it is also possible to apply an IS NULL predicate when the other one is an IS NOT NULL. It is especially interesting to notice that the access predicate doesn’t reference at all the NOT NULL column!

SELECT /*+ index(t) */ * FROM t WHERE n1 IS NULL AND n2 IS NOT NULL

Plan hash value: 780655320

----------------------------------------------
| Id  | Operation                   | Name   |
----------------------------------------------
|   0 | SELECT STATEMENT            |        |
|   1 |  TABLE ACCESS BY INDEX ROWID| T      |
|*  2 |   INDEX RANGE SCAN          | I_N123 |
----------------------------------------------

   2 - access("N1" IS NULL)
       filter("N2" IS NOT NULL)

SELECT /*+ index(t) */ * FROM t WHERE n1 IS NOT NULL AND n2 IS NULL

Plan hash value: 3029444779

----------------------------------------------
| Id  | Operation                   | Name   |
----------------------------------------------
|   0 | SELECT STATEMENT            |        |
|   1 |  TABLE ACCESS BY INDEX ROWID| T      |
|*  2 |   INDEX SKIP SCAN           | I_N123 |
----------------------------------------------

   2 - access("N2" IS NULL)
       filter(("N2" IS NULL AND "N1" IS NOT NULL))


Next Page »