Oracle Core Essential Internals for DBAs and Developers

Oracle Core Essential Internals for DBAs and Developers Jonathan Lewis

Oracle Core: Essential Internals for DBAs and Developers Copyright 2011 by Jonathan Lewis All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN-13 (pbk): 978-1-4302-3954-3 ISBN-13 (electronic): 978-1-4302-3955-0 Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. President and Publisher: Paul Manning Lead Editor: Jonathan Gennick Development Editor: James Markham Technical Reviewer: Tanel Põder Editorial Board: Steve Anglin, Mark Beckner, Ewan Buckingham, Gary Cornell, Morgan Ertel, Jonathan Gennick, Jonathan Hassell, Robert Hutchinson, Michelle Lowman, James Markham, Matthew Moodie, Jeff Olson, Jeffrey Pepper, Douglas Pundick, Ben Renow-Clarke, Dominic Shakeshaft, Gwenan Spearing, Matt Wade, Tom Welsh Coordinating Editor: Jessica Belanger Copy Editor: Kimberly Burton Production Support: Patrick Cunningham Indexer: SPi Global Artist: SPi Global Cover Designer: Anna Ishchenko Distributed to the book trade worldwide by Springer Science+Business Media, LLC, 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer-sbm.com, or visit www.springeronline.com. For information on translations, please e-mail rights@apress.com, or visit www.apress.com. Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use. ebook versions and licenses are also available for most titles. For more information, reference our Special Bulk Sales ebook Licensing web page at www.apress.com/bulk-sales. The information in this book is distributed on an as is basis, without warranty. Although every precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work. Any source code or other supplementary materials referenced by the author in this text is available to readers at http://www.apress.com. For detailed information about how to locate your book s source code, go to http://www.apress.com/source-code/.

Contents at a Glance About the Author... x About the Technical Reviewer... xi Acknowledgments... xii Introduction... xiii Chapter 1: Getting Started...... 1 Chapter 2: Redo and Undo... 5 Chapter 3: Transactions and Consistency... 25 Chapter 4: Locks and Latches... 59 Chapter 5: Caches and Copies... 93 Chapter 6: Writing and Recovery... 121 Chapter 7: Parsing and Optimizing... 159 Chapter 8: RAC and Ruin... 199 Appendix: Dumping and Debugging... 231 Glossary... 245 Index... 255 iii

Contents About the Author... x About the Technical Reviewer... xi Acknowledgments... xii Introduction... xiii Chapter 1: Getting Started...... 1 Oracle in Processes... 1 Oracle in Action... 4 Summary... 4 Chapter 2: Redo and Undo... 5 Basic Data Change... 5 The Approach... 6 An Example... 6 Debriefing... 9 Summary of Observations... 11 ACID... 11 Redo Simplicity... 14 Undo Complexity... 19 Read Consistency... 20 Rollback... 21 Summary... 22 iv

CONTENTS Chapter 3: Transactions and Consistency... 25 Conflict Resolution... 25 Transactions and Undo... 26 Start and End of Transaction... 28 The Transaction Table... 29 Reviewing the Undo Block... 32 Data Block Visits and Undo... 35 Setting the Scene... 35 The Interested Transaction List... 36 Concurrent Action... 39 Commit SCN... 45 Commit Cleanout... 46 Delayed Block Cleanout... 48 Transaction Table Rollback... 51 LOBs... 57 Summary... 57 Chapter 4: Locks and Latches... 59 First Things, First..... 59 Arrays... 59 Pointers... 60 Linked Lists... 60 Hash Tables... 62 Latches... 66 Logic Behind Latches... 66 Latch Activity Statistics... 69 Latch Scalability... 75 Locks... 77 Infrastructure... 77 v

CONTENTS A Graphic Image of v$lock... 78 Deadlock... 82 Lock Modes... 85 Latches for Locks... 86 KGL Locks (and Pins)... 88 Locks and Pins... 89 Summary... 91 Chapter 5: Caches and Copies... 93 Memory Management... 93 Granules... 94 Granules and Buffers... 95 Multiple Data Caches... 97 Granules and Buffer Pools... 99 Buffer Pools... 100 Working Data Sets... 101 The LRU/TCH Algorithm... 103 LRU/TCH in action... 104 Relinking a Buffer... 106 REPL_AUX... 107 Finding Data... 109 Pinning Buffers... 112 Logical I/O... 114 Updates... 115 Loading a Hash Chain... 115 Read-Consistent Copies... 116 Physical I/O... 117 Tablescans... 117 Summary... 119 vi

CONTENTS Chapter 6: Writing and Recovery... 121 Targets... 121 Log Writer... 122 Log Writer Writes... 125 PL/SQL Optimization... 126 ACID Anomaly... 129 Commit Enhancements... 130 Mechanics... 131 Redo Wastage... 134 Private Redo... 136 Database Writer... 137 Buffer Headers... 137 Checkpoint Queues... 138 Incremental Checkpointing... 140 Database Writer Interactions... 142 Database Writer Meets Log Writer... 142 Database Writer and LRU... 143 Checkpoints and Queues... 145 Buffer Header Queues... 150 Checkpoints and Log Files... 151 Recovery... 152 Media Recovery... 154 Standby Databases... 154 Flashback Database... 155 Side Effects... 156 Summary... 156 vii

CONTENTS Chapter 7: Parsing and Optimizing... 159 Understanding SQL... 159 Parsing... 160 Optimizing... 160 Interpreting the tkprof Summaries... 161 The Dictionary Cache... 164 The Structure... 167 Dictionary Cache Activity... 170 What Is a Parse Call?... 173 Cursor Caching... 174 Holding Cursors... 176 The Library Cache... 178 Shared Pool Structures... 181 The Fine Structure of the Shared Pool... 184 and Action!... 190 Parsing and Optimizing... 192 Executing, Locking, and Pinning... 194 Mutexes... 195 Summary... 196 Chapter 8: RAC and Ruin... 199 The Big Picture... 200 Staying Alive... 202 What s the Point?... 203 High Availability... 203 Scalability... 204 The Grid... 205 viii

CONTENTS How Can It Work?... 207 The GRD... 207 Masters and Shadows... 210 GCS and GES... 213 Cache Fusion... 215 Consequences... 218 Recovery... 220 Sequences... 222 Caching Sequences... 223 Sequence Internals... 223 Ordered Sequences... 225 Sequences and Indexes... 227 Summary... 230 Appendix: Dumping and Debugging... 231 oradebug... 231 Suspending Processes... 231 Dumps... 232 Peeking at Memory... 235 Dumps from SQL... 238 oradebug Alternatives... 238 Data File Blocks... 239 Log Files... 240 Guesswork... 243 Glossary... 245 Index... 255 ix

About the Author Jonathan Lewis is a qualified teacher with a mathematics degree from Oxford University. Although his interest in computers came to light at the tender age of about 12 in the days when high-technology meant you used a keyboard, rather than a knitting needle to punch holes in cards it wasn t until he was four years out of university that he moved into computing professionally. Apart from an initial year as an incompetent salesman (apparently the correct answer to any question is Yes, not Yes, but it s going to take several weeks of configuration and programming ), he has been self-employed his entire career in the computer industry. Jonathan s initiation into Oracle was on version 5.1 running on a PC, which he used to design and build a risk-management system for the crude-trading floor of one of the major oil companies. (He had written the first version of the system using a PC program called dbase III which did use tables and indexes and made some claims to being a relational database management system.) With Oracle, he found out what a proper relational database management system ought to be able to do, and was immediately hooked. Since that day of revelation, Jonathan has focused exclusively on the use and abuse of the Oracle RDBMS. After three or four years of contract work helping to build large systems, he decided to move from the contractor market to the consultant market, and now shares his time evenly between shortterm consultancy work, holding seminars, and research. Jonathan is well-known in the international Oracle scene, having worked in 50 different countries and a dozen US states in the course of the last 10 years. He is a strong supporter of user groups particularly the UK user group (www.ukoug.org), of course and whenever the opportunity arises, he tries to find time for speaking to user groups outside the United Kingdom, sometimes as short, evening events at the end of a day s consultancy work. He also has a blog about Oracle (http://jonathanlewis. wordpress.com) and contributes regularly (when he's not fully occupied writing books) to various magazines, forums and discussion groups. Jonathan just celebrated his silver wedding anniversary to Diana (currently a primary school teacher and head of mathematics after many years of being an accountant). They have two children: Anna (now in her final year at Oxford University) and Simon (in his first year at York University). Despite the numerical backgrounds and mathematical skills of their parents, neither of the younger generation is the slightest bit interested in a career in computing. x

About the Technical Reviewer Tanel Põder is an internationally-acclaimed Oracle performance specialist, having helped solve complex problems for customers in more than twenty countries on five continents. He specializes in advanced performance tuning, end-to-end troubleshooting, and other complex (and, therefore, interesting) tasks, such as migrating very large databases with minimal downtime. Tanel has optimized the performance of Exadata installations starting from Exadata V1, and he plans to go even deeper with his current Exadata performance and troubleshooting research. He is a co-author of Expert Oracle Exadata (Apress, 2011). Tanel is one of the world s first Oracle Certified Masters, an Oracle ACE Director, and a proud member of the OakTable Network. He regularly speaks at conferences worldwide and publishes his articles, scripts, and tools in his blog at http://blog.tanelpoder.com. xi

Acknowledgments Above and beyond all others I have to thank my wife Diana for the support she has given me while I was writing. Without her love and patience, there would have been no book. I can t count the number of cups of tea prepared (and often left to go cold), the late nights (or early mornings) suffered, or the suppression of complaints over the things I should have done instead of write; this book required a lot of focus, and without someone at my side to deal with all the distractions of life, I wouldn t have got it written. A technical book requires good reviewers, of course, and I d like to thank Tanel Põder (http://tech.e2sn.com/ and http://blog.tanelpoder.com/) who was my official technical reviewer. His comments, questions and corrections were most helpful and instructive. I also had a couple of unofficial reviewers in Martin Widlake (http://mwidlake.wordpress.com) and Timur Akhmadeev (http://timurakhmadeev.wordpress.com/), who have also been most helpful in making the book what it is. The role I gave them was to tell me what was missing, what needed to be removed, and what bits were ambiguous; inevitably, though, they were also most useful at pointing out technical errors. If you like the diagrams in this book, you can thank Martin, in particular, because he kept telling me he wanted a visual image of some of the things I had described. Andrey Nikolaev (http://andreynikolaev.wordpress.com/) was a special reviewer for Chapter 4. There is a huge volume of information about the private workings of Oracle on the Internet and a large part of that volume is simply repetition of material that has slowly been going out of date for years. The information available about latching is the best demonstration of this phenomenon; partly, no doubt, because it is a very difficult thing to investigate. Andrey has done far more than any other author I have seen on the internet to bring our understanding of latches (and mutexes) up to date so I asked him if he would review what I had said on the topic in Chapter 4. I am most grateful for the comments and corrections he made. Despite the excellence of my reviewers, you will still find errors and omissions in this book the blame for any such errors and omissions is mine. Some examples will be deliberate simplifications, some will simply be things that I got wrong. A huge fraction of this book is about internal mechanisms, and neither I nor my reviewers have any access to special internal documentation that tells us about those mechanisms; we re just ordinary people who apply the scientific method to the things we can observe and measure. Over time, and with peer group review, we get to a consensus about how things work until new evidence appears. There are many other people whose names I could mention, and some whose names I never knew; people who have contributed some fragment to the content of this book either by supplying answers, by asking questions, or by hiring me to look at interesting problems. There are always plenty of questions on forums, such as OTN (http://forums.oracle.com) and Oracle-L (www.freelists.org/ archive/oracle-l), describing real-world issues with Oracle, and lots of people with the knowledge and experience to offer insights and solutions. Without a continuous flow of new questions, it s easy to get stuck in a rut and think you know it all so I can safely say that much of my knowledge about Oracle has been learned because of research than has been prompted by the questions I see on the internet xii

Introduction When I wrote Practical Oracle 8i, there was a three-week lag between publication and the first e-mail asking me when I was going to produce a 9i version of the book thanks to Larry Ellison s timing of the launch of 9i. That question has been repeated many times (with changes in version number) over the last 12 years. This book is about as close as I m going to come to writing a second edition of the book but it only covers the first chapter (and a tiny bit of the second and third) of the original. There were two things that encouraged me to start writing again. First, was the number of times I saw questions of the form: How does Oracle do XXX? Second, was the realization that it s hard to find answers to such questions that are both adequate and readable. Generally, you need only hunt through the manuals and you will find answers to many of the commonly-asked questions; and if you search the internet, you will find many articles about little features of how Oracle works. What you won t find is a cohesive narrative that put all the right bits together in the right order to give you a picture of how the whole thing works and why it has to work the way it does. This book is an attempt to do just that. I want to tell you the story of how Oracle works. I want to give you a narrative, not just a collection of bits and pieces. Targets Since this book is only a couple of hundred pages and the 11g manuals extend to tens of thousands of pages, it seems unlikely that I could possibly be describing the whole thing, so let me qualify the claim. The book is about the core mechanics of the central database engine the bit that drives everything else; essentially it boils down to undo, redo, data caching, and shared SQL. Even then I ve had to be ruthless in eliminating lots of detail and interesting special cases that would make the book too long, turgid, and unreadable. Consider, for example, the simple question: How does Oracle do a logical I/O?, then take a look at structure x$kcbsw, which is a list of all the functions that Oracle might call to visit a block. You will find (for 11.2.0.2) that there are 1,164 different functions for doing a logical I/O do you really want a detailed breakdown of all the options, or would a generic description of the common requirements be sufficient? The problem of detail repeats itself at a different level how much rocket science do you want to know; and how much benefit would anyone get from the book be if I did spend all my time writing about some of the incredibly intricate detail. Again, there s a necessary compromise to reach between completeness, accuracy, and basic readability. I think the image I ve followed is one that I first saw expressed by Andrew Holdsworth of Oracle s Real-World Performance Group at Oracle OpenWorld in 2006. In a presentation about the optimizer and how to collect statistics, he talked about the 90/9/1 methodology, as follows: 90 percent of the time the default sample works 9 percent of the time a larger sample works 1 percent of the time the sample size is irrelevant xiii

INTRODUCTION It s an enhancement of the famous 80/20 Pareto rule, and one that I think applies reasonably well to the typical requirement for understanding Oracle s internal mechanisms, but for the purposes of explaining this book, I want to rearrange the order as follows: 90 percent of the time you only need the barest information about how Oracle works to keep a system running adequately; 1 percent of the time you need to be a bit of a rocket scientist to figure out what s going wrong; and, I m aiming this book at the 9 percent group who could get a little more out of their databases and lose a little less time if they had a slightly better idea of how much work is going on under the covers. Where Next Some time ago Tanel Põder (my technical reviewer) made the following comment in answer to the question of when he was going to write a book on Oracle internals: The answer is never, if talking about regular, old-fashioned, printed-on-paper books. I think the subject just changes too fast. Also, it takes at least a year of full-time work to come up with a book that would be any good, and by the time of publishing, many details would already be outdated. This is a good answer, and adds weight to my comments about avoiding the 1 percent and sticking to the general requirements and approximations. Tanel s response to the problem is his living book at http://tech.e2sn.com/oracle. But paper is nice (even if it s electronic paper) and I believe the imposition of the book format introduces a difference between the content of a collection of internet articles (even very good ones) and the content a book. Again it comes back to narrative; there is a continuity of thought that you can get from a book form that doesn t work from collating short articles. As I write this introduction, I have 650 articles on my blog (a much greater volume of text than I have in this book); and although I might be able to draw a few articles together into a mini-series, if I tried to paste the whole lot together into a single book, it wouldn t be a terrible book even if I spent days trying to write linking paragraphs between articles. Even technical books need a cohesive narrative. To address the problems of a non-living book, I ve posted a set of pages on my blog at http://jonathanlewis.wordpress.com/oracle-core/, one page for each chapter of the book. Over time, this will report any errors or brief additions to the published version; but as a blog it will also be open for questions and comments. When asked about a second edition for my other books, I said there wouldn t be any. But with feedback from the readers, I may find that with this book, some of the topics could benefit from further explanation, or that there are popular topics I ve omitted, or even whole new areas that demand a chapter or appendix of their own. I ve offered my opening gambit to satisfy a popular requirement now it s up to you, the reader, to respond. xiv