Monday, July 29, 2013

12c New Features... A survey..

I'm working on some things right now related to Oracle Database 12c, and I'd like to do a little survey about your experience with upgrading to different versions of the Oracle Database. You can find the survey here. This is for DBA's only I'm afraid, not developers.  12c New Features content is coming as in more stuff on Exadata Patching and other goodies.

Sunday, July 28, 2013

Exadata Patching Part Five.... Patching the Compute Nodes

In the previous blog post we talked about patching the cell servers. Now, let's talk a little bit about updating the Compute nodes. To begin with, when you look closer at the contents of the QFSDP bundle patch you will find this set of files is included:

- Infrastructure/
|      --ExadataStorageServer/
|          -11.2.3.2.1/
|               -p14522699_112321_Linux-x86-64.zip
|               -p16432033_112321_Linux-x86-64.zip
|               -README-14522699.html
|               -README-16432033.html

Instantly I think this can get confusing, so let me explain. If you goto MOS and look up the first patch p14522699, you will find that this is the Storage Server Software Patch that you will use to update the Storage Servers (which I call the cell servers for some insane reason that I won't try to explain).

So, what is this other patch? What is  p16432033? While you might think this patch has something to do with the storage server (since it sits in a directory called ExadataStorageServer!) in fact it is the ISO image that can be used to update the compute nodes! I know, it's confusing but that is the way things are.

So, what is this patch you ask:

      --ExadataDBNodeUpdate/
|          -dbnodeupdate.zip
|          -README.txt

As it's under a directory you might think that it's the ISO for the compute node. Well, it is, in fact, the database patch for the GRID_HOME (GH) and ORACLE_HOME (OH) directories. So you will be forgiven for being a little confused.

All of these patches are clearly documented (wink) in MOS note 888828.1 (make this note your friend).

To further confuse things, the preferred way to patch your database server operating systems is not through the application of the ISO that is contained in the patch. The preferred way is to use the Oracle Unbreakable Linux Network (ULN) and YUM to download the most current OS release for the compute nodes.

The manual steps for updating a compute node are pretty involved. In an effort to reduce the overall effort and possibility of mistake, Oracle has created a script called the DB Node Update Utility (dbnodeupdate.sh). You can find more information on this tool using MOS note 1553103.1.

Most of the Exadata patches that I do, for various reasons, have to be done manually installing the updates from the current ISO image. If this is something you are going to need to do, reference note 1473002.1 for more information on how to setup the YUM repository for local use.

The bottom line is that for updating the Database Nodes, you will use YUM to grab the needed files in a YUM repository (either the ULN repository or one you create yourself) and then update the compute nodes.

Finally, note 1553103.1 nicely summarizes the various steps that are involved in the update of a compute node. To quote from the note:


A shortened list of the functionality provided by the dbnodeupdate.sh utility as follows:

  • Creates log file to track script execution and changes
  • Creates diag file with 'before patching' situation
  • Provides 'next steps'
  • Includes checks and workarounds for known issues
  • Creates and runs the backup utility 
  • Checks space requirements of /boot filesystem
  • Checks for left-over snapshots
  • Checks for required files and utilities
  • Includes 'check-only' option
  • Includes 'quiet mode'
  • Relinks all Database and Grid Infrastructure (GI) Homes
  • Enables / disables GI to stop/start
  • Checks for nfs/smbfs mounts and unmounts when possible
  • Validates provided media (zip, ISO, http, file://)
  • Validates user
  • Provides rollback option
  • Generates 'yum repo'-file
  • Validates yum.conf-file
  • Logs console history
  • Stops OSwatcher before patching
  • Unpacks media and helper zip-files
  • Stops db console and em agent
  • Checks for failed 'pvmove' actions
  • Checks for left over snapshots and failed backups
  • Supports multiple em agents
  • Checks for 3GB free space in "/"


I vote for automation! More coming soon!

Wednesday, July 24, 2013

Exadata Patching - Part Four...

So... you have downloaded the patch. Now what do you do. Good question.

Installing the Exadata patch sets *usually* involves the following:

1. Patch the cell servers.
2. Patch the Compute note OS.
3. Patch the GRID_HOME and ORACLE_HOME locations.
4. Run the database upgrade script on each database.

I say usually because there could be additional steps to perform, such as firmware updates, in any given patch set. There is also often updates to Enterprise Manager included in the patch sets, so you will want to consider those updates.

So, let's take a look at patching the cell servers in a bit more detail. In later posts I'll address the other steps.

Networking
I can't stress how important it is to be installing these patches from a location with reliable networking. While the internal Exadata network is very fast, redundant and reliable, it's an unfortunate truth that the networks outside the Exadata box may not always be the same. What you don't want is a slow or unreliable network connection that disconnects you in the middle of the the application of the patch. If your network is not reliable, you should consider other options such as the KDM, or directly connecting to the switch between the Exadata rack and your network if possible.
 
Patching the Cell Servers
The first thing we will want to do is patch the cell servers. Applying the cell server patch will update the entire cell server including OS updates, firmware updates and any other updates that are needed on the cell server (such as configuration related updates). This makes patching the cell servers much easier and it also ensures that the server images remain consistent.

Before you start to apply the cell server patch you will want to decide if you want to keep your Exadata rack up and available during the patch - which would require a rolling patch application, or if you want to take the rack down for about two hours to apply the patches.

The downside of doing a rolling patch is that you are looking at anywhere between and hour and two hours, per compute node, to apply the patch. Of course, the rack is up and available during this time, so the users don't notice anything. From an administrative point of view though this can represent a significant investment in time to upgrade the rack. If you are running a half-rack with 7 cell servers, for example, you are looking at anywhere from 7 to 14 hours to upgrade all of the cell servers. That's a long time for someone to sit and monitor the progress of the patch application!

The benefit of the rolling patch is clear though. If the patch fails, it only fails on one node, and that failure should not impact your availability. The loss of one cell server generally isn't something that's going to take the entire rack down. 

On the other hand, a non-rolling upgrade patches all of the cells at once. This has the benefit of shortening the time to upgrade the entire rack of cell servers - say 1 to 2 hours (I'd plan on 2 hours). On the other hand, upgrading all of the cell servers at once can be a bit of a frightening experience. This is because you are sitting there, hopeful, that the patch will apply successfully on all of the cells, that you have not made any mistakes and that unicorns fart rainbows.

If you can take the outage my personal preference is to always run the upgrade on one cell server (which allows the system to remain up) and then take the outage and upgrade the remaining cells all at once. I just like to see the patch apply successfully once so I get warm fuzzies, and generally I like to reduce the overall time it takes to apply patches for various reasons. Of course, if uptime is your principle goal, then you will want to do a rolling patch.

You can find the specific instructions on applying the current (July) patch to the cells on MOS. Note that you need to have access to Oracle MOS to access this note. Note that each patch has it's own install instructions, so you should review the patches to make sure that nothing has changed in the install process.

Cell Server Patching Prerequisites
While there are a number of steps that you will want to perform, there are some that I consider to be critical to applying the patch (at this time)? They are:

1. Make sure you can access the ILOM on each cell server. This is a critical step, so make sure you execute this on each ILOM.

2. I usually like to run the patches from one of the compute nodes. Determine which compute node you want to use to install the patch from and check ssh connectivity from that node to each of the cell servers. Really, if SSH wasn't working right then the Rack would probably be having problems anyway, but it's always a good idea to double check. The following command can be used to check for SSH connectivity (and user equivalence).

dcli -g cell_group -l root 'hostname -i'

dcli is a nice utility that provides the ability to run commands across a set of nodes via SSH. 
You might wonder what the cell_group business is about. This is a text file (usually in /home/oracle) that lists all the cell servers in the rack. Normally this file should have been created when the Exadata rack was installed.

Also, make sure you check the cyphers as indicated in the install documentation. 

3. Check the prerequisites that apply for the type of patch you are applying. For example, if you are doing a rolling patch you will need to adjust the disk_repair_time parameter for ASM. If you are patching all the nodes, you will be shutting down CRS and all associated services at this time.

4. While the patching program (patchmgr) will check for sufficient disk space, I like to manually check the available disk space on each cell server just to be sure.

Of course, this is just a few of the steps that you need to execute before you apply the patch. As I said before, the best thing in my mind to do is create a checklist that clearly calls out all the steps you intend to execute. While the documentation is good, it does contain a mix of instructions and I feel like it's a much easier and clearer process to follow a checklist first, referencing the documentation when required.

Applying the Patch
Once you have completed the prerequisites then it's time to patch the cell. To perform the upgrade you use the patchmgr utility. Usually you will do this from a compute node, while logged in as root. Oracle specifically says not to use the ILOM for applying patches, which makes sense if you think about it. :)

Once you are ready to patch the cells you generally use the patchmgr utility. The patchmgr utility provides an automated way of applying the patches to the storage cells.

When using the patchmgr utility you will first clean up any old patching related stuff, then you will run the prerequisite checks and then you will start your patching. Patchmgr will be in root of the directory where you unzipped the patch files too. The basic commands you would run are:

./patchmgr -cells cell_group -cleanup
This will clean up any previoius patches on all nodes in the cell_group group.
 
./patchmgr -cells cell_group -patch_check_prereq [-rolling]
This executes the prerequisite patch check. This will check the node(s) for space availability, current patch  compatibility and so on. Note that this will apply to all nodes on the cell_group list. If any errors appear when running the prerequisites, these should be corrected before proceeding and then you should run patchmgr with the cleanup and prereq check process, again.

 
./patchmgr -cells cell_group -patch [-rolling]
This executes the actual patch. Using the -rolling parameter will start a rolling patch upgrade. This will apply to all nodes on the cell_group list.

There have been times that I've had to fully path the cell_group location.

The biggest stress I find during the whole process is the rebooting of the cell servers. You really kind of find yourself praying that they will come back up. It can take time, so be patient.

Bottom LineThe bottom line is that the install of the patches is generally pretty straight forward process, but you should carefully follow the instructions provided by Oracle.

Next post, I'll talk about upgrading the Compute Node OS.








It's your birthday...

Happy birthday you....

Saturday, July 20, 2013

Exadata Patching - Part Three...

** If you find these posts helpful, please feel free to link my blog to you website, your blog or wherever it would make sense. Thanks! **

Part Three - Introduction and Disclaimer
So, we have discussed getting ready for patching. Let's talk a little bit about the general mechanics of patching. One note of caution. This blog post was written just as the July patch was released. While I've been through the readme, the directions and so on, I've not yet applied this patch. Also, since I don't have a TARDIS, I've not been privy the the future patches. The bottom line is that the patches might change, instructions for a given patch set might require you follow a different path than the one I'm going to lay out for you here. It's even possible that the method of patching will change. For example, Metalink document 888828.1 says that the patching frequency for Oracle Database 12c is yet to be determined. In fact, the current QFSDP does not patch Oracle Database 12c at all.

As with everything else, things change. I'll try to update my blog on a regular basis if something changes in a major way.

Also, there is a new tool called OPlan. In these posts, I'm going to assume that your not going to use OPlan, but it might be time to start putting it to work in kind of a test phase. OPlan provides a more automated method of helping you figure out what you need to do to patch your system, including dealing with dependencies and the like (note, I've not used it yet, but this is what I've read). However, there are some current restrictions to OPlan that generally precludes me from using it (such as multiple ORACLE_HOME directories). So, for now, I'm going to let OPlan mature  but I'll probably use it with the July patch in kind of a Beta stage and see how well it works. 

About the QFSDP
As I mentioned in my previous post, the QFSDP contains a number of different patches. Patching Exadata is not a fire and forget exercise. There are a lot of steps that you have to follow. Eventually I think that OPlan will simplify those steps, and also I know that the folks that are working on Exadata are always working to simplify the patching process. I have to say, the folks working on Exadata are amazing and as a DBA I am thoroughly impressed with it. Even though I work in a sales organization, I'm a technical guy and I hate selling. I hate marketing hype and hoopla. With respect to Exadata, I've been deeply, technically, involved in several POV's and I can tell you that Exadata is the real deal.

Slightly Off Course for a Moment
The only thing I don't like about it is that it works so well that it can mask really bad SQL and database designs... On the other hand, even good designs have performance issues and in the age of trying to reduce costs and consolidate, Exadata is an amazing machine. Frankly, I hate using the word product, and the word machine seems to black boxish to me. Still, I've seen to do some fantastic things and there is so much opportunity to then go modify code, design and applications beyond that. I'd throw in the rest of the Exa stack, Exalogic, Exalitics and so on, but this is about Exadata patching.

Finding the Current Patch
The July 2013 QFSDP is found by going to Metalink document 16784347. If you are reading this later on, you can find a link to the current patch in Metalink Document 888828.1, as I showed you earlier.

The first thing to notice on the patching document (in this case 16784347), is that  you need to select the correct release and platform as seen here:



More about the QFSDP
Once  you have done that you can click on the Read Me button to read the Readme file associated with the correct patch. If you look at the read me, it gives you a basic low-down on what is contained in the patch bundle. For example, it indicates that the patch bundle consists of three main, high level, components. These are:
  • Infrastructure
  • Database
  • Systems Management
In each of these components are contained sub-components. For example in the database component there is:
  1. Quarterly Database Patch for Exadata (Jul 2013 - 11.2.0.3.20) - contains updates to Oracle Database and Oracle Clusterware software (11.2.0.3.20 QDPE Jul 2013)

  2. OPatch - contains software to install/deinstall the 'Database component' patches (11.2.0.3.4)

  3. OPlan - contains software that facilitates the 'Database component' patch installation process by providing you with step-by-step patching instructions specific to your environment (12.1.0.1.1)
The readme also contains the instructions on preparing to install the QFSDP after you have downloaded it.


Along with the ability to access the readme file, you can also download the bundle patch set.

What's in the QFSDP Download?
Contained within the QFSDP download are a number of individual patch sets. For example, for the infrastructure part of the patch set you get the following files all packed in the zip file:

16784347
|
|- README.txt
|
|- README.html
|
|
|- Infrastructure/
|      --ExadataStorageServer/
|          -11.2.3.2.1/
|               -p14522699_112321_Linux-x86-64.zip
|               -p16432033_112321_Linux-x86-64.zip
|               -README-14522699.html
|               -README-16432033.html
|
|      --ExadataDBNodeUpdate/
|          -dbnodeupdate.zip
|          -README.txt
|
|      --InfiniBandSwitchSoftware/
|          -1.3.3-2/
|               -p12373676_112100_Linux-x86-64.zip
|               -p11891229_133_Generic.zip
|               -README.txt
|
|      --SunRackIIPDUMeteringUnitFirmware/
|          -1.04/
|               -p12871297_1040_Generic.zip
|               -README.txt
|

Note that there are directory structures for each component in the patch. We get the Storage cell update, the DBNode Update, the Infiniband Switch software and the PDUMetering Unit Firmware.

Note that the patches are all cumulative. So, if your behind by a quarter, just apply the most recent patch and you will be good.

Don't Fall For Over Patching
Also, just because a component is included in the patch, does not mean it needs to be applied. For example, the Infiniband switch patch (version 1.3.3-2) has been around for a long time (as you can see - again - in document 888828.1). How do you know what component versions are in your system already? Look at your Exacheck output. In that output, it will tell you all of the current versions installed on your system. Here is an example of that output (modified slightly for the purposes of space savings):



Note in this output (which is for just one node within an Exadata Machine)  that the current versions of each component is listed.  Compare this to the list in the patch readme. For example, if your Infiniband switch software is already at 1.3.3-2, then there is absolutely no need to install the infiniband part of this patch since you can see (near the bottom of the report) that the Infiniband switch is at version 1.3.3.2 already.

Always the Exceptions
There are always exceptions and the like that you have to deal with. After getting a good feeling for what is in the patch set and what is in your Exadata Rack, go back and make sure you read document 888828.1 again, carefully. Traverse EACH relative link in that document (there are a great number) for any information that might supersede the README's contained in the patch bundle.

There have been times, after a patch has been released, that additional notes are included on the Metalink document. These notes might have you use a different version of OPatch, or perhaps they will indicate that you need to  use a different individual patch set that that which is in the QFSPD.

I'll give you an example. In an earlier patch set they missed including the OS patch for the compute nodes. There is a note that indicates that you need to download a specific patch and apply it as a part of the overall QFSDP patch process. Here is an example of that specific note which applied to the January and April patch set:

There are a lot of complex dependencies to patching, as you can imagine. OPlan seeks to remedy some of that and as I said earlier I plan on at least "trying it out" this go around to see how well it manages those dependencies for me.


Battle Stations - The Critical Issues
You will also notice note 1270094.1 listed in ML document 888828.1. This note is critically important as it's lists all Exadata Critical Issues (indeed that is the name of the document). Notice that there is an item on the critical list for the Infinitude switch, along with an associated patch that you need to apply to the current IB 1.3.3.2 version. So, you might think you are all patched up, but until you read this, you would possibly be wrong. In the end, not only do you need to make sure that 1.3.3.2 is installed, but you need to make sure that patch 12373676 has been applied. It can get confusing, especially if you have not been keeping up with your patches.  The fact is that this patch was released a long time ago (way back in 2011) to the odds are that it's been applied. In fact, most of the issues listed here are older ones (the latest is Feb-2013 I believe).

Downloading the Patch
Speaking of things that have changed, this is one of them.
The way the latest patch set has been setup for distribution has changed in this release. I don't know if this will be the way of the future, but it's new. Previously the patch was distributed as a single ZIP file. Now, as of the July patch set (which is a hefty 4.2GB image when fully extracted) is sub-divided into several 1.5GB zip files that you will download and then install according to the install documentation. I'm not sure if this is the plan for distributing the future patches or not.

That's it for now. More coming in part four. I originally thought this would be a three part blog post but  it's seems that it will be more.... I expect to post part four in the next couple of days...

Thursday, July 18, 2013

Eeeegads... the patch is out and I'm behind - Exadata Patching Part II (edited - display issues fixed)

The Exadata July patch set is out! I feel like I'm behind on my Blog posts!

In my last post I talked about Exacheck, and how you needed to run and review Exacheck.  If you find items you need to correct, then now is the time to do it! Often, you can click on the view link of the report and it will take you to a more detailed view of the issue, and sometimes it will even include information on how to correct the problem.

One piece of advice, after you whittle down the items on the Exacheck that you feel like need to be addressed before you patch. If this is your first few times doing this, go ahead and open an SR with Oracle and send that Exacheck to support.  When opening the SR, list the items you think need immediate attention for patching and those that you think can wait and get them to double check your thinking, and provide resolution information on the critical items.

So, having reviewed your Exacheck, it's time to start looking at the actual patch.

If you check Metalink Note 888828.1 (that is a lot of 8's) you will find that the new QFSDP for Exadata has been released. QFSDP stands for (drum roll) Quarterly Full Stack Download Patch. Within this note you will find a section called the "Latest Releases and Patching News" - here is the current copy of this section:



As you can see, each QFSDP  has a patch number associated with it. That patch number has a link to the actual patch, that you can download. Additional information is also on the page where the link exists, such as which bugs are fixed in the patch set.

So, what should you review in preparing to apply a patch? There are a TON of notes that you should review before you try to do your first Exadata patch:

1461240.1 - Exadata Database Machine Software and Hardware Maintenance Planning Guide
This  note provides basic information on Engineered Systems patching. Strongly recommended first time read.

1270094.1 - Exadata Critical Issues - You should read this document prior to any patching exercise, and really, I'd read it regularly to keep up with critical issues on Exadata.

1473002.1 - Exadata YUM Repository Population and Linux Database Server Updating

You will want to review that last note really carefully, because YUM will come into play when installing your quarterly patch on the Compute nodes. Should you be downloading the update online or applying the update locally, you will need to get YUM setup in one way or another. So you might want to be working on that now.

So, what is in a QFSDP? Well, we will talk about that in my next Blog post.

For now, read the documentation, start to think about patching and we will talk more about patching in part three....

Saturday, July 06, 2013

Exadata - Preparing for the quarterly patch - Part One

If you are using Exadata you know it's coming. If all goes as scheduled, the next Exadata QFSDP (Quarterly Full Stack Download Patch) will be out later this month. If your running on Exadata you should know that it's very important to keep your patches up-to-date. That being the case, I thought I'd spend a few moments discussing what you can do to prepare for these patches.

The first thing you can do, and you should do this regularly, is to run an Exacheck report. Oracle Metalink Note 1070954.1 describes how to download the Exacheck report. The included user guide and best practices documents will describe how to install Exacheck and run it.

It's important to note that new versions of Exacheck are released fairly regurarly, and you want to download, install and run the most current version of Exacheck before you install the QFSDP. It's really a good practice to run Exacheck reports on a regular basis. This is a good method of cross-checking your Exadata system for any failures, errors and to note any configuration changes that might have slipped through.

I've run Exacheck before every patch that I've applied. In many cases we have found problems on the Exacheck that needed to be corrected before we could apply the patch. While many Exadata environments have ASR (phone home) setup for their Exadata machines, many do not for various reasons. If you are working with an Exadata machine and it's not setup with ASR, then you need to be extra vigilant reviewing your systems health. The beauty of Exadata is it's redundancy, but that redundancy can also mask a variety of problems that occur. If you don't catch and correct those problems, then things can and do eventually fail. If you find problems on the day you want to be applying patches, you might find your patching schedule thrown out the window as you scramble to repair a bad disk, replace a bad cable, or any number of other small things that can go wrong.

If you look on the Oracle Exadata Assessment Report, which is one of the reports that Exadata produces, you will see a message similar to this one:

Note! This version of exachk is considered valid for 31 days from today or until a new version is available

So, even Exacheck will warn you how much longer  you have to run the current version of Exacheck. 

So, you have your exacheck report. What are the things to look for before you install the patch sets? There are a number of things on the Exacheck that can get flagged and some are worth paying attention too, and others are less than important.

First, notice the disclaimer on the report that says:

NOTE : exachk is only one part of the MAA Best Practices recommendation methodology. My Oracle Support "Oracle Exadata Best Practices (Doc ID757552.1)" should be reviewed thoroughly as it is the driver for exachk and contains additional operational and diagnostic guidance that is not programmed within exachk.

So, Exacheck is just part of an overall set of best practices that Oracle recommends. Notice that it says you should review the note. It is not the purpose of Exacheck to remove the need for the DBA to think - rather it gives the DBA things to think about. Just because the report flags something does not mean it's bad and just because the report does not flag something does not mean things are good. At the end of the day, you have to use your experience to interpret the report and decide what is important in your environment and what is an exception.

One output from the Exacheck execution will be the Oracle Exadata Assessment Report. At the top of this report there will be a score allocated to your Exacheck system. This score is supposed to represent the overall health of the system. While this score is a good guide, it's a bit like the database hit ratio in that it can be misleading and it can also cause you to get target fixation. I do not recommend that you focus on lowering the score. Rather, I recommend that you review the result, determine which ones are critical and really apply to your environment and then correct those problems. I've seen people pay way to much attention to this score, and craft a goal that this number be as near 100 percent as possible rather than analyze the result and use the score as a general metric that you build reasonable thresholds around. Don't get score fixation.

The Exacheck reports contains various checks on the Exadata stack including:
  1. ASM check
  2. Cluster wide check
  3. Database check
  4. ORACLE_HOME check
  5. OS Check
  6. Patch check
  7. SQL check
  8. SQL parameter check
  9. Storage server check
  10. Infiniband switch check

These checks include a variety of things such as version checks, best practices checks and so on. So, the question is, what is truly important when it comes to Exacheck and making sure everything is ready for the patch to be applied.

Generally I will ignore database setting failures out of the gate. This isn't because these are not important, but generally a database parameter, on a working database, that is not set in alignment with best practices is not going to impact the application of the patches. So, let's say that you saw the following on your Exacheck report:

FAILDatabase CheckDatabase control files are not configured as recommendedAll Databases

Would this failure require resolution before you applied your patches. Probably not. Not that this isn't something to look and and figure out if your databases are secure, but this should not stop you from applying a database patch.

Now, what about this one:

WARNINGOS CheckFree space in root(/) filesystem is less than recommended.

Note that this is a warning and not a failure, which seems a bit odd to me. What would you do if you saw this error message? Would you apply your patches? No, of course not. Since one of the things we are going to patch is the compute node, we want to make sure that there is plenty of space on the root file system. Also, we are going to be backing up the boot partition on these boxes, and we want the boot partition as clean as possible before we start backing it up, or patching it. So, in my mind, in spite of the fact that this is reported as just a warning, it's really something important to look into.

What do you do if you get a result and you don't know how to react to it? Please, please - contact Oracle Support and ask them! Don't assume it's ok, or that it's not important. If you don't understand it, take steps to understand it and then determine if you need to resolve it.

So, what are the issues that I worry the most about?

  1. Network issues. Cabling, configuration, all are important.
  2. ILOM issues. If an ILOM is flaky, you have a real problem.DO NOT PATCH IF YOU HAVE AN ILOM ISSUE. PERIOD!
  3. Storage cell issues. The storage cells should all be consistently without error. If an error on the storage cell layer is flagged, it's something to pay attention to. Me personal rule, don't patch anything unless the storage cells are 100% healthy without question.
  4. Any database server OS checks that fail due to configuration, infiniband or other infrastructure related errors.
  5. Any database server verify-topology checks that fail.
  6. Any database errors that surface that indicate the versions are different, incorrect or other types of errors.
  7. If the Exacheck report has the following included in it: WARNING! The data collection activity appears to be incomplete for this exachk run. Please review the "Killed Processes" and / or "Skipped Checks" section and refer to "Appendix A - Troubleshooting Scenarios" of the "Exachk User Guide" for corrective actions.
All of this being said, I think the Exacheck reports says it best when it states:

FAIL, WARNING, ERROR and INFO finding details should be reviewed in the context of your environment.

Finally, once you have run Exacheck, reviewed it's findings and corrected those that need correction, it's a good idea to document the final Exacheck results for future reference.

So, now you have your Exacheck ready to go and all problems that need attention corrected! We are one step closer to being ready to apply that patch!

Well, I hope this blog entry was helpful. In the next entry I'll discuss some Exadata results I've seen in the past and the impacts of those results. Then, we will talk about other Quarterly bundle patch related topics. Your comments about your experience with Exacheck are most welcome!  Have a great weekend!







Friday, July 05, 2013

12c New Features Contribution - Want to get published?

This is related to my new Oracle Database 12c New Features book which will be out soon! I'm offering you a chance at getting your name in lights (sort of) and to be immortalized in this new book.  Note there is no compensation, award, prize or anything else guaranteed here. Just opportunity.

We are near the end of the actual writing of the 23c New Features book. My friend, Tom Kyte, has offered to add some running commentary which we are calling "Tom says". In the process, Tom is also offering up some technical comments. I had finished up Chapter 6, which includes coverage on a new feature called Pattern Matching. To be honest and embarrassed a bit, Tom replied that he didn't like my example. In fact, he kind of hated it. There were many reasons why, and I agreed with 90% of them.

So, after feeling really depressed I thought... Well, back to the drawing board with my example. Then it struck me that this is actually an opportunity.

So, I'm giving YOU an opportunity to be published. In the comments to this post (and please copy me in email at robertgfreeman@yahoo.com), respond with the following by midnight 7/15:

  1. Write a query using the new Oracle Database 12c Pattern Matching feature.
  2. Include sample output demonstrating the query.
  3. Provide all DDL required to re-create the query including the creation of the table (and any other objects), and creation of all data in that table.
  4. The query should be SIMPLE for a first time user to follow, but not so simple that it can be replicated with a SQL statement easily not using the pattern matching feature.
All comments with entries will be published on the blog here as long as they are not bogus, breaking copyrights (so don't just cut and paste from the Oracle documentation or just change it so slightly that you think no one will notice) or are otherwise in bad form.

One awesome entry will be selected for publication (assuming a get an awesome entry - I do not guarantee that any entry will be selected).

I'll contact you if you are selected and then we can work out the legal details. Note that if you are selected you will not be compensated for your entry or any additional follow-up work that you might be asked to do. You will have to warrant that it's your own and give Oracle Press rights to publish it under their copyright. Your name will appear in the acknowledgements of Oracle Database 12c New Features for all your jealous co-workers to oooohhh and aaawwwwhhhh at!

So, get cracking and learn about and use Oracle Database 12c Pattern Matching!

P.S. -- Here is a starting place....
http://docs.oracle.com/cd/E16655_01/server.121/e17749/pattern.htm



Wednesday, July 03, 2013

Oracle GoldenGate Book is out!

Lots of things Oracle Wise are going on.


I just saw that my new book on Oracle GoldenGate has shipped on Amazon. I'll say that this book was one of the toughest I've written for a lot of reasons. It was written during a time of great personal upheaval.... it has seen my life in the greatest ups and downs that I've ever experienced. This is part of the reason that it took so long to get the book printed. However, we worked hard to keep the book current and relevant - we added a new chapter at the end to cover the latest 11g features of GoldenGate. I'm sure that we will see GoldenGate 12c at some point in time in the not to distant future (I have absolutely no internal knowledge of this fact what-so-ever, this is a pure guess based on the fact that Oracle Database 12c has now been released).

I'm working on the last of the writing on my Oracle Database 12c New Features book! I just finished the initial write of the final "chapter" of the book and am now preparing to go back through all the chapters and test them against the final production code. We write the NF book mostly on the Beta code of a new database version, but we ALWAYS go through every chapter, every code example, and check it against the production code. I've noticed that other 12cv books tend to use the beta and that the early ones out don't seem to check their stuff against production. This has lead to examples that don't work, mis-named parameters and a host of other problems with the books that make them less than useful. I think this approach works best, but I'd be interested in your experiences.

More later on all that was promised and more.

edit: Scott made me realize that I probably did not give the people who helped me with the GoldenGate book and the New Features book too! I don't have all the names at hand, and I don't want to miss anyone, so I'll do that on a follow-on post.
 
Subscribe in a reader