Oh... Say it isn't so... Twinkies and Ding Dongs are bankrupt??

Look here! It' can't be...!!

Now I KNOW something is wrong with the world!!

Engineered Systems - Complexity vs. Simplicity...

Oracle started the Engineered Systems decade (my word!) with the introduction of Exadata. Now we have Exalogic and other machines that provide this engineered system concept that I generically like to call Exa* (Exa-Splat). A lot has been touted with respect to the feature sets of these systems, and I've seen enough POV's now (both first-hand and word of mouth) that I can tell you that the performance improvements are real and significant.

However, I think that the story of Engineered system is about much more than being able to take a query that ran before in 24 hours and now it runs in 3 minutes. Sure, that's great and very important, but there is a strategic reason for Engineered systems and I'm not sure that is getting as much play as it should, because I think it's important.

Does this sound like you? Your environment is this collection of loosely coupled systems. You have these great, powerful database servers but when you look at CPU it's sitting at some number like 20 percent. You have servers all over the place. Your behind on your patching. Your constantly standing up new systems, and all the related infrastructure (network, disk, etc) and it just takes time. Your time to provision has the customers complaining. Your spending all your time reacting to problems, to configuring new environments, to wondering why this switch does not work with this server. You spend a lot of time wondering how to integrate various pieces and which technologies are best and which are certified. If this sounds like you, I call you tactical man. You are tactical in your thinking. Your poor customers really don't have the ability to derive knowledge from you because you are busy standing up systems for them. They are building database designs that are amazingly awful, and you are unable to help them because you have to work on the next fire, the next problem.

Is this you:

(phone rings)
You: Hello
Fred: Hey this is Fred. We need help with a database design. What's this normalization thing?
You: Sorry Fred, I'm working on installing four copies of Linux right now, can't help.
Fred: Hey, I understand, you guys are over worked down there. We will just use Hibernate to generate the schema, no problem.
You: No... wai...
(click on the phone)...

Are you tactical man? I wrote some time ago about the holistic DBA and really... becoming the holistic DBA is all about transitioning from Tactical Man to Strategic Man. The truth is that engineered systems can help you in this transition. How you ask... First, look at this handy dandy graphic I whipped up:

This image is kind of an exercise in simplicity (and my limited graphics design abilities) but I think it makes a point. As we move from higher complexity to simplicity, we move from being tactical (reactive) in our work to more strategic and proactive. Also, look at the graphic... in moving in this direction we reduce the number of people required and we reduce the costs as well.

Engineered systems is part of this drive towards simplicity. With an engineered system, you let the vendor do the painstaking work of finding the right components, certifying them, and making them all work together like a well oiled machine. With Engineered systems we get the benefit of scale.... If one problem surfaces on a given system, because the system are all similarly engineered we can immediately apply the solution to the problem across the various enterprises running that engineered system. No more wondering about this or that combination of hardware, we know (more properly the vendor knows) and the vendor take responsibility for providing that remediation across the platforms that are deployed.

Other benefits include the reduction of the proliferation of systems and hardware, reducing the overall footprint that has to be managed. This reduces costs and allows you to again move up towards the strategic part of the graph. There are a number of benefits with respect to the implementation of  an engineered solution, be it Exadata or even the Oracle Database Appliance. We should not focus on just one part of the equation (performance) when asking ourselves, what do we do to improve productivity and become more visionary?

As a result... the DBA, the network engineer, the storage administrator, the Unix administrator, are all free to pursue tasks further up the scale, moving in the direction of strategic planning, proactive approaches to business problems and helping the customer do it right the first time. In being visionary about the systems we deploy we can improve our ability to react to strategic needs of the enterprise.

I know... this might sound like a sales pitch for Exa* products.... and I do work for Oracle, but everything I've said here is part of my experience, past and present. Tactical man is alive and the costs of keeping him in business are significant. Strategic man needs to be our goal, and with Strategic man comes a host of benefits including lowered costs and a better Enterprise. Engineered systems can help you migrate to becoming Strategic man...

Sure... really.... we all just love installing Linux and configuring Clusters... but is that really best for the Enterprise? I think not.

Writing a book.... Part 2

Greetings all...

So I'm in the middle of my new GoldenGate book. I've written chapters 3,4, and 5 and currently I'm working on Chapter 2. I know, that might not make sense but it did for me... :) At any rate, I've found GoldenGate to be a great little replication product...

I really think that this book is going to be a dynamite book and I think that GoldenGate is going to really come into the Enterprise more and more as time goes on for a number of reasons.

Don't Believe Everything You Read

When you write about a product you come across certain subtitles and problems that kind of make you wonder. In the latest case I was writing about the privileges that are required by the GoldenGate administrator account. I found a bit of a documentation error in that it does not include issuing a grant of the alter any table privilege (or alter table if you prefer) to the GoldenGate administrator account. This privilege is required to be able to issue the GoldenGate add trandata command which enables supplemental logging on a specific object within the database, a requirement for GoldenGate replication. Curious that this was missing.

The Concept of Least Privilege in Oracle Database Security
Even more curious is how my competition handled this problem and this leads to a discussion on the concept of least privilege and how even the most innocent of things can be a really bad. idea. In the data realm, security is strengthened by the notion of least privilege which is where a specific user account is given only the exact privileges it needs. In this day of hacking, break-in's and SQL injection, locking down the database is critical. All to often though, because it's easier, DBA's will grant administrative privileges to accounts, without considering the risks that this entails. The thought seems to be that in granting the DBA role to an account, we are saving time and money by not drilling down into the privileges that are really needed and only granting those specifically required privileges. Of course, we assume just because we have not been hacked yet, that we are immune from such nastiness and as a result we lazily grant DBA to anyone.

Maybe it's not hackers at all we need to protect ourselves from... maybe it's a software bug or design flaw (if you prefer) that we need to protect ourselves from. At any rate, granting DBA privileges willy nilly is just a really bad bad idea. There is a great deal of discussion on least privilege on the internet, and I think the concept is really pretty obvious to anyone of moderate intelligence. At least .... you would think.


Books are things that people read to learn how to do things, and how to do them right. Now, we all make mistakes and I'm the first one to stand up and make the claim that I'm far from perfect. But there are just some mistakes that one should not make. In this case, as I was chasing down the question "Why is the add trandata command not working - Eventual answer is because a privilege is missing from the documentation" I went to look at the competing Oracle GoldenGate works to see if they had anything to say on the topic. Much to my horror, I found both books espousing the notion of granting the DBA role to the GoldenGate administrator account! Holly cow! This is far from the Oracle recommendation and it's just plain bad practice. I had to shake my head after reading that in both books no less.

Back to Don't Believe Everything You Read
Which leads to the another point... about books and web pages.... be careful what you read (especially that Freeman guy, watch out for him). Make sure that you think about what you are doing as you are doing it. Not everyone is the Oracle expert they claim to be, and experts come with varying levels of expertise... I've seen web pages that have listed the "solution" to a specific problem with the Oracle database.... and while the solution stated works it is such a bad practice that the person recommending the change ought to be hung.... Ok, I'm not suggesting lynching be re-introduced as a common practice but maybe we ought to threaten a few people with pitchforks, billy clubs and march on them en-mase. This is where the trusted few really come in and make life better for us all. The Tom Kytes, Jonathan Lewises, Cary Millsaps, and the other recognized names in the Oracle community.

It's also where the Oracle documentation comes in, as well as Oracle support. It's also where YOU come in... Which leads to the last point...

File those SR's!!!
When you run into a problem of find a documentation bug what do you do? Do you find your own workaround and then just move on or do you file an SR on the problem and make sure that Oracle is aware of the issue? Look at how vast the documentation library is and you know the likelihood of incorrect or missing information is quite high. The folks that work on the documentation are amazing and do a great job, but as with any kind of writing mistakes happen... with a product as dynamic as Oracle, things are missed... You are an important part of the improvement cycle, so please, take a moment and file that SR or that Documentation bug and make life for all of us a little better.

Writing a book....

When I tell people I'm writing a book, I'm often asked how I like it. The process is long and can be frustrating, but it can be exciting too. I'm writing a new book now, on Oracle GoldenGate. I'm somewhat new to Oracle GoldenGate, but I've collected a number of contributors and I have also brought on a great technical editor to work with me. In the end, I know this is going to be an awesome book!

I thought I'd blog a little bit along the way as I write the book. I'll also blog about some things related to GoldenGate as I come across them. I'll share my writing moments, and the things I learn and experience along the way.

Before you start writing a book, there are a few hoops you have to go through. First, you put together the proposal for the book. The proposal tells the publisher about the book, who the audience it, why the book will sell, how big the book will be and so on. Along with the proposal, you also submit an outline that includes a table of contents, title and length of each chapter, and so on. The publisher takes all of this information, does a quick calculation to determine if they think they can make any money on the book and then decide if they will publish. This process may take 2-3 weeks if  your an established author, with a book idea that clearly makes sense. If you are a new author or the book idea is not as straight forward then the process may take longer as the publishing world wants to make sure they are not going to loose money on the deal.

Having agreed to the basic terms of the book, the publisher will then have you sign a contract for delivery of the book. Typically this outlines how many pages and general content that will be in the book. It also will outline payment of advances and royalties that you can expect to be paid on your book. Once you sign the contract, you are on the writing trail.

I strongly advise prospective authors not to start writing until you have a contract unless your plan is to self-publish if you can't get someone to do it for you. I've seen what seemed like certain deals fall through at the last minute, and I've seen other authors start writing and then regret it because the deal fell through. So, if your going to write, make sure you have the contract signed.

When you start writing, the publisher is going to want you to write to a specific template/format. Generally they will send you a Word template (or whatever editor they want you to use) that will have specific formats for various parts of your book. With Oracle Press, I have a MS Word template they sent me. I simply plug it into my Word document and I'm off to the races.

I've written a couple of chapters in the book so far. As I said, GoldenGate is a fairly new product for me, but I actually think that makes my writing better. First, as I write the chapters, I see this material from the point of view of my reader, typically someone who is new to the product. I think this helps me write content that helps the new person, and I also think that it helps me see the pitfalls that might beset them.

For example.... when you are using GoldenGate, you need to enable what Oracle calls supplemental logging. Supplemental logging causes Oracle to record additional information about a transaction in the redo stream. GoldenGate needs this additional information to properly replicate data changes over to target databases. Now, to enable supplemental logging you can use the GoldenGate interface called GGSCI, log into the database and then use the command add trandata to enable supplemental logging for a specific table or set of tables. If you look in the documentation it seems as easy as doing that once everything is setup. However, there is a bit of a gottya it seems in that the documentation (in as far as I've been able to find) is missing one bit of crucial information. This has to do with permissions. It does not seem to note anywhere that the GoldenGate administrator account that you log into needs to have the alter table privilege.

Oh, maybe it's hidden in there somewhere and I have not found it yet, but that is just my point. I had a hard time finding it. Now, if I was an old GoldenGate expert, like my technical editor is, then I might have missed that subtle point. Because I'm somewhat new to it, I catch things like this and actually put them in the book... as notes or as a part of the overall text, they are in there. I think that is part of what makes my books good (I say this with all humility). 

I also realize that by being new, I do miss out on some points-of-view that someone with more experience in the product might have. I try to deal with this in several ways. First, like I said, I have an awesome technical editor. He and I work hand in hand during the process, and truthfully, I use him more that most people use their technical editors. He catches things I might have missed, things I might have mis-understood and is there to help me with answers to questions. 

I also bring on experts as contributing authors. I really do like to share in the success of a book and give others opportunities to be a part of the process. Writing a technical book will not make you rich, but it does give you credentials in the professional world. It looks good on a resume and can certainly help your career.  As a leader and mentor, I like the people who work for me, or with me, to come with me on writing adventures so they can benefit from the experience. I also like having people I know on projects, because I know who they are and their work ethic. I also will bring on an expert or two, as a contributing author, for more advanced chapters either as the sole author of those chapters or as a major contributing author to those chapters. This way, the advanced chapters get the instant experience of someone who has a few hours of experience with the product and some time in the trenches.

In the end, this approach to book writing seems to work really well. I hope it will work as well with the GoldenGate book.

Next time I'll blog more about the writing process and the editing process. Sometimes the editing process is the hardest part of writing....