Read the article: Ten Dials To Set at DatabaseJournal.com

As it turns out there are specific reasons and causes for MySQL replication to get out of sync.  We’ll discuss what is and isn’t compliant in a MySQL Replicated environment, and then most importantly, we’ll discuss a tool that can help you verify your environment, and show you what is or isn’t in sync and why.

I’m presenting this webcast with O’Reilly on Thursday January 22.  If you’re interested, click here to register.

Of particular interest to our readers was his mention that Open Source will continue to grow in importance, as budgets get squeezed.  Limited budgets mean looking at every dollar, so this month’s Open Insights issue 51, we talk about stretching your database dollar, and where and when open source database technologies may be right for you.

In this piece we talk about some of the ways to verify your setup, and make sure your slave does not drift out of sync with the master, and to alert you if it does.

I do see differences in how younger people adopt social networks into their working habits versus older people who may be stuck with their existing habits that work for them.  On the other hand, I also think it has to do with personalities, as to whether someone finds the answers on their own, versus using social means & conversation to get to the answer.

Whenever a publisher releases a 2nd Edition of a book, you know it was well received the first time around.  So that’s a good sign that the material has gotten people buying.  I would say in the computer reference and howto market, that’s a very good indication that the material is well written and relevant.  I certainly found it to be the case with this title.

If you’re looking for a quick & no nonsense howto book on MySQL development, look no further.  The book focuses on Ruby, Perl, PHP, Python and Java as examples.  So if you’re doing development, specifically web development, you’ll get a lot out of this title.

Wow, that’s quite a list of authors, but when you look at the material, you see why.  This book is a very indepth look at the MySQL server.  Intended for the intermediate to advanced DBAs and developers who want to know the inner workings of the server, as well as how to use many of it’s advanced features.

For instance the chapter on replication was quite good.  Given that you probably setup replication in five minutes, and are wondering weeks or months later why it’s not working, this chapter will give you some answers.  Using non-deterministic functions?  Mixing MyISAM and InnoDB tables in the same transactions?  Seeing some errors in your slave error log that don’t make sense?  After  finding out that there is something wrong, you may be more surprised that your slave can be out of sync with the master, and not even let you know about it.  The chapter recommends Maatkit’s mk-table-checksum as an assistant to identifying these problems.

All in all the book is superb, so take a look at the review for details, and go get yourself a copy!!

Replication is fairly straightforward to setup, however your slave databases can get out of sync, or throw errors.  We investigate some of the reasons why, and help you identify those before they come back to bite you!

We’ll give and overview of MySQL specifically for Oracle DBAs.  Since you already have the requisite conceptual
framework as an Oracle DBA, a quickstart will help you get up to speed with how things are done in the MySQL
world.

Outline

Introduction
1. Installation
2. User Management, Authentication + Privileges
3. Backup and Recovery
4. Replication Solutions Compared to DataGuard
5. Transactions, Query Optimization + Profiling
6. Views and Partitioning
7. Server Tuning and Optimization
Conclusions

Learning Objective 1
Bring Oracle DBAs up to speed with MySQL

Learning Objective 2
Feature comparison between Oracle + MySQL, what’s there, what’s not etc.

Learning Objective 3
Tuning and Profiling MySQL from an Oracle perspective.

Attendee Prerequisites
Intermediate Oracle DBA experience required.

We looked at the technology involved to scale up based on their growth rates, but found it was becoming cost
prohibitive.  They asked us how it could be done cheaper.

Stepping back from the problem, we devised a very unorthodox solution, which upon inspection may seem rather
obvious.  Yet the business had been doing things a certain way for so long, they had not considered this alternate
view.

Outline

Introduction
1. Initial meetings with client, discuss problems
2. Review of database systems
3. Cost estimate based on current growth patterns.
4. Discussions with client, and further review
5. Creative problem solving to do more with less hardware.
Conclusions

Learning Objective 1
Measuring growth of database applications

Learning Objective 2
Discussing business needs, and projecting requisite technology needs and requirements.

Learning Objective 3
Balancing technology costs with business requirements, and using creative problem solving to increase

Attendee Prerequisites
Beginning DBA skills.
Basic understanding of internet platform, client, webserver, and database.

Proposal details

Proposer

Sean Hull (hullsean@gmail.com)

Speakers

Sean Hull (hullsean@gmail.com)

Description

Drupal is an open-source CMS that has a huge user base and community. I recently managed the MySQL side of the equation for a Drupal-based website of a large television brand. There are some documents out there for high performance MySQL with Drupal, I found them lacking. So I did extensive benchmarking, and profiling of queries, to find the best overall setup.

Session type: 45 minute conference session

Topics: Business and Case Studies

Abstract

Drupal is the open-source CMS du-jour. It has become very popular very quickly, but performance isn’t necessarily tip-top out of the box.

In our implementation of MySQL for a drupal-based website of a large television brand, we went down the path of doing our own benchmarking for our expected pageviews, and ares of the site we thought would get the most traffic. We benchmark the base set of modules, and our own extensions as well using apachebench, http_load, supersmack, and sql-bench. We then optimized the queries inside those modules using various MySQL profiling tools such as EXPLAIN, innotop, slow query log, and show status.

In this presentation we discuss the site, the business requirements, the benchmarking tools, profiling tools, and the final results. It provides a window into the workings of a large high-traffic internet website.

Proposal details

Proposer

Sean Hull (hullsean@gmail.com)

Speakers

Sean Hull (hullsean@gmail.com)

Description

MySQL has a number of great features for managing archival data, and very large tables. We look at the MERGE storage engine first. Then we look at views which can hide many tables behind the scenes. And lastly we look at partitioned tables.

Session type: 45 minute conference session

Topics: Data Warehousing and Business Intelligence

Abstract

MySQL has some very powerful features for large databases now available in 5.1.

We’ll look at the MERGE storage engine first. Then we’ll take a look at views, and the strengths and weaknesses there. And lastly we’ll consider partitioning as an option.

Each of these solutions has pros and cons. When you consider performance, ease of administration, and manageability, they all have tradeoffs.

Proposer

Sean Hull (hullsean@gmail.com)

Speakers

Sean Hull (hullsean@gmail.com)

Description

MySQL Replication is powerful technology, but many things can trip you up. Sometimes the slaves get out of sync with the master, or dump errors into their logfiles. Sometimes replication just can’t keep up. In this presentation will explain how to verify with checksums, avoid traps in your code, monitor, and even help you improve the performance of your replication setup.

Abstract

There are a few big gotchas with MySQL’s statement based replication. Non-deterministic functions can cause the slave to get out of sync, mixing innodb and MyISAM tables, to name just a few. What you need to do is watch for errant and incompatible SQL, and then verify that your setup is correct. Verification can be done with checksums on the tables, and comparing those against the slave copies.

We’ll discuss all this, and the tools and techniques you can use to bulletproof your own replication setup. We’ll also discuss performance tuning. If your slave is getting slowly further and further behind, there are techniques, parameters, and re-architecting that can improve its speed and performance.

This point is not made to be cynical.  Rather it asks us to take a hard look at complicated and interconnected pieces, accept that sometimes  there are things hidden in that complexity that we can’t know about, and to plan for that.  Plan for trouble, monitor (ie regulate) things as best you can, but not so much that the system doesn’t function.  And be as proactive as possible.

These truths probably hold for all complex relationships, whether they are financial, or complex technical systems.

I’ve posted my review of Backup and Recovery over at Amazon.

The book covers a wide range of topics, and is one part new features, one part Oracle 101, and one part summary of all of the Oracle products.  It is primarily targeted to DBAs, Developers and System Administrators, albeit on the novice side, however it’s writing style, and technical depth are not unweildy, so IT managers, and business users can surely learn a lot from this book as well.  The writing style is quite good, and overall it is an enjoyable read.

In particular I found this discussion Scaling Out by Jason Sobel.   He talks about the cache coherency challenges they had, and how they used MySQL & memcache to get the job done.

Intro to Oracle’s Automatic Workload Repository (AWR) 

Paul Boutin’s recent article in Valley Wag discusses Robert Cringley’s declaration of the Death of the Database  that has everyone all excited.  At root is the idea of cloud computing, and the likes of the Google’s of the world storing all of our data, and managing all the dirty messy database storage seemlessly for us.

Yes, I’ll give you that for many applications, and small websites, this will certainly be the future.  Who wants to manage a database for every website.  But for the large clients of Oracle databases, the terrabyte datastores, datawarehouses, Oracle applications, and Financials, the backend datastore will remain a requirement.  This isn’t necessarily because a third party can’t do the job better, or that it wouldn’t make a business sleep better at night leaving the database management to the experts.   Nor is it that security couldn’t be implemented properly, to make the data available only to the business, and invisible to the prying eyes of the DBAs down the line.  No all of those problems are solvable.

The problem is one of handing over the keys to the kingdom.  Take the worldwide GPS system, for example.   Currently Europeans, Russians, and Chinese alike rely on a US built satellite system for GPS service.  Imagine military operations relying on US technology were the US to get into a scuffle with the Russians or the Chinese.  In the end business wants to see their data, if not physically, then confident of where those servers are, and who touches the data, the hardware, the backups etc.

I do agree with Cringley and Boutin that cloud computing will change things, and continue to put pressure on the big database vendors like Oracle, but I don’t think it’ll put them out of business anytime soon.