Scouring the Business of Technology

Moving to a new blog

This will be my last post for this blog… Hope you enjoy my previous articles.

Meanwhile, I am consolidating all my other blogs into one which unifies my personal mission to bring a clear pathway toward a meaningful IT career.

Henceforth, all my blog posts will be published at ITJUMPSTART.WORDPRESS.COM.

Business Model of Apple


Apple has just 8 percent of the global phone market, but makes about 80 percent of the industry’s operating profits.

Wall Street analysts expect Apple Inc. to post a profit of $9.2 billion for the January to March quarter when it reports on Tuesday. That’s roughly in line with the profit expected from the world’s largest oil company, Exxon Mobil Corp.

The majority of the profit will come from iPhone sales, especially now that three of the four national US wireless carriers -AT&T, Sprint, and Verizon- sell the phone.

Siri-Controlled Car

Proof of concept of Siri remote car control, courtesy of Brandon Fiquett.

MySQL High Availability


  • Baron Schwartz aka Xaprb dislikes MMM as HA tool
  • Traditional setup: master-slave replication
  • At a specified interval – the heartbeat – the clustering solution will see if your MySQL instance is still running. The default heartbeat is often something like 10 seconds and minimum granularity tends to be one second. Problem: What happens between these seconds the clustering solution is completely unaware of. If you have 5000 trx/sec, you could have fifty-thousand failures before an attempt to fix the error is made.
  • The clustering solution takes small peeks into MySQL from the outside, but other than that MySQL remains a black box. Problem: Say that there are network errors. This causes your transactions to fail, and it also causes replication to fail. So which one failed first? If transactions started failing first, and replication was still working until the end, then you are fine and you can fail over to the other node. But if replication failed first, then you will lose data if you fail over, because not everything was replicated. Your clustering software has no idea, it wasn’t looking when the error happened.
  • Typically the clustering software itself will have some communication going on, which has the benefit that it verifies that the network connection between nodes is ok. This is in itself useful, sure. Problem: But just like above, if the clustering software detects that network has failed, it’s still mostly unaware of the state of MySQL. The failover decision is done blindly.
  • In a typical setup like above, the clustering software is actually just checking whether there is a network connection between the two MySQL nodes. If yes, that makes it happy. Problem: Nobody is really checking whether the application servers can really connect to those MySQL nodes! This is one of the most classic errors to make in software programming: when testing for error conditions, you are not testing the thing you actually want to know the answer to, but testing something else. Kind of like the guy in the joke who was searching for his keys under the lamp where there was light, not where he actually lost the keys.

Henrik Ingo’s favored solution: Galera (synchronous multi-master replication)

  • Thanks to synchronous replication and Galera’s quorum mechanism, no commits are lost anywhere. When the failure happens, it will be detected as part of the replication process. The Galera nodes themselves figure out which node is thrown out of the cluster, and the remaining ones – who “have quorum” – proceed to commit the transaction. Application nodes that connected to the failing node will of course receive errors as their commits fail.
  • There is no need for maintaining master-slave state, virtual ip or to do any failover. The application can connect to any of the Galera nodes and if the transaction succeeds, then it succeeds. If it fails, retry with another node.
  • As a side comment: since the replication is synchronous there is no slave lag as you are familiar from MySQL replication, which can also cause you to lose data. This is not a weakness of clustering frameworks, but a strength of Galera compared to classic MySQL replication that most people out there still are using.

Caveat: This works if you use JDBC (see Oracle blog example)

So in short, when using a Galera cluster, you should use mysql:loadbalance: in front of your JDBC connection string. This allows you to then give a list of MySQL nodes, which are all writeable masters. The JDBC driver will connect to any one of them and commit the transaction. If a node is not available, it will just try another one. (If a transaction was already in progress, it will fail with an exception, you can then retry it and it will just connect to a new node.)

Related links:

MySQL Master HA (MHA) tool – by Yoshinori Matsunobu

SkySQL commercial support for MHA

Three Database Clustering Users – by Josh Berkus


Networking as a Service

Courtesy: Cisco, GigaOM

Courtesy: Mozy

Note: This is a reprint from Open-E Blog

There is  plenty of talk about Bonding and Multipath IO, but it is very difficult to get solid information about either one. Typically what documentation can be found is very bulky and the most important practical questions go unanswered.

As a result the following questions are often heard:

  • When should I use bonding and when should I use multipath?
  • I was expecting better throughput with bonding, why am I not seeing this?
  • My RAID array shows 400MB/sec with local test, how can I get 400/MB sec outside?

Before we answer the above questions, let’s understand first how MPIO and bonding works.

MPIO allows a server with multiple NICs to transmit and receive I/O across all available interfaces to a corresponding MPIO-enabled server.  If a server has two 1Gb NICs and the storage server has two 1Gb NICs, the theoretical maximum throughput would be about 200 MB/s.

Link aggregation (LACP, 802.3ad, etc.) via NIC teaming does not work the same way as MPIO. Link aggregation does not improve the throughput of a single I/O flow.  A single flow will always traverse only one path.  The benefit of link aggregation is seen when several “unique” flows exist, each from different source. Each individual flow will be sent down its own available NIC interface which is determined by a hash algorithm. Thus with more unique flows, more NICs will provide greater aggregate throughput. Link aggregation will not provide improved throughput for iSCSI, although it does provide a degree of redundancy.

Bonding works between a server and switch. Numerous workstations using each using a single NIC connected to the switch will benefit from bonded connections between the switch and storage server.

MPIO works between a storage server and the client server, whether or not there is a switch in the path.

With these basics fact, it will now be easier to answer our questions.

Q: When do I need bonding, and when is multipath appropriate?

A: Bonding works for a NAS server with multiple workstations connected.

MPIO works between hosts and initiators on FC or iSCSI.  An example of MPIO configuration with a performance test showing 200MB/sec using dual Gb NIC’s is demonstrated step-by-step at: How to configure DSS V6 MPIO with Windows 2008 Server .

In short:

  • Bonding works for NAS
  • MPIO works for SAN

Get every new post delivered to your Inbox.

Join 53 other followers