Recovery Monkey: Musings on backups, storage, tuning and more

Choose a Topic:

Mon
5
Jan '09

So what exactly is IBM trying to do with the XIV?

By now most people dealing with storage know that IBM acquired the XIV technology. What IBM is doing now is trying to push the technology to everyone and their dog, for reasons we’ll get into…

I just hope IBM gets their storage act together since now they’re selling products made by 4-5 different vendors, with zero interoperability between them (maybe SVC is the “one ring to rule them all”?)

In a nutshell, the way XIV works is by using normal servers running Linux and the XIV “sauce” and coupling them together via an Ethernet backbone. A few of the nodes get FC cards and can become FC targets. A few more of the features:

  • Thin provisioning
  • Snaps
  • Synchronous (only) replication
  • Easy to use (there’s not much you can do with it)
  • Uses RAID-X (no global spares, merely there’s space on each drive, faster rebuilds are possible)
  • Only mirrored
  • A good amount of total cache per system since each server has several GB of RAM BUT the cache is NOT global (each node simply caches the data for its local disks).

IBM claims insane performance numbers with the XIV (“it will destroy DMX/USP!” — sure). But let’s take a look at how everything looks:

  • 180 maximum (or minimum) drives (you can get a half config but I think you always get the 180 drives but license half, I might be mistaken - I believe you have to make a commitment that you’ll buy the whole thing in 1 year)
  • Normal Linux servers do everything
  • Only SATA
  • The backbone is Ethernet, not FC or Infiniband (much, much higher latency is incurred by Ethernet vs the other technologies)

The way IBM claims they can sustain high speed is to not try and make the SATA drives get bound by their low transactional performance vs 15K FC drives or, even worse, SSDs. From what I understand (and IBM employees feel free to chime in) XIV:

  1. Ingests data using a few of the front-end nodes
  2. Tries to break up the datastream into 1MB chunks
  3. The algorithm tries to pseudo-randomly spread the 1MB chunks and mirror them among the nodes (the simple rule being that a 1MB chunk cannot have a mirror on the same server/shelf of drives!)

Obviously, by doing effectively as much as possible large block writes to the SATA drives and using the cache to great effect, one should be able to see the 180 SATA drives perform pretty much as fast as possible (ideally, the drives should be seeing streaming instead of random data). However (there’s always that little word…)

  1. There is no magic!
  2. If the incoming random IOPS are coming at too great a rate (OLTP scenarios), any cache can get saturated (the writes HAVE to be flushed to disk, I don’t care what array you have!) and it all boils down to the actual number of disks in the box. The box is said to do 20,000 IOPS if that happens - which I think is optimistic at 111 IOPS/drive! At any rate, 20,000 IOPS is less than what even small boxes from EMC or other vendors can do when they run out of cache. Where’s the performance advantage of XIV?
  3. The “randomization removing algorithm”, if indeed there’s such a thing in the box, will have issues with more than 1-2 servers sending it stuff
  4. See #1!

Like with anything, you can only extract so much efficiency out of a given system before it blows up.

An EMC CX4-960 could be configured with 960 drives. Even assuming that not all are used due to spares etc. you are left with a system with over 5 times the number of physical disks vs an XIV, tons more capacity etc. Even if the “magic” of XIV makes it more efficient, are those XIV SATA drives really 5 times more efficient (5 times would make it EQUAL to the 960 performance, XIV would have to be well over 5 times more efficient than an EMC box of equivalent size to beat the 960).

Let’s put it that way:

If my system was as efficient as IBM claims, and I had IBM’s money, it’d buy all the competitive arrays, even at several times the size of my box, and publicize all kinds of benchmarks showing just how cool my box is vs the competition. You just can’t find that info anywhere, though.

Regarding innovation: Other vendors have had similar chunklet wide striping for years now (HP EVA, 3Par, Compellent if I’m not mistaken, maybe more). 3Par for sure does hot sparing similar to an XIV (they reserve space on each drive). 3Par can also grow way bigger than XIV (over 1,000 drives).

So, if I want a box with thin provisioning, wide striping, sparing like XIV but the ability to choose among different drive types, why not just get a 3Par? What is the compelling value of XIV, short of being able to push 180 SATA drives well? Nobody has been able to answer this.

I’m just trying to understand XIV’s value prop since:

  1. It’s not faster unless you compare it to poorly architected configs
  2. It has less than 50% efficiency at best, so it’s not good for bulk storage
  3. It’s not cheap from what I’ve seen
  4. Burns a ton of power
  5. Cannot scale AT ALL
  6. Cannot tier within the box (NO drive choices besides 1TB SATA)
  7. Cannot replicate asynchronously
  8. Has no application integration
  9. No Quality of Service performance guarantees
  10. No ability to granularly configure it
  11. Is highly immature technology with a small handful of reference customers and a tiny number of installs! (I guess everyone has to start somewhere but do YOU want to be the guinea pig?)

Unless your needs are exactly what XIV provides, why would you ever buy one? Even if your space/performance needs are in the XIV neighborhood there are other far more capable storage systems out there for less money!

IBM is not stupid, or at least I hope not. So, what IBM is doing is pretty much handing out XIVs to whoever will take one. If you get one, think of yourself as a beta tester. Because I hardly believe that IBM bought the XIV IP without seeing some kind of roadmap, otherwise the purchase would be kinda stupid! If you are a beta-tester, be aware that:

  • XIV cheats with benchmarks that write zeros to the disk or read from not previously-accessed addresses
  • XIV will be super-fast with 1-2 hosts pushing it, push it realistically with a real number of hosts
  • Try to load up the box since if it’s not full enough you’ll get an extremely skewed view of performance - put even dummy data inside but fill it to 80% and then run benchmarks!
  • Test with your applications, not artificial benchmarks
  • Do not accept the box in your datacenter before you see a quote! In at least 3 cases that I know of IBM drops off the box without giving you even a ballpark figure. I think that’s insane.

And last, but not least: I keep hearing and reading about the following being true, I’d love IBM engineers to disprove it:

If you remove 2-3 drives from different trays simultaneously from a loaded system then you will suffer a catastrophic failure (logically makes sense looking at how the chunks get allocated but I’d love to know how it works in real life). And before someone tells me that this never happens in real life, It’s personally happened to me at least once (lost 2 drives in rapid succession) and many other people I know that have any serious real-world experience…

D

'

On current smartphones

The time has come for me to get a new phone (my current one can’t keep up with the demands and the speed or lack thereof ends up frustrating me).

So I’ve been looking at the plethora of devices out there - Berries, Windows Mobile, iPhone, etc (disclaimer: I’ve been a Blackberry user for many years now).

For me, the ideal mobile device needs to:

  • Synchronize seamlessly all my Exchange stuff
  • Be able to display PDF and office docs (not necessarily edit them)
  • Be a great phone (reception, clarity)
  • Have tethering ability
  • Be fast when I multitask on it
  • Offer GPS (almost all current ones do)
  • Have a decent supply of third-party apps
  • Be able to last me for a whole day (NOT just a business day) of pretty heavy usage
  • Have not so much an intuitive OS but an OS that doesn’t get in the way
  • Let me input text very, very fast (I’m writing this on my phone now)
  • Be tough (Mil-Spec would be great)
  • The ability to play music/videos is not essential but is nice to have (all do it now)
  • Camera nice-to-have but it doesn’t need to be amazing
  • Should be able to have a decent web browser
  • Shouldn’t be ridiculously huge…

So here’s the Executive Summary if your needs coincide with mine:

- Get a Blackberry Bold

For the nitty-gritty:

We HAVE to mention the iPhone, it’s a marvel of social engineering, industrial design and amazing marketing/branding. Of course the battery life utterly sucks if you try to use it the way I’d need to but, most importantly: I cannot type on the sucker! I don’t have abnormally large sausage fingers, indeed I believe my digits are downright elegant, yep I just cannot type fast or accurately on the iPhone (this paragraph might have taken me 10 mins to write on it). So we stike that one out.

Then you have the new Blackberry Storm, also touch-screen. On this one, the entire screen is a gigantic button that you need to press in order for it to register. I found that this approach seems to make it way more accurate for me than the iPhone. The battery life and build are also great. Too bad that the hardware can’t keep up, it feels decidedly slow, more so than the iPhone. Scratch that one too.

Then we have the narrow-form-factor Berries. Can’t type on them quickly. Out they all go.

Next are the various and sundry Windows Mobile devices. Almost too much choice here, huge third-party support, some great hardware from a few vendors. But I find that the OS really gets in my way and all of them also feel amazingly slow. Battery life is no great shakes, either.

Nokia has some good ones, the E71 is my favorite, but they don’t sync that elegantly with Exchange plus the keyboard is weird. Great build, though. If you like its keyboard go for it. OS can take some getting used to…

What remains is the Blackberry Bold. Sure, size-wise it feels like holding a slipper against your head (fortunately size was never a very important criterion) but it passes almost all the other tests! It also lets you send/receive emails while on the phone, feels fast, and has an amazing keyboard. Probably because it’s slipper-sized…

Well-made but it’s so nice that it needs a decent case to get ruggedized so you keep it looking nice, in which case it’ll look more like a size 13 boot against my face and I won’t be able to see just how nice it is anyway.

Am I alone in believing that many people would gladly pay a premium for a sleek, ruggedized device that doesn’t look like a Casio G-Shock? I’d be totally OK with the silly and easily disfigured plastic chromed bits being replaced by Kevlar or rubber, a scratch-proof screen, the ability to withstand immersion for 30′, successful drop-testing from 1 story to concrete, flexible circuit boards (not the ultra-thin ribbon type, you can get boards that are almost rubbery), Mil-Spec connectors, port covers…

It’s all possible, it just adds to the cost. But I guarantee most professionals will pay $100-200 more for the ruggedized model that doesn’t need clunky cases. Ericsson, Siemens and Nokia all had standard phones (never made it to the US) that would fit the above description with the exception of the scratch-proof screen (the Ericsson one was pretty amazing - they suggested you wash it to get rid of the dirt - albeit pretty large), but they slowly stopped making them. They weren’t even much more expensive than the plain models!

The old, thick Blackberries used to be pretty tough, I dropped mine onto concrete many, many times (drop-kicked it once) and the only damage was that the vibrating thingy inside stopped working 100% of the time, a no-no among Berry addicts. It did look scratched but it wasn’t painted on so the scratches weren’t that visible.

I hear the iPhone can be tough, at least the original one. The 3G - not so sure. A colleague had his stop working after he dropped it 3 feet. It landed on its back (should be an easy knock to absorb), you can’t even see a scratch on it. Unacceptable, IMO.

D