mySQL, noSQL, and Key Value datastores

Monolithic RDBMs are losing ground to key-value data stores, particularly persistent distributed in nature. mySQL mounting problems was perhaps the key reason (pun intended) people looked elsewhere. Google's brilliant engineers realized that a key/value data model can satisfy the needs of almost every class of application that needs a datastore backend.

Key/value datastores are simple to build, easy to understand, easy to optimize, easy to scale. The, now famous, CAP theorem states that it is not practically possible to guarantee consistency, availability and partitioning resilience/tolerance all at the same time; one of those traits has to be sacrificed. Again, most applications really do not require all three to function. The CAP theorem is most likely derived from the Project Triangle mode.

Most web-based applications are built on simple data models. Most web-applications eventually suffer from service capacity and availability issues(i.e scalability woes). It is trivial to scale out(vertically) application logic processing(application servers), HTTP requests processing(web servers, load balancers).
It is not easy to scale out an RDBMS. Some expensive systems(Oracle, etc) provide ways to address those issues (e.g Oracle RAC) but its expensive to deploy them, and most of them rely on a shared everything setup which just doesn't work in the long run. (Shared nothing is really the way to go).

Google released a bunch of papers ( actually, a bazillion of papers ), many of them defining and shaping the development of future related technologies. Namely, the papers describing GFS, BigTable, MapReduce (and of course, the paper the changed everything, "The Anatomy of a Large-Scale Hypertextual Web Search Engine" ) steered everyone to the right direction.

In the datastores domain, Hadoop/HBase, Radix, Cassandra and others, based on BigTable and Amazon's Dynamo papers, all relying on the simple key/value datastore model, are gaining market share - rightly so. Coupled with Memmache and similar services(in-memory key/value stores) they are solving the problems of service capacity and availability. This is a paradigm shift. Its a downhill for heavy-footprint, complex and inflexible datastore systems. They wont go away but will not be such a valuable(pun intended) component in tomorrow's technology landscape.

We are going to gradually migrate from RDBMs - though, we are not relying that much on them nowadays - to a key/value datastore (we are currently building one, also based on BigTable and Dynamo ). If nothing else, those simple systems are both simple and beautiful (for the most part).

on Apple's iPad launch : the Aftermath

The iPad was launched yesterday. What was, perhaps, the most hyped and anticipated product of this eon, finally became known as Steve Jobs held it proudly in his hands. These kind of products usually wind up being the recipients of all kinds of crazy expectations. Everyone wants the next-big-thing to do everything, better, in new radical ways and, if that is not enough, they expect even more in the end (in the case of Apple, that comes after Job's "one more thing" statement). All in all, most people are not pleased with the iPad. I am not one of those people. Here is my take on (some of) the 'issues' raised so far.

The name
'iPad' indeed feels wrong. I was expecting it to be called 'tablet' or 'Applet tablet' or something along those lines. Of course, I failed to consider the fact that this would not be a tablet, according to Apple at least. This is not a netbook either. In fact, this is a product that carves a new niche and defines it. So Apple didn't use tablet in the title. I remember when the first Macbook came out. People didn't like the name. They don't mind about it now - in fact they may even come to like it.

Multitasking
The iPad does not allow multiple apps to run at the same time. This made much sense on the iPhone, but would make little to no sense to a laptop or netbook. Well, again, this is neither of those. My gut feeling tells me the reason Apple chose to go with this is threefold.

  • The existing iPhone apps are built and executed in a way that just doesn't translate well to a multitasking environment. It is not impossible of course. Apple decided to play safe with this one.
  • The RAM on this thing would not allow for a multiple applications to run in parallel efficiently. Lets just say that Apple, again, decided to play safe with this one too.
  • Apple's vision for the iPad is very specific. A consumption device that does some things extremely well, one thing at a time.

Cameras
I would have loved it if iPad had a front-facing camera so that I could video chat with my brother. Well, actually, I never use video chat, but that need is valid for enough folks to make it an important omission. Apple has been keen on adding an iSight camera on just about every Mac product it has released. iMacs, Macbooks, the cinema displays, you name it. They love it when people video chat over iChat, use PhotoBooth to go silly and whatnot. What became apparent with the iPhone and even more apparent with the iPad is that Apple, presumably, has a very good set of reasons that led it to (at least for now) make it impossible (one way or another) to do so on this class of devices. It is likely that AT&T and other carriers are to blame here. Apple is giving up some features in exchange for others(better deals with the carriers?). Win some, lose some.

Adobe Flash support
Apple doesn't like Flash. It can't be more obvious than that. They could list a number of technical and semi-valid reasons as to why this is a bad thing, but none of that would matter. I personally couldn't care less about Flash support on Safari, but the vast majority of potential users would, especially the folks in the US where, I hear, Hulu has become the go-to site for all things entertainment there. Unless things go way south for Apple, I don't see it changing its stance on the subject.

On screen keyboard
When the iPhone came out, naysayers and pessimists sure had lots to say about the onscreen virtual keyboard. Nowadays though, people seem to actually prefer those kind of keyboards to the traditional physical ones, me included. Why waste device physical space, weight and looks for a 'real' keyboard, which is there even if you don't want it, when this new virtual keyboard works for you? I have been trying to type using both hands for the past few days. I can type now at least x2 many words/minute than I was able to do so in the past, when I was using just my thumb and I hope, expect, to get better at it. On the iPad, which features even bigger keys, things should be even better for me.

Books and magazines
I wanted an ebooks reader for quite some time now. I was hoping for an ebooks store tightly integrated to iTunes and the iTunes store, the ability to subscribe to magazines (Wired, Edge, ..) and have them delivered to my 'subscriptions inbox' ( with a nice badge indicating new subscriptions count; me and Stelios would sure love that ) and a reader that would provide all benefits PDF readers come with, but with even more thrown in. Well, it won't really work like that on the iPad, at least for now, but this iBooks application sure looks sweet and well done. Apparently, each magazine, newspaper or other content provider will come up with its own solution to the digital content challenge ( NYTimes app demoed was pretty impressive ), which is one way to do it. Apple is playing safe there, again. There is no subscriptions aggregation place/app, or anything like that. Wired will need to build its own app and same will be true for everyone else. I can't wait to see what they will come up with.

All in all, this is v1.0 of a new product that, again, occupies and defines a new category on its own. Recall v1.0 of OS X, the iMac, the iPod, or any other product, produced by any company. Most v1.0 products are there to establish a baseline. Evolution bless them with more feature in later releases. This will certainly be the case with the iPad too. I never needed a tablet device ( my MBP 17" is everything I would 'ever' need ), but I am so buying one for me and Dora ( and for my brother and Dimitris if they themselves wont get one ). Exciting times (ahead).

On important technologies: LLVM, CocoaTouch, Caching, Multi-core designs

As far as I am concerned, LLVM, CocoaTouch and memory based cache servers make up the the set of software technologies that will affect all things computing next year onwards.

LLVM is going to push code compilation and optimization to the next level. Building a new language is borderline trivial using LLVM technology. You produce the IR and the LLVM backend takes care of everything for you. I will be surprised f the 'fastest' Javascript implementation for 2009 won't be based on LLVM.

Apple's CocoaTouch is so well done, so well thought out (we are still on iPone SDK 1.0 and that speaks volumes) that it will be hard not to imagine Apple advancing and reusing the technology on, say, tablets and even making available the CocoaTouch extensions to OS X existing frameworks features set.

The ever increasing complexity of web-based services along with the rising number of users of those services and the need to sustain a user experience that depends on responding to user's actions as fast as possible, calls for the kind of tools and services that utilize intelligent RAM based caching. By caching just about everything, thanks to the ratio of reads/writes, gets/puts, resources (CPU, disks, etc) use drops by orders of magnitude while at the same time satisfying the need for a perceived fast responses to a matching requests. Developers and researchers most likely will come up with even better systems, ones that deal with cache coherency transparently, mirroring and synchronization, etc. This new realization may even render expensive, large and over-complicated systems irrelevant(e.g Oracle RDBMS). The only potential problem is the saturation of the network links, which is another class of problems researchers should look into in the near future.

On the CPUs side, everyone seems to have finally agreed that we can no longer scale vertically. We have to scale horizontally by exploiting parallelization and multi-core designs, potentially coupled with technologies such as NUMA. The Cell processor, Sun's UltraSPARC T, Intel's and AMD Multi-Core designs are build on that principles. Using all those cores('threads') efficiently, both in therms of throughput, scheduling and access to system resources is not (going to be) an easy task, but the benefits and the need to go forward justify this new approach, if not make it necessary. I personally couldn't be more excited about the possibilities availed by those architectures.

Mark Papadakis

Moires, Heraklio, Crete, Greece
Bytes conjurer. Seeking knowledge 24x7
About MarkP

Favorite Quotations

  • Focused, hard work is the real key to success. Keep your eyes on the goal, and just keep taking the next step towards completing it. If you aren't sure which way to do something, do it both ways and see which works best.
  • Focus is a matter of deciding what things you are not going to do.
  • Simple is Beautiful
  • In the information age, the barriers [to entry into programming] just aren't there. The barriers are self imposed. If you want to set off and go develop some grand new thing, you don't need millions of dollars of capitalization. You need enough pizza and Diet Coke to stick in your refrigerator, a cheap PC to work on, and the dedication to go through with it. We slept on floors. We waded across rivers.
  • Fear is the path to the Dark Side. Fear leads to anger, anger leads to hate, hate leads to suffering.
  • Easy is what I know, difficult is what I don't.

    Activity Log

  • 18.03 20:28  @stevedekorte : I know at least one person close to me (@stelabouras) who 'd go to great lengths to be granted beta access to this game :)
  • 14.03 21:12  Unity 3D (http://unity3d.com/) is one of the most promising,exciting and important technologies/products of this gen.Definitely one to watch




Search