Wednesday, December 19, 2007

Storage

As data keeps growing exponentially, the need for more advanced storage technologies has increased. We take a look at the upcoming technologies that will make storage simpler in future

Data growth in any organization is an irreversible process. You can control and manage data, but you can't reduce it. Hence, there will always be the need for efficient ways of storing and managing data. Since data is generated by umpteen types of applications in zillions of formats, some pretty innovative ways are required for storing it. That's why storage has always remained such a hot topic, and it will continue to remain so as long as there's growing data.

Market drivers
The ever increasing amount of data brings with it not only requirements for more storage capacity, but also issues of security, physical space for hosting more storage devices, concerns of managing them, the speed of accessing data, and much more. So much so that storage has become the driver for innovation across so many industries. It has provided an opportunity to the software development world for creating applications to store, back up, retrieve, index, and search data. It has given the security industry an opportunity to find new ways of securing the growing volumes of data. The storage industry itself is coping with the challenge of cramming higher storage capacity in smaller form factors. The growing volume of data is also creating opportunities in the storage management and virtualization space.


Even the consumer electronics industry is banking on storage, because most consumer appliances and even digital gadgets require lots of it.

MP3 players, smartphones, digital cameras and camcorders, Digital Video Recorders, are just a few examples that require storage.Plus, storage is also driving niche markets like IP Surveillance.

Glimpses 2007
1) Hard drive capacities hit the 1 Terabyte mark.
2) Tiny personal NAS boxes became ubiquitous this year. New security standards emerged for encrypting data at rest, like the IEEE 1619.3.
3) Email archival solutions became widespread
4) USB based flash drive capacities exceeded 4 GB.
5) NAS boxes, ILM, Continuous Data Protection, Document and Content Management Systems were on the purchase list.

Hot trends this year
We've had a pretty eventful year as far as storage goes. Several foundation stones have been laid this year that will set the tone for things to come in the future.

While the trend of increasing capacities and reducing costs in hard drives has been around for a long time now, this year they finally hit the 1 TB mark. In fact, the competition to bring out higher capacity drives has become very similar to the GHz wars of the past in microprocessors, which continued until they reached the limit. Thankfully, hard drive capacities have not yet reached their limits. We'll see capacities beyond 1 TB in a single hard drive in the coming years.

Solid State storage drives also hit the market this year, and many laptop vendors launched their models with them. So imagine a laptop with no magnetic hard drive-just a single solid state drive with flash memory inside and a SATA interface. The technology behind high capacity solid state drives is NAND, which stands for 'Not AND'. In the digital world, NAND is a kind of logic gate.

Jargon Buster
Storage Terms you should know for next year


1) ILM (Information Lifecycle Management): A comprehensive approach for administering Storage systems on computing devices. The information system's data and associated metadata are effectively managed right from its creation and initial storage, till the time it becomes obsolete and is deleted.
2) CDP (Continuous Data Protection): It is nothing in line with traditional back up or RAID/replication/mirroring. It refers to backup of data in such a manner that every change made to the data is automatically saved. Essentially it captures every version of the data that the user saves. You can restore data to any point in time.
3) Storage Virtualization: Commonly used in Storage Area Network, it is mainly the pooling of all physical storage from various network devices into what appears to be a single storage device which is managed from a central console.
4) VTL (Virtual Tape Library): It's mainly a virtualization technology for data storage used primarily for data backup and recovery purposes. It presents a storage component as a tape drive or library for use with existing backup software.
5) SSD (Solid State Drives): A data storage device which uses solid-state memory to store persistent data. SSDs comprise of NAND flash which is non-volatile or SDRAM which is volatile. They are already available as 32GB SSD and 64GB SSD from various vendors and 128GB will be available soon. Super Computers.


So in effect, storage has become even more ubiquitous than before. Just about every industry demands it today for different applications. In enterprises, branch office automation led to the need for anytime anywhere access to data, which made the concept of Wide Area File Services more popular. Data center consolidation and infrastructure centralization led to greater demands for more storage and backup. The need to standardize and comply led to the popularity of email archival solutions, and even new data encryption technologies.

Network Attached Storage, or NAS for short, saw terrific growth this year. In fact, the interesting thing about this market was that NAS boxes became available for everyone, right from personal users to data centers. Interestingly, 1 TB NAS for personal use or for small offices became commonplace this year.


Information Lifecycle Management solutions were also on top of the storage purchase chart for many enterprises. Likewise, several other storage terms were pretty common place this year. There was a lot of interest in Continuous Data Protection or CDP for instance. With growing amount of content, organizations started considering content and document management solutions.

Key Predictions for the future
Now we come to the exciting part. What all can you expect in the storage arena next year. So here's our list of predictions to help you be prepared for next year:


Storage Virtualization implementations will rise
This is one of the most talked about areas in storage for a long time, and has not been able to take off as well as its cousin in the server world. This year saw some traction in storage virtualization, with vendors introducing many new products and early bird implementations. So the momentum has just started, and we're likely to see a lot more action in storage virtualization next year. While considering storage virtualization, don't get blinded by its benefits. Look at the other side as well. That's because it's not as easy as adding an abstraction layer on top of your existing heterogeneous storage infrastructure and have a virtualized storage environment. Be prepared to add more storage equipment to manage your existing ones. Likewise, there are many different ways of doing storage virtualization, and choosing the right one requires a discussion in itself.
Solid State drives will become more commonplace
We all know that solid state memory is much faster than magnetic memory. It has no moving parts, runs at much higher speed, and is not affected by wear and tear. That's why RAM is faster than a hard drive. However, solid state memory is also much more expensive, which is why we don't see everyone rushing to replace all magnetic storage with it. But there are quite a few interesting developments in this area. Many notebooks vendors this year started shipping their products with solid state drives inside. A race has started amongst memory manufacturers to introduce higher capacity solid state drives, with the latest being A-Data's 128 GB SSD.


Currently, if you were to buy a laptop with a SSD inside, then you have to shell out at least 40-50K extra for it. But the growing competition amongst flash drive manufacturers and economies of scale will eventually bring down prices, and you might find lots of SSD based notebooks in the market next year. The advantages are many. SSDs are thinner, lighter, and faster, which should improve system performance, make notebooks lighter and perhaps more power friendly as well.


A peek into the future
1) Storage virtualization will become widespread.
2) Solid State drives in notebooks will become common place.
3) Hard drive capacities will reach 2 TB+ range for desktops.
4) Virtual Tape Libraries will become more common.
5) Full Disk Encryption or FDE based drives will hit the market for data centers.

Full disk encryption based drives will emerge for storage security
One of the biggest challenges most organizations are facing today is securing their ever growing volumes of data. This is becoming even a bigger concern with so many laptops being used amongst organizations. Plus of course there's all the data being backed up to tape drives. All this data in computers, which is not travelling across a network is known as data at rest. The answer to doing this is to encrypt it. There are many ways of doing this, and we've seen many attempts at it over the past few years with vendors introducing various types of solutions for doing so. There are software programs that encrypt data on the fly, and even appliances that encrypt data before backing it up. This year however, another technology has started gaining ground, which aims to build encryption capabilities within the hard drive itself, a concept known as Full-Disk Encryption or FDE. This is currently being backed up by Seagate, IBM, and LSI, and even IEEE is creating a management standard, called 1619.3, to ensure interoperability.


If everything goes well, then you should see FDE based drives shipping next year. The logic behind having FDE based drives is also quite strong. If the logic of encrypting all data lies within the hard drive controller itself, it will improve performance. Moreover, even as more drives get added, the performance won't degrade because each drive will be handling its own encryption.

VTLs will grow in popularity
This is one area that's becoming strong competitor to tape based backup. Virtual Tape Libraries will become more popular next year, as vendors gear up with more offerings.


Storage has become so ubiquitous and there have been so many developments around it that one can go on and on talking about them. But we'll end the discussion here and bring you more news on it in our forthcoming issues.