Transforming Cloud Infrastructure to Support Big Data Storage and Workflows

As companies have turned to cloud-based services to store, manage and access big data, it has become clear that the cloud’s promise of virtually unlimited, on-demand increases in storage, computing, and bandwidth is hindered by a series of technical bottlenecks: transfer performance over the WAN, HTTP throughput within remote infrastructures, and size limitations of cloud object stores.

This session will discuss the principles of cloud object stores, using the examples of Amazon S3, Microsoft Azure, Akamai NetStorage and OpenStack Swift, and performance benchmarks of their native HTTP I/O. It will share best practices in orchestrating complex, large-scale big data workflows. It will also examine the requirements and challenges of such IT infrastructure designs (on-premise, in the cloud or hybrid), including integration of necessary high-speed transport technologies to power ultra-high speed data movement, and adoption of appropriate high-performance network-attached storage systems.

The session will also explore how organizations across different industries are using big data in the cloud for ever-greater efficiencies and innovation, including those in the media and entertainment industry and in the field of life sciences.

Learning Objectives

How to overcome the technical bottlenecks associated with cloud-based services
How to take advantage of the cloud for storing, managing and accessing big data
How to plan and implement complex, large-scale data workflows