Transforming Cloud Infrastructure to Support Big Data

Cloud systems promise virtually unlimited, on-demand increases in storage, computing, and bandwidth. As companies have turned to cloud-based services to store, manage and access big data, it has become clear that this promise is tempered by a series of technical bottlenecks: transfer performance over the WAN, HTTP throughput within remote infrastructures, and size limitations of the cloud object stores. This session will discuss principles of cloud object stores, using examples of Amazon S3, Microsoft Azure, and OpenStack Swift, and performance benchmarks of their native HTTP I/O. It will share best practices in orchestration of complex, large-scale big data workflows. It will also examine the requirements and challenges of such IT infrastructure designs (on-premise, in the cloud or hybrid), including integration of necessary high-speed transport technologies to power ultra-high speed data movement, and adoption of appropriate high-performance network-attached storage systems.

Learning Objectives

Attendees will learn methods to overcome the technical bottlenecks associated with using cloud-based services. Attendees will also gain insight into how to take advantage of the cloud for storing, managing and accessing big data using high-speed transport technologies and high-performance NAS systems.
Attendees will learn what it takes to plan and implement complex, large-scale data workflows. This includes the requirements and challenges of designing IT infrastructure, how to ensure secure file transfers and storage, and real-world examples from industry leaders.
Attendees will gain a better understanding of the differing requirements and challenges of on-premise, cloud and hybrid infrastructure designs.