It seems like everyone is talking about containers these days, but not everyone is talking about storage – and they should be. The first wave of adoption of container technology was focused on micro services and ephemeral workloads. The next wave of adoption won’t be possible without persistent, shared storage. That’s why the SNIA Ethernet Storage Forum is hosting a live webcast on November 17^th, “Current State of Storage in the Container World.” In this webcast, we will provide an overview of Docker containers and the inherent challenge of persistence when containerizing traditional enterprise applications.  We will then examine the different storage solutions available for solving these challenges and provide the pros and cons of each. You’ll hear:

• An Overview of Containers
  • Quick history, where we are now
  • Virtual machines vs. Containers
  • How Docker containers work
  • Why containers are compelling for customers
  • Challenges
  • Storage
• Storage Options for Containers
  • NAS vs. SAN
  • Persistent and non-persistent
• Future Considerations
  • Opportunities for future work

This webcast should appeal to those interested in understanding the basics of containers and how it relates to storage used with containers. I encourage you to register today! We hope you can make it on November 17^th. And if you’re interested in keeping up with all that SNIA is doing with containers, please sign up for our Containers Opt-In Email list and we’ll be sure to keep you posted.

 

SNIA's 18th Storage Developer Conference is officially a success, with 124 general and breakout sessions;  Cloud Interoperability, Kinetic Storage, and SMB3 plugfests; ten Birds-of-a-Feather Sessions, and amazing networking among 450+ attendees.  Sessions on NVMe over Fabrics won the title of most attended, but Persistent Memory, Object Storage, and Performance were right behind.  Many thanks to SDC 2016 Sponsors, who engaged attendees in exciting technology discussions. For those not familiar with SDC, this technical industry event is designed for a variety of storage technologists at various levels from developers to architects to product managers and more.  And, true to SNIA's commitment to educating the industry on current and future disruptive technologies, SDC content is now available to all - whether you attended or not - for download and viewing. You'll want to stream keynotes from Citigroup, Toshiba, DSSD, Los Alamos National Labs, Broadcom, Microsemi, and Intel - they're available now on demand on SNIA's YouTube channel, SNIAVideo. All SDC presentations are now available for download; and over the next few months, you can continue to download SDC podcasts which combine audio and slides. The first podcast from SDC 2016 - on hyperscaler (as well as all 2015 SDC Podcasts) are available here, and more will be available in the coming weeks. SNIA thanks all its members and colleagues who contributed to make SDC a success! A special thanks goes out to the SNIA Technical Council, a select group of acknowledged industry experts who work to guide SNIA technical efforts. In addition to driving the agenda and content for SDC, the Technical Council oversees and manages SNIA Technical Work Groups, reviews architectures submitted by Work Groups, and is the SNIA's technical liaison to standards organizations. Learn more about these visionary leaders at http://www.snia.org/about/organization/tech_council. And finally, don't forget to mark your calendars now for SDC 2017 - September 11-14, 2017, again at the Hyatt Regency Santa Clara. Watch for the Call for Presentations to open in February 2017.

by Marty Foltyn

Persistent memory discussions are capturing the minds of SNIA members and colleagues.  At last month’s SNIA Storage Developer Conference, NVM (non-volatile memory) and NVMe sessions were standing-room-only, and opinion sharing continued into animated hallway discussions.  I encourage you to check out the many presentations on the SNIA SDC website, and to download the live recordings of the keynotes here.

SNIA continued their education on persistent memory at this week’s Memcon in Santa Clara CA. SNIA’s booth was packed with attendees asking questions like what is the difference between the different kinds of NVDIMMs (you’ll want to check out our new snia_nvdimm_infographic), and is NVDIMM a standard (indeed, it is, JEDEC just released the DDR4 NVDIMM-N Design Standard Revision 1.0 last month, and you can download the link from our website).

The work being done within SNIA on persistent memory is contributing to a seachange in the industry – the convergence of memory and storage – perhaps the most revolutionary change since the invention of the transistor more than 60 years ago.  To learn more, check out this interview with Jim Pappas, SNIA’s Vice-Chairman and co-chair of the SNIA Solid State Storage Initiative.  And mark your calendar for January 18, 2017, when SNIA will hold the 5th annual Persistent Memory Summit in San Jose CA.  The latest details can be found here.

 

 

The first installment of our "colorful" Webcast series, "Everything You Wanted To Know about Storage But Were Too Proud To Ask – Part Chartreuse," covered the fundamental elements of storage systems. If you missed it, you can check it out on-demand. On November 1^st, we'll be back at it, focusing on the network aspect of storage systems with "Everything You Wanted To Know About Storage But Were Too Proud To Ask – Part Mauve." As with any technical field, it's too easy to dive into the jargon of the pieces and expect people to know exactly what you mean. Unfortunately, some of the terms may have alternative meanings in other areas of technology. In this Webcast, we look at some of those terms specifically and discuss them as they relate to storage networking systems. In particular, you'll find out what we mean when we talk about:

• Channel versus Busses
• Control Plane versus Data Plane
• Fabric versus Network

Register now for Part Mauve of "Everything You Wanted To Know About Storage But Were Too Proud to Ask. For people who are familiar with data center technology, whether it be compute, programming, or even storage itself, some of these concepts may seem intuitive and obvious... until you start talking to people who are really into this stuff. This series of Webcasts will help be your Secret Decoder Ring to unlock the mysteries of what is going on when you hear these conversations. We hope to see you there! Update: If you missed the live event, it's now available  on-demand. You can also  download the webcast slides.    

The first installment of our “colorful” Webcast series, “Everything You Wanted To Know about Storage But Were Too Proud To Ask – Part Chartreuse,” covered the fundamental elements of storage systems. If you missed it, you can check it out on-demand. On November 1^st, we’ll be back at it, focusing on the network aspect of storage systems with “Everything You Wanted To Know About Storage But Were Too Proud To Ask – Part Mauve.”

As with any technical field, it’s too easy to dive into the jargon of the pieces and expect people to know exactly what you mean. Unfortunately, some of the terms may have alternative meanings in other areas of technology. In this Webcast, we look at some of those terms specifically and discuss them as they relate to storage networking systems.

In particular, you’ll find out what we mean when we talk about:

• Channel versus Busses
• Control Plane versus Data Plane
• Fabric versus Network

Register now for Part Mauve of “Everything You Wanted To Know About Storage But Were Too Proud to Ask.

For people who are familiar with data center technology, whether it be compute, programming, or even storage itself, some of these concepts may seem intuitive and obvious… until you start talking to people who are really into this stuff. This series of Webcasts will help be your Secret Decoder Ring to unlock the mysteries of what is going on when you hear these conversations. We hope to see you there!

 

 

 

Based on record-breaking registration and attendance at our recent SNIA Cloud webcast, Intro to Containers, Container Storage and Docker, It’s clear that containers is a hot topic that folks want to learn more about – especially from a vendor-neutral authority like SNIA. If you missed the live event, it’s now available on-demand together with the webcast slides. We were bombarded with questions at the live webcast and we ran out of time before we could answer them all, so as promised, here are answers from our expert presenters, Chad Thibodeau and Keith Hudgins. Oh, and please don’t forget to register for part two of this webcast, Containers: Best Practices and Data Management Services, on December 7, 2016. Q: Would you please highlight key challenges a company may face to move from a hypervisor to container environment? CT: The main challenge that gets raised in moving from virtual machines to containers is around security as when deployed on bare-metal, all of the containers share the core operating system. However, there are arguments that containers can still be effectively isolated. KH: Primarily paring down your applications to their minimum running requirements. This can be quite difficult with long-entrenched legacy applications! Q: With a VM you allocate a finite amount of vCPU and RAM, with a high degree of confidence that those resources will be available to whatever workload is running in the VM. Is that also true of containers - does (or can) the workload get a guaranteed allocation of CPU and memory resources? CT: Keith, I'll let you address this one from the application microservice; from the storage side, an SLA or Quality-of-Service can be defined for a container volume if the storage provider offers this capability. KH: By default, you don't allocate CPU or ram availability. Most containers are small enough that it's not a consideration. However, if you need to specify priority, we have a method to do that. Please review the docs here. Q: Where are microservices most useful? Are there certain environments where they are more likely to be deployed & which verticals or type of solutions/apps will see more benefit? CT: Microservices can apply to applications within most verticals; for financials it was mentioned that Goldman Sachs is planning on containerizing 90% of their existing applications to web-service such as Netflix. Some of the determining factors are whether the application(s) would benefit from what container technology provides such as rapid deployment, lightweight, portability, and the ability to scale beyond typical monolithic applications. KH: Microservices are most useful with network-facing applications that don't require heavy transactional control. Note that it *is* possible to build transactional microservices, but the best practices on that route hasn't been optimized yet. Q: What OS version / Hypervisor, support containerization, are working towards cutting the "noisy neighbor" issue? CT: Containers are supported by both MS Windows and Linux operating systems. The specific version of Linux OS will be more dependent upon the level of capabilities included (Keith, more your area) and MS Windows Server 2016 is the first release of Windows with container (Docker) support. KH: Docker supports running containers under Windows and Linux kernels. We don't care whether it's on metal or virtualized. It's possible to set affinity groups in a production Docker installation to help manage noisy neighbor issues, but note that fundamentally Docker is NOT a multi-tenant system. Q: What is "stateful database"? How does it differ from regular databases? CT: Most databases are stateful such as Oracle, MySQL, Cassandra, MongoDB or Redis. The confusion may be around the Gartner quote which stated "Stateful Database Applications" in which they simply meant that databases are examples of stateful applications. KH: Any database is by definition stateful. A "stateless" container is one that is running a process that doesn't store persistent data to disk. This could be a caching system, web application server, load balancer, queue runner, or pretty much any component that doesn't need to store data. Everything else is "stateful" and needs some way to shove that data into a reliable datastore. Q: What factors should be considered when choosing between containers and virtual servers for a given project/use case? CT: The driving factors for container deployments are: portability, minimal footprint (low overhead since no hypervisor or guest OS), rapid provisioning and de-commissioning, scalability and largely open-source based. If any (or all) of these are deemed valuable to you, then you should consider container deployment technology. KH: That's a very broad question! It helps to understand that a container is simply a wrapper around one process that is running on a container host. So it'd be one database service, or one web app server, for example. If you can break up your application into a bunch of these single processes and chain those processes together via networking (DB serves data through the network layer to your cache, which supplies the web app, which is behind the load balancer, etc) then it's a great candidate for containerization. Q: Sounds like a lightweight hypervisor? CT: Containers have been compared to virtual machines as a "lightweight VM"; however, there are distinctions mostly around the fact that the hardware resources are not virtualized for containers, but rather the application is abstracted. KH: That's not a bad way to start thinking about it. However, you don't have a second kernel underneath the hypervisor, so there's no hardware abstraction. Also, in general you don't want to run a full OS stack per container, just what you need for the application. That way your containers are lean and efficient. Q: So is there a practical limit to the number of users you need to have for an app in order for containers/microservices to be preferable vs. traditional apps? CT: Not necessarily. It is more about what you are trying to achieve with the application and the requirements you have around things like: platform agnostic, portability, ease-of-deployment, scalability, etc. But I wouldn't put a hard number on when containers make more sense over virtual machines or bare-metal deployments for that matter. KH: Nope! Microservices is far more about making it easier to build and maintain your applications than it is about scaling. Like anything, you can over-abstract your application design and go extra silly with it, but it's fundamentally about a better way of managing your applications' lifecycle than it is about how many users you can push through the pipe. Q: So is graph and memory the same thing? CT: Keith, I'll let you address this one. KH: Nope. Graph refers to our copy-on-write storage for images at runtime. Our docs can explain it way better than I can in a Q&A session. Look here for more info. Q: Similar to the Docker Container Networking, are there any specific efforts going on around Docker Storage? For example, are you (or will you be) building any products to support features that you mentioned (such as 'Storage vMotion' like capabilities)? CT: Keith, I'll let you address this one. However, there are initiatives and activities on the storage side around providing vMotion like capabilities for the data and application state. KH: It's always a possibility. There's nothing I can say right now, but stay tuned. Q: Let me shift gear, here, where does containerization work with NFV, and how should one correlate to the ask of Telco provider(s)? CT: Keith, I'll let you address this one--should be right up your alley. KH: While this webinar is fundamentally about storage technologies, Docker does have a very broad ecosystem of network partners. NFV is a very broad topic and can't easily be covered in one quick bite, but there are definitely efforts using Docker as both an enablement technology for NFV, as well as integrating Docker's built-in networking capabilities in an NFV scope for application delivery. Q: It would be helpful to circle back at the end and summarize what is Open Source and what is a commercial product, I'm trying to grasp what you miss out on by staying just Open Source. I know that excludes the Universal Control Plane but I don't yet see what UCP delivers, what its USP is. CT: Keith, I'll let you address this one--should be right up your alley. KH: UCP is the only unique commercial component of Docker. It combines a web-based GUI with role-based access control (RBAC) to make it easier to control security and access to Docker components in a production environment. We do maintain a separate codebase for our commercially supported Engine and Registry, but that's mainly done to maintain a more stable release, with critical patches backported from the upstream open source projects. Fundamentally, CS Engine and DTR are the same product as their open source upstreams, only on a slower, more stable release cycle. Click here for an overview, and links to some more detailed information on what's involved in our commercial products: Q: Is there demand for concurrent access, across container hosts, to persistent data? If so, what are those use case scenarios? CT: Yes, actually if you think about a micro-service architecture, you will most likely have many containers accessing a common set of container data volumes simultaneously. This is exactly the reason for persistent storage--if the containers running the application services get migrated or moved to other physical nodes, they need to maintain access to their respective container data volumes in many cases. KH: What a great question! That demand is small, but there. In most cases, persistent data is maintained concurrently through clustering processes (database replication, object storage, etc) but there are some edge cases for large file processing (rendering, big data needs) where there are some asks for that capability. Q: is it possible to run windows applications on Linux container and vice versa? CT: To my knowledge, you should be able to run Linux applications within the recently announced Windows Server 2016 supported containers (see article here), but you can't run Windows applications within Linux containers. KH: No. A container is essentially a process running in a named concurrency group under the kernel. Therefore, you need a Linux kernel to run Linux processes, and likewise for Windows. It will be possible to run Windows and Linux containers under the same management umbrella very soon. We're waiting on some network features to roll out in Windows Server 2016 SP1 for that capability. Q: is it a good idea to run legacy apps in a container? Exactly what is the relation between microservices and containers? Container-like technology used to be popular in various UNIX OSes. What is different now? is the best choice a microservice in a container & spin multiple instantiations fast? CT: Keith, I'll let you address this one--should be right up your alley. KH: Container technology is still popular in several UNIX OSes. Under the hood, a Docker Linux container isn't very different from a Solaris Zone. The difference is primarily the lifecycle tools to build and maintain your containers from both the developer and operations sides. The newer generation of container runtimes is simply much easier to use than older methods. From a Docker perspective: Docker Hub, the ease of use of the 'Docker' CLI command tools, and clustering capabilities in Engine are the main differences. As always, design your architecture to fit your team, user, and application needs. However, if you do want to use a microservices approach, maintaining each part of your application stack as a suite of microservices does make running them widely parallel a strong approach. Q: Is a micro service self-contained with respect to data requirements. Can a service that depends on an external datasource be a micro service? CT: A micro service is by definition self-contained; however, it also would typically connect to one or more container data volumes. Regarding accessing external data sources, not exactly sure what is meant here, but the micro services can be running on one physical server with the container data volumes being created and managed on a separate DAS/SAN/NAS storage platform. KH: Yes, absolutely. An API broker for an external, legacy datastore is a good example of a microservice. Q: How are container images qualified so that they can be trusted for automated pulls? CT: Keith, I'll let you address this one--should be right up your alley. I believe that there is NOT a vetting or certification process done by Docker when posting to either the public Hub or to a trusted registry. This would be the responsibility of the container image developer. KH: In a few ways. First, containers are rarely built from scratch. They are normally built from base images released by trusted providers like Microsoft, Ubuntu, Red Hat, etc. First you should prove trust in that base image through similar methods as you would a VM image. In a Docker Datacenter install, a user with Admin rights can then bring those base images into Docker Trusted Registry (DTR) and then also do a review of internal images built on top of that base before blessing them to go into production. There are also 3rd party security scanning technologies you can use, should that be a concern. Q: For stateless applications, can Docker help apply updates to the application without taking a downtime? For example, a container is running version n of an application and version n+1 needs to be deployed without causing a downtime to users, could one spin a new container with version n+1 of the application and deploy it? CT: If the application is truly stateless, then it shouldn't matter if they are torn down and restarted on another physical server/node to allow the application of a new patch or OS update on the original node. However, this would need to be correctly architected. KH: Yes. Using Docker Engine in Swarm mode, we provide a command 'Docker service update' to do exactly that. Check the docs. Q: Flocker vs. Convoy vs. others - could you talk about these interfaces and their adoption? CT: ClusterHQ (Flocker) has developed a generic storage volume plugin that they then provided back to the Docker community to incorporate the Docker engine. I'm not very familiar with Convoy, but it appears to be a Rancher-developed storage plugin that they have made available as open source, but it is not part of the Docker release. KH: Flocker and Convoy are brokerage-type volume drivers that have the capability to connect with several storage backends. Each has its own API to talk to and manage volumes under the hood. It's also possible to integrate directly with Docker's volume API. If you're mainly interested in integrating purely with Docker Engine, a direct Docker Volume API plugin is the best approach. However, both Flocker and Convoy provide some ease-of-use features and capabilities that might make it attractive to go their routes. Volume API docs are here. Q: Does the link in communication between different containers that run microservices incur the very load we are trying to escape from monolithic approach? CT: Keith, I'll let you address this one--should be right up your alley. KH: That's a very philosophical question! I'd argue that using modern API methods like REST over HTTP is so lightweight that the distributed approach makes more sense. Q: Docker Swarm? CT: Keith, I'll let you address this one. KH: Swarm is our clustering technology to chain together several Docker Engine hosts into one big cluster. Prior to Engine 1.12, it was a standalone product. After 1.12, we added SwarmKit into Engine to make building and maintaining swarms much easier. For more info, check out old Swarm docs and new Swarm docs. Q: Do the applications need to be re-written/revised to take advantage of Container approach? CT: Most legacy or monolithic applications will need to be refactored to best take advantage of a micro-service architecture. KH: Typically, yes. Web applications are already built in a distributed way, so they're the easiest to convert. Q: Do microservices implement Unikernels? CT: Keith, I'll let you also address this. My take: Containers run microservices and leverage the entire OS (Linux kernel and all of its libraries, drivers, etc.). A unikernel is a very small and minimalistic kernel that doesn't contain the additional bloat of the full kernel and therefore, is considered to run that much faster and leaner. Docker acquired a unikernels company and will most likely provide support for running microservices with unikernels and how they may provide a container like wrapper. KH: Not directly. Unikernels are a new method to run arbitrary runtimes under a single kernel. Docker is currently doing some early work with microkernel technology to improve containers, but it's not rolled into core Engine yet. Q: Can a Docker container run on bare metal instead of a host OS directly. If yes, what benefits does this approach provide? CT: A Docker container requires a host OS to run; however, when we refer to "bare-metal" we are referring to a "non-virtualized server". The key benefit this provides is that you don't waste efficiencies by eliminating the hypervisor and guest OS and it is also much more manageable as with the hypervisor and guest OS scenario, you have to manage and maintain all of the VMs that may have different guest OS's and versions. KH: No. A Docker container needs Docker Engine to run, so you'll need to run Engine under a supported OS on the metal. Running Engine on a physical server means your containers will get full "on the iron" IO, since there's no hypervisor abstraction layer between your container and the hardware it's running on. Q: Are packaged software companies like Oracle moving to containerization? CT: Oracle is developing product offerings that are containerized applications. They would be best able to address your question. KH: Here is Oracle's GitHub repository of their official Docker containers and here is their official images in Docker Hub. See? I told you there was no shortage of questions! If you still have one, please comment in this blog below and we’ll get back to you as soon as we can. Follow us on Twitter @sniacloud_com to stay up-to-date on what SNIA Cloud is doing with containers. And don’t forget to register for part two of this webcast, Containers: Best Practices and Data Services, on December 7^th. We hope to see you there!    

Based on record-breaking registration and attendance at our recent SNIA Cloud webcast, Intro to Containers, Container Storage and Docker, It’s clear that containers is a hot topic that folks want to learn more about – especially from a vendor-neutral authority like SNIA. If you missed the live event, it’s now available on-demand together with the webcast slides.

We were bombarded with questions at the live webcast and we ran out of time before we could answer them all, so as promised, here are answers from our expert presenters, Chad Thibodeau and Keith Hudgins. Oh, and please don’t forget to register for part two of this webcast, Containers: Best Practices and Data Management Services, on December 7, 2016.

Q: Would you please highlight key challenges a company may face to move from a hypervisor to container environment?

CT: The main challenge that gets raised in moving from virtual machines to containers is around security as when deployed on bare-metal, all of the containers share the core operating system. However, there are arguments that containers can still be effectively isolated.

KH: Primarily paring down your applications to their minimum running requirements. This can be quite difficult with long-entrenched legacy applications!

Q: With a VM you allocate a finite amount of vCPU and RAM, with a high degree of confidence that those resources will be available to whatever workload is running in the VM. Is that also true of containers – does (or can) the workload get a guaranteed allocation of CPU and memory resources?

CT: Keith, I’ll let you address this one from the application microservice; from the storage side, an SLA or Quality-of-Service can be defined for a container volume if the storage provider offers this capability.

KH: By default, you don’t allocate CPU or ram availability. Most containers are small enough that it’s not a consideration. However, if you need to specify priority, we have a method to do that. Please review the docs here.

Q: Where are microservices most useful? Are there certain environments where they are more likely to be deployed & which verticals or type of solutions/apps will see more benefit?

CT: Microservices can apply to applications within most verticals; for financials it was mentioned that Goldman Sachs is planning on containerizing 90% of their existing applications to web-service such as Netflix. Some of the determining factors are whether the application(s) would benefit from what container technology provides such as rapid deployment, lightweight, portability, and the ability to scale beyond typical monolithic applications.

KH: Microservices are most useful with network-facing applications that don’t require heavy transactional control. Note that it *is* possible to build transactional microservices, but the best practices on that route hasn’t been optimized yet.

Q: What OS version / Hypervisor, support containerization, are working towards cutting the “noisy neighbor” issue?

CT: Containers are supported by both MS Windows and Linux operating systems. The specific version of Linux OS will be more dependent upon the level of capabilities included (Keith, more your area) and MS Windows Server 2016 is the first release of Windows with container (Docker) support.

KH: Docker supports running containers under Windows and Linux kernels. We don’t care whether it’s on metal or virtualized. It’s possible to set affinity groups in a production Docker installation to help manage noisy neighbor issues, but note that fundamentally Docker is NOT a multi-tenant system.

Q: What is “stateful database”? How does it differ from regular databases?

CT: Most databases are stateful such as Oracle, MySQL, Cassandra, MongoDB or Redis. The confusion may be around the Gartner quote which stated “Stateful Database Applications” in which they simply meant that databases are examples of stateful applications.

KH: Any database is by definition stateful. A “stateless” container is one that is running a process that doesn’t store persistent data to disk. This could be a caching system, web application server, load balancer, queue runner, or pretty much any component that doesn’t need to store data. Everything else is “stateful” and needs some way to shove that data into a reliable datastore.

Q: What factors should be considered when choosing between containers and virtual servers for a given project/use case?

CT: The driving factors for container deployments are: portability, minimal footprint (low overhead since no hypervisor or guest OS), rapid provisioning and de-commissioning, scalability and largely open-source based. If any (or all) of these are deemed valuable to you, then you should consider container deployment technology.

KH: That’s a very broad question! It helps to understand that a container is simply a wrapper around one process that is running on a container host. So it’d be one database service, or one web app server, for example. If you can break up your application into a bunch of these single processes and chain those processes together via networking (DB serves data through the network layer to your cache, which supplies the web app, which is behind the load balancer, etc) then it’s a great candidate for containerization.

Q: Sounds like a lightweight hypervisor?

CT: Containers have been compared to virtual machines as a “lightweight VM”; however, there are distinctions mostly around the fact that the hardware resources are not virtualized for containers, but rather the application is abstracted.

KH: That’s not a bad way to start thinking about it. However, you don’t have a second kernel underneath the hypervisor, so there’s no hardware abstraction. Also, in general you don’t want to run a full OS stack per container, just what you need for the application. That way your containers are lean and efficient.

Q: So is there a practical limit to the number of users you need to have for an app in order for containers/microservices to be preferable vs. traditional apps?

CT: Not necessarily. It is more about what you are trying to achieve with the application and the requirements you have around things like: platform agnostic, portability, ease-of-deployment, scalability, etc. But I wouldn’t put a hard number on when containers make more sense over virtual machines or bare-metal deployments for that matter.

KH: Nope! Microservices is far more about making it easier to build and maintain your applications than it is about scaling. Like anything, you can over-abstract your application design and go extra silly with it, but it’s fundamentally about a better way of managing your applications’ lifecycle than it is about how many users you can push through the pipe.

Q: So is graph and memory the same thing?

CT: Keith, I’ll let you address this one.

KH: Nope. Graph refers to our copy-on-write storage for images at runtime. Our docs can explain it way better than I can in a Q&A session. Look here for more info.

Q: Similar to the Docker Container Networking, are there any specific efforts going on around Docker Storage? For example, are you (or will you be) building any products to support features that you mentioned (such as ‘Storage vMotion’ like capabilities)?

CT: Keith, I’ll let you address this one. However, there are initiatives and activities on the storage side around providing vMotion like capabilities for the data and application state.

KH: It’s always a possibility. There’s nothing I can say right now, but stay tuned.

Q: Let me shift gear, here, where does containerization work with NFV, and how should one correlate to the ask of Telco provider(s)?

CT: Keith, I’ll let you address this one–should be right up your alley.

KH: While this webinar is fundamentally about storage technologies, Docker does have a very broad ecosystem of network partners. NFV is a very broad topic and can’t easily be covered in one quick bite, but there are definitely efforts using Docker as both an enablement technology for NFV, as well as integrating Docker’s built-in networking capabilities in an NFV scope for application delivery.

Q: It would be helpful to circle back at the end and summarize what is Open Source and what is a commercial product, I’m trying to grasp what you miss out on by staying just Open Source. I know that excludes the Universal Control Plane but I don’t yet see what UCP delivers, what its USP is.

CT: Keith, I’ll let you address this one–should be right up your alley.

KH: UCP is the only unique commercial component of Docker. It combines a web-based GUI with role-based access control (RBAC) to make it easier to control security and access to Docker components in a production environment. We do maintain a separate codebase for our commercially supported Engine and Registry, but that’s mainly done to maintain a more stable release, with critical patches backported from the upstream open source projects. Fundamentally, CS Engine and DTR are the same product as their open source upstreams, only on a slower, more stable release cycle. Click here for an overview, and links to some more detailed information on what’s involved in our commercial products:

Q: Is there demand for concurrent access, across container hosts, to persistent data? If so, what are those use case scenarios?

CT: Yes, actually if you think about a micro-service architecture, you will most likely have many containers accessing a common set of container data volumes simultaneously. This is exactly the reason for persistent storage–if the containers running the application services get migrated or moved to other physical nodes, they need to maintain access to their respective container data volumes in many cases.

KH: What a great question! That demand is small, but there. In most cases, persistent data is maintained concurrently through clustering processes (database replication, object storage, etc) but there are some edge cases for large file processing (rendering, big data needs) where there are some asks for that capability.

Q: is it possible to run windows applications on Linux container and vice versa?

CT: To my knowledge, you should be able to run Linux applications within the recently announced Windows Server 2016 supported containers (see article here), but you can’t run Windows applications within Linux containers.

KH: No. A container is essentially a process running in a named concurrency group under the kernel. Therefore, you need a Linux kernel to run Linux processes, and likewise for Windows. It will be possible to run Windows and Linux containers under the same management umbrella very soon. We’re waiting on some network features to roll out in Windows Server 2016 SP1 for that capability.

Q: is it a good idea to run legacy apps in a container? Exactly what is the relation between microservices and containers? Container-like technology used to be popular in various UNIX OSes. What is different now? is the best choice a microservice in a container & spin multiple instantiations fast?

CT: Keith, I’ll let you address this one–should be right up your alley.

KH: Container technology is still popular in several UNIX OSes. Under the hood, a Docker Linux container isn’t very different from a Solaris Zone. The difference is primarily the lifecycle tools to build and maintain your containers from both the developer and operations sides. The newer generation of container runtimes is simply much easier to use than older methods. From a Docker perspective: Docker Hub, the ease of use of the ‘Docker’ CLI command tools, and clustering capabilities in Engine are the main differences. As always, design your architecture to fit your team, user, and application needs. However, if you do want to use a microservices approach, maintaining each part of your application stack as a suite of microservices does make running them widely parallel a strong approach.

Q: Is a micro service self-contained with respect to data requirements. Can a service that depends on an external datasource be a micro service?

CT: A micro service is by definition self-contained; however, it also would typically connect to one or more container data volumes. Regarding accessing external data sources, not exactly sure what is meant here, but the micro services can be running on one physical server with the container data volumes being created and managed on a separate DAS/SAN/NAS storage platform.

KH: Yes, absolutely. An API broker for an external, legacy datastore is a good example of a microservice.

Q: How are container images qualified so that they can be trusted for automated pulls?

CT: Keith, I’ll let you address this one–should be right up your alley. I believe that there is NOT a vetting or certification process done by Docker when posting to either the public Hub or to a trusted registry. This would be the responsibility of the container image developer.

KH: In a few ways. First, containers are rarely built from scratch. They are normally built from base images released by trusted providers like Microsoft, Ubuntu, Red Hat, etc. First you should prove trust in that base image through similar methods as you would a VM image. In a Docker Datacenter install, a user with Admin rights can then bring those base images into Docker Trusted Registry (DTR) and then also do a review of internal images built on top of that base before blessing them to go into production. There are also 3rd party security scanning technologies you can use, should that be a concern.

Q: For stateless applications, can Docker help apply updates to the application without taking a downtime? For example, a container is running version n of an application and version n+1 needs to be deployed without causing a downtime to users, could one spin a new container with version n+1 of the application and deploy it?

CT: If the application is truly stateless, then it shouldn’t matter if they are torn down and restarted on another physical server/node to allow the application of a new patch or OS update on the original node. However, this would need to be correctly architected.

KH: Yes. Using Docker Engine in Swarm mode, we provide a command ‘Docker service update’ to do exactly that. Check the docs.

Q: Flocker vs. Convoy vs. others – could you talk about these interfaces and their adoption?

CT: ClusterHQ (Flocker) has developed a generic storage volume plugin that they then provided back to the Docker community to incorporate the Docker engine. I’m not very familiar with Convoy, but it appears to be a Rancher-developed storage plugin that they have made available as open source, but it is not part of the Docker release.

KH: Flocker and Convoy are brokerage-type volume drivers that have the capability to connect with several storage backends. Each has its own API to talk to and manage volumes under the hood. It’s also possible to integrate directly with Docker’s volume API. If you’re mainly interested in integrating purely with Docker Engine, a direct Docker Volume API plugin is the best approach. However, both Flocker and Convoy provide some ease-of-use features and capabilities that might make it attractive to go their routes. Volume API docs are here.

Q: Does the link in communication between different containers that run microservices incur the very load we are trying to escape from monolithic approach?

CT: Keith, I’ll let you address this one–should be right up your alley.

KH: That’s a very philosophical question! I’d argue that using modern API methods like REST over HTTP is so lightweight that the distributed approach makes more sense.

Q: Docker Swarm?

CT: Keith, I’ll let you address this one.

KH: Swarm is our clustering technology to chain together several Docker Engine hosts into one big cluster. Prior to Engine 1.12, it was a standalone product. After 1.12, we added SwarmKit into Engine to make building and maintaining swarms much easier. For more info, check out old Swarm docs and new Swarm docs.

Q: Do the applications need to be re-written/revised to take advantage of Container approach?

CT: Most legacy or monolithic applications will need to be refactored to best take advantage of a micro-service architecture.

KH: Typically, yes. Web applications are already built in a distributed way, so they’re the easiest to convert.

Q: Do microservices implement Unikernels?

CT: Keith, I’ll let you also address this. My take: Containers run microservices and leverage the entire OS (Linux kernel and all of its libraries, drivers, etc.). A unikernel is a very small and minimalistic kernel that doesn’t contain the additional bloat of the full kernel and therefore, is considered to run that much faster and leaner. Docker acquired a unikernels company and will most likely provide support for running microservices with unikernels and how they may provide a container like wrapper.

KH: Not directly. Unikernels are a new method to run arbitrary runtimes under a single kernel. Docker is currently doing some early work with microkernel technology to improve containers, but it’s not rolled into core Engine yet.

Q: Can a Docker container run on bare metal instead of a host OS directly. If yes, what benefits does this approach provide?

CT: A Docker container requires a host OS to run; however, when we refer to “bare-metal” we are referring to a “non-virtualized server”. The key benefit this provides is that you don’t waste efficiencies by eliminating the hypervisor and guest OS and it is also much more manageable as with the hypervisor and guest OS scenario, you have to manage and maintain all of the VMs that may have different guest OS’s and versions.

KH: No. A Docker container needs Docker Engine to run, so you’ll need to run Engine under a supported OS on the metal. Running Engine on a physical server means your containers will get full “on the iron” IO, since there’s no hypervisor abstraction layer between your container and the hardware it’s running on.

Q: Are packaged software companies like Oracle moving to containerization?

CT: Oracle is developing product offerings that are containerized applications. They would be best able to address your question.

KH: Here is Oracle’s GitHub repository of their official Docker containers and here is their official images in Docker Hub.

See? I told you there was no shortage of questions! If you still have one, please comment in this blog below and we’ll get back to you as soon as we can. Follow us on Twitter @sniacloud_com to stay up-to-date on what SNIA Cloud is doing with containers. And don’t forget to register for part two of this webcast, Containers: Best Practices and Data Services, on December 7^th. We hope to see you there!

 

 

Today's storage world would appear to have been divided into three major and mutually exclusive categories: block, file and object storage. The marketing that shapes much of the user demand would appear to suggest that these are three quite distinct animals, and many systems are sold as exclusively either SAN for block, NAS for file or object. And object is often conflated with cloud, a consumption model that can in reality be block, file or object. A fixed taxonomy that divides the storage world this way is very limiting, and can be confusing; for instance, when we talk about cloud. How should providers and users buy and consume their storage? Are there other classifications that might help in providing storage solutions to meet specific or more general application needs? What about customers who need file access performance beyond what one storage box can provide? Which options support those who want scale-out solution like object storage with file protocol semantics? To clear up the confusion, the SNIA Ethernet Storage Forum is hosting a live Webcast, "Clustered File Systems: No Limits." In this Webcast we will explore clustered storage solutions that not only provide multiple end users access to shared storage over a network, but allow the storage itself to be distributed and managed over multiple discrete storage systems. You'll hear:

• General principles and specific clustered and distributed systems and the facilities they provide built on the underlying storage
• Better known file systems like NFS, IBM Spectrum Scale (GPFS) and Lustre, along with a few of the less well known
• How object based systems like S3 have blurred the lines between them and traditional file based solutions

This Webcast should appeal to those interested in exploring some of the different ways of accessing & managing storage, and how that might affect how storage systems are provisioned and consumed. POSIX and other acronyms may be mentioned, but no rocket science beyond a general understanding of the principles of storage will be assumed. Contains no nuts and is suitable for vegans! As always, our experts will be on hand to answer your questions on the spot. Register now for this October 25^th event. Update: If you missed the live event, it's now available  on-demand. You can also  download the webcast slides.

Today’s storage world would appear to have been divided into three major and mutually exclusive categories: block, file and object storage. The marketing that shapes much of the user demand would appear to suggest that these are three quite distinct animals, and many systems are sold as exclusively either SAN for block, NAS for file or object. And object is often conflated with cloud, a consumption model that can in reality be block, file or object.

A fixed taxonomy that divides the storage world this way is very limiting, and can be confusing; for instance, when we talk about cloud. How should providers and users buy and consume their storage? Are there other classifications that might help in providing storage solutions to meet specific or more general application needs? What about customers who need file access performance beyond what one storage box can provide? Which options support those who want scale-out solution like object storage with file protocol semantics?

To clear up the confusion, the SNIA Ethernet Storage Forum is hosting a live Webcast, “Clustered File Systems: No Limits.” In this Webcast we will explore clustered storage solutions that not only provide multiple end users access to shared storage over a network, but allow the storage itself to be distributed and managed over multiple discrete storage systems. You’ll hear:

• General principles and specific clustered and distributed systems and the facilities they provide built on the underlying storage
• Better known file systems like NFS, IBM Spectrum Scale (GPFS) and Lustre, along with a few of the less well known
• How object based systems like S3 have blurred the lines between them and traditional file based solutions

This Webcast should appeal to those interested in exploring some of the different ways of accessing & managing storage, and how that might affect how storage systems are provisioned and consumed. POSIX and other acronyms may be mentioned, but no rocket science beyond a general understanding of the principles of storage will be assumed. Contains no nuts and is suitable for vegans!

As always, our experts will be on hand to answer your questions on the spot. Register now for this October 25^th event.

 

A new SNIA specification offers a unified approach to managing storage and servers in environments like hyperscale and cloud infrastructures. SNIA on Storage recently sat down with SNIA member volunteers from the Scalable Storage Management Technical Work Group (SSM TWG), who just announced the completion of Version 1.0 of the SNIA Swordfish^TM storage management specification, to learn more.

SNIA on Storage (SOS): What prompted SNIA to begin work in this area?

SSM TWG: We have been looking at what IT administrators are doing in today’s data centers, and what they will need to do in the future with storage equipment and storage services. We wanted to simplify the way storage is allocated, monitored, and managed, so the SNIA SSM TWG was formed and members collaborated to develop SNIA Swordfish, a specification that extends the DMTF’s (Distributed Management Task Force) Redfish™. Redfish is an open industry standard specification and schema that specifies a RESTful interface and utilizes JavaScript Object Notation and Open Data Protocol to help customers integrate solutions within their existing tool chains.

SOS: What does SNIA Swordfish do?

SSM TWG: Our SNIA Swordfish specification is based on Redfish’s easy-to-use RESTful interface, and provides a scalable storage management application programming interface (API) that can handle all necessary storage management functionality, including file and block storage provisioning, volume mapping and masking, replication, and capacity and health reporting tools. SNIA Swordfish can assist in handling tasks such as locating storage with a suitable class of service and allocating it to a server or a virtual machine.

SOS: How will SNIA Swordfish fit into data center management?

SSM TWG: SNIA Swordfish was designed to support management use cases that focus on what IT administrators need to do with storage equipment and storage services in a data center. Because it extends the DMTF Redfish API into storage management, it helps provide a unified approach for managing servers, storage, and network fabrics. This unified approach will make it easier to create management software tools that can deal with operations involving multiple management domains. SNIA Swordfish is also designed to support vendor specific functionality alongside standardized functionality, to allow for future needs.

SOS: Which companies are working on SNIA Swordfish in the TWG?

SSM TWG: Our TWG members represent many of the leading companies in the storage industry today, including Broadcom, Dell EMC, HPE, Intel, Microsoft, NetApp, Nimble Storage, and VMware.

SOS: Where can IT administrators and data center managers learn more about SNIA Swordfish?

SNIA Swordfish will be a highlight of the SNIA Storage Developer conference September 19-22 in Santa Clara, CA, with overview and deep dive presentations. These presentations will be made publicly available on the SDC website at www.storagedeveloper.org. SNIA Swordfish has its own webpage at www.snia.org/swordfish. Those who would like to submit feedback on the specification can do so either via the SSM TWG (SNIA members) or the SNIA Feedback Portal (all).