The COVID-19 Pandemic has amplified cybersecurity concerns, particularly related to the cloud. Threat actors have recognized a unique opportunity to exploit pandemic-related vulnerabilities through social engineering attacks, business email compromise, work from home, and other remote weak points. This results in increased risk and occurrence of ransomware attacks and data breaches that can disrupt or totally compromise organizations’ ability to conduct business. Partnering with a cybersecurity company Melbourne can help businesses strengthen their defenses against these evolving threats. These security incidents can also subject victims to liability for violations of privacy and data breach notification laws. Then there’s the issue of facing allegations that could impact your career and personal life. Internet-related offences are increasingly common, and many individuals are caught up in investigations without realising the full implications. For anyone dealing with online fraud, hacking accusations, or other cyber-related charges, having an experienced internet crimes attorney who understands digital laws and defence strategies can be the key to securing a favourable outcome. The right legal support can help you navigate these complex cases effectively. The SNIA Cloud Storage Technologies Initiative (CSTI) will be taking on this important topic with a live webcast on January 20, 2021, “Data Privacy and Data Protection in the COVID Era” where our SNIA experts will discuss:

• The changing threat landscape due to COVID-19
• Common security failures and their consequences
• Recent attacker exploits
• Data protection concerns:
  • Strategies to combat malware
  • Minimizing ransomware risks
• How emerging technologies (5G, IoT, AI, etc.) expand the threat landscape

Register today to learn key considerations to mitigate the cybersecurity risks resulting from the COVID pandemic.

The COVID-19 Pandemic has amplified cybersecurity concerns particularly related to the cloud. Threat actors have recognized a unique opportunity to exploit pandemic-related vulnerabilities through social engineering attacks, business email compromise, work from home and other remote weak points. This results in increased risk and occurrence of ransomware attacks and data breaches that can disrupt or totally compromise organizations’ ability to conduct business. These security incidents can also subject victims to liability for violations of privacy and data breach notification laws. The SNIA Cloud Storage Technologies Initiative (CSTI) will be taking on this important topic with a live webcast on January 20, 2021, “Data Privacy and Data Protection in the COVID Era” where our SNIA experts will discuss:

• The changing threat landscape due to COVID-19
• Common security failures and their consequences
• Recent attacker exploits
• Data protection concerns:
  • Strategies to combat malware
  • Minimizing ransomware risks
• How emerging technologies (5G, IoT, AI, etc.) expand the threat landscape

Register today to learn key considerations to mitigate the cybersecurity risks resulting from the COVID pandemic.

The SNIA Networking Storage Forum (NSF) recently took on the topics surrounding data reduction with a 3-part webcast series that covered Data Reduction Basics, Data Compression and Data Deduplication. If you missed any of them, they are all available on-demand.

In Not Again! Data Deduplication for Storage Systems” our SNIA experts discussed how to reduce the number of copies of data that get stored, mirrored, or backed up. Attendees asked some interesting questions during the live event and here are answers to them all.

Q. Why do we use the term rehydration for deduplication?  I believe the use of the term rehydration when associated with deduplication is misleading. Rehydration is the activity of bringing something back to its original content/size as in compression. With deduplication the action is more aligned with a scatter/gather I/O profile and this does not require rehydration.

A. "Rehydration" is used to cover the reversal of both compression and deduplication. It is used more often to cover the reversal of compression, though there isn't a popularly-used term to specifically cover the reversal of deduplication (such as "re-duplication").  When reading compressed data, if the application can perform the decompression then the storage system does not need to decompress the data, but if the compression was transparent to the application then the storage (or backup) system will decompress the data prior to letting the application read it. You are correct that deduplicated files usually remain in a deduplicated state on the storage when read, but the storage (or backup) system recreates the data for the user or application by presenting the correct blocks or files in the correct order.

Q. What is the impact of doing variable vs fixed block on primary storage Inline?

A. Deduplication is a resource intensive process. The process of sifting the data inline by anchoring, fingerprinting and then filtering for duplicates not only requires high computational resources, but also adds latency on writes. For primary storage systems that require high performance and low latencies, it is best to keep these impacts of dedupe low. Doing dedupe with variable-sized blocks or extents (for e.g. with Rabin fingerprinting) is more intensive than using simple fixed-sized blocks. However, variable-sized segmentation is likely to give higher storage efficiency in many cases. Most often this tradeoff between latency/performance vs. storage efficiency tips in favor of applying simpler fixed-size dedupe in primary storage systems.

Q. Are there special considerations for cloud storage services like OneDrive?

A. As far as we know, Microsoft OneDrive avoids uploading duplicate files that have the same filename, but does not scan file contents to deduplicate identical files that have different names or different extensions. As with many remote/cloud backup or replication services, local deduplication space savings do not automatically carry over to the remote site unless the entire volume/disk/drive is replicated to the remote site at the block level. Please contact Microsoft or your cloud storage provider for more details about any space savings technology they might use.

Q. Do we have an error rate calculation system to decide which type of deduplication we use?

A. The choice of deduplication technology to use largely depends on the characteristics of the dataset and the environment in which deduplication is done. For example, if the customer is running a performance and latency sensitive system for primary storage purposes, then the cost of deduplication in terms of the resources and latencies incurred may be too high and the system may use very simple fixed-size block based dedupe. However, if the system/environment allows for spending extra resources for the sake of storage efficiency, then a more complicated variable-sized extent based dedupe may be used. About error rates themselves, a dedupe storage system should always be built with strong cryptographic hash-based fingerprinting so that the error rates of collisions are extremely low. Errors due to collisions in a dedupe system may lead to data loss or corruption, but as mentioned earlier these can be avoided by using strong cryptographic functions.

Q. Considering the current SSD QLC limitations and endurance... Can we say that if a right choice for deduped storage?

A. In-line deduplication either has no effect or reduces the wear on NAND storage because less data is written. Post-process deduplication usually increases wear on NAND storage because blocks are written then later erased--due to deduplication--and the space later fills with new data. If the system uses post-process deduplication, then the storage software or storage administrator needs to weigh the space savings benefits vs. the increased wear on NAND flash. Since QLC NAND is usually less expensive and has lower write endurance than SLC/MLC/TLC NAND, one might be less likely to use post-process deduplication on QLC NAND than on more expensive NAND which has higher endurance levels.

Q. On slides 11/12 - why not add compaction as well - "fitting" the data onto respective blocks and "if 1k file, not leaving the rest 3k of 4k block empty"?

A. We covered compaction in our webcast on data reduction basics “Everything You Wanted to Know About Storage But Were Too Proud to Ask: Data Reduction.” See slide #18 below.

Again, I encourage you to check out this Data Reduction series and follow us on Twitter @SNIANSF for dates and topics of more SNIA NSF webcasts.

The SNIA Networking Storage Forum (NSF) recently took on the topics surrounding data reduction with a 3-part webcast series that covered Data Reduction Basics, Data Compression and Data Deduplication. If you missed any of them, they are all available on-demand. In Not Again! Data Deduplication for Storage Systems” our SNIA experts discussed how to reduce the number of copies of data that get stored, mirrored, or backed up. Attendees asked some interesting questions during the live event and here are answers to them all. Q. Why do we use the term rehydration for deduplication?  I believe the use of the term rehydration when associated with deduplication is misleading. Rehydration is the activity of bringing something back to its original content/size as in compression. With deduplication the action is more aligned with a scatter/gather I/O profile and this does not require rehydration. A. “Rehydration” is used to cover the reversal of both compression and deduplication. It is used more often to cover the reversal of compression, though there isn’t a popularly-used term to specifically cover the reversal of deduplication (such as “re-duplication”).  When reading compressed data, if the application can perform the decompression then the storage system does not need to decompress the data, but if the compression was transparent to the application then the storage (or backup) system will decompress the data prior to letting the application read it. You are correct that deduplicated files usually remain in a deduplicated state on the storage when read, but the storage (or backup) system recreates the data for the user or application by presenting the correct blocks or files in the correct order. Q. What is the impact of doing variable vs fixed block on primary storage Inline? A. Deduplication is a resource intensive process. The process of sifting the data inline by anchoring, fingerprinting and then filtering for duplicates not only requires high computational resources, but also adds latency on writes. For primary storage systems that require high performance and low latencies, it is best to keep these impacts of dedupe low. Doing dedupe with variable-sized blocks or extents (for e.g. with Rabin fingerprinting) is more intensive than using simple fixed-sized blocks. However, variable-sized segmentation is likely to give higher storage efficiency in many cases. Most often this tradeoff between latency/performance vs. storage efficiency tips in favor of applying simpler fixed-size dedupe in primary storage systems. Q. Are there special considerations for cloud storage services like OneDrive? A. As far as we know, Microsoft OneDrive avoids uploading duplicate files that have the same filename, but does not scan file contents to deduplicate identical files that have different names or different extensions. As with many remote/cloud backup or replication services, local deduplication space savings do not automatically carry over to the remote site unless the entire volume/disk/drive is replicated to the remote site at the block level. Please contact Microsoft or your cloud storage provider for more details about any space savings technology they might use. Q. Do we have an error rate calculation system to decide which type of deduplication we use? A. The choice of deduplication technology to use largely depends on the characteristics of the dataset and the environment in which deduplication is done. For example, if the customer is running a performance and latency sensitive system for primary storage purposes, then the cost of deduplication in terms of the resources and latencies incurred may be too high and the system may use very simple fixed-size block based dedupe. However, if the system/environment allows for spending extra resources for the sake of storage efficiency, then a more complicated variable-sized extent based dedupe may be used. About error rates themselves, a dedupe storage system should always be built with strong cryptographic hash-based fingerprinting so that the error rates of collisions are extremely low. Errors due to collisions in a dedupe system may lead to data loss or corruption, but as mentioned earlier these can be avoided by using strong cryptographic functions. Q. Considering the current SSD QLC limitations and endurance… Can we say that if a right choice for deduped storage? A. In-line deduplication either has no effect or reduces the wear on NAND storage because less data is written. Post-process deduplication usually increases wear on NAND storage because blocks are written then later erased–due to deduplication–and the space later fills with new data. If the system uses post-process deduplication, then the storage software or storage administrator needs to weigh the space savings benefits vs. the increased wear on NAND flash. Since QLC NAND is usually less expensive and has lower write endurance than SLC/MLC/TLC NAND, one might be less likely to use post-process deduplication on QLC NAND than on more expensive NAND which has higher endurance levels. Q. On slides 11/12 – why not add compaction as well – “fitting” the data onto respective blocks and “if 1k file, not leaving the rest 3k of 4k block empty”? A. We covered compaction in our webcast on data reduction basics “Everything You Wanted to Know About Storage But Were Too Proud to Ask: Data Reduction.” See slide #18 below.

Again, I encourage you to check out this Data Reduction series and follow us on Twitter @SNIANSF for dates and topics of more SNIA NSF webcasts.

As a year we will never forget drew to a close, SNIA on Storage sat down (virtually of course!) with Computational Storage Technical Work Group Co-Chairs Jason Molgaard of Arm and Scott Shadley of NGD Systems and Computational Storage Special Interest Group Chair Eli Tiomkin of NGD Systems to take the pulse of 2020 and anticipate 2021 computational storage advances.

SNIA On Storage (SOS): Jason, Scott, and Eli, thanks for taking the time to chat. Where was computational storage 12 months ago and how did it progress in 2020?

Scott Shadley (SS): The industry launched the computational storage effort in late 2018, so 2019 was a year of beginning education and understanding of the technology concepts to encourage the “ask” for computational storage.  All new technology takes time to develop, so 2020 saw the beginning of implementation and growth with customer solutions beginning to be publicized and multiple vendors beginning to promote.

Jason Molgaard (JM): I agree. In 2019 the question was, “What is computational storage?” and the belief that it might never happen.  By early 2020, we saw much more interest and understanding of what computational storage was and how it could play a role in product development and deployment. 

Eli Tiomkin (ET): SNIA established the Computational Storage Special Interest Group in early 2020 as a great way to start to spread the word and make people aware of how compute could meet storage. As the year progressed, more players joined the market with devices that offered viable solutions and SNIA gained more members interested in contributing to the growth of this technology. 

SS:  We really saw the launch of the computational storage market in 2020 with multiple solutions of merit and also third-party industry analysts and experts writing on the technology.  The Computational Storage Technical Work Group, launched in 2019, brought 45+ companies together to begin to craft a standard for computational storage architectures and a programming model. In 2020 that effort branched out to other standards groups like NVM Express to propel standards even further.

JM:  Now, nearing the end of 2020, everyone has some vested interest in computational storage.

SOS:  Who are some of the “everyones” who have a vested interest?

JM:  First interest is from the developers– who are looking at “What should I make” and “How does it work”. They are seeing the interest from the knowledge gained by customers and prospects.  Users acquire devices and ask, ‘How will I use this?” and “Where will it give me benefits in my data center?”. They are interested in how they can use computational storage implementations in their industry for their purposes.

SS:  Computational storage at the end of 2020 is no longer simply a concept discussed at the CTO level as a forward-looking implementation but is now getting into the business units and those doing the real work.  That is the key – moving from R&D to the market by way of the business unit.

SOS:  Is this because users are understanding the hows and whys of compute moving closer to storage?

SS:  SNIA has done a huge amount of work this year to make computational storage visible and the connection between compute and storage understandable with outbound publicity around the technology and the weight it carries.  SNIA drove folks to pay attention, and the industry has responded making sure computational storage is on customer roadmaps.

ET:  SNIA’s 2020 activity to make computational storage noticeable has gotten results.  Our 2021 goal in the SIG is to take everything we did in 2020 and multiply it two to three times to draw even more attention to computational storage’s benefits for cloud, edge storage, and networking.  We want to make users always consider computational storage to solve problems and make outcomes more efficient.  We will be increasing the SIG’s identification and education on computational storage real world deployments in 2021 with videos, demonstrations, and developer bootcamps.

SOS: Thinking good things for the future, where do you see computational storage in five years?

SS: I see computational storage where persistent memory is today or even more advanced, with more opportunities and more deployments.  By 2025, 10% of all solid state drives could be computational based.

JM:  I agree with the 10%, and it could even be more looking at the kinds of industries that will have more widespread adoption. There will be higher adoption in end point applications as it is an easy way to gain a lot of compute into existing storage needs.  Data centers will also be clear winners but there some players may be more reluctant to adopt computational storage.

SS; I see an emerging growth market for data storage at the edge where the problem is to move data from the edge to some core location – cloud, on premise, etc.  The ability to put computational storage at the end point – the edge - gives SNIA the perfect opportunity to engage the industry and educate where the technology will get its success as compared to the core data center.

ET:  I double that and see for the edge as it evolves and grows, computational storage will be a natural selection for storage and compute at the edge. I would even say further if the data center hyperscaler would have started today from a technology point of view, we would have seen computational storage deployed in most data center infrastructures.  But getting into the existing infrastructure and pushing compute and storage as it is today is difficult so we might now be playing with some existing swim lanes.  However, as the edge evolves, it will have a natural tendency to go with NVMe SSDs with computational storage as a perfect fit for edge applications.

SOS:  Any further thoughts?

SS:  We at SNIA are very bullish on computational storage but have to be cautiously optimistic. We are not saying this has to happen, but rather that we at SNIA in the Technical Work Group and Special Interest Group can make it happen by how well we work as an organization in the industry with real customers who will deploy computational storage to drive success in the market.  SNIA is well versed with the capability to understand this new architecture and help others to understand that it is not scary but safe.  SNIA can provide that support to drive the technology.

SOS:  I have always been impressed by the cross-vendor cooperation and collaboration of SNIA members in putting technology forward and advancing standards and education in the marketplace.

SS: It is a great effort so let’s have some fun and make 2021 the year of computational storage deployments!  If you are interested, come join us!

There is a new wave of cognitive services based on video and image analytics, leveraging the latest in machine learning and deep learning. In a recent SNIA Cloud Storage Technologies Initiative (CSTI) webcast “How Video Analytics is Changing the Way We Store Video,” we looked at some of the benefits and factors driving this adoption, as well as explored compelling projects and required components for a successful video-based cognitive service. This included some great work being done in the open source community. In the course of the presentation, there were several discussion points and questions that arose. Our SNIA presenters, Glyn Bowden from HPE and Kevin Cone from Intel provide the answers. Q: The material several times mentioned an open source project for video analytics.  Is that available for everyone to view and contribute? A. Absolutely. The Open Visual Cloud Product (OVCP) is located on GitHub at: https://github.com/OpenVisualCloud. Contributions are welcome, and there are a significant number of contributors already involved in the project. There were several examples on the versatility of OVCP, and it was noted how extensible the project could be with addition of new tools and models. Q. Glyn talked about some old video platforms. Did people really capture video like that? Did the dinosaurs roam the tape archives back in those days? A. Ha! Glyn would like everyone to know that while that was the way video used to be captured, he was not around during the time of the dinosaurs. Q. Is there a reason to digitize and store old video of such poor quality? A. Glyn demonstrated how much of this video can still be valuable, but he also discussed how it was difficult to capture and index. Clearly, there are significant storage implications in digitizing too much old video, though cloud storage certainly provides a variety of solutions. Q. There was a good example of video analytics in smart cities. Is there a role for computational storage in this type of application? A. Not only is there a role for computational storage, there’s a significant need for smart networking. Kevin and Glyn provided some cases where the network might do local analytics. In fact, there was a recent SNIA webcast on “Compute Everywhere: How Storage and Networking Expand the Compute Continuum” that discussed some aspects of the edge and cloud interaction. Q. There was a good discussion on governance of video data. One discussion point was around the use of video in public safety and law enforcement. Would it be the case that smart city video might also be useful as a legal tool, and would have different retention rules as a result? Are there other examples of something like this? A. There are a variety of rules on archiving and retention of data that may be used in public safety. This is a pretty fluid area. Another example would be of videos where children are present, as there are significant privacy issues. The EU leads in the legislative efforts in this area, and they have a number of rules & guidelines that are outlined here. Q. Digital camera pickups have the ability to see beyond the human visual spectrum. Are there uses for video analytics in the IR and UV spaces? A. Kevin mentioned the use of IR as an indicator of remote temperature monitoring. Glyn said that this might also be an example of legal hazards, where there could be a violation of health protections. So, governance is likely to play a role in this area. Q. What are some differences between analytics and storage of video at the edge and storage in the cloud or data center? A. Video storage at the edge is likely a temporary thing. It could be stored there or analyzed there to reduce the latency of decision-making. Or it’s possible that it would be analyzed to determine how much of the video should be permanently archived. Cloud storage is more permanent, and analytics in the cloud is more likely to generate metadata that would be used to make policy at the edge.

Data in transit provides a large attack surface for bad actors. Keeping data secure from threats and compromise while it’s being transmitted was the topic at our live SNIA Networking Storage Forum (NSF) webcast, Securing Data in Transit. Our presenters, Claudio DeSanti, Ariel Kit, Cesar Obediente, and Brandon Hoff did an excellent job explaining how to mitigate risks. We had several questions during the live event. Our panel of speakers have been kind enough to answer them here. Q. Could we control the most important point – identity, that is, the permission of every data transportation must have an identity label, so that we can control anomalies and misbehaviors easily? A. That is the purpose of every authentication protocol: verify the identity of entities participating in the authentication protocol on the basis of some secret values or certificates associated with the involved entity. This is similar to verifying the identity of a person on the basis of an identity document associated with the person. Q. What is BGP? A. BGP stands for Border Gateway Protocol, it is a popular routing protocol commonly used across the Internet but also leveraged by many customers in their environments. BGP is used to exchange routing information and next hop reachability between network devices (routers, switches, firewall, etc.). In order to establish this communication among the neighbors, BGP creates a TCP session in port 179 to maintain and exchange BGP updates. Q. What are ‘north-south’ and ‘east west’ channels? A. Traditionally “north-south” is traffic up and down the application or solution “stack” such as from client to/from server, Internet to/from applications, application to/from database, application to/from storage, etc. East-West is between similar nodes–often peers in a distributed application or distributed storage cluster. For example, east-west could include traffic from client to client, between distributed database server nodes, between clustered storage nodes, between hyperconverged infrastructure nodes, etc. Q. If I use encryption for data in transit, do I still need a separate encryption solution for data at rest? A. The encryption of data in transit protects the data as it flows through the network and blocks attack types such as eavesdropping, however, once it arrives to the target the data is decrypted and saved to the storage unencrypted unless data at rest encryption is applied. It is highly recommended to use both for best protection, data at rest protection protects the data in case the storage target is accessed by an attacker. The SNIA NSF did a deep dive on this topic in a separate webcast “Storage Networking Security Series: Protecting Data at Rest.” Q. Will NVMe-oFÔ use 3 different encryption solutions depending upon whether it’s running over Fibre Channel, RDMA, or IP? A. When referring to data in transit, the encryption type depends on the network type, hence, for different networks we will use different data-in-motion encryption protocols, nevertheless, they can all be based on Encapsulating Security Protocol (ESP) with same cipher suite and key exchange methods. Q. Can NVMe-oF over IP already use Transport Layer Security (TLS) for encryption or is this still a work in progress? Is the NVMe-oF spec aware of TLS? A. NVMe-oF over TCP already supports TLS 1.2. The NVM Express Technical Proposal TP 8011 is adding support for TLS 1.3. Q. Are there cases where I would want to use both MACsec and IPSec, or use both IPSec and TLS?  Does CloudSec rely on either MACSec or IPSec? A. Because of the number of cyber-attacks that are currently happening on a daily basis, it is always critical to create a secure environment in order to protect confidentially and integrity of the data. MACsec is enabled in a point-to-point Ethernet link and IPSec could be classified as to be end-to-end (application-to-application or router-to-router). Essentially you could (and should) leverage both technologies to provide the best encryption possible to the application. These technologies can co-exist with each other without any problem. The same can be said if the application is leveraging TLS. To add an extra layer of security you can implement IPSec, for example site-to-site to IPSec VPN. This is true especially if the communication is leveraging the Internet. CloudSec, on the other hand, doesn’t rely on MACsec because MACsec is a point-to-point Ethernet Link technology and CloudSec provides the transport and encryption mechanism to support a multi-site encryption communication. This is useful where more than one data center is required to provide an encryption mechanism to protect the confidentially and integrity of the data. The CloudSec session is a point-to-point encryption over Data Center Interconnect on two or more sites. CloudSec key exchange uses BGP to guarantee the correct information gets the delivered to the participating devices. Q. Does FC-SP-2 require support from both HBAs and switches, or only from the HBAs? A. For data that moves outside the data center, Fibre Channel Security Protocols (FC-SP-2) for Fibre Channel or IPsec for IP would need to be supported by the switches or routers. No support would be required in the HBA. This is most common use case for FC-SP-2.  Theoretically, if you wanted to support FC-SP-2 inside the secure walls of the data center, you can deploy end-to-end or HBA-to-HBA encryption and you won’t need support in the switches.  Unfortunately, this breaks some switch features since information the switch relies on would be hidden. You could also do link encryption from the HBA-to-the switch, and this would require HBA and switch support.  Unfortunately, there are no commercially available HBAs with FC-SP-2 support today, and if they become available, interoperability will need to be proven. This webcast from the Fibre Channel Industry Association (FCIA) goes into more detail on Fibre Channel security. Q. Does FC-SP-2 key management require a centralized key management server or is that optional? A. For switch-to-switch encryption, keys can be managed through a centralized server or manually. Other solutions are available and in production today. For HBAs, in most environments there would be thousands of keys to manage so a centralized key management solution would be required and FC-SP provides 5 different options. Today, there are no supported key management solutions for FC-SP-2 from SUSE, RedHat, VMware, Windows, etc. and there are no commercially available HBAs that support FC-SP-2. This webcast was part of our Storage Networking Security Webcast Series and they are all available on demand. I encourage you to take a look at the other SNIA educational webcasts from this series:

• Understanding Storage Security and Threats
• Protecting Data at Rest
• Encryption 101
• Key Management 101
• Security & Privacy Regulations
• Applied Cryptography

---

Recently, the SNIA Compute, Memory, and Storage Initiative (CMSI) hosted a wide-ranging discussion on the “compute everywhere” continuum.  The panel featured Chipalo Street from Microsoft, Steve Adams from Intel, and Eli Tiomkin from NGD Systems representing both the start-up environment and the SNIA Computational Storage Special Interest Group. We appreciate the many questions asked during the webcast and are pleased to answer them in this Q&A blog. Our speakers discussed how, in modern analytics deployments, latency is the fatal flaw that limits the efficacy of the overall system.  Solutions move at the speed of decision, and microseconds could mean the difference between success and failure against competitive offerings.  Artificial Intelligence, Machine Learning, and In-Memory Analytics solutions have significantly reduced latency, but the sheer volume of data and its potential broad distribution across the globe prevents a single analytics node from efficiently harvesting and processing data. Viewers asked questions on these subjects and more. Let us know if you have any additional questions by emailing cmsi@snia.org. And, if you have not had a chance to view the entire webcast, you can access it in the SNIA Educational Library. Q1: The overlay of policy is the key to enabling roles across distributed nodes that make “compute everywhere” an effective strategy, correct? A1: Yes, and there are different kinds of applications.  Examples include content distribution or automation systems, and all of these can benefit from being able to run anywhere in the network.  This will require significant advancements in security and trust as well. Q2: Comment: There are app silos and dependencies that make it difficult to move away from a centralized IT design.  There’s an aspect of write-once, run-everywhere that needs to be addressed. A2: This comes to the often-asked question on the differences between centralized and distributed computing.  It really comes down to the ability to run common code anywhere, which allows digital transformation.  By driving both centralized and edge products, the concept of compute everywhere can really come to life. Q3: Comment: There are app silos and app dependencies, for instance three tier apps, that make it difficult to move away from centralized consolidated IT design. What are the implications of this? A3: Data silos within a single tenant, and data silos that cross tenants need to be broken down.  The ability to share data in a secure fashion allows a global look to get results. Many companies view data like oil, it’s their value.  There needs to be an ability to grant and then revoke access to data. The opportunity for companies is to get insight from their own data first, but then to share and access other shared data to develop additional insight.  We had a lively discussion on how companies could take advantage of this. Emerging technologies to automate the process of anonymizing or de-identifying data should facilitate more sharing of data. Q4: Comment: The application may run on the edge, but the database is on-prem.  But that’s changing, and the ability to run the data analytics anywhere is the significant change.  Compute resources are available across the spectrum in the network and storage systems.  There is still need for centralized compute resources, but the decisions will eventually be distributed.  This is true not only inside a single company, but across the corporate boundary. A4: You have the programming paradigm to write-one, run-everywhere. You can also expose products and data.  The concept of data gravity might apply to regulatory as well as just size considerations. Q5: There’s the concept of geo-fencing from a storage perspective, but is that also from a routing perspective? A5: There are actually requirements such as GDPR in Europe that define how certain data can be routed.  What’s interesting is that the same kind of technology that allows network infrastructure to route data can also be used to help inform how data should flow.  This is not just to avoid obstacles, but also to route data where it will eventually need to be collected in order to facilitate machine learning and queries against streaming data, especially where streaming data aggregates. Q6: Eli Tiomkin introduced the concept of computational storage.  The comment was made that moving compute to the storage node enables the ability to take an analytics model and distribute that across the entire network. A6: As data becomes vast, the ability to gain insight without forcing continuous data movement will enable new types of application and deployments to occur. Q7: When do you make the decision to keep the data on-prem and bring the analytics to the data store rather than take the data to the service itself?  Or what are the keys to making the decision to keep the data on your premise instead of moving it to a centralized database? When would you want to do one vs. the other? A7: The reason the data should be processed on the edge is because it’s easier to compare the results to new data as it’s aggregated at the source.  There are latency implications of moving the data to the cloud to make all the decisions, and it also avoids excess data movements. In addition to data gravity considerations there might be regulation barriers.  Additionally, some of the decisions that customers are expecting to make might have to scale to a metro area.  An example would be using retail data to influence digital signage.  We provided several other examples in the discussion. Q8: “Routing” traditionally means that data needs to be moved from one point to the next as fast as possible.  But perhaps intelligent routing can be used to make more deliberate decisions in when and where to move and secure data.  What are the implications of this? A8: What it really represents is that data has different value at different times, and also at different locations.  Being able to distribute data is not just an act of networking, but also an act of balancing the processing required to gain the most insight.  There’s a real need for orchestration to be available to all nodes in the deployment to best effect. Q9: It seems like the simple answer is to compute at the edge and store in the cloud. Is this true? A9: It really depends on what you want to store and where you need to store it. You might find your insight immediately, or you might have to store that data for a while due to audit considerations, or because the sought-after insight is a trend line from streaming sources. So likely, a cache of data is needed at the edge.  It depends on the type of application and the importance of the data. When you’re improving your training models, the complexity of the model will dictate where you can economically process the data.  So the simple answer might not always apply. An example would be where there is a huge cache of data at the edge but archive/data lake in the cloud. For instance, consider the customer support arm of a cellular network with a dashboard indicating outages, congestion, and trending faults in order to address a customer who is complaining of poor service. The need to quickly determine whether the problem is their phone, a basestation, or the network itself drives the need to have compute and store distributed everywhere. Large cellular networks produce 100+ Terabytes of data a day in telemetry, logging, and event data. Both maintaining the dashboard and the larger analytics tasks for predictive maintenance requires a distributed approach. Q10: How can you move cloud services like AI/ML to on-prem, when on-prem might have a large database.  Many of the applications depend on the database and it might be difficult to move the application to the edge when the data is on-prem. A10: The real question is where you run your compute.  You need a large dataset to train an AI model, and you’ll need a large processing center to do that.  But once you have the model, you can run the data through the model anywhere, and you might get different insight based on the timeliness of the decision needed. That might not mean that you can throw away the data at that point.  There’s a need to continue to augment the data store and make new decisions based on the new data. Q11: So how would the architecture change as a result? A11: Compute everywhere implies that the old client-server model is expanding to suggest that compute capability needs to be coordinated between compute/store/move capabilities in the end device, on-premises infrastructure, local IT, metro or network edge compute resources, zones of compute, and in the cloud. Compute everywhere means client to client and server to server, peers of servers and tiers of servers. Cloud gaming is an early example of compute everywhere. Gaming PCs & Gaming Console inter-acting in peer-to-peer fashion while simultaneously interacting with edge and cloud gaming servers each inter-acting within its tiers and peers. AI is becoming a distributed function like gaming driving demand for compute everywhere and just like gaming, some AI functions are best done in or close to the end device and others nearby, and still other further away in highly centralized locations. Q12: Outside of a business partnership or relationship, what are other cases where users would generally agree to share data? A12: As we’ve seen trends change due to the current pandemic, there are many cities and municipalities that would like to keep some of the benefits of reduced travel and traffic.  There’s an opportunity to share data on building automation, traffic control, coordination of office and work schedules, and many other areas that might benefit from shared data.  There are many other examples that might also apply.  Public sources of data from public agencies, in some geographies, are or will be mandated to share their collected data. We should anticipate that some government statistic data will be available by subscription, just like a news feed. Q13: Efficient interactions among datacenters and nodes might be important for the decisions we need to make for future compute and storage.  How could real-time interactions affect latency? A13: The ability to move the compute to the data could significantly reduce the latency of decision-making.  We should see more real-time and near-real-time decisions will simultaneously be made through a network of edge clusters. Distributed problems, like dynamically managing traffic systems across a large metro area will leverage distributed compute and store edge clusters to adjust metered on-ramps, stop lights, traffic signage in near real-time. Imagine what kinds of apps and services will emerge if insights can be shared near instantaneously between edge compute clusters. Put succinctly, some distributed problems, especially those exposed in streaming data from people and things, will require distributed processing operating in a coordinate way in order to resolve. Q14: Who’s dog barked at the end of the talk? A14: That would be Jim’s dog valiantly defending the household from encroaching squirrels. Q15: Will there be more discussions on this topic? A15: Well, if you’d like to hear more, let us at SNIA know and we’ll find more great discussion topics on compute everywhere.

Internet of Payments (IoP) enables payment processing over many kinds of IoT devices and has also led to the emergence of the micro-transaction. It’s an area of rapid growth. Recently, the SNIA Cloud Storage Technologies Initiative (CSTI) hosted a live webcast Technology Implications of Internet of Payments. The talk was hosted by me, with Glyn Bowden from HPE and Richard George from HLPS providing expert insight. In the course of the conversation, several comments and questions arose, and they are summarized here. Feel free to view the entire discussion and provide us with feedback on the talk. We are also always interested on topics you’d like to see us cover in the future. Q. When considering digitization of assets, currency is not locked to a solid standard. That is, they are not based on specific physical assets. Many new digital currencies are therefore unstable. But the proposition here is that they would be more secure because they’re locked to real physical value. Is that correct? A. There is significant volatility in digital assets, mostly because of speculation. Being able to, say, fractionalize your home into digital assets stabilizes the specific currency and creates value that is locked to the growing value of the asset itself. The physical asset can be locked to a fiat currency. For those interested in trading on a reliable crypto exchange platform, a Coinbase Review can provide valuable insights. If you're planning to have an investment for your child's future, here is the guide to junior ISAs to help you. Q. Comment: The fact that the currency is digital means that it can be shared on a currency exchange.  The example used was that assets in the game Fortnight can be bought and sold on eBay. Those who love playing Contra may use these contra hard corps cheats to play more game levels. A. Yes, exactly, this is a new way to create a wealth of exchanges. The assets themselves are readily exchanged, and this gets us back to a more traditional bartering of desired goods and services that have been extant for centuries. Q. Is this the real opportunity to move back to a barter and exchange system? You can value your own assets and bargain them for other assets? A. Absolutely. This is a way for people and organizations to generate value that they see for their assets. There is an opportunity to make liquid approximately $250T of assets. Q. The reach of these assets is astounding. Will this change global micro-lending based on the real assets that global start-up businesses own or create? A. Yes. This is a peer-to-peer investment and exchange, and it can be opened up to hundreds of thousands of assets and individuals. These peer-to-peer transactions will impact both individuals and governments, and create significant efficiencies in the trading of value. To learn more about the wonders of trading, articles like the Kiana Danial course have you covered. Q. How does trust impact currency? In the digital environment, a loss of trust would essentially make the specific currency valueless rapidly. A. Yes, there’s significant need for technology to establish a common trust model, and for all parties in the transaction to commit to it. Richard and Glyn provided a great example in the presentation, so make sure you watch it to see. Glyn also provided a high-level architecture that could authenticate the transaction. Q. The credentialing process and the creation of storage repositories is a way to create the trust in the currency. The example of a third-world farmer lets the farmer create a true chain of trust that can be used by large global entities to establish value. So, can the final customer be assured that the asset came from not only a trusted source but also from an ethical one? A. Yes, and this will create significant value to the originating entity as well as others in the chain of value. The technology chain adds transparency to the transaction which opens it up to public scrutiny. Q. How does this affect cloud storage vendors that participate as part of the transaction infrastructure? A. Vendors who process the transactions and manage the currency exchange can gain insight from both the data and the data flow of transactions. Q. What is HLPS? A. Health Life Prosperity Shared Ltd. is a financial technology company focused on using digital assets to help people in the UK purchase homes.  Richard is an expert in digital assets and payments.

Internet of Payments (IoP) enables payment processing over many kinds of IoT devices and has also led to the emergence of the micro-transaction. It’s an area of rapid growth. Recently, the SNIA Cloud Storage Technologies Initiative (CSTI) hosted a live webcast Technology Implications of Internet of Payments. The talk was hosted by me, with Glyn Bowden from HPE and Richard George from HLPS providing expert insight. In the course of the conversation, several comments and questions arose, and they are summarized here. Feel free to view the entire discussion and provide us with feedback on the talk. We are also always interested on topics you’d like to see us cover in the future. Q. When considering digitization of assets, currency is not locked to a solid standard. That is, they are not based on specific physical assets. Many new digital currencies are therefore unstable. But the proposition here is that they would be more secure because they’re locked to real physical value. Is that correct? A. There is significant volatility in digital assets, mostly because of speculation. Being able to, say, fractionalize your home into digital assets stabilizes the specific currency and creates value that is locked to the growing value of the asset itself. The physical asset can be locked to a fiat currency. Q. Comment: The fact that the currency is digital means that it can be shared on a currency exchange.  The example used was that assets in the game Fortnight can be bought and sold on eBay. A. Yes, exactly, this is a new way to create a wealth of exchanges. The assets themselves are readily exchanged, and this gets us back to a more traditional bartering of desired goods and services that have been extant for centuries. Q. Is this the real opportunity to move back to a barter and exchange system? You can value your own assets and bargain them for other assets? A. Absolutely. This is a way for people and organizations to generate value that they see for their assets. There is an opportunity to make liquid approximately $250T of assets. Q. The reach of these assets is astounding. Will this change global micro-lending based on the real assets that global start-up businesses own or create? A. Yes. This is a peer-to-peer investment and exchange, and it can be opened up to hundreds of thousands of assets and individuals. These peer-to-peer transactions will impact both individuals and governments, and create significant efficiencies in the trading of value. Q. How does trust impact currency? In the digital environment, a loss of trust would essentially make the specific currency valueless rapidly. A. Yes, there’s significant need for technology to establish a common trust model, and for all parties in the transaction to commit to it. Richard and Glyn provided a great example in the presentation, so make sure you watch it to see. Glyn also provided a high-level architecture that could authenticate the transaction. Q. The credentialing process and the creation of storage repositories is a way to create the trust in the currency. The example of a third-world farmer lets the farmer create a true chain of trust that can be used by large global entities to establish value. So, can the final customer be assured that the asset came from not only a trusted source but also from an ethical one? A. Yes, and this will create significant value to the originating entity as well as others in the chain of value. The technology chain adds transparency to the transaction which opens it up to public scrutiny. Q. How does this affect cloud storage vendors that participate as part of the transaction infrastructure? A. Vendors who process the transactions and manage the currency exchange can gain insight from both the data and the data flow of transactions. Q. What is HLPS? A. Health Life Prosperity Shared Ltd. is a financial technology company focused on using digital assets to help people in the UK purchase homes.  Richard is an expert in digital assets and payments.