{"id":301,"date":"2021-08-31T15:40:49","date_gmt":"2021-08-31T15:40:49","guid":{"rendered":"https:\/\/connect-community.org\/\/2021-8-31-applying-swarm-learning-to-challenges-created-by-data-at-the-edge\/"},"modified":"2021-10-14T08:51:18","modified_gmt":"2021-10-14T13:51:18","slug":"2021-8-31-applying-swarm-learning-to-challenges-created-by-data-at-the-edge","status":"publish","type":"post","link":"https:\/\/connect-community.org\/2021-8-31-applying-swarm-learning-to-challenges-created-by-data-at-the-edge\/","title":{"rendered":"Applying swarm learning to challenges created by data at the edge"},"content":{"rendered":"<h3 style=\"white-space: pre-wrap;\"><strong><em>Moving to an edge-to-cloud technology world brings a host of new data-driven challenges. Understand how swarm learning presents a solution to an increasingly complex environment.<\/em><\/strong><\/h3>\n<div class=\" image-block-outer-wrapper layout-caption-below design-layout-inline combination-animation-none individual-animation-none individual-text-animation-none \" data-test=\"image-block-inline-outer-wrapper\">\n<figure class=\" sqs-block-image-figure intrinsic \" style=\"max-width: 400px; overflow: hidden;\">\n<div class=\"image-block-wrapper\" data-animation-role=\"image\">\n<div class=\"sqs-image-shape-container-element has-aspect-ratio \" style=\"position: relative; padding-bottom: 66.75%; overflow: hidden;\"><noscript><img decoding=\"async\" src=\"https:\/\/connect-community.org\/\/wp-content\/uploads\/2021\/10\/Swarm-Learning.png\" alt=\"Swarm-Learning.png\" \/><\/noscript><img decoding=\"async\" class=\"thumb-image lazyload\" data-src=\"https:\/\/connect-community.org\/\/wp-content\/uploads\/2021\/10\/Swarm-Learning.png\" alt=\"Swarm-Learning.png\" data-image=\"https:\/\/connect-community.org\/\/wp-content\/uploads\/2021\/10\/Swarm-Learning.png\" data-image-dimensions=\"400x267\" data-image-focal-point=\"0.5,0.5\" data-load=\"false\" data-image-id=\"612e4cd198897348344e9873\" data-type=\"image\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" \/><\/div>\n<\/div>\n<\/figure>\n<\/div>\n<p class=\"\" style=\"white-space: pre-wrap;\">Many of us can recall the film\u00a0<em>A Beautiful Mind<\/em>\u00a0based on the life of the mathematician Dr. John Nash. For some it was a difficult movie to watch but we all saw how Nash developed his models in support of what became the Nash Equilibrium that equally applied to economics as it did to game theory. Previously, earlier mathematicians had described win\/loss outcomes whereas Nash added both win\/win and loss\/loss outcomes. These ideas underpin much of what has followed in the development of game theory.<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">\u00a0The Nash Equilibrium as applied to game theory has also played a part in swarm learning. Have you seen starlings swirling in harmonious patterns with no apparent leader or seen video clips of similar actions taken by schools of fish? It is this apparent leaderless performance that now captivates many in IT and for good reason. Perhaps more impressive and closer to home, how will this technology be showcased?<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">In August 2017 CBS&#8217;s\u00a0<em>60 Minutes<\/em>\u00a0featured a swarm of autonomous drones witnessed over California\u2019s desert. \u201cNow a new generation of drones is coming.\u00a0Only this time they are autonomous\u2014able to operate on their own without humans controlling them from somewhere with a joystick,\u201d CBS reported. \u201cThink of it as the coming swarm, and if that sounds like the title of a sci-fi mini-series, well, as we first reported earlier this year, it&#8217;s already a military reality. We saw it with our own eyes and captured it on camera.\u201d<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\"><strong>Conundrum at the edge<\/strong><\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">As we move to a technology world that is edge-to-cloud driven, it soon becomes apparent that the industry will experience difficulties. Expressed as simply as can be said, deploy enough edge intelligence to support an enterprise and you soon run into barriers created as a result of capturing massive amounts of data even as bandwidth and the inherent fall out from lengthening latency create an unimaginable environment.<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">There has been a culture among mathematicians to model just about everything. Remember the scene in\u00a0<em>A Beautiful Mind<\/em>\u00a0where the camera captures Dr. Nash following a flock of pigeons? When a student asked what he was doing he explained how he thought he could extract an equation capable of explaining their movements. This is now referred to as flocking behavior\u2014within a group, individual members stay close enough so as not to be separated while giving themselves enough distance to move. Pigeons, as do starlings and fish, swarm intuitively sharing nothing other than an insight into what they individually need to do.<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\"><strong>Swarm learning presents a solution<\/strong><\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">For HPE engineers there has been a growing interest in swarm learning and it\u2019s hard to ignore two factors influencing this latent interest in swarms. Among nations there is a growing awareness that their sovereignty can be compromised even as there are industries where privacy is of uttermost importance. This\u00a0has led to both protect data storage as well as restrict data transmission. In a cloud-to-edge model, there is no counting how many locations edge deployments may include nor is there any real way to determine where data ultimately ends up residing. With the potential for the edge to produce yottabytes of data, as predicted by some industry analysts who keeping upping this number with each analysis provided (IDC: 33 zettabytes in 2018, 175 ZB in 2025), suitable communications infrastructure neither exists nor was already in place, to keep up with the potential demand. This has been the subject of two articles published to the HPE web site.<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">In the article,\u00a0<a href=\"https:\/\/www.hpe.com\/us\/en\/insights\/articles\/how-swarm-learning-enables-data-sharing-while-protecting-data-so-2106.html\" target=\"_blank\" rel=\"noopener\"><span style=\"text-decoration: underline;\">How swarm learning provides data insights while protecting data sovereignty<\/span><\/a>, moving data where sovereign rights are the issue, the most important aspect of swarm learning is addressed: \u201cThe beauty of swarm learning is that there is no central node which aggregates the data,&#8221; says\u00a0Dr. Eng Lim Goh, senior vice president and chief technology officer for artificial intelligence at Hewlett Packard Enterprise.\u00a0&#8220;The swarm network acts as a union by sharing insights directly with all participants of the respective learning. There is no central custodian collecting all learnings or insights.&#8221;<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">Article author Christopher Null expands on this observation: &#8220;Doing business across national borders, and even across U.S. state lines, means that data sovereignty rules, which subject data to the regulations of the region in which that data is collected, must be followed with an even greater level of care. Swarm learning offers a way to end-run data sovereignty complexities by ensuring data never has to leave the location in which it is collected.&#8221;<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">In the second article, the topic of swarm learning as it applies to edge deployments introduces us to\u00a0<a href=\"https:\/\/www.hpe.com\/us\/en\/insights\/articles\/swarm-learning-and-the-artificially-intelligent-edge-1908.html\" target=\"_blank\" rel=\"noopener\"><span style=\"text-decoration: underline;\">Swarm learning and the artificially intelligent edge:<\/span><\/a><\/p>\n<blockquote>\n<p class=\"\" style=\"white-space: pre-wrap;\">&#8220;Today, we create artificial intelligence models by sending data collected at the edge to a central point. The data is used to \u201ctrain\u201d the model, and then the model is pushed back out to all devices at the edge.\u00a0 Yet, it\u2019s often impractical to send all of the data collected at the edge to a central server for computing. Latency, heat and energy demands, compliance issues, and mounting transportation and opportunity costs are to blame.<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">However, swarm learning has additional functionality that obviates the need for a central leader: The AI modeling is done completely by the devices at the edge. Because\u00a0the AI training in swarm learning is done at the edge, using the compute available on the clients, the back and forth to a central control is removed. With swarm learning, the compute-intensive training in machine learning can be done\u00a0in situ\u00a0by yoking edge devices together.&#8221;<\/p>\n<\/blockquote>\n<p class=\"\" style=\"white-space: pre-wrap;\">Within the HPE NonStop community there is a culture not too dissimilar to that existing among mathematicians. Perhaps not to the same degree, but in terms of overlapping of ideas and testing potential opportunities and solutions it has been demonstrated for many years that the NonStop community has been instrumental in many of the most recent projects pursued by NonStop development.<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">Virtualization, support of the Intel x86 architecture and the imminent support of HPE GreenLake have all been fueled by a culture of cooperation and oftentimes, little more than imaginative debates. This has been the way with the NonStop community and it has been how those who participate share ideas. Swarm behavior may not be the target of discussions today but then again, where insights, models and data intersect, perhaps there are\u00a0opportunities when it comes to providing a reliable, robust and indeed fault-tolerant server for edge intelligence to register for initial models.<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">This will be a topic covered in the upcoming\u00a0<a href=\"https:\/\/nonstoptbc.com\/\" target=\"_blank\" rel=\"noopener\"><span style=\"text-decoration: underline;\">NonStop Technical Boot Camp (TBC)<\/span><\/a>\u00a0October 5-7, 2021 in Denver. HPE Master Technologist Justin Simonds will be providing the NonStop community with further insights into swarm learning and it will surely attract an audience. He says:\u00a0&#8220;The ability to share models without having to share actual information will be a real game changer. Swarm learning will allow companies to benefit from each other without sharing customer or internal information. No risk sharing! Swarm learning is parallel processing for AI.&#8221;<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">If it is just these soundbites that interest you then you are not alone. Already there is a developing sense that for some industries\u2014healthcare and fraud detection to name a few\u2014swarm learning is about to have its impact felt. In portraying Dr. Nash, actor Russell Crowe gave one of his best performances. This had a lot to do with the intriguing way a complex topic was addressed so simplistically. It wasn\u2019t just about observing pigeons as even the modern dating scene was addressed at one point.<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">HPE is pursuing swarm learning with a sense of urgency and this is perhaps best conveyed when Null quotes Patrick Edlund, HPE Head of Communication for Germany, Austria, and Switzerland:<\/p>\n<blockquote>\n<p class=\"\" style=\"white-space: pre-wrap;\"><em>&#8220;<\/em>The world is going to be decentralized,&#8221; says Edlund, &#8220;so the majority of data will be created at the edge. That&#8217;s the thing we have to solve in the future, because in order to get the value out of that data, you must share it somehow.&#8221;<\/p>\n<\/blockquote>\n<p class=\"\" style=\"white-space: pre-wrap;\">Register today for\u00a0<a href=\"https:\/\/nonstoptbc.com\/\" target=\"_blank\" rel=\"noopener\"><span style=\"text-decoration: underline;\">NonStop Technical Boot Camp (TBC)<\/span><\/a>\u00a0October 5-7, 2021 in Denver, CO.<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">Please visit us for more information on the\u00a0<a href=\"https:\/\/www.hpe.com\/us\/en\/servers\/nonstop.html\" target=\"_blank\" rel=\"noopener\"><span style=\"text-decoration: underline;\">HPE NonStop family of systems<\/span><\/a>.<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">\n<hr \/>\n<p class=\"\" style=\"white-space: pre-wrap;\"><strong><em>Meet Server Experts blogger Karen Copeland, Manager, Worldwide HPE NonStop Product Management<\/em><\/strong><\/p>\n<div class=\" image-block-outer-wrapper layout-caption-below design-layout-inline combination-animation-none individual-animation-none individual-text-animation-none \" data-test=\"image-block-inline-outer-wrapper\">\n<figure class=\"sqs-block-image-figure\" style=\"overflow: hidden;\">\n<div class=\"image-block-wrapper\" data-animation-role=\"image\">\n<div class=\"sqs-image-shape-container-element \" style=\"position: relative; overflow: hidden;\"><noscript><img decoding=\"async\" src=\"https:\/\/connect-community.org\/\/wp-content\/uploads\/2021\/10\/KarenCopeland-HPE.jpg\" alt=\"Karen Copeland\" \/><\/noscript><img decoding=\"async\" class=\"thumb-image lazyload\" data-src=\"https:\/\/connect-community.org\/\/wp-content\/uploads\/2021\/10\/KarenCopeland-HPE.jpg\" alt=\"Karen Copeland\" data-image=\"https:\/\/connect-community.org\/\/wp-content\/uploads\/2021\/10\/KarenCopeland-HPE.jpg\" data-image-dimensions=\"\" data-image-focal-point=\"0.5,0.5\" data-load=\"false\" data-image-id=\"612e4d5a4f630239a255953f\" data-type=\"image\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" \/><\/div>\n<\/div><figcaption class=\"image-caption-wrapper\">\n<div class=\"image-caption\">\n<p class=\"\" style=\"white-space: pre-wrap;\">Karen Copeland<\/p>\n<\/div>\n<\/figcaption><\/figure>\n<\/div>\n<p class=\"\" style=\"white-space: pre-wrap;\"><em>Karen has worked in various roles at different management levels within the HPE NonStop business, with leadership positions for Software Engineering IT functions, configuration management functions, release functions, and process improvement. She has led application development teams creating business applications running on NonStop systems that still support the NonStop business today.<\/em><\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\">\n<strong>Compute Experts<\/strong><br \/>\nHewlett Packard Enterprise<\/p>\n<p class=\"\" style=\"white-space: pre-wrap;\"><a href=\"https:\/\/twitter.com\/hpe_compute\" target=\"_blank\" rel=\"noopener\"><span style=\"text-decoration: underline;\">twitter.com\/hpe_compute<\/span><\/a><br \/>\n<a href=\"https:\/\/www.linkedin.com\/showcase\/hpe-servers-and-systems\/\" target=\"_blank\" rel=\"noopener\"><span style=\"text-decoration: underline;\">linkedin.com\/showcase\/hpe-servers-and-systems\/<\/span><\/a><br \/>\n<a href=\"https:\/\/www.hpe.com\/servers\" target=\"_blank\" rel=\"noopener\"><span style=\"text-decoration: underline;\">hpe.com\/servers<\/span><\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Moving to an edge-to-cloud technology world brings a host of new data-driven challenges. Understand how swarm learning presents a solution to an increasingly complex environment. Many of us can recall&hellip;<\/p>\n","protected":false},"author":1,"featured_media":52663,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_seopress_robots_primary_cat":"none","_seopress_titles_title":"","_seopress_titles_desc":"","_seopress_robots_index":"","_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0,"content-type":"","footnotes":""},"categories":[410],"tags":[],"coauthors":[],"class_list":["post-301","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news","category-410","description-off"],"_links":{"self":[{"href":"https:\/\/connect-community.org\/wp-json\/wp\/v2\/posts\/301","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/connect-community.org\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/connect-community.org\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/connect-community.org\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/connect-community.org\/wp-json\/wp\/v2\/comments?post=301"}],"version-history":[{"count":0,"href":"https:\/\/connect-community.org\/wp-json\/wp\/v2\/posts\/301\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/connect-community.org\/wp-json\/wp\/v2\/media\/52663"}],"wp:attachment":[{"href":"https:\/\/connect-community.org\/wp-json\/wp\/v2\/media?parent=301"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/connect-community.org\/wp-json\/wp\/v2\/categories?post=301"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/connect-community.org\/wp-json\/wp\/v2\/tags?post=301"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/connect-community.org\/wp-json\/wp\/v2\/coauthors?post=301"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}