{"id":398,"date":"2020-09-17T10:49:39","date_gmt":"2020-09-17T14:49:39","guid":{"rendered":"https:\/\/www.macloo.com\/ai\/?p=398"},"modified":"2020-09-26T09:52:17","modified_gmt":"2020-09-26T13:52:17","slug":"using-mario-to-understand-neural-networks","status":"publish","type":"post","link":"https:\/\/www.macloo.com\/ai\/2020\/09\/17\/using-mario-to-understand-neural-networks\/","title":{"rendered":"Using Super Mario to understand neural networks"},"content":{"rendered":"\n<p>I doubt I will ever program a neural network, but I&#8217;m trying to understand how they work \u2014 and how they are trained \u2014 <em>well enough<\/em> to make assumptions about how the systems work. What I want to be able to do is raise questions when I hear about a new-to-me AI system. I don&#8217;t want to take it on faith that a system is safe and likely to function well.<\/p>\n\n\n\n<p>Ultimately I want to help my journalism and communications students understand this too.<\/p>\n\n\n\n<p>Last week I discussed here a video about <a href=\"https:\/\/www.macloo.com\/ai\/2020\/09\/10\/what-is-a-neural-network-and-how-does-it-work\/\">how neural networks work<\/a>. Some time before I found that video, I had watched this one a couple of times. It&#8217;s from 2015 and it&#8217;s only 6 minutes long. It&#8217;s been viewed on YouTube more than 9 million times. In fact, it&#8217;s pretty close to <strong>1 billion views<\/strong>!<\/p>\n\n\n\n<figure class=\"wp-block-embed-youtube aligncenter wp-block-embed is-type-video is-provider-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<div class=\"jetpack-video-wrapper\"><iframe loading=\"lazy\" title=\"MarI\/O - Machine Learning for Video Games\" width=\"739\" height=\"416\" src=\"https:\/\/www.youtube.com\/embed\/qv6UVOQ0F44?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe><\/div>\n<\/div><\/figure>\n\n\n\n<p>Video game designer <a rel=\"noreferrer noopener\" href=\"https:\/\/twitter.com\/sethbling\" target=\"_blank\">Seth Bling<\/a> demonstrates a fully trained neural network that plays Mario expertly. Then he shows us how the system looks at the start, when the Mario character just stands in one place and dies every time. This is the untrained neural network, when it &#8220;knows&#8221; nothing.<\/p>\n\n\n\n<p>Unlike the example in my earlier post \u2014 where the <strong>input<\/strong> to the neural network was an image of a handwritten number, and the <strong>output<\/strong> was the number (thereby &#8220;reading&#8221; the image) \u2014 here the <strong>input<\/strong> is the game state, which changes by the split second. The <em>game state<\/em> is a simplified digital representation of the Mario character, the surfaces he can run on or jump to, and any obstacles or rewards that are present. The <strong>output<\/strong> is which button should be pressed \u2014 holding down <em>right<\/em> continuously makes Mario run toward the right without stopping.<\/p>\n\n\n\n<p>So the <strong>output layer<\/strong> in this neural network is the set of all possible actions Mario can take. For a human playing the game, these would be the buttons on the game controller.<\/p>\n\n\n\n<p>In the training, Mario has a &#8220;fitness level,&#8221; which is a number. When Mario is dying all the time, that number stays around 2. When Mario reaches the end of the level without dying (but without scoring extra points), his fitness is 528. So by &#8220;looking at&#8221; the fitness level, the neural net assesses success. If the number has increased, then keep doing the same thing.<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\"><p>&#8220;The more lines and neurons you have, the more nuanced the decisions can be.&#8221;<\/p><cite>\u2014Seth Bling<\/cite><\/blockquote>\n\n\n\n<p>Of course there are more actions than only moving right. Training the neural net to make Mario jump and perform more actions required many generations of neural nets, and only the best-performing ones were selected for the next generation. After 34 generations, the fitness level reached 4,000.<\/p>\n\n\n\n<p>One thing I especially like about this video is the simultaneous visual of real Mario running in the real game level, along with a representation of the neural net showing its pathways in green and red. There is no code and no math in this video, and so while watching it, you are only thinking about how the connections come to be made and <em>reinforced<\/em>.<\/p>\n\n\n\n<p>The method used is called <a rel=\"noreferrer noopener\" href=\"https:\/\/en.wikipedia.org\/wiki\/Neuroevolution_of_augmenting_topologies\" target=\"_blank\">NeuroEvolution of Augmenting Topologies<\/a> (NEAT), which I&#8217;ve read almost nothing about \u2014 but apparently it enables the neural net <em>to grow itself,<\/em> essentially. Which is kind of mind blowing.<\/p>\n\n\n\n<p>Bling <a href=\"https:\/\/pastebin.com\/ZZmSNaHX\" target=\"_blank\" rel=\"noreferrer noopener\">shared his code here<\/a>; it&#8217;s written in the <a href=\"https:\/\/en.wikipedia.org\/wiki\/Lua_(programming_language)\" target=\"_blank\" rel=\"noreferrer noopener\">Lua<\/a> language.<\/p>\n\n\n\n<p><a rel=\"license\" href=\"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/\"><img decoding=\"async\" alt=\"Creative Commons License\" style=\"border-width:0\" src=\"https:\/\/i.creativecommons.org\/l\/by-nc-nd\/4.0\/88x31.png\"><\/a><br>\n<small><span xmlns:dct=\"http:\/\/purl.org\/dc\/terms\/\" property=\"dct:title\"><strong>AI in Media and Society<\/strong><\/span> by <span xmlns:cc=\"http:\/\/creativecommons.org\/ns#\" property=\"cc:attributionName\">Mindy McAdams<\/span> is licensed under a <a rel=\"license\" href=\"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/\">Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License<\/a>.<br>\nInclude the author&#8217;s name (Mindy McAdams) and a link to the original post in any reuse of this content.<\/small><\/p>\n\n\n\n<p>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I doubt I will ever program a neural network, but I&#8217;m trying to understand how they work \u2014 and how they are trained \u2014 well enough to make assumptions about how the systems work. What I want to be able to do is raise questions when I hear about a new-to-me AI system. I don&#8217;t&hellip; <a class=\"more-link\" href=\"https:\/\/www.macloo.com\/ai\/2020\/09\/17\/using-mario-to-understand-neural-networks\/\">Continue reading <span class=\"screen-reader-text\">Using Super Mario to understand neural networks<\/span> <span class=\"meta-nav\" aria-hidden=\"true\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[7,5],"tags":[75,76,15,96],"class_list":["post-398","post","type-post","status-publish","format-standard","hentry","category-games","category-machine-learning","tag-mario","tag-neat","tag-neural_network","tag-reinforcement_learning"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/posts\/398","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/comments?post=398"}],"version-history":[{"count":9,"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/posts\/398\/revisions"}],"predecessor-version":[{"id":407,"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/posts\/398\/revisions\/407"}],"wp:attachment":[{"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/media?parent=398"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/categories?post=398"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/tags?post=398"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}