{"id":446,"date":"2020-09-24T13:28:19","date_gmt":"2020-09-24T17:28:19","guid":{"rendered":"https:\/\/www.macloo.com\/ai\/?p=446"},"modified":"2020-09-24T13:28:19","modified_gmt":"2020-09-24T17:28:19","slug":"imagining-words-as-numbers-in-n-dimensional-space","status":"publish","type":"post","link":"https:\/\/www.macloo.com\/ai\/2020\/09\/24\/imagining-words-as-numbers-in-n-dimensional-space\/","title":{"rendered":"Imagining words as numbers in n-dimensional space"},"content":{"rendered":"\n

The vocabulary<\/strong> of a neural network is represented as vectors<\/strong> \u2014 which I wrote about yesterday<\/a>. This enables many related words to be “close to” one another, which is how the network perceives similarity and difference. This is as near as a computer comes to understanding meaning<\/em> \u2014 which is not very near at all, but good enough<\/em> for a lot of practical applications of natural language processing.<\/p>\n\n\n\n

A previous way of representing vocabulary for a neural network was to assign just one number to each word. If the neural net had a vocabulary of 20,000 words, that meant it had 20,000 separate inputs in the first layer \u2014 the input layer. (I discussed neural nets in an earlier post here<\/a>.) For each word, only one input was activated. This is called “one-hot encoding.”<\/p>\n\n\n\n

Representing words as vectors (instead of with a single number) means that each number in the array for one word is an input for the neural net. Among the many possible inputs, several<\/em> or many<\/em> are “hot,” not just one.<\/p>\n\n\n\n

\"\"
Photo by Timothy Newman<\/a> on Unsplash<\/a><\/em><\/figcaption><\/figure>\n\n\n\n

As I was sorting this in my mind today, reading and thinking, I had to think about how to convey to my students (who might have no computer science background at all) this idea of words. The word itself doesn’t exist. The word is represented<\/em> in the system as a list of numbers. The numbers have meaning; they locate the the word-object in a mathematical space, for which computers are ideally suited. But there is no word.<\/p>\n\n\n\n

Long ago in school I learned about the signifier<\/em> and the signified<\/em>. Together, they create a sign<\/em>. Language is our way of representing the world in speech and in writing. The word is not the thing itself; the map is not the territory. And here we are, building a representation of human language in code, where a vocabulary of tens of thousands of human words exists in an imaginary space consisting of numbers \u2014 because numbers are the only things a computer can use.<\/p>\n\n\n\n

I had a much easier time understanding the concepts of image recognition than I am having with NLP.<\/p>\n\n\n\n

\"Creative<\/a>
\nAI in Media and Society<\/strong><\/span> by Mindy McAdams<\/span> is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License<\/a>.
\nInclude the author’s name (Mindy McAdams) and a link to the original post in any reuse of this content.<\/small><\/p>\n\n\n\n

.<\/p>\n","protected":false},"excerpt":{"rendered":"

The vocabulary of a neural network is represented as vectors \u2014 which I wrote about yesterday. This enables many related words to be “close to” one another, which is how the network perceives similarity and difference. This is as near as a computer comes to understanding meaning \u2014 which is not very near at all,… Continue reading Imagining words as numbers in n-dimensional space<\/span> →<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[2],"tags":[93,91],"class_list":["post-446","post","type-post","status-publish","format-standard","hentry","category-nlp","tag-vectors","tag-words"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/posts\/446","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/comments?post=446"}],"version-history":[{"count":6,"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/posts\/446\/revisions"}],"predecessor-version":[{"id":453,"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/posts\/446\/revisions\/453"}],"wp:attachment":[{"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/media?parent=446"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/categories?post=446"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.macloo.com\/ai\/wp-json\/wp\/v2\/tags?post=446"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}