{"id":6204,"date":"2016-09-19T14:35:39","date_gmt":"2016-09-19T14:35:39","guid":{"rendered":"https:\/\/adamchandler.me\/blog\/?p=6204"},"modified":"2016-09-26T19:12:20","modified_gmt":"2016-09-26T19:12:20","slug":"linked-heavy-metal-and-natural-language-processing-part-1","status":"publish","type":"post","link":"https:\/\/adamchandler.me\/blog\/2016\/09\/19\/linked-heavy-metal-and-natural-language-processing-part-1\/","title":{"rendered":"Linked: \u201cHeavy Metal and Natural Language Processing &#8211; Part 1\u201d"},"content":{"rendered":"<p>via <a href=\"#\">Degenerate State<\/a>:<\/p>\n<blockquote><p>As you can see, Five Finger Death Punch have the highest number of swear words in their lyrics, and Pig Destroyer have the most complex wordplay. It also suggests that bands that swear more seem to use more complex words.<\/p>\n<p>While this is an interesting way to represent the bands, it is limited in what it captures. In what follows, I&#8217;m going to explore more general ways we can looking at natural language, focusing on the &#8220;bag-of-words&#8221; model.<\/p>\n<p>A &#8220;bag-of-words&#8221; model is one where we only care about the frequencies with which each word appears in the text of a document. In other words we throw away all information about the relative ordering of words. This approach obviously loses some information about the document being analyses. For example, the phrases &#8220;dog bites man&#8221; and &#8220;man bites dog&#8221; would end up with the same representation, despite referring to very different events. However, we do capture some information about the phrases, namely that both of them refer to a &#8220;dog&#8221; and to a &#8220;man&#8221;. This suggests that in looking at word frequencies we are capturing some information about the &#8220;topic&#8221; of the models.<\/p><\/blockquote>\n<p>Most Metal = Burn<\/p>\n<p>Least Metal = Particularly<\/p>\n<p>Great.<\/p>\n","protected":false},"excerpt":{"rendered":"via Degenerate State: As you can see, Five Finger Death Punch have the highest number of swear words&hellip;\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_seo_schema_type":"","apple_news_api_created_at":"","apple_news_api_id":"","apple_news_api_modified_at":"","apple_news_api_revision":"","apple_news_api_share_url":"","apple_news_cover_media_provider":"image","apple_news_coverimage":0,"apple_news_coverimage_caption":"","apple_news_cover_video_id":0,"apple_news_cover_video_url":"","apple_news_cover_embedwebvideo_url":"","apple_news_is_hidden":"","apple_news_is_paid":"","apple_news_is_preview":"","apple_news_is_sponsored":"","apple_news_maturity_rating":"","apple_news_metadata":"\"\"","apple_news_pullquote":"","apple_news_pullquote_position":"","apple_news_slug":"","apple_news_sections":[],"apple_news_suppress_video_url":false,"apple_news_use_image_component":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"csco_singular_sidebar":"","csco_page_header_type":"","csco_custom_appearance":"","csco_disable_excerpt_posts_layout":false,"csco_page_load_nextpost":"","csco_post_video_location":[],"csco_post_video_location_hash":"","csco_post_video_url":"","csco_post_video_bg_start_time":0,"csco_post_video_bg_end_time":0,"_jetpack_hide_featured_image":false,"_jetpack_featured_image_duplicate_dismissed":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[7],"tags":[],"class_list":["post-6204","post","type-post","status-publish","format-standard","category-linked","cs-entry","cs-video-wrap"],"apple_news_notices":[],"jetpack_publicize_connections":[],"jetpack_shortlink":"https:\/\/wp.me\/pb3IC4-1C4","jetpack-related-posts":[{"id":5866,"url":"https:\/\/adamchandler.me\/blog\/2016\/07\/05\/linked-aw-so-cute-allo-helps-you-respond-to-shared-photos\/","url_meta":{"origin":6204,"position":0},"title":"Linked: &#8220;Aw, so cute!&#8221;: Allo helps you respond to shared photos","author":"Adam Chandler","date":"July 5, 2016","format":false,"excerpt":"via Google Research Blog on Allo (their new messaging app w\/ AI built right in): We utilize Google's image recognition technology, developed by our Machine Perception team, to associate images with semantic entities \u2014 people, animals, cars, etc. We then apply a machine learned model that maps those recognized entities\u2026","rel":"","context":"In &quot;Linked&quot;","block_context":{"text":"Linked","link":"https:\/\/adamchandler.me\/blog\/category\/linked\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":7247,"url":"https:\/\/adamchandler.me\/blog\/2017\/12\/08\/linked-taco-bells-website-is-absolutely-insane\/","url_meta":{"origin":6204,"position":1},"title":"Linked: &#8220;Taco Bell&#8217;s Website Is Absolutely Insane&#8221;","author":"Adam Chandler","date":"December 8, 2017","format":false,"excerpt":"via Cracked: Here's only a quarter of the words Taco Bell uses to describe the general concept of fountain drinks: Hey, this is a cup. It comes in four sizes, and you can fill it with the fountain drink of your choice. You can put ice in it, too, if\u2026","rel":"","context":"In &quot;Linked&quot;","block_context":{"text":"Linked","link":"https:\/\/adamchandler.me\/blog\/category\/linked\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":6956,"url":"https:\/\/adamchandler.me\/blog\/2017\/08\/03\/linked-my-interview-with-apple\/","url_meta":{"origin":6204,"position":2},"title":"Linked: &#8220;My Interview with Apple&#8221;","author":"Adam Chandler","date":"August 3, 2017","format":false,"excerpt":"via Cult of Mac (Blogspot): I don't think I've ever stumbled through an interview like that in my life. I was so nervous, and all the waiting and panicking in general didn't add much placidity to my state of mind. Working for Apple is a dream for me, and all\u2026","rel":"","context":"In &quot;Linked&quot;","block_context":{"text":"Linked","link":"https:\/\/adamchandler.me\/blog\/category\/linked\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":3196,"url":"https:\/\/adamchandler.me\/blog\/2011\/10\/10\/steve-jobs-in-his-own-words\/","url_meta":{"origin":6204,"position":3},"title":"\u2605 &#8220;Steve Jobs in his own words&#8221;","author":"Adam Chandler","date":"October 10, 2011","format":false,"excerpt":"Engadget has a nice roundup. \u00a0","rel":"","context":"In &quot;Linked&quot;","block_context":{"text":"Linked","link":"https:\/\/adamchandler.me\/blog\/category\/linked\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":7443,"url":"https:\/\/adamchandler.me\/blog\/2018\/05\/21\/screwing-your-vocal-minority-500ish-words\/","url_meta":{"origin":6204,"position":4},"title":"Screwing Your Vocal Minority \u2013 500ish Words","author":"Adam Chandler","date":"May 21, 2018","format":false,"excerpt":"Screwing Your Vocal Minority \u2013 500ish Words: All of this is to say, it\u2019s fucking insane that Twitter has killed their native Mac client not once, but twice. I don\u2019t care if it only had a hundred thousand users, a thousand users, or a hundred users, it\u2019s the abandonment of\u2026","rel":"","context":"In &quot;Linked&quot;","block_context":{"text":"Linked","link":"https:\/\/adamchandler.me\/blog\/category\/linked\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":5860,"url":"https:\/\/adamchandler.me\/blog\/2016\/07\/04\/linked-medium-a-publishers-perspective\/","url_meta":{"origin":6204,"position":5},"title":"Linked: &#8220;Medium: A Publisher&#8217;s Perspective&#8221;","author":"Adam Chandler","date":"July 4, 2016","format":false,"excerpt":"via MondayNote: Our findings were actually consistent with those of Medium\u2019s own data scientist Mike Sall who notes, in this post: Overall, 74% of posts are under 3 minutes long and 94% are under 6 minutes long. Mike Sall and his team ended up determining that the optimal length for\u2026","rel":"","context":"In &quot;Linked&quot;","block_context":{"text":"Linked","link":"https:\/\/adamchandler.me\/blog\/category\/linked\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/adamchandler.me\/blog\/wp-json\/wp\/v2\/posts\/6204","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/adamchandler.me\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/adamchandler.me\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/adamchandler.me\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/adamchandler.me\/blog\/wp-json\/wp\/v2\/comments?post=6204"}],"version-history":[{"count":0,"href":"https:\/\/adamchandler.me\/blog\/wp-json\/wp\/v2\/posts\/6204\/revisions"}],"wp:attachment":[{"href":"https:\/\/adamchandler.me\/blog\/wp-json\/wp\/v2\/media?parent=6204"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/adamchandler.me\/blog\/wp-json\/wp\/v2\/categories?post=6204"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/adamchandler.me\/blog\/wp-json\/wp\/v2\/tags?post=6204"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}