{"id":747,"date":"2018-10-03T18:33:55","date_gmt":"2018-10-03T18:33:55","guid":{"rendered":"http:\/\/raheelyawar.com\/?p=747"},"modified":"2018-10-03T18:33:55","modified_gmt":"2018-10-03T18:33:55","slug":"get-specifics-with-golang-memory-profiler","status":"publish","type":"post","link":"http:\/\/raheelyawar.com\/?p=747","title":{"rendered":"Get Specifics with Golang Memory Profiler"},"content":{"rendered":"<p><img data-recalc-dims=\"1\" decoding=\"async\" class=\"size-medium wp-image-768 alignright\" src=\"https:\/\/i0.wp.com\/raheelyawar.com\/wp-content\/uploads\/2018\/08\/go-gopher-miner.jpeg?resize=300%2C86\" alt=\"\" width=\"300\" height=\"86\" srcset=\"https:\/\/i0.wp.com\/raheelyawar.com\/wp-content\/uploads\/2018\/08\/go-gopher-miner.jpeg?resize=300%2C86 300w, https:\/\/i0.wp.com\/raheelyawar.com\/wp-content\/uploads\/2018\/08\/go-gopher-miner.jpeg?resize=768%2C220 768w, https:\/\/i0.wp.com\/raheelyawar.com\/wp-content\/uploads\/2018\/08\/go-gopher-miner.jpeg?resize=1024%2C293 1024w, https:\/\/i0.wp.com\/raheelyawar.com\/wp-content\/uploads\/2018\/08\/go-gopher-miner.jpeg?w=1500 1500w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/p>\n<p>I created a web-server using Golang that ran machine learning algorithms for a <a href=\"http:\/\/www.toggo.de\/spiele\/trolljaeger\/abenteuer-in-den-trollhoehlen-4774.htm\">video game<\/a>. I was aware that the algorithms will take a large number of resources as the player data increases but I saw very large and inconsistent memory spikes that could use up to\u00a04 to 6 gigs of RAM before going back to normal. Working with a web server that only had an 8GB of RAM in total and shared it with multiple services, I couldn&#8217;t have it so I attempted to investigate.<\/p>\n<p>Golang has a pretty good <a href=\"https:\/\/golang.org\/pkg\/runtime\/pprof\/\">documentation<\/a> for its profiler. I would recommend going through it before doing anything else since it shows the complete ability of the profiler. I worked with it to produce the following code.<\/p>\n<p><script src=\"https:\/\/gist.github.com\/RaheelYawar\/431ea1751da4caa1ff9171467cbd9d03.js\"><\/script><\/p>\n<p>When your Golang application runs the<\/p>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">executeMemoryProfiler()<\/pre>\n<p>function, it will export the memory profile in the &#8220;mem.prof&#8221; file. The next step is to render this to show a memory trace. The documentation and many examples point you towards the following command:<\/p>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">go tool pprof mem.prof<\/pre>\n<p>You can then type &#8220;web&#8221; in the terminal to render your memory profile using a web browser. The problem is that you don&#8217;t see a lot of information which you can work with. I found a helpful <a href=\"https:\/\/austburn.me\/blog\/go-profile.html\">blog post<\/a> that addressed this issue. Try again but use the following command:<\/p>\n<pre class=\"brush: plain; title: ; notranslate\" title=\"\">go tool pprof --alloc_space mem.prof<\/pre>\n<p>Again type &#8220;web&#8221; to render the profile. Now, you should see a more detailed flowchart of execution and the memory allocated to each function in that chart.<\/p>\n<p>I hope this helps out someone, there isn&#8217;t a whole lot of information available about the quirks of memory profiling so I thought it would be a good idea to have a very basic post about it. Thanks for reading.<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I created a web-server using Golang that ran machine learning algorithms for a video game. I was aware that the algorithms will take a large number of resources as the player data increases but I saw very large and inconsistent memory spikes that could use up to\u00a04 to 6 gigs of RAM before going back [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[36],"tags":[37,38,39],"class_list":["post-747","post","type-post","status-publish","format-standard","hentry","category-golang","tag-golang","tag-memory","tag-profiling"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p9NqGD-c3","_links":{"self":[{"href":"http:\/\/raheelyawar.com\/index.php?rest_route=\/wp\/v2\/posts\/747","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/raheelyawar.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/raheelyawar.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/raheelyawar.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/raheelyawar.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=747"}],"version-history":[{"count":9,"href":"http:\/\/raheelyawar.com\/index.php?rest_route=\/wp\/v2\/posts\/747\/revisions"}],"predecessor-version":[{"id":937,"href":"http:\/\/raheelyawar.com\/index.php?rest_route=\/wp\/v2\/posts\/747\/revisions\/937"}],"wp:attachment":[{"href":"http:\/\/raheelyawar.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=747"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/raheelyawar.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=747"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/raheelyawar.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=747"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}