{"id":246,"date":"2021-09-09T23:31:58","date_gmt":"2021-09-09T20:31:58","guid":{"rendered":"https:\/\/dev-word.com\/?p=246"},"modified":"2022-01-03T08:14:01","modified_gmt":"2022-01-03T06:14:01","slug":"homelab-kubernetes-cluster-part-0","status":"publish","type":"post","link":"https:\/\/dev-word.com\/index.php\/2021\/09\/09\/homelab-kubernetes-cluster-part-0\/","title":{"rendered":"Homelab Kubernetes Cluster &#8211; Part 0"},"content":{"rendered":"\n<p>Kubernetes has become a standard in cloud deployment. As a result, deploying your application in the cloud using Kubernetes or tools based on it is very common. In the beginning, playing with Minikube satisfied our needs, curiosity, and knowledge-seeking. But, as we move towards production environments and start to understand high availability and scalability, we realize we need more powerful toys with lots of CPU, RAM, and storage for testing and experimenting. In the following months, I will describe in a step-by-step guide how I built my homelab cluster for Kubernetes, what were the mistakes I made and how I solved them, and what you should consider before going on this road.<\/p>\n\n\n\n<h1 class=\"wp-block-heading\">End-Goal<\/h1>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"768\" height=\"1024\" src=\"https:\/\/dev-word.com\/wp-content\/uploads\/2021\/09\/IMG_1242-768x1024.jpg\" alt=\"\" class=\"wp-image-251\" srcset=\"https:\/\/dev-word.com\/wp-content\/uploads\/2021\/09\/IMG_1242-768x1024.jpg 768w, https:\/\/dev-word.com\/wp-content\/uploads\/2021\/09\/IMG_1242-225x300.jpg 225w, https:\/\/dev-word.com\/wp-content\/uploads\/2021\/09\/IMG_1242-1152x1536.jpg 1152w, https:\/\/dev-word.com\/wp-content\/uploads\/2021\/09\/IMG_1242-1536x2048.jpg 1536w, https:\/\/dev-word.com\/wp-content\/uploads\/2021\/09\/IMG_1242-1200x1600.jpg 1200w, https:\/\/dev-word.com\/wp-content\/uploads\/2021\/09\/IMG_1242-scaled.jpg 1920w\" sizes=\"auto, (max-width: 768px) 100vw, 768px\" \/><\/figure><\/div>\n\n\n\n<p>The end goal is to have an up and running&nbsp;<a rel=\"noreferrer noopener\" target=\"_blank\" href=\"https:\/\/kubernetes.io\/docs\/concepts\/overview\/what-is-kubernetes\/\"><strong>Kubernetes<\/strong><\/a>&nbsp;cluster at home, ready to run and scale applications.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Virtualization<\/h3>\n\n\n\n<p>I will build the cluster using <strong><a href=\"https:\/\/www.proxmox.com\/en\/\">Proxmox VE 7.0<\/a>,<\/strong> an excellent open-source virtualization management platform based on <strong><a href=\"https:\/\/www.linux-kvm.org\/page\/Main_Page\">KVM<\/a><\/strong>, with over 15 years of proved history. I found it to be very flexible and easy to use. In addition, it offers out-of-the-box features like automatic VM backup, notifications, replication, fencing, a Ceph Interface, and many more.<\/p>\n\n\n\n<p>The obvious alternative would have been<a rel=\"noreferrer noopener\" target=\"_blank\" href=\"https:\/\/www.vmware.com\/products\/vsphere.html\">&nbsp;<strong>VMWare vSphere<\/strong><\/a>, but the price for three physical hosts with two CPUs was, at the time of this writing, around 600 euros per year. If you are willing to pay this amount of money for the virtualization technology, I suggest going for it. It is more common in software companies, and you will probably have more chances to run into VMWare than Proxmox.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Kubernetes distribution<\/h3>\n\n\n\n<p>I will be using a <strong>Kubernetes (v1.21+)<\/strong> distribution cluster using <strong>Ubuntu<\/strong> <strong>Server 20.04<\/strong> virtual machines and <a href=\"https:\/\/microk8s.io\/\"><strong>Microk8s<\/strong><\/a>, Canonical&#8217;s lightweight Kubernetes distribution. At this point, I will branch out into how to run a <a href=\"https:\/\/k3s.io\/\"><strong>K3s<\/strong><\/a>(Rancher&#8217;s Kubernetes distribution) and <a href=\"https:\/\/www.okd.io\/\"><strong>OKD<\/strong><\/a>(RedHat community Kubernetes distribution) cluster as they are the most used alternatives to Microk8s.<\/p>\n\n\n\n<p>For storage, I will be using multiple technologies such as&nbsp;<strong>NFS<\/strong>&nbsp;and&nbsp;<strong>iSCSI<\/strong>&nbsp;on a separate&nbsp;<a rel=\"noreferrer noopener\" target=\"_blank\" href=\"https:\/\/www.truenas.com\/\"><strong>TrueNAS<\/strong><\/a>&nbsp;server,&nbsp;<a rel=\"noreferrer noopener\" target=\"_blank\" href=\"https:\/\/docs.ceph.com\/en\/latest\/\"><strong>Ceph<\/strong><\/a>, and&nbsp;<a rel=\"noreferrer noopener\" target=\"_blank\" href=\"https:\/\/min.io\/\"><strong>MinIO<\/strong><\/a>. I will also detail how to install and configure a TrueNAS server to work with Kubernetes and Proxmox.<\/p>\n\n\n\n<h1 class=\"wp-block-heading\">Considerations<\/h1>\n\n\n\n<p>Having a homelab sounds very cool, and when you see a complete rack in your home office, it definitely feels fulfilling. Add some Prometheus dashboards, and you have an absolute masterpiece. Soon, you&#8217;ll be inviting fellow developers over to show them your new pet and brag about how powerful it is and how many things it can be.<\/p>\n\n\n\n<p>But&#8230;You should consider the following aspects:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Hardware cost money<\/h3>\n\n\n\n<p>Depending on your end goal and on how much you are willing to spend on the metal pieces, you should decide on a budget and stick to it. Next, you must consider infrastructure parts such as HDDs, SSDs, cables, routers, switches, UPSs, and probably an air conditioner.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Things will break<\/h3>\n\n\n\n<p>Two of my second-hand HDDs crashed in my first two months, then one cooler, and then an SFP+ cable. These are pretty cheap components, but you should take into account that they will require maintenance.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Heat &amp; Noise<\/h3>\n\n\n\n<p>In the beginning, I was planning to keep them in the custom &#8220;rack&#8221; that I built (see the picture at the top) in my living room. I soon discovered that the heat produced couldn&#8217;t be ignored, and I had to keep my AC full throttle all day long. The noise, on the other hand, was unbearable. Even from another room with the door closed, I could hear them. <\/p>\n\n\n\n<p>Long story short, I ended up building a two square meters room dedicated to servers and storage where I installed an air conditioner, a dehumidifier, and a temperature and humidity sensor. And, of course, a very thick door.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Know the hardware<\/h3>\n\n\n\n<p>Be sure you understand the limitations of the hardware before buying it. The price difference between an&nbsp;<a rel=\"noreferrer noopener\" target=\"_blank\" href=\"https:\/\/support.hpe.com\/hpesc\/public\/docDisplay?docId=emr_na-c03235277\"><strong>HP Proliant 380p G8<\/strong>&nbsp;<\/a>and an&nbsp;<a rel=\"noreferrer noopener\" target=\"_blank\" href=\"https:\/\/support.hpe.com\/hpesc\/public\/docDisplay?docId=emr_na-c04438844\"><strong>HP Proliant 380 G9<\/strong><\/a>&nbsp;has a reason behind it. Check the performance of CPUs and RAM against your requirements. If you plan to deploy CPU-intensive applications, you should focus more on the CPU, especially on high frequency per core. If you plan to host websites or deploy microservices with low to medium resource requirements, you should focus more on RAM.<\/p>\n\n\n\n<p>On the storage part, it&#8217;s the same story &#8211; HDDs or SSDs, Hardware RAID, or Software RAID. Check your I\/O disk requirements. You might find out that HDDs are more than enough for your needs.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Learning curve<\/h3>\n\n\n\n<p>It&#8217;s not easy. If you come from the Software Development world, as I do, then you&#8217;ll have to strengthen your patience and buy a lot of coffee. There will be long nights, you&#8217;ll reach long-forgotten forums and threads, and there will be days when you&#8217;ll go to bed with more questions than you woke up with, which will put you out of your comfort zone big time.<\/p>\n\n\n\n<h1 class=\"wp-block-heading\">Conclusion<\/h1>\n\n\n\n<p>Either you plan to host your applications or play around with Kubernetes, it&#8217;s a good skill for a developer to have. It will broaden your knowledge, and you will have to get used to things such as Switches &amp; Routers, Networking &amp; the OSI model, Linux &amp; Scripting, Virtualization, and so on.<\/p>\n\n\n\n<p>These skills will probably become &#8220;nice to have&#8221; for the full-stack position shortly. According to&nbsp;<a rel=\"noreferrer noopener\" target=\"_blank\" href=\"https:\/\/www.cncf.io\/wp-content\/uploads\/2020\/11\/CNCF_Survey_Report_2020.pdf\"><strong>CNCF&#8217;s survey<\/strong><\/a>, in 2020, <strong>83% <\/strong>of the respondents are using Kubernetes in <strong>production<\/strong>. This rate increased by 5% from the previous year, and these trends confirm that the adoption will continue.<\/p>\n\n\n\n<p>Stay tuned for the next article to find out more about the hardware choices, topology, and budget.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This is the first article in a series where I will explain how to build a Kubernetes cluster on top of Proxmox, running on HP Proliant servers at home.<\/p>\n","protected":false},"author":1,"featured_media":257,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[46,51,43,45],"tags":[49,50,47,52,48],"class_list":["post-246","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cluster","category-homelab","category-kubernetes","category-virtualization","tag-cluster","tag-homelab","tag-kubernetes","tag-microk8s","tag-proxmox"],"_links":{"self":[{"href":"https:\/\/dev-word.com\/index.php\/wp-json\/wp\/v2\/posts\/246","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dev-word.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dev-word.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dev-word.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dev-word.com\/index.php\/wp-json\/wp\/v2\/comments?post=246"}],"version-history":[{"count":10,"href":"https:\/\/dev-word.com\/index.php\/wp-json\/wp\/v2\/posts\/246\/revisions"}],"predecessor-version":[{"id":272,"href":"https:\/\/dev-word.com\/index.php\/wp-json\/wp\/v2\/posts\/246\/revisions\/272"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dev-word.com\/index.php\/wp-json\/wp\/v2\/media\/257"}],"wp:attachment":[{"href":"https:\/\/dev-word.com\/index.php\/wp-json\/wp\/v2\/media?parent=246"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dev-word.com\/index.php\/wp-json\/wp\/v2\/categories?post=246"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dev-word.com\/index.php\/wp-json\/wp\/v2\/tags?post=246"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}