Building a home lab

A home lab is a specialized computing device designed for specific tasks such as running development containers, networked storage, and various functional applications. It offers a controlled and customizable environment suitable for computing, learning, or hobby purposes.

A home server can serve as a Network Attached Storage (NAS) media server for your local network, delivering a streamlined and organized method for storing and accessing media and files. Additionally, it can host applications and virtualized systems like Home Assistant, Jupyter notebooks, and web servers. With GPU support, a custom home server can accommodate a broader range of workloads, including training and inference for large language models, genome assembly, 3D rendering, and more.

As a note to the reader, this was written in two parts. The original server we built focused on storage and hosting local applications 24/7. It consumes around 200W and has enough resources for things that need to run persistently. The second build we added is a 1000W unit with a 4090 GPU and 16 core CPU for multimedia and machine learning related workloads. Having local GPU support was essential, but not neccessarily something I want to run all the time, so it made sense to run it as a separate appliance.

Picking components

Before diving into the build process, determine the primary functions your server(s) will serve and the performance expectations you have. Consider factors such as budget, available space, and potential future upgrades. By clearly outlining these aspects, you'll be better equipped to choose suitable components for your build.

Invest time in researching the components that align with your requirements and ensure compatibility between them. Remember that using consumer-grade components in certain areas can help reduce costs without significantly impacting performance, however for applicances you hope to run 24/7, it is important to pick components that can handle such loads.

Build 1

Build 1 (aka Earthlab), our mid-tier multi-purpose server, features a 2U server chassis, providing reasonable space for expansion and compatibility with most server racks. The Asus AM4 TUF X570-Plus motherboard was chosen for its compatibility with the latest AMD Ryzen processors and support for PCIe 4.0. We opted for the AMD Ryzen 5 5600X CPU, which offers excellent performance for the price and can handle a wide range of tasks.

For our GPU, we selected the ZOTAC GeForce GTX 1650, which offers solid performance at an affordable price. Our memory choice is 64GB of Corsair DDR4 RAM, providing enough capacity for running multiple applications and processes.

Storage is a combination of high-capacity Western Digital HDDs set up in RAID 6, SATA SSDs for frequently accessed data, and a high-performance NVMe SSD for the operating system and essential applications. Finally, a reliable and efficient power supply, the CORSAIR RM650x, powers our server.

These are components I had access to, so they're just here as a reference, not recommendations. With that said, this build has been performing great and recently hit 30 days of persistent uptime without a hitch. As far as a average bit for a NAS and couple apps and dev boxes go this works very well.

Build 2

The second build is for more serious workloads (...and some gaming). For this build, I started with a pre-built gaming PC for most of the components. I'll be using almost all the componenst besides the power supply, chassis and an additioanl GPU.

At the time of writing I couldn't find the server chassis I would need to comfortably fit two 4090s GPUs, but I did find out that a 4U chassis almost fits everything, except the GPU power cables that will stick out the side. Given these are 4090s, I wouldn't want to stress those cables with any sharp bends. You'd want atleast a 5U or 6U chassis for this unless you can find a way to mount the 4090s on the side.

Assembly

When assembling your server, focus on proper airflow and cable management for optimal performance and easier maintenance down the line. A well-organized build will not only be more efficient but also simpler to troubleshoot if issues arise.

Once the hardware assembly is complete, install your preferred operating system (e.g., Linux, Windows Server, Proxmox) and set up the required services, applications, and network configurations.

Software

Managing a home server is not unlike managing any other server.There are a lot of options in terms of software you can run. The best choice for you depends on your specific needs, preferences, and familiarity with the platform. Some factors to consider include ease of use, hardware compatibility, feature set, scalability, and cost.

As the base operating system for the hardware, I was mostly considering between Proxmox and TrueNAS SCALE.

Proxmox

An excellent choice for home labs due to its user-friendly web-based interface, robust feature set, and support for both VMs and containers. It's open-source, free to use, and offers a good balance between functionality and ease of setup. Proxmox is ideal if you want an all-in-one solution for managing storage, virtualization, and networking.

TrueNAS SCALE

TrueNAS SCALE is a Linux-based, open-source, and free-to-use NAS operating system that also supports virtualization capabilities. It is developed by iXsystems, the same company that is behind the well-known FreeNAS/TrueNAS Core operating system. TrueNAS SCALE is designed to provide a reliable and feature-rich NAS solution with the added benefits of virtualization options.

Key features of TrueNAS SCALE:

• Easy-to-use interface: TrueNAS SCALE offers a user-friendly web interface that makes managing your NAS and virtualization tasks a breeze, even for those with limited experience in server administration.

• Strong support for ZFS file system: ZFS is a powerful file system that offers data integrity, snapshots, and efficient data compression. TrueNAS SCALE provides excellent support for ZFS, ensuring optimal performance and data protection.

• Compatibility with VMs and containers: In addition to NAS functionality, TrueNAS SCALE is capable of running virtual machines (VMs) and containers through its built-in virtualization platform. This allows you to host various applications, services, and development environments on your home lab server.

• Scalability: TrueNAS SCALE is designed to scale with your needs. As your storage and virtualization requirements grow, you can easily expand your server's capacity and capabilities.

After considering both Proxmox and TrueNAS SCALE, I opted for TrueNAS SCALE for my home lab server due to its focus on NAS functionality, strong support for the ZFS file system, and compatibility with both VMs and containers. As my primary interest was in having a robust NAS solution with additional virtualization options, TrueNAS SCALE was the perfect fit for my needs.

It is worth noting that TrueNAS SCALE is relatively new software. Generally speaking, it's been stable in its functionality as a NAS, but the apps and VM functionality have been somewhat buggy. They were mostly UI bug and not neccessarily stability related. Give this, if your main focus is virtualization, it could better to run Proxmox.

Maintenance and uptime

Proper maintenance and monitoring of your server are essential to ensure its reliability, security, and performance. Establishing a regular maintenance schedule and following best practices will help you achieve better uptime and extend the life of your server.

Monitoring

• System performance: Keep an eye on your server's CPU, RAM, and disk usage. Monitoring tools such as htop, top, or Glances can help you track system performance in real-time. Setting up alerts for high resource usage can help you identify potential issues before they become critical.

• Log files: Regularly review log files to identify potential issues, security threats, or application errors. Tools like Logwatch, Graylog, or the ELK stack (Elasticsearch, Logstash, Kibana) can help automate log analysis and alert you to potential problems.

• Uptime: Monitor your server's uptime using tools such as Uptime Robot, Pingdom, or a self-hosted solution like Nagios or Zabbix. These tools can notify you if your server becomes unreachable or experiences downtime.

Updates and Security

• Operating system updates: Keep your server's operating system up-to-date by regularly installing security patches and software updates. This will help protect your server from vulnerabilities and improve stability. Set up a schedule to check for updates at least once a week, and more frequently for critical security updates.

• Application updates: Similarly, keep your server's applications and services updated to their latest stable versions. This ensures you have the latest features and security improvements. Regularly review the release notes for any significant changes or known issues.

• Backups: Regularly back up your server's data to protect against data loss or corruption. Implement a 3-2-1 backup strategy: keep at least three copies of your data, store two of them on different storage media, and have at least one copy offsite.

• Firewall and security: Configure and maintain a properly configured firewall to protect your server from unauthorized access. Regularly review firewall rules and update them as needed. Use intrusion detection systems (IDS) like Snort or Suricata to monitor network traffic for malicious activity.

• User account management: Regularly review user accounts and permissions, removing any unused or unnecessary accounts. Enforce strong passwords and use multi-factor authentication where possible.

By following a couple maintenance and monitoring practices, you can ensure your server remains secure, performs optimally, and achieves better uptime. Remember that consistency is key, so set up a maintenance schedule and stick to it.

Building a home lab can be a rewarding experience, offering a centralized platform for learning, experimentation, and hosting. By carefully selecting components and software tailored to your specific needs, you can create a powerful and versatile server that can grow and adapt with you over time.

While building and maintaining a home lab server comes with challenges, including the initial investment and ongoing costs, the utility it brings outweighs those drawbacks. A home lab can provide an excellent platform for learning, experimenting, and hosting a variety of projects, applications, and services. By carefully considering your requirements, budget, and available space, you can create a custom home server that meets your needs and allows for future upgrades and expansion.

Whether you're a maker, technology enthusiast, or someone looking to localize their digital resources, a home lab server can be a valuable addition to your home network.

Changes and upgrades

As with most technology, a home lab server is not a static entity. With this in mind, this is an ongoing section of changes and upgrades that I've made to the server since setting it up.

Upgrading to ECC RAM

One such upgrade that can enhance the overall performance and reliability of the server is the switch to ECC RAM. ECC (Error-Correcting Code) memory is engineered to detect and correct data corruption, ensuring your server operates at peak performance with minimal risk of errors. This suggestion, courtesy of a Redditor, is particularly applicable when you're expecting to run a system 24/7.