EKS Auto Mode vs. Luna: Choosing the Right Scaling Strategy for Your Kubernetes Workloads

Running Kubernetes on AWS using Elastic Kubernetes Service (EKS) offers a robust platform for container orchestration, but the challenge of managing the underlying compute infrastructure persists. This limitation can be addressed through various approaches, including the fully managed simplicity of EKS Auto Mode or the granular control offered by an intelligent Kubernetes cluster autoscaler like Luna. In this post, we’ll explore the advantages of each, helping you choose the best scaling strategy for your workloads.

---

Introduction

EKS Auto Mode is a fully managed solution aimed at reducing operational complexity for Kubernetes clusters on AWS. It automates essential tasks like node provisioning, scaling, and lifecycle management, offering an ideal entry point for teams new to EKS or operating simpler workloads.

In contrast, compute autoscalers like Luna offer greater flexibility and customization, allowing you to optimize your infrastructure for the demands of complex and/or resource-intensive workloads.

Understanding the nuances of these approaches is key to selecting the optimal scaling solution for your Kubernetes deployments.

EKS Auto Mode: The Allure of Simplicity

EKS Auto Mode shines in its simplicity. AWS takes on the heavy lifting of managing your worker nodes, handling everything from provisioning and scaling to OS patching and even instance type selection. This "swift lift" approach offers several key advantages:

• Reduced Operational Burden: By automating core infrastructure management tasks, Auto Mode frees up your team to focus on application development and deployment.
• Simplified Security Posture: Auto Mode defaults to Bottlerocket, a purpose-built, security-focused container operating system. Bottlerocket's minimal attack surface, CIS Level 1 benchmark certification, and FIPS 140-3 compliance provide a strong foundation for secure container workloads.
• Streamlined Upgrades: Leveraging Karpenter under the hood, Auto Mode automates node refreshes and ensures consistent patching, minimizing security risks and maintaining cluster stability.
• Simplified Setup with Built-in Add-ons: Essential EKS add-ons, such as the EBS CSI driver for persistent storage and the ALB Ingress Controller for load balancing, are automatically deployed during cluster creation, further simplifying the setup process.

However, this simplicity comes at a cost. Auto Mode's opinionated approach introduces several limitations:

• Irreversible Activation: Once EKS Auto Mode is enabled on a cluster, it cannot be disabled. This irreversible change requires careful consideration before activation, as it commits the cluster to the Auto Mode management permanently.
• Limited Node Configuration Flexibility: EKS Auto Mode offers minimal control over node shapes and configurations. You cannot include or exclude specific instance sizes, or fine-tune the infrastructure to meet specialized workload requirements. This lack of flexibility means that Auto Mode's node provisioning is based on a predefined set of instance types selected by AWS.
• Limited Customization: EKS Auto Mode restricts customization at the node level. You are unable to modify kernel parameters, install custom system packages, or adjust kubelet settings. These limitations make it challenging to meet the requirements of workloads that depend on specific OS configurations or custom software installations.
• Spot Support:
  While EKS Auto Mode simplifies operations, it does not leverage or support spot instances for cost savings, unlike some advanced autoscalers like Luna. This could result in higher operational costs for workloads where spot instances could be safely utilized.
• Bottlerocket Dependency: The reliance on Bottlerocket, while beneficial for security, prevents the use of custom Amazon Machine Images (AMIs), which might be necessary for specific software or compliance requirements.
• Potential for IP Address Exhaustion: Auto Mode utilizes prefix delegation, assigning /28 CIDR blocks to each node. In VPCs with limited IP address space, this can lead to IP exhaustion issues, preventing the creation of new nodes and halting cluster scaling altogether.
• Default Networking Overhead:
  EKS Auto Mode relies on AWS-managed networking configurations, which can introduce inefficiencies in specific scenarios, such as cross-AZ traffic or high-latency workloads, due to default routing setups.
• Reduced Visibility: The automated nature of Auto Mode reduces direct visibility into the node provisioning and configuration processes, making detailed troubleshooting more reliant on AWS's logging and monitoring tools.

When Does EKS Auto Mode Shine?

Auto Mode is ideal for:

• Small, Simple Clusters: Perfect for teams running standard workloads without complex resource needs.
• New Users: A smooth on-ramp for Kubernetes beginners, focusing on applications without delving into infrastructure.
• Testing and Experimentation: Auto Mode's streamlined setup makes it ideal for quickly creating and tearing down temporary clusters for testing, prototyping, or experimentation.
  

Luna: Embracing Flexibility and Control

For teams managing larger or more complex clusters, Luna’s flexibility and control offer significant advantages.

What Does Luna Offer?

Luna provides a dynamic, customizable approach to autoscaling that empowers you to fine-tune every aspect of node management:

• Highly Flexible Instance Selection: Luna dynamically selects appropriate node shapes based on workload requirements such as CPU, memory, architecture (including ARM), and other criteria. This flexibility ensures that the infrastructure is tailored to meet the unique demands of your applications.
• Spot Instance Support for Cost Optimization: Luna enables the use of spot instances, provisioning cost-effective nodes when desired, and based on availability. By incorporating spot instances and mixed instance types, Luna significantly reduces infrastructure costs while maintaining high availability.
• Granular Instance Control: Inclusion and exclusion lists allow you to define allowed and disallowed instance types/families, optimizing for cost, performance, or specific hardware requirements.
• Cost-Driven Instance Selection: Luna dynamically selects the least expensive, available instance shape that meets workload requirements, minimizing infrastructure spending.
• Hardware Specialization: Supports GPU acceleration and other specialized hardware for resource-intensive applications.
• Support for Custom AMIs: Luna allows you to choose a specific AMI or use your own custom AMI, enabling fine-grained control over the OS and installed software.
• Advanced Scheduling Capabilities: Features like node taints, tolerations, and node affinity allow precise control over pod placement, Luna provisions the appropriate nodes to support this placement as required.
• Serverless-like Experience: Luna automates much of the underlying node management, offering a simplified operational experience similar to EKS Auto Mode but with greater flexibility.
  

Key Benefits of Luna

• Unparalleled Flexibility: Ideal for environments requiring specific configurations, hardware accelerations, or software setups.
• Advanced Cost Optimization with Spot: Spot instance utilization can drastically reduce infrastructure costs compared to on-demand-only nodes.
• Scalable for Large Clusters: As clusters grow in complexity and size, Luna ensures scalability without sacrificing control.
• Enhanced Workload Support: Handles diverse and complex workloads better than Auto Mode, offering tailored solutions for every use case.
• Fine-Grained Control: If your workloads demand specific instance types, OS configurations, or hardware acceleration (like GPUs), an Intelligent Kubernetes Cluster Autoscaler such as Luna is essential.
• Ease of Deployment, Configuration, and Upgrades: Compared to other autoscalers, Luna streamlines the deployment and configuration process for autoscaling within your EKS clusters. While it requires slightly more setup than EKS Auto Mode, it offers greater flexibility and customization with relatively low effort. Additionally, Luna supports smooth upgrades, ensuring new features and improvements can be rolled out with minimal disruption to cluster operations.
  

Choosing the Right Approach

The decision between EKS Auto Mode and Luna boils down to your priorities and workload characteristics:

• Consider Choosing EKS Auto Mode if:
  • You’re running small, straightforward clusters with minimal customization needs.
  • You’re new to Kubernetes and want a streamlined experience.
  • Your team prioritizes ease of use over granular control.
• Consider Choosing Luna if:
  • You need precise control over infrastructure, including custom AMIs and hardware configurations.
  • Your workloads demand advanced scheduling, cost optimization, or specialized resources like GPUs.
  • You’re managing large clusters with busty workloads and/or diverse application requirements.
    

Conclusion

Kubernetes compute scaling within EKS requires choosing a solution that aligns with your operational priorities, workload complexity, and cost management goals. EKS Auto Mode simplifies Kubernetes management with automation and preconfigured settings, making it an excellent choice for smaller clusters, standard workloads, or teams looking for a low-maintenance entry point. Its ease of use allows you to focus on deploying applications without being bogged down by infrastructure details.

On the other hand, an Intelligent Kubernetes Cluster Autoscaler like Luna offers the flexibility, control, and cost optimization needed for growing, complex, bursty, or resource-intensive deployments. Whether you're fine-tuning node configurations, optimizing for diverse workload requirements, or leveraging advanced features like spot instances, Luna provides the autoscaling necessary to efficiently scale clusters tailored to your unique needs and workloads.

The choice isn’t about one being inherently better than the other—it’s about understanding your requirements. For teams prioritizing simplicity and rapid deployment, Auto Mode is worth considering as a viable option. For those needing advanced scaling capabilities and greater customization, Luna’s robust feature set provides unmatched value. By carefully evaluating these factors, you can adopt the solution that delivers the best results for your Kubernetes journey on AWS.


Author:
Justin Willoughby (Principal Solutions Architect, Elotl)


Disclaimer: The features and limitations of EKS Auto Mode as described in this blog are based on the author’s understanding at the time of publication. AWS may update or change these features over time, and readers are encouraged to consult the official AWS documentation for the most up-to-date information.