←All Projects

NFS Ganesha for Cost-Effective RWM Storage

 · Part of my work at Carousell

Abstract:  Implemented NFS Ganesha on Kubernetes to provide a cost-effective ReadWriteMany (RWM) storage solution, overcoming the limitations of standard GCP storage offerings.

Tech: #Kubernetes#NFS#GCP#Persistent Storage#Cost Optimization

The Challenge

A key challenge when running certain stateful applications on Google Kubernetes Engine (GKE) is the need for persistent storage with a ReadWriteMany (RWM) access mode, which allows a volume to be mounted as read-write by multiple pods simultaneously. The native GCP solutions for this, like Filestore, were prohibitively expensive for our use case and budget. This limitation blocked the deployment of several important workloads to our Kubernetes clusters.

The Solution

To solve this problem, I researched and implemented a solution using NFS Ganesha, an open-source NFS server that runs in user space. The project involved:

  1. Architecture: Designing a resilient architecture where an NFS Ganesha server could be deployed within our Kubernetes cluster, backed by a standard, cost-effective Google Persistent Disk (in ReadWriteOnce mode).
  2. Implementation: Deploying and configuring the NFS Ganesha server, which then exports the underlying block storage as an NFS share.
  3. Integration: Configuring a PersistentVolume (PV) that points to this internal NFS server, which can then be claimed by multiple pods via a PersistentVolumeClaim (PVC) with ReadWriteMany access.

This innovative approach provided a fully functional and budget-friendly RWM storage layer, unblocking critical application deployments and improving resource utilization without incurring high cloud storage costs.