# The Painful Journey of Deploying a Helm Chart to Kubernetes with Go

In the fast-paced world of software development, time-sensitive projects can often lead to both breakthroughs and challenges. Recently, I embarked on a project to deploy a Helm chart to a Kubernetes cluster using Go. This was because the helm Terraform provider is currently not working as I would expect it to and does not allow you to build an Azure Kubernetes Cluster and Deploy a Helm chart in the same run.  
  
Despite my best efforts, a critical bug in a dependency and several deployment hurdles made the journey incredibly frustrating. Here's an account of my experience and the code I used, highlighting the pain points and lessons learned.

#### The Goal

The objective was simple: automate the deployment of a Helm chart to a Kubernetes cluster. This included setting up the Kubernetes Cert-Manager objects and leveraging the [`github.com/mittwald/go-helm-client`](http://github.com/mittwald/go-helm-client) package to manage Helm operations. Theoretically, it was straightforward. In practice, it proved to be anything but.

### The Code

Here is the code I used for this project. Note that sensitive information has been omitted for security reasons.

```go
package main

import (
	"bytes"
	"context"
	"fmt"
	"log"
	"os"
	"path/filepath"
	"time"

	"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
	"github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v5"
	certmanagerv1 "github.com/cert-manager/cert-manager"
	helmclient "github.com/mittwald/go-helm-client"
	"github.com/mittwald/go-helm-client/values"
	"helm.sh/helm/v3/pkg/repo"
	core "k8s.io/api/core/v1"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/client-go/kubernetes"
	_ "k8s.io/client-go/plugin/pkg/client/auth"
	clientcmd "k8s.io/client-go/tools/clientcmd"

	"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
	"github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v5"
)

func main() {
	var outputBuffer bytes.Buffer
	ctx := context.Background()
	mycontext, _ := context.WithTimeout(ctx, 80000*time.Second)

	certmanageremail := "your-email@example.com"
	subid := "your-subscription-id"
	rg := "your-resource-group"
	cluster := "your-cluster-name"

	fmt.Println("Connecting to Cluster Via Azure Call\n")
	myaksconnect := connectToAks(ctx, subid, rg, cluster)
	mypublicip := "0.0.0.0"
	fmt.Printf("Moved %v to KubeConfig File", &myaksconnect.Name)
	fmt.Println("Connecting to Cluster using KubeConfig File\n")
	kubeClient := connectToK8s()

	fmt.Println("Building Helm Client\n")
	opt := &helmclient.Options{
		Namespace:        "default",
		RepositoryCache:  "/tmp/.helmcache",
		RepositoryConfig: "/tmp/.helmrepo",
		Debug:            false,
		Linting:          true,
		DebugLog:         func(format string, v ...interface{}) {},
		Output:           &outputBuffer,
	}

	myHelmClient, err := helmclient.New(opt)
	if err != nil {
		panic(err)
	}

	fmt.Println("Deploying nginx-ingress\n")
	nginxchartRepo := repo.Entry{
		Name: "nginx-ingress",
		URL:  "https://kubernetes.github.io/ingress-nginx",
	}

	if err := myHelmClient.AddOrUpdateChartRepo(nginxchartRepo); err != nil {
		log.Fatal(err)
	} else {
		fmt.Printf("Added Chart Repo %s\n", nginxchartRepo.Name)
	}

	if err := myHelmClient.UpdateChartRepos(); err != nil {
		log.Fatal(err)
	} else {
		fmt.Printf("Updating Chart Repo\n")
	}

	nginxchartSpec := helmclient.ChartSpec{
		ReleaseName:     "nginx-ingress/nginx-ingress",
		ChartName:       "nginx-ingress",
		Namespace:       "nginx-ingress",
		CreateNamespace: true,
		SkipCRDs:        false,
		Wait:            true,
		ValuesOptions: values.Options{
			StringValues: []string{
				"rbac.create=false",
				"controller.service.externalTrafficPolicy=Local",
				fmt.Sprintf("controller.service.loadBalancerIP=%v", mypublicip),
				"controller.replicaCount=2",
				"controller.nodeSelector.kubernetes\\.io/os=linux",
				"defaultBackend.nodeSelector.kubernetes\\.io/os=linux",
				"controller.admissionWebhooks.patch.nodeSelector.kubernetes\\.io/os=linux",
				"controller.publishService.enabled=true",
				"controller.service.beta.kubernetes.io/azure-load-balancer-health-probe-request-path=/healthz",
			},
		},
	}
	nginxInstalledHelmChart, err := myHelmClient.InstallChart(mycontext, &nginxchartSpec, nil)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("Status of Chart Install %v,\n", *nginxInstalledHelmChart.Info)

	fmt.Println("Deploying cert-manager\n")
	certmanagerchartRepo := repo.Entry{
		Name: "cert-manager",
		URL:  "https://charts.jetstack.io",
	}

	if err := myHelmClient.AddOrUpdateChartRepo(certmanagerchartRepo); err != nil {
		log.Fatal(err)
	} else {
		fmt.Printf("Added Chart Repo %s\n", certmanagerchartRepo.Name)
	}

	if err := myHelmClient.UpdateChartRepos(); err != nil {
		log.Fatal(err)
	} else {
		fmt.Printf("Updating Chart Repo\n")
	}

	certmanagerchartSpec := helmclient.ChartSpec{
		ReleaseName:     "cert-manager/cert-manager",
		ChartName:       "cert-manager",
		Version:         "v1.14.5",
		Namespace:       "cert-manager",
		CreateNamespace: true,
		ValuesOptions: values.Options{
			StringValues: []string{
				"extraArgs = {--dns01-recursive-nameservers=1.1.1.1:53}",
				"controller.nodeSelector.kubernetes\\.io/os=linux",
			},
		},
		SkipCRDs: false,
		Wait:     true,
	}
	certmanagermyInstalledHelmChart, err := myHelmClient.InstallChart(mycontext, &certmanagerchartSpec, nil)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Printf("Status of Chart Install %v,\n", *certmanagermyInstalledHelmChart.Info)

	// Further deployments omitted for brevity...

	fmt.Println("Finished Deployment\n")
}

func connectToK8s() *kubernetes.Clientset {
	home, exists := os.LookupEnv("HOME")
	if !exists {
		home = "C:\\Users\\your-user"
	}

	configPath := filepath.Join(home, ".kube", "config")

	config, err := clientcmd.BuildConfigFromFlags("", configPath)
	if err != nil {
		log.Panicln("Failed to create K8s config")
	}

	clientset, err := kubernetes.NewForConfig(config)
	if err != nil {
		log.Panicln("Failed to create K8s clientset")
	}

	return clientset
}

func connectToAks(ctx context.Context, subid string, rg string, aksname string) *armcontainerservice.ManagedClustersClientGetAccessProfileResponse {
	cred, err := azidentity.NewDefaultAzureCredential(nil)
	if err != nil {
		log.Fatalf("Failed to obtain a credential: %v", err)
	}

	clientFactory, err := armcontainerservice.NewClientFactory(subid, cred, nil)
	if err != nil {
		log.Fatalf("Failed to create client: %v", err)
	}
	res, err := clientFactory.NewManagedClustersClient().GetAccessProfile(ctx, rg, aksname, "clusterUser", nil)
	if err != nil {
		log.Fatalf("Failed to finish the request: %v", err)
	}
	return &res
}

func Ptr[T any](v T) *T {
	return &v
}
```

### The Challenges

1. **Dependency Bugs:** The biggest hurdle was a bug in the [`github.com/mittwald/go-helm-client`](http://github.com/mittwald/go-helm-client) package. Despite following the documentation and best practices, the Helm charts would not deploy correctly. I reported the issue here: [mittwald/go-helm-client#209](https://github.com/mittwald/go-helm-client/issues/209). This bug made deploying a working Helm chart impossible, derailing the project timeline.
    
2. **Time Sensitivity:** This project was highly time-sensitive. Every minute spent troubleshooting the bug was a minute closer to the deadline, adding to the stress and frustration.
    
3. **Cert-Manager Objects:** Deploying Kubernetes Cert-Manager objects using the native Kubernetes Go package was another pain point. Despite their theoretical simplicity, the objects did not deploy as expected, which was a significant setback, as Cert-Manager is critical for managing SSL/TLS certificates.
    

### Lessons Learned

1. **Dependency Management:** Always vet your dependencies thoroughly. Consider contingency plans for potential issues with third-party libraries in critical projects.
    
2. **Community and Support:** Don't hesitate to reach out to the community or report issues.
    
3. **Time Management:** Factor in potential delays when working with new or less-tested tools. Buffer time can help you better manage unexpected challenges.
    

### Conclusion

While this project was fraught with challenges, it was a valuable learning experience. I learned a ton about the internals of Kubernetes and Helm, which has improved my understanding of how to build Apps for Kubernetes. I also hope it gives you an insight into how it could be done.  
  
Note: Please don't use this code, as it currently does not work. Once I have fixed the bug in the helm package, I will write another blog post about it.