Resources Guide

In hash a resource is the main abstraction, where everything is a kind of a resource, just like in Linux kern everything is a file, in hash every thing is a resource.

Introduction

Resources are defined using yaml files and a bunch of other files based on their type, for example: terraform resources have a yaml file that defines this is a terraform resource along with a path for all the terraform files for this resource the same thing goes for Microservice resource as an example, we have a resource file and a bunch of source code files for the microservice.

Resources are implemented using plugins, there are some built-in resources defined in the resources package:

  • Env: This is a special resource that is used as a target for some actions on resources.

  • Terraform: This resource is used to manage terraform code in the mono repository, it creates a workspace, per environment, it creates outputs for every terraform output defined in the code, it also ensures that terraform is formatted correctly when testing the resource.

  • DockerImage: This resource is used to manage a single Docker Image defined in a docker file, it uses the DockerRegsitry target in an env to push images to the docker registry, it allows you to make sure that no image uses the latest tag in your docker file.

  • Kustomize: This resource is used to manage a bunch of k8s manifests defined using a kustomization file, it uses the K8S target from the env to deploy and test kubernetes manifests, when testing it uses dry run against the cluster.

  • GoMicroService: This resource represents a simple microservice written in go, it expects a docker file for the docker image, a k8s directory for k8s manifests to be present, it needs DockerRegsitry and K8S targets to operate.

These are some of the built-in resources, we can add our own resources as described resources plugin tutorial.

Environment Targets

An Env is a special resource, it is used by some resources when running actions on them, for example when we deploy a Kustomize resource we need to select a different k8s cluster for every deployment, the data about this cluster is stored in the K8S target for the environment, so we define a K8S target in environment development that connects to development cluster and the same thing for environment staging and production, these targets are defined in Env resource files as follows:

apiVersion: hash.io/v1alpha1
kind: Env
metadata:
    name: dev
depends_on:
    - id: Terraform:gke
      action1: deploy
      action2: deploy
spec:
    targets:
      - kind: K8STarget
        name: k8s
        spec:
          use_gcloud: true
          cluster_name: dev
          project_name: hashkern-demo
          cluster_region: europe-west3
      - kind: DockerRegistryTarget
        name: docker
        spec:
          registry_url: $Terraform:shared-gcp-tf.deploy.registry_url.value
          service_account: $Terraform:shared-gcp-tf.deploy.repo_writer_email.value

We can see an environment definition in the previous listing, the name of env is dev, it depends in an a Terraform resource called gke, which actually creates the kubernetes cluster, this means that if we try to deploy a kustomize resource here hash kern will ensure for us that the GKE cluster is created first and then deploy kustomize. The depends_on is a list of objects which contains the ID of the dependency, action1 is the action that we want to execute on current resource and action2 is the action that we must execute on the other resource, this means the following

To deploy environment of name dev, you have first to deploy the Terraform resource with the name gke.

The targets in spec defines our targets here, for K8S target we have its name as k8s this might be needed later in our Road Map, we have its specs, the specs here say that we have to use gcloud to authenticate to the cluster, in other cases we might have different specs for different clusters depending on how to authenticate to them.

We have another target of kind DockerRegistryTarget which has two keys in its specs, these keys are actually outputs from another Terraform resource, this automatically adds another dependency on this resource, the format of the value is as follows

$<Resource ID>.<action>.<output_name>

When we have this, its adds a dependency on running the desired action on the right resource.

Overriding resource specs using environments

We want to have different parameters and values per environment, for example if we have a terraform resource that creates a GKE cluster maybe we want to use different name, size, network for each environment, we can define variables to make this value different but how can we code this in the resource??

To achieve this we need to do two things, first enable developers to provide variables in the resource’s specs with values and then override these values in environments, lets look how to do this.

Lets say we have this terraform resource file

apiVersion: hash.io/v1alpha1
kind: Terraform
metadata:
    name: gke
spec:
    path: .
    match:
      - "*.tf"
    variables:
      gke_name: development

Now we ensure that the variable named gke_name will have the value development but how can we change it in staging? Ofcourse we are not going to edit the resource file every time we want to deploy to a different environment, that would be crazy we’re going to tell hash kern that when it tries to run an action on this resource, it must override the value for variables in its specs, this is how we do it.

apiVersion: hash.io/v1alpha1
kind: Env
metadata:
    name: development
spec:
    Terraform-gke:
        variables:
            gke_name: development

This is for development and for staging we have this

apiVersion: hash.io/v1alpha1
kind: Env
metadata:
    name: staging
spec:
    Terraform-gke:
        variables:
            gke_name: staging

And for the resource we have this:

apiVersion: hash.io/v1alpha1
kind: Terraform
metadata:
    name: gke
spec:
    path: .
    match:
      - "*.tf"

This means that when we run an action on resource gke in the development environment, it will use development as value for gke_name, a resource developer doesn’t need to take this into account, the value for variables key in the resource is automatically patched by hash kern before the resource’s action method is called.

The key Terraform-gke in the env’s specs means apply this patch to a resource called gke of type Teraform, if we had Terraform only then it means apply this patch to all Terraform resources.

Hash Templates

Templates is a unique feature for hash kern that allows developers to extend the functionality of the tools that they use on their own, templates are files that end with .hash extension, these files are rendered by hash kern before running an action on a resource, their rendered content will be filled by outputs from other resources or from artifacts, a template file is rendered as a Jinja2 template, here is an example of a template:

apiVersion: apps/v1
kind: Deployment
metadata:
name: new
spec:
selector:
    matchLabels:
    app: new
template:
    metadata:
    labels:
        app: new
    spec:
    containers:
    - name: new
        image: {{ artifacts.get_artifact("DockerImage:test", "publish", "image_url").getData() }}
        resources:
        limits:
            memory: "256Mi"
        requests:
            memory: "256Mi"
        envFrom:
        - secretRef:
            name: {{ globals.get("Terraform:secrets", "build", "secret_name").get("value") }}

This template represents a kubernetes deployment, it uses an artifact from a DockerImage resource and a gloabl output from a Terraform resource.

the line

image: {{ artifacts.get_artifact("DockerImage:test", "publish", "image_url").getData() }}

uses a Jinja2 template syntax to call the get_artifact method on the artifacts objects, this method takes three arguments, the first one is the resources’s ID, the second one is the action that must be called on this resource to get this output, and the last one is the output name, then we call getData on the resulting object to get the actual image URL.

this line

name: {{ globals.get("Terraform:secrets", "build", "secret_name").get("value") }}

uses also Jinja2 template syntax to call get method on globals object, this method takes theree arguments, the first is the resource’s ID, the second is the action that must be called on this resource to get the output and the last one is the output’s name, then we call get method on the resulting object because terraform outputs are actually objects with a value attribute.

These templates enable us to link resource’s together in ways that weren’t possible before, when we directly use kustomize, we will never get this feature that allows us to use for example terraform outputs in kustomize or use a docker image URL in kustomize that is an output from another resource, only templates enable us to do this.