If you are a devops/cloud/platform engineer, developing CI CD pipelines at any scale is what you have encountered many times. However, for years the tools to support us writing pipeline logic as code are mostly based on YAML + Bash for GitHub Actions, GitLab CI, Harness, CircleCI,… and Groovy (for Jenkins)
So have you ever thought of writing pipeline logic using a programming language you prefer and run it anywhere? Look no further, Dagger tool is available to support you.
After using it for a while in my personal project, these are the top highlights of this tool:
Use top languages like Golang, Typescript/Javascript, Python. More to come in future
Commands are run on container, which means there is no OS ‘s library/tool/framework dependency
It’s very fast thanks to built-in caching mechanism
For simplicity, I will demonstrate how to use it using Golang. Please note that
Basic understanding of Docker, container, CI/CD, and any programming language is crucial before proceeding
Your local machine/cloud instances MUST have Docker and Golang 1.20 installed
Setup project and install Dagger
- Create an empty folder/clone a GitHub repo and open your terminal and run these commands
# Init go project
go mod init github.com/your-username/mynewci
# Install Dagger and dependencies
go get dagger.io/dagger@latest
go mod tidy
Create a simple CI pipeline
I will take Node.js microservice using Typescript as example to develop the pipeline
- Create a file named
ci.go
with the following content.
package main
import (
"context"
"fmt"
"dagger.io/dagger"
)
// Declare logic to run simple CI for your Node.js service
func Build(ctx *context.Context, client *dagger.Client) {
// Get host environment where this program will run on.
host := client.Host()
// Declare the host directory to mount directory to container
src := host.Directory(".", dagger.HostDirectoryOpts{
Exclude: []string{"node_modules/", "*.go"}, // Ignore some files and folders
})
_, err := client.
Pipeline("My first CI").
Container(). // Init container
From("node:18-alpine"). // Use Node.js 18 as base image
WithMountedDirectory("/src", src). // Mount current host dir to container
WithWorkdir("/src"). // Declare workdir
WithExec([]string{"yarn", "install"}). // install npm packages
WithExec([]string{"yarn", "build"}). // Build code (compile from TS to JS)
WithExec([]string{"yarn", "lint"}). // Check code linting
WithExec([]string{"yarn", "test"}). // Run unit test
Directory("./build").
Export(*ctx, "./build") // Capture generated build folder and mount back to host
if err != nil {
fmt.Println(err)
// Exit program if there's error in pipeline
panic(err)
}
}
- Create a file named
publish.go
with the following content.
package main
import (
"context"
"fmt"
"dagger.io/dagger"
)
// Build Docker image and publish to Docker Hub
func PublishImage(ctx *context.Context, client *dagger.Client) {
// Get host environment where this program will run on.
host := client.Host()
// Get environment variables of the host environment
username, _ := host.EnvVariable("USERNAME").Value(*ctx)
name, _ := host.EnvVariable("IMAGE_NAME").Value(*ctx)
tag, _ := host.EnvVariable("TAG").Value(*ctx)
imgPath := fmt.Sprintf("%s/%s:%s", username, name, tag)
_, err := client.
Pipeline("Publish to Docker Hub").
Host().
Directory(".").
DockerBuild(dagger.DirectoryDockerBuildOpts{
Dockerfile: "./Dockerfile.prod"
}). // Specify Dockerfile on host directory to use for building image
Publish(*ctx, imgPath) // Trigger `docker push` to push to Docker Hub
if err != nil {
fmt.Println(err)
panic(err)
}
}
Last thing to make this work is to declare main function and call functions above in there.
Let’s create
main.go
as below
package main
import (
"context"
"errors"
"os"
)
func initClient(ctx *context.Context) (*dagger.Client, error) {
// Init Dagger client with STDOUT log of all pipeline actions
client, err := dagger.Connect(*ctx, dagger.WithLogOutput(os.Stdout))
if err != nil {
return nil, err
}
return client, nil
}
func main() {
ctx := context.Background() // Declare context
client, err := initClient(&ctx)
if err != nil {
panic(err)
}
// Close Dagger client once finish
defer client.Close()
// Build code
Build(&ctx, client)
// Publish image to Docker Hub
PublishImage(&ctx, client)
}
At this stage, you should have code repo with structure like this
.
├── ci.go
├── go.mod
├── go.sum
├── main.go
└── publish.go
Tip!!! You can setup Makefile
to quickly run common commands like go mod tidy
or go build .
Build and run pipeline
go build .
# This is at the end of the module name when running "go mod init"
./mynewci
Voila! Once running the binary you will see logs of the pipeline running.
Integrate with CI CD tools
Dagger is NOT a replacement for CI CD tools you know like Jenkins, or GitHub Actions. It’s rather a tool to help move the logic of the pipeline away from those tools and allow you to “write once, run anywhere” with minimal setup.
To integrate with your existing tools, you need to have at least:
Docker
Golang/Node.js/Python installed on running agent
Necessary environment variables for your pipeline logic
Where to go from here
Dagger offers more features than those I just shared. Don’t forget to take a look at https://docs.dagger.io/ to find suitable functionality for your needs.
Dagger is still in beta and is subject to changes. But from what the Dagger team has done so far, it is impressive :)
Hope you find this post useful, and happy coding!!!
\Cover image is originally from [dagger.io/*](https://dagger.io/)