Introduction to Pulumi

November 2, 2020

Pulumi is an open-source multi-language and multi-cloud development platform, which allows you to manage your whole cloud infrastructure, like VMs, networking, databases, modern architectures, serverless functions, and containers, just by coding.

‍

Using Pulumi you will be able to:

Create modern applications:
You can use programming languages you already know, as long as they are supported by Pulumi, using the IDEs and tools you are used to work with.
Deploy to any cloud with any CI/CD tool:
No matter on what cloud you are working on, you still can use your favorite language and tools. Developers and system operators can collaborate to get DevOps best practices, making it really easy to integrate Pulumi with any tool.
Manage cloud environments:
Have visibility of every environment that you have, knowing who changed what. You can easily set up environments configurations to keep your secrets safe.

‍

Infrastructure as Code

Infrastructure as code (sometimes referred to as programmable or software-defined infrastructure) is a very important DevOps practice for every project, its objective is to make the process of achieving CI/CD more maintainable and scalable. By writing high-level code to deploy, update, and destroy cloud infrastructure, it becomes really easy to maintain and scale if you follow good practices. It isn’t the same to have to create new services manually in your cloud console, that calling a function one more time, or adding some more lines of code.

As the infrastructure is also code, it can go through the same version control, automated testing process, and different steps of a continuous integration and continuous delivery (CI/CD) pipeline that developers use for their application code.

We live in a world where everything tends to be automated, having that in mind, designing code that can be run automatically every time something happens, or in a determined time, is a great idea. You can schedule your code to deploy the whole infrastructure early in the morning, so you have it available for the day, and in the evening, when you aren’t using any resource, destroy cloud resources so you don’t waste money.

Learning how to deploy infrastructure as code might need some training, but when you understand how it works, and get familiar with it, it becomes a really useful tool for your projects.

‍

Architecture & Concepts

Pulumi is a modern infrastructure as code platform. It includes a CLI, libraries, and a hosted service that, working together, deliver a robust way of provisioning, updating, and managing cloud infrastructure.

Pulumi has its own programming model to deploy infrastructure, it manages different resources using the Pulumi SDK, this programming model and SDK introduces new concepts, such as programs, projects, and stacks to manage the resources’ deployment, and inputs and outputs to manage dependencies between resources, that you have to understand and use in order to make your deploy successful.

‍

Programming model

The Pulumi’s programming model contains core concepts that you need to know to write your infrastructure as code.

A Pulumi project is conformed by three components:

‍

Project

A project is a directory that includes a program and a .yaml file with configurations that lets Pulumi know how to run the program.

The Pulumi.yaml file contains important metadata such as the name and a description of the project, the programming language chosen, and more optional configurations.

‍

Program

A program is a collection of files that the project uses to run. These files are written in the programming language that is defined in the Pulumi.yaml file.

As Pulumi programs are written in familiar languages, you can use their natives libraries, but in order to deploy your infrastructure, you will also need to use Pulumi’s packages and their resource types. You can set your program to generate outputs that will be available in the stack that is running it.

When you run your program using the Pulumi CLI, you will see a preview of the resources that are going to be created, updated, or destroyed, depending on your code and their current status.

‍

Stacks

A stack is an instance of a project, you can have as many stacks as you want, they are usually used for different cloud environments, stages, or branches. Each stack has its own Pulumi.<stack>.yaml file, so it can be configured independently.

You can do basic operations like create, select, and destroy stacks using the CLI, also, it provides you more specific features such as listing, tagging, importing and exporting stacks, and viewing their outputs.

‍

Best practices

There are some best practices and different approaches when organizing your code when using Pulumi. Namely, we could have a Monolithic or a Micro-stacks approach.

‍

Monolithic

It’s very common to start using Pulumi with a monolithic approach, where a single project defines the infrastructure and application resources for an entire vertical service including all the different stacks used for different environments (ex., production, staging, testing).

These are some benefits of using a monolithic approach:

Simplicity: Having a simple project and a collection of stacks is the simplest thing you can do.
Versioning: Having all code in one project to version control your project.
Agility: All of the above means that using a monolithic approach will almost always lead to the best productivity and therefore agility. For small projects or teams, this is usually the right place to start.

Micro-stacks

Using a monolithic structure is where most users begin, but in turn, choosing a finer-grained decomposition of projects and stacks brings its own advantages.

Micro-stacks is a pattern where the project is broken into separately managed smaller projects. For example, defining the cluster infrastructure in one project and consume it from another. It is possible to even break it in more projects for different environments, said production, staging, and testing.

This approach raises the question of managing dependencies between your stacks. To manage inter-stack dependencies, Pulumi uses stack exports, where in one stack you export a service output property and in another one you consume it.

Here are a few examples of how to split up a monolithic project:

In a microservice architecture, each one gets its own project
Application container images may be rebuilt and published independently of infrastructure projects.
Core, low-level infrastructure (networks, cluster orchestrators) may be independent of other infrastructure and application resources.

Let’s see the advantages of this approach:

Independence: Certain projects deploy at different cadences. For example, a service that updates every day may not be appropriate to live in the same project that a critical infrastructure that changes infrequently.
Security: You can ensure that only allowed personal approve changes to critical infrastructure.
Complexity and Performance: Putting all services in one place may increase build times. Breaking apart pieces that can be built independently can increase agility and improve performance, particularly when they evolve at different rates and/or are managed by different teams.

‍

How it works

Pulumi uses states to define what operation should be done to each resource.

The Pulumi state model is based on four different components:

Language host: The language that you used to write your infrastructure as code runs the Pulumi program and makes a registration request for each resource that has been declared.
Resource providers: SDKs that use plugins for different cloud providers to deploy resources to the cloud.
States: Each resource has a current state and desired state, which can be changed when Pulumi performs operations in them.
Deployment engine: Pulumi uses a deployment engine to connect the previous components and decide what to deploy to the cloud.

‍

‍

You can interact in different ways with a project, and Pulumi will act according to that:

Deploy: Each resource is created with its name, some properties, and its current and desired state.
Update: Each already created resource properties and desired state can be changed, if needed, Pulumi will update/destroy the resource, if not, it will stay the same. If there are new resources that haven’t been created before, they will be created now.
Destroy: Each already created resource desired state will change to “deleted”, and Pulumi will delete every resource.

Basically, the Pulumi deployment engine does the following operations for each resource that has to be deployed:

Receives a resource registration request from the language host.
Checks if the resource depends on another resource that hasn’t been created yet.
Checks the current state of the resource.
Compares the current state with the desired one, and decide what operation should the resource provider run:

If the resource hasn’t been created yet:

Create it.

If the resource has been created:

If the current state is the same as the desired one, doesn’t do anything.
If the current state is different from the desired one, updates/delete it, so it reaches its desired state.

5. Updates the resource’s state.

For a component it works the same way, but the language host will make multiple resources registration requests, and the children will run when the father’s current state equals it’s desired one.

You can have a deeper insight into how Pulumi works reading the second part of this publication →

‍

Start deploying your infrastructure

If you have come all the way down here, you’re probably wondering “How do I start deploying my infrastructure using Pulumi?”.

The first thing you should do, if you haven’t already, is choosing the cloud provider and language that you are going to use. With that decision already made, the Pulumi getting started documentation will guide you to install Pulumi and configure the cloud provider and language you chose before, after that, it will show you how to create and deploy a basic project, also, you’ll be able to destroy the whole stack.

Once you have a basic project deployed, it’s time to dig into the documentation and start developing your own solutions. Check the API reference to deploy different services to the cloud.

‍