Terraform and NREC: Part III - Dynamics¶
Last changed: 2024-04-17
This document builds on Terraform Part 1 and Part 2, and extends the code base to make it more dynamic. We also make use of more advanced functionality in Terraform such as output handling and local variables.
The goal with this document is to show how Terraform can be used to set up a real environment on NREC. We will create:
An SSH key pair
Four web servers running CentOS
One database server running Ubuntu
A volume that is attached to the database server
Security groups that allow access to the different servers, as well as allowing the web servers to access the database server
The files used in this document can be downloaded:
Variables file¶
In order to keep the management of the Terraform infrastructure easy and intuitive, it is a good idea to consolidate definitions and variables into a single file or a small set of files. Terraform supports a concept of default values for variables, which can be overridden. In our example, we are opting for a single file that contains all variables, with default values, used throughout the code:
1# Variables
2variable "region" {
3}
4
5variable "name" {
6 default = "myproject"
7}
8
9variable "ssh_public_key" {
10 default = "~/.ssh/id_rsa.pub"
11}
12
13variable "network" {
14 default = "IPv6"
15}
16
17variable "volume_size" {
18 default = 20
19}
20
21# Security group defaults
22variable "allow_ssh_from_v6" {
23 type = list(string)
24 default = []
25}
26
27variable "allow_ssh_from_v4" {
28 type = list(string)
29 default = []
30}
31
32variable "allow_http_from_v6" {
33 type = list(string)
34 default = []
35}
36
37variable "allow_http_from_v4" {
38 type = list(string)
39 default = []
40}
41
42variable "allow_mysql_from_v6" {
43 type = list(string)
44 default = []
45}
46
47variable "allow_mysql_from_v4" {
48 type = list(string)
49 default = []
50}
51
52# Mapping between role and image
53variable "role_image" {
54 type = map(string)
55 default = {
56 "web" = "GOLD CentOS 8"
57 "db" = "GOLD Ubuntu 21.04 LTS"
58 }
59}
60
61# Mapping between role and flavor
62variable "role_flavor" {
63 type = map(string)
64 default = {
65 "web" = "m1.small"
66 "db" = "m1.medium"
67 }
68}
69
70# Mapping between role and number of instances (count)
71variable "role_count" {
72 type = map(string)
73 default = {
74 "web" = 4
75 "db" = 1
76 }
77}
Notice that the region variable (highlighted) is empty and doesn’t have a default value. For this reason, the region must always be specified in some way when running Terraform:
$ ~/terraform plan
var.region
Enter a value:
As shown above, when a default value isn’t specified in the code Terraform will ask for it interactively.
Also note that the allow_ssh_from_v6, allow_ssh_from_v4
etc. (highlighted) variables are empty lists. It is expected that we
specify these in the terraform.tfvars
file, explained in the next
section.
Local variables file¶
Terraform supports specification of local variables that completes or overrides the variable set given in variables.tf. We can do this on command line:
terraform -var 'region=bgo'
This does not scale, however. Terraform has an option -var-file
that takes one argument, a variables file:
terraform -var-file <file>
An example file terraform.tfvars
that
complements our variables.tf could look like this:
1# Region
2region = "osl"
3
4# This is needed to access the instance over ssh
5allow_ssh_from_v6 = [
6 "2001:700:100:12::7/128",
7 "2001:700:100:118::67/128"
8]
9allow_ssh_from_v4 = [
10 "129.240.12.7/32",
11 "129.240.118.67/32"
12]
13
14# This is needed to access the instance over http
15allow_http_from_v6 = [
16 "2001:700:200::/48",
17 "2001:700:100::/41"
18]
19allow_http_from_v4 = [
20 "129.177.0.0/16",
21 "129.240.0.0/16"
22]
23
24# This is needed to access the instance over the mysql port
25allow_mysql_from_v6 = [
26 "2001:700:100:118::67/128"
27]
28allow_mysql_from_v4 = [
29 "129.240.118.67/32"
30]
Here, we specify the region and the addresses to be used for the security group. Since this file is named terraform.tfvars it will be automatically included when running terraform commands. If we were to name it as e.g. production.tfvars, we would need to specify which file to use on the command line, like this:
$ terraform plan -var-file production.tfvars
$ terraform apply -var-file production.tfvars
Read more about variables here: Terraform variables
Using variables¶
Terraform supports a variety of different variable types, and should be familiar to anyone who has used modern programming languages. We’re using string, list (array) and map (hash) variables. In this example, we have divided our original one-file setup into 3 files, in addition to the local variables file:
main.tf |
Our main file. |
secgroup.tf |
Since the security group definitions are rather verbose, we have separated these from the main file. |
variables.tf |
Variable definitions with default values. |
terraform.tfvars |
Local variables. |
We’ll take a look at main.tf. The first part, containing the SSH key pair resource, is as before but using variables:
1# SSH key
2resource "openstack_compute_keypair_v2" "keypair" {
3 region = var.region
4 name = "${terraform.workspace}-${var.name}"
5 public_key = file(var.ssh_public_key)
6}
Next, we’ll look at our security groups in secgroup.tf. We now have three of them:
1# Security group SSH + ICMP
2resource "openstack_networking_secgroup_v2" "instance_ssh_access" {
3 region = var.region
4 name = "${terraform.workspace}-${var.name}-ssh"
5 description = "Security group for allowing SSH and ICMP access"
6}
7
8# Security group HTTP
9resource "openstack_networking_secgroup_v2" "instance_web_access" {
10 region = var.region
11 name = "${terraform.workspace}-${var.name}-web"
12 description = "Security group for allowing HTTP access"
13}
14
15# Security group MySQL
16resource "openstack_networking_secgroup_v2" "instance_db_access" {
17 region = var.region
18 name = "${terraform.workspace}-${var.name}-db"
19 description = "Security group for allowing MySQL access"
20}
Since these are web and database serves, we create a security group for allowing HTTP for the web servers and port 3306 for the database server, in addition to allowing SSH and ICMP. The security group rules for SSH and ICMP are pretty much the same as before, but using variables:
1# Allow ssh from IPv4 net
2resource "openstack_networking_secgroup_rule_v2" "rule_ssh_access_ipv4" {
3 region = var.region
4 count = length(var.allow_ssh_from_v4)
5 direction = "ingress"
6 ethertype = "IPv4"
7 protocol = "tcp"
8 port_range_min = 22
9 port_range_max = 22
10 remote_ip_prefix = var.allow_ssh_from_v4[count.index]
11 security_group_id = openstack_networking_secgroup_v2.instance_ssh_access.id
12}
13
14# Allow ssh from IPv6 net
15resource "openstack_networking_secgroup_rule_v2" "rule_ssh_access_ipv6" {
16 region = var.region
17 count = length(var.allow_ssh_from_v6)
18 direction = "ingress"
19 ethertype = "IPv6"
20 protocol = "tcp"
21 port_range_min = 22
22 port_range_max = 22
23 remote_ip_prefix = var.allow_ssh_from_v6[count.index]
24 security_group_id = openstack_networking_secgroup_v2.instance_ssh_access.id
25}
26
27# Allow icmp from IPv4 net
28resource "openstack_networking_secgroup_rule_v2" "rule_icmp_access_ipv4" {
29 region = var.region
30 count = length(var.allow_ssh_from_v4)
31 direction = "ingress"
32 ethertype = "IPv4"
33 protocol = "icmp"
34 remote_ip_prefix = var.allow_ssh_from_v4[count.index]
35 security_group_id = openstack_networking_secgroup_v2.instance_ssh_access.id
36}
37
38# Allow icmp from IPv6 net
39resource "openstack_networking_secgroup_rule_v2" "rule_icmp_access_ipv6" {
40 region = var.region
41 count = length(var.allow_ssh_from_v6)
42 direction = "ingress"
43 ethertype = "IPv6"
44 protocol = "ipv6-icmp"
45 remote_ip_prefix = var.allow_ssh_from_v6[count.index]
46 security_group_id = openstack_networking_secgroup_v2.instance_ssh_access.id
47}
Notice that we now use implicit iteration over the number of entries listed in the “allow_from” variables, which are empty lists in variables.tf but are properly defined in terraform.tfvars.
Let’s take a look at the security group rules defined for HTTP and MySQL access:
1# Allow HTTP from IPv4 net
2resource "openstack_networking_secgroup_rule_v2" "rule_http_access_ipv4" {
3 region = var.region
4 count = length(var.allow_http_from_v4)
5 direction = "ingress"
6 ethertype = "IPv4"
7 protocol = "tcp"
8 port_range_min = 80
9 port_range_max = 80
10 remote_ip_prefix = var.allow_http_from_v4[count.index]
11 security_group_id = openstack_networking_secgroup_v2.instance_web_access.id
12}
13
14# Allow HTTP from IPv6 net
15resource "openstack_networking_secgroup_rule_v2" "rule_http_access_ipv6" {
16 region = var.region
17 count = length(var.allow_http_from_v6)
18 direction = "ingress"
19 ethertype = "IPv6"
20 protocol = "tcp"
21 port_range_min = 80
22 port_range_max = 80
23 remote_ip_prefix = var.allow_http_from_v6[count.index]
24 security_group_id = openstack_networking_secgroup_v2.instance_web_access.id
25}
26
27# Allow MySQL from IPv4 net
28resource "openstack_networking_secgroup_rule_v2" "rule_mysql_access_ipv4" {
29 region = var.region
30 count = length(var.allow_mysql_from_v4)
31 direction = "ingress"
32 ethertype = "IPv4"
33 protocol = "tcp"
34 port_range_min = 3306
35 port_range_max = 3306
36 remote_ip_prefix = var.allow_mysql_from_v4[count.index]
37 security_group_id = openstack_networking_secgroup_v2.instance_db_access.id
38}
39
40# Allow MYSQL from IPv6 net
41resource "openstack_networking_secgroup_rule_v2" "rule_mysql_access_ipv6" {
42 region = var.region
43 count = length(var.allow_mysql_from_v6)
44 direction = "ingress"
45 ethertype = "IPv6"
46 protocol = "tcp"
47 port_range_min = 3306
48 port_range_max = 3306
49 remote_ip_prefix = var.allow_mysql_from_v6[count.index]
50 security_group_id = openstack_networking_secgroup_v2.instance_db_access.id
51}
The resource definition for the HTTP access, as well as the first two resource definitions for MySQL access, follows the same logic as that of the SSH and ICMP rules. The last two MySQL rules are different:
1# Allow MYSQL from web servers (IPv4)
2resource "openstack_networking_secgroup_rule_v2" "rule_mysql_from_web_access_ipv4" {
3 region = var.region
4 count = 1
5 direction = "ingress"
6 ethertype = "IPv4"
7 protocol = "tcp"
8 port_range_min = 3306
9 port_range_max = 3306
10 remote_group_id = openstack_networking_secgroup_v2.instance_web_access.id
11 security_group_id = openstack_networking_secgroup_v2.instance_db_access.id
12}
13
14# Allow MYSQL from web servers (IPv6)
15resource "openstack_networking_secgroup_rule_v2" "rule_mysql_from_web_access_ipv6" {
16 region = var.region
17 count = 1
18 direction = "ingress"
19 ethertype = "IPv6"
20 protocol = "tcp"
21 port_range_min = 3306
22 port_range_max = 3306
23 remote_group_id = openstack_networking_secgroup_v2.instance_web_access.id
24 security_group_id = openstack_networking_secgroup_v2.instance_db_access.id
25}
Here, we use rather advanced functionality for security groups in
Openstack. We can allow IP addresses from other security groups
(source groups) access by specifying remote_group_id
rather than
remote_ip_prefix
. It is possible to achieve the same using
remote_ip_prefix
, however it is less elegant [1].
We’ll circle back to main.tf:
1# Web servers
2resource "openstack_compute_instance_v2" "web_instance" {
3 region = var.region
4 count = lookup(var.role_count, "web", 0)
5 name = "${var.region}-web-${count.index}"
6 image_name = lookup(var.role_image, "web", "unknown")
7 flavor_name = lookup(var.role_flavor, "web", "unknown")
8
9 key_pair = "${terraform.workspace}-${var.name}"
10 security_groups = [
11 "default",
12 "${terraform.workspace}-${var.name}-ssh",
13 "${terraform.workspace}-${var.name}-web",
14 ]
15
16 network {
17 name = "IPv6"
18 }
19
20 lifecycle {
21 ignore_changes = [image_name,image_id]
22 }
23
24 depends_on = [
25 openstack_networking_secgroup_v2.instance_ssh_access,
26 openstack_networking_secgroup_v2.instance_web_access,
27 ]
28}
29
30# Database servers
31resource "openstack_compute_instance_v2" "db_instance" {
32 region = var.region
33 count = lookup(var.role_count, "db", 0)
34 name = "${var.region}-db-${count.index}"
35 image_name = lookup(var.role_image, "db", "unknown")
36 flavor_name = lookup(var.role_flavor, "db", "unknown")
37
38 key_pair = "${terraform.workspace}-${var.name}"
39 security_groups = [
40 "default",
41 "${terraform.workspace}-${var.name}-ssh",
42 "${terraform.workspace}-${var.name}-db",
43 ]
44
45 network {
46 name = "IPv6"
47 }
48
49 lifecycle {
50 ignore_changes = [image_name,image_id]
51 }
52
53 depends_on = [
54 openstack_networking_secgroup_v2.instance_ssh_access,
55 openstack_networking_secgroup_v2.instance_db_access,
56 ]
57}
We now define two different instance resources. One for web servers and one for the database server. They use different values defined in variables.tf for image, flavor etc. Lastly, we define a volume resource and attach this volume to the database server:
1# Volume
2resource "openstack_blockstorage_volume_v2" "volume" {
3 name = "database"
4 size = var.volume_size
5}
6
7# Attach volume
8resource "openstack_compute_volume_attach_v2" "attach_vol" {
9 instance_id = openstack_compute_instance_v2.db_instance[0].id
10 volume_id = openstack_blockstorage_volume_v2.volume.id
11}
Making changes¶
Terraform maintains the state of the infrastructure it manages in the workspace directory. It is possible to make simple changes just by updating and applying the code. If we wanted to scale down the number of web servers from 4 to 2, we would change this line in variables.tf:
1# Mapping between role and number of instances (count)
2variable "role_count" {
3 type = map(string)
4 default = {
5 "web" = 4
6 "db" = 1
7 }
8}
After changing the count from 4 to 2 here (the highlighted
line), we can run terraform plan
:
$ terraform plan
...
Plan: 0 to add, 0 to change, 2 to destroy.
...
Applying this with terraform apply
will then destroy two of the
web servers. Similarly, if we were to increase the web server count
from 4 to 5, Terraform would add a new web server.
Complete example¶
A complete listing of the example files used in this document is provided below.
1# Define required providers
2terraform {
3 required_version = ">= 1.0"
4 required_providers {
5 openstack = {
6 source = "terraform-provider-openstack/openstack"
7 }
8 }
9}
10
11# Configure the OpenStack Provider
12# Empty means using environment variables "OS_*". More info:
13# https://registry.terraform.io/providers/terraform-provider-openstack/openstack/latest/docs
14provider "openstack" {}
15
16# SSH key
17resource "openstack_compute_keypair_v2" "keypair" {
18 region = var.region
19 name = "${terraform.workspace}-${var.name}"
20 public_key = file(var.ssh_public_key)
21}
22
23# Web servers
24resource "openstack_compute_instance_v2" "web_instance" {
25 region = var.region
26 count = lookup(var.role_count, "web", 0)
27 name = "${var.region}-web-${count.index}"
28 image_name = lookup(var.role_image, "web", "unknown")
29 flavor_name = lookup(var.role_flavor, "web", "unknown")
30
31 key_pair = "${terraform.workspace}-${var.name}"
32 security_groups = [
33 "default",
34 "${terraform.workspace}-${var.name}-ssh",
35 "${terraform.workspace}-${var.name}-web",
36 ]
37
38 network {
39 name = "IPv6"
40 }
41
42 lifecycle {
43 ignore_changes = [image_name,image_id]
44 }
45
46 depends_on = [
47 openstack_networking_secgroup_v2.instance_ssh_access,
48 openstack_networking_secgroup_v2.instance_web_access,
49 ]
50}
51
52# Database servers
53resource "openstack_compute_instance_v2" "db_instance" {
54 region = var.region
55 count = lookup(var.role_count, "db", 0)
56 name = "${var.region}-db-${count.index}"
57 image_name = lookup(var.role_image, "db", "unknown")
58 flavor_name = lookup(var.role_flavor, "db", "unknown")
59
60 key_pair = "${terraform.workspace}-${var.name}"
61 security_groups = [
62 "default",
63 "${terraform.workspace}-${var.name}-ssh",
64 "${terraform.workspace}-${var.name}-db",
65 ]
66
67 network {
68 name = "IPv6"
69 }
70
71 lifecycle {
72 ignore_changes = [image_name,image_id]
73 }
74
75 depends_on = [
76 openstack_networking_secgroup_v2.instance_ssh_access,
77 openstack_networking_secgroup_v2.instance_db_access,
78 ]
79}
80
81# Volume
82resource "openstack_blockstorage_volume_v2" "volume" {
83 name = "database"
84 size = var.volume_size
85}
86
87# Attach volume
88resource "openstack_compute_volume_attach_v2" "attach_vol" {
89 instance_id = openstack_compute_instance_v2.db_instance[0].id
90 volume_id = openstack_blockstorage_volume_v2.volume.id
91}
1# Security group SSH + ICMP
2resource "openstack_networking_secgroup_v2" "instance_ssh_access" {
3 region = var.region
4 name = "${terraform.workspace}-${var.name}-ssh"
5 description = "Security group for allowing SSH and ICMP access"
6}
7
8# Security group HTTP
9resource "openstack_networking_secgroup_v2" "instance_web_access" {
10 region = var.region
11 name = "${terraform.workspace}-${var.name}-web"
12 description = "Security group for allowing HTTP access"
13}
14
15# Security group MySQL
16resource "openstack_networking_secgroup_v2" "instance_db_access" {
17 region = var.region
18 name = "${terraform.workspace}-${var.name}-db"
19 description = "Security group for allowing MySQL access"
20}
21
22# Allow ssh from IPv4 net
23resource "openstack_networking_secgroup_rule_v2" "rule_ssh_access_ipv4" {
24 region = var.region
25 count = length(var.allow_ssh_from_v4)
26 direction = "ingress"
27 ethertype = "IPv4"
28 protocol = "tcp"
29 port_range_min = 22
30 port_range_max = 22
31 remote_ip_prefix = var.allow_ssh_from_v4[count.index]
32 security_group_id = openstack_networking_secgroup_v2.instance_ssh_access.id
33}
34
35# Allow ssh from IPv6 net
36resource "openstack_networking_secgroup_rule_v2" "rule_ssh_access_ipv6" {
37 region = var.region
38 count = length(var.allow_ssh_from_v6)
39 direction = "ingress"
40 ethertype = "IPv6"
41 protocol = "tcp"
42 port_range_min = 22
43 port_range_max = 22
44 remote_ip_prefix = var.allow_ssh_from_v6[count.index]
45 security_group_id = openstack_networking_secgroup_v2.instance_ssh_access.id
46}
47
48# Allow icmp from IPv4 net
49resource "openstack_networking_secgroup_rule_v2" "rule_icmp_access_ipv4" {
50 region = var.region
51 count = length(var.allow_ssh_from_v4)
52 direction = "ingress"
53 ethertype = "IPv4"
54 protocol = "icmp"
55 remote_ip_prefix = var.allow_ssh_from_v4[count.index]
56 security_group_id = openstack_networking_secgroup_v2.instance_ssh_access.id
57}
58
59# Allow icmp from IPv6 net
60resource "openstack_networking_secgroup_rule_v2" "rule_icmp_access_ipv6" {
61 region = var.region
62 count = length(var.allow_ssh_from_v6)
63 direction = "ingress"
64 ethertype = "IPv6"
65 protocol = "ipv6-icmp"
66 remote_ip_prefix = var.allow_ssh_from_v6[count.index]
67 security_group_id = openstack_networking_secgroup_v2.instance_ssh_access.id
68}
69
70# Allow HTTP from IPv4 net
71resource "openstack_networking_secgroup_rule_v2" "rule_http_access_ipv4" {
72 region = var.region
73 count = length(var.allow_http_from_v4)
74 direction = "ingress"
75 ethertype = "IPv4"
76 protocol = "tcp"
77 port_range_min = 80
78 port_range_max = 80
79 remote_ip_prefix = var.allow_http_from_v4[count.index]
80 security_group_id = openstack_networking_secgroup_v2.instance_web_access.id
81}
82
83# Allow HTTP from IPv6 net
84resource "openstack_networking_secgroup_rule_v2" "rule_http_access_ipv6" {
85 region = var.region
86 count = length(var.allow_http_from_v6)
87 direction = "ingress"
88 ethertype = "IPv6"
89 protocol = "tcp"
90 port_range_min = 80
91 port_range_max = 80
92 remote_ip_prefix = var.allow_http_from_v6[count.index]
93 security_group_id = openstack_networking_secgroup_v2.instance_web_access.id
94}
95
96# Allow MySQL from IPv4 net
97resource "openstack_networking_secgroup_rule_v2" "rule_mysql_access_ipv4" {
98 region = var.region
99 count = length(var.allow_mysql_from_v4)
100 direction = "ingress"
101 ethertype = "IPv4"
102 protocol = "tcp"
103 port_range_min = 3306
104 port_range_max = 3306
105 remote_ip_prefix = var.allow_mysql_from_v4[count.index]
106 security_group_id = openstack_networking_secgroup_v2.instance_db_access.id
107}
108
109# Allow MYSQL from IPv6 net
110resource "openstack_networking_secgroup_rule_v2" "rule_mysql_access_ipv6" {
111 region = var.region
112 count = length(var.allow_mysql_from_v6)
113 direction = "ingress"
114 ethertype = "IPv6"
115 protocol = "tcp"
116 port_range_min = 3306
117 port_range_max = 3306
118 remote_ip_prefix = var.allow_mysql_from_v6[count.index]
119 security_group_id = openstack_networking_secgroup_v2.instance_db_access.id
120}
121
122# Allow MYSQL from web servers (IPv4)
123resource "openstack_networking_secgroup_rule_v2" "rule_mysql_from_web_access_ipv4" {
124 region = var.region
125 count = 1
126 direction = "ingress"
127 ethertype = "IPv4"
128 protocol = "tcp"
129 port_range_min = 3306
130 port_range_max = 3306
131 remote_group_id = openstack_networking_secgroup_v2.instance_web_access.id
132 security_group_id = openstack_networking_secgroup_v2.instance_db_access.id
133}
134
135# Allow MYSQL from web servers (IPv6)
136resource "openstack_networking_secgroup_rule_v2" "rule_mysql_from_web_access_ipv6" {
137 region = var.region
138 count = 1
139 direction = "ingress"
140 ethertype = "IPv6"
141 protocol = "tcp"
142 port_range_min = 3306
143 port_range_max = 3306
144 remote_group_id = openstack_networking_secgroup_v2.instance_web_access.id
145 security_group_id = openstack_networking_secgroup_v2.instance_db_access.id
146}
1# Variables
2variable "region" {
3}
4
5variable "name" {
6 default = "myproject"
7}
8
9variable "ssh_public_key" {
10 default = "~/.ssh/id_rsa.pub"
11}
12
13variable "network" {
14 default = "IPv6"
15}
16
17variable "volume_size" {
18 default = 20
19}
20
21# Security group defaults
22variable "allow_ssh_from_v6" {
23 type = list(string)
24 default = []
25}
26
27variable "allow_ssh_from_v4" {
28 type = list(string)
29 default = []
30}
31
32variable "allow_http_from_v6" {
33 type = list(string)
34 default = []
35}
36
37variable "allow_http_from_v4" {
38 type = list(string)
39 default = []
40}
41
42variable "allow_mysql_from_v6" {
43 type = list(string)
44 default = []
45}
46
47variable "allow_mysql_from_v4" {
48 type = list(string)
49 default = []
50}
51
52# Mapping between role and image
53variable "role_image" {
54 type = map(string)
55 default = {
56 "web" = "GOLD CentOS 8"
57 "db" = "GOLD Ubuntu 21.04 LTS"
58 }
59}
60
61# Mapping between role and flavor
62variable "role_flavor" {
63 type = map(string)
64 default = {
65 "web" = "m1.small"
66 "db" = "m1.medium"
67 }
68}
69
70# Mapping between role and number of instances (count)
71variable "role_count" {
72 type = map(string)
73 default = {
74 "web" = 4
75 "db" = 1
76 }
77}
Footnotes