diff --git a/website/source/docs/configuration/from-1.5/expressions.html.md b/website/source/docs/configuration/from-1.5/expressions.html.md new file mode 100644 index 000000000..8c2069a26 --- /dev/null +++ b/website/source/docs/configuration/from-1.5/expressions.html.md @@ -0,0 +1,222 @@ +--- +layout: "docs" +page_title: "Expressions - Configuration Language" +sidebar_current: configuration-expressions +description: |- + HCL allows the use of expressions to access data exported + by resources and to transform and combine that data to produce other values. +--- + +# Expressions + +_Expressions_ are used to refer to or compute values within a configuration. +The simplest expressions are just literal values, like `"hello"` or `5`, but +HCL also allows more complex expressions such as references to data exported by +resources, arithmetic, conditional evaluation, and a number of built-in +functions. + +Expressions can be used in a number of places in HCL, but some contexts limit +which expression constructs are allowed, such as requiring a literal value of a +particular type or forbidding. Each language feature's documentation describes +any restrictions it places on expressions. + +The rest of this page describes all of the features of Packer's +expression syntax. + +## Types and Values + +The result of an expression is a _value_. All values have a _type_, which +dictates where that value can be used and what transformations can be +applied to it. + +HCL uses the following types for its values: + +* `string`: a sequence of Unicode characters representing some text, like + `"hello"`. +* `number`: a numeric value. The `number` type can represent both whole + numbers like `15` and fractional values like `6.283185`. +* `bool`: either `true` or `false`. `bool` values can be used in conditional + logic. +* `list` (or `tuple`): a sequence of values, like + `["us-west-1a", "us-west-1c"]`. Elements in a list or tuple are identified by + consecutive whole numbers, starting with zero. +* `map` (or `object`): a group of values identified by named labels, like + `{name = "Mabel", age = 52}`. + +Strings, numbers, and bools are sometimes called _primitive types._ +Lists/tuples and maps/objects are sometimes called _complex types,_ _structural +types,_ or _collection types._ + +Finally, there is one special value that has _no_ type: + +* `null`: a value that represents _absence_ or _omission._ If you set an + argument of a source or module to `null`, Packer behaves as though you + had completely omitted it — it will use the argument's default value if it has + one, or raise an error if the argument is mandatory. `null` is most useful in + conditional expressions, so you can dynamically omit an argument if a + condition isn't met. + +### Advanced Type Details + +In most situations, lists and tuples behave identically, as do maps and objects. +Whenever the distinction isn't relevant, the Packer documentation uses each +pair of terms interchangeably (with a historical preference for "list" and +"map"). + +However, module authors and provider developers should understand the +differences between these similar types (and the related `set` type), since they +offer different ways to restrict the allowed values for input variables and +source arguments. + +### Type Conversion + +Expressions are most often used to set values for the arguments of resources and +child modules. In these cases, the argument has an expected type and the given +expression must produce a value of that type. + +Where possible, Packer automatically converts values from one type to +another in order to produce the expected type. If this isn't possible, Packer +will produce a type mismatch error and you must update the configuration with a +more suitable expression. + +Packer automatically converts number and bool values to strings when needed. +It also converts strings to numbers or bools, as long as the string contains a +valid representation of a number or bool value. + +* `true` converts to `"true"`, and vice-versa +* `false` converts to `"false"`, and vice-versa +* `15` converts to `"15"`, and vice-versa + +## Literal Expressions + +A _literal expression_ is an expression that directly represents a particular +constant value. Packer has a literal expression syntax for each of the value +types described above: + +* Strings are usually represented by a double-quoted sequence of Unicode + characters, `"like this"`. There is also a "heredoc" syntax for more complex + strings. String literals are the most complex kind of literal expression in + Packer, and have additional documentation on this page: + * See [String Literals](#string-literals) below for information about escape + sequences and the heredoc syntax. + * See [String Templates](#string-templates) below for information about + interpolation and template directives. +* Numbers are represented by unquoted sequences of digits with or without a + decimal point, like `15` or `6.283185`. +* Bools are represented by the unquoted symbols `true` and `false`. +* The null value is represented by the unquoted symbol `null`. +* Lists/tuples are represented by a pair of square brackets containing a + comma-separated sequence of values, like `["a", 15, true]`. + + List literals can be split into multiple lines for readability, but always + require a comma between values. A comma after the final value is allowed, + but not required. Values in a list can be arbitrary expressions. +* Maps/objects are represented by a pair of curly braces containing a series of + ` = ` pairs: + + ```hcl + { + name = "John" + age = 52 + } + ``` + + Key/value pairs can be separated by either a comma or a line break. Values + can be arbitrary expressions. Keys are strings; they can be left unquoted if + they are a valid [identifier](./syntax.html#identifiers), but must be quoted + otherwise. You can use a non-literal expression as a key by wrapping it in + parentheses, like `(var.business_unit_tag_name) = "SRE"`. + +## References to Named Values + +Packer makes one named values available. + +The following named values are available: + +* `source..` is an object representing a + [source](./sources.html) of the given type + and name. + +## String Literals + +HCL has two different syntaxes for string literals. The +most common is to delimit the string with quote characters (`"`), like +`"hello"`. In quoted strings, the backslash character serves as an escape +sequence, with the following characters selecting the escape behavior: + +| Sequence | Replacement | +| ------------ | ----------------------------------------------------------------------------- | +| `\n` | Newline | +| `\r` | Carriage Return | +| `\t` | Tab | +| `\"` | Literal quote (without terminating the string) | +| `\\` | Literal backslash | +| `\uNNNN` | Unicode character from the basic multilingual plane (NNNN is four hex digits) | +| `\UNNNNNNNN` | Unicode character from supplementary planes (NNNNNNNN is eight hex digits) | + +The alternative syntax for string literals is the so-called Here Documents or +"heredoc" style, inspired by Unix shell languages. This style allows multi-line +strings to be expressed more clearly by using a custom delimiter word on a line +of its own to close the string: + +```hcl +< "" "" { + # Block body + = # Argument +} +``` + +- _Blocks_ are containers for other content and usually represent the + configuration of some kind of object, like a source. Blocks have a + _block type,_ can have zero or more _labels,_ and have a _body_ that contains + any number of arguments and nested blocks. Most of Packer's features are + controlled by top-level blocks in a configuration file. +- _Arguments_ assign a value to a name. They appear within blocks. +- _Expressions_ represent a value, either literally or by referencing and + combining other values. They appear as values for arguments, or within other + expressions. + +For full details about Packer's syntax, see: + +- [Configuration Syntax](./syntax.html) +- [Expressions](./expressions.html) + +## Code Organization + +The HCL language uses configuration files that are named with the `.pkr.hcl` +file extension. There is also [a JSON-based variant of the +language](./syntax-json.html) that is named with the `.pkr.json` file +extension. + +Configuration files must always use UTF-8 encoding, and by convention are +usually maintained with Unix-style line endings (LF) rather than Windows-style +line endings (CRLF), though both are accepted. + +## Configuration Ordering + +The ordering of root blocks is not significant. The order of `provisioner` or +`post-processor` blocks within a `build` is the only major feature where block +order matters. diff --git a/website/source/docs/configuration/from-1.5/syntax-json.html.md b/website/source/docs/configuration/from-1.5/syntax-json.html.md new file mode 100644 index 000000000..a264b5f12 --- /dev/null +++ b/website/source/docs/configuration/from-1.5/syntax-json.html.md @@ -0,0 +1,276 @@ +--- +layout: "docs" +page_title: "JSON Configuration Syntax - Configuration Language" +sidebar_current: configuration-json-syntax +description: |- + In addition to the native syntax that is most commonly used with Packer, + the HCL language can also be expressed in a JSON-compatible syntax. +--- + +# JSON Configuration Syntax + + +Most Packer configurations are written in [the native HCL +syntax](./syntax.html), which is designed to be easy for humans to read and +update. + +Packer also supports an alternative syntax that is JSON-compatible. This +syntax is useful when generating portions of a configuration programmatically, +since existing JSON libraries can be used to prepare the generated +configuration files. + +The JSON syntax is defined in terms of the native syntax. Everything that can +be expressed in native syntax can also be expressed in JSON syntax, but some +constructs are more complex to represent in JSON due to limitations of the +JSON grammar. + +Packer expects native syntax for files named with a `.pkr.hcl` suffix, and JSON +syntax for files named with a `.pkr.json` suffix. + +The low-level JSON syntax, just as with the native syntax, is defined in terms +of a specification called _HCL_. It is not necessary to know all of the details +of HCL syntax or its JSON mapping in order to use Packer, and so this page +summarizes the most important differences between native and JSON syntax. If +you are interested, you can find a full definition of HCL's JSON syntax in [its +specification](https://github.com/hashicorp/hcl/blob/hcl2/hclsyntax/spec.md). + +## JSON File Structure + +At the root of any JSON-based Packer configuration is a JSON object. The +properties of this object correspond to the top-level block types of the +Packer language. For example: + +```json +{ + "variables": { + "example": "value" + } +} +``` + +Each top-level object property must match the name of one of the expected +top-level block types. Block types that expect labels, such as `variable` shown +above, are represented by one nested object value for each level of label. +`source` blocks expect two labels, so two levels of nesting are required: + +```json +{ + "source": { + "amazon-ebs": { + "example": { + "instance_type": "t2.micro", + "ami_name": "ami-abc123" + } + } + } +} +``` + +After any nested objects representing the labels, finally one more nested +object represents the body of the block itself. In the above example the +`instance_type` and `ami_name` arguments for `source "amazon-ebs" "example"` +are specified. + +Taken together, the above two configuration files are equivalent to the +following blocks in the native syntax: + +```hcl +variables { + example = "value" +} + +source "amazon-ebs" "example" { + instance_type = "t2.micro" + ami_name = "ami-abc123" +} +``` + +Within each top-level block type the rules for mapping to JSON are slightly +different (see [Block-type-specific Exceptions][inpage-exceptions] below), but the following general rules apply in most cases: + +* The JSON object representing the block body contains properties that + correspond either to argument names or to nested block type names. + +* Where a property corresponds to an argument that accepts + [arbitrary expressions](./expressions.html) in the native syntax, the + property value is mapped to an expression as described under + [_Expression Mapping_](#expression-mapping) below. For arguments that + do _not_ accept arbitrary expressions, the interpretation of the property + value depends on the argument, as described in the + [block-type-specific exceptions](#block-type-specific-exceptions) + given later in this page. + +* Where a property name corresponds to an expected nested block type name, + the value is interpreted as described under + [_Nested Block Mapping_](#nested-block-mapping) below, unless otherwise + stated in [the block-type-specific exceptions](#block-type-specific-exceptions) + given later in this page. + +## Expression Mapping + +Since JSON grammar is not able to represent all of the Packer language +[expression syntax](./expressions.html), JSON values interpreted as expressions +are mapped as follows: + +| JSON | Packer Language Interpretation | +| ------- | ------------------------------------------------------------------------------------------------------------- | +| Boolean | A literal `bool` value. | +| Number | A literal `number` value. | +| String | Parsed as a [string template](./expressions.html#string-templates) and then evaluated as described below. | +| Object | Each property value is mapped per this table, producing an `object(...)` value with suitable attribute types. | +| Array | Each element is mapped per this table, producing a `tuple(...)` value with suitable element types. | +| Null | A literal `null`. | + +When a JSON string is encountered in a location where arbitrary expressions are +expected, its value is first parsed as a [string template](./expressions.html#string-templates) +and then it is evaluated to produce the final result. + +If the given template consists _only_ of a single interpolation sequence, +the result of its expression is taken directly, without first converting it +to a string. This allows non-string expressions to be used within the +JSON syntax. + +## Nested Block Mapping + +When a JSON object property is named after a nested block type, the value +of this property represents one or more blocks of that type. The value of +the property must be either a JSON object or a JSON array. + +The simplest situation is representing only a single block of the given type +when that type expects no labels, as with the `tags` nested block used +within `source` blocks: + +```json +{ + "source": { + "amazon-ebs": { + "example": { + "tags": { + "key": "value" + } + } + } + } +} +``` + +The above is equivalent to the following native syntax configuration: + +```hcl +source "amazon-ebs" "example" { + tags { + key = "value" + } +} +``` + +When the nested block type requires one or more labels, or when multiple +blocks of the same type can be given, the mapping gets a little more +complicated. For example, the `provisioner` nested block type used +within `source` blocks expects a label giving the provisioner to use, +and the ordering of provisioner blocks is significant to decide the order +of operations. + +The following native syntax example shows a `source` block with a number +of provisioners of different types: + +```hcl +source "amazon-ebs" "example" { + # (source configuration omitted for brevity) + + provisioner "shell-local" { + inline = ["echo 'Hello World' >example.txt"] + } + provisioner "file" { + source = "example.txt" + destination = "/tmp/example.txt" + } + provisioner "shell" { + inline = [ + "sudo install-something -f /tmp/example.txt", + ] + } +} +``` + +In order to preserve the order of these blocks, you must use a JSON array +as the direct value of the property representing this block type, as in +this JSON equivalent of the above: + +```json +{ + "source": { + "amazon-ebs": { + "example": { + "provisioner": [ + { + "shell-local": { + "inline": ["echo 'Hello World' >example.txt"] + } + }, + { + "file": { + "source": "example.txt", + "destination": "/tmp/example.txt" + } + }, + { + "shell": { + "inline": ["sudo install-something -f /tmp/example.txt"] + } + } + ] + } + } + } +} +``` + +Each element of the `provisioner` array is an object with a single property +whose name represents the label for each `provisioner` block. For block types +that expect multiple labels, this pattern of alternating array and object +nesting can be used for each additional level. + +If a nested block type requires labels but the order does _not_ matter, you +may omit the array and provide just a single object whose property names +correspond to unique block labels. This is allowed as a shorthand for the above +for simple cases, but the alternating array and object approach is the most +general. We recommend using the most general form if systematically converting +from native syntax to JSON, to ensure that the meaning of the configuration is +preserved exactly. + +### Comment Properties + +Although we do not recommend hand-editing of JSON syntax configuration files +-- this format is primarily intended for programmatic generation and consumption -- +a limited form of _comments_ are allowed inside JSON objects that represent +block bodies using a special property name: + +```json +{ + "source": { + "amazon-ebs": { + "example": { + "//": "This instance runs the scheduled tasks for backup", + + "instance_type": "t2.micro", + "ami_name": "ami-abc123" + } + } + } +} +``` + +In any object that represents a block body, properties named `"//"` are +ignored by Packer entirely. This exception does _not_ apply to objects +that are being [interpreted as expressions](#expression-mapping), where this +would be interpreted as an object type attribute named `"//"`. + +This special property name can also be used at the root of a JSON-based +configuration file. This can be useful to note which program created the file. + +```json +{ + "//": "This file is generated by generate-outputs.py. DO NOT HAND-EDIT!", +} +``` diff --git a/website/source/docs/configuration/from-1.5/syntax.html.md b/website/source/docs/configuration/from-1.5/syntax.html.md new file mode 100644 index 000000000..5bc4c4690 --- /dev/null +++ b/website/source/docs/configuration/from-1.5/syntax.html.md @@ -0,0 +1,119 @@ +--- +layout: "docs" +page_title: "Syntax - Configuration Language" +sidebar_current: configuration-syntax +description: |- + HCL has its own syntax, intended to combine declarative + structure with expressions in a way that is easy for humans to read and + understand. +--- + +# HCL Configuration Syntax + +Other pages in this section have described various configuration constructs +that can appear in HCL. This page describes the lower-level syntax of the +language in more detail, revealing the building blocks that those constructs +are built from. + +This page describes the _native syntax_ of HCL, which is a rich language +designed to be easy for humans to read and write. The constructs in HCL can +also be expressed in [JSON syntax](./syntax-json.html), which is harder for +humans to read and edit but easier to generate and parse programmatically. + +This low-level syntax of HCL is defined in terms of a syntax called _HCL_, +which is also used by configuration languages in other applications, and in +particular other HashiCorp products. It is not necessary to know all of the +details of HCL in order to use Packer, and so this page summarizes the most +important details. If you are interested, you can find a full definition of HCL +syntax in [the HCL native syntax +specification](https://github.com/hashicorp/hcl/blob/hcl2/hclsyntax/spec.md). + +## Arguments and Blocks + +HCL syntax is built around two key syntax constructs: +arguments and blocks. + +### Arguments + +An _argument_ assigns a value to a particular name: + +```hcl +image_id = "abc123" +``` + +The identifier before the equals sign is the _argument name_, and the expression +after the equals sign is the argument's value. + +The context where the argument appears determines what value types are valid +(for example, each source type has a schema that defines the types of its +arguments), but many arguments accept arbitrary +[expressions](./expressions.html), which allow the value to +either be specified literally or generated from other values programmatically. + +### Blocks + +A _block_ is a container for other content: + +```hcl +source "amazon-ebs" "example" { + ami_name = "abc123" + + tags { + # ... + } +} +``` + +A block has a _type_ (`source` in this example). Each block type defines +how many _labels_ must follow the type keyword. The `source` block type +expects two labels, which are `amazon-ebs` and `example` in the example above. +A particular block type may have any number of required labels, or it may +require none as with the nested `tags` block type. + +After the block type keyword and any labels, the block _body_ is delimited +by the `{` and `}` characters. Within the block body, further arguments +and blocks may be nested, creating a hierarchy of blocks and their associated +arguments. + +HCL uses a limited number of _top-level block types,_ which +are blocks that can appear outside of any other block in a configuration file. +Most of Packer's features (including resources, input variables, output +values, data sources, etc.) are implemented as top-level blocks. + +## Identifiers + +Argument names, block type names, and the names of most Packer-specific +constructs like resources, input variables, etc. are all _identifiers_. + +Identifiers can contain letters, digits, underscores (`_`), and hyphens (`-`). +The first character of an identifier must not be a digit, to avoid ambiguity +with literal numbers. + +For complete identifier rules, Packer implements +[the Unicode identifier syntax](http://unicode.org/reports/tr31/), extended to +include the ASCII hyphen character `-`. + +## Comments + +HCL supports three different syntaxes for comments: + +* `#` begins a single-line comment, ending at the end of the line. +* `//` also begins a single-line comment, as an alternative to `#`. +* `/*` and `*/` are start and end delimiters for a comment that might span + over multiple lines. + +The `#` single-line comment style is the default comment style and should be +used in most cases. Automatic configuration formatting tools may automatically +transform `//` comments into `#` comments, since the double-slash style is +not idiomatic. + +## Character Encoding and Line Endings + +Packer configuration files must always be UTF-8 encoded. While the +delimiters of the language are all ASCII characters, Packer accepts +non-ASCII characters in identifiers, comments, and string values. + +Packer accepts configuration files with either Unix-style line endings +(LF only) or Windows-style line endings (CR then LF), but the idiomatic style +is to use the Unix convention, and so automatic configuration formatting tools +may automatically transform CRLF endings to LF. diff --git a/website/source/docs/extending/custom-builders.html.md b/website/source/docs/extending/custom-builders.html.md index 63cfdb40b..1cd2b6094 100644 --- a/website/source/docs/extending/custom-builders.html.md +++ b/website/source/docs/extending/custom-builders.html.md @@ -31,7 +31,8 @@ method should do. ``` go type Builder interface { - Prepare(...interface{}) error + ConfigSpec() hcldec.ObjectSpec + Prepare(...interface{}) ([]string, []string, error) Run(context.Context, ui Ui, hook Hook) (Artifact, error) } ``` diff --git a/website/source/docs/extending/custom-post-processors.html.md b/website/source/docs/extending/custom-post-processors.html.md index e940047b3..7f717346a 100644 --- a/website/source/docs/extending/custom-post-processors.html.md +++ b/website/source/docs/extending/custom-post-processors.html.md @@ -36,6 +36,7 @@ explaining what each method should do. ``` go type PostProcessor interface { + ConfigSpec() hcldec.ObjectSpec Configure(interface{}) error PostProcess(context.Context, Ui, Artifact) (a Artifact, keep, mustKeep bool, err error) } diff --git a/website/source/docs/extending/custom-provisioners.html.md b/website/source/docs/extending/custom-provisioners.html.md index 7cfd55939..0d6d7f15b 100644 --- a/website/source/docs/extending/custom-provisioners.html.md +++ b/website/source/docs/extending/custom-provisioners.html.md @@ -35,8 +35,9 @@ explaining what each method should do. ``` go type Provisioner interface { + ConfigSpec() hcldec.ObjectSpec Prepare(...interface{}) error - Provision(Ctx, Ui, Communicator, new(interface{})) error + Provision(context.Context, Ui, Communicator, map[string]interface{}) error } ``` diff --git a/website/source/guides/hcl/component-object-spec/index.html.md.erb b/website/source/guides/hcl/component-object-spec/index.html.md.erb new file mode 100644 index 000000000..f548abdc8 --- /dev/null +++ b/website/source/guides/hcl/component-object-spec/index.html.md.erb @@ -0,0 +1,44 @@ +--- +layout: guides +page_title: Generating code for config spec. +sidebar_current: hcl-component-object-spec +description: |- + Learn how to generate the HCL2 configuration of your component easily. +--- + +# Auto Generate the HCL2 code of a plugin + +From v1.5, Packer can be configured using HCL2. Because Packer has so many +builders, provisioner & post-processors, we relied on code generation to +iterate more easily. The good new is that you can benefit from this code +generator to get the HCL2 spec code of your component simply. It's a Go binary +package and is located in [`cmd/mapstructure-to-hcl2`](https://github.com/hashicorp/packer/tree/master/cmd/mapstructure-to-hcl2). + +Say you want to configure the `Config` struct of a `Builder` in a package +located in `my/example-plugin/config.go`. Here are some simple steps you can +follow to make it HCL2 enabled: + +* run `go install github.com/hashicorp/packer/cmd/mapstructure-to-hcl2` + +* Add `//go:generate mapstructure-to-hcl2 -type Config` at the top of +`config.go` + +* run `go generate ./my/example-plugin/...` + + This will generate a `my/example-plugin/config.hcl2spec.go` file containing + the configuration fields of `Config`. + +* Make sure that all the nested structs of `Config` are also auto generated the + same way. + +* Now we only need to make Builder implement the interface by adding the +following snippet: + + ```go + func (b *Builder) ConfigSpec() hcldec.ObjectSpec { return b.config.FlatMapstructure().HCL2Spec() } + ``` + + From now on every time you add or change a field of Config you will need to + run the `go generate` command again. + +A good example of this is the [Config struct of the amazon-ebs builder](https://github.com/hashicorp/packer/blob/master/builder/amazon/ebs/builder.go) \ No newline at end of file diff --git a/website/source/guides/hcl/from-json-v1/index.html.md.erb b/website/source/guides/hcl/from-json-v1/index.html.md.erb index 07ccdbf2b..cee1b0ac9 100644 --- a/website/source/guides/hcl/from-json-v1/index.html.md.erb +++ b/website/source/guides/hcl/from-json-v1/index.html.md.erb @@ -1,13 +1,13 @@ --- layout: guides page_title: Transforming Packer v1 files for Packer v1.5.0 -sidebar_current: hcl +sidebar_current: hcl-from-json-v1 description: |- Learn how to manually move from a Packer v1 working JSON build file to a working v1.5.0 HCL file. --- -# Transforming Packer v1 files for Packer v1.5 +# Transforming Packer v1 config files to HCL2 for Packer v1.5 -> **Note:** Starting from version **1.5.0** Packer can read HCL2 files. diff --git a/website/source/guides/hcl/index.html.md.erb b/website/source/guides/hcl/index.html.md.erb index 5d66496bf..555d06c94 100644 --- a/website/source/guides/hcl/index.html.md.erb +++ b/website/source/guides/hcl/index.html.md.erb @@ -50,3 +50,41 @@ It could be super handy for a boot_command. EOF } ``` + +## Building blocks can be split in files + +Currently Packer offers the `source` and the `build` root blocks. These two +building blocks can be defined in any order and a `build` can import one or more +`source`. Usually a `source` defines what we currently call a builder and a +`build` can apply multiple provisioning steps to a source. For example: + +```hcl +# folder/sources.pkr.hcl +source "amazon-ebs" "example-1" { + ami_name = "example-1-ami" +} + +source "virtualbox-iso" "example-2" { + boot_command = < +/install/vmlinuz noapic +... +EOF +} +``` + +```hcl +# folder/build.pkr.hcl +build { + sources = [ + "source.amazon-ebs.example-1", + "source.virtualbox-iso.example-2" + ] + + provisioner "shell" { + inline = [ + "echo it's alive !" + ] + } +} +``` diff --git a/website/source/layouts/docs.erb b/website/source/layouts/docs.erb index c6f13a3e3..6f833a900 100644 --- a/website/source/layouts/docs.erb +++ b/website/source/layouts/docs.erb @@ -1,9 +1,24 @@ <% wrap_layout :inner do %> <% content_for :sidebar do %>