How to Output DEBUG-Level Logs in Terraform

This is a quick note on how to change the log output level in Terraform.

Introduction

When troubleshooting, logs are an important source of information.

If the default logs don't provide enough details, increasing the log level can be helpful.

In this post, I’ll summarize how to change the log level in Terraform, and also take a quick look at the relevant source code.

Note: This article was translated from my original post.

Outputting DEBUG-Level Logs in Terraform

To change the log level in Terraform, set the environment variable TF_LOG.

Ref. Terraform | HashiCorp Developer

For example, to output DEBUG-level logs during terraform plan, set TF_LOG to DEBUG:

# Set TF_LOG only for this command
TF_LOG=DEBUG terraform plan

# Set TF_LOG and then run the command
export TF_LOG=DEBUG
terraform plan

You can set the following log levels with TF_LOG:

• TRACE
• DEBUG
• INFO
• WARN
• ERROR

Choose the appropriate level depending on how much information you need.

Setting Log Levels Separately for Terraform and Providers

You can set different log levels for Terraform itself and for the provider.

To set the log level for Terraform, use the environment variable TF_LOG_CORE.

To set the log level for the provider, use TF_LOG_PROVIDER:

# Set Terraform log level to DEBUG and run plan
TF_LOG_CORE=DEBUG terraform plan

# Set Provider log level to DEBUG and run plan
TF_LOG_PROVIDER=DEBUG terraform plan

Both TF_LOG_CORE and TF_LOG_PROVIDER support the same log levels as TF_LOG:

• TRACE
• DEBUG
• INFO
• WARN
• ERROR

Appendix: Reading Terraform's Source Code

Let’s also take a look at the part of the Terraform source code that handles log level settings.

// newHCLogger returns a new hclog.Logger instance with the given name
func newHCLogger(name string) hclog.Logger {
    logOutput := io.Writer(os.Stderr)
    logLevel, json := globalLogLevel()

    if logPath := os.Getenv(envLogFile); logPath != "" {
        f, err := os.OpenFile(logPath, syscall.O_CREAT|syscall.O_RDWR|syscall.O_APPEND, 0666)
        if err != nil {
            fmt.Fprintf(os.Stderr, "Error opening log file: %v\n", err)
        } else {
            logOutput = f
        }
    }

    return hclog.NewInterceptLogger(&hclog.LoggerOptions{
        Name:              name,
        Level:             logLevel,
        Output:            logOutput,
        IndependentLevels: true,
        JSONFormat:        json,
    })
}

// NewLogger returns a new logger based in the current global logger, with the
// given name appended.
func NewLogger(name string) hclog.Logger {
    if name == "" {
        panic("logger name required")
    }
    return &logPanicWrapper{
        Logger: logger.Named(name),
    }
}

// NewProviderLogger returns a logger for the provider plugin, possibly with a
// different log level from the global logger.
func NewProviderLogger(prefix string) hclog.Logger {
    l := &logPanicWrapper{
        Logger: logger.Named(prefix + "provider"),
    }

    level := providerLogLevel()
    logger.Debug("created provider logger", "level", level)

    l.SetLevel(level)
    return l
}

// CurrentLogLevel returns the current log level string based the environment vars
func CurrentLogLevel() string {
    ll, _ := globalLogLevel()
    return strings.ToUpper(ll.String())
}

func providerLogLevel() hclog.Level {
    providerEnvLevel := strings.ToUpper(os.Getenv(envLogProvider))
    if providerEnvLevel == "" {
        providerEnvLevel = strings.ToUpper(os.Getenv(envLog))
    }

    return parseLogLevel(providerEnvLevel)
}

func globalLogLevel() (hclog.Level, bool) {
    var json bool
    envLevel := strings.ToUpper(os.Getenv(envLog))
    if envLevel == "" {
        envLevel = strings.ToUpper(os.Getenv(envLogCore))
    }
    if envLevel == "JSON" {
        json = true
    }
    return parseLogLevel(envLevel), json
}

Ref. terraform/internal/logging/logging.go at c047958b5708e4f500fe61000a662c169d048fff · hashicorp/terraform · GitHub

As you can see, the global log level is retrieved via globalLogLevel and passed to newHCLogger, while the provider log level is retrieved with providerLogLevel and used in NewProviderLogger.

The valid log levels are defined here:

    // ValidLevels are the log level names that Terraform recognizes.
    ValidLevels = []string{"TRACE", "DEBUG", "INFO", "WARN", "ERROR", "OFF"}

Ref. terraform/internal/logging/logging.go at c047958b5708e4f500fe61000a662c169d048fff · hashicorp/terraform · GitHub

Since strings.ToUpper is used when parsing log level values, capitalization doesn’t matter.

Conclusion

That’s it for how to change the log output level in Terraform.

Just knowing this can help you troubleshoot faster. It’s worth adding to your toolbox.

Follow @bioerrorlog

[Related Articles]

en.bioerrorlog.work

References

• Terraform | HashiCorp Developer
• terraform/internal/logging/logging.go at 271352620b43872aa99f7837f9c1c243b417a45c · hashicorp/terraform · GitHub