Skip to content

Commit 92520a9

Browse files
Ivan De Marinolaurapacilio
Ivan De Marino
and
laurapacilio
authored
New SDKv2/logging documentation section (#1011)
* New SDKv2/logging documentation section, including new Logging HTTP Transport page * Tweak the release notes for PR 1006 * Apply suggestions from code review * Streamline the links at the end of the website doc for HTTP Transport Co-authored-by: Laura Pacilio <[email protected]>
1 parent 69b5079 commit 92520a9

File tree

4 files changed

+247
-1
lines changed

4 files changed

+247
-1
lines changed

Diff for: ‎.changelog/1006.txt

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
```release-note:feature
2-
helper/logging: New `NewLoggingHTTPTransport()` and `NewSubsystemLoggingHTTPTransport()` functions, providing `http.RoundTripper` Transport implementations that log request/response using [terraform-plugin-log](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-log)
2+
helper/logging: New `NewLoggingHTTPTransport()` and `NewSubsystemLoggingHTTPTransport()` functions, providing `http.RoundTripper` Transport implementations that log request/response using [terraform-plugin-log](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-log) ([#546](https://github.com/hashicorp/terraform-plugin-sdk/issues/546))
33
```
44

55
```release-note:note

Diff for: ‎website/data/plugin-sdk-nav-data.json

+13
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,19 @@
9696
}
9797
]
9898
},
99+
{
100+
"title": "Logging",
101+
"routes": [
102+
{
103+
"title": "Overview",
104+
"path": "logging"
105+
},
106+
{
107+
"title": "HTTP Transport",
108+
"path": "logging/http-transport"
109+
}
110+
]
111+
},
99112
{
100113
"title": "Testing",
101114
"routes": [

Diff for: ‎website/docs/plugin/sdkv2/logging/http-transport.mdx

+208
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
---
2+
page_title: Plugin Development - Logging HTTP Transport
3+
description: |-
4+
SDKv2 provides a helper to send all the HTTP transactions to structured logging.
5+
---
6+
7+
# HTTP Transport
8+
9+
Terraform's public interface has included `helper/logging`: [`NewTransport()`](https://github.com/hashicorp/terraform-plugin-sdk/blob/main/helper/logging/transport.go) since v0.9.5. This helper is an implementation of the Golang standard library [`http.RoundTripper`](https://pkg.go.dev/net/http#RoundTripper) that lets you add logging at the `DEBUG` level to your provider's HTTP transactions.
10+
11+
We do not recommend using this original helper because it is designed to log the entirety of each request and response. This includes any sensitive content that may be present in the message header or body, presenting security concerns.
12+
13+
Instead, we recommend using the [terraform-plugin-log](https://www.terraform.io/plugin/log) library to produce logs for your provider. This library does not present the same security concerns and provides [log filtering](https://www.terraform.io/plugin/log/filtering) functionality. This page explains how to set up the new `RoundTripper()` helper to log HTTP Transactions with `terraform-plugin-log`.
14+
15+
16+
# Setting Up Logging for HTTP Transactions
17+
18+
The recommended logging helper for SDK is built on top of [terraform-plugin-log](https://www.terraform.io/plugin/log). This lets you leverage the features from our structured logging framework without having to write an entire implementation of `http.RoundTripper`.
19+
20+
There are two functions inside `helper/logging` that target a specific logging setup for your provider. Refer to [“Writing Log Output”](https://www.terraform.io/plugin/log/writing) for details.
21+
22+
* `NewLoggingHTTPTransport(transport http.RoundTripper)`: Use this method when you want logging against the `tflog` Provider root logger.
23+
* `NewSubsystemLoggingHTTPTransport(subsystem string, transport http.RoundTripper)`: Use this method when you want logging against a `tflog` Provider [Subsystem logger](https://www.terraform.io/plugin/log/writing#subsystems). The `subsystem` string you use with `NewSubsystemLoggingHTTPTransport()` must match the [pre-created subsystem logger name](https://www.terraform.io/plugin/log/writing#create-subsystems).
24+
25+
To set up HTTP transport, you must create the HTTP Client to use the new transport and then add logging configuration to the HTTP request context.
26+
27+
28+
### Creating the HTTP Client
29+
30+
After you create the transport , you must use it to set up the `http.Client` for the provider. The following example sets up the client in `schema.Provider` `ConfigureContextFunc`. The client is identical to the default Golang `http.Client`, except it uses the new logging transport.
31+
32+
```go
33+
func New() (*schema.Provider, error) {
34+
return &schema.Provider{
35+
// omitting the rest of the schema definition
36+
37+
ConfigureContextFunc: func (ctx context.Context, rsc *schema.ResourceData) (interface{}, diag.Diagnostics) {
38+
39+
// omitting provider-specific configuration logic
40+
41+
transport := logging.NewLoggingHTTPTransport(http.DefaultTransport)
42+
client := http.Client{
43+
Transport: transport,
44+
}
45+
46+
return client, diag.Diagnostics{}
47+
}
48+
}
49+
}
50+
51+
52+
## Adding Context to HTTP Requests
53+
54+
All calls to the `tflog` package must contain an SDK provided `context.Context` that stores the logging implementation. Providers written with `terraform-plugin-sdk` must use context-aware functionality, such as the [`helper/schema.Resource` type `ReadContext` field](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema#Resource.ReadContext).
55+
56+
The following example uses [`http.NewRequestWithContext()` function](https://pkg.go.dev/net/http#NewRequestWithContext) to create an HTTP request that includes the logging configuration from the `context.Context`.
57+
58+
```go
59+
// inside a context-aware Resource function
60+
req, err := http.NewRequestWithContext(ctx, "GET", "https://www.terraform.io", nil)
61+
if err != nil {
62+
return fmt.Errorf("Failed to create a new request: %w", err)
63+
}
64+
65+
res, err := client.Do(req)
66+
if err != nil {
67+
return fmt.Errorf("Request failed: %w", err)
68+
}
69+
defer res.Body.Close()
70+
```
71+
72+
Use the [`(http.Request).WithContext()` method](https://pkg.go.dev/net/http#Request.WithContext) to set the context for the `http.Request` if the request is generated separately from where the `context.Context` is available.
73+
74+
75+
## HTTP Transaction Log Format
76+
77+
The logging transport produces two log entries for each HTTP transaction: one for the request and one for the response.
78+
79+
### Request Example
80+
81+
The following example shows a log generated from an HTTP Request to [https://terraform.io](https://terraform.io).
82+
83+
```text
84+
2022-07-26T18:54:08.880+0100 [DEBUG] provider: Sending HTTP Request: Accept-Encoding=gzip Content-Length=0 \
85+
Host=www.terraform.io User-Agent=Go-http-client/1.1 \
86+
tf_http_op_type=request tf_http_req_method=GET \
87+
tf_http_req_uri=/ tf_http_req_version=HTTP/1.1 tf_http_trans_id=7e80e48d-8f32-f527-1412-52a8c84359e7
88+
```
89+
90+
The following example shows the same logs after you enable [logging in JSON format](https://www.terraform.io/internals/debugging).
91+
92+
```json
93+
{
94+
"@level": "debug",
95+
"@message": "Sending HTTP Request",
96+
"@module": "provider",
97+
"@timestamp": "2022-07-26T18:54:08.880+0100",
98+
99+
"Accept-Encoding": "gzip",
100+
"Content-Length": "0",
101+
"Host": "www.terraform.io",
102+
"User-Agent": "Go-http-client/1.1",
103+
104+
"tf_http_op_type": "request",
105+
"tf_http_req_body": "",
106+
"tf_http_req_method": "GET",
107+
"tf_http_req_uri": "/",
108+
"tf_http_req_version": "HTTP/1.1",
109+
"tf_http_trans_id": "7e80e48d-8f32-f527-1412-52a8c84359e7"
110+
}
111+
```
112+
113+
### Response Example
114+
115+
The following example shows logs from a [https://terraform.io](https://terraform.io) HTTP response.
116+
117+
```text
118+
2022-07-26T18:54:10.734+0100 [DEBUG] provider: Received HTTP Response: Age=9 \
119+
Cache-Control="public, max-age=0, must-revalidate" Content-Type=text/html \
120+
Date="Tue, 26 Jul 2022 13:16:46 GMT" Etag="... ABCDEFGH..." Server=Vercel \
121+
Strict-Transport-Security="max-age=63072000" X-Frame-Options=SAMEORIGIN X-Matched-Path=/ \
122+
X-Nextjs-Cache=HIT X-Powered-By=Next.js X-Vercel-Cache=STALE \
123+
X-Vercel-Id=lhr1::iad1::lx2h8-99999999999999-fffffffffff \
124+
tf_http_op_type=response tf_http_res_body="... LOTS OF HTML ..." tf_http_res_status_code=200 \
125+
tf_http_res_status_reason="200 OK" tf_http_res_version=HTTP/2.0 \
126+
tf_http_trans_id=7e80e48d-8f32-f527-1412-52a8c84359e7
127+
```
128+
129+
the following example shows the same logs in JSON format.
130+
131+
```json
132+
{
133+
"@level": "debug",
134+
"@message": "Received HTTP Response",
135+
"@module": "provider",
136+
"@timestamp": "2022-07-26T18:54:10.734+0100",
137+
138+
"Age": "9",
139+
"Cache-Control": "public, max-age=0, must-revalidate",
140+
"Content-Type": "text/html",
141+
"Date": "Tue, 26 Jul 2022 13:16:46 GMT",
142+
"Etag": "... ABCDEFGH...",
143+
"Server": "Vercel",
144+
"Strict-Transport-Security": "max-age=63072000",
145+
"X-Frame-Options": "SAMEORIGIN",
146+
"X-Matched-Path": "/",
147+
"X-Nextjs-Cache": "HIT",
148+
"X-Powered-By": "Next.js",
149+
"X-Vercel-Cache": "STALE",
150+
"X-Vercel-Id": "lhr1::iad1::lx2h8-99999999999999-fffffffffff",
151+
152+
"tf_http_op_type": "response",
153+
"tf_http_res_body": "... LOTS OF HTML ...",
154+
"tf_http_res_status_code": 200,
155+
"tf_http_res_status_reason": "200 OK",
156+
"tf_http_res_version": "HTTP/2.0",
157+
"tf_http_trans_id": "7e80e48d-8f32-f527-1412-52a8c84359e7"
158+
}
159+
```
160+
161+
### Log Information
162+
163+
Each log contains the following information, which is represented as [fields](https://www.terraform.io/plugin/log/writing#fields) in the JSON format.
164+
165+
| Log field name | Description | Possible values | Applies to |
166+
|---------------------------:|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------|:------------------:|
167+
| `tf_http_op_type` | Which HTTP operation log refers to | [`request`, `response`] | Request / Response |
168+
| `tf_http_trans_id` | Unique identifier used by Request and Response that belong to the same HTTP Transaction | A universally unique identifier ([UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier)) | Request / Response |
169+
| `tf_http_req_body` | Request body | | Request |
170+
| `tf_http_req_method` | Request method | A canonical [HTTP methods](https://developer.mozilla.org/en-US/docs/Web/HTTP/Method) | Request |
171+
| `tf_http_req_uri` | Request URI | Ex. `"/path"` | Request |
172+
| `tf_http_req_version` | Request HTTP version | Ex. `"HTTP/1.1"` | Request |
173+
| `tf_http_res_body` | Response body | | Response |
174+
| `tf_http_res_status_code` | Response status code | A canonical [HTTP status code](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status) | Response |
175+
| `tf_http_res_status_reason | Response status reason | Canonical textual description of the corresponding `tf_http_res_status_code` | Response |
176+
| `tf_http_res_version` | Response HTTP version | Ex. `"HTTP/2.0"` | Response |
177+
| (Other fields) | Request / Response headers. One field per header. If the header contains a single value, the log field value is set to that value. Otherwise, the field value is a slice of strings. | | Request / Response |
178+
179+
180+
## Filtering Sensitive Data
181+
182+
To [filter logs](https://www.terraform.io/plugin/log/filtering), you must configure the `context.Context` before before it is added to the `http.Request`.
183+
184+
The following example masks all the header values of HTTP Requests containing an `Authorization` and `Proxy-Authorization` credentials.
185+
186+
```go
187+
// inside a context-aware Resource function
188+
ctx := tflog.SubsystemMaskFieldValuesWithFieldKeys(ctx, "my-subsystem", "Authorization")
189+
ctx = tflog.SubsystemMaskFieldValuesWithFieldKeys(ctx, "my-subsystem", "Proxy-Authorization")
190+
191+
req, err := http.NewRequestWithContext(ctx, "GET", "https://www.terraform.io", nil)
192+
if err != nil {
193+
return fmt.Errorf("Failed to create a new request: %w", err)
194+
}
195+
196+
res, err := client.Do(req)
197+
if err != nil {
198+
return fmt.Errorf("Request failed: %w", err)
199+
}
200+
defer res.Body.Close()
201+
```
202+
203+
204+
# Links
205+
206+
* [Plugin Development - Logging](https://www.terraform.io/plugin/log) - Learn more about the logging framework
207+
* [terraform-plugin-log - tflog](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-log/tflog) - Read the Golang documentation for the logging framework
208+
* Read the Golang documentation for [`NewLoggingHTTPTransport()`](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2/helper/logging#NewLoggingHTTPTransport) and [`NewSubsystemLoggingHTTPTransport()`](https://pkg.go.dev/github.com/hashicorp/terraform-plugin-sdk/v2/helper/logging#NewSubsystemLoggingHTTPTransport)

Diff for: ‎website/docs/plugin/sdkv2/logging/index.mdx

+25
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
---
2+
page_title: Plugin Development - Logging
3+
description: |-
4+
High-quality logs are important when debugging your provider. Learn to set-up logging and write meaningful logs.
5+
---
6+
7+
# Logging
8+
9+
Terraform Plugin SDKv2 integrates with the structured logging framework [terraform-plugin-log](https://www.terraform.io/plugin/log). High-quality logs are critical to quickly [debugging your provider](https://www.terraform.io/plugin/debugging).
10+
11+
## Managing Log Output
12+
13+
Learn how to use [environment variables and other methods](https://www.terraform.io/plugin/log/managing) to enable and filter logs.
14+
15+
## Writing Log Output
16+
17+
Learn how to [implement code in provider logic](https://www.terraform.io/plugin/log/writing) to output logs.
18+
19+
## Filtering Log Output
20+
21+
Learn how to [implement code in provider logic](https://www.terraform.io/plugin/log/filtering) to omit logs or mask specific log messages and structured log fields.
22+
23+
## Log HTTP Transactions
24+
25+
Learn how to [set up the Logging HTTP Transport](./logging/http-transport) to log HTTP Transactions with the [structured logging framework](https://www.terraform.io/plugin/log).

0 commit comments

Comments
 (0)