http

This example uses channels, waitgroup and timeouts to make concurrent, bounded http requests.

cd go-bits/http
go run -race main.go

The tests can be run to verify behaviour.

cd go-bit/http
go test ./... -race

main.go

\

1. A context with a timeout is created for use in cancelling ongoing goroutines.
2. An http client is created with a timeout specifying the time limit for requests.
3. A new request client is created.
4. Calling client.GetResultChannel returns a channel of result.
5. We range over the channel of result and output either the status code from a successful request or the error from an unsuccessful request.

client.go

• The client struct uses the do func to make http requests and the concurrencyLimit to limit the number of concurrent requests.
• The result and status structs are used to hold the responses from the http requests.

GetResultChannel

1. Sending to the semaphoreChan will block when the channel buffer is full, thereby limiting the number of concurrent requests.
2. The resultChan is used to hold the results from the http requests and is returned immediately allowing main to range over.
3. A sync.WaitGroup is created and used to track the goroutines started to make the http requests.
4. The WaitGroup is counter is incremented by the number of urls that will be requested concurrently.
5. The urls are ranged over and a goroutine is started for each request.
6. A select is used to determine whether the context has been cancelled or timed out, if so the WaitGroup is decremented.
7. If context.Done() cannot be received from, the semaphoreChan is sent to.
8. An http request is made by calling c.GetResult (see below) and the result is sent to the resultsChan.
9. The semaphoreChan is received from thereby reducing the number of struct{} in the channel by one.
10. The WaitGroup is decremented.
11. An additional goroutine is started and uses Wait to block until all of the goroutines started to make http requests have completed or have timed out or been cancelled. Both the semaphoreChan and resultsChan are then closed, the latter is required as ranging over the resultsChan in main.go will block otherwise.

GetResult

1. A NewRequestWithContext is used to allow cancellation/timeout of the request.