Skip to content
This repository was archived by the owner on Sep 9, 2020. It is now read-only.

Integration Test Benchmarks #558

Closed
wants to merge 1 commit into from
Closed

Integration Test Benchmarks #558

wants to merge 1 commit into from

Conversation

jmank88
Copy link
Collaborator

@jmank88 jmank88 commented May 11, 2017

This change adds benchmarks for the in-process integration test cases. Here is an example execution:

BenchmarkIntegration/ensure/empty/case1-4         	       1	2270968333 ns/op
BenchmarkIntegration/ensure/empty/case2-4         	       1	1059966456 ns/op
BenchmarkIntegration/ensure/empty/case3-4         	       1	1092412167 ns/op
BenchmarkIntegration/ensure/override/case1-4      	       1	1669498530 ns/op
BenchmarkIntegration/ensure/pkg-errors/case1-4    	2000000000	         0.00 ns/op
BenchmarkIntegration/ensure/update/case1-4        	       1	2178060316 ns/op
BenchmarkIntegration/ensure/update/case2-4        	       1	2308889406 ns/op
BenchmarkIntegration/init/case1-4                 	       1	1550706487 ns/op
BenchmarkIntegration/init/case2-4                 	       1	1210377305 ns/op
BenchmarkIntegration/init/case3-4                 	       1	1249999250 ns/op
BenchmarkIntegration/init/manifest-exists-4       	2000000000	         0.00 ns/op
BenchmarkIntegration/init/skip-hidden-4           	       1	1073756377 ns/op
BenchmarkIntegration/remove/force/case1-4         	       1	1374232234 ns/op
BenchmarkIntegration/remove/specific/case1-4      	       1	1288340159 ns/op
BenchmarkIntegration/remove/specific/case2-4      	       1	1100542801 ns/op
BenchmarkIntegration/remove/unused/case1-4        	       1	1238545492 ns/op
BenchmarkIntegration/status/case1/dot-4           	       1	3104462186 ns/op
BenchmarkIntegration/status/case1/json-4          	       1	3085201266 ns/op
BenchmarkIntegration/status/case1/table-4         	       1	3166151319 ns/op
PASS
ok  	github.com/golang/dep/cmd/dep	36.556s

Using built-in benchmarking at this high of a level seems a bit silly, but it was simple to adapt the existing integration testing structure with the testing.TB interface. I'm happy to evolve this into something more customized if that makes sense.

@sdboyer
Copy link
Member

sdboyer commented May 13, 2017

I appreciate the initiative, but yeah, I'm not sure this tells us anything terribly useful. The times here are dominated by network interactions, and are biased towards the case where the repo cache isn't already populated - highly unrepresentative of the typical case.

Let's do open up an issue to discuss what areas we might be able to more meaningfully benchmark, though - @sectioneight might also be interested in that.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants