Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug: tests.lib.log: race condition #83

Open
katcipis opened this issue Mar 21, 2017 · 2 comments
Open

bug: tests.lib.log: race condition #83

katcipis opened this issue Mar 21, 2017 · 2 comments
Assignees
@katcipis
Labels
bug

Comments

@katcipis
Copy link
Contributor

Logs:

cd tests/azure && go test -parallel 30  -timeout 30m -race -run= ./... -args -logger file
==================
WARNING: DATA RACE
Write at 0x00c4200db880 by goroutine 31:
  os.(*file).close()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/os/file_unix.go:143 +0x124
  os.(*File).Close()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/os/file_unix.go:132 +0x55
  github.com/NeowayLabs/klb/tests/lib/log.newFile.func1()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/log/log.go:55 +0x41
  runtime.call32()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/runtime/asm_amd64.s:479 +0x4b
  testing.(*common).Fatal()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/testing/testing.go:490 +0x7c
  github.com/NeowayLabs/klb/tests/lib/retrier.(*Retrier).Run()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/retrier/retrier.go:61 +0x808
  github.com/NeowayLabs/klb/tests/lib/nash.(*Shell).Run()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/nash/nash.go:55 +0x236
  github.com/NeowayLabs/klb/tests/azure_test.testResourceGroupDelete()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/azure/resourcegroup_test.go:64 +0x3af
  testing.tRunner()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/testing/testing.go:610 +0xc9

Previous read at 0x00c4200db880 by goroutine 57:
  os.(*File).write()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/os/file_unix.go:249 +0x8a
  os.(*File).Write()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/os/file.go:142 +0x6f
  log.(*Logger).Output()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/log/log.go:166 +0x397
  log.(*Logger).Printf()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/log/log.go:173 +0x8f
  github.com/NeowayLabs/klb/tests/lib/retrier.retryUntilDone.func1()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/retrier/retrier.go:80 +0xf8

Goroutine 31 (running) created at:
  testing.(*T).Run()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/testing/testing.go:646 +0x52f
  github.com/NeowayLabs/klb/tests/azure_test.TestResourceGroup()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/azure/resourcegroup_test.go:78 +0x9a
  testing.tRunner()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/testing/testing.go:610 +0xc9

Goroutine 57 (running) created at:
  github.com/NeowayLabs/klb/tests/lib/retrier.retryUntilDone()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/retrier/retrier.go:82 +0xe1
  github.com/NeowayLabs/klb/tests/lib/retrier.(*Retrier).Run()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/retrier/retrier.go:49 +0x1f2
  github.com/NeowayLabs/klb/tests/lib/nash.(*Shell).Run()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/nash/nash.go:55 +0x236
  github.com/NeowayLabs/klb/tests/azure_test.testResourceGroupDelete()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/azure/resourcegroup_test.go:64 +0x3af
  testing.tRunner()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/testing/testing.go:610 +0xc9
==================
--- FAIL: TestStorageAccount (0.00s)
    --- FAIL: TestStorageAccount/StorageAccount_Create (334.01s)
    	retrier.go:61: &{{{{0 0} 0 0 0 0} [] {0xc420119140} false false false false false 0xc42008f680 2 TestStorageAccount/StorageAccount_Create {63625719651 584166723 0xace8c0} 941 0xc420067f80 0xc420144000 []} true 0xc4200710e0} 
    		
    		work "Shell.Run:./testdata/create_storage_account.sh" failed, errors in order:
    		error[0]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[1]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[2]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[3]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[4]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[5]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[6]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[7]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[8]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[9]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[10]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[11]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[12]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[13]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[14]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[15]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[16]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[17]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[18]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[19]: error: exit status 8, executing script: ./testdata/create_storage_account.sh
    		error[20]: retrier timeout
--- FAIL: TestVnet (0.01s)
    --- FAIL: TestVnet/Vnet_Create (334.44s)
    	retrier.go:61: &{{{{0 0} 0 0 0 0} [] {0xc42008fbc0} false false false false false 0xc42008f8c0 2 TestVnet/Vnet_Create {63625719651 586196444 0xace8c0} 1897 0xc420066ba0 0xc420066c00 []} true 0xc4200710e0} 
    		
    		work "Shell.Run:./testdata/create_vnet.sh" failed, errors in order:
    		error[0]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[1]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[2]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[3]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[4]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[5]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[6]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[7]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[8]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[9]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[10]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[11]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[12]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[13]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[14]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[15]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[16]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[17]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[18]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[19]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[20]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[21]: retrier timeout
--- FAIL: TestNsg (0.01s)
    --- FAIL: TestNsg/Nsg_Create (348.65s)
    	retrier.go:61: &{{{{0 0} 0 0 0 0} [] {0xc42008fa40} false false false false false 0xc42008f440 2 TestNsg/Nsg_Create {63625719651 585058651 0xace8c0} 2182 0xc420066ae0 0xc420066b40 []} true 0xc4200710e0} 
    		
    		work "Shell.Run:./testdata/create_nsg.sh" failed, errors in order:
    		error[0]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[1]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[2]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[3]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[4]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[5]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[6]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[7]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[8]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[9]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[10]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[11]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[12]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[13]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[14]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[15]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[16]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[17]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[18]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[19]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[20]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[21]: retrier timeout
--- FAIL: TestNic (0.01s)
    --- FAIL: TestNic/Nic_Create (349.31s)
    	retrier.go:61: &{{{{0 0} 0 0 0 0} [] {0xc420119a40} false false false false false 0xc42008f380 2 TestNic/Nic_Create {63625719651 591717571 0xace8c0} 1676 0xc4201448a0 0xc420144900 []} true 0xc4200710e0} 
    		
    		work "Shell.Run:./testdata/create_nsg.sh" failed, errors in order:
    		error[0]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[1]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[2]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[3]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[4]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[5]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[6]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[7]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[8]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[9]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[10]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[11]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[12]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[13]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[14]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[15]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[16]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[17]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[18]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[19]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[20]: error: exit status 8, executing script: ./testdata/create_nsg.sh
    		error[21]: retrier timeout
--- FAIL: TestSubnet (0.00s)
    --- FAIL: TestSubnet/testSubnetCreate (349.80s)
    	retrier.go:61: &{{{{0 0} 0 0 0 0} [] {0xc420118840} false false false false false 0xc42008f740 2 TestSubnet/testSubnetCreate {63625719651 581120697 0xace8c0} 1442 0xc420067980 0xc4200679e0 []} true 0xc4200710e0} 
    		
    		work "Shell.Run:./testdata/create_vnet.sh" failed, errors in order:
    		error[0]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[1]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[2]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[3]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[4]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[5]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[6]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[7]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[8]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[9]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[10]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[11]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[12]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[13]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[14]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[15]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[16]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[17]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[18]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[19]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[20]: error: exit status 8, executing script: ./testdata/create_vnet.sh
    		error[21]: retrier timeout
--- FAIL: TestRouteTable (0.01s)
    --- FAIL: TestRouteTable/RouteTable_AddVirtualApplianceRoute (347.91s)
    	retrier.go:61: &{{{{0 0} 0 0 0 0} [] {0xc42015c9c0} false false false false false 0xc42008f5c0 2 TestRouteTable/RouteTable_AddVirtualApplianceRoute {63625719651 592216833 0xace8c0} 1367 0xc420145800 0xc420145860 []} true 0xc4200710e0} 
    		
    		work "Shell.Run:./testdata/create_route_table.sh" failed, errors in order:
    		error[0]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[1]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[2]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[3]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[4]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[5]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[6]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[7]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[8]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[9]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[10]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[11]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[12]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[13]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[14]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[15]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[16]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[17]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[18]: retrier timeout
    --- FAIL: TestRouteTable/RouteTable_Create (348.41s)
    	retrier.go:61: &{{{{0 0} 0 0 0 0} [] {0xc420118fc0} false false false false false 0xc42008f5c0 2 TestRouteTable/RouteTable_Create {63625719651 593652282 0xace8c0} 1747 0xc420067ec0 0xc420067f20 []} true 0xc4200710e0} 
    		
    		work "Shell.Run:./testdata/create_route_table.sh" failed, errors in order:
    		error[0]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[1]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[2]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[3]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[4]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[5]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[6]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[7]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[8]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[9]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[10]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[11]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[12]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[13]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[14]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[15]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[16]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[17]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[18]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[19]: retrier timeout
    --- FAIL: TestRouteTable/RouteTable_AddInternetRoute (350.21s)
    	retrier.go:61: &{{{{0 0} 0 0 0 0} [] {0xc42015c6c0} false false false false false 0xc42008f5c0 2 TestRouteTable/RouteTable_AddInternetRoute {63625719651 591191770 0xace8c0} 1549 0xc4201451a0 0xc420145200 []} true 0xc4200710e0} 
    		
    		work "Shell.Run:./testdata/create_route_table.sh" failed, errors in order:
    		error[0]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[1]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[2]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[3]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[4]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[5]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[6]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[7]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[8]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[9]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[10]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[11]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[12]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[13]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[14]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[15]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[16]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[17]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[18]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[19]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[20]: error: exit status 8, executing script: ./testdata/create_route_table.sh
    		error[21]: retrier timeout
--- FAIL: TestAvailabilitySet (0.00s)
    --- FAIL: TestAvailabilitySet/AvailabilitySet_Create (350.08s)
    	retrier.go:61: &{{{{0 0} 0 0 0 0} [] {0xc42014c240} false false false false false 0xc42008f2c0 2 TestAvailabilitySet/AvailabilitySet_Create {63625719651 586494644 0xace8c0} 1194 0xc42000c3c0 0xc42000c420 []} true 0xc4200710e0} 
    		
    		work "Shell.Run:./testdata/create_avail_set.sh" failed, errors in order:
    		error[0]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[1]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[2]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[3]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[4]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[5]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[6]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[7]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[8]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[9]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[10]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[11]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[12]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[13]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[14]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[15]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[16]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[17]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[18]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[19]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[20]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[21]: retrier timeout
    --- FAIL: TestAvailabilitySet/AvailabilitySet_Delete (355.08s)
    	retrier.go:61: &{{{{0 0} 0 0 0 0} [] {0xc42014c3c0} false false false false false 0xc42008f2c0 2 TestAvailabilitySet/AvailabilitySet_Delete {63625719651 586571493 0xace8c0} 1466 0xc42000c9c0 0xc42000ca20 []} true 0xc4200710e0} 
    		
    		work "Shell.Run:./testdata/create_avail_set.sh" failed, errors in order:
    		error[0]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[1]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[2]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[3]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[4]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[5]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[6]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[7]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[8]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[9]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[10]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[11]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[12]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[13]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[14]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[15]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[16]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[17]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[18]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[19]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[20]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[21]: retrier timeout
==================
WARNING: DATA RACE
Write at 0x00c4201580c0 by goroutine 30:
  os.(*file).close()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/os/file_unix.go:143 +0x124
  os.(*File).Close()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/os/file_unix.go:132 +0x55
  github.com/NeowayLabs/klb/tests/lib/log.newFile.func1()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/log/log.go:55 +0x41
  runtime.call32()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/runtime/asm_amd64.s:479 +0x4b
  testing.(*common).Fatal()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/testing/testing.go:490 +0x7c
  github.com/NeowayLabs/klb/tests/lib/retrier.(*Retrier).Run()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/retrier/retrier.go:61 +0x808
  github.com/NeowayLabs/klb/tests/lib/azure/fixture.(*ResourceGroup).Delete()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/azure/fixture/resourcegroup.go:67 +0x228
  github.com/NeowayLabs/klb/tests/azure_test.testResourceGroupCreate.func1()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/azure/resourcegroup_test.go:34 +0xf9
  runtime.call64()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/runtime/asm_amd64.s:480 +0x4b
  testing.(*common).Fatal()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/testing/testing.go:490 +0x7c
  github.com/NeowayLabs/klb/tests/lib/retrier.(*Retrier).Run()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/retrier/retrier.go:61 +0x808
  github.com/NeowayLabs/klb/tests/lib/nash.(*Shell).Run()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/nash/nash.go:55 +0x236
  github.com/NeowayLabs/klb/tests/azure_test.testResourceGroupCreate()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/azure/resourcegroup_test.go:42 +0x40c
  testing.tRunner()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/testing/testing.go:610 +0xc9

Previous read at 0x00c4201580c0 by goroutine 145:
  os.(*File).write()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/os/file_unix.go:249 +0x8a
  os.(*File).Write()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/os/file.go:142 +0x6f
  log.(*Logger).Output()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/log/log.go:166 +0x397
  log.(*Logger).Printf()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/log/log.go:173 +0x8f
  github.com/NeowayLabs/klb/tests/lib/retrier.retryUntilDone.func1()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/retrier/retrier.go:80 +0xf8

Goroutine 30 (running) created at:
  testing.(*T).Run()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/testing/testing.go:646 +0x52f
  github.com/NeowayLabs/klb/tests/azure_test.TestResourceGroup()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/azure/resourcegroup_test.go:77 +0x6b
  testing.tRunner()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/testing/testing.go:610 +0xc9

Goroutine 145 (running) created at:
  github.com/NeowayLabs/klb/tests/lib/retrier.retryUntilDone()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/retrier/retrier.go:82 +0xe1
  github.com/NeowayLabs/klb/tests/lib/retrier.(*Retrier).Run()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/retrier/retrier.go:49 +0x1f2
  github.com/NeowayLabs/klb/tests/lib/azure/fixture.(*ResourceGroup).Delete()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/azure/fixture/resourcegroup.go:67 +0x228
  github.com/NeowayLabs/klb/tests/azure_test.testResourceGroupCreate.func1()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/azure/resourcegroup_test.go:34 +0xf9
  runtime.call64()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/runtime/asm_amd64.s:480 +0x4b
  testing.(*common).Fatal()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/testing/testing.go:490 +0x7c
  github.com/NeowayLabs/klb/tests/lib/retrier.(*Retrier).Run()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/retrier/retrier.go:61 +0x808
  github.com/NeowayLabs/klb/tests/lib/nash.(*Shell).Run()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/lib/nash/nash.go:55 +0x236
  github.com/NeowayLabs/klb/tests/azure_test.testResourceGroupCreate()
      /home/travis/gopath/src/github.com/NeowayLabs/klb/tests/azure/resourcegroup_test.go:42 +0x40c
  testing.tRunner()
      /home/travis/.gimme/versions/go1.7.linux.amd64/src/testing/testing.go:610 +0xc9
==================
--- FAIL: TestResourceGroup (0.00s)
    --- FAIL: TestResourceGroup/Delete (301.09s)
    	retrier.go:61: &{{{{0 0} 0 0 0 0} [] {0xc420119440} false false false false false 0xc42008f500 2 TestResourceGroup/Delete {63625719651 582311023 0xace8c0} 1269 0xc420144120 0xc420144180 []} true 0xc4200710e0} 
    		
    		work "Shell.Run:./testdata/create_resource_group.sh" failed, errors in order:
    		error[0]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[1]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[2]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[3]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[4]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[5]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[6]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[7]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[8]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[9]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[10]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[11]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[12]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[13]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[14]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[15]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[16]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[17]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[18]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[19]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[20]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[21]: retrier timeout
    --- FAIL: TestResourceGroup/Create (612.10s)
    	retrier.go:61: &{{{{0 0} 0 0 0 0} [] {0xc4201192c0} false false false false false 0xc42008f500 2 TestResourceGroup/Create {63625719651 584679245 0xace8c0} 1454 0xc420144060 0xc4201440c0 []} true 0xc4200710e0} 
    		
    		work "Shell.Run:./testdata/create_resource_group.sh" failed, errors in order:
    		error[0]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[1]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[2]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[3]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[4]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[5]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[6]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[7]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[8]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[9]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[10]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[11]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[12]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[13]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[14]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[15]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[16]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[17]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[18]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[19]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[20]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[21]: error: exit status 8, executing script: ./testdata/create_resource_group.sh
    		error[22]: retrier timeout
    	retrier.go:61: &{{{{0 0} 0 0 0 0} [9 114 101 116 114 105 101 114 46 103 111 58 54 49 58 32 38 123 123 123 123 48 32 48 125 32 48 32 48 32 48 32 48 125 32 91 93 32 123 48 120 99 52 50 48 49 49 57 50 99 48 125 32 102 97 108 115 101 32 102 97 108 115 101 32 102 97 108 115 101 32 102 97 108 115 101 32 102 97 108 115 101 32 48 120 99 52 50 48 48 56 102 53 48 48 32 50 32 84 101 115 116 82 101 115 111 117 114 99 101 71 114 111 117 112 47 67 114 101 97 116 101 32 123 54 51 54 50 53 55 49 57 54 53 49 32 53 56 52 54 55 57 50 52 53 32 48 120 97 99 101 56 99 48 125 32 49 52 53 52 32 48 120 99 52 50 48 49 52 52 48 54 48 32 48 120 99 52 50 48 49 52 52 48 99 48 32 91 93 125 32 116 114 117 101 32 48 120 99 52 50 48 48 55 49 48 101 48 125 32 10 9 9 10 9 9 119 111 114 107 32 34 83 104 101 108 108 46 82 117 110 58 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 34 32 102 97 105 108 101 100 44 32 101 114 114 111 114 115 32 105 110 32 111 114 100 101 114 58 10 9 9 101 114 114 111 114 91 48 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 49 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 50 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 51 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 52 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 53 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 54 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 55 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 56 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 57 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 49 48 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 49 49 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 49 50 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 49 51 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 49 52 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 49 53 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 49 54 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 49 55 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 49 56 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 49 57 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 50 48 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 50 49 93 58 32 101 114 114 111 114 58 32 101 120 105 116 32 115 116 97 116 117 115 32 56 44 32 101 120 101 99 117 116 105 110 103 32 115 99 114 105 112 116 58 32 46 47 116 101 115 116 100 97 116 97 47 99 114 101 97 116 101 95 114 101 115 111 117 114 99 101 95 103 114 111 117 112 46 115 104 10 9 9 101 114 114 111 114 91 50 50 93 58 32 114 101 116 114 105 101 114 32 116 105 109 101 111 117 116 10] {0xc4201192c0} false true false true false 0xc42008f500 2 TestResourceGroup/Create {63625719651 584679245 0xace8c0} 1454 0xc420144060 0xc4201440c0 []} true 0xc4200710e0} 
    		
    		work "ResourceGroup.Delete" failed, errors in order:
    		error[0]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[1]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[2]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[3]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[4]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[5]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[6]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[7]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[8]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[9]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[10]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[11]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[12]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[13]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[14]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[15]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[16]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[17]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[18]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[19]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[20]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[21]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[22]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[23]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[24]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[25]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[26]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[27]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[28]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[29]: resources.GroupsClient#Delete: Failure responding to request: StatusCode=404 -- Original Error: autorest/azure: Service returned an error. Status=404 Code="ResourceGroupNotFound" Message="Resource group 'klb-resgroup-tests-904321' could not be found."
    		error[30]: retrier timeout
--- FAIL: TestVM (0.00s)
    --- FAIL: TestVM/VM_Create (1531.96s)
    	retrier.go:61: &{{{{0 0} 0 0 0 0} [] {0xc42008fd40} false false false false false 0xc42008f800 2 TestVM/VM_Create {63625719651 578299760 0xace8c0} 1849 0xc420066c60 0xc420066cc0 []} true 0xc4200710e0} 
    		
    		work "Shell.Run:./testdata/create_avail_set.sh" failed, errors in order:
    		error[0]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[1]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[2]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[3]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[4]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[5]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[6]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[7]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[8]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[9]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[10]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[11]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[12]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[13]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[14]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[15]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[16]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[17]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[18]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[19]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[20]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[21]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[22]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[23]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[24]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[25]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[26]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[27]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[28]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[29]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[30]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[31]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[32]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[33]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[34]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[35]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[36]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[37]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[38]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[39]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[40]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[41]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[42]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[43]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[44]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[45]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[46]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[47]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[48]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[49]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[50]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[51]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[52]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[53]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[54]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[55]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[56]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[57]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[58]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[59]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[60]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[61]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[62]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[63]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[64]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[65]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[66]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[67]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[68]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[69]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[70]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[71]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[72]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[73]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[74]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[75]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[76]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[77]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[78]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[79]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[80]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[81]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[82]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[83]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[84]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[85]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[86]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[87]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[88]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[89]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[90]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[91]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[92]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[93]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[94]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[95]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[96]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[97]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[98]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[99]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[100]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[101]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[102]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[103]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[104]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[105]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[106]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[107]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[108]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[109]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[110]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[111]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[112]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[113]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[114]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[115]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[116]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[117]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[118]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[119]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[120]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[121]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[122]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[123]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[124]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[125]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[126]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[127]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[128]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[129]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[130]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[131]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[132]: error: exit status 8, executing script: ./testdata/create_avail_set.sh
    		error[133]: retrier timeout
FAIL
FAIL	github.com/NeowayLabs/klb/tests/azure	1531.981s
make: *** [test] Error 1
/home/travis/build.sh: line 162:  4512 Terminated              travis_jigger $! $timeout $cmd
@katcipis katcipis added the bug label Mar 21, 2017
@katcipis katcipis self-assigned this Mar 21, 2017
@katcipis
Copy link
Contributor Author

The problem is here:

	var errs []error
	for {
		// buffered channel avoid goroutine leak
		result := make(chan error, 1)

		go func() {
			l.Printf("retrier: executing work: %s", name)
			result <- work()
		}()

		select {
		case res := <-result:
			{
				if res == nil {
					return nil
				}
				l.Printf("%s: got error: %s", name, res)
				errs = append(errs, res)
			}
		case <-ctx.Done():
			{
				l.Printf("retrier: %s: timeouted, returning all errors", name)
				return append(
					errs,
					errors.New("retrier timeout"),
				)
			}
		}
		// Sometimes we geet an error like: Number of write requests
		// for subscription 'x'
		// exceeded the limit of '1200' for time interval '01:00:00'.
		// Please try again after '5' minutes
		// Before trying again lets backoff a little.
		time.Sleep(backoff)
	}

To prevent Close being called at the same time as a Write I would have to wait all pending work jobs that are running before cancelling the test (and eventually calling tearDown).

The problem is that I do this on purpose exactly to guarantee that if some work goes haywire I guarantee that our teardown function WILL be executed, guaranteeing that resources are ALWAYS cleanedup, even when something locks up forever.

Now I can't think on a better solution then implementing a concurrency safe logger that prevents Write/Read being called concurrently with Close.

There is a discussion about this, and solving this on the os.File don't make sense, but on our particular case it makes sense:

golang/go#10001

@katcipis
Copy link
Contributor Author

After talking with @tiago4orion it seems like a good idea to just leak the fd and not close it at all :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@katcipis katcipis

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@katcipis