Developer docs on how to manually test on windows

[springmeyer]

Currently it is ideal to manually test the Windows app before releasing a new build (due to #1208 (comment)). By "manually test" we mean just run the installer on a windows machine and make sure the application starts and is generally usable.

So, we need some better docs about how anyone can quickly do this.

My opinion is that testing on AWS is the fastest (to avoid anyone ever having to download a Windows VM) so I propose adding a wiki page (with images/screenshots) that walks through:

• What AMI to test with, how to launch it through the AWS GUI
• How to open up the windows firewall to be able to download Studio via IE
• How to launch Studio to ensure it starts okay
• How to debug common problems (e.g. seeing and stopping potential zombie processes in the task manager)

Once done this could be linked to from https://github.com/mapbox/mapbox-studio/blob/mb-pages/CONTRIBUTING.md so that anyone cutting a new release can do this if necessary (if any changes have been made to the shell portion of mapbox-studio).

I think we should document this against a vanilla windows AWS AMI, but in the future to make it easier it would be very nice to have the AMI pre-configured so that:

• Chrome is already installed (and therefore the firewall is opened enough to actually install chrome and other stuff from the net)
• Studio downloads link is the default page, etc
• No visual studio version is installed (so that we can ensure the vccredist.exe gets installed on the fly correctly)

Game plan:

• @amyleew and @springmeyer collaborate on documenting this
• @GretaCB tries out our docs for the next release that includes Mapnik omnivore refactor #1199.

/cc @BergWerkGIS for ideas on what a customized AMI for easy testing could look like.

/cc @rclark @yhahn and @camilleanne @bsudekum for spiritual support

[yhahn]

@springmeyer hrm, at least in my experience in doing Windows development with Studio I've muched preferred having a VM. Testing on an EC2 may make sense if you're just trying to see if things work but as soon as you hit a bug and you want to work on fixing it being able to work locally seems like a perk.

Here's the gist I shared with @camilleanne, @samanpwbb and @bsudekum for getting set up on a windows vm for node development:

• https://gist.github.com/yhahn/604a1858aa500e708ad6

[springmeyer]

@yhahn true. I guess what I'm proposing is a really low cost method of just testing startup and basic functioning and not a solution to help do deep-dive debugging.

However personally I only use an EC2 now for deep-dive debugging because I have one set up with sublime and other tools and setting up one locally has too much mental resistance.

Anyway, just synced up with @bsudekum on this. We spitballed this general flow:

• ./scripts/build-travis.sh starts responding to commit messages with something like -m [test-win {hash or branch}]
• the windows package is made and posted to s3 like when you do -m [publish {hash}]
• then we restore a saved instance (this would be better than a fresh instance since old studio installs will be there which simulates a user environment more)
• a per-boot script runs and downloads the packaged build
• travis dumps the url to the new ec2 instance in the logs, you grab it and plug in into Microsoft Remote Desktop
• you login via MRD and test the app
• travis kills the machine once it hits its 50 min limit (other ideas welcome)

[yhahn]

👍 to EC2s for CI like this flow -- I wonder though whether this could come after the victory lap @bsudekum was planning on for abstracting a lot of our app building/testing CI to be shared between tilemill + studio?

[wilhelmberg]

firewall is opened enough to actually install chrome and other stuff from the net

No need to turn off firewall or fiddle with IE options/network zones.
Just disable IE Enhanced Security in Server Manager:
http://blog.blksthl.com/2012/11/28/how-to-disable-ie-enhanced-security-in-windows-server-2012/

then we restore a saved instance (this would be better than a fresh instance since old studio installs will be there which simulates a user environment more)

I think two machines would be good:

• one vanilla AMI: to see if it works on a clean machine
• one saved instance: to simulate the update process

travis kills the machine once it hits its 50 min limit (other ideas welcome)

I think there should be some kind of flag we can pass in, if the machine should power down after some automatic tests or if you want to log in remotely.
Would be nasty if the machine is powered down by travis when you are in the middle of your testing session.

I must admit that manually shutting down a machine with Windows 8.1 and Windows Server 2012 is a lot more hassle than it used to be and it should be:
Where is Shutdown button in Windows Server 2012