VS 2013 Test Explorer shows no JS tests


This is a repost of a discussion I started.

I'm using chutzpah test adapter for the test explorer 3.0.1. When I start VS and go to the Test Explorer, I can see the output in the "Tests" console. It shows ------ Discover test started ------ and then has a timeout for every one of my test files: "Error: Timeout occured when executing test file".
I do have the chutzpah logging enabled, and it spits out lots of good stuff, but every discovered test file ends with phantom complaining that it timed out.

I was considering upgrading to 3.2.1, but when I installed it locally, I started getting failures when running tests from chutzpah context menu or console. The tests themselves succeeded, but they started failing because of functionality in a document.ready function...which previously didn't seem to be executed during test execution. I believe the two problems are related. I was not worried about the js tests not showing up in test explorer because they were still executable from the context menu or console. But if 3.2.1 is going to start failing because the execution of the document.ready function, then I realize I need a better picture into what's really going on.

The sample I'm attaching does not have any references to a document.ready function, so I think that I may be off on that thought...but it is still a problem with 3.2.1 where the execution of the document.ready function causes errors. It seems that in previous versions (at least in 3.0.1), the document.ready was not being run).

I included the chutzpah.log that was generated by the test discovery process.

Any help you can provide is much appreciated.

file attachments


mmanela wrote Mar 21, 2014 at 5:21 PM

When I run your test from the command line it seems to work fine:
 chutzpah.console.bat C:\Users\mmanela\Desktop\ChutzpahRepro\WebApplication1\Scripts\Tests\UtilitiesTests.js
results in

Chutzpah console test runner  (64-bit .NET 4.0.30319.34011)
Copyright (C) 2013 Matthew Manela (http://matthewmanela.com).

chutzpah.dll:     Version

File: C:\Users\mmanela\Desktop\ChutzpahRepro\WebApplication1\Scripts\Tests\UtilitiesTests.js
     3 total, 0 failed, took 0.08 seconds

Tests complete: 3
Code Coverage Results     Average Coverage: 44%
     31% for C:\Users\mmanela\Desktop\ChutzpahRepro\WebApplication1\Scripts\Utilities.js
     100% for C:\Users\mmanela\Desktop\ChutzpahRepro\WebApplication1\Scripts\Tests\UtilitiesTests.js

=== 3 total, 0 failed, took 1.64 seconds ===
Does that work for you?

jmicksch wrote Mar 21, 2014 at 5:42 PM

Yes, running from chutzpah console command line works for the sample project. The document.ready problem is not expressed in this sample. I think that if I can figure out why the VS test explorer is failing to show these tests, I will be able to figure out the document.ready problem myself...it very well could be our bad practices.

My concern here is that the 3.0.1 test explorer fails to display the tests, complaining about timeout.

As a side note, is there anything about the chutzpah test execution that significantly changed between 3.0.1 and 3.2.1? If not, then I may end up posting another issue related to the document.ready problem.

I did create an extremely simple web project that does discover the test and expose it in the test explorer. I don't understand what's different between the 2.

mmanela wrote Mar 21, 2014 at 6:17 PM

I am able to repro your issue. It looks like codecoverage is the issue. If you set it to false it works fine.
I am not sure why it would work outside of TestExplorer but not when run from it. That make very little sense to me so I need to investigate further.

jmicksch wrote Mar 21, 2014 at 6:32 PM

Thanks for narrowing it down to that setting. Please let me know what you discover. :)

mmanela wrote Mar 22, 2014 at 6:15 AM

I found the issue. When Chutzpah runs as part of the test explorer it has two mode: discovery and execution. In discovery mode Chutzpah tries to be faster and runs the files but not the bodies of the tests. This is causing an issue though with what blanket.js is doing with code coverage.

I will fix this for the next release.

jmicksch wrote Mar 24, 2014 at 4:30 PM

Thanks so much for your research into this issue. I look forward to getting the fixed version into place.

SeriousM wrote May 8, 2014 at 2:32 PM

Any updates on this?

mmanela wrote May 8, 2014 at 3:29 PM

Working on the fix and plan to do a new release in a couple weeks at the latest.

mmanela wrote May 17, 2014 at 3:29 PM

Released in 3.2.2