There is a great post about running TFS Builds with Chutzpah at http://blogs.msdn.com/b/visualstudioalm/archive/2012/07/09/javascript-unit-tests-on-team-foundation-service-with-chutzpah.aspx.  But I am running into an issue with it not finding any tests.

No test is available in C:\Builds\4\..\Scripts\Test\site\tests.  Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.

I checked on the server, and the files are there, any ideas?

Here is the real error that is coming back.

Error: Method 'ToXml' in type 'Chutzpah.VS2012.TestAdapter.ChutzpahAdapterSettings' from assembly 'Chutzpah.VS2012.TestAdapter, Version=2.1.0.162, Culture=neutral, PublicKeyToken=1ca802c37ffe1896' does not have an implementation.

That is a really strange error message. By itself it doesn't make much sense to me. Is there any stack trace or other information with that error?

Can you setup a simple solution that will let me reproduce the issue?

Unfortunately, there is no stack trace from the Build Server in this case.  I have run the Chutzpah extension locally on the same solution and it works fine.

I found that the ToXml error that I was getting was only coming back when I tried to edit the build definition.  Here is the whole message when I try to edit the definition.

Summary: There were 0 failures, 1 errors and 1 warnings loading custom activities and services.

Error: Method 'ToXml' in type 'Chutzpah.VS2012.TestAdapter.ChutzpahAdapterSettings' from assembly 'Chutzpah.VS2012.TestAdapter, Version=2.1.0.162, Culture=neutral, PublicKeyToken=1ca802c37ffe1896' does not have an implementation.

Warning: Could not load file or assembly 'file:///C:\Users\...\AppData\Local\Temp\VSTFSBuild\...\...\phantomjs.exe' or one of its dependencies. The module was expected to contain an assembly manifest.

Here is a link to my SkyDrive with a solution that contains a simple test file to reproduce the issue.

http://sdrv.ms/UMbUN7

We are having very similar issues. After following the same blog post mentioned above, we get the same two errors (Method 'ToXml' and 'phantonjs.exe or one of it's dependencies...'). We've configured two test runs in the build definition - one for test*.dll and second one for **\*.js - and from a practical standpoint, it seems the test runner simply does not execute the second run, only the first.

We're kind of stuck at this point...

That is odd. I have never run it in TFS Build before so I need to test this out and see what is going on.  

Once I get some time I will dig into this further.

That would great.  JavaScript is a bit part of the projects we do now, and with continuous delivery, we want to make sure tests can be run automatically on our build server.  Thanks again.

I am unable to reproduce your issue. I followed the steps outlines in http://blogs.msdn.com/b/visualstudioalm/archive/2012/07/09/javascript-unit-tests-on-team-foundation-service-with-chutzpah.aspx?CommentPosted=true#commentmessage and I was able to run the tests successfully as part of a integrated build.

Here are the steps I followed:

1. Unziped the solution you attached

2. Downloaded the Chutzpah Test Adapter from http://visualstudiogallery.msdn.microsoft.com/f8741f04-bae4-4900-81c7-7c9bfb9ed1fe

3. I unziped the test adapter contents into a folder named Chutzpah which I included in the solution.

4. I followed the steps in the blog post to set the Chutzpah folder as the "Version Control Path to custom Assemblies"

5. I updated the test assembly file specification to **\tests.js

6. I *did* not set the tests.js to "Copy Always" as the blog said. When I did this it ran that file multiple times for some reason.

7. After these steps I was able to checkin and run the build successfully.

Did you procedure differ?

Did you copy all files from Chutzpah to your build assemblies folder, or only the ones outlined in the blog post?

When you ran the build, did it show you the JavaScript tests in the test results?  The build ran successfully for me to, but it would not show any results for the tests.  When I open the build definition it shows the errors described above.  Thanks for looking into this.

1. I copied *all* the files from the .vsix.

2. It did report the test results. I then updated one of the tests to have a failing assert and it detected that correctly also.

It seems that rexmorgan and I have the same problem.  I know that I put in two test runs as well, one for Test.*dll and one for **\*.js.  Does that fail your results if you add the additional test run?

I started over with a fresh solution & build definition, repeated the steps above* and still seeing the same behavior - the test run which specifies .js simply does not fire. If we have two test runs (one for .dll, one for .js), we get "1 test run completed" - with the .NET unit test results shown. If the .dll test run is removed, we get "No Test Results".

*Which is more or less what we had previously (mainly just keeping the Chutzpah files in a central location instead of inside the actual test project).

@rexmorgan

The solution I posted at http://sdrv.ms/UMbUN7 only contains JavaScript unit tests.  Does that solution work for you?

The setup I did has only 1 test run and it was looking just for **\tests.js. I can try later with adding multiple runs. 

@mmanela

My test runs specify **\*.js.  It could be that when it does not receive a specific file to test it fails.  Or maybe if it picks up any files that are not unit tests, it fails.

I repeated the steps precisely and also introduced the additional conditions that makes our setup different and they both fail in exactly the same way (js tests simply do not run) - do there must be an environmental issue that's different. Are you running .NET 4.5 on the build server?

I ran my build on the hosted tfs (tfs.visualstudio.com). I am not sure if the build agents there are different.

It seems like it could be related to an assembly difference in Microsoft.VisualStudio.TestPlatform.ObjectModel.dll.

The problem is at VS2012.TestAdapter\ChutzpahAdapterSettings.cs in the ToXml method.  It could be the override does not match that of the loaded assembly of Microsoft.VisualStudio.TestPlatform.ObjectModel.dll.

That seems plausible. What version of VS/TFS are you using?

We are using TFS 2012 and VS 2012 with .NET 4.5.

When I put Microsoft.VisualStudio.TestPlatform.ObjectModel.dll in the custom assemblies folder, the Build Definition gave the error that that assembly was already loaded, so I removed it from the folder.

We removed that dll thinking the same, no difference. Also on 2012 / 4.5

@mmanela

Do you get an error with the "Microsoft.VisualStudio.TestPlatform.ObjectModel.dll" assembly when you try to edit your build definition?

No I don't get any error.

I'm getting the same exact problem.  It looks like I have a newer version of the Microsoft.VisualStudio.TestPlatform.ObjectModel.dll on my machine.  My machine has version 11.0.51106.1 in  C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow.  Whereas the adapter seems to be using version 11.0.50907.0.  It looks like the signature for ToXml hasn't changed from what you've implemented so must be a DLL hell problem?  Maybe if if the app.config allowed a binding redirect?

<runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="Microsoft.VisualStudio.TestPlatform.ObjectModel" publicKeyToken="whatever" culture="neutral" />
        <bindingRedirect oldVersion="11.0.50907.0" newVersion="11.0.51106.1" />
      </dependentAssembly>