chutzpah Wiki Rss Feed

Updated Wiki: Home

mmanela — Sun, 19 May 2013 17:16:50 GMT

Welcome to Chutzpah - A JavaScript Test Runner

Pronunciation: hutz·pah

Chutzpah is an open source JavaScript test runner which helps you integrate JavaScript unit testing into your website. It enables you to run JavaScript unit tests from the command line and from inside of Visual Studio. It also supports running in the TeamCity continuous integration server.

Chutzpah supports both the QUnit and Jasmine testing frameworks.
Chutzpah uses the PhantomJS headless browser to run your tests.

Notice: Known issue and workaround if you have both NVidia and Intel video drivers

Quick Downloads

Requirements

General

QUnit or Jasmine

Visual Studio integration

Visual Studio 2010
Visual Studio 2012

Features

Runs JavaScript unit tests from the command line

Run JavaScript unit tests from inside of Visual Studio

Right click menu to run tests

Shows test results in Error window

Shows test results in Ouput window

Integrates into VS 2012 Unit Test Explorer

New Comment on "Chutzpah.json Settings File"

mmanela — Sun, 28 Apr 2013 18:12:15 GMT

@saguir, That is a feature I would like to add and plan to in a future release.

Updated Wiki: Documentation

mmanela — Sun, 28 Apr 2013 17:38:18 GMT

Chutzpah Documentation

Using Chutzpah

Contributing to Chutzpah

Updated Wiki: Running against a remote url

mmanela — Sun, 28 Apr 2013 17:37:28 GMT

Running again a remote url

As of version 2.4 Chutzpah supports executing a remote test harness. If you have a html test file on a remote server (like http://somesite.com/test.html) you can tell Chutzpah to execute that file directly

chutzpah.console.exe http://somesite.com/test.html

Note: Chutzpah does not support running anything but .html files remotely. If you want to run Chutzpah directly on a .js, .ts or .coffee file they must be local on disk.

Updated Wiki: Running against a remote url

mmanela — Sun, 28 Apr 2013 17:37:00 GMT

Running again a remote url

chutzpah.console.exe http://somesite.com/test.html

Note: Chutzpah does not support running anything but .html files remotely. If you want to run Chutzpah directly on a .js, .ts or .coffee file they must be local on disk.

Updated Wiki: Running against a remote url

mmanela — Sun, 28 Apr 2013 17:36:50 GMT

Running again a remote url

chutzpah.console.exe http://somesite.com/test.html

Note: Chutzpah does not support running anything but .html files remotely. If you want to run Chutzpah directly on a .js, .ts or .coffee file they must be local on disk.

Updated Wiki: Chutzpah File References

mmanela — Sun, 28 Apr 2013 17:33:33 GMT

Chutzpah File References

One of the most important steps to having Chutzpah successfully run your tests is to set up your file references correctly. You accomplish this by adding a special file reference comment at the top of your files:

/// <reference path="someDependency.js" />

When Chutzpah sees this it will include that file in the generated test harness as well as recursively scan that file to find all of its references.

If you want to include a whole directory of files you can just specify a folder:

/// <reference path="some/folder" />

Chutzpah will the recursively scan that folder to find all references.

Excluding References

The reference syntax is also used by VS to give intellisense in JS files. This leads to an issue since sometimes you may reference a version of a file just for intellisense but you do not want Chutzpah to copy it. In this case you can include the chutzpah-exclude attribute to have it ignored by Chutzpah:

/// <reference path="someDependency.js" chutzpah-exclude="true" />

Alternate Reference Syntax

The reference syntax is also used by the TypeScript compiler for referencing files. This can cause an issue since Chutzpah expects you to reference your implementation files and TypeScript excepts you to reference your definition files. To mitigate this issue you can use a Chutzpah specific version of the reference command:

/// <chutzpah_reference path="someDependency.js" />

This works the same as the other syntax except that TypeScript (and VS) will ignore it.

Root Reference Path Mode

If you write a reference path like

/// <reference path="someDependency.js" />

then Chutzpah searched for that dependency relative to the current file. However, you can also specify a reference path like

/// <reference path="/someDependency.js" />

The beginning forward slash will tell Chutzpah to find that file at the root of the drive the file is in (e.g C:\). This behavior is configurable using a chutpah.json file. For example if you want the rooted reference path to be relative to a different directory you can place the a chutzpah.json file in that directory with the following content:

{
    "RootReferencePathMode":"SettingsFileDirectory"
}

Click this link to lean more of the Chutzpah.json Settings File

Updated Wiki: Chutzpah File References

mmanela — Sun, 28 Apr 2013 17:33:06 GMT

Chutzpah File References

/// <reference path="someDependency.js" />

/// <reference path="some/folder" />

Chutzpah will the recursively scan that folder to find all references.

Excluding References

/// <reference path="someDependency.js" chutzpah-exclude="true" />

Alternate Reference Syntax

/// <chutzpah_reference path="someDependency.js" />

This works the same as the other syntax except that TypeScript (and VS) will ignore it.

Root Reference Path Mode

If you write a reference path like

/// <reference path="someDependency.js" />

then Chutzpah searched for that dependency relative to the current file. However, you can also specify a reference path like

/// <reference path="/someDependency.js" />

{
    "RootReferencePathMode":"SettingsFileDirectory"
}

You can read more about the Chutzpah.json Settings File

Updated Wiki: Chutzpah.json Settings File

mmanela — Sun, 28 Apr 2013 17:31:42 GMT

Using the Chutzpah.json Settings File

The Chutzpah settings file is an .json file that you can place anywhere in your testing directories to inform Chutzpah of test specific settings. The Chuzpah settings file is completely optional and if the file or any of its properties are omitted Chutzpah will use the default values.

When Chutzpah is run on a file (lets say test.js) it scans the current directory for a file named chutzpah.json. If it does not find one it will recursively traverse up the directory chain looking for one. This behavior provides a lot of flexibility since you can provide multiple chutzpah.json files in your tree and each one will work over its sub-folder. This lets you have different settings for different groups of tests.

The settings file is a text file with json content of the following format:

{
    "Framework": "qunit|jasmine",
    
    "TestFileTimeout": "<Timeout in milliseconds>",
    
    "TestHarnessLocationMode": "TestFileAdjacent|SettingsFileAdjacent|Custom",
    
    "TestHarnessDirectory": "<Path to a folder>",

    "TypeScriptCodeGenTarget" : "ES3|ES5",
    
    "RootReferencePathMode":"DriveRoot|SettingsFileDirectory",
    
    "CodeCoverageIncludes": [],
    
    "CodeCoverageExcludes": []
}

The chutzpah.json file supports the following properties

Framework – Determines what testing framework to use. This will override the other detection methods. This is helpful if Chutzpah is having trouble figuring out what your intended framework is. If left out Chutzpah will automatically try to detect the framework.

TestFileTimeout - The time to wait for tests in a file to finish in milliseconds

TestHarnessLocationMode – Determines where to place the generated html test harness files. The default mode is TestFileAdjacent which means the harness is placed in the same directory as the test file. SettingsFileAdjacent means it is placed in the same directory as the chutzpah.json file. Custom mode lets you specify a custom path to the directory.

TestHarnessDirectory – When TestHarnessLocationMode is set to Custom this is either the relative or absolute path to the directory where to place the test harness.

TypeScriptCodeGenTarget – When running on a TypeScript file this property will tell the TypeScript compiler what type of code to generate. ES3 is for ECMAScript 3 and ES5 is for ECMAScript 5.

RootReferencePathMode – This property determines what directory a rooted reference path refers to. In a test file Chutzpah parses the <reference path="" /> tags to determine your test dependencies. By default (the DriveRoot setting) if you write <reference path="\a.js" /> Chutzpah will look for a.js under C:\ (assuming you are running your test in the C drive). However, if you set this property to SettingsFileDirectory Chutpah will now look for a.js relative to the location of the chutzpah.json file. This is useful if you have a large project and want to minimize the number of long reference paths.

CodeCoverageIncludes - The collection code coverage file patterns to include in coverage. These are in glob format. If you specify none all files are included.

CodeCoverageExcludes - The collection code coverage file patterns to exclude in coverage. These are in glob format. If you specify none no files are excluded.

Examples:
This file tells Chutzpah to use QUnit and to place the test harness next to the chutzpah.json file:

{
    "Framework": "qunit",
    "TestHarnessLocationMode": "SettingsFileAdjacent"
}

This file tells Chutzpah to convert TypeScript to ES5 code and to place the test harness in a custom directory:

{
    "TypeScriptCodeGenTarget" : "ES5",
    "TestHarnessLocationMode": "Custom",
    "TestHarnessDirectory": "../Some/Folder"
}

When run with Code Coverage this file tells Chutzpah to exclude files named jquery.js and include files that have a suffix of query.js or are named dogs.js:

{
    "CodeCoverageExcludes": ["*\\jquery.js"],
    "CodeCoverageIncludes": ["*query.js", "*\\dog.js"]
}

Updated Wiki: Chutzpah File References

mmanela — Sun, 28 Apr 2013 17:21:30 GMT

Chutzpah File References

/// <reference path="someDependency.js" />

/// <reference path="some/folder" />

Chutzpah will the recursively scan that folder to find all references.

Excluding References

/// <reference path="someDependency.js" chutzpah-exclude="true" />

Alternate Reference Syntax

/// <chutzpah_reference path="someDependency.js" />

This works the same as the other syntax except that TypeScript (and VS) will ignore it.

Updated Wiki: Chutzpah File References

mmanela — Sun, 28 Apr 2013 17:19:33 GMT

Chutzpah File References

/// <reference path="someDependency.js" />

/// <reference path="some/folder" />

Chutzpah will the recursively scan that folder to find all references.

Excluding References

/// <reference path="someDependency.js" chutzpah-exclude="true" />

Alternate Reference Syntax

/// <chutzpah_reference path="someDependency.js" />

This works the same as the other syntax except that TypeScript (and VS) will ignore it.

Updated Wiki: Documentation

mmanela — Sun, 28 Apr 2013 17:07:33 GMT

Chutzpah Documentation

Using Chutzpah

Contributing to Chutzpah

Updated Wiki: coverage

mmanela — Wed, 24 Apr 2013 04:21:55 GMT

Code Coverage in Chutzpah

Chutzpah’s new code coverage features is built on top of the great open source library Blanket.js. When you run with code coverage enabled it will use Blanket.js to instrument the files under test and gather the coverage results. You can perform code coverage from the command line and both Visual Studio plugins.

Command Line

To get code coverage from the command line you pass the /coverage flag to chutzpah.console.exe.

chutzpah.console.exe /coverage /path test.js

After this command finishes you will have a two files placed in your current directory

_Chutzpah.coverage.html – The rendered html results
_Chutzpah.coverage.json – The raw json results

When you open the results html file you will see something like:

Visual Studio Context Menu Extension

The Visual Studio Context Menu Extension now contains a Show Code Coverage command which will perform code coverage and open the results in a browser.

Visual Studio 2012 Test Adapter Extension

For the Unit Test Explorer adapter for Visual Studio 2012 Chutzpah piggybacks on the existing “Analyze Code Coverage” menu option and uses it to run with code coverage and show the results in a browser. Ideally, Chutzpah would integrate into the code coverage reporting built into VS but I was unable to figure out a good way to plug in to that. The side effect is that when you run this command VS will open their results window first (with nothing in it) and then Chutzpah will launch its results in a browser.

NOTE: I have noticed some performance issues when running “Analyze Code Coverage for All Tests” command. I think it might be VS doing extra work for no reason since it is trying to generate its own report.

Configuration Code Coverage

Being able to run code coverage is not enough since in many cases the results will not be accurate. For example, if your project uses jQuery you do not expect your test to cover all the lines of jQuery’s source code (that would be one impressive test). Therefore you need to exclude files/directories from participating in code coverage. There are two way you can do that with Chutpah: command line arguments or a chutzpah.json file.

If you are running Chutzpah from the command line there are two additional arguments you can pass coverageIncludes and coverageExcludes. These are a comma separated list of files/paths you want to include or exclude from coverage. They can use a glob format where * matches zero or more characters and ? matches one character. For example:

Exclude files that ends in matt.js

chutzpah.console.exe /path test.js /coverage /coverageExcludes *matt.js

Include only dog.js and any file with a single character extension

chutzpah.console.exe /path test.js /coverage /coverageIncludes "*\dog.js, *.?"

Include *query.js but exclude jQuery.js

chutzpah.console.exe /path test.js /coverage /coverageIncludes *query.js /coverageExcludes *\jquery.js

The other option for configuring code coverage is the chutzpah.json file. There are two new setting which you can use CodeCoverageExcludes and CodeCoverageIncludes. These settings work in the same way as the their command line counterpart. The main difference is that you must escape backslashes.

{
  "CodeCoverageExcludes": ["*\\jquery.js"],
  "CodeCoverageIncludes": ["*query.js", "*\\dog.js"]
}

Updated Wiki: Command Line Options

mmanela — Wed, 24 Apr 2013 04:20:14 GMT

Chutzpah console test runner (64-bit .NET 4.0.30319.18033)
Copyright (C) 2013 Matthew Manela (http://matthewmanela.com).

usage: chutzpah.console [options]
usage: chutzpah.console <testFile> [options]

Valid options:
/silent : Do not output running test count
/teamcity : Forces TeamCity mode (normally auto-detected)
/wait : Wait for input after completion
/failOnScriptError : Return a non-zero exit code if any script errors occurs
/debug : Print debugging information
/openInBrowser : Launch the tests in the default browser
/timeoutMilliseconds : Amount of time to wait for a test file to finish before failing. (Default
s to 5000)
/parallelism [n] : Max degree of parallelism for Chutzpah. Defaults to number of CPUs + 1
: If you specify more than 1 the test output may be a bit jumbled
/path path : Adds a path to a folder or file to the list of test paths to run.
: Specify more than one to add multiple paths.
: If you give a folder, it will be scanned for testable files.
: (e.g. /path test1.html /path testFolder)
/file path : Alias for /path
/vsoutput : Print output in a format that the VS error list recognizes
/coverage : Enable coverage collection
/coverageIncludes pat : Only instrument files that match the given shell patterns (in glob format
), comma separated.
/coverageExcludes pat : Don't instrument files that match the given shell pattern (in glob format
), comma separated.
/compilercache : File where compiled scripts can be cached. Defaults to a file in the temp
directory.
/compilercachesize : The maximum size of the cache in Mb. Defaults to 32Mb.
/testMode : The mode to test in (All, Html, AllExceptHTML, TypeScript, CoffeeScript,
JavaScript)
/junit filename : output results to JUnit-style XML file

Updated Wiki: coverage

mmanela — Wed, 24 Apr 2013 04:17:31 GMT

Code Coverage in Chutzpah

Command Line

To get code coverage from the command line you pass the /coverage flag to chutzpah.console.exe.

chutzpah.console.exe /coverage /path test.jsAfter this command finishes you will have a two files placed in your current directory

_Chutzpah.coverage.html – The rendered html results
_Chutzpah.coverage.json – The raw json results

When you open the results html file you will see something like:

Visual Studio Context Menu Extension

The Visual Studio Context Menu Extension now contains a Show Code Coverage command which will perform code coverage and open the results in a browser.

Visual Studio 2012 Test Adapter Extension

Configuration Code Coverage

chutzpah.console.exe /path test.js /coverage /coverageExcludes *matt.js

Include only dog.js and any file with a single character extension

chutzpah.console.exe /path test.js /coverage /coverageIncludes "*\dog.js, *.?"

Include *query.js but exclude jQuery.js

chutzpah.console.exe /path test.js /coverage /coverageIncludes *query.js /coverageExcludes *\jquery.js

{
  "CodeCoverageExcludes": ["*\\jquery.js"],
  "CodeCoverageIncludes": ["*query.js", "*\\dog.js"]
}

Updated Wiki: coverage

mmanela — Wed, 24 Apr 2013 04:16:53 GMT

Code Coverage in Chutzpah

Command Line

To get code coverage from the command line you pass the /coverage flag to chutzpah.console.exe.

chutzpah.console.exe /coverage /path test.jsAfter this command finishes you will have a two files placed in your current directory

_Chutzpah.coverage.html – The rendered html results
_Chutzpah.coverage.json – The raw json results

When you open the results html file you will see something like:

Cannot resolve image macro, invalid image name or id.

Visual Studio Context Menu Extension

The Visual Studio Context Menu Extension now contains a Show Code Coverage command which will perform code coverage and open the results in a browser.

Cannot resolve image macro, invalid image name or id.

Visual Studio 2012 Test Adapter Extension

For the Unit Test Explorer adapter for Visual Studio 2012 Chutzpah piggybacks on the existing “Analyze Code Coverage” menu option and uses it to run with code coverage and show the results in a browser. Ideally, Chutzpah would integrate into the code coverage reporting built into VS but I was unable to figure out a good way to plug in to that. The side effect is that when you run this command VS will open their results window first (with nothing in it) and then Chutzpah will launch its results in a browser.

Cannot resolve image macro, invalid image name or id.

NOTE: I have noticed some performance issues when running “Analyze Code Coverage for All Tests” command. I think it might be VS doing extra work for no reason since it is trying to generate its own report.

Configuration Code Coverage

chutzpah.console.exe /path test.js /coverage /coverageExcludes *matt.js

Include only dog.js and any file with a single character extension

chutzpah.console.exe /path test.js /coverage /coverageIncludes "*\dog.js, *.?"

Include *query.js but exclude jQuery.js

chutzpah.console.exe /path test.js /coverage /coverageIncludes *query.js /coverageExcludes *\jquery.js

{
  "CodeCoverageExcludes": ["*\\jquery.js"],
  "CodeCoverageIncludes": ["*query.js", "*\\dog.js"]
}

Updated Wiki: Documentation

mmanela — Wed, 24 Apr 2013 04:14:41 GMT

Chutzpah Documentation

Using Chutzpah

Contributing to Chutzpah

Updated Wiki: Home

mmanela — Wed, 24 Apr 2013 04:08:53 GMT

Welcome to Chutzpah - A JavaScript Test Runner

Quick Downloads

Requirements

General

QUnit or Jasmine

Visual Studio integration

Visual Studio 2010

Features

Runs JavaScript unit tests from the command line

Run JavaScript unit tests from inside of Visual Studio

Right click menu to run tests

Shows test results in Error window

Shows test results in Ouput window

Integrates into VS 2012 Unit Test Explorer

Updated Wiki: Home

mmanela — Wed, 24 Apr 2013 04:08:42 GMT

Welcome to Chutzpah - A JavaScript Test Runner

Quick Downloads

Requirements

General

QUnit or Jasmine

Visual Studio integration

Visual Studio 2010

Features

Runs JavaScript unit tests from the command line

Run JavaScript unit tests from inside of Visual Studio

Right click menu to run tests

Shows test results in Error window

Shows test results in Ouput window

Integrates into VS 2012 Unit Test Explorer

Updated Wiki: Chutzpah.json Settings File

mmanela — Wed, 24 Apr 2013 04:04:00 GMT

Using the Chutzpah.json Settings File

{
    "Framework": "qunit|jasmine",
    
    "TestFileTimeout": "<Timeout in milliseconds>",
    
    "TestHarnessLocationMode": "TestFileAdjacent|SettingsFileAdjacent|Custom",
    
    "TestHarnessDirectory": "<Path to a folder>",

    "TypeScriptCodeGenTarget" : "ES3|ES5",
    
    "RootReferencePathMode":"DriveRoot|SettingsFileDirectory",
    
    "CodeCoverageIncludes": [],
    
    "CodeCoverageExcludes": []
}

{
    "Framework": "qunit",
    "TestHarnessLocationMode": "SettingsFileAdjacent"
}

This file tells Chutzpah to convert TypeScript to ES5 code and to place the test harness in a custom directory:

{
    "TypeScriptCodeGenTarget" : "ES5",
    "TestHarnessLocationMode": "Custom",
    "TestHarnessDirectory": "../Some/Folder"
}

When run with Code Coverage this file tells Chutzpah to exclude files named jquery.js and include files that have a suffix of query.js or are named dogs.js:

{
    "CodeCoverageExcludes": ["*\\jquery.js"],
    "CodeCoverageIncludes": ["*query.js", "*\\dog.js"]
}

chutzpah Wiki Rss Feed

Updated Wiki: Home

Welcome to Chutzpah - A JavaScript Test Runner

Recent News

Quick Downloads

Contents

Requirements

Features

Runs JavaScript unit tests from the command line

Run JavaScript unit tests from inside of Visual Studio

New Comment on "Chutzpah.json Settings File"

Updated Wiki: Documentation

Chutzpah Documentation

Using Chutzpah

Contributing to Chutzpah

Updated Wiki: Running against a remote url

Running again a remote url

Updated Wiki: Running against a remote url

Running again a remote url

Updated Wiki: Running against a remote url

Running again a remote url

Updated Wiki: Chutzpah File References

Chutzpah File References

Excluding References

Alternate Reference Syntax

Root Reference Path Mode

Updated Wiki: Chutzpah File References

Chutzpah File References

Excluding References

Alternate Reference Syntax

Root Reference Path Mode

Updated Wiki: Chutzpah.json Settings File

Using the Chutzpah.json Settings File

Updated Wiki: Chutzpah File References

Chutzpah File References

Excluding References

Alternate Reference Syntax

Updated Wiki: Chutzpah File References

Chutzpah File References

Excluding References

Alternate Reference Syntax

Updated Wiki: Documentation

Chutzpah Documentation

Using Chutzpah

Contributing to Chutzpah

Updated Wiki: coverage

Code Coverage in Chutzpah

Command Line

Visual Studio Context Menu Extension

Visual Studio 2012 Test Adapter Extension

Configuration Code Coverage

Updated Wiki: Command Line Options

Updated Wiki: coverage

Code Coverage in Chutzpah

Command Line

Visual Studio Context Menu Extension

Visual Studio 2012 Test Adapter Extension

Configuration Code Coverage

Updated Wiki: coverage

Code Coverage in Chutzpah

Command Line

Visual Studio Context Menu Extension

Visual Studio 2012 Test Adapter Extension

Configuration Code Coverage

Updated Wiki: Documentation

Chutzpah Documentation

Using Chutzpah

Contributing to Chutzpah

Updated Wiki: Home

Welcome to Chutzpah - A JavaScript Test Runner

Recent News

Quick Downloads

Contents

Requirements

Features

Runs JavaScript unit tests from the command line

Run JavaScript unit tests from inside of Visual Studio

Updated Wiki: Home

Welcome to Chutzpah - A JavaScript Test Runner

Recent News