6
Vote

Improve line number detection

description

Chutzpah uses a simple heuristic to try to match test's to line numbers.
For example, given a qunit test file it will scan the filing recording the line number of each instance of "test(" and then after tests are run match the test run to the index of each "test(" call.

This is a very fragile implementation that fails in a couple scenarios.
  1. If you are using a framework that adds its own verbage for tests (like Pavalov.js) this matching completely fails.
  2. If you wrap your test in a loop, Chutzpah will fail to give correct line numbers.
I have not thought about a more robust implementation yet, thoughts are welcome.

comments

Qube wrote Feb 12, 2012 at 8:19 PM

I think the only way to have a truly robust line number matcher is to actually extract the information from the JavaScript engine itself. If phantomjs works like Chrome, it may be possible to extract line numbers from Error objects. So given a test like this:

test('something', function () {
});

You could inject something like this:

test(recordLineNumber(new Error()) + 'something', function () {
});

recordLineNumber would take the Error object, add it's line number to an array and return an empty string. That array of line numbers should then line up with the test results, and could be returned with them.

mmanela wrote Feb 12, 2012 at 10:03 PM

I agree, if we can extract line information by using error objects then we have some viable options. I plan to test to see if this approach works. If so, then we have two options.
  1. Pre-process the file the way you show above.
  2. Figure out if there is a hook or if we can monkey patch the testing framework so that we can override the test execution and determine the line number when the test executes.

Qube wrote Feb 12, 2012 at 10:37 PM

Yep, good call. It looks like it might be possible to extract the line number from the stack, so a patched version of the test framework might do the trick.

Qube wrote Feb 13, 2012 at 7:47 AM

I did a quick test of overriding the "it" function in Jasmine, and it worked in Chrome. You can go up the stack to the line executed at the suite level. It's pretty much perfect.

Unfortunately it didn't work at all in Phantom - no stack! The only way to get the line number is to throw an actual error inline with the test (doesn't work using the Error constructor).