Make tests engine agnostic#16272
Conversation
digitalinfinity
left a comment
There was a problem hiding this comment.
This is awesome- thanks for this!
There was a problem hiding this comment.
Is this the best we can do?
Can we at least test that it's a multiline string?
There was a problem hiding this comment.
@refack I guess that's something. I've added this:
// Test that it's a multiline string.
assert.ok(/\n/g.test(stack));
joyeecheung
left a comment
There was a problem hiding this comment.
LGTM, although I wouldn't even mention "V8 and ChakraCore", just say "different JavaScript engines"...
There was a problem hiding this comment.
Nit: should be "At least one line ... " or the assertion should be > 1.
cjihrig
left a comment
There was a problem hiding this comment.
LGTM. I agree with @joyeecheung that mentioning the engines by name is not necessary.
There was a problem hiding this comment.
@Trott, I know it's a ton of tedious work, but can these assert.throws() be modified to common.expectsError() for sake of consistency?
e.g.
common.expectsError(
() => console.count(Symbol('test')),
{ type: TypeError }
);There was a problem hiding this comment.
(feel free to say no for doing so in this PR :-) ...)
There was a problem hiding this comment.
Sounds like a great code-and-learn PR
|
@Trott Given that
Should we work with v8 and chakra to unify their error messages? |
We treat error message changes as breaking changes and therefore often test for exact error messages. However, this only applies to errors generated by Node.js core. For errors generated by V8, the TSC has determined that we do not consider it a breaking change when an error message changes.
We have a relatively easy way to find these errors in our tests because ChakraCore will often (usually? almost always?) need to modify such tests to match their error messages.
This PR changes several tests that node-chakracore needed to modify due to the overzealous error message checking (much of which I may have been responsible for either directly or indirectly, but hey, we all get to learn from our mistakes).
There are still more tests to be done, but this is a good start and reaction to this will inform the approach in the rest of the tests.
@nodejs/chakracore @nodejs/testing
Checklist
make -j4 test(UNIX), orvcbuild test(Windows) passesAffected core subsystem(s)
test