JSCheck is a testing tool for JavaScript. It was inspired by QuickCheck, a testing tool for Haskell developed by Koen Claessen and John Hughes of Chalmers University of Technology.
JSCheck is a specification-driven testing tool. From a description of the properties of a system, function, or object, it will generate random test cases attempting to disprove those properties, and then report its findings. That can be especially effective in managing the evolution of a program because it can show the conformance of new code to old code. It also provides an interesting level of self-documentation, because the executable specifications it relies on can provide a good view of the workings of a program.
The JSCheck module is loaded from the jscheck.js file. It produces a single function that is used to construct a jsc object.
import JSCheck from "./jscheck.js"; let jsc = JSCheck();
The source is available at https://github.com/douglascrockford/jscheck.
JSCheck is concerned with the specification and checking of claims. (QuickCheck called these properties. We use the term claim instead of property to avoid confusion with JavaScript's use of property to mean a member of an object.)
jsc.claim files a claim function that will later be tested by the jsc.check function, which randomly generates the cases that attempt to disprove the filed claims. You can set the number of
cases generated per claim.
To make a claim, you pass three or four components to jsc.claim(name, predicate, signature, classifier).
The name is descriptive text that is used in making the report.
The predicate is a function that returns a verdict of true if the claim holds. The predicate should do something with the system being tested, perhaps examining its result or examining the consistency of its data structures. If you are testing a set of functions that do encoding and decoding, the predicate can assert things like
function predicate(verdict, value) {
return verdict(value === decode(encode(value)));
}
You won't need to select the value. JSCheck can generate random values for you.
The first parameter to the predicate is always be a verdict function. The predicate function uses a verdict function to announce the result of the case (true if the case succeed, and false if it failed). A verdict function makes it possible to conduct tests that might be completed in a different turn, such as tests involving event handling, network transactions, or asynchronous file processing.
The remaining parameters must match the specifiers.
The signature is an array of specifiers that describe the types of the predicate's arguments. (From a procedural perspective, specifiers are generators, but are quite different from ES6 function*, so to slightly reduce confusion, we take a declarative view.)
JSCheck provides a small library of specifiers that you can use in your claim. For example, jsc.integer(10) declares that a parameter should be an integer between 1 and 10. jsc.wun_of(["Curly", "Larry", "Moe"]) declares that a parameter can be one of three strings. Some of the specifiers can be combined, so jsc.array(jsc.integer(10), jsc.character("a", "z")) declares that a parameter can be an array of 1 to 10 lowercase letters.
An array of specifiers can also contain constants (such as string, numbers, or objects), so you can pass anything you need to into the predicate. If you need to pass in a function, then you must wrap the function value with the jsc.literal specifier.
You can also create your own specifiers.
You can optionally pass a classifier function as part of the claim. The classifier receives the same arguments as the predicate (excluding the verdict). A classifier can do two things:
undefined. JSCheck will attempt to generate another case to replace it. It is recommended that the classifier reject fewer than 90% of the cases. If you are accepting less than 10% of the potential cases, then you should probably reformulate your claim or create a specialized specifier.A jsc object contains several functions.
The jsc.claim function creates claims, and the jsc.check function tests the claims by generating the cases and producing the reports.
Process all of the filed claims. This function does not return anything. Results are only delivered through your callback functions in the configuration object.
The configuration object can contain these properties:
nr_trials:The number of trials generated per claim. Default: 100. The number of proposed cases could be as many as 10 times this number, to allow for rejection by your classifier functions.
time_limit:The number of milliseconds the claims have to complete. Default: No time limit.
detail:The level of detail in the report. Default: 3.
- none: There will be no report.
- terse: There will be a minimal report, showing the pass score of each claim.
- failures: The individual cases that fail will be reported.
- classification: The classification summaries will also be reported.
- verbose: All cases will be reported.
on_fail:Callback for failing cases. Youron_failfunction will be given for each failed case an object containing these properties:
args:The array of arguments for the case.claim:The claim function.classifier:The classifier function, if there was one.classification:The classification string of the case, if there is one.name:The name of the claim.pass:false.predicate:The predicate function.serial:The case serial number.signature:The signature array.
on_lost:Callback for lost cases. A case is considered lost if the predicate did not return a boolean verdict within the allotted milliseconds. Youron_lostfunction will be given for each lost case an object containing these properties:
args:The array of arguments for the case.claim:The claim function.classifier:The classifier function, if there is one.classification:The classification string of the case, if there is one.name:The name of the claim.predicate:The predicate function.serial:The case serial number.signature:The signature array.
on_pass:Callback for passing cases. Youron_passfunction will be given for each passing case an object containing these properties:
args:The array of arguments for the case.claim:The claim function.classifier:The classifier function, if there is one.classification:The classification string of the case, if there is one.name:The name of the claim.pass:true.predicate:The predicate function.serial:The case serial number.signature:The signature array.
on_report:Callback for the detailed report. Youron_reportfunction will be given a string containing the results for all of the claims.
on_result:Callback for the the summary. Youron_resultfunction will be given an object summarizing the check, containing these properties:
pass:The number of cases that passed.fail:The number of cases that failed.lost:The number of cases that did not return a boolean verdict.ok:trueifpassis greater than 0 andfailandlostare both 0.total:The total number of cases.
At the conclusion of the check, all of the claims will be released.
The claim function takes a name, a predicate function, and an array of specifiers and files a claim. The claim function does not test the claim. The testing process is started by the check function.
The predicate function must return verdict(true) if a case passes, and return verdict(false) if the case fails. It takes a list of arguments that is generated by the array of specifiers.
The signature is an array of specifiers. The signature looks like a type declaration for the predicate function.
The classifier function is called before each call of the predicate function.
It gets a chance to determine if the random values in its arguments are reasonable trial.
It can return undefined if the trial should be rejected and a new trial generated to replace it.
The classifier function can instead return a descriptive string that classifies the case.
The classifications can be counted and displayed in the report when the detail is 4 or 5.
A specifier is a function that returns a function that can generate values of a particular type. The specifiers are used in building the signature that is used to construct a claim.
The any specifier returns any random JavaScript value. It is short for
jsc.wun_of([
jsc.integer(),
jsc.number(),
jsc.string(),
jsc.wun_of([
true, Infinity, -Infinity, falsy(), Math.PI, Math.E, Number.EPSILON
])
])
It can produce values such as
0.5166432844718089 "_g" 0 499 "" 419 "N"
The array specifier returns an array containing random stuff. It is short for jsc.array(jsc.integer(4), jsc.integer()). So for example,
jsc.array()
can produce arrays such as
[397, 67] [421, 613, 281] [7, 823] [11] ...
The array specifier takes an array as a template. It goes through the array, expanding the specifiers it contains. So, for example,
jsc.array([
jsc.integer(),
jsc.number(100),
jsc.string(8, jsc.character("A", "Z"))
])
can generate arrays like
[3,21.228644298389554,"TJFJPLQA"] [5,57.05485427752137,"CWQDVXWY"] [7,91.98980208020657,"QVMGNVXK"] [11,87.07735128700733,"GXBSVLKJ"] ...
The array specifier takes a number and a value, and produces an array using the number as the length of the array, populating the array with the value. So, for example,
jsc.array(3, jsc.integer(640))
can generate arrays like
[305,603,371] [561,623,477] [263,534,530] [163,148,17] ...
The boolean specifier produces true and false with equal probability. It is shorthand for jsc.boolean(0.5).
The boolean specifier produces true and false. If the bias is 0.50, it produces them with equal probability. A lower bias produces more falses, and a higher bias produces more trues.
The character specifier generates a character. It is short for jsc.character(32, 126). So, for example,
jsc.character()
can generate strings like
"*" "a" "J" "0" ...
The character specifier treats its argument as a char code, generating a character.
The character specifier generates characters within a given range.
The character specifier takes a string, and selects one of its characters. So for example,
jsc.string(8, jsc.character("abcdefgABCDEFG_$"))
produces values like
"cdgbdB_D" "$fGE_BAB" "gEFF_FAe" "AebGbAbd" ...
The falsy specifier generates falsy values: false, null, undefined, "", 0, and NaN.
The integer specifier generates prime numbers. Sometimes when testing formulas, it is useful to plug prime numbers into the variables.
The integer specifier generates an integer between 1 and n.
The integer specifier generates an integer between from and to.
The literal specifier generates the value without interpreting it. For most values (strings, numbers, boolean, objects, arrays), the literal specifier is not needed. It is needed if you want to pass a function value to a predicate, because function values are assumed to be the products of specifiers.
The number specifier produces random numbers between 0 and x.
The number specifier produces random numbers between from and to.
The object specifier produces random objects containing random keys and values. So for example.
jsc.object()
can generate objects like
{"adf*:J'mS%": ""}
{"_S-": 0.23757726117037237, "{U": 3, "[": null, ":vL|_": "Bl=2C"}
{"pV": 0.8472617215011269, "3:" :"xVvi`", "jGB8y": 5, "$<9BFhA": true}
{"1": 7, "H@C>": 0.01756326947361231}
...
The object specifier makes an object containing n random keys and values.
The object specifier takes an object as a template. It goes through the enumerable own properties of object, expanding the specifiers it contains. So, for example,
jsc.object({
left: jsc.integer(640),
top: jsc.integer(480),
color: jsc.wun_of(["black", "white", "red", "blue", "green", "gray"])
})
can generate objects like
{"left": 104, "top": 139, "color": "gray"}
{"left": 62, "top": 96, "color": "white"}
{"left": 501, "top": 164, "color": "white"}
{"left": 584, "top": 85, "color": "white"}
...
The object specifier takes an array of keys and produces an object using those keys. The values are taken from an array of values or a specifier. So for example,
jsc.object(
jsc.array(jsc.integer(3, 8), jsc.string(4, jsc.character("a", "z"))),
jsc.boolean()
)
can generate objects like
{"jodo": true, "zhzm": false, "rcqz": true}
{"odcr": true, "azax": true, "bnfx": true, "hmmc": false}
{"wjew": true, "kgqj": true, "abid": true, "cjva": false, "qsgj": true, "wtsu": true}
{"qtbo": false, "vqzc": false, "zpij": true, "ogss": false, "lxnp": false, "psso": true, "irha": true, "ghnj": true}
...
and
jsc.object(
["x", "y", "z"],
[jsc.integer(320), jsc.integer(240), jsc.integer(100)]
)
can generate objects like
{"x": 99, "y": 51, "z": 51}
{"x": 114, "y": 166, "z": 82}
{"x": 35, "y": 124, "z": 60}
{"x": 13, "y": 41, "z": 63}
...
The wun_of specifier takes an array of specifiers, and selects values from the array with equal probability. So, for example
jsc.wun_of([
jsc.number(),
jsc.boolean(),
null
])
produces values like
0.09817210142500699 0.3351482313591987 null false ...
The wun_of specifier takes an array of specifiers and an array of weights. The weights are used to adjust the probabilities. So for example,
jsc.wun_of( [1,2,3,4,5,6,7,8,9,10], [1,2,3,4,5,6,7,8,9,10] )produces values like
8 10 6 10 ...
The sequence specifier takes an array of values, and produces them in sequence, repeating the sequence as needed. So for example,
jsc.sequence([1, 2])
produces values like
1 2 1 2 ...
The string specifier makes random ASCII strings. It is short for jsc.string(jsc.integer(10), jsc.character()). So for example,
jsc.string()
produces strings like
"hZO*3" "m-W2@KL" ",P+po0#2 " "tlt^[ ui`V" ...
The string specifier generates the stringification of the value, using String. So for example,
jsc.string(jsc.integer(1000, 9999))
produces values like
"4791" "9523" "2774" "4288" ...
The string specifier generates strings by joining some number of values. So for example,
jsc.string(jsc.integer(1, 8), jsc.character("aeiou")))
produces values like
"ieauae" "uo" "iuieio" "euu" ...
Any number of number value pairs can be provided, so
jsc.string(
jsc.character("A", "Z"),
4, jsc.character("a", "z"),
6, jsc.character("1", "9"),
jsc.wun_of("!@#$%")
)
produces values like
"Zsopx171765#" "Nfafw851294%" "Gtyef393138%" "Lrxav768561!" ...
JSCheck provides a small set of specifiers that can be combined in many ways. For some purposes, you may need to create your own specifiers. It is easy to do. A specifier is a function that returns a function.
my_specifier = function specifier(param1, param2) {
// per claim processing happens in here
return function generator() {
// per case processing happen in here
return value;
};
}
The generator function that is returned is stored in the signature array, and will be called for each value that needs to be generated. It has access to the specifier's parameters. Its arguments might be other specifiers, so if an argument is a function, it uses the result of calling the function.
intermediate_value = (
typeof param1 === "function"
? param1()
: param1
);
Since JSCheck performs a useful specification and description function as well as a testing function, it is recommended (but not required) that claims be inserted into the relevant source code, and not in separate source files. ecomcon can make this easy to manage, so that claims can be removed automatically from production code. All of the calls to jsc can be hidden in special comments, which are activated during development, and removed by minification in production.
One difficulty in demonstrating testing systems is that the exposition of the system to be tested is usually significantly more complex than the testing tool being demonstrated. So in this case, we are testing a trivial function. We will make an incorrect claim. JSCheck will help us to find the error in the claim. It might seem counter productive to demonstrate bad claim making, but it is as important to get the claims right as it is to get the program right.
We are going to test the le function.
function le(a, b) {
return a <= b;
}
We construct a claim. Our predicate simply returns the result of le. It takes two integers, one with a max of 10 and another with a max of 20. We classify the cases by the relationship between the arguments.
import JSCheck from "./jscheck.js";
let jsc = JSCheck();
jsc.claim(
"Less than",
function predicate(verdict, a, b) {
return verdict(le(a, b));
},
[
jsc.integer(10),
jsc.integer(20)
],
function classifier(a, b) {
return (
a < b
? "lt"
: (
a === b
? "eq"
: "gt"
)
);
}
);
jsc.check({
detail: 4,
on_report: console.log
});
But when we check the claim, many cases fail. The on_report summary of the classification tells the story:
eq pass 7 gt pass 0 fail 22 lt pass 71
The predicate failed because 22 of the generated cases had an a that was larger than b. This is because jsc.integer(10) produces from the range 0 to 9, and jsc.integer(20) produces from the range 0 to 19. Sometimes the first value will be larger. This tells us that we should make the predicate more sophisticated, or we could have the classifier return undefined instead of "gt" to reject those cases.