Module:ScribuntoUnit
This module provides unit tests for other Lua modules. To test a module, you must create a separate test module, usually located at Module:Module name/testcases
. The module is tested with the ScribuntoUnit module, which verifies that the operations defined in the test module produce the expected results.
Test module structure
To make a test module (test suite), start with the following code:
<syntaxhighlight lang="lua"> local myModule = require('Module:MyModule') -- the module to be tested local ScribuntoUnit = require('Module:ScribuntoUnit') local suite = ScribuntoUnit:new() </syntaxhighlight>
After you have done this you can add individual test functions to the suite
object. Any function that begins with test
is treated as a test. (Other functions will be ignored by ScribuntoUnit, but can be used in the tests themselves.)
<syntaxhighlight lang="lua"> function suite:testSomeCall()
self:assertEquals('expected value', myModule.someCall(123)) self:assertEquals('other expected value', myModule.someCall(456))
end
function suite:testSomeOtherCall()
self:assertEquals('expected value', myModule.someOtherCall(123)) self:assertEquals('other expected value', myModule.someOtherCall(456))
end </syntaxhighlight>
The tests you write should make assertions, and ScribuntoUnit will check whether those assertions are true. For example, assertEquals
checks that both of the arguments it is given are equal. If ScribuntoUnit doesn't find an assertion to be true, then the test will fail and an error message will be generated. The error message will show which assertion failed verification (other checks on the assertions are not made at this time).
To finish the test module, you need to return the suite
object.
<syntaxhighlight lang="lua"> return suite </syntaxhighlight>
Running the tests
The tests can be run in two ways: through the Lua debug console, and from a wiki page using #invoke. If you are running the tests through the debug console, use the code require('Module:MyModule/testcases').run()
. If you are running them from a wiki page, use the code {{#invoke:MyModule/testcases|run}}
. This will generate a table containing the results. It is also possible to display a more compact table by using the code {{#invoke:MyModule/testcases|run|displayMode=short}}
.
Tests
Error messages
The last parameter of all the test methods is a message that is displayed if validation fails.
<syntaxhighlight lang="lua"> self:assertEquals("expected value", myModule.someCall(123), "The call to myModule.someCall(123) didn't return the expected value.") </syntaxhighlight>
assertTrue, assertFalse
<syntaxhighlight lang="lua"> self:assertTrue(expression, message) self:assertFalse(expression, message) </syntaxhighlight>
These test whether the given expression evaluates to true
or false
. Note that in Lua false
and nil
evaluate to false
, and everything else evaluates to true
.
<syntaxhighlight lang="lua"> self:assertTrue(2 + 2 == 4) self:assertTrue('foo') self:assertFalse(2 + 2 == 5) self:assertFalse(nil) </syntaxhighlight>
assertStringContains
<syntaxhighlight lang="lua"> self:assertStringContains(pattern, s, plain, message) </syntaxhighlight>
This tests whether pattern
is found in the string s
. If plain
is true, then pattern
is interpreted as literal text; otherwise, pattern
is interpreted as a ustring pattern.
If the string is not found, the error message shows the values of pattern
and s
; if s
is more than 70 characters long then a truncated version is displayed. This method is useful for testing specific behaviours in complex wikitext.
<syntaxhighlight lang="lua"> self:assertStringContains("foo", "foobar") -- passes self:assertStringContains("foo", "fobar") -- fails self:assertStringContains(".oo", "foobar") -- passes: matches "foo" self:assertStringContains(".oo", "foobar", true) -- fails: . is interpreted as a literal character </syntaxhighlight>
assertNotStringContains
<syntaxhighlight lang="lua"> self:assertNotStringContains(pattern, s, plain, message) </syntaxhighlight>
This is the opposite of assertStringContains
. The test will fail if pattern
is found in the string s
. If plain
is true, then pattern
is interpreted as literal text; otherwise, pattern
is interpreted as a ustring pattern.
<syntaxhighlight lang="lua"> self:assertNotStringContains("foo", "foobar") -- fails self:assertNotStringContains("foo", "fobar") -- passes self:assertNotStringContains(".oo", "foobar") -- fails: matches "foo" self:assertNotStringContains(".oo", "foobar", true) -- passes: . is interpreted as a literal character </syntaxhighlight>
assertEquals
<syntaxhighlight lang="lua"> self:assertEquals(expected, actual, message) </syntaxhighlight>
This tests whether the first parameter is equal to the second parameter. If both parameters are numbers, the values are instead compared using Template:Code with delta 1e-8 (0.00000001) since numbers are represented as floating points with limited precision.
<syntaxhighlight lang="lua"> self:assertEquals(4, calculator.add(2, 2)) </syntaxhighlight>
assertWithinDelta
<syntaxhighlight lang="lua"> self:assertWithinDelta(expected, actual, delta, message) </syntaxhighlight>
For two numbers, this tests whether the first is within a given distance (delta) from the second. This is useful to compare floating point numbers, which are used to represent numbers in the standard installation of Lua. (To be precise, it uses double-precision floating point numbers.) For example, on the version of Scribunto installed on the English Wikipedia, the expression 0.3 – 0.2 == 0.1
evaluates to false
. This is because in practice, the expression 0.3 – 0.2
equals 0.09999999999999997780…
and the number 0.1
equals 0.10000000000000000555…
. The slight error between the two means that Lua does not consider them equal. Therefore, to test for equality between two floating point numbers, we should accept values within a small distance (delta) of each other, not just equal values. Note that this problem does not affect integers, which can be represented exactly using double-precision floating point numbers up to values of 2^53.
<syntaxhighlight lang="lua"> self:assertWithinDelta(0.1, calculator.subtract(0.3, 0.2), 1e-10) </syntaxhighlight>
assertDeepEquals
<syntaxhighlight lang="lua"> self:assertDeepEquals(expected, actual, message) </syntaxhighlight>
This tests whether the first parameter is equal to the second parameter. If the parameters are tables, they are compared recursively, and their __eq metamethods are respected.
<syntaxhighlight lang="lua"> self:assertDeepEquals(table1, table2) </syntaxhighlight>
assertTemplateEquals
<syntaxhighlight lang="lua"> self:assertTemplateEquals(expected, template, args, message) </syntaxhighlight>
This tests whether the first parameter equals a template call. The second parameter is the template name, and the third parameter is a table of the template arguments.
<syntaxhighlight lang="lua"> self:assertTemplateEquals(4, 'add', {2, 2}) -- true if Template:Add equals 4 </syntaxhighlight>
Note that some tags written in XML notation cannot be tested correctly; see the note for the assertResultEquals
function below.
assertResultEquals
<syntaxhighlight lang="lua"> self:assertResultEquals(expected, text, message) </syntaxhighlight>
This tests whether the first parameter equals the expansion of any wikitext. The second parameter can be any wikitext.
<syntaxhighlight lang="lua"> self:assertResultEquals(4, 'Script error: No such module "Calculator".') </syntaxhighlight>
Note that some special tags written in XML notation, such as <pre>
, <nowiki>
, <gallery>
and <ref>
cannot be compared correctly. These tags are converted to strip markers before they are processed by Lua. Strip markers are unique, even when generated from identical input, so any tests testing these tags for equality will fail. This also applies to the assertTemplateEquals
and assertSameResult
functions.
assertSameResult
<syntaxhighlight lang="lua"> self:assertSameResult(text1, text2, message) </syntaxhighlight>
This tests whether the expansion of a given string of wikitext equals the expansion of another string of wikitext. This can be useful for verifying that a module behaves in the same way as a template it is intended to replace.
<syntaxhighlight lang="lua"> self:assertSameResult('Template:Add', 'Script error: No such module "Calculator".') </syntaxhighlight>
Note that some tags written in XML notation cannot be tested correctly; see the note for the assertResultEquals
function above.
assertThrows
<syntaxhighlight lang="lua"> self:assertThrows(fn, expectedMessage, message) </syntaxhighlight>
This tests whether a given function throws an exception. If expectedMessage
is not nil
, it will check that an exception was thrown with the given error message.
See also
------------------------------------------------------------------------------- -- Unit tests for Scribunto. ------------------------------------------------------------------------------- require('Module:No globals') local DebugHelper = {} local ScribuntoUnit = {} -- The cfg table contains all localisable strings and configuration, to make it -- easier to port this module to another wiki. local cfg = mw.loadData('Module:ScribuntoUnit/config') ------------------------------------------------------------------------------- -- Concatenates keys and values, ideal for displaying a template argument table. -- @param keySeparator glue between key and value (defaults to " = ") -- @param separator glue between different key-value pairs (defaults to ", ") -- @example concatWithKeys({a = 1, b = 2, c = 3}, ' => ', ', ') => "a => 1, b => 2, c => 3" -- function DebugHelper.concatWithKeys(table, keySeparator, separator) keySeparator = keySeparator or ' = ' separator = separator or ', ' local concatted = '' local i = 1 local first = true local unnamedArguments = true for k, v in pairs(table) do if first then first = false else concatted = concatted .. separator end if k == i and unnamedArguments then i = i + 1 concatted = concatted .. tostring(v) else unnamedArguments = false concatted = concatted .. tostring(k) .. keySeparator .. tostring(v) end end return concatted end ------------------------------------------------------------------------------- -- Compares two tables recursively (non-table values are handled correctly as well). -- @param ignoreMetatable if false, t1.__eq is used for the comparison -- function DebugHelper.deepCompare(t1, t2, ignoreMetatable) local type1 = type(t1) local type2 = type(t2) if type1 ~= type2 then return false end if type1 ~= 'table' then return t1 == t2 end local metatable = getmetatable(t1) if not ignoreMetatable and metatable and metatable.__eq then return t1 == t2 end for k1, v1 in pairs(t1) do local v2 = t2[k1] if v2 == nil or not DebugHelper.deepCompare(v1, v2) then return false end end for k2, v2 in pairs(t2) do if t1[k2] == nil then return false end end return true end ------------------------------------------------------------------------------- -- Raises an error with stack information -- @param details a table with error details -- - should have a 'text' key which is the error message to display -- - a 'trace' key will be added with the stack data -- - and a 'source' key with file/line number -- - a metatable will be added for error handling -- function DebugHelper.raise(details, level) level = (level or 1) + 1 details.trace = debug.traceback('', level) details.source = string.match(details.trace, '^%s*stack traceback:%s*(%S*: )') -- setmetatable(details, { -- __tostring: function() return details.text end -- }) error(details, level) end ------------------------------------------------------------------------------- -- when used in a test, that test gets ignored, and the skipped count increases by one. -- function ScribuntoUnit:markTestSkipped() DebugHelper.raise({ScribuntoUnit = true, skipped = true}, 3) end ------------------------------------------------------------------------------- -- Checks that the input is true -- @param message optional description of the test -- function ScribuntoUnit:assertTrue(actual, message) if not actual then DebugHelper.raise({ScribuntoUnit = true, text = string.format("Failed to assert that %s is true", tostring(actual)), message = message}, 2) end end ------------------------------------------------------------------------------- -- Checks that the input is false -- @param message optional description of the test -- function ScribuntoUnit:assertFalse(actual, message) if actual then DebugHelper.raise({ScribuntoUnit = true, text = string.format("Failed to assert that %s is false", tostring(actual)), message = message}, 2) end end ------------------------------------------------------------------------------- -- Checks an input string contains the expected string -- @param message optional description of the test -- @param plain search is made with a plain string instead of a ustring pattern -- function ScribuntoUnit:assertStringContains(pattern, s, plain, message) if type(pattern) ~= 'string' then DebugHelper.raise({ ScribuntoUnit = true, text = mw.ustring.format("Pattern type error (expected string, got %s)", type(pattern)), message = message }, 2) end if type(s) ~= 'string' then DebugHelper.raise({ ScribuntoUnit = true, text = mw.ustring.format("String type error (expected string, got %s)", type(s)), message = message }, 2) end if not mw.ustring.find(s, pattern, nil, plain) then DebugHelper.raise({ ScribuntoUnit = true, text = mw.ustring.format('Failed to find %s "%s" in string "%s"', plain and "plain string" or "pattern", pattern, s), message = message }, 2) end end ------------------------------------------------------------------------------- -- Checks an input string doesn't contain the expected string -- @param message optional description of the test -- @param plain search is made with a plain string instead of a ustring pattern -- function ScribuntoUnit:assertNotStringContains(pattern, s, plain, message) if type(pattern) ~= 'string' then DebugHelper.raise({ ScribuntoUnit = true, text = mw.ustring.format("Pattern type error (expected string, got %s)", type(pattern)), message = message }, 2) end if type(s) ~= 'string' then DebugHelper.raise({ ScribuntoUnit = true, text = mw.ustring.format("String type error (expected string, got %s)", type(s)), message = message }, 2) end local i, j = mw.ustring.find(s, pattern, nil, plain) if i then local match = mw.ustring.sub(s, i, j) DebugHelper.raise({ ScribuntoUnit = true, text = mw.ustring.format('Found match "%s" for %s "%s"', match, plain and "plain string" or "pattern", pattern), message = message }, 2) end end ------------------------------------------------------------------------------- -- Checks that an input has the expected value. -- @param message optional description of the test -- @example assertEquals(4, add(2,2), "2+2 should be 4") -- function ScribuntoUnit:assertEquals(expected, actual, message) if type(expected) == 'number' and type(actual) == 'number' then self:assertWithinDelta(expected, actual, 1e-8, message) elseif expected ~= actual then DebugHelper.raise({ ScribuntoUnit = true, text = string.format("Failed to assert that %s equals expected %s", tostring(actual), tostring(expected)), actual = actual, expected = expected, message = message, }, 2) end end ------------------------------------------------------------------------------- -- Checks that 'actual' is within 'delta' of 'expected'. -- @param message optional description of the test -- @example assertEquals(1/3, 9/3, "9/3 should be 1/3", 0.000001) function ScribuntoUnit:assertWithinDelta(expected, actual, delta, message) if type(expected) ~= "number" then DebugHelper.raise({ ScribuntoUnit = true, text = string.format("Expected value %s is not a number", tostring(expected)), actual = actual, expected = expected, message = message, }, 2) end if type(actual) ~= "number" then DebugHelper.raise({ ScribuntoUnit = true, text = string.format("Actual value %s is not a number", tostring(actual)), actual = actual, expected = expected, message = message, }, 2) end local diff = expected - actual if diff < 0 then diff = - diff end -- instead of importing math.abs if diff > delta then DebugHelper.raise({ ScribuntoUnit = true, text = string.format("Failed to assert that %f is within %f of expected %f", actual, delta, expected), actual = actual, expected = expected, message = message, }, 2) end end ------------------------------------------------------------------------------- -- Checks that a table has the expected value (including sub-tables). -- @param message optional description of the test -- @example assertDeepEquals({{1,3}, {2,4}}, partition(odd, {1,2,3,4})) function ScribuntoUnit:assertDeepEquals(expected, actual, message) if not DebugHelper.deepCompare(expected, actual) then if type(expected) == 'table' then expected = mw.dumpObject(expected) end if type(actual) == 'table' then actual = mw.dumpObject(actual) end DebugHelper.raise({ ScribuntoUnit = true, text = string.format("Failed to assert that %s equals expected %s", tostring(actual), tostring(expected)), actual = actual, expected = expected, message = message, }, 2) end end ------------------------------------------------------------------------------- -- Checks that a wikitext gives the expected result after processing. -- @param message optional description of the test -- @example assertResultEquals("Hello world", "{{concat|Hello|world}}") function ScribuntoUnit:assertResultEquals(expected, text, message) local frame = self.frame local actual = frame:preprocess(text) if expected ~= actual then DebugHelper.raise({ ScribuntoUnit = true, text = string.format("Failed to assert that %s equals expected %s after preprocessing", text, tostring(expected)), actual = actual, actualRaw = text, expected = expected, message = message, }, 2) end end ------------------------------------------------------------------------------- -- Checks that two wikitexts give the same result after processing. -- @param message optional description of the test -- @example assertSameResult("{{concat|Hello|world}}", "{{deleteLastChar|Hello world!}}") function ScribuntoUnit:assertSameResult(text1, text2, message) local frame = self.frame local processed1 = frame:preprocess(text1) local processed2 = frame:preprocess(text2) if processed1 ~= processed2 then DebugHelper.raise({ ScribuntoUnit = true, text = string.format("Failed to assert that %s equals expected %s after preprocessing", processed1, processed2), actual = processed1, actualRaw = text1, expected = processed2, expectedRaw = text2, message = message, }, 2) end end ------------------------------------------------------------------------------- -- Checks that a template gives the expected output. -- @param message optional description of the test -- @example assertTemplateEquals("Hello world", "concat", {"Hello", "world"}) function ScribuntoUnit:assertTemplateEquals(expected, template, args, message) local frame = self.frame local actual = frame:expandTemplate{ title = template, args = args} if expected ~= actual then DebugHelper.raise({ ScribuntoUnit = true, text = string.format("Failed to assert that %s with args %s equals expected %s after preprocessing", DebugHelper.concatWithKeys(args), template, expected), actual = actual, actualRaw = template, expected = expected, message = message, }, 2) end end ------------------------------------------------------------------------------- -- Checks whether a function throws an error -- @param fn the function to test -- @param expectedMessage optional the expected error message -- @param message optional description of the test function ScribuntoUnit:assertThrows(fn, expectedMessage, message) local succeeded, actualMessage = pcall(fn) if succeeded then DebugHelper.raise({ ScribuntoUnit = true, text = 'Expected exception but none was thrown', message = message, }, 2) end -- For strings, strip the line number added to the error message actualMessage = type(actualMessage) == 'string' and string.match(actualMessage, 'Module:[^:]*:[0-9]*: (.*)') or actualMessage local messagesMatch = DebugHelper.deepCompare(expectedMessage, actualMessage) if expectedMessage and not messagesMatch then DebugHelper.raise({ ScribuntoUnit = true, expected = expectedMessage, actual = actualMessage, text = string.format('Expected exception with message %s, but got message %s', tostring(expectedMessage), tostring(actualMessage) ), message = message }, 2) end end ------------------------------------------------------------------------------- -- Creates a new test suite. -- @param o a table with test functions (alternatively, the functions can be added later to the returned suite) -- function ScribuntoUnit:new(o) o = o or {} setmetatable(o, {__index = self}) o.run = function(frame) return self:run(o, frame) end return o end ------------------------------------------------------------------------------- -- Resets global counters -- function ScribuntoUnit:init(frame) self.frame = frame or mw.getCurrentFrame() self.successCount = 0 self.failureCount = 0 self.skipCount = 0 self.results = {} end ------------------------------------------------------------------------------- -- Runs a single testcase -- @param name test nume -- @param test function containing assertions -- function ScribuntoUnit:runTest(suite, name, test) local success, details = pcall(test, suite) if success then self.successCount = self.successCount + 1 table.insert(self.results, {name = name, success = true}) elseif type(details) ~= 'table' or not details.ScribuntoUnit then -- a real error, not a failed assertion self.failureCount = self.failureCount + 1 table.insert(self.results, {name = name, error = true, message = 'Lua error -- ' .. tostring(details)}) elseif details.skipped then self.skipCount = self.skipCount + 1 table.insert(self.results, {name = name, skipped = true}) else self.failureCount = self.failureCount + 1 local message = details.source if details.message then message = message .. details.message .. "\n" end message = message .. details.text table.insert(self.results, {name = name, error = true, message = message, expected = details.expected, actual = details.actual, testname = details.message}) end end ------------------------------------------------------------------------------- -- Runs all tests and displays the results. -- function ScribuntoUnit:runSuite(suite, frame) self:init(frame) local names = {} for name in pairs(suite) do if name:find('^test') then table.insert(names, name) end end table.sort(names) -- Put tests in alphabetical order. for i, name in ipairs(names) do local func = suite[name] self:runTest(suite, name, func) end return { successCount = self.successCount, failureCount = self.failureCount, skipCount = self.skipCount, results = self.results, } end ------------------------------------------------------------------------------- -- #invoke entry point for running the tests. -- Can be called without a frame, in which case it will use mw.log for output -- @param displayMode see displayResults() -- function ScribuntoUnit:run(suite, frame) local testData = self:runSuite(suite, frame) if frame and frame.args then return self:displayResults(testData, frame.args.displayMode or 'table') else return self:displayResults(testData, 'log') end end ------------------------------------------------------------------------------- -- Displays test results -- @param displayMode: 'table', 'log' or 'short' -- function ScribuntoUnit:displayResults(testData, displayMode) if displayMode == 'table' then return self:displayResultsAsTable(testData) elseif displayMode == 'log' then return self:displayResultsAsLog(testData) elseif displayMode == 'short' then return self:displayResultsAsShort(testData) else error('unknown display mode') end end function ScribuntoUnit:displayResultsAsLog(testData) if testData.failureCount > 0 then mw.log('FAILURES!!!') elseif testData.skipCount > 0 then mw.log('Some tests could not be executed without a frame and have been skipped. Invoke this test suite as a template to run all tests.') end mw.log(string.format('Assertions: success: %d, error: %d, skipped: %d', testData.successCount, testData.failureCount, testData.skipCount)) mw.log('-------------------------------------------------------------------------------') for _, result in ipairs(testData.results) do if result.error then mw.log(string.format('%s: %s', result.name, result.message)) end end end function ScribuntoUnit:displayResultsAsShort(testData) local text = string.format(cfg.shortResultsFormat, testData.successCount, testData.failureCount, testData.skipCount) if testData.failureCount > 0 then text = '<span class="error">' .. text .. '</span>' end return text end function ScribuntoUnit:displayResultsAsTable(testData) local successIcon, failIcon = self.frame:preprocess(cfg.successIndicator), self.frame:preprocess(cfg.failureIndicator) local text = '' if testData.failureCount > 0 then local msg = mw.message.newRawMessage(cfg.failureSummary, testData.failureCount):plain() msg = self.frame:preprocess(msg) if cfg.failureCategory then msg = cfg.failureCategory .. msg end text = text .. failIcon .. ' ' .. msg .. '\n' else text = text .. successIcon .. ' ' .. cfg.successSummary .. '\n' end text = text .. '{| class="wikitable scribunto-test-table"\n' text = text .. '!\n! ' .. cfg.nameString .. '\n! ' .. cfg.expectedString .. '\n! ' .. cfg.actualString .. '\n' for _, result in ipairs(testData.results) do text = text .. '|-\n' if result.error then text = text .. '| ' .. failIcon .. '\n| ' if (result.expected and result.actual) then local name = result.name if result.testname then name = name .. ' / ' .. result.testname end text = text .. name .. '\n| ' .. mw.text.nowiki(tostring(result.expected)) .. '\n| ' .. mw.text.nowiki(tostring(result.actual)) .. '\n' else text = text .. result.name .. '\n| ' .. ' colspan="2" | ' .. mw.text.nowiki(result.message) .. '\n' end else text = text .. '| ' .. successIcon .. '\n| ' .. result.name .. '\n|\n|\n' end end text = text .. '|}\n' return text end return ScribuntoUnit