monotone

monotone Mtn Source Tree

Root/notes/tester.txt

1A testsuite is a lua script which defines commonly used globals for the
2individual tests, and lists those tests in the "tests" table. Each test is a
3directory containing a lua script named "__driver__.lua", and optionally any
4number of data files. These data files can be copied to the scratch directory
5with "get(filename)".
6
7Each test is run in a scratch directory, named tester_dir/$TESTNAME relative to
8the directory in which the tester is run. If the test passes and the -d option
9is not given, this directory will be removed after the test finishes. The tester
10will leave a logfile named "tester.log" in tester_dir and in each scratch dir
11that isn't removed.
12
13tester should be called as
14tester testsuite [-dlh] [num [...]] [num..num [...]] [regex [...]]
15
16 -h print a help message
17 -l print test names only; don't run them
18 -d don't clean the scratch directories
19 num run a specific test
20 num..num run tests in a range
21 if num is negative, count back from the end
22 regex run tests with matching names
23
24On startup, the testsuite file is run. It should set up any global vars that the
25tests need, and add them to the "tests" table, as 'table.insert(tests,
26"test-name")'. "test-name" is the name of the directory for this test relative
27to testdir, which is initialized to srcdir (where the testsuite file is).
28
29After this the internal run_tests function is called, which parses the
30command-line arguments and runs whichever tests have been asked for. It also
31resets most of the environment between tests.
32
33
34Variable reference
35------------------
36
37tests
38
39Global table, containing all tests. The testsuite should contain lines
40of table.insert(tests, "test-name")
41
42test.root
43
44The scratch directory.
45
46test.name
47
48The name of the test.
49
50test.wanted_fail
51
52Set when an xfailed command succeeds.
53
54test.partial_skip
55
56Set this if you want to note that part of a test is being skipped.
57
58test.log
59
60The logfile for this individual test.
61
62srcdir
63
64Set to the dir containing the testsuite.
65
66testdir
67
68The directory containing all test stuff. Initially set to srcdir. Where
69all testsuite-related files are in a subdir except for the testsuite
70itself, the testsuite should set this to that dir. For example, the
71monotone testsuite sets this to srcdir.."/tests" .
72
73debugging
74
75Set to true if the -d option was given.
76
77logfile
78
79The general log.
80
81failed_testlogs
82
83List of logfiles for failed tests, to be appended to the general log.
84
85files
86
87Table of files for built-in commands (grep, cat, sort, etc) to use
88for i/o.
89
90posix_umask
91
92Set the umask on posix systems, do nothing on Windows. If a test uses
93this, it should have a relevant skip_if.
94
95initial_dir
96
97Global variable, set to the directory the test program was started in.
98
99
100Function reference
101------------------
102
103a) monotone Lua interface
104+++++++++++++++++++++++++
105
106regex.search()
107globish.match()
108parse_basic_io()
109
110These functions are the same as in monotone.
111
112
113b) File functions
114+++++++++++++++++
115
116mtime(filename) [logged]
117
118Returns the file modification time.
119
120mkdir(filename) [logged]
121
122Create the specified directory.
123
124chdir()
125
126Go to a directory. You probably want to use indir() instead.
127
128exists()
129isdir()
130
131File status checks.
132
133existsonpath(name) [logged]
134
135Returns true if "name" is on the path.
136
137numlines(filename) [logged]
138
139Count the lines in a file.
140
141open_or_err()
142
143Wrapper around io.open, that throws instead of returning an error code.
144
145fsize(filename)
146
147Return the size of the given file.
148
149readfile(filename) [logged]
150
151Return the contents of a file. The unlogged version is readfile_q.
152
153readstdfile(filename)
154
155Calls readfile(), taking filename as relative to the testsuite directory.
156
157writefile(filename [, dat]) [logged]
158
159If dat is given, set the file contents to dat. If dat is not given, only
160make sure that the file exists. The unlogged version is writefile_q.
161
162append(filename [, dat]) [logged]
163
164Append dat to the given file.
165
166copy(from, to) [logged]
167
168Copy a file or a directory tree. The unlogged version is unlogged_copy.
169
170rename(from, to) [logged]
171
172Like os.rename except that it removes the destination if it already
173exists (because os.rename on Windows will not replace an existing
174destination). The unlogged version is unlogged_rename.
175
176remove(filename) [logged]
177
178Remove a file or a directory tree. The unlogged version is
179unlogged_remove.
180
181getstd(name [, as])
182
183Copy a file or a directory tree named 'name' from the testsuite
184directory to the scratch directory, giving it the name 'as', or 'name'
185is as is not given.
186
187get(name [, as])
188
189Copy a file or a directory tree named 'name' from the test directory to
190the scratch directory, giving it the name 'as', or 'name' is as is not
191given.
192
193indir(dir, cmd)
194
195cmd is a command table, as would be passed to check(). This wraps what
196it would do with a chdir() pair to run it in the given directory.
197
198
199c) Command execution
200++++++++++++++++++++
201
202execute(path, ...) [internal]
203
204Run an external program, and return the program's return code.
205
206runcmd(cmd, prefix, bgnd) [internal]
207
208Run cmd under i/o redirection. cmd is a table giving either a function
209and arguments, or an external program name and agruments.
210
211pre_cmd() [internal]
212
213Set up the files used for i/o redirection of commands.
214
215post_cmd() [internal]
216
217Verify that a command gave the expected results, and throw an error if
218it did not.
219
220bg(torun, ret, stdout, stderr, stdin)
221
222Run a command in the background. The arguments are:
223
224torunlist of strings, {"program", "arg1", "arg2" ,...}
225retexpected return code
226std{out,err}
227false ignore output
228true save the output in a file named stdout or stderr
229string the string must match the output
230{string} the named file must match the output
231nil (not given) the output must be empty
232stdin
233true use existing "stdin" file
234nil, false empty input
235string contents of string are used as input
236{string} contents of named file are used as input
237
238The returned object has two methods:
239
240finish(timeout)
241wait for the given timeout, then kill the program and check
242the results. Returns true.
243wait(timeout)
244wait for the given timeout. If the program has ended, check
245the results and return true. Otherwise, return false.
246
247
248d) Checks
249+++++++++
250
251runcheck() [internal]
252
253check(), minus some of the input validation
254
255check(first, ...)
256
257Run a command, and verify the results:
258
259check(torun, ret, stdout, stderr, stdin)
260Inputs are the same as to bg().
261check(bool)
262Throw an error if the argument is false.
263check(number)
264Check that the argument is zero.
265
266If the command fails, abort the test.
267
268skip_if(bool)
269
270If the argument is true, abort the test (throw true) and mark it as
271skipped.
272
273xfail(...)
274
275Equivalent to xfail_if(true, ...).
276
277xfail_if(chk, ...)
278
279Give the ... arguments to check(), and catch any errors. If chk is false
280just rethrow anything caught, so this wrapper is transparent. If chk is
281true, then either mark the test for unexpected success (if the command
282succeeded), or abort and mark it as an expected failure (throw false).
283
284
285e) Printing and Logging
286+++++++++++++++++++++++
287
288P(...)
289
290Print arguments (written to stdout and to the main logfile).
291
292L(...)
293
294Log arguments (written to the test logfile, which will be appended to
295the main logfile if this test fails).
296
297log_file_contents(filename)
298
299Write the contents of the given file to the test log.
300
301getsrcline()
302
303Returns (test.name, sourceline), where sourceline is the line number of
304the oldest stack frame which is in the driver file for the current test
305
306locheader()
307
308Line getsrcline(), except that it returns a string suitable for
309prefixing log lines, as L(locheader(), message).
310
311err(what [, level])
312
313Cause the test to fail, by throwing an error. This is like error(),
314except that it records the getsrcline() of where the error was thrown.
315
316log_error() [internal]
317
318Write a caught error to the test log.
319
320
321f) Content functions (matching, comparison, etc.)
322+++++++++++++++++++++++++++++++++++++++++++++++++
323
324samefile(left, right)
325
326Return true if the given files have identical contents, or false
327otherwise.
328
329samelines(file, table)
330
331Split the file into lines, and check that each line matches the
332corresponding element of the table. This function doesn't care what the
333line endings are.
334
335greplines(file, table)
336
337Split the file into lines, and check that each line is matched by the
338regex given in the corresponding table element.
339
340grep(["-qv"], "what" [, "filename"])
341cat(...)
342tail(file, num)
343sort([filename])
344
345These functions each return a table, suitable for passing to pcall() or
346check(). They are meant to be used in place of the shell utilities of
347the same names.
348
349
350g) Utility functions
351++++++++++++++++++++
352
353run_tests() [internal]
354
355Parse command line arguments, load the testsuite, and run specified
356tests.
357
358go_to_test_dir() [internal]
359set_redirect() [internal]
360clear_redirect() [internal]
361clean_test_dir() [internal]
362leave_test_dir() [internal]
363save_env() [internal]
364restore_env() [internal]
365
366Misc. infrastructure.
367
368get_source_dir()
369
370Return the directory the testsuite file was in.
371
372set_env()
373
374Set an environment variable.
375
376get_ostype()
377
378Return a string representing the OS. You probably care mostly about the
379first word.
380
381include(filename)
382
383Load and execute a file, named relative to the testsuite directory.
384
385trim(string)
386
387Return the string, with leading and trailing whitespace removed.
388

Archive Download this file

Branches

Tags

Quick Links:     www.monotone.ca    -     Downloads    -     Documentation    -     Wiki    -     Code Forge    -     Build Status