You have unit tests, don’t you? Or at least you run some tests to make sure your recent changes broke something?
These regression tests are vital to any serious software development. What would your customers say when they get a new version breaking their day to day work? Well, at least I can’t afford that – and be it just that I’m not keen to fix those problems at a customer under high pressure. So I have regression tests.
Writing those tests for a long standing program can be quite time consuming, but you have to start somewhere. In this case it came out that my application takes an input file and produces an output file from that. So I asked my customers to send me plenty of their input files. The amount of files I got was – well, let’s say unexpected.
I took each file and produced the corresponding output file as a reference. The regression tests also take each input file, produce the output file and compares it with the reference file. If anything is different the test fails and I have to take a look why it fails. Sometimes the fail is intended and I have to update the reference file.
The big challenge was to write the unit tests for all these input files (> 1000). My first attempt was to put a loop into a test case that iterates over all files. That worked but had a major drawback: when one file failed, the complete test failed at that file and the remaining files were not checked at all.
Writing one test case for each file looked far less comfortable, so I wrote a little extension to DUnit that dynamically creates test cases for each file found in a given folder.
The extension introduces two new classes: TFileTestCase derived from TTestCase and TFolderTestSuite derived from TTestSuite.
To make use of these you derive a test case from TFileTestCase and implement any test methods as published just as you do for a normal test case. The current file for the test case can be taken from the new property TestFileName.
The critical part is to override the class method Suite, which returns an ITestSuite interface. Here you create an instance of TFolderTestSuite with some appropriate parameters. The required first parameter for TestClass can simply be given with self, which in a class method gives the current class type.
Here is the code for the extension unit:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 |
unit TestFrameworkExt; interface uses TestFramework; type TFileTestCaseClass = class of TFileTestCase; TFileTestCase = class(TTestCase) private FTestFileName: string; public constructor Create(const AMethodName, ATestFileName: string); reintroduce; overload; virtual; function GetName: string; override; property TestFileName: string read FTestFileName; end; TFolderTestSuite = class(TTestSuite) private FBaseFolder: string; FFileMask: string; FRecursive: Boolean; protected procedure AddMethodTests(TestClass: TTestCaseClass; const NameOfMethod: string); overload; virtual; procedure ProcessFile(Suite: ITestSuite; TestClass: TFileTestCaseClass; const NameOfMethod, FileName: string); virtual; procedure ProcessFolder(Suite: ITestSuite; TestClass: TFileTestCaseClass; const NameOfMethod, Path, FileMask: string; Recursive: Boolean); overload; virtual; procedure ProcessBaseFolder(Suite: ITestSuite; TestClass: TFileTestCaseClass; const NameOfMethod: string); virtual; public constructor Create(TestClass: TFileTestCaseClass; const ABaseFolder, AFileMask: string; ARecursive: Boolean); overload; procedure AddTests(testClass: TTestCaseClass); override; property BaseFolder: string read FBaseFolder; property FileMask: string read FFileMask; property Recursive: Boolean read FRecursive; end; implementation uses Types, IOUtils; constructor TFolderTestSuite.Create(TestClass: TFileTestCaseClass; const ABaseFolder, AFileMask: string; ARecursive: Boolean); begin FBaseFolder := ABaseFolder; FFileMask := AFileMask; FRecursive := ARecursive; inherited Create(TestClass); end; procedure TFolderTestSuite.AddMethodTests(TestClass: TTestCaseClass; const NameOfMethod: string); var suite: ITestSuite; begin if TestClass.InheritsFrom(TFileTestCase) then begin suite := TTestSuite.Create(NameOfMethod); AddSuite(suite); ProcessBaseFolder(suite, TFileTestCaseClass(TestClass), NameOfMethod); end else begin AddTest(TestClass.Create(NameOfMethod)); end; end; procedure TFolderTestSuite.AddTests(testClass: TTestCaseClass); var MethodIter : Integer; NameOfMethod : string; MethodEnumerator: TMethodEnumerator; begin { call on the method enumerator to get the names of the test cases in the testClass } MethodEnumerator := nil; try MethodEnumerator := TMethodEnumerator.Create(testClass); { make sure we add each test case to the list of tests } for MethodIter := 0 to MethodEnumerator.Methodcount-1 do begin NameOfMethod := MethodEnumerator.nameOfMethod[MethodIter]; AddMethodTests(testClass, NameOfMethod); end; finally MethodEnumerator.free; end; end; procedure TFolderTestSuite.ProcessBaseFolder(Suite: ITestSuite; TestClass: TFileTestCaseClass; const NameOfMethod: string); begin ProcessFolder(Suite, TestClass, NameOfMethod, BaseFolder, FileMask, Recursive); end; procedure TFolderTestSuite.ProcessFile(Suite: ITestSuite; TestClass: TFileTestCaseClass; const NameOfMethod, FileName: string); begin suite.AddTest(TestClass.Create(NameOfMethod, FileName)); end; procedure TFolderTestSuite.ProcessFolder(Suite: ITestSuite; TestClass: TFileTestCaseClass; const NameOfMethod, Path, FileMask: string; Recursive: Boolean); var list: TStringDynArray; S: string; tmp: ITestSuite; begin list := TDirectory.GetFiles(Path, FileMask); for S in list do ProcessFile(suite, TestClass, NameOfMethod, S); if Recursive then begin list := TDirectory.GetDirectories(path); for S in list do begin tmp := TTestSuite.Create(TPath.GetFileName(S)); suite.AddSuite(tmp); ProcessFolder(tmp, TestClass, NameOfMethod, S, FileMask, true); end; end; end; constructor TFileTestCase.Create(const AMethodName, ATestFileName: string); begin FTestFileName := ATestFileName; inherited Create(AMethodName); end; function TFileTestCase.GetName: string; begin result := TPath.GetFileNameWithoutExtension(TestFileName); end; end. |