|
Inspired by an article by Martin Fowler about Continuous Integration I tried to implement something similar for my company.
We are working with Delphi, Team Coherence, Finalbuilder and TestComplete. The later two are not really relevant for the implementation described here, the first two definitely are: The application is written in Delphi and it makes extended use of the TcVcsApi of Team Coherence.
The Team Coherence server is running on a Windows 2000 Server and for automated building and testing we use a dedicated build-and-test-station, nevertheless both tasks could be dedicated to the same machine.
Let's write down some specifications for the system:
- Automated compile checks and regression tests should be done after anyone checked in some changes. As the check in of sources is not always an atomic process, we will take precautions to let the users complete this task and give them a chance to establish source consistency.
- Each build and test will only notify the users, who actually checked in some sources. So in the following when talking about users we actually mean those.
- First we try to compile the sources. In case of error the users will get an error message, otherwise we send an OK message. Any error should be fixed immediately because it directly affects other programmers. As those errors mostly come from new files missing in the repository the fix should be no problem investigating the compiler output.
- After successful compilation we execute the regression tests and the users are informed about the results. A failed regression test has to be fixed before promoting the new version to QA.
- When all went well we deploy the files to the QA team and notify them having something to do.
So the first task is to check if anything was recently checked in. As the repository holds not only source files, but also language resources, documentation, test data and so on, we have to decide, what files are relevant for compilation and regression tests and so should be watched for check ins. The simplest approach to this is to watch only the source files, which should be located in a separate folder structure of the project. More elaborated solutions may include the test files and whatever seems appropriate.
The next question is: How do we know which revisions are checked in recently? What does "recently" mean in this context?
It turns out to be more effective to concentrate first on the second part of spec 1 which forces us only to do anything a little while after the latest check in. For this we calculate the latest allowed check in time some minutes before "now" and iterate through all files in our watched folder and subfolders testing the timestamp of the tip revision (which can be seen directly in the file info without stepping into the revisions) against our just calculated value. If we find a newer timestamp, we can calculate the time for the next check from it, thus guaranteeing not to be later than the given delay. Lets call this a "first level check". If all files are older than the specified value, we can dig deeper making a "second level check".
We define a file to be "changed" when there was a check in after the last successful regression test. We accomplished this by attaching a new version label to each set of revisions that made it through the regression tests. So we can iterate from the tip revision of each file down to the revision that has the label of the last positive test attached. Let's make an example:
Let "FileA" and "FileB" be two of our source files and let "V6.0.6.19" be the label marking the last successful regression test. A look into Team Coherence will show us
FileA:
FileB:
We can see that FileA didn't change since regression test V6.0.6.19, where FileB was changed two times since then.
So given the label "V6.0.6.19" for the last successful regression test, we iterate through all relevant files and for each file starting at the tip revision, we iterate down the revisions until we find the label "V6.0.6.19" or we reach the bottom of the revisions. If we found the label and the labelled revision is equal to the tip revision, the file didn't change. If the revisions are different the file has changed. If the label wasn't found at all, the file is new and we count this definitely as a change.
After finding some changed files we launch the build and test process notifying it about the timestamp we used and the list of users who made changes. The timestamp is necessary because of possible check ins during our check. The build process will get only revisions that are not newer than this timestamp and attach a special testing label to them. After a successful test this label will be removed.
Assuming the compile and regression tests didn't raise any problems and attached the new version label to the revisions, we get noticed of this label and take this one for our future checks. If someone by chance hit us during this process and checked something in, we will recognize this on our next check and start the process again. As the delay time is normally shorter than the time needed for the tests, the next check will start soon after the build process finished.
Now in the rare case that the test will fail we will find those changed files on every check we make, because they are changed referred to the reference label. As the last regression test failed, the old label of the last successful regression test remains the same. If we don't handle this properly we will start new tests over and over again spamming the users with error messages they already know and keeping them away from fixing the bugs.
To solve this we look for this testing label when iterating the revisions. If we find it in the tip revision, we don't need another test. But if we find it on a non-tip revision, someone has probably made a change to the buggy revision and this is worth a new test.
There is still some ugly thing in the whole system - can you see it? When nothing has changed after the last successful regression test, the timestamp of all files are old enough to stand the first level check and so we always go into a second level check. This is obviously unnecessary, as no new check in should be easy to detect. This is one case where a server side check-in triggered add-in will have an advantage. But we can cope with this, too. Remember, on the first level check we look at the timestamp of each file notifying when it is "too young". Now when we remember the timestamp of the youngest file anyway, we can easily detect a newer youngest file right in the first level check. If the timestamp of the youngest file is not newer than that from the last check, nothing happened.
|
