Regression Framework & Testing Framework

For a validation person, its imperative to have a regression framework of choice. I work in Virtual Platforms where I use .NET framework with Visual studio for my work. Several of my models come from other environments as well. In this article, I am going to describe infrastructure for build and testing models, with an example of virtual platform project.

For a given project I would describe frameworks as example, though several other possible frameworks are available. For maintaining your source code, you need a Source code repository - such as SVN. You need to setup a test Test Framework - Py.test (python.org) or Junit. And you need to build your models on a desired frequency such as daily, weekly, or other duration. You can use a continous build Framework - Cruisecontrol (ccnet.org)


Test Framework: Py.test
Unit level testing is very important for software projects. For complex large projects, development teams need to maintain a nightly regression infrastructure to avoid any 'bad code' making into tree of your code base and destroying the project.

py lib has several namespaces that help automate testing, such as: py.test - write/ deploy unit and functional tests across multiple machines; and others py.code, py.path. py.xml, py lib scripts, py.io, py.log etc.


py.test is a command line tool to collect, run and report about automated tests. It can be used to run a single test to several 10's of K tests. By default, all python modules with a test_*.py filename are inspected for finding tests:

    * functions with a name beginning with test_
    * classes with a leading Test name and test prefixed methods.
    * unittest.TestCase subclasses


Regression framework

I can use Cruisecontrol to setup nightly or weekly builds for a software project. Cruisecontrol will provide automated web reports/ email notifications. CruiseControl is a framework for a continuous build process. It includes, but is not limited to, plugins for email notification, Ant, and various source control tools. A web interface is provided to view the details of the current and previous builds. It allows one to perform a continuous integration of any software development process.CruiseControl is free, open-source software, distributed under a BSD-style license. It was originally created by employees of ThoughtWorks to allow for continuous integration on a project they were working on. It was later extracted into a stand-alone application.There is a version of CruiseControl for .NET called CruiseControl.NET (aka CCNet) and a version for Ruby called CruiseControl.rb.