Automatic Class Tester

Reviewed: 1.1.2925.23829 (Oct 30, 2008)

jferris — Thu, 30 Oct 2008 17:25:24 GMT

Rated 5 Stars (out of 5) - Not only is this great for helping us maintain our code coverage goals, it helps find mis-wired properties for us. Even though we use automatic properties wherever possible, there are times that we cannot use them and bring the risk of human error to the party. Indispensable tool!

COMMENTED ISSUE: Add support for Abstract Types and Interfaces to the TypeFactory

furie — Thu, 07 Feb 2008 15:28:30 GMT

Use a mock repository to add support for abstract types (abstract classes and interfaces) to the TypeFactory
Comments: ** Comment from web user: furie **

This can be done now by extending the TypeFactory an instance of it to a tester constructor.
using Castle.Core.Interceptor;
using Castle.DynamicProxy;
public class MyTypeFactory : TypeFactory
{
static ProxyGenerator proxyGenerator = new ProxyGenerator();
public override object CreateRandomValue(Type type)
{
if (type.IsInterface)
return proxyGenerator.CreateInterfaceProxyWithoutTarget(type, new StandardInterceptor());
return base.CreateRandomValue(type);
}
public override bool CanCreateInstance(Type type)
{
bool result = base.CanCreateInstance(type);
if (!result)
result = type.IsInterface && !type.IsGenericTypeDefinition;
retur result;
}

COMMENTED ISSUE: PropertyChecker to optionally test properties throw ArgumentNullException when setting null value

furie — Thu, 07 Feb 2008 15:16:59 GMT

Often a precondition of a property is that the value set is non null and therefore the property should throw an ArgumentNullExtension. Would be good to switch this checking on at class level. Also optionally checking string.Empty throws would be great also.
Comments: ** Comment from web user: furie **

Actually I think this would be better implemented as a seperate 'NullParam' tester that tested Constructors, Properties, and Methods.

CREATED ISSUE: PropertyChecker to optionally test properties throw ArgumentNullException when setting null value

furie — Thu, 07 Feb 2008 11:12:34 GMT

COMMENTED ISSUE: Add support for Abstract Types and Interfaces to the TypeFactory

furie — Thu, 07 Feb 2008 10:59:53 GMT

Use a mock repository to add support for abstract types (abstract classes and interfaces) to the TypeFactory
Comments: ** Comment from web user: furie **

Consider using Castle DynamicProxy 2 to create an instance of an abstract class. It's lightweight, quick and simple. RhinoMocks uses it to create its mocks.

CREATED ISSUE: Strongly signed assemblies

igorbrejc — Wed, 30 Jan 2008 08:18:51 GMT

Hello,

Thanks for a great tool and a great idea. The only thing I really miss is being able to use the library in a strongly signed test projects. Would it be possible for you to include this in the next release?

Best regards,
Igor

Source code checked in

molmorg — Wed, 16 Jan 2008 09:52:45 GMT

Added AddPropertyExclusions and AddConstructorExclusion to the AssemblyTester

COMMENTED ISSUE: Compare two provided objects instead of comparing randomly generated value

Jericho — Fri, 11 Jan 2008 17:14:27 GMT

Comparing randomly genrated values is great for detecting bugs where a public property uses the wrong private field. Another test that would also be very useful is to compare all the properties of two provided objects. What I'm envisioning is something like this (this is only psendo-code):

[TestMethod]
public void TestPerson()
{
Person p1 = new Person();
p1.Name = "Bob Smith";
string personId = MyDAL.Save(p1);
Person p2 = MyDAL.Fetch(personId);
PropertyTester tester = new PropertyTester(p1, p2);
tester.TestProperties();
}
Comments: ** Comment from web user: Jericho **

I just submitted a patch with the code for a "PropertyComparer" class that will compare all the properties on two objects to make sure that the values are the same.

Patch Uploaded

Jericho — Fri, 11 Jan 2008 17:13:05 GMT

Jericho has uploaded a patch.

Description:
I wrote a new tester to compare the properties of two provided objects. Here's a unit test example that demonstrates how it works:

using System;
using System.Collections.Generic;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using TheJoyOfCode.QualityTools.Tests.TestSubjects;
using Rhino.Mocks;

namespace TheJoyOfCode.QualityTools.Tests
{
[TestClass]
public class PropertyComparerTest
{
[TestMethod]
public void CompareProperties_Good()
{
DummyGood dummy1 = new DummyGood();
dummy1.Bool = true;

DummyGood dummy2 = new DummyGood();
dummy2.Bool = true;

PropertyComparer comparer = new PropertyComparer(dummy1, dummy2);
comparer.CompareProperties();
}

[TestMethod]
[ExpectedException(typeof(PropertyCompareException))]
public void CompareProperties_PropertiesDifferent() {
DummyGood dummy1 = new DummyGood();
dummy1.Bool = true;

DummyGood dummy2 = new DummyGood();
dummy2.Bool = false;

PropertyComparer comparer = new PropertyComparer(dummy1, dummy2);
comparer.CompareProperties();
}

[TestMethod]
[ExpectedException(typeof(PropertyCompareException))]
public void CompareProperties_TypesDifferent() {
DummyGood dummy1 = new DummyGood();
dummy1.Bool = true;

DummyNoEvent dummy2 = new DummyNoEvent();
dummy2.SomeInt = 8;

PropertyComparer comparer = new PropertyComparer(dummy1, dummy2);
comparer.CompareProperties();
}
}
}

CLOSED ISSUE: Add support for creation of Arrays to TypeFactory

molmorg — Wed, 09 Jan 2008 17:31:08 GMT

TypeFactory can't currently create arrays (e.g. object[])

Source code checked in

molmorg — Wed, 09 Jan 2008 17:31:06 GMT

Added support for creation of Arrays. The array is created with a random number of empty elements (up to 50).

UPDATED WIKI: AssemblyTester

molmorg — Wed, 09 Jan 2008 16:19:11 GMT

The AssemblyTester

The AssemblyTester uses the PropertyTester and ConstructorTester and can remove much of the coding previously required to use them by applying these tests to all the types in an assembly, subject to exclusions that you can specify.

How it works

To test an assembly simply create an instance of the AssemblyTester passing it the name of the Assembly to test.

AssemblyTester tester = new AssemblyTester("TheJoyOfCode.QualityTools");

Now you can call the AssemblyTesters TestAssembly method which looks like this:

     public void TestAssembly(bool testProperties, bool testConstructors)

The testProperties and testConstructors parameters should be fairly self explanatory!

The example below aims to test both constructors and properties in the "TheJoyOfCode.QualityTools.Tests.DummyProject" assembly.

     AssemblyTester tester = new AssemblyTester("ExampleAssembly");
     tester.TestAssembly(true, true);

The AssemblyTester will throw an exception with a detailed description of errors encountered when testing an assembly. Here's an example:

AssemblyTesterTest.AssemblyTester_PassWithExclusions : FailedTheJoyOfCode.QualityTools.AssemblyTestException: 
Testing the assembly TheJoyOfCode.QualityTools.Tests.DummyProject produced the following errors:
------------
The get value of the 'Dummy4' property on the type 'TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.IncorrectProperties' 
did not equal the set value (in: '', out: 'fe6faacc-3ea8-45a0-8c5c-dc2e0021065b')
 [TestProperties,Type:TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.IncorrectProperties]
 
The value of the 'Dummy1' property did not equal the value set with the 'dummy1' 
constructor parameter (in: 'cf04ca44-73dc-4147-bb3e-2a1a0d2c52b3', out: '819e635d-4dc6-4e6c-b141-75f0212cb242')
 [TestConstructors,Type:TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.IncorrectConstructors]
 
The type TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.GenericClass`1 is an open 
Generic Definition (e.g. Class<T>. Only closed generic types Class<int> are currently supported.
 [TestConstructors,Type:TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.GenericClass`1[T]]
 
Cannot create an instance of TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.GenericClass`1[T] 
because Type.ContainsGenericParameters is true. 
[TestProperties,Type:TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.GenericClass`1[T]]

Not all types in an assembly qualify for automatic testing of constructors or properties. Looking at the errors produced above it is clear that the two first errors are should be fixed (A property which is not mapped correctly and a constructor which does not map correctly to the corresponding property). The last two errors indicates that the AssemblyTester can not test this particular type because it contains generic parameters.

Exclusions

To still use the AssemblyTester on this module we can skip these last two issues. This can be done in two ways, specifying one or more types to exclude or specifying a namespace to exclude.

Excluding a type

The example excludes the class which is not qualified for automatic testing to the AssemblyTester.

     AssemblyTester tester = new AssemblyTester("TheJoyOfCode.QualityTools.Tests.DummyProject");
     tester.Exclusions.AddType(typeof(GenericClass<>));
     tester.TestAssembly(AssemblyTester.TestTarget.Properties | AssemblyTester.TestTarget.Constructors);

When running the test again the AssemblyTester only produces the following errors.

TheJoyOfCode.QualityTools.AssemblyTestException: Testing the assembly 
TheJoyOfCode.QualityTools.Tests.DummyProject produced the following errors:
------------
The get value of the 'Dummy4' property on the type 'TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.IncorrectProperties' 
did not equal the set value (in: '', out: '1e5101b5-e9e9-4d69-b13c-83863f70a7b6')
 [TestProperties,Type:TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.IncorrectProperties]
 
The value of the 'Dummy1' property did not equal the value set with the 'dummy1' 
constructor parameter (in: '2bf2002e-b505-4413-a8f9-97036c40f720', out: '8a447a00-2f17-4d27-bb25-1fc66bf2c39d')
 [TestConstructors,Type:TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.IncorrectConstructors]

The errors above are errors which the AssemblyTester has been designed to detect; errors in mapping properties to fields and mapping values to constructors. So to make the test pass we need to fix those errors!

Excluding a namespace

The example below excludes all types in the namepace "ExampleAssembly.SomeNamespace" namespace but does not exclude types in any sub namespaces.

     tester.Exclusions.AddNamespace("ExampleAssembly.SomeNamespace", false);

We could specify that all types within a numespace and sub-namespaces are excluded like so:

     tester.Exclusions.AddNamespace("ExampleAssembly.SomeNamespace", true);

For example, this would also exclude any type in the "ExampleAssembly.SomeNamespace.SomeSubNamespace" too.

You can have as many exclusions as you like and can combine type exclusions with namespace exclusions. Easy peasy.

If you need to exclude a type from an assembly test because of one funky property or constructor you can always target that type directly with the PropertyTester or ConstructorTester and exclude the property/type specifically. Best of both worlds.

Adding your own exclusion rules

It's easy to extend the exclusion policies available in the AssemblyTester by creating your own class that inherits from TestExclusion. Imagine you want to be able to exclude any class that starts with the letter 'S'. First we need a new

private class StartsWithSExclusion : TestExclusion
{
    public override bool IsExcluded(Type type)
    {
        return type.Name.StartsWith("S");
    }
}

Now we just have to add that type to the Exclusions as follows:

     tester.Exclusions.Add(new StartsWithSExclusion());

... and any types whose name starts with an S won't be tested.

FAQ

Q: It's nice that I can exclude whole types and namespaces using the AssemblyTester but why can't I specify a single property on a type that should be excluded.
A: This feature is deliberately not included to date. The practice in this case is to exclude the type from the assembly test like so:

AssemblyTester tester = new AssemblyTester("MyType, MyAssembly");
tester.Exclusions.AddType(typeof(TypeWithUntestableProperty));
tester.TestAssembly(true, true);

Now, add another test to target that class using the PropertyTester directly:

PropertyTester tester = new PropertyTester(new TypeWithUntestableProperty));
tester.TestProperties();

UPDATED WIKI: AssemblyTester

molmorg — Wed, 09 Jan 2008 15:35:04 GMT

The AssemblyTester

How it works

To test an assembly simply create an instance of the AssemblyTester passing it the name of the Assembly to test.

AssemblyTester tester = new AssemblyTester("TheJoyOfCode.QualityTools");

Now you can call the AssemblyTesters TestAssembly method which looks like this:

     public void TestAssembly(bool testProperties, bool testConstructors)

     AssemblyTester tester = new AssemblyTester("ExampleAssembly");
     tester.TestAssembly(true, true);

The AssemblyTester will throw an exception with a detailed description of errors encountered when testing an assembly. Here's an example:

AssemblyTesterTest.AssemblyTester_PassWithExclusions : FailedTheJoyOfCode.QualityTools.AssemblyTestException: 
Testing the assembly TheJoyOfCode.QualityTools.Tests.DummyProject produced the following errors:
------------
The get value of the 'Dummy4' property on the type 'TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.IncorrectProperties' 
did not equal the set value (in: '', out: 'fe6faacc-3ea8-45a0-8c5c-dc2e0021065b')
 [TestProperties,Type:TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.IncorrectProperties]
 
The value of the 'Dummy1' property did not equal the value set with the 'dummy1' 
constructor parameter (in: 'cf04ca44-73dc-4147-bb3e-2a1a0d2c52b3', out: '819e635d-4dc6-4e6c-b141-75f0212cb242')
 [TestConstructors,Type:TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.IncorrectConstructors]
 
The type TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.GenericClass`1 is an open 
Generic Definition (e.g. Class<T>. Only closed generic types Class<int> are currently supported.
 [TestConstructors,Type:TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.GenericClass`1[T]]
 
Cannot create an instance of TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.GenericClass`1[T] 
because Type.ContainsGenericParameters is true. 
[TestProperties,Type:TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.GenericClass`1[T]]

Exclusions

To still use the AssemblyTester on this module we can skip these last two issues. This can be done in two ways, specifying one or more types to exclude or specifying a namespace to exclude.

Excluding a type

The example excludes the class which is not qualified for automatic testing to the AssemblyTester.

     AssemblyTester tester = new AssemblyTester("TheJoyOfCode.QualityTools.Tests.DummyProject");
     tester.Exclusions.AddType(typeof(GenericClass<>));
     tester.TestAssembly(AssemblyTester.TestTarget.Properties | AssemblyTester.TestTarget.Constructors);

When running the test again the AssemblyTester only produces the following errors.

TheJoyOfCode.QualityTools.AssemblyTestException: Testing the assembly 
TheJoyOfCode.QualityTools.Tests.DummyProject produced the following errors:
------------
The get value of the 'Dummy4' property on the type 'TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.IncorrectProperties' 
did not equal the set value (in: '', out: '1e5101b5-e9e9-4d69-b13c-83863f70a7b6')
 [TestProperties,Type:TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.IncorrectProperties]
 
The value of the 'Dummy1' property did not equal the value set with the 'dummy1' 
constructor parameter (in: '2bf2002e-b505-4413-a8f9-97036c40f720', out: '8a447a00-2f17-4d27-bb25-1fc66bf2c39d')
 [TestConstructors,Type:TheJoyOfCode.QualityTools.Tests.DummyProject.WithErrors.IncorrectConstructors]

Excluding a namespace

The example below excludes all types in the namepace "ExampleAssembly.SomeNamespace" namespace but does not exclude types in any sub namespaces.

     tester.Exclusions.AddNamespace("ExampleAssembly.SomeNamespace", false);

We could specify that all types within a numespace and sub-namespaces are excluded like so:

     tester.Exclusions.AddNamespace("ExampleAssembly.SomeNamespace", true);

Adding your own exclusion rules

private class StartsWithSExclusion : TestExclusion
{
    public override bool IsExcluded(Type type)
    {
        return type.Name.StartsWith("S");
    }
}

Now we just have to add that type to the Exclusions as follows:

     tester.Exclusions.Add(new StartsWithSExclusion());

... and any types whose name starts with an S won't be tested.

FAQ

AssemblyTester tester = new AssemblyTester("MyType, MyAssembly");
tester.Exclusions.AddType(typeof(TypeWithUntestableProperty));
tester.TestAssembly(true, true);

And add another test to target that class using the PropertyTester

/// TODO + unfinished +

Source code checked in

Bruusi — Wed, 09 Jan 2008 12:28:28 GMT

Fixed documentation references.

NEW POST: Problem with DateTime property

Jericho — Mon, 07 Jan 2008 21:21:12 GMT

I figured it out: the miliseconds are not preserved in my class. I guess I'll simply ignore this property.

Source code checked in

molmorg — Mon, 07 Jan 2008 17:57:18 GMT

Source code checked in

molmorg — Mon, 07 Jan 2008 17:43:19 GMT

Added pluggable TypeFactory (implements ITypeFactory) and fixed a bug in one of the error messages (in and out reversed). Added test using RhinoMocks to verify that GetSet, Get and Set only properties.

CREATED ISSUE: Compare two provided objects instead of comparing randomly generated value

Jericho — Mon, 07 Jan 2008 16:14:48 GMT

CREATED ISSUE: Exclude class property when testing Assembly

Jericho — Mon, 07 Jan 2008 16:05:18 GMT

The Assembly tester currently allows to exclude an entire namespace or an entire class but it would be great to also be able to exclude a specific property of a specific class. For example:

[code]
[Test]
public void AssemblyTester_PassWithExclusions()
{
AssemblyTester tester = new AssemblyTester("ExampleAssembly");
tester.Exclusions.AddProperty("ExampleAssembly.SomeNamespace.SomeClass", "PropertyName");
tester.TestAssembly(true, true);
}
[/code]

NEW POST: Problem with DateTime property

Jericho — Mon, 07 Jan 2008 15:47:47 GMT

When testing one of my classes I am getting an error regarding a DateTime property but the get value and the set value match:

Test method WebExApiTest.WebExXmlTypePropertiesTest.TestEventSchedule threw exception: TheJoyOfCode.QualityTools.PropertyTestException: The get value of the 'StartDate' property on the type 'WebExApi.EventSchedule' did not equal the set value (in: '1/1/0001 12:02:34 AM', out: '1/1/0001 12:02:34 AM').

Can anybody examplain why I'm getting an error even though the values appear to be the same?