xUnit.net - Unit Testing for .NET

UPDATED RELEASE: xUnit.net 1.1

Sun, 18 May 2008 18:15:32 GMT

Placeholder for the next xUnit.net release

NEW POST: Anyone having trouble with F# / xUnit

BradWilson — Sat, 17 May 2008 21:54:01 GMT

I've opened a bug to track this problem.

CREATED ISSUE: Cannot run static tests in static class

BradWilson — Sat, 17 May 2008 21:52:14 GMT

If the class is declared as static, then xUnit.net skips it because of the "IsAbstract" tests in TypeUtility.IsTestClass.

If the class is not declared as static, the tests run as expected.

This seems to have affected recent drops of F# and testing, because the "default" class that it creates is marked as a static class, but it should also be fixed for C#/VB.NET for users who want to write static tests in static classes.

NEW POST: Anyone having trouble with F# / xUnit

BradWilson — Sat, 17 May 2008 21:35:57 GMT

Now I'm actually able to reproduce the behavior with C#, without using F#. It appears that a static class is marked as "abstract". The question is, did the framework change or did something inside of xUnit.net change? Because clearly, xUnit.net used to be able to run F# tests without any problems.

using System;
using System.Reflection;

public static class Foo { }

public class MainClass {
    public static void Main() {
        Console.WriteLine("IsAbstract = {0}", typeof(Foo).IsAbstract);
    }
}

When I run this problem, I see "true" output.

The investigation continues...

Source code checked in

BradWilson — Sat, 17 May 2008 20:32:26 GMT

Fixed a bug where an assertion is thrown in xunit.gui when loading an assembly with no tests in it.

UPDATED WIKI: Home

jimnewkirk — Sat, 17 May 2008 18:04:19 GMT

About the Project

Blog posts on xUnit.net

May 16, 2008 - David Tchepak uses xUnit.net to test the first iteration of his game (Link)
May 14, 2008 - Bembeng Arifin on using xunit.console.exe from within Visual Studio 2008 Express (Link)
May 9, 2008 - Damon Payne explores concurrent unit testing with xUnit.net (Part 0, Part 1)
May 5, 2008 - Torkel Ödegaard on Watin and xUnit.net integration testing (Link)
April 22, 2008 - Moq mocking framework now uses xUnit.net for its unit testing (Link)
April 18, 2008 - Scott Hanselman's Weekly Source Code reviews xUnit.net's extensibility APIs (Link)
April 4, 2008 - Matt Podwysocki looks at our new ASP.NET MVC Preview 2 support (Link)
January 31. 2008 - Ben Hall on: xUnit.net, xUnit.net Extensions, Creating Your Own Extensions

How Do I...?

Links to Resources

.NET Framework 2.0 Redistributable (x86, x64)
TestDriven .NET
Resharper
CruiseControl .NET
NUnit

NEW POST: Anyone having trouble with F# / xUnit

cmiles — Sat, 17 May 2008 17:02:10 GMT

Thanks for looking into that Brad - that information is great. I posted this up on hubFS also, and will add your info about the class being abstract to that post to see if anyone has any interesting information to add -

Thanks again,

Charles Miles

NEW POST: Anyone having trouble with F# / xUnit

BradWilson — Sat, 17 May 2008 15:24:34 GMT

The problem appears to be that the F# class that is created is marked as abstract in the CLR, which causes this to skip over the class:

public static class TypeUtility
{
    public static bool IsTestClass(ITypeInfo type)
    {
        return !type.IsAbstract && (HasRunWith(type) || ContainsTestMethods(type));
    }
}

Just to verify that it really is abstract, I tried to make one with Activator in the immediate window while I was broken in the debugger:

System.Activator.CreateInstance(type.Type)
'System.Activator.CreateInstance(type.Type)' threw an exception of type 'System.MissingMethodException'
    base {System.MissingMemberException}: {"Cannot create an abstract class."}
    Message: "Cannot create an abstract class."

And it really is abstract.

I don't know if this was a recent change for F#. There were no flags that I could see in the Visual Studio project that would cause it to create (or not) abstract classes. The dizzying array of compiler switches (from the command line) weren't much help, either. :) When I see Matt online next, I'll ask him what experience he's had with this.

Thanks,
Brad

NEW POST: Anyone having trouble with F# / xUnit

cmiles — Sat, 17 May 2008 03:25:54 GMT

After looking at this just a little more I think I have two examples that show my question/confusion/problem more clearly.

The first example is in a style that I read about on several blogs and thought looked great as an option for use in F# because - as I understood it - no class type is needed for each test/fixture. When I build the code below and run the console runner against the resulting dll it reports Total Tests: 0. Is this syntax not possible/supported? Or are there other code constructs/compilation options needed that I am not aware of to get this to work?

#light

#r @"C:\Users\charles\Documents\Dev\xunit-1.0\xunit.dll"

open Xunit

[<Fact>]
let Test_nic = Assert.Equal("t", "t")

The second example is in a style that I see when people give examples of testing with nunit in F#. Building this and running the console runner against the resulting dll gives the expected result of Total Tests: 1.

#light

#r @"C:\Users\charles\Documents\Dev\xunit-1.0\xunit.dll"

open Xunit

type inclass_tests() = class

    [<Fact>]
    member x.Test_InClass() =
        Assert.Equal("t", "t")

end

Thanks for any insight and help,
Charles Miles

NEW POST: Anyone having trouble with F# / xUnit

cmiles — Fri, 16 May 2008 19:35:25 GMT

New F# Project in Visual Studio 2008 with F# 1.9.4.15. New F# Source file, code below , build (exe is the default but I tried both exe and dll) - run the console runner on the resulting dll or exe - it will not report any tests in the dll (one other person on hubFS also replorted this experience in reply to a post of mine there).

This code compiles/builds without reporting any errors - it is taken from http://weblogs.asp.net/podwysocki/archive/2008/04/04/relooking-at-xunit-net-rc2.aspx - I am using this code from Matthew Podwysocki to test in order to try and avoid any syntax issues related to my newness to F#.

Happy to send dlls or any other needed info - hope this is a decent test case let me know if you need something else/different.

(reference path to xunit will need to be changed in the code below)

#light

#r @"C:\Users\charles\Documents\Dev\xunit-1.0\xunit.dll"

open System
open System.Collections.Generic
open Xunit

type Stack<'t> = class
val elements : LinkedList<'t>

new() = { elements = new LinkedList<'t>() }

member x.IsEmpty
    with get() = x.elements.Count = 0

member x.Push element =
    x.elements.AddFirst(element:'t)

member x.Top
    with get() =
      if x.elements.Count = 0 then
        raise (InvalidOperationException("cannot top an empty stack"))
      x.elements.First.Value

member x.Pop =
    let top = x.Top
    x.elements.RemoveFirst()
    top
end

[<Fact>]
let NoElementsShouldBeEmpty () =
let stack = new Stack<string>()
Assert.True(stack.IsEmpty)

NEW POST: Anyone having trouble with F# / xUnit

BradWilson — Fri, 16 May 2008 16:30:30 GMT

To the best of my knowledge, TestDriven.NET will not work with F# tests, because TestDriven.net doesn't know how to parse F#. Same problem with ReSharper, I would imagine.

However, the other runners -- GUI, console, MSBuild -- should all be able to see F# tests. Do you have a simple repro case?

NEW POST: Anyone having trouble with F# / xUnit

cmiles — Fri, 16 May 2008 14:59:32 GMT

I am having trouble with xUnit 1.0 and F# 1.9.4.15 - both console test runner and TDD.NET do not see any tests in my F# code even though I have several functions with Asserts with [<Fact>] as their attribute.

I assumed that the trouble was with my code so I set up a project with the code here: http://weblogs.asp.net/podwysocki/archive/2008/04/04/relooking-at-xunit-net-rc2.aspx and built it - but the console runner and TDD.NET still did not see any tests?

My tests do work in C# with the same dll referenced and same console runner.

I am a beginner with F# so I think that it is fairly likely that the is something I do not understand about F# rather than a problem with xUnit - can anyone confirm they are using xUnit 1.0 and F# 1.9.4.15 together successfully? Or better yet does anyone have any suggestions on troubleshooting this problem?

Thanks,

Charles Miles

charles@cmiles.info

COMMENTED FEATURE: Collection asserts

slorion — Fri, 16 May 2008 09:16:09 GMT

Collection asserts are sorely missing ... ie AreEqual, IsSubsetOf, AreEquivalent, AllItemsAreUnique, AllItemAreInstanceOf, etc.
Comments: ** Comment from web user: slorion **

The thing is, talking for myself, I already got these comparers and much more with PowerCollections. I am giving feedback on what I feel could be added to this project to make it more useful. Having these available for everyone without the need to use 3rd party library or write our own would be nice. The lexicographical comparison (the one you implemented) already cover a lot of ground. The other obvious comparison is where order does not matter (comparing sets) which you also coded. Beyond that, I can hardly see any other.

Now, the whole point of my comment about the sample is that it is bugged.

- it should return 1 when x has more elements than y and -1 if x has less elements than y (currently, it returns the inverse of that).
- it should handle null collections
- it should handle null values inside collections (at least not crash)

Now that I care that much about it, but I thought I should take 1 min of my time and notice you. Seeing the way it goes, I will stop doing that.

About null values, I don't see where there can be confusion: null is equal to null and null is less than anything else, that's how every .NET comparers handle them.

COMMENTED FEATURE: Collection asserts

jimnewkirk — Fri, 16 May 2008 07:45:30 GMT

Collection asserts are sorely missing ... ie AreEqual, IsSubsetOf, AreEquivalent, AllItemsAreUnique, AllItemAreInstanceOf, etc.
Comments: ** Comment from web user: jimnewkirk **

I really do not want to get into a back and forth discussion about this. The reason we put in the extensibility point was to not have these types of disagreements over what at times can be considered one preference over another. It is my belief that when comparing two collections you will get into these types of arguments/discussions. That is why I dont believe it belongs in the framework,

Your point about null values really nails this point for me - how do you compare the collections when there are null values? I just dont think there is answer that works that is why I think you can take what we have as a starting point or not and just write one that handles the cases you believe are important and make sense in your environment. You only have to write it once and then check it into your own source tree, its not like the code is duplicated.

COMMENTED FEATURE: Collection asserts

slorion — Fri, 16 May 2008 07:06:24 GMT

Collection asserts are sorely missing ... ie AreEqual, IsSubsetOf, AreEquivalent, AllItemsAreUnique, AllItemAreInstanceOf, etc.
Comments: ** Comment from web user: slorion **

Fine have it your way. Your sample is still bugged.

COMMENTED FEATURE: Collection asserts

BradWilson — Fri, 16 May 2008 06:53:19 GMT

Collection asserts are sorely missing ... ie AreEqual, IsSubsetOf, AreEquivalent, AllItemsAreUnique, AllItemAreInstanceOf, etc.
Comments: ** Comment from web user: BradWilson **

You're missing my point, which is: there are clearly going to be several variations on what the "correct" comparer is for arrays, so we are NOT going to write the "one true comparer". Write your own.

COMMENTED FEATURE: Collection asserts

slorion — Fri, 16 May 2008 06:50:53 GMT

Collection asserts are sorely missing ... ie AreEqual, IsSubsetOf, AreEquivalent, AllItemsAreUnique, AllItemAreInstanceOf, etc.
Comments: ** Comment from web user: slorion **

My rules ?? This is per the interface documentation ! Read the doc ... There is no official vs non-official. You implement the interface as documented, that's it.

COMMENTED FEATURE: Collection asserts

BradWilson — Fri, 16 May 2008 02:24:08 GMT

Collection asserts are sorely missing ... ie AreEqual, IsSubsetOf, AreEquivalent, AllItemsAreUnique, AllItemAreInstanceOf, etc.
Comments: ** Comment from web user: BradWilson **

Your comments are an excellent example of why we will not be making "one official set of comparers". Your rules might not be the rules other people want to use.

COMMENTED FEATURE: Collection asserts

slorion — Thu, 15 May 2008 23:06:49 GMT

Collection asserts are sorely missing ... ie AreEqual, IsSubsetOf, AreEquivalent, AllItemsAreUnique, AllItemAreInstanceOf, etc.
Comments: ** Comment from web user: slorion **

You got the comparers wrong in the sample:

var x = new int[] { 1, 2, 3 };
var y = new int[] { 1, 2 };

int result = CollectionEqualityComparer<int>.Compare(x, y);

result should be 1, not -1, since x > y. The code also does not handle null values. As you can see, the code is simple, but having a proper tested implementation included would be valuable since it is easy to make mistakes.

COMMENTED FEATURE: A test with no asserts should yield a warning

kevwil — Thu, 15 May 2008 22:05:43 GMT

A test which has no asserts should result in a warning (perhaps even an error?), given the presence of Assert.Throws<> and Assert.DoesNotThrow<>.
Comments: ** Comment from web user: kevwil **

I disagree. Sometimes I write tests with only mock object expectations, because the code under test only interacts with dependencies and has no other local logic.

xUnit.net - Unit Testing for .NET

UPDATED RELEASE: xUnit.net 1.1

NEW POST: Anyone having trouble with F# / xUnit

CREATED ISSUE: Cannot run static tests in static class

NEW POST: Anyone having trouble with F# / xUnit

Source code checked in

UPDATED WIKI: Home

About the Project

Recent News

Blog posts on xUnit.net

How Do I...?

Links to Resources

NEW POST: Anyone having trouble with F# / xUnit

NEW POST: Anyone having trouble with F# / xUnit

NEW POST: Anyone having trouble with F# / xUnit

NEW POST: Anyone having trouble with F# / xUnit

NEW POST: Anyone having trouble with F# / xUnit

NEW POST: Anyone having trouble with F# / xUnit

COMMENTED FEATURE: Collection asserts

COMMENTED FEATURE: Collection asserts

COMMENTED FEATURE: Collection asserts

COMMENTED FEATURE: Collection asserts

COMMENTED FEATURE: Collection asserts

COMMENTED FEATURE: Collection asserts

COMMENTED FEATURE: Collection asserts

COMMENTED FEATURE: A test with no asserts should yield a warning