xUnit.net - Unit Testing for .NET

Source code checked in, #33927

BradWilson — Fri, 03 Jul 2009 23:38:52 GMT

Added Ctrl+A to search box, assemblies box, and test list box. The UI can be manipulated (in particular, refined) while tests are running without crashing. Diabled unload/reload menu items while tests are running. Fixed an insidious bug in the progress control which only manifests when your test count > the pixel width of the progress control.

Source code checked in, #33926

BradWilson — Fri, 03 Jul 2009 22:29:41 GMT

Added right click to assemblies to allow unload and reload

Source code checked in, #33925

BradWilson — Fri, 03 Jul 2009 22:04:04 GMT

Asynchronously updating UI for the win. Oh, plus a counter!

Source code checked in, #33922

BradWilson — Fri, 03 Jul 2009 18:19:29 GMT

Re-enabled TD.NET tests in xunit.tests.msbuild

Source code checked in, #33921

BradWilson — Fri, 03 Jul 2009 18:17:33 GMT

Re-enabled TestDriven.net tests using real TestDriven.Framework.dll

Source code checked in, #33920

BradWilson — Fri, 03 Jul 2009 18:05:45 GMT

Updated ObjectModel so that last run status is contained within the TestMethod class. Simplified ITestMethodRunnerCallback by removing TestPassed, TestFailed, and TestSkipped (now just check the last run status in TestFinished). Updated the GUI to use the new API. Moved SkipReason out of TestMethod, and put it into TestSkippedReason instead. Tests which have a skip reason now automatically start being marked as skipped.

Source code checked in, #33919

BradWilson — Fri, 03 Jul 2009 16:29:27 GMT

Converted TestClassTests to Moq. Renamed PassedResultAcceptance to PassedResultAcceptanceTests.

Source code checked in, #33918

jimnewkirk — Fri, 03 Jul 2009 16:10:59 GMT

Removed StubLogger, replaced with Moq

Commented Issue: Runner should deal with ambiguity against non-Fact methods (and/or TD.NET should deal with runner objections)

bartelink2 — Fri, 03 Jul 2009 11:11:16 GMT

Similar to 7973, running CTP1

// Running this class via TD.NET leaves the runner hanging with a "Testing Started" message in the toolbar
public class WithAmbiguityXUnitCantHandle
{
// Running this via TD.NET works
[Fact]
public void CanDoItThisWay()
{
CanDoItBothWays(true);
}

// Running this via TD.NET works
[Fact]
public void CanDoItBothWays()
{
CanDoItBothWays(false);
}

// Needs to be public for this to repro
public void CanDoItBothWays(bool param)
{
}
}

Running via runner.gui:-

running System.ArgumentException: Ambiguous method named CanDoItBothWays in type WithAmbiguityXUnitCantHandle
at Xunit.Sdk.Executor.RunTests..ctor(Executor executor, String _type, List`1 _methods, Object _handler)
at Xunit.ExecutorWrapper.RethrowWithNoStackTraceLoss(Exception ex)
at Xunit.ExecutorWrapper.CreateObject(String typeName, Object[] args)
at Xunit.ExecutorWrapper.RunTests(String type, List`1 methods, Predicate`1 callback)
at Xunit.TestRunner.<>c__DisplayClasse.<RunTests>b__c()
at Xunit.TestRunner.PreserveWorkingDirectoryAndCatchExceptions(TestRunnerDelegate func)

Making the impl method non-public or renaming it to remove the conflict [obviously] resolves it
Comments: ** Comment from web user: bartelink2 **

Not 100 sure making the duplicate private resolves it anymore but unfortunately dont have time to verify right now...

Created Issue: Runner should deal with ambiguity against non-Fact methods (and/or TD.NET should deal with runner objections)

bartelink2 — Fri, 03 Jul 2009 11:08:42 GMT

Created Issue: When exeption is thrown in derived DataAttribute class, tests run infinitely

matthaw — Thu, 02 Jul 2009 19:39:38 GMT

Xunit never cancels the test run if the derived DataAttribute class throws an exception, but infinitely executes.

public class KillIt : DataAttribute {
public override IEnumerable<object[]> GetData(MethodInfo method, Type[] paramTypes) {
throw new Exception();
}
}

public class Rar {
[Theory, KillIt]
public class ShouldntHang(string a) {
Assert.True(true);
}
}

New Post: Support Visual Studio Test Runner (Please Read and Respond!)

bartelink2 — Wed, 01 Jul 2009 21:56:05 GMT

Not sure if I was sub'd to Roy at the time (think I unsub'd and resub'd for a bit back then) but I, and I assume others also scanned his post and took things on board to a degree, which certainly delayed me evaluating. Had I seen the rebuttal in time (not sure why it didnt register - I'm sub'd to your blog for ages...) I'd have wasted less time with other frameworks. Keep up the great work and continue to be strong on keeping the core lean.

New Post: Support Visual Studio Test Runner (Please Read and Respond!)

BradWilson — Wed, 01 Jul 2009 15:29:17 GMT

There's not much I can do about Roy's post. New things scare people. :)

We're aware of the desire for people to be able to publish results into TFS, and although we're not going to get to that for 1.5, it's definitely on our list.

New Post: Support Visual Studio Test Runner (Please Read and Respond!)

bartelink2 — Wed, 01 Jul 2009 08:07:53 GMT

@@ianceicys:

@ianceicys

@ianceicys: And another thing. You'll run into ExpectedException. I was able to transform 200 in a flash using a CodeRush template, and a CR optimise usings worked really well to clean up references. Well worth assessing how many you have and how long its going to take you to convert them all to Assert.Throws

@BradWilson: Re the actual question:

VS integration is irrelevant for me as I use RedGreen or TD.NET - I'm trying to get away from MSTEST and MSTESTism. I guess xUnit.net can make the world a better place by creating less friction for people with hard selling jobs to do, but Roy Osherove's IMHO largely misinformed post put me off for far longer. But then I've never worked in an env rife with MSTESTism :D

GUI runner, I will prob never use (Yes I know its implemented), but the NUnit auto-reload and re-run feature is killer - to dock on another monitor. Its easy to show to anyone that thinks they want an integrated runner.

TeamCity is done and was critical.

I'd like clean TeamBuild support though (PM me if you want to see scripts I ended up with and/or any other shortcircuiting of my last few days rantings here and on SO)

Other two, not qualified to comment

New Post: Support Visual Studio Test Runner (Please Read and Respond!)

bartelink2 — Wed, 01 Jul 2009 07:43:18 GMT

@ianceicys: Just been down the road, look at my recent posts on StackOverflow - they should give good pointers. Re conversion, I did a pile of of removals of [TestClass], replaces of IsNull, IsEqual, IsNotEqual, Assert.Fail, TestMethod. Then you need to replace any TestInitalize/Cleanups, but they are less common. One thing you'll probably run into is VSTT private accessors - I ended up generating ones with publicize.exe and checking the bins in as VS2008 Test References get ignored without it (trying to run without VS on the build agent). Then I ended up having to install VS for now as to get test results into the TFS repoisitory you need to do a mstest/publish, which only works with VSTT and TFC installed on the agent box. (The projects still need to reference Microsoft.VisualStudio.QualityTools.UnitTests if you use a publicize.exe _Accessor assembly, but this gets onto an agent box with TeamBuild anyway IIRC)

Before you start this work, ask yourself what you're gaining in business value out of this exercise though (I gained more focused test development with less bad crutches leading one in bad directions. I thought I was gaining the ability not to have to install VSTT on a build box. Right now, I'm happy I did it, but not aws happy as UI was exepcting to be.

So, what are you hoping to gain?

New Post: Support Visual Studio Test Runner (Please Read and Respond!)

ianceicys — Tue, 30 Jun 2009 22:39:23 GMT

I have a team that is using Xunit and we are struggling to get the tests to work with an integrated Team Build. We primilary use MSTEST unit tests, is there an easy way to convert xunit tests to MStest tests?

Thanks,

New Post: How to run multiple assemblies?

bartelink2 — Tue, 30 Jun 2009 13:53:41 GMT

As covered in http://stackoverflow.com/questions/1063779/msbuild-task-for-combining-nunit-or-mstest-xml-results, I'm running into lots of side issues re not being able to run multiple assemblies, and I'm assuming others are too. While I've done a batched run and can parallelise it as a result, any perf gains I make are negated by the mstest /publish commands queuing up to send stacks of duplicated binaries up alongside the test results. If achieving similarity to the NUnit task's interface was to be achievable it would be a win for people porting NUnit tests and automation around it... (Don't mean to sound like I'm moaning - xUnit.net is a glorious project overall)

Created Issue: Xunit.Runner.MSBuild.xunit should be a ToolTask

bartelink2 — Tue, 30 Jun 2009 09:25:02 GMT

As covered in http://stackoverflow.com/questions/1059230/trapping-error-status-in-msbuild/1059672#1059672, in build rigs that handle N test assemblies, one generally wants to be able to batch invoke them, but still run remainign ones in the presence of failures. It would seem that the standard way of achieving this is by deriving from ToolTask instead of Task. There's already a hacked version being posted at http://jonnekats.wordpress.com/2009/05/07/integrate-xunit-tests-into-your-daily-team-build/ demonstrating the need

Updated Wiki: Home

BradWilson — Mon, 29 Jun 2009 16:09:46 GMT

On This Page...

About the Project
Frequently Asked Questions
Recent News
How Do I...
Blog Posts on xUnit.net
Links to Resources

About the Project

Frequently Asked Questions

Recent News

2009-05-25	Released: xUnit.net 1.5 CTP 1
2009-02-21	xUnit.net Contrib launches, with Resharper runner as its first project. Read more here
2008-11-26	Documented the XML format used in xUnit.net
2008-11-26	Released: xUnit.net 1.1
2008-08-20	Should we support Visual Studio's test runner? Discuss it here
2008-06-29	RunWithNUnit and xunitext.nunit.dll moved from Main to Samples. Read more here

How Do I...?

Blog Posts on xUnit.net

2009-06-27	Justin Etheredge has an MSDN Magazine article on testable ASP.NET MVC apps with xUnit.net, Ninject, and Moq. (Link)
2009-06-25	Ben Hall has written a [Browser] attribute for xUnit.net and WatiN. (Link)
2009-06-10	Ade Miller talks about using xUnit.net theories to validate interface implementations. (Link)
2009-06-04	Peter Mounce has integrated xUnit.net with rake-dotnet. (Link)
2009-05-30	Mark Needham describes how to run your .NET 4.0 tests using xUnit.net. (Link)
2009-05-27	Ben Hall is working on a Behavior-style syntax for xUnit.net called "xUnit.GWT". (Blog, Source, Example, Report)
2009-04-20	Chris Marinos discusses using xUnit.net + F# to write BDD-style tests. (Link)
2009-04-14	Steven Livingstone-Pérez discussing xUnit.net's integration with TeamCity. (Link)
2009-04-10	Peli de Halleux announces the release of Pex 0.10, with support for xUnit.net. (Link)
2009-03-22	Alex Zeitler has a screencast demonstrating xUnit.net & xUnit.net BDD Extensions & Resharper (in German). (Link)
2008-12-15	Matt Podwysocki integrates xUnit.net with FsCheck for F#. (Link)
2008-12-01	Ade Miller writes about rounding errors and a margin-of-error comparer he wrote. (Link)
2008-09-19	Fredrik Kalseth writes about his specification-style BDD work with xUnit.net. (Link)
2008-08-28	Björn Rochel blogs about his attempts at a BDD framework on top of xUnit.net. (Link)
2008-08-24	Phil Haack discusses some experiments we did with a specification API ("SubSpec"), available in the samples. (Link)
2008-05-16	David Tchepak uses xUnit.net to test his game (Part 1, Part 2)
2008-05-14	Bembeng Arifin on using xunit.console.exe from within Visual Studio 2008 Express (Link)
2008-05-09	Damon Payne explores concurrent unit testing with xUnit.net (Part 0, Part 1)
2008-05-05	Torkel Ödegaard on Watin and xUnit.net integration testing (Link)
2008-04-22	Moq mocking framework now uses xUnit.net for its unit testing (Link)
2008-04-18	Scott Hanselman's Weekly Source Code reviews xUnit.net's extensibility APIs (Link)
2008-04-04	Matt Podwysocki looks at our new ASP.NET MVC Preview 2 support (Link)
2008-01-31	Ben Hall on: xUnit.net, xUnit.net Extensions, Creating Your Own Extensions

Links to Resources

.NET Framework 2.0 Redistributable (x86, x64)
TestDriven.NET
Resharper
TeamCity
CruiseControl .NET
MSBuild Reference
NUnit

Source code checked in, #33797

BradWilson — Mon, 29 Jun 2009 05:28:37 GMT

Added XML doc comments to the new Object Model classes.