CodePlex Home
Register | Sign In | CodePlex Home

Table of Contents

Attributes

Note: any testing framework attributes that are not in this list have no corresponding attribute in xUnit.net.

NUnit 2.x MbUnit 2.4 MSTest xUnit.net Comments
[Test] [Test] [TestMethod] [Fact] Marks a test method.
[TestFixture] [TestFixture] [TestClass] n/a xUnit.net does not require an attribute for a test class; it looks for all test methods in all public (exported) classes in the assembly.
[ExpectedException] [ExpectedException] [ExpectedException] Assert.Throws or Record.Exception xUnit.net has done away with the ExpectedException attribute in favor of Assert.Throws. See Note 1.
[SetUp] [SetUp] [TestInitialize] Constructor We believe that use of [SetUp] is generally bad. However, you can implement a parameterless constructor as a direct replacement. See Note 2.
[TearDown] [TearDown] [TestCleanup] IDisposable.Dispose We believe that use of [TearDown] is generally bad. However, you can implement IDisposable.Dispose as a direct replacement. See Note 2.
[TestFixtureSetUp] [TestFixtureSetUp] [ClassInitialize] IUseFixture<T> To get per-fixture setup, implement IUseFixture<T> on your test class. See Note 3
[TestFixtureTearDown] [TestFixtureTearDown] [ClassCleanup] IUseFixture<T> To get per-fixture teardown, implement IUseFixture<T> on your test class. See Note 3
[Ignore] [Ignore] [Ignore] [Fact(Skip="reason")] Set the Skip parameter on the [Fact] attribute to temporarily skip a test.
n/a [Timeout] [Timeout] [Fact(Timeout=n)] Set the Timeout parameter on the [Fact] attribute to cause a test to fail if it takes too long to run. Note that the timeout value for xUnit.net is in milliseconds.
[Property] n/a [TestProperty] [Trait] Set arbitrary metadata on a test
n/a [Row], [RowTest] [DataSource] [Theory], [XxxData] Theory (data-driven test). See Note 4


Note 1: Long-term use of [ExpectedException] has uncovered various problems with it. First, it doesn't specifically say which line of code should throw the exception, which allows subtle and difficult-to-track failures that show up as passing tests. Second, it doesn't offer the opportunity to fully inspect details of the exception itself, since the handling is outside the normal code flow of the test. Assert.Throws allows you to test a specific set of code for throwing an exception, and returns the exception during success so you can write further asserts against the exception instance itself.

Note 2: The xUnit.net team feels that per-test setup and teardown creates difficult-to-follow and debug testing code, often causing unnecessary code to run before every single test is run. For more information, see http://jamesnewkirk.typepad.com/posts/2007/09/why-you-should-.html and http://www.agileprogrammer.com/dotnetguy/articles/SetupTeardown.aspx.

Note 3: xUnit.net provides a new way to think about per-fixture data with the use of the IUseFixture<T> interface. By implementing this interface, you tell the test system that you wish to have certain fixture data provided to you before each test is run, the type of which is the T in the generic part of the interface. The runner will create a single instance of the fixture data and pass it through to your SetFixture() method before running each test. All the tests share the same instance of fixture data. After all the tests have run, the runner will dispose of the fixture data, if it implements IDisposable.

Note 4: The extensions library (xunit.extensions.dll) ships with support for data-driven tests call Theories. Mark you test with the [Theory] attribute (instead of [Fact]), then decorate it with one or more [XxxData] attributes, including [InlineData], [PropertyData], [ExcelData], [SqlServerData], and [OleDbData]. See the unit tests in the test.xunitext project for examples.

Assertions

xUnit.net has removed most instances of the words "Are" and "Is". The NUnit 2.2.10 Assert class is used for comparison purposes; note that NUnit 2.4 offers both NUnit 2.2.x style and new "natural language" assertions. Note: any testing framework assertions which are not in this list have no corresponding assertion in xUnit.net.

NUnit 2.2.x MbUnit 2.4 MSTest xUnit.net Comments
AreEqual AreEqual AreEqual Equal MSTest and xUnit.net support generic versions of this method
AreNotEqual AreNotEqual AreNotEqual NotEqual MSTest and xUnit.net support generic versions of this method
AreNotSame AreNotSame AreNotSame NotSame
AreSame AreSame AreSame Same
Contains Contains and In Contains (on CollectionAssert) Contains MbUnit's Contains works on strings, and In works on containers
DoAssert n/a n/a n/a
n/a NotIn DoesNotContain (on CollectionAssert) DoesNotContain
n/a n/a n/a DoesNotThrow Ensures that the code does not throw any exceptions
Fail Fail Fail n/a xUnit.net alternative: Assert.True(false, "message")
Greater Greater n/a n/a xUnit.net alternative: Assert.True(x > y)
Ignore Ignore Inconclusive n/a
n/a Between n/a InRange Ensures that a value is in a given inclusive range (note: NUnit and MSTest have limited support for InRange on their AreEqual methods)
IsAssignableFrom IsAssignableFrom n/a IsAssignableFrom
IsEmpty IsEmpty n/a Empty
IsFalse IsFalse IsFalse False
IsInstanceOfType IsInstanceOfType IsInstanceOfType IsType
IsNaN IsNan n/a n/a xUnit.net alternative: Assert.True(double.IsNaN(x))
IsNotAssignableFrom IsNotAssignableFrom n/a n/a xUnit.net alternative: Assert.False(obj is Type);
IsNotEmpty IsNotEmpty n/a NotEmpty
IsNotInstanceOfType IsNotInstanceOfType IsNotInstanceOfType IsNotType
IsNotNull IsNotNull IsNotNull NotNull
IsNull IsNull IsNull Null
IsTrue IsTrue IsTrue True
Less Less n/a n/a xUnit.net alternative: Assert.True(x < y)
n/a NotBetween n/a NotInRange Ensures that a value is not in a given inclusive range
n/a n/a n/a Throws Ensures that the code throws an exact exception
Last edited Apr 29 at 5:02 PM by BradWilson, version 28
Comments
rhwilburn Oct 16 2007 at 8:10 AM 
This is a really great testing framework. I think its the best I have seen as it has scaled back all of the annotation bloat from setup and tear down methods which are not necessary as they are implied by the life of the object that hosts the test.

Gabriel Mar 17 2008 at 2:12 PM 
Are there plans to add the equivalent of NUnits [Category] attribute?

BradWilson Apr 1 2008 at 1:14 AM 
Gabriel, yes, in the sense that you can use [Trait], but no, in the sense that none of the runners make any use of traits for decide what should or should not be run. IMO, it's better to make separate test assemblies than it is to use Categories.

cristianp May 26 2008 at 6:03 PM 
Would you support [Rollback] Attributes in the test? (as mbunit and nunit extensions). And what about Assert.That as the nunit.framework.syntax in NUnit 2.4? I guess it would make the Tests Asserts a lot more clear.

BradWilson Aug 26 2008 at 7:32 AM 
@cristianp, we have [AutoRollback] as an extension in xunitext.dll

chaliy Sep 8 2008 at 12:06 PM 
Guys, could you please explain reasons why you remove(not implemented) message for assertion methods?

BradWilson Sep 8 2008 at 4:12 PM 
@chilay, the reason we removed messages from everything except True/False is because the asserts should be descriptive enough. Assert.Equal(42, someValue, "someValue should've been 42") doesn't add a lot of value, and if you can't tell what an assert is testing when you write it, then it's probably broken anyway. Worst case, you can add a comment to the assert method. We left the messages on True/False, because these tend to be the catchall handlers used when a specific assertion is not present.

mknaup Nov 21 2008 at 10:49 PM 
Are there any plans for a similar extension like NUnitForms? Or how do you test GUI applications with XUnit?

DanielPuzey Jan 28 at 5:11 PM 
I'd have to disagree, Brad - surely any level of context you can add is useful? Why should someValue be 42? The example you gave may be useless, but an example like Assert.Equal(42, someValue, "The query did not return the expected number of results") is less so.

By forcefully excluding the reason, aren't you making it necessary to inspect the test code to work out why the test failed - instead of it being clear from the output?

I do like the use of pre-existing mechanisms - such as IDisposable for cleanup and constructors for initialisation. It makes the test classes much more portable and reusable outside of the test framework.

kolorahl Feb 28 at 1:58 AM 
No, I'd have to agree with Brad. "The query did not return the expected number of results" is actually worse than "someValue does not equal 42." But not when you're specifically using the variable name "someValue." Let's say you got a query result and stored the count in 'queryResCount'. Now the generated exception message is more like "queryResCount does not equal 42." You see, the variable name clearly states what value you're comparing and the number 42 tells you exactly what your expected size was. Your example simply used "expected number of results." Well, what is the expected number?

The system was implemented more to prevent people from being lazy or accidentally skipping over data that should be displayed. I think that as long as you name your variables well the messages should be pretty good at telling you what went wrong for what reason.

iSpeakGeek Apr 17 at 1:13 PM 
I'd vote for being able to add context to my assertions - consider a test where we are verifying that changes are isolated e.g.

Assert.Equal(1, calculator.NumeracyLevel(qualifications));
Assert.Equal(0, calculator.LiteracyLevel(qualifications));
Assert.Equal(0, calculator.SomeOtherLevel(qualifications));

If either of the later two assertions fail, I can obviously find out what's wrong by looking at the line number. That's fine until I run this as a Theory test or if I have more than one or two numeric assertions. You might argue in the latter case that I should re-write my test to keep it to a small number of assertions, but in my situation that would make the individual tests less thorough or much harder to maintain.

With Theory tests it may not be obvious that I have multiple points of failure unless I carefully inspect my results to see whether the line numbers for my numeric assertions are different.

I'd also argue that comments can be useful in tracking down the failure. I've frequently found myself adding notes like:

// If this fails here, it's probably because of something like...

or

// If this fails here multiple times in the theory test then the test data may be incomplete - has the data changed?

Having these entries in the code is useful, but I could isolate which failing tests to concentrate on quicker if I could see these notes in the test output.

So, given that being able to optionally include messages in the assertions does no harm and can be useful in certain situations, I'd say it'd be better not to limit the users of the test framework wouldn't it?

Cheers,

John

BradWilson Apr 17 at 8:16 PM 
@iSpeakGeek, to the best of my knowledge, all runners except Resharper will actually show you the values that were passed into the theory as parameters to the name of the test method. For example, this test:

public class MyTests {
[Theory, InlineData(27), InlineData(42)]
public void NumberMustBeAwesome(int value) {
Assert.Equal(42, value);
}
}

Shows the failure like this with the console runner:

1) MyTests.NumberMustBeAwesome(27) : Assert.Equal() Failure

colinjack Sep 23 at 2:38 PM 
Do you think there is any chance you could add something like a category attribute in the future, I don't like having to split out slower tests out to a seperate project.

bartelink Sep 23 at 3:53 PM 
There is a Category in Extensions (it's derived from Trait). The GUI runner has trait-based filtering, see http://feedproxy.google.com/~r/BradWilson/~3/SNM_fROJjc8/xunitnet-15-shipped.html

Be careful with mixing 'slow' and 'fast' tests - integration tests should be called as such and probably separated into assemblies with that name
Updating...
© 2006-2009 Microsoft | About CodePlex | Privacy Statement | Terms of Use | Code of Conduct | Advertise With Us | Version 2009.10.27.15987