NUNIT vs VSTS ?? Which is better ?

Hi,
I am trying to figure out best testing tool for my project. I have
narrowed down my requirements to two tools NUNIT and VSTS unit. But I have
used neither and I have to use only one of them. Hence can someone who has
used them before share his/her experience on them so that I can get a better
idea and make a proper choice ?
Any link or suggestions are also welcome.

Thanks and Regards,
Parag
MTS,Persistent Systems Private Limited

Peter
Tue Mar 07 01:54:23 CST 2006

They both really do the same thing. Here are the reasons that I would
choose NUnit over MSs version.

1) Unit testing in VS can only be done through a high-end version of the
product. If all your developers do not use the team version, then they will
not be able to use the MS testing framework.
2) NUnit has been around for years with a proven track records.
3) NUnit if available for many versions of the framework and also Mono (I
think).
4) NUnit does not couple the testing to just an MS environment.
5) External tools may work better with NUnit such as build systems.


"Parag" <parag_kulkarni@persistent.co.in> wrote in message
news:ucZX$raQGHA.5152@TK2MSFTNGP10.phx.gbl...
> Hi,
> I am trying to figure out best testing tool for my project. I have
> narrowed down my requirements to two tools NUNIT and VSTS unit. But I have
> used neither and I have to use only one of them. Hence can someone who has
> used them before share his/her experience on them so that I can get a
> better idea and make a proper choice ?
> Any link or suggestions are also welcome.
>
> Thanks and Regards,
> Parag
> MTS,Persistent Systems Private Limited
>
>

William
Tue Mar 07 02:08:34 CST 2006

I have used both. nunit is nice, but VS test is suit. I use it in the VSDev
version.

--
William Stacey [MVP]

"Parag" <parag_kulkarni@persistent.co.in> wrote in message
news:ucZX$raQGHA.5152@TK2MSFTNGP10.phx.gbl...
| Hi,
| I am trying to figure out best testing tool for my project. I have
| narrowed down my requirements to two tools NUNIT and VSTS unit. But I have
| used neither and I have to use only one of them. Hence can someone who has
| used them before share his/her experience on them so that I can get a
better
| idea and make a proper choice ?
| Any link or suggestions are also welcome.
|
| Thanks and Regards,
| Parag
| MTS,Persistent Systems Private Limited
|
|

NoSpamMgbworld
Tue Mar 07 10:21:28 CST 2006

They are pretty equivalent. The VSTS tags are a bit closer to the Attributes
throughout the framework, which is nice. If you are using Team System, I
would go ahead and use VSTS. If not, nUnit is a fine alternative. I use both,
depending on the project location.

--
Gregory A. Beamer
MVP; MCP: +I, SE, SD, DBA

***************************
Think Outside the Box!
***************************


"Parag" wrote:

> Hi,
> I am trying to figure out best testing tool for my project. I have
> narrowed down my requirements to two tools NUNIT and VSTS unit. But I have
> used neither and I have to use only one of them. Hence can someone who has
> used them before share his/her experience on them so that I can get a better
> idea and make a proper choice ?
> Any link or suggestions are also welcome.
>
> Thanks and Regards,
> Parag
> MTS,Persistent Systems Private Limited
>
>
>

William
Tue Mar 07 15:23:09 CST 2006

There are two things that I really like more about VS. Better overall
integration with VS. And you can right-click a method and gen the test stub
automatically. You also get code coverage results with integration to show
what lines where not covered. Not sure what nunit offers today in that
regard.

--
William Stacey [MVP]

"Cowboy (Gregory A. Beamer) - MVP" <NoSpamMgbworld@comcast.netNoSpamM> wrote
in message news:AF277C17-A2E8-41AB-9C58-200F638C7DE1@microsoft.com...
| They are pretty equivalent. The VSTS tags are a bit closer to the
Attributes
| throughout the framework, which is nice. If you are using Team System, I
| would go ahead and use VSTS. If not, nUnit is a fine alternative. I use
both,
| depending on the project location.
|
| --
| Gregory A. Beamer
| MVP; MCP: +I, SE, SD, DBA
|
| ***************************
| Think Outside the Box!
| ***************************
|
|
| "Parag" wrote:
|
| > Hi,
| > I am trying to figure out best testing tool for my project. I have
| > narrowed down my requirements to two tools NUNIT and VSTS unit. But I
have
| > used neither and I have to use only one of them. Hence can someone who
has
| > used them before share his/her experience on them so that I can get a
better
| > idea and make a proper choice ?
| > Any link or suggestions are also welcome.
| >
| > Thanks and Regards,
| > Parag
| > MTS,Persistent Systems Private Limited
| >
| >
| >

John
Tue Mar 07 15:38:37 CST 2006

While NUnit doesnt offer strong vstudio integration, or coverage there
are a number of 3rd party tools that do this integration .... I
personally prefer TestRunner.

That being said, I really believe that the market positioning of VSTS by
MS puts it out of reach for most independent developers and small
development shops. That being said, IMHO, I personally see NUnit as the
best option.




William Stacey [MVP] wrote:
> There are two things that I really like more about VS. Better overall
> integration with VS. And you can right-click a method and gen the test stub
> automatically. You also get code coverage results with integration to show
> what lines where not covered. Not sure what nunit offers today in that
> regard.
>

Jon
Tue Mar 07 15:42:18 CST 2006

William Stacey [MVP] <william.stacey@gmail.com> wrote:
> There are two things that I really like more about VS. Better overall
> integration with VS. And you can right-click a method and gen the test stub
> automatically.

Does it assume a "one test method per concrete method" model though?
That's what Eclipse assumes when you ask it to generate a test skeleton
- and it's almost never what I actually want. (I just start with a bare
test class.)

It's also not useful for true TDD, where the tests come before the code
they're calling.

> You also get code coverage results with integration to show
> what lines where not covered. Not sure what nunit offers today in that
> regard.

You can run NUnit over NCover. I haven't used it much myself, but I've
done enough to know it works.

--
Jon Skeet - <skeet@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

William
Tue Mar 07 16:06:24 CST 2006

| That being said, I really believe that the market positioning of VSTS by
| MS puts it out of reach for most independent developers and small
| development shops. That being said, IMHO, I personally see NUnit as the
| best option.

Remember you don't need VSTS to use it. You need at lease VSDev IIRC. I
have VSDev with the MSDN. Am I wrong?

I thought same about NUnit before I really used VS. One day I *forced
myself to use VS for good sized library. I would not go back now.
Different strokes. Code coverage is big.

--
William Stacey [MVP]

William
Tue Mar 07 16:31:50 CST 2006

| Does it assume a "one test method per concrete method" model though?
| That's what Eclipse assumes when you ask it to generate a test skeleton
| - and it's almost never what I actually want. (I just start with a bare
| test class.)

Not 100% sure what you mean. By default, if you just have a new class, it
will stub out (pretty nice) each method (w/ overload), constructor and
properties IIRC. And even the private methods if desired. You can then add
any amount of new test methods, delete or change existing ones. You can do
any amount of tests (high, low, range, expected exceptions, etc) inside one
method or break them out as you want. Get nice coverage results, see what
failed. Click on failed tests to jump to that code. Click on uncovered
results to jump to that method and see the uncovered lines in color, etc.
Lot more going on here then just red and green traffic lights.

| It's also not useful for true TDD, where the tests come before the code
| they're calling.

They encourage and support either. Possibly the easiest way is to stub out
your class with exception in the body, then gen your tests over the class.
Then just keep adding code to your class as you go and iterating your tests.
You could also just stub out your tests without any class first, but not
sure how valuable that would be. Its all just code, so you can do what you
want.

| > You also get code coverage results with integration to show
| > what lines where not covered. Not sure what nunit offers today in that
| > regard.
|
| You can run NUnit over NCover. I haven't used it much myself, but I've
| done enough to know it works.

Have nothing against NUnit. I can only suggest trying the test and code
coverage in VS (if you got the MSDN w/ VSDev). I think you will be very
happy with the whole solution.

--
William Stacey [MVP]

John
Tue Mar 07 17:17:45 CST 2006

code coverage is available when you use TestRunner and NUnit as well
.... at a fraction of the cost


William Stacey [MVP] wrote:
> | That being said, I really believe that the market positioning of VSTS by
> | MS puts it out of reach for most independent developers and small
> | development shops. That being said, IMHO, I personally see NUnit as the
> | best option.
>
> Remember you don't need VSTS to use it. You need at lease VSDev IIRC. I
> have VSDev with the MSDN. Am I wrong?
>
> I thought same about NUnit before I really used VS. One day I *forced
> myself to use VS for good sized library. I would not go back now.
> Different strokes. Code coverage is big.
>

Chris
Tue Mar 07 18:25:42 CST 2006

"Parag" <parag_kulkarni@persistent.co.in> wrote in message:

[Nunit Vs VSTS]

There are pros and cons to both. The NUnit Stuff is free, and free is
compelling.

The VSTS stuff has integration into Visual Studio. NUnit doesn't. On the
other hand NUnit has some nice VS add-in's that give it integration -
arguably a cleaner integration than VSTS has.

VSTS has Code Coverage metrics, Nunit doesn't. This is a huge win. On the
other hand, NUnit has NCover, which gives you code coverage.

Both Nunit and VSTS have NAnt and MSBuild tasks to run tests and perform
continous integration.

The VSTS test infrastructure is integrated into the Profiler which is very
nice. NUnit has no equilivent that I've been able to find. With Compuware
not offering a Community Edition of their VS 2005 profiler (and the product
being so expensive smaller shops and indiviual consultants can't buy it) ,
this is a compelling win.

The VSTS has the memory profiling stuff built in as well. NUnit has no
equilivent. The SciTech Memory Profiler for $100 is far better than the MS
one, and at a realistic price point.

For 95% of the developers I work with, Team System isn't an option due to
cost. Microsoft's failure to include the Unit Test infrastructure with VS
Pro means almost everyone is stuck with NUnit for testing.

Overall, I like NUnit better - if only I could talk Compuware into giving me
a copy of their DevPartner Pro suite I would be all set.

--
Chris Mullins
Coversant, Inc.

Chris
Tue Mar 07 18:29:51 CST 2006

"William Stacey [MVP]" wrote

> Remember you don't need VSTS to use it. You need at lease
> VSDev IIRC. I have VSDev with the MSDN. Am I wrong?

Unfortunatly, I think so.

http://msdn.microsoft.com/vstudio/products/compare/

The Professional Version I often use doesn't have any of the testing stuff,
and unless you got the very, very expensive MSDN subscription, that's the
version of VS.NET that comes with MSDN.

> Code coverage is big.

http://ncover.org/site/

--
Chris Mullins

Chris
Tue Mar 07 18:32:20 CST 2006

[I've responded to so many of your posts this last week, I feel like I'm
stalking you]

"William Stacey [MVP]" <william.stacey@gmail.com> wrote

> | You can run NUnit over NCover. I haven't used it much myself, but I've
> | done enough to know it works.
>
> Have nothing against NUnit. I can only suggest trying the test and code
> coverage in VS (if you got the MSDN w/ VSDev). I think you will be very
> happy with the whole solution.

It's nice - even nicer, to me, is the performance profiler and memory
profiler integration with the VSTS infrastructure. I also love the way you
can rollup smaller tests into big test runs using the Web Stress tools. Very
slick design by Microsoft.

--
Chris Mullins

William
Tue Mar 07 20:55:32 CST 2006

cool. Have not tried the memory profiler stuff yet.

--
William Stacey [MVP]

Jon
Wed Mar 08 01:20:55 CST 2006

William Stacey [MVP] <william.stacey@gmail.com> wrote:
> | Does it assume a "one test method per concrete method" model though?
> | That's what Eclipse assumes when you ask it to generate a test skeleton
> | - and it's almost never what I actually want. (I just start with a bare
> | test class.)
>
> Not 100% sure what you mean. By default, if you just have a new class, it
> will stub out (pretty nice) each method (w/ overload), constructor and
> properties IIRC.

What *exactly* do you mean by stubbing out? Creating the methods in the
production class, or creating methods in the test class to match the
ones in the production class.

My point is that if I've got a single method in the production class, I
may well want three or four test methods with names indicating what
aspect of that method they'll be testing. Sometimes two methods are
tested by the same set of tests. In other words, creating a "matching"
set of methods in the test code has never seemed that useful to me.

> And even the private methods if desired. You can then add
> any amount of new test methods, delete or change existing ones. You can do
> any amount of tests (high, low, range, expected exceptions, etc) inside one
> method or break them out as you want. Get nice coverage results, see what
> failed. Click on failed tests to jump to that code. Click on uncovered
> results to jump to that method and see the uncovered lines in color, etc.
> Lot more going on here then just red and green traffic lights.

Good to hear VS is finally catching up with Eclipse a bit :)

> | It's also not useful for true TDD, where the tests come before the code
> | they're calling.
>
> They encourage and support either. Possibly the easiest way is to stub out
> your class with exception in the body, then gen your tests over the class.

No, that's not test-driven development. With test-driven development
you don't know what methods you'll have in your class until you've
written your tests.

> Then just keep adding code to your class as you go and iterating your tests.
> You could also just stub out your tests without any class first, but not
> sure how valuable that would be. Its all just code, so you can do what you
> want.

Yes, but different IDE features can certainly help TDD. I seem to
remember that 2005 is capable of creating methods that you call even
when they don't exist just like Eclipse does. That's what really helps