Adventures in Linq, System.IO.FileInfo, and Unit Testing

I am working with another developer on a team website.  The shared server that we use has recently been running out of disk space and we don’t know why.  I whipped up a quick console program to help figure out what is going on.  My first thought was to create a static List of FileInfo in my program and then populate it via a recursive method.  I coded it like so:

static List<FileInfo> fileInfos = new List<FileInfo>();

        private static void SearchDirectory(string directoryName)

        {

            DirectoryInfo directoryInfo = new DirectoryInfo(directoryName);

            foreach (FileInfo fileInfo in directoryInfo.GetFiles())

            {

                fileInfos.Add(fileInfo);

            }

 

            foreach (DirectoryInfo subDirectoryInfo in directoryInfo.GetDirectories())

            {

                SearchDirectory(subDirectoryInfo.FullName);

            }

        }

 

And then I sorted the fileInfos object via some LINQ:

        private static void SortDirectoryInfo()

        {

            var fileSort = (from fi in fileInfos

                            orderby fi.Length descending

                            select fi);

 

            int i = 0;

            foreach (FileInfo fileInfo in fileSort)

            {

                if (i < 40)

                {

                    Console.WriteLine(fileInfo.Name + ":" +fileInfo.Length.ToString());

                }

                i++;

            }

        }

 

This actually worked (and the culprit was the TFS server consuming too much disk space) but it is obviously not good code and does not take advantage of the power of C# 3.0.

My first attempt to clean up the code was to write a unit test for both methods.  I changed the scope of the methods to public and then tried to generate the unit test.  I then ran into the problem with brittle code – the SortDirectoryInfo method depends on an external class that may or may not exist.  Note that I did not even check to see if it exists in the method.  So I re-wrote the method to take a parameter of a List of FileInfos.  Doing so meant that I could drop the global fileInfos object and stick it into the Main method.  I also had to change the SearchDirectory to take in that fileInfos object.

        public static void SortDirectoryInfo(List<FileInfo> fileInfos)

        {

            …Implementation

        }

I then generated some Unit Tests and quickly ran into the problem of combining Unit Tests with Integration Tests.  Without an adequate Mocking framework, what is a good test?  I went with the least common denominator principle – have a directory that I know has, at least, 1 file.  The test looks like this:

        [TestMethod()]

        public void SearchDirectoryInProgramFilesTestReturnsAtLeastOneFile()

        {

            string directoryName = @"C:\Program Files";

            List<FileInfo> fileInfos = new List<FileInfo>();

            Program.SearchDirectory(directoryName, fileInfos);

            int notExpected = 0;

            int actual = fileInfos.Count;

            Assert.AreNotEqual(notExpected, actual);

        }

 

This seems like a good  1st pass.  I have no idea if the data coming back is right (or if the subdirectory search is working) so I probably need tests to those conditions such as:

·         SearchDirectoryInNoFileBaseButFilesInSubdirectoryReturnsAtLeastOneFile

·         SearchDirectoryInNoFileBaseandNoFileSubdirectoryReturnsNoFiles

 

With this test under my belt (it passed) I went about cleaning up my code.   I first wanted to remove the recursion because the GetFiles() has an overload that does exactly that and I would rather have Microsoft do something than me.  I ripped out the code in the function and replaced it with this:

        public static void SearchDirectory(string directoryName, List<FileInfo> fileInfos)

        {

            DirectoryInfo directoryInfo = new DirectoryInfo(directoryName);

            foreach (FileInfo fileInfo in directoryInfo.GetFiles("*.*",

                     SearchOption.AllDirectories))

            {

                fileInfos.Add(fileInfo);

            }

        }

 

Good news, the test still passed

I then wanted to get rid of that magic number variable in the SortDirectoryInfo function.  I changed the LINQ to this :

            var fileSort = (from fi in fileInfos

                            orderby fi.Length descending

                            select fi).Take(20);

 

And my tests still passed (tell me again why I wrote code before using Unit Tests)?

Feeling good, I then took aim and the SearchDirectory and SortDirectory functions were separate.  I know that Martin et al think that functions should only do 1 thing but in this case, I can use LINQ to directly access the fileInfos without that intermediate variable and function, the code becomes more maintainable.  I re-wrote the function to be this

        public static void SearchDirectory(string directoryName)

        {

            DirectoryInfo directoryInfo = new DirectoryInfo(directoryName);

            var fileSort = (from fi in directoryInfo.GetFiles("*.*", SearchOption.AllDirectories)

                            orderby fi.Length descending

                            select fi).Take(20);

 

            foreach (FileInfo fileInfo in fileSort)

            {

                Console.WriteLine(fileInfo.Name + ":" + fileInfo.Length.ToString());

            }

        }

 

and then realized that my code was no longer testable.  I looked that the function and surmised that it is doing two things – just not the two things I originally thought.  It is calculating some data AND printing it to the console.  To make my code more testable, I separated those two functions

        public static IEnumerable<FileInfo> SearchDirectory(string directoryName)

        {

            DirectoryInfo directoryInfo = new DirectoryInfo(directoryName);

            IEnumerable<FileInfo> fileSort = (from fi in directoryInfo.GetFiles("*.*",

                            SearchOption.AllDirectories)

                            orderby fi.Length descending

                            select fi).Take(20);

            return fileSort;

        }

 

And

        private static void PrintResults(IEnumerable<FileInfo> fileInfos)

        {

            foreach (FileInfo fileInfo in fileInfos)

            {

                Console.WriteLine(fileInfo.Name + ":" + fileInfo.Length.ToString());

            }

        }

 

With the calling code as so:

        static void Main(string[] args)

        {

           

            Console.WriteLine("—Start—");

            IEnumerable<FileInfo> fileInfos = SearchDirectory(@"C:\Program Files");

            PrintResults(fileInfos);

            Console.WriteLine("—-End—-");

            Console.ReadLine();

        }

 

Note that I changed the return value from var (me being lazy) to the actual class – in this case it is IEnumerable of type FileInfo.  The last challenge was thinking of good Unit Tests for the return value – IEnumerable<FileInfo>.  I used a for..each and make a counting variable but I wonder if there is a different pattern that I could use to make a cleaner test. 

Advertisements

Fakes, Mocks, and Stubs

I have started looking at MOQ more closely.  I found there was a dearth of material.  The best article on how to implement a basic Mock is found here.  Note that tutorials found on the Mock homepage by Nixusg and Steve Walther use an older version of Mock and do not work.

I was still a bit confused of WHY I would want to use a Mock versus a Fake when I came across a great explanation in Pragmatic Unit Testing on page 104.  Basically, it boils down to the size of your project and what you need to test.  If you have a rudimentary class, you can just create a fake object in your test’s setup and assign it all of the needed properties.  If you are testing behavior of a class, you can set up the dependent object that the behavior depends on (and use DI) with the way you want to tests to perform.  For example, I have a basic DTO like this:

    public class Product

    {

        public Product() { }

        public Product(int productId, string productName, string description,

            double price, DateTime dateCreated, DateTime dateModified)

        {

            this.ProductId = productId;

            this.ProductName = productName;

            this.Description = description;

            this.Price = price;

            this.DateCreated = dateCreated;

            this.DateModified = DateModified;

        }

 

        public int ProductId { get; set; }

        public string ProductName { get; set; }

        public string Description { get; set; }

        public double Price { get; set; }

        public DateTime DateCreated { get; set; }

        public DateTime DateModified { get; set; }

 

        public override string ToString()

        {

            return String.Format("{0} : {1}", ProductId, ProductName);

        }

    }

And I have an Interface for a Builder that works on the DTO like this:

    public interface IProductRepository

    {

        IList<Product> FindAll();

        Product FindByName(string productName);

        Product FindById(int productId);

        bool Save(Product product);

    }

I then added a Test Project to my solution and added 2 Testing Classes – One using a Fake and one using a Mock.  The Fake was set up like this:

    class TestProductRepository : IProductRepository

    {

        List<Product> products = new List<Product>();

 

        public TestProductRepository()

        {

            products.Add(new Product(1, "Test Product One",

                "Description For One",12.4, DateTime.Now, DateTime.Now));

            products.Add(new Product(2, "Test Product Two",

                "Description For Two",1.8, DateTime.Now, DateTime.Now));

            products.Add(new Product(3, "Test Product Trhee",

                "Description For Three",6.5, DateTime.Now, DateTime.Now));

        }

 

        public IList<Product> FindAll()

        {

            return products;

        }

 

        public Product FindByName(string productName)

        {

            foreach (Product product in products)

            {

                if (product.ProductName == productName)

                {

                    return product;

                }

            }

            return null;

        }

 

With the Unit tests like this

        [TestMethod()]

        public void FindAllTest()

        {

            IProductRepository target = CreateIProductRepository();

            int expected = 3;

            int actual = target.FindAll().Count;

            Assert.AreEqual(expected, actual);

        }

 

The problem comes in when the interface that is tested has an enormous number of methods and properties.  Using Mocking, you can set up you Mock and only worry about the properties and methods that you care about. 

Consider the same tests with a Mock:

internal virtual Mock<IProductRepository> CreateIProductRepositoryMock()

        {

 

            IList<Product> products = new List<Product>

            {

                new Product {ProductId=1, ProductName="Product One",

                    Description = "Product One Description", Price=5.6},

                new Product {ProductId=2, ProductName="Product Two",

                    Description = "Product Two Description", Price=6.7},

                new Product {ProductId=3, ProductName="Product Three",

                    Description = "Product Three Description", Price=2.1}

            };

 

            Mock<IProductRepository> mockIProductRepository =

                new Mock<IProductRepository>();

            mockIProductRepository.Setup(mr =>

      mr.FindAll()).Returns(products);

            mockIProductRepository.Setup(mr => mr.FindById(It.IsAny<int>()))

                .Returns((int productId) =>

                new Product { ProductId = productId,

                    ProductName = "Product One Description" });

            mockIProductRepository.Setup(mr =>

          mr.FindByName(It.IsAny<string>()))

                .Returns((string productName) =>

                new Product { ProductId = 1, ProductName = productName });

            mockIProductRepository.Setup(mr =>

                mr.Save(It.IsAny<Product>())).Returns(true);

 

            return mockIProductRepository;

        }

With the Unit Tests like this

        [TestMethod()]

        public void FindAllTest()

        {

            Mock<IProductRepository> mock = CreateIProductRepositoryMock();

            int expected = 3;

            int actual = mock.Object.FindAll().Count;

            Assert.AreEqual(expected, actual);

 

        }

 

Even in this basic example, you can see how fewer line of code the Mock took and how much more control the Lambda gives you.   Mock was harder because I was not as familiar with the Lambda.  However, once I figured out the Lambda, I started admiring how powerful Mocking can be.  You definitely can write better and more comprehensive test with Mocks.

MVC Storefront – Review

So I went through about 20 hours of the MVC Storefront found here.  For me, it started out well.  However, after episode #6 or so, there was little, if any, MVC being discussed.  Rather, it was about Mocking, CardSpace, etc…  If you want to learn MVC, you can watch the 1st 6 videos and be done.  The amuzing thing in the series is that Rob starts out welcoming people to comment but by the 10th episode he is saying things like "I am sure some of your wonks out there won’t like this but…"  I guess the lesson here is that be careful of what you wish for.
 
On a seperate tack, I cracked open
 
last night and went through the Apendix (Intro to Java Script) and Chapter 1.  It is well written, however I still am gasping at the complexity of JQuery/JavaScript for all of this client-side processing.  I still believe you start with an AJAX Update Panel and work out from there if the performance degrades or the user experience is not optimal.
 

Java Script Injection in MVC

I set up a Hello World project based on the webcast found here.  I set up a model, a controller, and a view and tried to run a simple JavaScript injection to see if I could replicate the error.  However, I could not get the attack to work because the MVC layers protection on out of the box.

 

This is great – but in the name of understanding, I started rolling back the protections to get my attack to work.  I first thought I had to add a ValidateRequest = false in my page and web.config, just as the error page directed.   I added this

<%@ Page Title="" ValidateRequest="false" Language="C#" …

 to the page but I got the same Server Error.  I checked the web.config and found that the ValidateRequest is already off for the pages

<add verb="*" path="*.mvc" validate="false"

 I then realized that I needed to disable validation on the controller like this.

    [ValidateInput(false)]

    public class CommentsController : Controller

That got me closer – I would then enter in tags and the data was persisting to the database:

 

A quick glance at the auto-generated code in the View told me that the Html.Encode is applied out of the box – which protects you from the script running (< is &lt, etc…)

<%= Html.Encode(Model.CommentDesc) %>

Once I remove the encode statement, I got closer to the desired attack – the JavaScript was running on my page from the data in the database – however, the closing tag was not rendered so I got this error:

 

 

And the offending code looked like this:

 

I am at a high enough wall to stop this endeavor for now.  Suffice to say, with the out of the box features of MVC, sites should be reasonably protected from JavaScript injections.

 

 

 

MVC – Dealing with the Identity Column

I am following a MVC  introduction tutorial found here.  I have gone though it a couple of times and I am coming up to speed with the basics of MVC.  I found an interesting error though.  I created a controller called movie and then coded up the functions for the insertion of a movie record like this:

        public ActionResult Create()

        {

            return View();

        }

 

        [AcceptVerbs(HttpVerbs.Post)]

        public ActionResult Create(Movie movieToCreate)

        {

            if (ModelState.IsValid)

            {

                context.AddToMovieSet(movieToCreate);

                context.SaveChanges();

                return View(movieToCreate);

            }

            else

            {

                return View();

            }

        }

I then created a Unit Test to check my work like this (note the exclusion of the primary key):

        public void CreateTest()

        {

            MovieController target = new MovieController();

            Movie movieToCreate = new Movie();

            movieToCreate.Title = "AAA";

            movieToCreate.Director = "BBB";

            movieToCreate.DateReleased = DateTime.Now;

 

            ActionResult actual = target.Create(movieToCreate);

            Assert.IsNotNull(actual);

        }

 

Things worked great so I then created a View using the code generator (Right Click -> AddView) to create a data entry page.  I then deleted the ID field because it is an identity key in the database:

Unfortunately, when I ran it, I got a validation error:

So I added back the ID Field and placed in a dummy value:

And the data took!  Note the new ID number:

 

This illustrates my biggest problem with MVC so far – you need to rely almost exclusively on the Code Gen (and know the conventions) to get anything done and when things don’t work as expected, it is really hard to pinpoint the problem.

In this case, I found the error was that I didn’t tell the controller that the ID should not be bound.  I changed the signature of the Create Function to this

public ActionResult Create([Bind(Exclude="Id")]Movie movie)

and it worked like a charm