Mapping data between types on a Web Service can be a collosal pain. There are various frameworks out there that can speed things up, but sometimes you just need to do it by hand. Fortunately some of the features introduced with C# 3 can help speed things up.

Say the type on the server looks like this:

class ServerType {
    public int ProductId { get; set; }
}

And your client type looks like:

class ClientType {
    public int Id { get; set; }
}

[This is a gross simplification, typically the types have more differences, but it gets the point across.]

Let’s say that the web service takes a List<ServerType> as one of its parameters too. Typically before C# you would need to do something like this:

public List ConvertClientList(List<ClientType> clientTypes) {
    List serverTypes = new List<ServerType>();
    foreach (ClientType clientType in clientTypes) {
        ServerType serverType = new ServerType();
        serverType.ProductId = clientType.Id;
        serverTypes.Add(serverType);
    }
    return serverTypes;
}

Ugh. That is far too much code and typing. Fortunately we can drop that to a one-liner with LINQ and Class Initializers:

var serverTypes =
    clientTypes.Select(c => new ServerType { ProductId = c.Id }).ToList();

Bingo. Let’s walk through this a little bit just to be clear. On every item in the clientTypes List, the Select function is being called. It is instantiating a new ServerType object and setting the ProductId property. The default payload of the function in select is to return the expression, hence the lack of a ‘return’ statement; it is implied. Select returns an IQueryable by default to allow more chaining of expressions. Here we don’t need to do that, so we convert the whole thing back to a List.

The Select function is very powerful. It let’s you transform a collection of objects of one type to something else entirely. The result doesn’t even have to be a known type! For further magic on Select, read this article.


Disclaimer: I have not worked as a full time software tester, so this point of view is from a developer who would love to see the following qualities in QA personnel that I work with.

Sometime early in my career as a software developer, I learned that my code was not perfect and needed testing before being released1. Since then I have worked with a couple of good testers, and unfortunately, some truly terrible ones as well. I would very much prefer to work with the good ones, so I thought I would list some characteristics that could help a would-be tester become a great one.

This is the biggest: Your goal is not a checklist. This one is partially attributable to poor management. Testers who are graded on the number of bugs they find tend to try to file as many as they can. Your goal is actually not to find a ton of bugs; your goal is to release the highest quality software possible. I can forgive many testing sins if just this one maxim is embraced.

Understand the application. I am always shocked to read a bug report that is completely at odds with what is written in the application's requirements. "The app is supposed to work that way," is far too often the answer. One way to mitigate this is to include testers in the design of the product they will be testing. This includes knowing which developer/department to assign the bugs. In most cases the poor UI guys have 95% of the bugs assigned to them and end up serving as triage for the rest of the group. A good tester knows that even though an error dialog is popping up in the UI, it may very well be because of a problem with the database.

Know your tools. If you are testing a web app, you had better have tons of extensions for debugging, header analysis, and browser tweaking installed. You need to know how to take a good screenshot and highlight the problematic area. This goes hand in hand with "Record important information" in a few paragraphs. Good tools make it easier to specify what the problem is. Taking a screenshot of your entire desktop and pasting it into Word is always a bad idea.2 Knowing the bug tracker inside and out is mandatory.

Leave your ego at the door. This goes for both testers and developers. There is naturally an adversarial relationship between both tribes; one group exists to find problems with the others' work, but it doesn't have to be ugly. I once worked with a guy who hated every single QA person he met because his ego would not let him accept that his code had bugs. Conversely, you are not King of the World for finding an obvious defect that the programmer should have caught.

Record important information. Taking a screenshot of an exception screen is a little better than writing "it dun broke" in the description field, but not by much. Does this exception happen every time? With different information? In different browsers? Are there any unique or special rules applied to this specific instance?

I once worked on a product that needed different rules applied to different customers' accounts. These rules caused a bunch of validation issues and various one-off bugs that couldn't be reproduced in the development environment (since the rules were different for development and production). It was beyond frustrating to get bug reports that lacked any bit of helpful information to track these nasty issues down, and resulted in many cannot-reproduce/close/re-open wars.

Be specific too! I recently received a bug report that said that the UI I created was different from the wireframe provided by the designer. It included a screenshot of my UI and one of the wireframe. I would have had to play the spot-how-many-differences game. Instead, a good bug would have been to list what was wrong.

You don't need to know how to program. Though a little bit of knowledge helps. Knowing what an Exception is and how one shows up would be beyond helpful. Knowing how they are thrown in the app you are testing would be even better.

Just the other day, I ran into an issue where an Exception was being thrown, but was actually masking the real problem. The Exception stated that there was a problem with the AuthenticationService I was using. I spent a few minutes trying to figure out what happened since nobody had recently worked on that particular service. We soon realized that the service itself was fine; the problem was actually in the config file shared by all services. The AuthenticationService was just the first one called, so it bombed first. Having a solid knowledge of how an Exception works and why the application is throwing them would avoid red herring problems like this.

This list is by no means exhaustive, but I do feel that if more developers were working with more testers that had those qualities, software in general would be much better.

1. That would have been somewhere between 10 minutes and an hour of writing my first bit of code.

2. Why is this so awful? First, any decent bug tracker will automatically make images easy to browse through and view; hiding it in a Word doc prevents that. Second pasting a screen capture will automatically save it as an uncompressed bitmap and result in a large file size. Even over a corporate intranet, it would still take far too long to download and extract. Seriously, friends don't let friends use a take poor screenshots.

This was cross posted on my personal site.

Say you had an object model that was generated from a database like so:

public class Lecture {
	public List<Speaker_Lecture> Speakers { get; set; }
}

public class Speaker_Lecture {
	public Lecture Lecture { get; set; }
	public Speaker Speaker { get; set; }
}

public class Speaker {
	public string Name { get; set; }
}

Unfortunately, you’re stuck with that semi-ugly Speaker_Lecture model, and cannot change it. How would you get a list of all the Speakers for a set of Lectures?

Without LINQ, I imagine you’d create an empty List of Speakers and then have nested foreach loops to go through the list of Lectures, and then the Speakers within those Speaker_Lectures. Kind of ugly.

But with LINQ, you can collapse it to one [long] line:

var speakers = lectures.Select(
    lec => lec.Speakers.Select(spk => spk.Speaker))
    .Aggregate((list, next) => next).Distinct();

And you’ll get a nice happy IEnumerable of Speaker.

What is it doing there? Well, the first select extracts the Speaker_Lecture objects from the lectures. Then, it extracts the Speakers from those objects. However, at this point, the Select function gives us a collection of a collection, which we do not want. Fortunately, Aggregate will walk through those collections and flatten them out to a single list. Distinct removes any duplicates.


Recently, I had a Dictionary<string, string> object that I wanted to display as a single string, essentially like “[key1] => value1  [key2] => value2” and so on. Back in the olden days of .NET 2.0 and prior, you could have just done it like this:

string s = ""; // Or a StringBuilder
foreach (string key in dict.Keys) {
    s += string.Format("[{0}] => {1}  ", key, dict[key]);
}

This was my first thought to solve the issue, but I wasn’t exactly thrilled with it. I knew that in Python this would be a relatively simple reduce (as in Map/Reduce) operation and could be done on one line.

I thought that there must be a similar way using LINQ to get a one-liner. There is no Extension Method called “Reduce,” but the equivalent is Aggregate. The only catch is that for the Aggregate function, the generic accumulator type must be the same as what is in the dictionary, which in this case was a key/value pair.

The solution is still technically a one liner, but I split it up for readability’s sake. You can use Select to munge the pair into a single string.

IDictionary foo; // Fill foo in here
string prettyPrint = foo.
    Select(pair => string.Format("[{0}] => {1} ", pair.Key, pair.Value)).
    Aggregate((acc, s) => acc += s);

If you have a bit of a Functional programming background, this makes sense, though if you do not, it may not be as readable.


I like frameworks. I spent far too much time in my programming infancy reinventing the wheel, and I don’t want to ever have to write more code than I have to again. Thus, I like relying on frameworks to take the tedium out of certain parts of the craft.

However, a problem arises when you don’t know what that framework is doing. Recently I spent many hours tracking down one specific bug that needed a single line changed to fix. In this specific case it was in an ASP.NET WebForms app. A <button> tag was used to make some fancy ajaxy calls via jQuery and PageMethods and the onclick event. The problem was that the form that was supposed to be displayed only appeared for a few seconds, then vanished again.

I poured through the code, set many breakpoints, broke out both Firebug and the Safari debugger (both proved useless due to the actual problem). Despite all of this, I still couldn’t figure out what the heck was going on. Finally, after I was ready to jump off the parking garage, the light bulb clicked on.

The <button> tag behaves like <input type=”submit” />; it sends a postback. And since the form was wrapped in an UpdatePanel, it was executing both the jQuery ajax, as well as the ASP.NET ajax, which were clobbering each other. In Firefox, the jQuery code finished first, and was then overwritten by the Page_Load from the UpdatePanel, causing our freshly rendered form to go away.

Changing the <button> to <input type=”button” /> fixed things. There are plenty of other fixes for this issue, but that’s what we used.

ASP.NET WebForms and ASP.NET Ajax like to abstract many of the “complicated” web development concepts away from the developer. I’m not going to comment on whether this is a good or bad thing, but it can present issues if you are using some mechanism and are not aware of what is going on under the hood. These problems usually don’t even manifest until late in a project- when the bosses are breathing down your neck and wondering why the app isn’t finished yet.

Does this mean we should abandon frameworks entirely and code everything in C (or assembler)? Certainly not, but if you are going to use that extra super duper happy UpdatePanel, you had better know what it is going to do to everything inside of it.


I gave two talks at Desert Code Camp this last weekend on ASP.NET MVC. The talks went well enough, though the time was far too short to cover everything I wanted to. I did get requests to share the code created during the sessions. So, here it is, for the most part.

http://krazyyak.com/sogeti/DemoBlog.zip

I say “for the most part” because it’s not the exact same stuff written during the presentation. I went through, cleaned up, and commented many of the methods and functions in everything.

The stuff with StructureMap is still pretty rough; I’m not a StructureMap expert, but I want to learn more about it. I’ll hopefully be able to blog about it in July.

As far as a disclaimer, this code is public domain and should be only used for demonstration/educational purposes. I offer no warranties on it, etc. Actually using it for a blog engine is probably a bad idea.

Feel free to discuss it here and ask questions.


In the project I am currently working on, we are using JavaScript and jQuery in just about everything. One of the frustrations that I have been seeing is that I’ll render a view representing an object, but also want a JSON version of that object to manipulate. I am also quite anal when it comes to keeping my code clean and separating the concerns; I did not want to have JavaScript strewn throughout my HTML or vice versa.

I considered a couple of options. The first just involved making an AJAX call to the server and having it return a JsonResult. This is clean, but I did not want to effectively double my calls to the server.

I also considered extending the MVC framework and writing my own ActionResult class that returned both HTML and JSON. However, that also “felt” wrong; like I was short-circuiting the framework. I would have had to rely on JavaScript on the client side to parse the resulting object and place the HTML, not something I want to be doing with all of my views.

I let it sit on the backburner for a while, and then today came up with a pretty good solution. It’s not ideal, but it works well enough for me. I created a simple HtmlHelper extension that just dumps the JSON out into a variable:

using System.Web.Script.Serialization;

    public static class JsonExtensions {
        public static string Json(this HtmlHelper html, string variableName) {
            return Json(html, variableName, html.ViewData.Model);
        }

        public static string Json(this HtmlHelper html, string variableName, object model) {
            TagBuilder tag = new TagBuilder("script");
            tag.Attributes.Add("type", "text/javascript");
            JavaScriptSerializer jsonSerializer = new JavaScriptSerializer();
            tag.InnerHtml = "var " + variableName + " = " + jsonSerializer.Serialize(model) + ";";
            return tag.ToString();
        }
    }

You call it from a page like so:

<%= Html.Json("foo") %>
<%= Html.Json("bar", Model.Something) %>

This is not an ideal solution, you are still technically putting JavaScript in the HTML. But, it does not make an extra call to the server, and the markup in the IDE is still very clean.

Another approach I considered was to have an Action that returned a JavaScriptResult that consisted of the same thing. You could then add a script tag like:

<script type="text/javascript" src="/javascript/MyObject/1000">

The catch though is that it would cause more overhead, especially if the app was designed in such a way that the retrievals are expensive (ours is).

I’m still ruminating on that perfect scenario, but for now I’ll be doing it this way.


Just prior to their 1.0 release, the ASP.NET MVC dev team added a nice feature to prevent CSRF attacks, the AntiForgeryToken. In brief, a CSRF attack is when a 3rd party gets one of your users to accidentally run a malicious script that accesses normally restricted URLs. The AntiForgeryToken pattern allows the web server to reject requests that come from places it was not expecting. More details can be found here. If you are a web developer and aren’t familiar with CSRF attacks, you need to fix that.

Anyways, the AntiForgeryToken bit is all well and good, but what if you are using jQuery (or another library) to handle your AJAX calls? Say you have an Action method like this:

[AcceptVerbs(HttpVerbs.Post), ValidateAntiForgeryToken]
public ActionResult DeleteAccount(int accountId) {
    // delete stuff
}

And you call it via:

$.post('/home/DeleteAccount', { accountId: 1000 }, function() {
    alert('Account Deleted.');
});

Since the POST does not include the AntiForgeryToken, it will fail.

Fortunately, it doesn’t take much brainpower to fix this. All the client side component of AntiForgeryToken does is put the token in a basic hidden field. So, you just need to pull that data out and include it in your AJAX call.

var token = $('input[name=__RequestVerificationToken]').val();

$.post('/home/DeleteAccount', { accountId: 1000, '__RequestVerificationToken': token }, function() {
    alert('Account Deleted.');
});

Do note that if you have multiple forms on the page with multiple AntiForgeryTokens, you will have to specify which one you want in your jQuery selector. Another gotcha is if you are using jQuery’s serializeArray() function, you’ll have to add it a bit differently:

var formData = $('#myForm').serializeArray();
var token = $('input[name=__RequestVerificationToken]').val();
formData.push({ name: '__RequestVerificationToken', value: token });

$.post('/home/DeleteAccount', formData, function() {
    alert('Account Deleted.');
});

Again, if you are making a web site that has any sort of interactivity at all, you need to be aware of these kinds of attacks. The tools to prevent them are readily available for you to use.


I’m a big fan of the ASP.NET MVC framework. I love the fine grained control it gives you over everything. But what I love most is that you can unleash the fully armed and operational firepower of jQuery.

Recently, I’ve been working on a site that is using both MVC and jQuery to make AJAX calls. Typically the method being called returned HTML (via a PartialView). But, if an error was thrown, I wanted to return a JSON object encapsulating it.

Unfortunately, jQuery could not decide how to handle a response that could be either HTML, or a JSON object. You can tell jQuery what to expect, or let it try to figure out what the response dataType is. Here’s the behavior of my Action method:

[AcceptVerbs(HttpVerbs.Post)]
public ActionResult HandleAjax(int id) {
    try {
        var myObj = RetrieveData(id);
        return PartialView("myview", myObj);
    } catch (Exception ex) {
        return Json(new { success = false, message = ex.Message });
    }
}

The problem is that regardless of what gets returned, jQuery only recognizes it as a string, and treats it as HTML.

I did think of one solution, but it used pretty poor design: it attempted to parse the responseText as JSON. If that was successful, it meant the error object was returned. But, if an exception was thrown, it was the HTML. The problem is that it was using error handling mechanisms to capture an expected result. Bad news; I’m not even going to post the code.

Fortunately, someone pointed out that if I set the http status code to 500, jQuery will send the result to the error() callback function, as opposed to success(). Thus, if you added:

this.Response.StatusCode = 500;

In the catch block, you can then use jQuery’s error() callback to do exactly as you’d expect:

$.ajax({
    data: { id: 100 },
    dataType: 'html',
    error: function(XMLHttpRequest, textStatus, errorThrown) {
        var errorObj = JSON.parse(XMLHttpRequest.responseText);
        handleError(errorObj);
    },
    success: function(data) {
        $('#resultContainer').html(data);
    },
    url: '/home/HandleAjax'
});

This also makes use of Crockford’s JSON library to parse the responseText string to a JSON object.

It is this transparency that makes me sing the praises of the MVC framework. Such a mechanism is certainly possible with classic WebForms, but it is hidden under layers of abstraction, and likely would have required mixing of markup, script, and server side code.


I kind of left my last post on this subject dangling, but now I’m ready to come clean.  I was proceeding merrily along, removing the tightly coupled LINQ to SQL generated code into something nice and loose, but ran into some serious snags when trying to update the database objects that had joins associated with it.

The code became more and more unwieldy; I was creating extra loader classes for longer inheritance chains, and more and more abstractions to get everything to work properly. At some point, a little voice in the back of my head started saying, this is supposed to be EASIER? Finally it reached a point where it simply did not appear that I would be able to meet my goal of defining my class models separately from LINQ to SQL’s generated ones.

So, I gave up and let the idea die while I focused on other things. Occasionally I did revisit it, but only met with the same conclusion.

About a month ago I became interested in the new ASP.NET MVC framework. Rob Conery has been publishing a series of videos that detail his process of creating an online store. While watching those, I realized that he was implementing the very idea that I had wanted to do here. And amazingly, the code to do it was surprisingly simple and small.

After studying it and poking around, here I am today with the answer! It lies in what Rob calls the Repository pattern. It boils down to this: you have your model classes defined somewhere (in the sample app, I’m using LinqExample.Core), an IRepository interface that states the methods that will interact with the database, and the implementation (LinqRepository here). Rob goes on to create a Service class that sits between the UI and Repository, but for this example, I’m leaving that out.

public interface IPersonRepository {
    IQueryable<Person> GetAll();
    Person GetSingle(int personId);
    IQueryable<TpsReport> GetReports(int personId);

    int SavePerson(Person person);
    int SaveReport(TpsReport report, Person forThisPerson);
}

This also makes testing easy. I can cook up a TestPersonRepository that returns dummy data, allowing any UI testing to avoid touching the database.

Next, we create the implementation of this interface using LINQ to SQL. Add the .dbml CropperCapture[13] file and drag the two tables onto the workspace as you would normally. The key part here is to change the Context Namespace and Entity Namespace to something other than the default (an empty string). I used LinqExample.Data.Repository. This has the effect of namespacing each generated class. Thus, they will not conflict with the model classes we have already defined.

Our example actually looks similar to what I laid out in Part 3, however I do not use the Entity types for the joining objects, but rather IQueryable<T>. This allows a lazy evaluation of the objects, and let’s LINQ to SQL’s binding work better. Additionally, I reverted all of the properties in the classes to the basic auto-properties that C# 3.0 brings us.

public class Person {
    public int Id { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public IQueryable<TpsReport> Reports { get; set; }
}

The key piece from Rob’s code is how he pulls the data using LINQ to SQL. Rather than just returning the auto-generated classes, he selects the actual Person or TpsReport class and the initializers to populate them:

public IQueryable<LinqExample.Core.Person> GetAll() {
    var people = from pe in this.db.Persons
                 select new Person {
                     Id = pe.id,
                     FirstName = pe.fname,
                     LastName = pe.lname,
                     Reports = this.GetReports(pe.id)
                 };
    return people;
}

At this point you may be thinking "All you did is create a class and just do a bunch of right/left property setting." While that is true (and the right/left bit does get tedious), it does more than that by return IQueryable<T>.

Returning as an IQueryable<T> has an extra advantage of letting you do further queries without having to throw away parts of the initial return set. Thus, if you chose to implement the Service layer, you could keep GetAll() in the Repository implementation, and have GetSingle(), GetWithLastName(), and others in the Service assembly.

If you returned an IList<T> instead, you would need to call ToList(), which would run the whole query right then, and forsake all the lazy evaluation benefits from IQueryable<T>. Of course, you may want to do that, but that is what the Service Layer is for.

Updating the database is not quite as simple, but doesn’t involve black magic either. The gist of it is to check to see if this is an update or an insert call, and to perform accordingly. Here is SavePerson(), but SaveReport() is very similar

public int SavePerson(Person person) {
    using (Repository.LinqExampleDbDataContext saver = new Repository.LinqExampleDbDataContext()) {
        Repository.Person pe = saver.Persons.SingleOrDefault(p => p.id == person.Id);
        bool isNew = false;
        if (pe == null) {
            isNew = true;
            pe = new Repository.Person();
        }
        pe.fname = person.FirstName;
        pe.lname = person.LastName;
        if (isNew) {
            saver.Persons.InsertOnSubmit(pe);
        }
        saver.SubmitChanges();
        return pe.id;
    }
}

And, you’re done! This is a little easier to swallow than what I had going in Part 3, no XML to muck around with, and is far simpler than the code soup I was cooking up before abandoning that approach.

Should you decide to go with another technology, be it the Entity Framework, NHibernate, or even a non SQL Server database, you would only need to create an IRepository implementation for that particular framework/database. It can be tedious for large quantities of tables, but that is the price you might have to pay for loose coupling and a friendlier design. Of course, you could refactor that to automatically set properties through Reflection or something, but that is another article altogether.


Source code for this post is located here.