XmlSerializerFormat is NOT supported by WCF Rest 4.0 in Medium Trust

Here’s one for you googlers (or bingites): Using the XmlSerializerFormat attribute on methods that you expose over WCF Rest (i.e. services using the webHttpBinding) is not supported if your service is running in Medium Trust (as it most likely is if you’re using Shared Hosting, like Rackspace Cloud Sites).

This is contrary to what you might expect from reading documents like this, which say that XmlSerializer is supported in partial trust. I guess they forgot to test the webHttpBinding scenario. I should be clear: webHttpBinding itself works fine in Medium Trust: it’s using XmlSerializerFormat with webHttpBinding that will give you problems.

That’s the short story.

Here’s the amplified version.

How I learned: the hard way

I’m building a web service that my friends running MacOSX want to be able to use. Out in the Apple orchards they’re not so keen on SOAP as we are who sit behind Windows. So I turned to WCF Web Http, now part of .Net 4.0, which lets you take control of your urls and keep your messages to the bare necessities.

With WCF REST you have the choice of XML or JSON. I chose XML, and by default WCF will use the DataContractSerializer. That produces reasonably clean XML, but it is picky about ordering of elements within messages, it annotates elements with type attributes that are often unnecessary, and it has a rather clunky syntax for serializing arrays.

Then I read about the XmlSerializerFormat attribute. If you apply that to your Operation Contracts, it switches to using the XmlSerializer, and then, by decorating your data contracts with the appropriate attributes, you can take complete control of the xml which gets written to the wire.

All well and good – while the service was running on my machine. But when I pushed the bits to our web host, who run all websites under Medium Trust, it was a different story. All the calls to the service threw a ProtocolException with the helpful message: “(400) Bad Request”.

Turning on includeExceptionDetailInFaults, then spying on the interaction using Fiddler shed more light on the situation. I was getting back a stack trace with this at the top:

at System.ServiceModel.Dispatcher.UnwrappedTypesXmlSerializerManager.BuildSerializers()
   at System.ServiceModel.Dispatcher.UnwrappedTypesXmlSerializerManager.GetOperationSerializers(Object key)
   at System.ServiceModel.Dispatcher.SingleBodyParameterXmlSerializerMessageFormatter.EnsureSerializers()
   at System.ServiceModel.Dispatcher.SingleBodyParameterXmlSerializerMessageFormatter.GetInputSerializers()
   at System.ServiceModel.Dispatcher.SingleBodyParameterMessageFormatter.ReadObject(Message message)
   at System.ServiceModel.Dispatcher.SingleBodyParameterMessageFormatter.DeserializeRequest(Message message, Object[] parameters)

A bit of hunting around using Reflector revealed that the BuildSerializers method always calls XmlSerializer.FromMappings, and if you check that out on MSDN, you’ll see that it requires Full Trust.

So there’s no getting away from it. You can’t use XmlSerializer together with WebHttpBinding under Medium Trust. I did try the usual work-around for getting XmlSerializer to work in Medium Trust – pre-generating the serialization assemblies at build time: but that was before Reflector showed me the clinching evidence that my efforts were futile.

So what are your options? Either stick with DataContractSerializer, or switch to JSON¹. Or, if time is on your side, wait for the next version of WCF which has a new extensible Web API specially designed to let you take complete control of your message formats and such like.

---

1. Decorate your methods like this to switch to JSON format:

[ServiceContract()]
public interface IService
{
    [OperationContract]
    [WebInvoke(UriTemplate = "/MyOperation", 
               Method = "POST",
               RequestFormat = WebMessageFormat.Json,
               ResponseFormat = WebMessageFormat.Json)]
    Response MyOperation(Dto dto);
}

Let’s synchronize our watches: Determining Clock Skew with Christian’s Algorithm and the Reactive Framework

In every playground race, there’s always one child who sets off somewhere between “On your marks” and “Get set”, gaining an unfair advantage over the honest ones who wait for “Go!”. I faced a similar problem in the design of my Windows Phone 7 multi-player game, Simon Squared, though it was unsynchronized clocks and network latency that were to blame rather than cheats.

The multi-player mode in the game relies on puzzles being shown to all players at the same time, and players race to complete them. Since some puzzles can be solved in a matter of seconds, players who get to see them even slightly sooner than their competitors stand a good chance of getting to the solution first. So I needed a plan for avoiding the digital equivalent of playground squabbles.

Any such plan has to take into account network latency: an unpredictable amount of time can elapse between the server shouting “Go!”, and the phones hearing the message. So I got my server to send out a message announcing that it would shout “Go!” at time T, where T is 4 seconds on from the time on the server’s clock (and broadcast in UTC obviously). This gives even the most far-flung phone plenty of time to receive the message, check its own clock, and then start a count-down.

Bet you’ve spotted the flaw in that straight-away! What if the server’s clock and the phones’ clocks are out of sync?

An Overview of Christian’s Algorithm

That’s where Christian’s algorithm comes in. It gives us a simple method of broadcasting the time at the server to a client whilst allowing for network latency. It works like this:

1. Client sends a message to the server: “What’s the time?” [adding ‘Mr. Wolf’ is optional]. Crucially, it notes the time that it sent the message (call it T_sent)
2. Server responds as quick as it can, giving the time according to its own clock, T_server.
3. When Client gets the message, it notes the time of receipt (call it T_received). Then it does some maths: the round-trip time, RTT, is T_received – T_sent . So assuming that the server responded instantly, and that the network latency was the same in both directions, that means that the server actually sent the message RTT/2 seconds ago. Thus, at the instant Client receives the message, the time at the server is T_server + RTT/2. Then the Client can compare with its own clock and determine the difference – the clock skew.

Since network latency can vary with each request we can get an improved result by trying the whole exercise several times then cherry picking the result which had the shortest round trip time, since that minimises the error in our estimate of the network latency.

So what does that look like in code?

The Server Side – with WCF REST

Well first we need the server to be able to serve up the time:

[AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
[ServiceContract]
public class TimeService
{
    [WebGet(UriTemplate = "/?timeAtClient={clientReportedTime}")]
    public TimeCheck GetTime(string clientReportedTime)
    {
        return new TimeCheck()
                   {
                       ClientReportedTime = DateTimeOffset.Parse(clientReportedTime, CultureInfo.InvariantCulture, DateTimeStyles.AssumeUniversal),
                       TimeAtServer = DateTimeOffset.UtcNow,
                   };
    }
}

This is making use of WCF 4’s REST capabilities. Notice the WebGet attribute on the method, with it’s UriTemplate. If I add the appropriate route  to my Global.ascx file,

routes.Add(new ServiceRoute("Time", new WebServiceHostFactory(), typeof(FeedbackService)));

that will ensure that HTTP GET requests like http://localhost/MyApp/Time/?timeAtClient=2011-03-17T20:19:30.2196972Z get routed to the CheckTime method with the appropriate part of the query string passed into the clientReportedTime parameter.

So that's the server side done.

RestSharp and Rx for Asynchronous Web Requests

Then the client needs to be able to call the server. I’ve been making use of the excellent RestSharp library here:

_restClient = new RestClient(ServerUrl);

...

private IObservable<TimeCheck> GetServerTime()
{
    var request = new RestRequest("Time?timeAtClient={timeNow}");
    request.AddUrlSegment("timeNow", DateTime.UtcNow.ToString("O"));

    return ExecuteRequest<TimeCheck>(request);
}

private IObservable<TResult> ExecuteRequest<TResult>(RestRequest restRequest) where TResult : new()
{
    var subject = new AsyncSubject<TResult>();

    _restClient.ExecuteAsync<TResult>(restRequest, response => HandleResponse(response, subject));

    return subject;
}

private void HandleResponse<T>(RestResponse<T> response, AsyncSubject<T> subject)
{
    subject.OnNext(response.Data);
    subject.OnCompleted();
}

This is where things start hotting up. The first part of GetServerTime is obvious enough: in lines 7-8 I’m constructing a RestRequest consisting of a url and the appropriate query string parameter. Then I Execute it. Now since we’re talking Windows Phone here, I can’t make synchronous web requests – the phone UI has to remain responsive. So I can’t wait and then return the response from the method.

Instead I return an IObservable, an interface you might not be familiar with. This creature is the brain-child of the clever folks on the Reactive Framework team. They like to call IObservable the mathematical dual of IEnumerable. And what they mean is this: IEnumerable is used when you want to pull elements one by one out of a collection. IObservable is used when you want elements to be pushed one by one to you. You can think of an IObservable as being a stream of events. And just like you can build queries on top of IEnumerable, using Where and Select, so you can build queries on top of IObservable.

Here I’m using an IObservable as the channel to push back the result of the Web request when it becomes available – you can see that happening in the HandleResponse method, which is the callback that RestRequest calls when it gets a response back from the web server.

Finally – the Algorithm

Finally we get to the implementation of the algorithm itself: and with the underpinnings in place, it turns out to be one big LINQ query:

public IObservable<TimeSpan> DetermineClockSkew()
{
    var timeOffset = (from tick in Observable.Interval(TimeSpan.FromMilliseconds(100)).Take(5)
                      from timeCheck in GetServerTime().Timestamp()
                      let estimatedOneWayLatency =
                          (timeCheck.Timestamp.UtcDateTime - timeCheck.Value.ClientReportedTime).
                              TotalMilliseconds/2
                      let estimatedTimeAtServer =
                          timeCheck.Value.TimeAtServer + TimeSpan.FromMilliseconds(estimatedOneWayLatency)
                      let predictedClockSkew = estimatedTimeAtServer - timeCheck.Timestamp
                      select new {predictedClockSkew, estimatedOneWayLatency})
        .MinBy(p => p.estimatedOneWayLatency)
        .Select(p => p.predictedClockSkew);

    return timeOffset;
}

Let's step through this line by line:

• Line 1: We start things off by generating a stream of 5 tick events, at 100 millisecond intervals – we’re going to ping the server 5 times, and use the result with the shortest round trip time
• Line 2: Each time we hear a tick we fire off a what’s-the-time request to the server using GetServerTime which we discussed above. The Timestamp() method will stamp each response that we get back from the server with the time at which we received it.
• Line 3: For each response we get back from the server we estimate the one-way latency from server to client by taking half of the round-trip time (when we send a request to the server we send our own time, and the server echoes this back in the ClientReportedTime property of the message)
• Line 4: Now that we have an estimate of how long the message took to get from the server, we can estimate the time at the server at this precise moment.
• Line 5: We can now figure out the difference between the clock on the phone and the server’s clock by comparing the time at which we received the server’s response with our estimate of the server’s time at that moment.
• Line 6: We stash our estimate of the latency, and the estimate of the clock skew into an anonymous type
• Line 7 - 8: We look over all 5 attempts at estimating the clock skew, and we take the one which had the smallest latency. Notice how MinBy returns an IObservable rather than an actual value as it might in good old Linq-to-Objects: it has to be this way, otherwise the whole method would block waiting for all the web requests to complete, rather defeating the our use of asynchronous calls elsewher.

So there you have it: thanks to Christian, the WCF REST framework and the Rx team, a neat and elegant way of determining clock skew between server and phone.

See it in Action

Remember, for a limited time only you can see this in action by downloading my game to your phone or Emulator (for free!) and playing with your friends. Full source code to the whole game is also available.

Finally, judging for the Windows Phone 7 app contest finishes on March 31st, so make sure you check out the entries, and be sure to tell Red Gate what you think of mine!

Simon Squared – We have Multi-player: Days 4, 5 and (ahem!) 6

OK. So you let me have one more day, and I took 3. But I got it working. After 6 days of work, I can show you Simon Squared in all its Windows Azure powered multi-player glory:

Simon Squared Multi-player, nicely showcasing my Multiple-instances-of-the-WP7-emulator hack

So what took me so long? Surely after getting the Windows Phone 7 emulator to talk to the Windows Azure emulator, finding a UI library for XNA and enabling multiple instances of the WP7 emulator to run at once, everything else should have been easy? If only!

Here’s what happened.

Day 4

9:00 Start by setting up a Windows Azure subscription, making use of my MSDN subscription benefits: it's pretty generous: 750 hours/month of Compute Time, 10 GB storage, 7/14 GB traffic in/out.

I get a little worried when the sign up screen tells me that I'll receive an activation email "within 24 hours" - but I'm relieved to see that within a couple of minutes of being redirected to the Azure management console (very swish Silverlight app) my subscription appears, and I can start creating deployments.

10:11 Got side-tracked into a discussion with my boss about possibilities for future products and offerings based on the work we've been doing here

12:45 Implemented the phone side behaviour for running the count down to the games beginning. Making heavy use of the Reactive Framework (which comes built into WP7) for hiding away the messiness of asynchronous calls.

13:45 Problem with WCF REST: if I return a derived type of message from my operation, the client only receives the properties defined on the base type. This is the problem that the KnownType attribute solves elsewhere in WCF, but it doesn’t seem to be working for WCF REST.

15:22 Discovered that WCF REST uses XmlSerializer by default rather than DataContractSerializer - so KnownTypes woudn't work. Also discovered that RestSharp is using XmlSerializer when sending data to the server, so not serializing inherited types in the way that DCS is expecting.

15:39 Now got inherited message types passing correctly: WCF Rest and RestSharp are both pretty pluggable, so I was able to implement a custom ISerializer in RestSharp, and a custom MediaTypeProcessor in WCF Rest, both using DataContractSerializer instead of XmlSerializer

17:30 Caught out by Http response caching. The symptom was that I’d launch a fresh HttpWebRequest,  but get back stale data. Investigation showed that the requests weren’t ever leaving the phone – it never showed up in Fiddler. I spent ages prodding my Reactive Framework query, convinced that it wasn’t issuing the requests properly. Then I remembered Http Caching. In my haste, and newbie-ness, I solved the problem by putting a NoCache on all the responses on the server side. A better way would probably have been to set the WebRequest.CachePolicy.

Day 5

09:00 Thought occurs to me that I might get on faster if I don't have to keep waiting for the Azure Emulator to start up and shut down every time I build my server. Why not just run my server project directly in IIS? I try it – and WCF Http stops working. I eventually discover that the CTP bits I’m using don’t yet support HTTPS, and I had a HTTPs binding configured on my default Website in IIS.

10:30 Fortunately Cassini works - why did it take me an hour and half to discover that!

10:43 Start work on refactoring puzzle generation, so I can do it on the server side. The goal is that the server will generate rounds of 5 puzzles, and send them out to all the players in advance. Within a round, the server will send out a message saying “start puzzle X at Time T” – and thus synchronize the experience of each of the players

12:00 Working on updating the Phone side to play the puzzles that server sends out.

13:16 Finished refactoring the Game screen so that it is backed by a GameController - allowing me to implement a SinglePlayerGameController and MultiPlayerGameController.

17:20 Got some simple state machines working on the client and server. It looks something like this:

17:40 With two instances of the emulator running, I can now see the same puzzles displayed on each. The problem is, they’re horribly out of sync – player 1 might get the puzzle two seconds ahead of player 2, which for small puzzles means he’s certain to win!

Day 6

09:21 Implemented Christian’s algorithm to estimate the clock skew between server and phone. The implementation is pretty straight-forward using the Reactive Framework, and quite elegant, if I do say so myself. Remind me to show you, sometime.

10:33 Couldn’t figure out why changes I was making in the code weren’t having any effect when I ran the game. Then discovered that Visual Studio had some how set itself to “Never Build on Run”.

12:00 It’s working! Multiple players now receive the puzzles beautifully in sync, one after another. As soon as one player completes a puzzle, it vanishes from all the other players screens, and the count-down to the next puzzle begins

13:20 Implemented a scoreboard to show the position at the end of each round

14:30 Having deployed the game to my phone, I challenge Vinod to a dual – me on the emulator, him on the phone. I now discover a serious downside to being a games developer: once a game reaches playability, you inevitably get sucked into playing when you should just be testing.

15:35 Use Expression Design 4 to recreate a game logo that Neil designed using Word. What do you think?

17:05 Spitting and polishing. Tidying up the various screens. I might not have time to make them pretty, but at least I can make the buttons big enough to touch. That’s the downside of testing with an emulator using a mouse – it doesn’t alert you to the imprecision of fat fingers!

17:30 Waiting for Windows Azure to deploy the server. Visual Studio 2010 is supposed to be able to package and deploy your project straight into the cloud, but I couldn’t get that to work (something about it not being able to resolve the URL of my storage account). Instead, I uploaded the package manually.

17:45 Still waiting for Windows Azure to start up my role instance. Give up and go home.

21:00 Wife gives me permission to interrupt our traditional Thursday evening film (The Prestige last night, – recommended) to see whether the Azure role has started up yet. It hasn’t, so in the time-honoured way I tell it to reboot – and that fixes it. A few minutes later, she’s thrashing me – mainly because she’s using the phone, and I’m using the emulator over Remote Desktop (my laptop is old, and doesn’t support XNA games in the emulator) – in that setup the emulator doesn’t respond to my mouse moves!

What I need is another phone.

Ah, what’s this I see – the Windows Phone Marketplace Developer Newsletter, announcing Windows Phone 7 Handsets for Developers. The Newsletter says that local evangelists have codes for free phones to give away. I think Mike Ormond is the WP7 evangelist for the UK …

Multi-player enabling my Windows Phone 7 game: Day 3 – The Server Side

I’m building a game to enter in Red Gate’s Windows Phone 7 App competition. I built the core of the game in 3 days from a standing start. Now I’ve challenged myself to make the game multi-player in 3 days, using Windows Azure for the server side.  Day 1 was spent getting up to speed with Windows Azure and networking on WP7. On Day 2 I got the UI sketched out. Here’s Day 3:

8:30 Begin the day with excellent news. With barely any badgering at all Red Badger have kindly offered me an non-expiring build of their XPF UI library for XNA. Thanks chaps!

And thanks to everyone who's shown support by tweeting the last couple of blog posts. I can't tell you how motivating it is as I set to work again this morning.

8:45 Microsoft have released the January 2011 Update to the Windows Phone Developer Tools (on the 4th of February mind, but I'll let that pass). This is the one that brings the promised Copy-Paste support and some performance updates. Being the good boy I am, I install it before beginning anything else.

9:32 Staring work on the server. I've never designed anything RESTful before, so I've taken a quick look at a couple of articles: one at quite a high level, and one in a little more detail.  A logical place to begin is starting a game. So what I think I need to do is define a /Game resource on my server, and make it respond to POST requests.

11:45 Hooked up AutoFac on the server so that I can inject dependencies into my REST services.

12:00 A good chunk of the day gone, and I still haven't implemented starting a game. WCF isn't playing nice at the moment: I can't get it to respond to the POST request I'm making to my Service.

12:01 Writing the above fixed the problem! Well, not quite. As soon as I had written that, I thought of fiddling with the UriTemplate on the WebInvoke attribute. Since I was posting a request to the /Games resource, and I had registered my Games service using routes.AddServiceRoute<GamesService>("Games", serviceConfiguration) I thought I could get away without specifying a value for UriTemplate on WebInvoke. Turned out I needed to specify "/"!

12:05 Next: list all available games, so that a player can choose one to join.

12:45 Done! I can now create several games through the Start a Game screen, and see them all listed on the Join a Game screen.

16:15 The afternoon passes in a blur. I’ve added several operations to my service, so that the basic game setup screens are working. This is what I’ve got on the server side:

Resource	Http Method	Meaning
/Games	POST	Start a Game
/Games	GET	List existing games
/Games/{id}/Players	POST	Join a game
/Games/{id}/Players	GET	List players belonging to a game

Notice any conspicuous absences? That’s right: there are no operations to support actually playing a game. Where has the time gone? But before I can move on, I should at least check that this service works if multiple devices connect to it.

16:25 Fustrated: the Microsoft Windows Phone 7 emulator is single-instance. How can I test a multi-player game if I can only run one copy of it?! I’ve got to find a way round this.

17:30 Eureka! After a journey deep into the bowels of the Microsoft.SmartDevice.Connectivity API and its associated XML datastore I’ve found a way to have multiple instances of the emulator running at once, and even debug them simultaneously.

Hey! Hang on just a minute: there’s already a post on my blog about that – is someone with a time machine playing around with me here?

But never mind that. My 3 days are up, and all I’ve got to show for it are the game sign-up screens. So where do we go from here? Well, I’m going to make my excuses and plead for an extra day – just one more day, and I’m sure I can get it working!

My excuses:

• It took me ages to get the Windows Phone 7 emulator talking to the Windows Azure emulator: the good folks at Microsoft haven’t connected the dots here.
• A good two hours or more of my first day was taken up with the hunt for a working UI library for XNA. Thanks to Red Badger for saving me further frustrating hours.
• Running multiple instances of the emulator really shouldn’t be that hard.

So what do you think? Can I have another day?

Multiplayer-enabling my Windows Phone 7 game: Day 1

I’m building a game to enter in Red Gate’s Windows Phone 7 App competition. The first challenge I set myself was to build the core of the game in 3 days from a standing start. Then Red Gate moved the goal posts. So I’ve set myself a new challenge: make the game multi-player in 3 days, using Windows Azure for the server side. Here’s Day 1:

9:00 Started researching networking capabilities in WP7. MSDN has a useful summary. In short, HttpWebRequest and WebClient are supported, as are parts of WCF. One notable absence are Duplex Services – so it looks like I’ll be doing my own client side polling. ChannelFactory.CreateChannel is not supported, so service proxies cannot be generated dynamically - they have to be created using Add Service Reference, or slsvcutil on the command line. Since I’m not a great fan of all the boilerplate that the Add Service wizard spits out, I think I’ll give WCF on the client side a miss. This gives me the idea of creating a RESTful service that I can call using plain old HttpWebRequests.

9:16 Given the recent hoo-ha  over Windows Phone 7 data leaks, I think I should do my bit to save bytes and minimise data usage. I'Since I’m quite keen on Google’s Protocol Buffers format, I’m pleased to discovered that ProtoBuf.Net, Marc Gravell's implementation of the Google Protocol Buffers format supports WP7.

9:22 Starting work on a Windows Azure project for testing communication between client and server. It’s neat how the Cloud project in VS 2010 has a Windows Azure emulator.

10:38 Downloaded and built Microsoft's WCF Web APIs Preview 3. This extends the REST support that WCF has, as of .Net 4.0. Using this, and helpful blog post from Jesus Rodriguez, got a test service sending back responses using Protocol buffers. One thing that caught me out was failing to decorate my data contract classes with the [ProtoContract] and [ProtoMember] attributes. The serializer just returned an empty stream without that. Now to get the phone talking to the server.

11:05 An interruption: the boss arrives back in the office with an HTC HD7 for me to test the game on!

Signed up for a developer account to allow the phone to be unlocked for testing my apps. To my dismay, I discovered that I can't unlock the phone for deploying test apps until the publisher verification process has been completed. What?! I can understand why Microsoft would want to check that our company is who it claims it is before it lets us submit Apps to the market place; but to my own phone? I think I know who I am!

11:39 Encountered my first real problem: Windows Azure compute emulator only listens on the loopback address, 127.0.0.1; this means that the Windows Phone 7 emulator can't talk to it. Seems I’m not the first to have encountered this. I tried a number of solutions:

• Using NCat, a Port fowarding tool. No joy with this at all.
• TcpTrace, another kind of port forwarding tool, worked, but not when Fiddler was listening in.

13:00 Thanks to some suggestions from Phil Haack I discovered that Fiddler will do the job on its own (is there anything it can’t do?). Following these instructions, I set up Fiddler as a reverse proxy. This means my WP7 app can connect to MyMachine:8888, and Fiddler will shuttle the traffic to and from the Azure emulator.

13:10 Discovered RestSharp, a library that builds on top of HttpWebRequest to simplify making calls to RESTful services. In particular, it can deserialize the Response bytes into objects for you.

13:17 Trying to write a ProtoBuf deserializer for RestSharp, but Protobuf is throwing  MissingMethodException. It looks like I might have to use pre-built Serialization assemblies

14:40: Following Mark Gravell's post on using Protobuf-net with WP7, I mange to get deserialization of messages working on WP7 by pre-building a serialization assembly. I have to resort to subterfuge to get the resulting Serializer assembly added to my WP7 project: The Add Reference dialog refuses to do it, claiming that the assembly is not a bona-fide WP7 assembly, so have to add it to the .csproj file by hand.

15:00 In order to allow users to setup multiplayer games, I’m going to need to show some kind of UI. WP7 doesn’t allow you to mix and match Silverlight and XNA within one App. I don’t really fancy rolling my own UI library, so I start casting around for one. A helpful post on the XNA forums provides a list libraries for XNA in general, though only a couple of them support WP7.

15:35 Start investigating Nuclex UserInterface library, a UI library for XNA that claims support for WP7. Initially it looks quite promising: I can get it to render a couple of buttons. But I can’t make the buttons respond when I tap them.

17:15. I give up on Nuclex, and look for something else. The most promising looking framework is XPF, from Red Badger. It’s ambitious: it aims to replicate some of the core features of Silverlight for the XNA framework, including layout, data binding and animation. They have nightly builds available now, but they’re only valid for 30 days from the build date, and there’s no word yet on a release date, or indeed what the cost of a license will be. Do I take a chance on it?

What are your thoughts? Have I missed a framework that will save me hours? Get in touch: @samuel_d_jack on Twitter, or by email.

A Fiddler plug-in for inspecting WCF Binary encoded messages

If ever you're needing to debug the interaction between a Web Service and its clients, Microsoft’s Fiddler is the tool to use - this includes WCF Services so long as they're using a HTTP transport. The only thing Fiddler won't do is decode messages that are sent using WCF's proprietary Binary encoding - until today, that is: at lunch time, I took advantage of Fiddler's neat extensibility mechanism and created a rough-and-ready Inspector that will translate binary messages from gobbledegook to plain xml for your debugging pleasure.

You can download the plug-in and source from MSDN Code Gallery. To use it, just drop the plug-in  in the Inspectors folder of your Fiddler installation. Once you've reloaded Fiddler, switch to the Inspectors tab and look for WCF Binary.

Implementation Notes

• There’s a very helpful page on the Fiddler site which tells you how to build Inspectors in .Net.
• Fiddler gives each Inspector the raw bytes of each message, and it can do with it what it likes. Here’s how I decode a WCF Binary encoded message:

using System;
using System.Runtime.Serialization;
using System.ServiceModel.Channels;

...

private static readonly BufferManager _bufferManager = BufferManager.CreateBufferManager(int.MaxValue, int.MaxValue);

...

private string GetWcfBinaryMessageAsText(byte[] encodedMessage)
{
    var bindingElement = new BinaryMessageEncodingBindingElement();
    var factory = bindingElement.CreateMessageEncoderFactory();
    var message = factory.Encoder.ReadMessage(new ArraySegment<byte>(encodedMessage), _bufferManager);
    return message.ToString();
}

If your WCF service is unexpectedly receiving a null parameter value, try this…

An interesting problem ate up an hour and a half of my life yesterday morning. It was all to do with a WCF service.

The parameter that I was carefully formulating on one end of the wire and passing to my service proxy was not being received by my service at the other end - instead it was getting a null value. It worked the day before - why had it stopped working overnight?

Intercepting the http traffic with the help of the wonderful Fiddler (and a useful article about Fiddlering .Net by Rick Strahl) confirmed that a serialized version of the parameter was indeed being sent down the pipe and was arriving at the server - so why wasn't my service getting it?

After much head-scratching, I thought of checking the history of our Server project in the Source code repository. That gave me the clue I needed. In the file containing the interface which defines the service's contract, I noticed that the name of the parameter on the method in question had been changed.

Now at this point I should explain that, partially inspired by this article, we have chosen to eschew the usual route of having Visual Studio create our Service proxies for us. We prefer to share the assembly containing the Service contract interfaces and Data Contract objects between the client and the server projects. This approach works very nicely most of the time. It saves us having to "Update Service Reference" every time we change the shape of the sever side, and it reduces our dependence on the magic that Visual Studio usually works on our behalf.

Where this approach does get a little bit messy is when you want to have asynchronous versions of the operations on the client side (Begin[Operation] and End[Operation] methods). Clearly we don't need these on the contract implemented by the Server, so what we end up doing is creating a copy of the Contract named something like IContractClient which contains both the synchronous and also asynchronous versions of the operations.

Now can you see where this is going?

The problem turned out to be that the name of the parameter on the client version of the contract was different to that on the server version of the contract. The message on the wire contained the client’s (incorrect) name for the parameter. So presumably, when WCF received the message, it failed to match the parameters with those expected by the method on the server, shrugged its shoulders, and passed in a null value to the method.

But the story doesn't end there. I found the client version of the contract interface, made what I thought was the necessary correction, and whilst Visual Studio was rebuilding I announced to my colleague that I'd fixed the problem. Only to find, when I tested it, that it was still broken.

More head-scratching, and Fiddlering: the old parameter name was still being passed through. Huh? Clean and Rebuild to make definitely sure the change had stuck. Still no go.

And then the final ray of light. On the client side we were using the asynchronous version of the operation, so it was the Begin[Operation] method that I'd corrected. And I'd left the synchronous version untouched.

Once that was corrected everything started working, and I could finally begin the day's work.

In conclusion, for all you googlers:

If your WCF service is receiving null values instead of the parameters you are expecting, check that the parameter names on your methods on the client version of the contract match those on the server version of the contract. And on the client side, be sure to check both the synchronous and asynchronous definitions of your methods.

How-to: Create a Server Certificate for a WCF Service

One of the things I took a while to get used to when I started working with WCF were the certificates that are needed to secure Services.

Microsoft have configured WCF to be secure by default. This means that developers need to be much more imaginative in finding ways to open up gaping security holes in applications: there are several incantations you need to perform if you want to send passwords unencrypted over the wire, for example.

As a consequence however, if you do need to exchange security credentials, and are not in a situation where you can use Windows Credentials, you might find yourself needing to specify a Certificate that can be used to encrypt messages exchanged by client and server. You know you need one if you get errors like

System.InvalidOperationException: The service certificate is not provided. Specify a service certificate in ServiceCredentials.

The books I consulted (Learning WCF and Programming WCF Services) talked a lot about creating Certificates that could be used in a test environment (using makecert and the like), but almost nothing about creating certificates for a production environment. Using Test Certificates even during development is also less than ideal: because they are not properly signed by the relevant authorities you have to configure WCF not to do the full security checks on them; apparently there are also some performance problems.

After digging around a bit, I figured out that the X.509 certificates that WCF says it needs are the same thing as the SSL certificates you can purchase from Thawte, Verisign, and co - call me stupid for taking so long, but I haven't seen this actually spelled out. This led me to discover an easier way of generating test certificates. Comodo offer a very nice service letting you create free test SSL certificates that are valid for 90 days. As far as I can tell, they don't stop you recreating the certificates at the end of that time either. Because they don't validate domain names for these test certificates, you can use this service to create certificates for machines on your intranet. Their prices for full certificates seem very reasonable, particularly if you need them only for intranet use.

Getting and setting up a certificate

In the hope that I'll save somebody else some time, these are the steps I follow to set up my WCF service with a certificate:

1. Generate a Certificate Signing Request (CSR). The easiest way to do this is with IIS. Here's how if you use IIS 5.1 or 6.0 (see the bottom of this post for instructions using IIS7). Don't do this on a server that's serving up a site - it will change the IIS Certificate for the site. You can however create the certificate on one computer and export it to another.
   1. In IIS, right click on one of your websites (there'll only be the Default Web Site if you're doing this in Windows XP), then select Properties. Switch to the Directory Security tab.
      
   2. Click the Server Certificate button.
   3. In the IIS Certificate Wizard, select "Create a new Certificate".
      
   4. (If you see three options (Renew, Remove, Replace) that means you already have an SSL certificate configured. If you're sure it's safe to do so, you can select Remove to get back to the position of generating a new one. )
   5. Fill out the screens that follow. The most important one to get right is "Common name". Here you should put the same DNS name that you intend client computers to use when connecting to the Server. If, for example, you only intend to connect from the same machine, then it's fine to use "localhost". Use the server's name if you intend clients to connect over an intranet. For servers being accessed from elsewhere, include the full domain name, e.g. mymachine.mydomain.com.
   6. In the last but one step of the wizard, make sure you note down where IIS puts the Certificate Request File.
2. Get the certificate signed. This is where you'll need to use a Certification Authority (such as Thawte or Verisign, or one you set up yourself). As I mentioned, I've found that Comodo is the easiest to use for creating Test Certificates.
   1. On the Comodo website look for the "Get It Free Now" link; or one of the other options if you're ready to splash out and buy a full certificate. Follow the link through to the Certificate request page.
   2. At this point you'll need to open up the Certificate Request File that IIS created, and copy-and-paste its contents into the appropriate box on the web page.
      
   3. Complete the rest of the request at the site, then wait for an email to come through with your certificate zipped up inside it.
3. Import the certificate into IIS. Now you'll need to hop back to IIS.
   1. Get to the IIS Certificate Wizard as we did in step one.
   2. This time, it should give you the option to "Process the pending request"
   3. Follow through, and select the file that you got back from Comodo, or other CA.
   4. Complete the Wizard: the certificate is now available for use in IIS if you need it.
4. Make sure the Certificate is available for WCF. To do this we'll need to use the Certificate Manager Snap-in.
   1. Click the Start button, type mmc, then press enter. This will display an empty Management Console.
      
   2. Click File->Add/Remove Snap-in.
   3. In the Add/Remove dialog, click Add.
      
   4. Select "Certificates" from the list and click Add. When prompted in the following dialog, select to manage certificates for the Computer account on the Local Computer.
   5. Close and OK back to the main Console Window. The Certificates node should now be there.
   6. Check that a Certificate with your machine's domain name appears in the Personal\Certificates folder.
      If you have created a certificate for localhost, you can use this Console to export the Certificate, and import it on another machine, to save having to create multiple test certificates. The same applies if you have created the certificate on a machine different to the Server that will use it. Right-click the certificate, then select All Tasks->Export to export from this machine; to Import on another machine, bring up this same console, then right click the Personal/Certificates folder and select All Tasks->Import.
5. Now you're ready to configure WCF. These steps assume you're using the config file to configure your service.
   1. Open the Services config file. Look for the behaviour element that's used by your service. Under it, add a <serviceCredentials> node, then, inside that a <serviceCertificate> node. It should look something like this:

      <configuration>
       ...
        <system.serviceModel>
          <behaviors>
            ...
            <serviceBehaviors>
              <behavior name="[YourServiceBehaviourName]">
                <serviceCredentials>
                  <serviceCertificate findValue="localhost" storeLocation="LocalMachine"
                    storeName="My" x509FindType="FindBySubjectName" />
          </serviceCredentials>
          ...

      Update the findValue attribute to include the domain name of your machine - whatever you put for the Common Name in the Certificate Signing Request. For completeness, I should mention that the value in the storeName attribute corresponds with the top level folder under the Certificates node in the Certificates Snap-in (thus the "Personal" folder becomes the "My" store). If for some reason IIS puts your certificate in a different folder, then you might need to change the value in storeName.
6. That's it. Your newly secured service is good to go.

Generating Certificate Requests in IIS7

[Update 10/11/08]

Kevan Brewer kindly sent me instructions for generating certificate requests in IIS7. These replace steps 1 and 3 above.

1. Generate a Certificate Request.
   1. Click Start -> Run and type inetmgr.
   2. Click on the root node in IIS manager and double-click Server Certificates from the centre panel.
   3. In the Actions panel on the right, click on Create Certificate Request...
   4. Fill in the form paying particular attention to the Common Name field on the first page. On the last page, click the browse button, browse to the directory of your choice and enter a file name e.g. testcertificate. (I found that if I just typed a name into the box, it either created the file somewhere random or didn’t create it at all.)
2. Get the Certificate Signed. This is the same as step 2 above.
3. Import the Certificate into IIS.
   1. Back in IIS Manager, click on Complete Certificate Request... on the Actions panel.
   2. Browse to the directory to where the zipped certificate files were extracted, change the file type to *.* since, by default, it only shows files with a .cer extension. Select the file which most closely matches the name you entered for the Common Name field in step 1.4.
   3. Enter a suitable name for the friendly name (e.g. test certificate) and click OK.