Monday, 28 July 2008

Project Euler Problem 16: Calculating Large Powers of Two

I always feel like I've got good value-for-brain-cycles when I can reuse a solution to a problem. I had that satisfaction when I came to look at Problem 16 in the Project Euler series: calculating the sum of the digits of the number 21000. Why? Because I realised that calculating 21000 (which we clearly need to do to calculate the sum of its digits) is just the same as summing a list of very big integers, which we did to solve Problem 13.

How is that? On the face of it calculating an exponent looks quite different to calculating a sum. But take a look at this line of thought:

2n = 2 x 2n-1
   = 2n-1 + 2n-1
   = (2 x 2n-2) + (2 x 2n-2)
   = (2n-2 + 2n-2) + (2n-2 + 2n-2)
   = ...

Bottom up duck. Picture credit: Glisglis, http://flickr.com/photos/glisglis/365003674/Or if you prefer bottom-up thinking (you are a duck, perhaps):

21 = 2
22 = 2 x 2 = 2 + 2
23 = 2 x (2 x 2) = (2 + 2) + (2 + 2)
24 = 2 x [2 x ( 2 x 2)] = [(2 + 2) + (2 + 2)] + [(2 + 2) + (2 + 2)]
25 = you get the idea

Thus we have a recursive definition for calculating a power of two in terms of the addition operation: just add together two of the previous power of two. Putting my Functional Programming Thinking cap on immediately brings to mind the Unfold method, which generates a sequence by applying a transformation to the previous element. Used in combination with my code from an earlier solution to sum up two numbers represented as digits, this should give us what we need.

In order to know how many terms of the sequence to Unfold to get us up to 21000 we'll use the trick of smuggling some additional information into the terms of the sequence using an anonymous type: in the following code the Power property in the anonymous type indicates which power of two is held in the Digits sequence. Digits is a number represented as a sequence of digits, least significant digit first.

The full code, as always, is available on Code Gallery.

[EulerProblem(16, Title = "What is the sum of the digits of the number 2^1000?")]
public class Problem16
{
    public long Solve()
    {  
        return new {Power = 1, Digits = (IEnumerable<int>) new int[] {2}}
            .Unfold((previous) => new { Power = previous.Power + 1, Digits = SumNumbers(previous.Digits, previous.Digits) })
            .SkipWhile(item => item.Power < 1000)
            .First() // this will be the term containing 2^1000
            .Digits // take the Digits property of the anonymous type
            .Sum(); // sum the digits of 2^1000
    }

    /// <summary>
    /// Calculate the sum of two numbers represented as a sequence of digits. Numbers should be given
    /// with least significant digits first
    /// </summary>
    /// <param name="number1"></param>
    /// <param name="number2"></param>
    /// <returns>The sum of the two numbers as a sequence of digits</returns>
    private IEnumerable<int> SumNumbers(IEnumerable<int> number1, IEnumerable<int> number2)
    {
        // put the numbers together in an array so that we can zip them easily
        var numbers = new[] {number1, number2};

        // process the entire list of numbers column by column
        // ie, process least significant digits of all numbers first, then the next column of digits, etc.
        return numbers
        .Zip()
            // work through each column, calculating which digit represents the sum for that column,
            // and what the carryover is
        .Aggregate(
            // initialise an annoymous data structure to hold our workings out as we
            // move column-by-column.
            // Digits holds the digits of the sum that we've calculated so far
            // CarryOver holds whatever has been carried over from the previous column
            new { Digits = Stack<int>.Empty, CarryOver = 0 },
            (previousResult, columnDigits) =>
            {
                var columnSum = columnDigits.Sum() + previousResult.CarryOver;
                var nextDigit = columnSum % 10;
                var carryOver = columnSum / 10;
                return new { Digits = previousResult.Digits.Push(nextDigit), CarryOver = carryOver };
            },
            // to complete the aggregation, we need to add the digits of the final carry-over
            // to the digits of the sum;
            // note that because completeSum.Digits is a stack, when it is enumerated
            // we get the items back in reverse order - ie most significant digit first; hence the call to Reverse()
            // to make the order of the digits in the sum we return consistent with the input digits
            (completeSum) => completeSum.CarryOver == 0 ? 
                completeSum.Digits.Reverse() 
                : completeSum.CarryOver.Digits().Concat(completeSum.Digits).Reverse());
    }
}

I also had to update the Zip method to fix a bug. Here's the updated version:

public static IEnumerable<IEnumerable<T>> Zip<T>(this IEnumerable<IEnumerable<T>> sequences)
{
    // get the enumerators for each sequence
    var enumerators = sequences.Select(sequence => sequence.GetEnumerator()).ToList();

    var activeEnumerators = new List<IEnumerator<T>>();
    var items = new List<T>();

    while (enumerators.Count > 0)
    {
        items.Clear();
        activeEnumerators.Clear();

        foreach (var enumerator in enumerators)
        {
            if (enumerator.MoveNext())
            {
                items.Add(enumerator.Current);
                activeEnumerators.Add(enumerator);
            }
        }

        if (items.Count > 0)
        {
            yield return items;
        }

        enumerators.Clear();
        enumerators.AddRange(activeEnumerators);
    }
}

Friday, 25 July 2008

Inadvertent MSDN Humour

I just came across this whilst looking up the documentation for the WCF <dns> config element on MSDN:

MSDN Humour 

I'm guessing that this is the work of an overzealous link-shortening script. The links actually point to www.microsoft.com and msdn.microsoft.com. I wonder how long it will take Microsoft to dehumourise this?

Thursday, 24 July 2008

We're hiring

Are you looking for a new outlet for your talent? Is your current role not stretching you enough? Then read on: I might have just the thing for you. Paragon Simulation, the company I work for, is looking to recruit smart developers with the right blend of mathematical, analytical, and people skills to power our next round of business modelling work - and a new product development.

The official vacancies page has more details. The headlines are:

  • The job is based at our offices in Halesowen, near Birmingham (in the UK), just off M5 junction 3 - Google shows you where.
  • We're offering up to £31k, or up to £43k for the Senior role - both roles with additional bonuses and benefits.
  • We're looking for people with a strong maths or stats background. If you've previously been involved in some kind of computer modelling, especially business modelling, you'll be ideal.
  • You'll be working on a variety of modelling projects (possibly involving simulation tools like Witness or Flexsim or even building bespoke modelling tools in C#/.Net), with opportunities to get involved in the development of our new product (still under wraps at the moment).

If you're interested in applying please send your CV to me (after de-obfuscating the following address): sam [dot] jack [snail-shell symbol traditionally associated with email addresses] paragonsimulation [dot] com. Please include "Simulation Job" in the subject line.

But before you do that you might like to know a bit more about the company, and what we do.

Our Work

Paragon sells its services to companies in a number of sectors, ranging from the Nuclear, through the Environmental, to the Financial and Public Sectors. You should see the case studies if you want to get a feel for the breadth of work that we do. As you can imagine, hopping between sectors like this presents a very interesting challenge to the people that work here: clients have often commented on how quickly we learn the intricacies of their unique business - but that's what it takes to create the insightful models they need.

You can read more in an earlier post about my experience on a couple of modelling projects.

Working for Paragon

I find Paragon a great company to work for. There aren't many of us (currently fewer than 20), but as a Company we have an influence beyond our size. All the organisations we work for are very much larger than ours, and we engage them at the highest level. Everybody on a project has to make a strong contribution in many different roles: analyst, developer and tester. We love to find better ways of working: if you have great ideas, and want to be heard, this is the place to be.

We have a relaxed working environment: open plan, no dress-code, flexible working hours, table football in the kitchen - that kind of thing. There's a good personal development program in place with regular reviews and a clearly laid out career path. We try to ensure that everybody has some scheduled "Personal Development Time" every month to work on something not directly related to their current projects - reading, learning, working on projects of their own. As part of this, the company makes it very easy to get hold of any new books we need for the job. Here's the stack on my desk at the moment (we do also have bookshelves, by the way - I just feel cleverer when I have a pile of books to back me up):

22072008

In terms of the software development environment, we're very much a Microsoft shop. We're fortunate to have the kind of environment which means that we can take up new technologies where ever we think there's a case for them. So we can lay claim to all the latest buzzwords like Visual Studio 2008, .Net 3.5, C#3.0, LINQ, LINQ-to-SQL, WPF, WCF, SQL Server 2005, VSTO, etc., etc. It goes without saying that we have all the staples like Version Control and Bug tracking in place. We also have our Build Server that does Continuous Integration (using CruiseControl.Net) and some automated testing (we favour MbUnit). And you might like to know that we're all Resharper fans round here.

For Simulation work, I've already mentioned that we use Witness and Flexsim, both Discrete Event Simulation packages. Witness is one of the most mature Simulation engines, and is very good at building models that are configurable at run time. It can be programmed in a language that has similarities to Basic. Flexsim is a relative newcomer. Its speciality is 3d visualisation of processes, so we use it when we really want to engage a customer in how their process will look when it is in operation.

Then there's spreadsheet modelling. We have been using Excel with VBA for these kinds of projects, but with some of our recent models hitting 120Mb in size, I think we're pushing the limits of what this combination of technologies can achieve. If you have experience in using other technologies (perhaps VSTO) for serious spreadsheet modelling, or other ideas about how we can manage such large beasts, we'd love to hear from you.

So what are you waiting for? Get your CV up to date, and get it to me.

Tuesday, 22 July 2008

What I do at work

When I launched this blog a couple of months ago, I made a vague promise that I would talk a bit about what I do at work. I know your repeat visits have all been in the hope of me keeping that promise. Right?

[Content temporarily removed]

To be revealed...

I can't say much about my most recent and current projects, because they have to do with a new product that Paragon is developing and keeping under wraps . What I think I am allowed to say is that it is to do with spreadsheet modelling and data management.

At the risk of allowing you to join the dots and deduce more than you should know, I'll mention that this project has given us an opportunity to build a client/server application with WCF; we've had lots of fun with LINQ-to-SQL; and we've had plenty of character-building experience with VSTO and managed Excel add-ins - all lubricated with plenty of C#3.0 goodness.

If any of this sounds more appealing to you than your current line of work, look out for my up-coming post on some positions we have open.

[Content temporarily removed]