A little nerd humor for the day. I got asked by some executives to “prove” that what we were doing was working.  That wasn’t so hard.  The next question was “how would you show return on investment for additional budget.”  I thought on it for a few moments, and wrote the following on the board.  The engineers got it.

That’s a screen grab from Visual Studio, which is now my desktop image.

I later shared this with some of my developer buddies.  What ensued is also pretty funny:

Friend 1: “Shouldn’t that be a property (e.g., Developers.AreHappy)?”

Friend 2: “It could be a complex function returning different values depending on the level of Developer.Fickle property."

Friend 1: “The FxCops will come after you for violating policy CA1024.  To Friend 2’s point, you could get a reprieve if making devs happy is a time-intensive operation. If that’s the case, however, you should review your entire architecture and implementation.”

I love my nerdy friends.

I have been asked by a few people how sites like WP7AppList get their data.  The Windows Phone Marketplace, which you access on your PC via Zune software, uses XML to get data over the wire.  I wanted to share a couple of code snippets which might help an erstwhile data junkie on their way.  This code works.  It may not be the most elegant solution, but it works, and I wanted to share it with others in case they wanted to see how to parse the XML, or how to write LINQ queries against it.

Caveat – this is a geek enthusiast post.  I used Fiddler to figure out how to parse the XML.  This was something I did over Christmas break to give me a project about which I could be excited, and learn some more about parsing XML with LINQ.  I also wanted to do some large database stuff, and this crawler throws off a ton of data.  I did not use an proprietary knowledge about how our backend systems are working.  This is all done against the public XML feeds.

First up, we are going to need to create some data structures to catch all of the inbound data.  You can use anonymous types with LINQ, but I liked having a measure of control, and having the ability to handle null values and potential errors in the feed.

public class ZestAppData
{
    public string Title { get; set; }
    public string Id { get; set; }
    public DateTime ReleaseDate { get; set; }
    public DateTime Updated { get; set; }
    public string Version { get; set; }
    public string ShortDescription { get; set; }
    public decimal AverageUserRating { get; set; }
    public int UserRatingCount { get; set; }
    public string ImageId { get; set; }

    public IList<ZestCategory> Categories = new List<ZestCategory>();
    public IList<ZestPublisher> Publisher = new List<ZestPublisher>();
    public IList<ZestOffer> Offers = new List<ZestOffer>();
}

public class ZestCategory
{
    public string Id { get; set; }
    public string IsRoot { get; set; }
    public string Title { get; set; }
}

public class ZestOffer
{
    public string OfferId { get; set; }
    public string MediaInstanceId { get; set; }
    public decimal Price { get; set; }
    public string PriceCurrencyCode { get; set; }
    public string LicenseRight { get; set; }
    public List<string> PaymentType = new List<string>();
}

public class ZestPublisher
{
    public string Id { get; set; }
    public string Name { get; set; }
}

You are also going to want to have a bunch of variables defined for the URLs where the XML is coming from, the XML namespaces, etc:

const string BaseAppsUrl = "http://catalog.zune.net";
const string BaseImageUrl = "http://image.catalog.zune.net";

const string ZestVersion = "/v3.2/";
const string ZestImageVersion = "/v3.0/";

const string BaseApps = "apps/";
const string BaseImage = "image/";

const string BaseAppsResource = "?clientType=WinMobile%207.0&store=Zest&orderby=downloadRank";
const string BaseCommentsResource = "/reviews/?store=Zest&chunkSize=10";
const string BaseImageResource = "?width=240&height=240";

ZestCrawlEntities ZestCrawlContext;

XNamespace ns = "http://www.w3.org/2005/Atom";
XNamespace zestns = "http://schemas.zune.net/catalog/apps/2008/02";

public string LangCode = "en-us"; //setting the default value

public List<string> ValidLangCodes = new List<string>(
    new string[] {  "en-us", "en-gb", "de-de",
                    "fr-fr", "es-es", "it-it",
                    "en-au", "de-at", "fr-be",
                    "fr-ca", "en-ca", "en-hk",
                    "en-in", "en-ie", "es-mx",
                    "en-nz", "en-sg", "de-ch",
                    "fr-ch" });

public string AppAfterMarkerUrl { get; set; }
public bool HasMoreApps = true;
public string AppsResponseString { get; set; }
public XElement ReturnedAppsXml;

Have a look at the ValidLangCodes list.  that’s the coding we have on the URLs for country specific data.  So if you want to get the data from Mexico, us “es-mx.”  The first two letters are the language code, and the second two are the country code.  If an app is listed in the feed, it is active.  The list returned is ordered, meaning the first app is ranked #1.  I am pulling the ALL APPs list, which is the orderby clause on the BaseAppsResource.

The ZextCrawlContext is the ADO.NET DB model.  Create your own and stuff the data however you want.

Now that we have the code segments, you are going to need a way to get the XML from MSFT servers.

public void GetAppsResponse()
{
    string FullUrl;
    bool done = false;

    if (!String.IsNullOrEmpty(AppAfterMarkerUrl))
    {
        FullUrl = AppAfterMarkerUrl;
    }
    else
    {
        FullUrl = BaseAppsUrl + ZestVersion + LangCode + "/"
            + BaseApps + BaseAppsResource;
    }

    while (!done)
    {
        try
        {
            var request = WebRequest.Create(FullUrl) as HttpWebRequest;
            request.KeepAlive = false;

            var response = request.GetResponse() as HttpWebResponse;

            if (request.HaveResponse == true && response != null)
            {
                var reader = new StreamReader(response.GetResponseStream());
                ReturnedAppsXml = XElement.Parse(reader.ReadToEnd());
                done = true;
            }
        }
        catch
        {
            Console.WriteLine("yeah, your connection was likely aborted");
            done = false;
        }
    }
}

Now comes the fun part.  Remember, the XML is coming over the wire, and it comes 100 elements at a time.  So you have to parse the stream, stuff them somewhere and get the next stream.  Included in the XML returned is the token for how you request the next bit of XML. (note, yes I know I am using RegEx where I could be using String.Replace; also sorry about the wonky formatting, but my blog has width issues)

public IEnumerable<ZestAppData> GetAppEntries()
{
    //first we have to parse the feed which came back
    IEnumerable<ZestAppData> entries =
        from e in ReturnedAppsXml.Elements(ns + "entry")
        select new ZestAppData
        {

            Title = e.Element(ns + "title").Value,

            Id = Regex.Replace(e.Element(ns + "id").Value, "(urn:uuid:)(.)", "$2"),

            ReleaseDate = DateTime.Parse(e.Element(zestns + "releaseDate").Value),

            Updated = DateTime.Parse(e.Element(ns + "updated").Value),

            ShortDescription = e.Element(zestns + "shortDescription") == null
                ? "" : e.Element(zestns + "shortDescription").Value,

            AverageUserRating = decimal.Parse(e.Element(zestns + "averageUserRating").Value),

            UserRatingCount = int.Parse(e.Element(zestns + "userRatingCount").Value),

            Version = e.Element(zestns + "version").Value,

            ImageId = Regex.Replace(e.Element(zestns + "image").Element(zestns + "id").Value, "(urn:uuid:)(.)", "$2"),

            Categories = (
                from category in e.Elements(zestns + "categories").Elements(zestns + "category")
                select new ZestCategory
                {
                    Id = category.Element(zestns + "id").Value,
                    Title = category.Element(zestns + "title").Value,
                    IsRoot = category.Element(zestns + "isRoot").Value
                }).ToList(),

            Publisher = (
                from publisher in e.Elements(zestns + "publisher")
                select new ZestPublisher
                {
                    Id = publisher.Element(zestns + "id").Value,
                    Name = publisher.Element(zestns + "name").Value
                }).ToList(),

            Offers = (
                from offer in e.Elements(zestns + "offers").Elements(zestns + "offer")
                select new ZestOffer
                {
                    OfferId = offer.Element(zestns + "offerId").Value,
                    MediaInstanceId = offer.Element(zestns + "mediaInstanceId").Value,
                    Price = decimal.Parse(offer.Element(zestns + "price").Value),
                    PriceCurrencyCode = offer.Element(zestns + "priceCurrencyCode").Value,
                    LicenseRight = offer.Element(zestns + "licenseRight").Value,
                    PaymentType = (
                        from paymenttype in offer.Elements(zestns + "paymentTypes").Elements()
                        select paymenttype.Value).ToList()
                }).ToList()
        };

    //now I need to get the AfterMarkerUrl from the XML feed
    var afterMarker =
        from e in ReturnedAppsXml.Elements(ns + "link")
        where e.Attribute("rel").Value == "next"
        select (string)e.Attribute("href").Value;

    if (afterMarker.Count() > 0)
    {
        AppAfterMarkerUrl = BaseAppsUrl + afterMarker.Single();
    }
    else
    {
        HasMoreApps = false;
    }

    return entries;
}

Now you have all the data you need to crawl the marketplace whenever you want.  The LINQ stuff is really, really fast.  Crawling the marketplaces can be a bit slow.  I crawl each one individually when my code runs, and I store app lists for each of the markets.

One of the mistakes I made was having ZestAppData.Udpated be a DateTime and not a Date.  I only crawl once per day, so I don’t need all the extra data.  The Zest feeds update daily, I think every couple of hours.

We’ve made no secret of the fact that we believe we have a great platform for developers.  The canvas we have delivered is enabling amazing creativity to flourish.  Some developers, however, have taken a wait and see approach to developing for Windows Phone.  For those who have taken the plunge, some developers are making more money on Windows Phone 7 than Android, even though we have less handsets in market.

Yes, this is a data point of one, but it’s a public data point about which we can talk.  There are many more stories like this on the horizon, but this is the first one that is verified by a third party.

Fruit Ninja has made 7x more money on Windows Phone 7 than Android.

That is the claim of the article.  Why is this?  There are many potential reasons, and I don’t want to venture into what could quickly devolve into a discussion based on opinions.  What we can say is that the data shows that the Windows Phone Marketplace works for developers.  It shows that customers of Windows Phone are spending money to buy apps.  It shows that that Windows Phone has great potential.

UPDATE (03/09/11 11:20A): The Xylogic data is what it is.  I won’t vouch for it, so short of taking this post down, I can only say that I cannot verify their Android data.  Is it a fair assumption that any overstatement/understatement they make for Windows Phone is same for Android?  Maybe.  I am not going to surface Fruit Ninja numbers as that’s not my place.  Unless Xylogic made a mistake in our favor (overstating $$ on Windows Phone) and against Android (understating $$ on Android), the ratio will hold.

Besides, and here’s the point that everyone seems to be missing – we have sold way fewer phones and Android.  Way.  Even if we were at parity on revenues, which platform is more appealing?  Don’t let the source of the data get in the way of the undeniable fact: the dynamics and structure of the marketplace on Android creates a challenge for developers to make money.  When Angry Birds can’t make money with per unit sales, that’s a sign. [END UPDATE]

We are still learning, and will continue to share what we learn with our developer community about what apps are working, how to market their apps, get noticed, etc.  However, the fact remains that for this developer, Windows Phone 7 has been well worth their time.

The smart money is to bet on the train that is leaving the station, not the one that’s gone.  Again, public data point of one, but this train is picking up steam.  As CNet noted, our mobile fortunes are tied to app developers.  Getting developers info like this is incredibly important to demonstrating that Windows Phone is a viable, credible, profitable platform for developers.  We will turn ourselves inside out to get developers whatever they need to be successful.

Still on the fence?  If you are an Android developer, and looking for a switch, reach out to us.  For developers committed to building on Windows Phone, we will take care of you.  Every developer matters.

Today, we posted a new article up at the Windows Phone Developer Blog about how developers are having great results using the Trial API and Microsoft Mobile Ad control.  The most astonishing piece of data for me what that more than half of the Trial API apps convert to a sale within one day, and most of those within 2 hours of the download.

While the stories of existing devs having great success is music to my ears, what’s really exciting for me is the expansion of the geographies which can support developers.  We have just announced our Global Publishing Program, which allows us to extend the ability to be a published Windows Phone 7 developer to countries where we do not yet have Marketplace.

Publishers will sign up to support an entire region, allowing developers to submit applications to Marketplace through the publisher.  We have launched with our first partner, Yalla Apps, a new publisher supporting the Middle East and Africa region.  So what new countries can we support?  Here’s the list:

Afghanistan, Algeria, Angola, Bahrain, Benin, Botswana, Burkina Faso, Cameroon, Cape Verde, Central African Republic, Chad, Comoros, Congo, Côte d’Ivoire, Cyprus, Democratic Republic of Congo, Djibouti, Egypt, Equatorial Guinea, Ethiopia, French Polynesia, Gabon, Gambia, The, Ghana, Guinea, Guinea-Bissau, Iraq, Israel, Jordan, Kenya, Kuwait, Lebanon, Lesotho, Liberia, Libya, Madagascar, Malawi, Mali, Mauritania, Mauritius, Mayotte, Morocco, Mozambique, Namibia, New Caledonia, Niger, Nigeria, Oman, Pakistan, Palestinian Authority, Qatar, Reunion, Rwanda, Saudi Arabia, Senegal, Seychelles, Sierra Leone, Somalia, South Africa, Swaziland, Tanzania, Togo, Tunisia, Turkey, Uganda, United Arab Emirates, Yemen, Zambia, Zimbabwe

Turkey, Saudi Arabia, Pakistan, Israel & Egypt were our top 5 countries in terms of tools downloads from that list, representing multiple tens of thousands of developers.  Now each and every one of them can start publishing their amazing apps and games for Windows Phone.  This also means that the 3 developers in Somalia, 2 in Equatorial Guinea and Swaziland, and the 1 developer in Central African Republic, who have downloaded our tools, well, we now have a solution for you.  In fact, if you are one of the 8 developers in one of those countries, reach out to me – I’d love to hear your story.

More country support is coming.  We are listening to each and every one of you and working like mad to ensure that every developer who wants to build for Windows Phone 7 can.  Every developer matters.

Last week when I read Jamie’s excoriation of the Blackberry Playbook dev platform (so many people forwarded it to me), it made me sad.  Sad because a developer was hitting friction on a platform on which he clearly wanted to develop apps.

Tyler Lessard from RIM responded to this open air criticism, and I give him points for his embracing this publicly.  I look forwarded to meeting Tyler at some point in the future.

There are some in the community that said Jamie was bitching and moaning.  He was.  They say he gave up too easily.  Perhaps.  This issue does elucidate a more pressing issue, which is the plethora of choices available to the discerning developer, and the decreasing ability to get it wrong if you are a platform provider.  Further, with Twitter and blogs being so public – they are forever etched in stone people! (i.e. the search index) – it’s hard for a platform provider to not to feel like they are under constant and heavy fire.

Yet, the very tools which quickly bring scorn and ire to some can provide for a happy ending to others.  In just under 4 hours, I was able to reach out to someone in another country, whom I have never met, and had never heard of prior to last week, and prompt him to action.

Even if we build the platform one developer at a time, that’s one more developer than we had last week.  Every developer matters.  Every. Single. One.  I effing love my job.

It’s not often I find myself kerfuffled, but that’s what happened when I was permitted to be a guest to discuss Windows Phone 7 on Windows Weekly, hosted by Leo Laporte and Paul Thurrott.  I’ve been a long time listener of several of the TWiT network podcasts, and fondly remember staying up late in grad school to catch Dev Null on “The Site” on MSNBC.  It was a lively conversation, and I have received many emails from people I don’t know with their thoughts on the show and WP7.  I love hearing from the community, and really look forward to meeting more developers building cool apps and games.

We also sponsored a couple of .Net Rocks shows with Carl and Richard (show 1) so that we could talk about the tools release, but more importantly so that we could enable 6 developers to talk about what they are building for Windows Phone 7.  We really wanted to give a microphone to devs doing cool stuff; specifically guys who are working without the aid of PR firms and other means to get the opportunity to promote themselves.  We know that there are going to be many apps built for Windows Phone 7 by small teams, and we really want to help get them noticed.

Stop the presses!  I’ve got some news.  The great team at GDGT is coming to Seattle, and they are having a bit of a party and meetup.  There should be a great deal of nerdosity going on there, which is awesome in its own right.  This will be further accentuated by the fact that the Windows Phone 7 team will be there to show off some devices.

Under normal circumstances, that would be pretty cool.  However, we’re not in the “pretty cool” business here.  I think I have made it clear, we are in the epic shit business.  Apparently, the nerdosity is expected to flow especially high at this event, and the request was made for the developer platform team for Windows Phone 7 to make an appearance.  Challenge thrown down.  Challenge accepted.  Challenge contorted into something all together different.

Since announcing the email alias to which you can send your device requests (about 2 weeks ago), we’ve been blown away by the response.  We’re still making our way through those requests, getting them into a CRM system and assigning them to our field evangelists.  That’s a tale of logistical complexity for another day.  However, the whole point of having preview devices is to get them into the hands of developers, right?  So we have two things to share.

First, if you are a developer in the area, and want to come to the event, bring your laptops with your Windows Phone 7 XAPs.  We will have phones on site to which you can deploy your code.  You can see it running on a real Windows Phone 7.  You get to share it with the crowd.  That’s not bad, but certainly not epic.  No, no.  Epic is in reserve for those true hackers who show up with the goods.  For those who have got the best XAPs, we are bringing a couple of phones with us, and some developer agreements to sign.  That’s right…the “A” gamers will be walking out with phones on which to continue their coding.

Are you game?  Spread the word, and see you there.

In the last couple of days since Andy’s announcement that all employees would get free Windows Phone 7s, we’ve seen a lot of buzz about the offer, including some around why we were doing it.

So why did we do it?  Put simply, we did it for the employees.  If you speak to any software developer or hacker, they will tell you that they love writing code.  They love solving problems.  They love creating things.  They love sharing their work with their friends and peers.  They love the satisfaction of seeing something work on a screen.  Unfortunately, when you work for a company, most of them generally have pretty stringent rules about moonlighting, and the ownership of IP.  For software companies, this usually includes code and side projects.  Microsoft is no different.

With this new mobile app era upon us, the Windows Phone team felt it important that we enable our own employees to participate.  There are so many talented, technical people at Microsoft.  It’s not just professional software developers…many Microsofties love coding, and they work on projects in their spare time.

We believe most Microsofties want to write apps for Windows Phone 7.  Before last week, and the changes announced in our policies, they couldn’t.  They would not have been able to profit from those projects.  We made these changes specifically to address the desire of our own employees to express themselves in code in the app marketplace.  A funny thing happens when you remove friction and barriers among a group of creative people.  I can’t wait to see what they create.

The internal response has been overwhelming.  I can’t count the number of languages in which we heard “thank you” while at our global sales summit last week.

I cannot believe I have only been working on the Windows Phone 7 team for three months.  So much has happened, it feels like a lifetime.  What’s been most impressive to me has been the pace of the engineering team.  Considering how many technologies from across the company we are bringing to WP7, it’s absolutely amazing what they are doing.

This week, during Andy Lees’s keynote at the Microsoft Worldwide Partner Conference, we announced that the Windows Phone Developer Tools have gone BETA.  Beta means that we have a near final version of the tools for building apps and games for Windows Phone 7.

I wanted to share with the community just how excited the developer community has been.  As a data guy, I like numbers because they tend not to lie.  Since the Mix10 release in March of the first CTP tools (and then the refresh in April), we have seen over 200K downloads of the Windows Phone Developer Tools.  That was before we shipped a Beta, and certainly before we had 5M devices in market to fuel demand.  At some point we’ll release the Beta download numbers, but based on the number of people registered for the live training course next week (hint: it’s got a comma in it), there’s a lot of people who want to build apps for Windows Phone 7.

For me, that’s a pretty solid indicator of the excitement levels, how much our developer community is anticipating getting involved with WP7, and that it is time to get serious about building apps and games, or building out your business, for Windows Phone 7.  The opportunities are plenty, even if you don’t want to build apps.  We showed one of our partners, Preemptive Solutions, on stage at WPC showing how they are building analytics tools so that app developers can build better apps.  Get creative, and start building.  Stay nerdy my friends.

There was a story which surfaced yesterday about vulnerabilities exposed in a wide swath of Android apps.  SMobile Systems conducted research in the Android app space and found that some 20% of the apps allow third-party apps to gain access to sensitive or private information.

It would be easy to dismiss this article by pointing out that by simply downloading an app, a customer is making a explicit allowance to an app to access data on the phone.  What I found troubling about the findings wasn’t necessarily the point about access to the data, but rather that, 5% of the apps surveyed could make calls and 2% could send premium SMS messages.  Talk about a surprise cell phone bill.

The key word in the above statements, however, is “could.’”  Yes, customers make the decision to download those apps, but they have no way of knowing with certainty what those apps are doing behind the scenes.  UPDATE: Ben points out below that the customer us warned of all the APIs used, which is true, but they aren’t told *how* they are used.

Further, because of the multi-tasking architecture of Android, the apps have the potential to be doing a bunch of bad things in the background when the phone is not in use.

Google has been quick to point out that the architecture of Android would limit what actual damage one of these apps could do, but that’s really not the point.  What is being lost in this discussion is that there is no curation of the Android marketplace.  For all the grumbling and grousing about the Apple AppStore, their review process would likely catch these abuses.  There is no such level of certification for the Android marketplace.  Customers don’t want to think about needing anti-spyware software for their phone, as the article implies is one solution for Android.

The Windows Phone Marketplace certainly believes in the curation model, and we have placed user security as a top priority.  This is one of the main reasons that we have our app certification process, and why (UDPATE: “at least in version 1”) apps are run in sandboxes, with no access to any data other than its own isolated storage, or the ability to communicate with other apps.  UPDATE: The goal is to ensure that absolute best customer experience when using their phone.

UDPATE 6/25/10

I’m not one who believes in conspiracy theories or anything, but I do find the timing of this announcement from Google that they can remotely wipe apps from phones a bit curious.  I have to go do some digging, but I’d be interested to know if the Android developer agreement has specificity about what would constitute grounds for a remote wipe.