If you are looking for Cheap Windows Web Hosting but in a solid web hosting providers then Both Godaddy and Hostgator provide windows hosting. at Godaddy Both Windows Hosting and Linux Hosting are same price all you have to do it choice which you want while filling online forms. But at Hostgator Windows Hosting is expensive than Linux Hosting. The Basic personal plan cost you $8 per month if you pay by month and $7 per month if you pay annually. There are another couple of good Windows Hosting providers like ASP.net but even they are expensive.
Unlike with Linux Hosting where most of the things are free and open source, You need to buy license every software in Windows Hosting. So it's a costly service for web hosting providers. Server license, SQL license and other small things. Microsoft never give anything for free. But Some developers need Windows Hosting for .net frame based web sites and web applications. So they have to buy Windows web hosting even thought it's expensive. You need .net frame work and IIS web server to hosting these window based applications.
One of the biggest drawbacks in Windows Hosting is you can't use cPanel since it's written in PHP and made for Linux systems and My SQL. Also If you are running IIS then usually you can't run Apache or any other Linux / PHP based HTTP servers. Also Windows Hosting use My SQL or Oracle in high-end operations. No My SQL. So most common web applications like Wordpress, Joomla or web forum scripts like VBulletin, SMF or myBB can't be run in windows Hosting because these are written in PHP. You need Linux Hosting for these.
For now when we talk about cheapest web hosting and Windows based hosting I think Godaddy is rank #1. They offer same price for both Windows and Linux packages and that's nice. There are lots of web hosting providers offer cheap Windows hosting but I'm not sure how they keep running their business with price under $5. Probably using resellers and I always prefer to use web hosting providers who use their own Data centers or dedicated servers. You don't get too many problems this way. Reason for most of cheap web hosting problems is these are run by providers who only has reseller accounts and they don't have a control over their servers. So they have to wait for server owners sort out things when there is problem with servers. This usually take long and make clients angry.