Feed Sign in with OpenID OpenID

Simon Willison’s Weblog

No more usernames in URLs

This one could get very interesting. Microsoft have announced that an upcoming update to Internet Explorer will remove the ability to include usernames in URLs completely. This is in response to the growing problem of so called “phishing” scams, which use trick URLs to con important information such as passwords and credit card details out of unsuspecting browser users.

Phishing is big business. In this article on SecurityFocus, a loose transcript is provided of a talk by an FBI agent who explains how phishing is used by organised crime gangs in Eastern Europe:

This is bad enough and it’s also cruelly funny, but the scary part came in when Dave started talking about the other group behind the explosion of viruses and Trojans: Eastern European hackers, backed by organized crime, such as the Russian mafia. In other words, the professionals.

These people are after one thing: money. The easiest way to illegally acquire money now is through the use of online tools like Trojans, or through phishing: set up a fake Web site for PayPal or eBay or Amazon, and then convince the naive to enter their usernames, passwords, and credit card information. Viruses and spam also intersect in this nasty spiderweb. Viruses help spread Trojans, and Trojans are used to turn unsuspecting users’ computers into spam factories, or hosts for phishing expeditions, and thus furthering the spread of all the elements in this process: viruses, Trojans, spam, and phishing. It’s a vicious cycle, and unfortunately, it appears to be getting worse. The FBI is working as hard as it can, but the nations of Eastern Europe are somewhat powerless to solve the problem at this time.

IE is so susceptible to this kind of attack that it’s not even funny. In addition to the “invisible username” bug I covered last month, a recent discovery compounds the problem by allowing dangerous executable files to pose as safe file types when downloaded from the web. New Explorer hole could be devastating has the full details.

Microsoft’s solution is drastic to say the least. Passing the username as part of a URL has been part of the makeup of the internet since at least 1994, and the ability is baked in to a huge range of web client and server software. It’s described in RFC 23996. The feature is rarely used however, and the overall effect of its removal from IE is hard to judge. Off the top of my head I can think of only one site that uses it for legitimate reasons: FilePlanet, which incorporates it in to the site’s download queuing system (at least last time I checked).

There’s an interesting contrast to be made here between open and closed development methodologies. The Mozilla project has had a bug open on this issue for over two years, which has drawn over 170 comments with plenty of great ideas but no approved solution. Microsoft on the other hand have remained silent on the issue until (we can only assume) the bad publicity surrounding it forced them to act, at which point they announced a fix that appears to gly in the face of commonly accepted web standards—but does undoubtedly solve the problem. Of course, with no chance for user feedback prior to the decision it amounts to little less than a decree from God—which correlates directly to their inarguable domination of the browser market, at least in terms of market share.

Of course, the millions of IE users who decline to upgrade their browser will remain just as susceptible as they always were (unless they stop clicking links)—a fact for which we can hardly blame Microsoft. It does however mean that phishing will remain a lucrative scam for a long time to come.

Posted 30th January 2004 at 8:14 am

This is No more usernames in URLs by Simon Willison, posted on 30th January 2004.

View blog reactions

Next: I've sold out!

Previous: Cold War check point

23 comments

  1. A bad decision in my opinion. There are valid reasons for encoding username.password in a URL, and the upcoming change in IE will force us to either:
    • change our complete authentication scheme (whose main goal is to be as unintrusive to the user as possible)
    • drop IE as a supported browser for our product
    • something totally different
    What I don't understand is that MicroSoft or the Mozilla people didn't implement something like the warning Opera uses - which both covers the username/password and the %01 / %00 problems that are plaguing IE and Mozilla.

    Jens-Christian Fischer - 30th January 2004 09:36 - #

  2. To me, this rather suggests that there's something really broken in Internet explorer which makes this, apparentley trivial bug really hard to fix. Afterall, there are dozens of ways of fixing the more general problem of URL spoofing without disabling functionality. Rather than using the Opera-style dialog box (which I regard as unfriendly), I would be tempted to have the status bar display something like site:$hostname+$path username:$username password:$password. I even started writing a Mozilla patch to do just that - but soon found that my grasp of C++ was just good enough to make the browser crash everytime you hovered over a link, but not good enough to fix the problem. I guess maybe i should have another look sometime soon.

    My question is, after this, and the admission that Internet Explorer 7 won't break backward compatibility (i.e. won't come with improved support for CSS, DOM, etc., at least for text/html), why does anyone trust Microsoft to implement any standards correctly? Is there even any value in standards which the main player has shown a contemptuous disreagard for?

    jgraham - 30th January 2004 10:58 - #

  3. As much as I like to bash Microsoft occasionally myself, on this point I have to strongly disagree with you guys.

    This time, Microsoft are actually the good guys, *they* are following the standards. While the RFC2396 (btw.: you've made a typo there) cited by Simon does indeed allow the user:password@host scheme for URIs, RFC1738, clearly states that the URL scheme for HTTP is http://hos>:port/path?searchpart.

    Because more specific RFC override general ones, it is actually violating the standard to use the user:password scheme in HTTP-URIs. However, afaik that doesn't affect FTP-URIs, so the authentication scheme could still be used where it's useful

    As a side note: here in Germany, the two biggest Web-Hosters (Strato & 1&1), each of whom is hosting literally several millions of Websites, are offering so-called "@-domains" (as an Alternative to subdomains) based on the user:password@host-scheme. When MS implements this fix, they are clearly going to run into problems :)

    Thus, Microsoft ist not only complying with the HTTP-standard and protecting its users, it also kills useless marketing-crap like those "@-domains"

    Gerrit Kaiser - 30th January 2004 11:21 - #

  4. Erhm, RFC1738 certainly doesn't "override" RFC2396, which actually states that "it revises and replaces the generic definitions in RFC 1738 and RFC 1808.".

    BTW, Simon, any chance you could switch the anti-comment-spam redirects to something that doesn't break the back button?

    Morten Frederiksen - 30th January 2004 11:49 - #

  5. Gerrit - I've read RFC1738 what must be a dozen times and never noticed that! It actually says "No user name or password is allowed" (section 3.3).

    Marcus - 30th January 2004 11:54 - #

  6. Lol, they actually advise you -as you've said- to stop clicking links! The most effective step that you can take to help protect yourself from malicious hyperlinks is not to click them.

    Martijn - 30th January 2004 11:55 - #

  7. Morten - to quote the relevant portion of RFC2396: (emphasis mine)

    This document updates and merges "Uniform Resource Locators" [RFC1738] and "Relative Uniform Resource Locators" [RFC1808] in order to define a single, generic syntax for all URI. It excludes those portions of RFC 1738 that defined the specific syntax of individual URL schemes

    Admitted, "override" might not be the most suitable description. But the point remains: in my understanding, the username:password@host-scheme is not allowed in HTTP.

    Gerrit Kaiser - 30th January 2004 12:30 - #

  8. Morten, if you read a little further, RFC 2396 also says "[this RFC] excludes those portions of RFC 1738 that defined the specific syntax of individual URL schemes".

    (An HTTP URL is an HTTP URL, not a generic URL.)

    Fredrik Lundh - 30th January 2004 12:38 - #

  9. Thanks for the heads up. This means that anyone wanting to view my website won't be able to for too much longer. Fortunately, I have very few readers, and my website has not in the past lent itself well to permanent linking.

    Mark Humphrey - 30th January 2004 16:14 - #

  10. What? Your saying I can't access hax0red pr0n sites anymore with IE? That's it! I'm switching to Mozilla!

    huphtur - 30th January 2004 16:27 - #

  11. Mark: if you were referring to the site linked to from your signature you'll be fine. The new limitation applies only to URLs like http://username@domain.com/path - http://www.domain.com/~username will be unaffected as that's not using the username for authentication, just for viewing a particular user's public HTML directory.

    Simon Willison - 30th January 2004 16:37 - #

  12. I noticed in the titles on your abbreviations, URL = Universal Republic of Love?

    Brian - 30th January 2004 18:30 - #

  13. Yup - I got that one from Tim Bray :)

    Simon Willison - 30th January 2004 18:56 - #

  14. The worst part about removing support as a supposed "fix" is that it won't do a blasted thing to fix the problem. You can still hide valid addresses from IE using the %01 bug.

    Using Apache virtual hosts, ServerAlias directives and wildcard DNS, you can still redirect to a "phishing" page with little difficulty. It's laughably easy to grab portions of the URL preceding the %01 using a CGI script or any of the mod_(perl|python|ruby) language extensions.

    Tim Adamec - 30th January 2004 22:27 - #

  15. Well, it's about bloody time. Hopefully Microsoft stepping in and doing this will prompt other browser makers to step up and do the same. I hate to say this about people I've never met before, and know hardly anything about, but whoever decided to allow the username:password@site scheme was a complete and utter moron. How could you not see the huge, gigantic security flaws inherent in that?

    It's all very well and good to talk about valid uses, but I doubt my expericne is in any way different from the norm when I say that the amount of sites I've seen using the @ scheme legitimately is absolutely minuscule, whereas the abuse of it is far more widespread and far more important. Is anyone honestly saying that a few 'clever' uses of the @ url scheme outweigh unprepared users being tricked out of hundreds or thousands of dollars?

    Opera's scheme of handling it may be a bit better, but if you implemented this in most browsers, I can all but guarantee that most users would just hit ok and ignore the alert. Any other method of display will almost certainly be so passive that most people won't recognise it, and expecting everyone who uses the web to understand exactly how URLs work is ludicrous.

    So all that said, does anyone here have a better solution, that allows retaining the @ scheme while making sure 99.9% of people will never be fooled, using it to fake a different URL? Or is this just opportunistic Microsoft bashing? I dislike IE's lack of support for most standards too, but this is one that's just crying out to be ignored, rescinded, deleted and never spoken of again.

    Lach - 31st January 2004 00:49 - #

  16. So all that said, does anyone here have a better solution...

    The best solution involves 2 steps: First, as jgraham said above, display the following in the status bar when you hover over a link:

    site:http://www.myevilsite.com/evil/page - username:www.microsoft.com - password:mypassword

    ...in the status bar when you hover over a link. Secondly, if you then do not show the username and password at all in the address bar, then the only people who could possibly fall for this would be people who don't look in the address bar at all (and there's nothing that can be done for them).

    Also, as Tim Adamec said above, removing support for usernames and passwords may not even solve the problem at all if this is possible:

    http://www.microsoft.com%00.myevilsite.com

    Tom Pike - 31st January 2004 09:09 - #

  17. Why couldn't Microsoft do something similar to the effect you get when entering a URL with certain characters (like spaces) in it? As soon as you press enter (or in the case of a link, as soon as you click it), the URL is rewritten to replace each space with a %20.

    If this route was taken, security for legitimate username:password links would actually be improved in a way similar to HTTP POST data; the data is retained for each subsequent link, but with the added benefit of not leaving a trail of passwords in your history. At the same time, Phishing schemes would be reined in because even if you tricked the user into clicking the username:password link, a quick look at the address bar would reveal the true URL.

    Charles - 31st January 2004 09:11 - #

  18. Well, I guess Tom beat me to the punch by 2 minutes on my last post :)

    Charles - 31st January 2004 09:14 - #

  19. I wonder how do you know for sure that there was no user feedback about the decision. It seems that you make the same wrong accusations you made before about Microsoft. Well of course you are not accusing them, you mostly bash, but the point is that, you have no idea whether user feedback is involved or not. More likely they did have user feedback, it is just that they don't like to take the feedback from anybody, cause then you will have lots of Microsoft bashers going to the site and posting stupid comments. They will make all sorts of stupid bashing and at the end Microsoft will not be able to improve the product. Also keep it in mind that anybody even suggesting removing support for IE in its product is not a serious business anyway. Nobody will care about that sort of mindset and stupidity. And actually I bet the only time that person will suggest that will be here for ranting and bashing, otherwise he will be out of job pretty soon. The user name scheme was useful at the begining of the web. So I don't see a big problem. If you remember it, everything was sort of safe those times. You didn't have to worry about anything. But it shows that when people initially build these technologies they didn't think about malicious people ruining everything. IP design is one of them, which allows DOS attacks.

    Serge - 31st January 2004 09:19 - #

  20. Also, as Tim Adamec said above, removing support for usernames and passwords may not even solve the problem at all if this is possible:

    http://www.microsoft.com%00.myevilsite.com

    Or even if it's not. If we're talking dumb users here then why does anyone expect them to notice that HTTP://WWW.MICR0S0FT.COM isn't the same as HTTP://WWW.MICROSOFT.COM? Alternativly, how many users won't realise that http://www.microsoft.virusalert.com wasn't an official microsoft site.

    This type of change throws out the baby, and puts the bathwater to bed

    jgraham - 31st January 2004 10:47 - #

  21. That should read:

    Alternatively, how many users will realise that http://www.microsoft.virusalert.com isn't an official microsoft site?

    jgraham - 31st January 2004 11:01 - #

  22. I don't know what RFC you guys have been reading, but according to RFC1738 it states:

    3.1 Common Internet Scheme Syntax

    //<user>:<password>@<host>:<port>/<url-path>

    Thus, Microsoft is not conforming to the defined standards, and are breaking compatability in a very large way by implementing this in their browser. And this, just as I started to have some hope for IE.

    Tukon - 4th February 2004 01:07 - #

  23. The only document that I could find so far on the HTTP URL scheme versus the "generic" URL scheme (described in the RFCs mentioned above) is the BNF for specific URL schemes.
    What do you think? Does it mean the user:pwd format in HTTP URLs isn't standard?

    Dumky - 7th February 2004 23:25 - #

Comments are closed.

Previously hosted at http://simon.incutio.com/archive/2004/01/30/noMoreUsernames

A django site