Simon Willison’s Weblog

This was definitely a good move on the part of the Movable Type developers, though I wish it had been implemented in such a way that the URL could be seen without first following the link.

Jacques Distler has patched his installation so that the URL appears as part of the query string, and others have released patches and plugins to turn off the URL redirect feature altogether.

Also, I haven't tested 2.661 too much yet, but I believe that the URL redirect only applies to URLs in the URL field, and not to URLs in the comment field, so this may not be a complete solution for victims of comment spam that also allow HTML in comments. (Please correct me if I'm wrong about this.)

jacob - 21st January 2004 20:28 - #

This kinda stinks for those following the directions posted on my site. I understand how well this will help fight spammers, but PageRank does get good milage out of legitimate posts.

Jemal - 21st January 2004 22:28 - #

Congrats Simon! It seems you are responsible for improving thousands of weblogs! Woo!

Because I'm always thinking of the next step, have you ever given any thought to whitelists - a list of validated weblogs that can bypass the redirect system? Yes this could become another security problem, but I think if it is properly designed and maintained a whitelist solution could turn things back around for weblog comments.

But still, I'm happy to see your ideas and efforts pay off for the greater good! :D

Stephen - 21st January 2004 23:36 - #

While an effective solution to the problem, I suppose, provided that it works, killing pagerank does in effect kill Google, it's algorithm as it applies to sites linked to as such, and therefore destroy some of the social architecture involved.

The idea of authority through linking and those links being thought of as votes (greatly simplifying pagerank here) is what allowed Google to provide us with something so much better than Alta-Vista (remember how bad that became?).

Maybe it's just me, but by providing links that have no pagerank you are - in one sense - giving in to the spammers. Noteworthy though is that none of this will really matter when some worth adversary appears for the great SE that is google.

Mike - 21st January 2004 23:47 - #

I think this will just make spammers post links in the body of their spam (maybe on small things like "."). Joi Ito is getting people leaving real comments just for the GoogleJuice of name links, and now they know about it I don't think they'll stop.

While this is a good start, making the destination URL invisible is kinda bad. It seems like links aren't a 6 Apart speciality; I wish they'd make their documentation and forum links a little less scary :/ eg:
http://www.movabletype.org/docs/mtmanual_tags.html #archive%20templates%20and%20master%20archive%20in dex http://www.movabletype.org/support/index.php?act=S T&f=12&t=33797&s=715e46a3f8ce5569ccabc41cc59ad5c9

I wonder how comments getting stripped of GoogleJuice will affect weblogs' rankings? I imagine that this will lead to a loss of GoogleJuice for weblogs as well as spammers. Is that a bad or a good thing?

oli - 22nd January 2004 04:23 - #

Apologies for being the voice of dissent, but I was terribly, terribly disappointed that Six Apart chose this as one of their anti-spam measures. I was never very keen on it when your blog was the only one doing it, Simon, and now I find that a load of the MT blogs I regularly read are also doing it! It may be a good solution from the point of a developer, or the owner of the blog being spammed, but it punishes the user by delaying page loads, obscuring the URL that you're following and potentially (and rather essentially) breaking the 'back' button. Interestingly, overnight, at least three blogs I read which had upgraded have already removed this feature.

Phil Wilson - 22nd January 2004 09:02 - #

Because I'm always thinking of the next step, have you ever given any thought to whitelists - a list of validated weblogs that can bypass the redirect system?

I like this idea. I think I might have a go at implementing it for my blog. I really dislike the anti-PageRank karma that the redirects provide. I'd prefer that a site's readers get their google juice if they aren't spammers. A whitelist seems like a good place to start.

Scott Johnson - 22nd January 2004 12:32 - #

I think this will just make spammers post links in the body of their spam (maybe on small things like ".").

If you'll notice the way links work on Simon's blog, you'll see that all links are redirects. MT doesn't quite do this yet, but it's definitely coming soon.

Scott Johnson - 22nd January 2004 12:35 - #

I think there is also another way of killing pagerank-passing without doing redirects: Add random parameters to the url

How it works? Simple. Google assigns pagerank on a per page basis where a page is a always a full url, that means /index.html and /index.html?foo=bar are different. The PR of one is in no relation to the PR of the other one. So when we add parameters to urls in comments the PR flows to a "whole new" page. No ppl. might ask: Isn't it worse, since spammers will be included in google with duplicate pages? Well not when we choose the parameters wisely. We know that google doesn't like session-ids and it doesn't like multiple parameters (I think currently two are ok, three are too much), so now we know how to craft a url that will apall Google, won't pass PR to the "actual" page and has little usability problems (we can name the parameters such that humans at least now which part is fake).

An Example:A link to - www.domain.tld - would become:

www.domain.tld?followingparameteres=fake&please=re move&id=1234567890ABCDEF1234567890ABCDEF12

Sencer - 22nd January 2004 13:24 - #

Apologies if I'm completely off-track here, but...

If you'll notice the way links work on Simon's blog, you'll see that all links are redirects. MT doesn't quite do this yet, but it's definitely coming soon.

I realise how links work here, and yes it is strange that MT2.66 only goes half the way - it won't take spammers too long to realise. However I'd be interested if comment link redirection became a default for MT, although this seems the logical step. While I can see the reason behind it in fighting spam, it seriously breaks Google. If enough websites do it Google will most likely start taking redirection into account, and you're back where you started (I don't know how possible that is though :-)

I'm actually a little surprised in the implementation in MT2.66; author name link redirection the default, there's no conf file override, and the URLs are just ugly (unlike Simon's). I can't see where the link goes, and then there's that ugly redirect page in the middle. I imagine it will also mess up the referrer log entries for any visitors arriving via redirect (correct me if I'm wrong-I can't see any evidence below). I wish they'd copied what Simon had done a little more accurately ;-)

I imagine MT3's comment registration (plus the other anti-spam features in 2.66) will be an adequate alternative to redirects for most people (a changed comments script name and MT-Blacklist is still working flawlessly for me ;-) It seems more logical than killing the link tag. Am I missing the point?

I'd prefer that a site's readers get their google juice if they aren't spammers

My feelings exactly. It's just knowing who the spammer is that's the problem huh ;-)

oli - 22nd January 2004 14:11 - #

Perhaps the blogging software should have a method that the owner of the site can use to approve posts. The posts would still be readable immediately, but methods to prevent Google from adding page rank would be used until the post is approved. Then contributers would get rewarded, after a delay.

David Robarts - 31st January 2004 21:44 - #