The Isuzu Experiment



Update (9/5): Please don’t do this: vandalizing the site is not a good way to test it. If you want to test Wikipedia, please do it non-destructively.

Joi Ito points to an ongoing discussion regarding the authority of wikipedia as a source of information and knowledge. The discussion was prompted by an article in the Syracuse Post-Standard that suggests, in part, that wikipedia “take[s] the idea of open source one step too far” by allowing the user to make corrections.

The article has been correctly ridiculed by many, including Mike at Techdirt. In a later posting, he suggests an experiment: why not go to a certain page, insert something provably incorrect, and see how long it lasts.

No matter which side of the debate you find yourself on, this sounds like an interesting experiment. So, I have made not one, but 13 changes to the wikipedia site. I will leave them there for a bit (probably two weeks) to see how quickly they get cleaned up. I’ll report the results here, and repair any damage I’ve done after the period is complete. My hypothesis is that most of the errors will remain intact.

Does that invalidate Wikipedia? Certainly not! If anything, the general correctness and extent of Wikipedia is a tribute to humankind. It suggests the Kropotkin may be right: that the “survival of the fittest” requires that the fittest cooperate. It means that there are very few Vandals like me who are interfering with its mission.

This entry was posted in Uncategorized and tagged . Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

10 Comments

  1. James Ogilvie
    Posted 8/16/2005 at 5:44 am | Permalink

    Alex: I applaud you for having the spherical obloids to conduct this experiment off your own back. The results do seem to show that the organic nature of Wikipedia does work. To the people who criticize Wikipedia based on the fact that it may have on occasion some slightly erroneous information: may I remind you that most countries (yours included) have for years been teaching scandalously erroneous accounts of history to their children, from state-sanctioned history books! Wikipedia will, over time, help to address this in a very positive manner.

  2. Posted 11/1/2005 at 1:45 pm | Permalink

    Je trouve la discussion vraiment trsè intéressante et je suis moi-même une contributrice occasionnelle de Wikipédia. J’ai donc un peu développé (en français, pour une fois) l’idée sur mon blog, à cette adresse : http://www.mariesg-becker.org/blog/index.php/2005/11/01/77-wikipedia
    Pour dire que rien ne fera descendre Wikipédia dans mon estime. A mon avis…

  3. id_ologist
    Posted 9/1/2006 at 7:24 pm | Permalink

    Some people have complained that the experiment was announced before it began. Perhaps someone else is running a similar experiment without telling anyone anything?

  4. Posted 1/11/2007 at 11:48 am | Permalink

    My concern is also that some information is deleted or left out.

    My suggestion about many listing currently in the encylopedia, especially political ones, is that a wide spectrum of ‘opinions’ be provided in appropriate settings.

    That is there is good, bad and evil in most all things, and we must be careful not to stereotype most things as ‘black or white’, things change over time, and observations are relative to time and space.

    There is a resistance to change, even thought the change is truthful, and we should also judge this vehicle on the basis of its ability to overcome ‘paradigm paralysis, and communicate to the world, important truths that may be in conflict with the ‘power forces’ at play.

    Caesar J. B. Squitti

  5. johnp
    Posted 1/16/2007 at 6:19 am | Permalink

    Even if you “prove” that Wikipedia’s not perfect, that’s fine. All I need to go is use it to see for myself that it’s pretty darn good. Good is good enougn for me:)

  6. Dimitar Panayotov
    Posted 1/24/2007 at 4:38 am | Permalink

    I will make some try to summarize what I have read by far on this topic.

    First, I don’t think that announcing the experiment was influential to Wikipedia in any way; the people there are committed (and they proved it when they defeated Alex’s small-scale experiment) to providing accurate information and they obviously take seriously their commitment. Therego, it is very unlikely that they go to Google in hunt for information about vandals which plot to defeat Wikipedia and is very likely that they take their time for their mission/commitment. Will somebody argue this? I think it’s obvious.

    Second, I agree with many people here: why spit at Wikipedia for inaccuracies when the most paper sources of information are (in _times_) more inaccurate. Also: why requiring 100% accurate information for anything? Virtually everything beyond the traditional and well-known sciences as physics and chemistry (not just these, of course) is a temporary information in a sense, and is subject to change over time in a variety of ways: ambiguous statements will be clarified, insufficient details will be filled, more opinions on the topic will be listed, new discoveries will be made, and so on. Also, if you want 100% accuracy, you should pay a team of scientists and analyzers for it; there is no free lunch and most of the information on the whole Web is published _voluntarily_ with the free will and the blessing of the author. Anyone who thinks that he/she will ever stumble upon perfect information (beyond the simplest things like what is “electrical impedance”) was perhaps never been an author of information which were of use to other humans. In short, at this point I say that yes, Wikipedia surely has errors in it, but it has been developed in such a way that it could evolve (fixing the errors, growing the information, clarifying). The paper encyclopedias and other sources cannot do this, at least not as fast and not as cheap in the terms of money.

    Third: about what is “accurate”. I agree with other humans here that some information is not really an information, but just an information, if you follow me. Example: politics, tries for analysis of commercial products (like processors and video cards), religious articles. They (and many others) are all subjective, mainly because the authors have relied on what other humans said (which could always be a lie due to political goals or commercial interests) or because the topic is not exactly scientific. My experience with Wikipedia was that the authors have _tried_ to give some information in a scientific way, for example to say that the Christians are worshipping Jesus because of his sacrifice on the cross and also they could point that other Christians are worshipping other humans and actions more, like those who worship Holy Maria in greater extent (no criticisms here, please, I never actually read the Christian article on Wikipedia). But these are just scientific methods when one tries to classify the information he/she gots. Anyone felt offended? Well, there are a number of forums where you can argue forever on these topics. Again, Wikipedians _tried_ to give some information. How far they succeeded, they and they alone can judge themselves only. No other humans. If there is fault here, it is the subjectivity of the topic itself. Will somebody support me at this point?

    Fourth: it’s almost all about the money and about the resistance to change, or even both at a time. Somebody used Alex’s experiment to try to spit at Wikipedia. Big deal! Smart humans always analyze the performer’s reasons _first_ and before analyzing the deed itself. My personal thought is that if commercial encyclopedian (or paid news site) owners are trying to ruin Wikipedia’s reputation, this is because if their users know better Wikipedia, their money flow will eventually go dry. Their users are accustomed that Mr. Actual News or Mrs. Expert In Everything will inform them for money and so they question the reliability of a free source like Wikipedia — and yes, when you give money, you expect perfect servicing. But apart from these bad words I said, I do not think that commercial informers and free sources of information can be compared at all. They exist in different universes. And finally (on this point) let us not forget that if humankind stops to rely on The Big Brother to supply it with everything (including information) and if the humankind becomes careful in thinking and responsible of what it produces and how this can help others, then the humankind will not be so easily manipulated so that not to stop some world leaders to destroy foreign countries with reasons which nobody actually believes in (and many other horrible acts throughout the history, too). So, such humans will try to compromise and ruin Wikipedia and the like as long as they can breathe and speak at all, be sure of that! Since I have read here that some countries (unfortunately, mine too; it’s Bulgaria) are teaching kids with false history, I assume there are humans who support me on this point. :)

    Fifth: how Wikipedians did the job. Does it really matter that they used RC Patrol? Everybody with a normal level of IQ could say that they anticipated such abuse and they have taken care that their machines will help them coping with this problem in the best possible regard. So, isn’t it stupid somebody to say that if it weren’t RSS, they could never do it. Well, if it weren’t some guys which never cared for Earth’s atmosphere, perhaps today you would still go to your job with a horse, right? :) It’s the result which is important. How do they do it — it is their business. Are you one of them? No. So, what do you care? The system obviously works. How it works, you don’t care. It would be more useful if the critics proposed to Wikipedians something which will make their life even more easier and help them more to fulfill their noble mission, not to speak like a child — “if I did not injured myself on the leg yesterday, today I would outrun you”. Pfew, how could such humans ever write such stupidities on a public anyway? Don’t they feel little stupid about it? Please, somebody tell me.

    Sixth: the useful things that could be done. While so many humans took the time to write about the topic, I did not seen not a one of them to have been gone through changelogs of even a single Wikipedia article and write about it. That would be far more useful than criticizing the scientificness of the Alex’s approach. Yes, I am in this number also, I know. I agree with few others here which pointed that analyzing changelogs will be far more painless and harmless for Wikipedians. I personally know four people which regularly take their lunch time for correcting slight errors (or, more often, making clarifications and eliminating ambiguities) in Wikipedian articles they read, since they are almost experts on the topic and sometimes they need the whole picture. It’s nothing hard, really: get your lunch in a box from the near restaurant, return to office while nobody is there, eat slowly and easy and do the edit job (also easy, else errors will occur and you will become vandal without actually knowing it).

    If anybody could point me to Wikipedian article with a good chance for finding old errors and their fixtures, I will handle a single article changelog analysis myself. I even intend to find article in my area (programming) and do a job I feel it’s worth doing. I will post it when I am done.

    Summary:

    It’s all about wanting to contribute and to help. Everybody can criticize. It’s not always necessary to do it — the people which post information to such big source like Wikipedia are expected to be (and proved to be) responsible enough to criticize themselves alone in order to achieve development of the good information.

    I am looking forward to see comments from everybody which felt affected by my post, including Alex.

    Best wishes.

    —————————–
    [Dimitar Panayotov, Bulgaria]

  7. alex
    Posted 1/24/2007 at 11:13 am | Permalink

    Dimitar: Very thoughtful (and extensive!) comment. I agree with much of what you have to say here. I am particularly emphaticly in agreement with your sixth point. A detailed analysis of how particular changes are made, and wherever possible, the context and motivation of those changes, would be a really strong contribution to better understanding the “magic” of how Wikipedia works.

    I suspect that the minimal effort to make a change (e.g., during a lunch break) is what allows it to grow at the periphery. If so, it seems to me that having a clear model of this would allow for its applicatoin in other contexts. Think of it as a kind of antidote to the broken window hypothesis. Seeing that things are getting cleaned up encourages others to take just a moment to do the same. But understanding those micro-motivations would get us a lot closer to understanding Wikipedia.

  8. John Horrigan
    Posted 2/12/2007 at 6:46 pm | Permalink

    I love wikipedia so much, my name is John and I am a athletic director at Cadillac High School. Just wanted to say, Alex is my hero!!!!!!!!

  9. alex
    Posted 2/19/2007 at 10:33 am | Permalink

    Caroline writes:

    Here’s the new (correct as of date of email, Feb 18, 2007) for the History Flow graphics gallery:

    http://www.research.ibm.com/visual/projects/history_flow/gallery.htm

  10. Posted 2/28/2007 at 3:09 pm | Permalink

    The Zero Void Show Says:
    “Most of what wikipedia states is reliable and it’s delieberately not decieving. Thats already known and accepted to be true and reliable information [on Wikipedia]. So for the last time, did you hve to do this experiement?”

    Is easy to say that after TWO years of the original post… =)

21 Trackbacks

  1. By Denken Über on 9/8/2004 at 1:57 pm

    Wikipedia, edición colaborativa y autoridad
    Para los que no lo conocen Wikipedia es una enciclopedia online, abierta y colaborativa, cuyo esfuerzo ya suma ediciones en varios idiomas. Empezó en Inglés en Marzo del 2001 (ya tienen más de 300.000 articulos)y en Mayo ya estaba online…

  2. By The Now Economy on 9/8/2004 at 8:51 pm

    Decentralized Authoring Can Be Self-Healing
    BoingBoing in Wikipedia proves its amazing self-healing powers pointed us to The Isuzu Experiment, which goes like this: Joi Ito points to an ongoing discussion regarding the authority of wikipedia as a source of information and knowledge. The discussi…

  3. By Blog.org on 10/30/2004 at 11:31 am

    Wikipedia roundup
    Wikipedia and its critics, plus mention of H2G2 and various versions of the Encyclopaedia Brittannica.

  4. By Denken Ãœber on 11/22/2004 at 11:00 am

    Wikipedia, edición colaborativa y autoridad
    Para los que no lo conocen Wikipedia es una enciclopedia online, abierta y colaborativa, cuyo esfuerzo ya suma ediciones en varios idiomas. Empez� en Ingl�s en Marzo del 2001 (ya tienen m�s de 300.000 articulos)y en Mayo ya estaba online…

  5. By I Speak of Dreams on 11/27/2004 at 12:34 am

    Reliability and Veracity of Blogs and Wikipedia
    Why is it important what an uninformed, small-town librarian thinks of blogs and Wikipedia?

  6. […] From BoingBoing, an experiment to introduce false information into Wikipedia. Results: all 13 introduced errors were corrected within a matter of hours. The very idea of Wikipedia rocks; the fact that it actually exists is just off the scale of coolness. […]

  7. […] Explicar las bondades de las comunidades online y la “reputación” a un periodista a veces es complicado; explicarselo a uno que no acepta siguiera mirar ejemplos sobre el tiempo que un error anónimo y sin aviso, tarda en ser corregido en Wikipedia es peor. Alex Havalis introdujo 13 errores en diferentes artículos de Wikipedia. […]

  8. […] Det är ett förfarande som nog börjar bli lite väl vanligt nu. Först var förmodligen Alex Halavais med sitt Isuzu-experiment, men många har följt i hans spår – något som Halavais själv tog avstånd från redan 2004. Vinterns Seigenthaler-kontrovers (som jag uppmärksammade i december) kan på sätt och vis också föras till kategorin “experiment med faktafel”. […]

  9. […] Felaktiga uppgifter kan inte kvarstå länge i ett verk som läses av så många människor, läsare som dessutom själva kan ändra uppgifter. BBC-artikeln anför bland annat som exempel hur professorn och bloggaren Alex Halavais medvetet matade in ett antal felaktiga uppgifter. Enligt Halavais’ egen bloggpost om The Isuzu Experiment tog det inte lång tid innan felen hade rättats (och den klentrogne professorns hypoteser kommit på skam): All the changes were “factual” in nature, though some fairly obscure. And *all* were identified and removed within a couple of hours. […]

  10. […] This was behind a pay-wall before now, but the Chronicle of Higher Ed has an article on the relationship between Wikipedia and academia. I was quoted in there, and it focusses a little on the Isuzu Experiment but I guess that’s not all bad. As always, I think that there is space there for missed nuance (the nature of a short article), and I feel like I should hedge some of the things there. For example, although I do mostly make changes anonymously, in part because I am a known wikispammer and I don’t want the name to influence people’s perception of the edit. However, in the case of the Com Theory article, I did do it as myself. It doesn’t show up in the history because the article was moved (from “Theories of Communication”). And I know that traditional institutional scholars do contribute to Wikipedia, and that contribution is often valued. […]

  11. […] clear—I am not among those “some people,” and I would never use my caprice (the “Isuzu Experiment”) as anything approaching substantial evidence. These icons link to social bookmarking sites where […]

  12. […] Alexander (2004). The Isuzu Experiment. [Elektronisk]. Tillgänglig <http://alex.halavais.net/the-isuzu-experiment&gt; samt <http://alex.halavais.net/return-of-joe-isuzu&gt; […]

  13. […] Halavais blogginlägg om experimentet Halavais svarar på frågor kring experimentet Halavais ändringar på Wikipedia Ännu en undersökning Etiketter Wiki, wikis […]

  14. […] can easily be subjected to vandalism. What’s interesting though is in 2004, a man by the name of Alex Halavais who was quite a sceptic of Wikipedia’s conducted an experiment to test the reliability. He would […]

  15. By Web 2.0 – Week 2 (part 1) on 12/3/2011 at 10:13 am

    […] involved that “goodness and truth” will prevail; and studies have shown this to be true (Halavais, 2004). Caution should nevertheless be exercised. In the classroom the obvious application is in creating […]

  16. […] A. 2004. “The Isuzu Experiment,” blog entry at A Thaumaturgical Compendium (29 August), at http://alex.halavais.net/index.php?p=794, accessed 8 December […]

  17. By Wiki del 2 « isakglans on 5/29/2012 at 5:32 am

    […] Alex. (2004-08-29). The Isuzu experiment [elektronisk]. Tillgänglig: < http://alex.halavais.net/the-isuzu-experiment […]

  18. By Lärkontrakt « isakglans on 5/29/2012 at 9:51 am

    […] Alex. (2004-08-29). The Isuzu experiment [elektronisk]. Tillgänglig: < http://alex.halavais.net/the-isuzu-experiment […]

  19. By Web Indexing » In Praise of Wikipedia on 6/8/2012 at 2:11 am

    […] Many receive an RSS feed of recent changes so are able to spot errors and vandalism immediately: http://alex.halavais.net/news/index.php?p=794 .  Editors who do not have the time or knowledge to update pages that obviously need work can add […]

  20. […] Wikipedia gir alle den samme muligheten til Ã¥ bidra, noe som gir mulighet for interessant kunnskapsutveksling, men ogsÃ¥ for “spamming” og sabotasje. Med sine over 23 millioner artikler, er Wikipedia et dugnadsleksikon som har vist seg Ã¥ fungere, mot alle odds. Magasinet Nature viste i 2005 at artiklene ofte er av overraskende høy kvalitet, og oppslagsverket har vist seg motstandsdyktig mot sabotasje, blant annet i forbindelse med “The Isuzu Experiment”. […]

  21. […] accuracy gathered a lot of public and early research interest. Take for example the (in)famous ISUZU experiment from 2004 which was basically an ethno-methodological experiment looking at how the community […]

Post a Comment

Your email is never published nor shared. Required fields are marked *

*
*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>