The core issue of collaborative editing, that of accuracy and trust, has reached a point in debate where research is needed to advance the practice of content use and development. Hiawatha Bray of the Boston Globle offered a Wikipedia criticism in July, calling it One great source — if you can trust it:
For it lacks one vital feature of the traditional encyclopedia: accountability. Old-school reference books hire expert scholars to write their articles, and employ skilled editors to check and double-check their work. Wikipedia’s articles are written by anyone who fancies himself an expert…
“I think it’s exactly the right price,” said Michael Ross, senior vice president of corporate development at Encyclopaedia Britannica Inc. in Chicago. Major articles in Britannica are signed by the author; all articles are vetted by an experienced team of editors and scholars. The libraries that pay $1,500 for a set of bound volumes or the family that pays $60 a year for an Internet subscription are buying confidence as well as information. … Ross admits to reading and enjoying Wikipedia, and has even gotten ideas there for future Britannica articles. But the absence of traditional editorial controls makes Wikipedia unsuited to serious research. “How do they know it’s accurate?” Ross asks. “People can put down anything.”..
In 2002, Wikipedia was criticized because it couldn’t scale and have in-depth articles. Turns out that more was put down than expected, surpassing the Britannica.
Hiawatha raised a key issue, that of quality and reputation, and his piece highlighted Wikipedia’s ambition to publish a first print version. Coupling emergent content development and formal editorial process is a very competitive business model for print. But if the public learns to use and trust the content that emerges in Wikipedia as an authority, it is even more disruptive.
This week Al Fasoldt, a Post-Standard Columnist in Syracuse NY claimed Wikipedia is untrustworthy, based upon an interview with a high school librarian:
“As a high school librarian, part of my job is to help my students develop critical thinking skills,” Stagnitta wrote. “One of these skills is to evaluate the authority of any information source. The Wikipedia is not an authoritative source. It even states this in their disclaimer on their Web site.”
Wikipedia, she explains, takes the idea of open source one step too far for most of us.
Mike from Techdirt takes the columnist to task for misunderstanding Wikipedia:
Mike took another step of contacting the reporter, and the exchange led him to ask, whom do you trust, the wiki or the reporter?
The quality of Wikipedia Articles, at the very least, at a moment in time are better than they were before and will improve over time. Mike offered a Techdirt Challenge: I pointed to the Wikipedia page on Syracuse, NY where he apparently lives, and suggested he change something on the page, to make it provably, factually incorrect — and see how long it lasted. Alex Halavais, for one, is taking the Challenge. While the results of the challenge (update: 13/13)will provide some valuable insight, it lacks an untampered collection methodology and introduces unfair costs to the system itself.
Joi Ito rightly condemns Mr. Fasoldt’s assertion and views this issue as traditional vs. collective authority:
Shelley Powers delves into the issue of truth and authority:
The reason, according to those with more modern views, is though the authors could be considered ‘authorities’ on the topic, they don’t have the ‘truth’ because the truth, in this instance, is held by those who have new, and fresh insight into the existing material–they have reached an epiphany the others, weighed down by the mass of research material and outdated ideas, can’t hope to achieve.
According to these blessed with such insight, they have truth without authority, while the historians have authority, but can’t possibly understand the truth. Who you trust then, depends less on authority or even truth than it does on who you want to believe–literally whose interpretation rings your bell the most.
The Manifesto for the Reputation Society describes Wikipedia as reputation for the community as a whole by helping to create a public good where there is more flexibility as reputation and other motivations substitute for direct reciprocity. As the Manifesto hints, Wikipedia is considering codification:
Which brings me to an lingering thought — that explicitly codifying reputation introduces a cost which can constrain commons-based peer production. Wikipedia was never supposed to work, somehow does because of good club theory and transaction costs, and has gained a reputation as a resource. Introducing reputation for contributors or articles is the greatest risk to the Wikipedia community. Getting a base study on factual accuracy can help inform this decision as well as educate the public on how to use and participate with this commons resource.
I’ve been quitely forming a group of journalism schools, media centers and experts to engage in the Wemedia Project, which begins with a formal Wikipedia Article fact checking excercise and publishing findings. The USC Annenberg Center has already announced their support and next month we will begin the collaborative research process within a Socialtext Workspace. Without getting into defining truth, you can separate issue of fact, value or policy. The approach is to apply a formal fact checking process to a sample of articles to gain a baseline measure of factual accuracy and explore issues of reputation.
More to come, suggestions appreciated.