NationStates Jolt Archive

COMMENTARY: Imagine entering a topic or phrase into a search engine and getting back a reference to every book ever published containing that topic or phrase. Can you say, "Universal Online Library," boys and girls? Talk about making reasearch easier! :)

What do you think will be some of the effects ( both positive and negative ) of having this incredible library available to every human on the planet, which is the vision of those working on this project?


Scan This Book! (http://www.nytimes.com/2006/05/14/magazine/14publishing.html?_r=1&th&emc=th&oref=slogin)


By KEVIN KELLY
Published: May 14, 2006

In several dozen nondescript office buildings around the world, thousands of hourly workers bend over table-top scanners and haul dusty books into high-tech scanning booths. They are assembling the universal library page by page.

The dream is an old one: to have in one place all knowledge, past and present. All books, all documents, all conceptual works, in all languages. It is a familiar hope, in part because long ago we briefly built such a library. The great library at Alexandria, constructed around 300 B.C., was designed to hold all the scrolls circulating in the known world. At one time or another, the library held about half a million scrolls, estimated to have been between 30 and 70 percent of all books in existence then. But even before this great library was lost, the moment when all knowledge could be housed in a single building had passed. Since then, the constant expansion of information has overwhelmed our capacity to contain it. For 2,000 years, the universal library, together with other perennial longings like invisibility cloaks, antigravity shoes and paperless offices, has been a mythical dream that kept receding further into the infinite future.

Until now. When Google announced in December 2004 that it would digitally scan the books of five major research libraries to make their contents searchable, the promise of a universal library was resurrected. Indeed, the explosive rise of the Web, going from nothing to everything in one decade, has encouraged us to believe in the impossible again. Might the long-heralded great library of all knowledge really be within our grasp?

Brewster Kahle, an archivist overseeing another scanning project, says that the universal library is now within reach. "This is our chance to one-up the Greeks!" he shouts. "It is really possible with the technology of today, not tomorrow. We can provide all the works of humankind to all the people of the world. It will be an achievement remembered for all time, like putting a man on the moon." And unlike the libraries of old, which were restricted to the elite, this library would be truly democratic, offering every book to every person.

But the technology that will bring us a planetary source of all written material will also, in the same gesture, transform the nature of what we now call the book and the libraries that hold them. The universal library and its "books" will be unlike any library or books we have known. Pushing us rapidly toward that Eden of everything, and away from the paradigm of the physical paper tome, is the hot technology of the search engine.

1. Scanning the Library of Libraries

Scanning technology has been around for decades, but digitized books didn't make much sense until recently, when search engines like Google, Yahoo, Ask and MSN came along. When millions of books have been scanned and their texts are made available in a single database, search technology will enable us to grab and read any book ever written. Ideally, in such a complete library we should also be able to read any article ever written in any newspaper, magazine or journal. And why stop there? The universal library should include a copy of every painting, photograph, film and piece of music produced by all artists, present and past. Still more, it should include all radio and television broadcasts. Commercials too. And how can we forget the Web? The grand library naturally needs a copy of the billions of dead Web pages no longer online and the tens of millions of blog posts now gone — the ephemeral literature of our time. In short, the entire works of humankind, from the beginning of recorded history, in all languages, available to all people, all the time.

This is a very big library. But because of digital technology, you'll be able to reach inside it from almost any device that sports a screen. From the days of Sumerian clay tablets till now, humans have "published" at least 32 million books, 750 million articles and essays, 25 million songs, 500 million images, 500,000 movies, 3 million videos, TV shows and short films and 100 billion public Web pages. All this material is currently contained in all the libraries and archives of the world. When fully digitized, the whole lot could be compressed (at current technological rates) onto 50 petabyte hard disks. Today you need a building about the size of a small-town library to house 50 petabytes. With tomorrow's technology, it will all fit onto your iPod. When that happens, the library of all libraries will ride in your purse or wallet — if it doesn't plug directly into your brain with thin white cords. Some people alive today are surely hoping that they die before such things happen, and others, mostly the young, want to know what's taking so long. (Could we get it up and running by next week? They have a history project due.)

[ This article is NINE pages long. Read the rest of the article (http://www.nytimes.com/2006/05/14/magazine/14publishing.html?_r=1&th&emc=th&oref=slogin). ]

Maybe they will buy project gutenberg.

We need to hoard knowledge and make it harder for people to find. It appears my Illuminati colleagues have got all mixed up.

Ooops.

So the sum of all human knowledge is going to be availabe for free on Google? A lot of people are going to get shafted out of a lot of money.


Oh, and is it going to record all the porn too?

Hmm... A complete repository of all human knowledge. In my head. To use as I see fit. Knowledge of explosives and adhesives and pharmaceuticals and chemistry. Imagine the possibilities. :)

Hmm... A complete repository of all human knowledge. In my head. To use as I see fit. Knowledge of explosives and adhesives and pharmaceuticals and chemistry. Imagine the possibilities. :)
You're gonna make a mud bomb aren't ya?

You're gonna make a mud bomb aren't ya?

You think I can cover a small New England town?

So the sum of all human knowledge is going to be availabe for free on Google? A lot of people are going to get shafted out of a lot of money.

I doubt every single book will be scanned.

Why?

Caus there is always some selfish bastard out there who holds the one copy of say, a medieval manuscript on sword fighting (sorry, personal hobby of mine), that he/she doesn't want anyone else to see because it's theirs and they don't want anyone else to have it, despite the fact that there are thousands of people out there who do want to have a look at it and bring it to the attention of the world.

Gits.

From the days of Sumerian clay tablets till now, humans have "published" at least 32 million books, 750 million articles and essays, 25 million songs, 500 million images, 500,000 movies, 3 million videos, TV shows and short films and 100 billion public Web pages.
99 billion of which are porn and 99% of the remainder are forums like this one or emo blogs whinging about their lot in life and threatening to cut themselves.

Do we really want these to be our legacy to future generations?

You think I can cover a small New England town?
Why such a small target? Go for New England I say.

Just sounds like further ways of controlling information to me.

I doubt every single book will be scanned.

Why?

Caus there is always some selfish bastard out there who holds the one copy of say, a medieval manuscript on sword fighting (sorry, personal hobby of mine), that he/she doesn't want anyone else to see because it's theirs and they don't want anyone else to have it, despite the fact that there are thousands of people out there who do want to have a look at it and bring it to the attention of the world.

Gits.
Ah, hadn't thought of that. Hmmmm, google needs a crack squad of book stealing nerds for cases such as this.

99 billion of which are porn and 99% of the remainder are forums like this one or emo blogs whinging about their lot in life and threatening to cut themselves.

Do we really want these to be our legacy to future generations?

He who does not learn from history is doomed to repeat it.

Ah, hadn't thought of that. Hmmmm, google needs a crack squad of book stealing nerds for cases such as this.

Ninjas. *nod*

Ah, hadn't thought of that. Hmmmm, google needs a crack squad of book stealing nerds for cases such as this.

It's going to be bloody hard.

Just imagine trying to get the Vatican to open up its vaults...

Why such a small target? Go for New England I say.

WMWs(Weapons of Mass Wackiness) need small-scale testing before being deployed on a larger scale. :)

It's going to be bloody hard.

Just imagine trying to get the Vatican to open up its vaults...
I agree with LG, ninjas are needed.

Though in all seriousness I can't see the Vatican even contemplating this. Though we might get luck and get a Pope in favour of it in a few years. This one certainly isn't in the prime of life.

WMWs(Weapons of Mass Wackiness) need small-scale testing before being deployed on a larger scale. :)
You would have to make sure you got just the right consistancy of mud.

I wonder what this will do for universities as well?

No longer any need for a library, nor the book shops on each campus.

Research could be done via links to the specific books.

Essays and dissertations could be completed without needing to visit any archives? (No more thrill of the chase :( )

Education wouldn't really be needed at a higher level in many cases as all the info you would need would be in such a library.

This could change so much.

Hmm, this sounds a bit like the Hitchhikers guide, only more extensive since it will contain more info about the earth then the words 'mostly harmless' :D

I wonder what this will do for universities as well?

No longer any need for a library, nor the book shops on each campus.

Research could be done via links to the specific books.

Essays and dissertations could be completed without needing to visit any archives? (No more thrill of the chase :( )

Education wouldn't really be needed at a higher level in many cases as all the info you would need would be in such a library.

This could change so much.

And don't forget, they think that the 'technology of tomorrow' would allow all that information to be downloaded onto an iPod. It would make libraries totally irrelevant. And what would be done with all the actual books?

Hmm, this sounds a bit like the Hitchhikers guide, only more extensive since it will contain more info about the earth then the words 'mostly harmless' :D
As long as it still says Don't Panic on the cover.

And what would be done with all the actual books?

The greatest campfire humanity has ever known :D

The greatest campfire humanity has ever known :D
Wonderful. I'll bring the uber-marshmallows.

You would have to make sure you got just the right consistancy of mud.

Indeed. Not all muds are created equal. *nod*

Oh Joy. More cut & paste plagiarism from people too lazy to, like, read and, you know, write.

And don't forget, they think that the 'technology of tomorrow' would allow all that information to be downloaded onto an iPod. It would make libraries totally irrelevant. And what would be done with all the actual books?

Well, we wouldn't need new ones me thinks, that'd save a crap load of paper.

The actual books? Their value would fall and as Blaatschappen points out we could have a very warm November 5th here in the UK :D

And don't forget, they think that the 'technology of tomorrow' would allow all that information to be downloaded onto an iPod. It would make libraries totally irrelevant. And what would be done with all the actual books?

By then, we probably wouldn't need books. They'd be e-books. *nod*

Indeed. Not all muds are created equal. *nod*
Some mud is more equal than other mud.

Oh Joy. More cut & paste plagiarism from people too lazy to, like, read and, you know, write.


Yeah. Exams would go out of the window if you had a real time link readable through some nice high tech glasses or something similar...

Oh Joy. More cut & paste plagiarism from people too lazy to, like, read and, you know, write.It will be easier to detect such plagiarism too, with all possible sources in one convenient database.

Of course, especially for older texts, there is still a large problem of getting from a scanned text to a computer text (rather than leave it as an image). Even using printed books writing recognition is hardly perfect.
Searching through all known texts, reliably, is still a long way off.

Yeah. Exams would go out of the window if you had a real time link readable through some nice high tech glasses or something similar...
The solution? EMP weapons. If your glasses look too high tech.......ZAP. No more cheating. Nobody would dare let their mobile phone go off in clas anymore.

The solution? EMP weapons. If your glasses look too high tech.......ZAP. No more cheating. Nobody would dare let their mobile phone go off in clas anymore.

Heh, if they are in a exam room and their mobile phone goes off they get thrown out anyway. Stupid kids who 'forget' are just plain idiotic.

Members of my family are exam vigilators and have to regularly throw out people who 'forget'. They often claim they didn't know, but are pointed to the signs on the door, on the front of their exam papers and the announcement made at the start of the exam. Ignorance of the rules is no defence.

Heh, if they are in a exam room and their mobile phone goes off they get thrown out anyway. Stupid kids who 'forget' are just plain idiotic.

Members of my family are exam vigilators and have to regularly throw out people who 'forget'. They often claim they didn't know, but are pointed to the signs on the door, on the front of their exam papers and the announcement made at the start of the exam. Ignorance of the rules is no defence.
Most definately. But people would be so much more attentive to the rules if they thought their expensive technology would get fried.

I wonder what this will do for universities as well?

No longer any need for a library, nor the book shops on each campus.

Research could be done via links to the specific books.

Essays and dissertations could be completed without needing to visit any archives? (No more thrill of the chase :( )

Education wouldn't really be needed at a higher level in many cases as all the info you would need would be in such a library.

This could change so much.
This attitude actually worries me, because it leads easily to the idea that there's nothing left to discover, that all the needed information in the world is out there, and it's just a matter of finding it. I'm a teacher, and I love the idea of having such an incredible resource at my fingertips, but we need to be careful that we don't come to rely on it for everything.

This attitude actually worries me, because it leads easily to the idea that there's nothing left to discover, that all the needed information in the world is out there, and it's just a matter of finding it. I'm a teacher, and I love the idea of having such an incredible resource at my fingertips, but we need to be careful that we don't come to rely on it for everything.

I'm training to become a teacher and I wholly agree with that.

Just out of curiosity what age range do you teach?

I'm on an experience for KS2 (7-11 years old) currently and will be training for the same age range next year.

I'm training to become a teacher and I wholly agree with that.

Just out of curiosity what age range do you teach?

I'm on an experience for KS2 (7-11 years old) currently and will be training for the same age range next year.
I teach at the university level, and I have to deal with that trend I described above every day. It's disturbing.

I teach at the university level, and I have to deal with that trend I described above every day. It's disturbing.

I see it at my uni too.

When I actually went to archives, handled primary material and did research for both of my dissertations rather than just use the standard secondary sources, my profs. were rather suprised and encouraged :) Nice to hear, but worrying that more students don't do it.

I see it at my uni too.

When I actually went to archives, handled primary material and did research for both of my dissertations rather than just use the standard secondary sources, my profs. were rather suprised and encouraged :) Nice to hear, but worrying that more students don't do it.
It's worse than that--I've had students who seem to be convinced that they can find the answers to any question--even personal philosophical ones--simply by doing the right google search. They have this sense that every iota of knowledge necessary to living a full life has already been discovered and catalogued somewhere, and the key is just being able to find it. Self-discovery is lacking.

Some mud is more equal than other mud.

The proletariat mud will unite and overthrow the aristocratic mud.

just you wait and see.

It's worse than that--I've had students who seem to be convinced that they can find the answers to any question--even personal philosophical ones--simply by doing the right google search. They have this sense that every iota of knowledge necessary to living a full life has already been discovered and catalogued somewhere, and the key is just being able to find it. Self-discovery is lacking.

*Head hits desk*

Ow...

Indeed self discovery is lacking, those who wish to put this library together are those who want to change the world and make something of themselves. Most likely by changing themselves and learning as they go, but it will only perpetuate what you said. It's both a shame and a treasure at the same time.

Never the less, i think throughout history that mindset has been there, it's just that there are those few who wish to discover for themselves what they want to know who change the world.

The solution? EMP weapons. If your glasses look too high tech.......ZAP. No more cheating. Nobody would dare let their mobile phone go off in clas anymore.
You know they already have I-Pod disrupters in Japan. When activated they cause every I-Pod within a certain range to play whatever music is loaded into the disputer.

They have this sense that every iota of knowledge necessary to living a full life has already been discovered and catalogued somewhere, and the key is just being able to find it.
You mean it hasn’t? With the exception of scientific knowledge, everything else has been done at least one. Of course, you may have to go a bit more in depth than a Google search, but don’t ever for a second believe that anything you’ve come up with, besides specific situational responses, is unique.

You know they already have I-Pod disrupters in Japan. When activated they cause every I-Pod within a certain range to play whatever music is loaded into the disputer.
I'd make the iPod disruptor read out huge amounts of lorem ipsum.

And don't forget, they think that the 'technology of tomorrow' would allow all that information to be downloaded onto an iPod. It would make libraries totally irrelevant. And what would be done with all the actual books?
As with computers, there will always be a minority of nuevo-Luddites who refuse to be stampeded into the next millenium. Books will still be around for a long time, IMHO.

Oh Joy. More cut & paste plagiarism from people too lazy to, like, read and, you know, write.
Perhaps, but it will be relatively easy to catch them. Just program your PC or whatever to compare all passages to all known sources and if there's an unattributed match ... ya got 'em! Mwahahahaha! :D

Some mud is more equal than other mud.
Would one of the "pillars" be "Mud is clean?" :D

Perhaps, but it will be relatively easy to catch them. Just program your PC or whatever to compare all passages to all known sources and if there's an unattributed match ... ya got 'em! Mwahahahaha! :D
We do that already. More often than not, however, writing teachers catch cheaters simply by reading the paper, thinking about the student, and saying to ourselves "there's no freaking way that idiot wrote this paper." :D Then we go online to get confirmation.

Yeah. Exams would go out of the window if you had a real time link readable through some nice high tech glasses or something similar...
No. Exams in the traditional sense would become obsolete. Since all data is available all the time, what would be tested is ability to find it, compile it, and present it in an intelligent, easily comprehended form. There would no longer be a need to remember vast amounts of information. Exams would be on how to make use of it.

This attitude actually worries me, because it leads easily to the idea that there's nothing left to discover, that all the needed information in the world is out there, and it's just a matter of finding it. I'm a teacher, and I love the idea of having such an incredible resource at my fingertips, but we need to be careful that we don't come to rely on it for everything.
I suspect you may be overstating the case a bit. There have always been those who can't be arsed to do a bit of work. But this technology, once in place, will also allow more research on the structure of knowledge itself. I suspect we will find more of the "holes," and that there will always be those who take great delight in filling them through experimentation, exploration, and discovery.

[ ever the eternal optimist ] :)

No. Exams in the traditional sense would become obsolete. Since all data is available all the time, what would be tested is ability to find it, compile it, and present it in an intelligent, easily comprehended form. There would no longer be a need to remember vast amounts of information. Exams would be on how to make use of it.
Interesting idea. I think something along those lines is already happening. In the history syllabus in Ireland for the leaving certificate everyone(studying history) has to do a research project for 60% of their grade. The course is also less about learning off dates of historical events and more commenting on their significance.

Interesting idea. I think something along those lines is already happening. In the history syllabus in Ireland for the leaving certificate everyone(studying history) has to do a research project for 60% of their grade. The course is also less about learning off dates of historical events and more commenting on their significance.
Repeat after me: "Yoda is always right." :D

Here's another interesting excerpt from the OP article:

"When books are digitized, reading becomes a community activity. Bookmarks can be shared with fellow readers. Marginalia can be broadcast. Bibliographies swapped. You might get an alert that your friend Carl has annotated a favorite book of yours. A moment later, his links are yours. In a curious way, the universal library becomes one very, very, very large single text: the world's only book."

I find that not only mind-boggling, but wonderful! :)

"So what happens when all the books in the world become a single liquid fabric of interconnected words and ideas? Four things: First, works on the margins of popularity will find a small audience larger than the near-zero audience they usually have now. Far out in the "long tail" of the distribution curve — that extended place of low-to-no sales where most of the books in the world live — digital interlinking will lift the readership of almost any title, no matter how esoteric. Second, the universal library will deepen our grasp of history, as every original document in the course of civilization is scanned and cross-linked. Third, the universal library of all books will cultivate a new sense of authority. If you can truly incorporate all texts — past and present, multilingual — on a particular subject, then you can have a clearer sense of what we as a civilization, a species, do know and don't know. The white spaces of our collective ignorance are highlighted, while the golden peaks of our knowledge are drawn with completeness. This degree of authority is only rarely achieved in scholarship today, but it will become routine."