Internet navigators think small

Buzzwords

Test your tech-talk IQ

Internet navigators think small

Researchers study how communities change the Web
Small-world online communities reflect the structure of the wider Web -- but they change the way people use the Internet, researchers say.

By Alan Boyle MSNBC


May 21 — Scientists who have mapped out the structure of the World Wide Web are focusing increasingly on the smaller picture: how it is that online villages coalesce. Such research can help you hook up more easily with the information and people you’re looking for — but they could also help marketers and politicians figure out more easily where you’re coming from.

• Yellow Pages

• Auctions at uBid

• Personals Channel

• Professional Tips

• Newsletters

• Shopping

SUDDENLY, WEB COMMUNITIES have become one of the hottest topics on the wired frontier: You can see it in the rapid commercialization of the grass-roots Weblog movement, as well as the latest research into how the Web organizes itself and how Internet users connect to each other.
Such research builds upon previous studies into the structure of the Web — a structure that scientists have found parallels natural phenomena ranging from genetic code to our planet’s biosphere and beyond.
Like those natural examples, the Web follows a power-law distribution — meaning there are a lot of sparsely connected sites and a very small number of highly connected sites.
Can the rise of small-scale communities, such as those that grow up around Weblogs, change that picture? So far, researchers say the answer is yes, no and maybe.
On the “no” side of the issue is Notre Dame physicist Albert-Laszlo Barabasi, who conducted one of the first studies of the Web’s connectedness and explores network theory in a new book titled "Linked.”

MSNBC Weblogs

•	Alan Boyle's Cosmic Log
•	Michael Moran's World Agenda
•	Chris Matthews' Hardball
•	Jan Herman's The Juice

“We have this very inhomogeneous Web structure — many pages and nobody points to them — and there are a few sites that have millions of links pointing to them. In the Weblog business, that is actually repeating itself,” Barabasi said. “Yes, you could have a Web site out there, but will anybody point to it?”
Gary Flake of the NEC Research Institute has a slightly different take on the Weblog trend: “I think it’s the next logical step, and while it is certainly changing the topology of the Web in measurable ways, I don’t think that this is necessarily a paradigm shift.”

MAPPING THE VILLAGES
Although they’re not specifically studying Weblogs, Flake and his NEC colleagues are focusing on how smaller communities develop on the Internet. However, such research requires a sophisticated Web crawler to identify the thousands or millions of global villages on the Internet.
“Our hope is that by the summer we’re actually going to be running the Web community algorithm, doing it for all possible clusterings,” he said.
Small-world clusters tend to have tight-knit links between a small number of users, with sparse connections to the wider network. Kevin Werbach, editor of the newsletter Release 1.0, said Weblogs could play a key role in the growth of such clusters on the Internet.
“There are communities that know about each other, and there are communities that don’t,” he said. “Because they’re a tool to make information available, Weblogs lower the threshold for communities to find each other.”

‘A Picture of Weblogs’

Clay Shirky, a writer and consultant who focuses on the effects of decentralizing technologies, said that the small-world trend is changing the way people use the Web, even though it may not change the overall topology. He pointed to the rise of such user-centric phenomena as Slashdot, Plastic and the hyperlinked sites known as Wikis.
“That has been an enormous change — the notion that this stuff can now be two-way, that this can involve readers and writers as equals,” he said. “What this does, however, is create a world where there is a very great deal of content to be sorted through.”
Which gets back to Barabasi’s point. Shirky said he could already see the development of a power-law distribution in his studies of the LiveJournal phenomenon, as well as his own experience last month when he posted an essay on the size limitations of Web communities.
“Looking at the traffic that came into my site, I had links from 400 different URLs out on the Web, many of them from Weblogs,” he said. “The top 40 percent of that traffic came from just three Weblogs.”
Just as the Google search engine gained dominance by taking advantage of the Web’s power-law distribution of links, experimental search engines such as Blogdex or Daypop could serve to organize the Weblog universe, Shirky said.
Flake said “the really interesting thing is how search engine companies and data mining companies are going to mine this information for useful things.”

REAL-WORLD APPLICATIONS

If the small-scale structures of the Web could be mapped in detail, that could open the way for subject-specific search engines — say, for sports car enthusiasts or anti-terrorism sleuths. Politicians could identify the communities most likely to take their side, or oppose them, on gun control or AIDS research or Middle East policy. There could be commercial applications as well, Barabasi said.
“Finding the community of sports car enthusiasts would allow car companies to most effectively market their new models by placing ads at several hubs of this community,” he wrote in “Linked.”
Werbach pointed to the growing number of companies selling Web-logging software, as well as the rise of corporate Weblogs.
“We’re not yet at the stage where Weblogs have been commercialized, but we’ll get there.”
Shirky is expecting a case of dot-com deja vu all over again.

"You can be sure, as with any technology, that once it becomes good, people will begin renaming what they have,” he said. “Watch the content management vendors start claiming that they have blogging functions. That’s an inevitability. There’s always a deliberate attempt to confuse the issue when the public finds something it likes.”
What could set the Weblog phenomenon apart is that it’s more of a social phenomenon than a technological trend, Shirky said.
“If the other Webloggers don’t treat you as one, and don’t link to you,” he said, “you’re going to be relegated to the long, flat plain of the power-law curve.”