How hard can it be?
Well for a corporation very hard. But fortunately my solution to this problem that seems to be vexing everyone lately doesn’t rely on a corporate sponsor, and places the future of web search in OUR hands.

My vision of search is that each website takes a small share of responsibility, indexing sites that website finds relevant and/or useful and sharing its results with other search engines that request the info via a standard API, much like blogs share feeds using standards such as RSS and ATOM. Each webmaster can control how much weight to give to each set of imported results in comparison to sites it indexes itself and how the results are merged. I have the following basic vision, but as different developers get involved in ironing out the final details of the standard things may change. None the less the basic principles of my idea are as follows:

API:
Each search engine should accept requests in either/both http and/or https, probably with search parameters passed in the query string. The results then get sent back in an standard XML format, which may look something like:

<results source="http://example-search-engine.com">
	<site weight="200" url="http://example-result.org" title="Example Results">
		<favicon url="http://example-result.org/favicon.ico" />
		<page weight="150" url="http://example-result.org/page1.php" title="">
			<match><title>This title contains the search terms</title></match>
			<match><p>This paragraph contains one of the search terms</p></match>
			<match><img src="someimage.jpg" alt="This image description contains one of the search terms" /></match>
		</page>
		<page weight="150" url="http://example-result.org/page2.php">
			<match><p>This paragraph contains one of the search terms</p></match>
		</page>
	</site>
	<site ...>
		...
	</site>
</results>

CMS:
With the API open, different CMS systems can be developed that both index selected sites and process results from other search engines that implement the API. Each CMS can process the results in their own way, and allow the webmaster to configure various options in their own way also. For example, they can either integrate imported results, show them separately in a side panel or some mixture of the two depending on source. They can determine whether to show multiple results for one site or whether to group them. They may be configured to show country of the webserver of each result, or warnings if they appear on a malware list, or any other number of options. They may have options for doing image search, or map results etc, linking to whatever service they choose for these specialist searches.

A basic CMS may involve no login at all, with configuration handled via a file, whilst others may implement the API as a plugin to existing CMS systems such as Wordpress, Elgg, MediaWiki or whatever. Some may allow each logged in user to configure their own preferences and weightings, suggest relevant sites to index, report sites as spam, etc.

Plugins:
More complex dedicated search systems may themselves allow plugins or feature advanced options. This may include scraping results from existing search engines such as Google, Bing and Yahoo, in the manner that <a href=”https://scroogle”>Scroogle</a> currently does with Google results. Again these results may have weightings applied to them by the webmasters.

Conclusion:
A distributed solution to search means we don’t have to have massive datacenters under our control in order to compete with Google, each of us can take responsibility for indexing portions of the web important to ourselves, and share that data with each other freely for the benefit of all. To make a distributed system work we don’t need to abandon the huge stores of data provided by google until we are ready, we can in the mean time scrape from them, but ultimately a distributed system, based on qualified trust, should prove better at promoting sites that are useful over sites with good SEO but useless content.

Google is becoming less and less useful as each search tends to produce 100 sites all with the same article scraped from wikipedia, interspersed with adfarms, and I can’t help think such sites wouldn’t make it to prominence in the distributed model, as it won’t be in any webmasters interest to index useless sites. Unlike google who actually profit from many of the adfarms that host google advertisements. But yet despite this I fail to see why Microsoft or Yahoo would prove better, as both also suffer from corrupt corporate agendas. Rather than looking for another closed top-down corporate ’saviour’ to rescue us from the latest tyrant, I believe a bottom up emergent approach can prove not only successful, but also something that will become the dominant means of doing search.

Doesn’t this exist already?
Often when I come up with a good idea, I’m not surprised to find that something like it already exists.  However I can’t find anything quite like this idea at present, although some similar Peer-to-Peer based systems exist, such as FAROO and YaCy.  However I don’t see P2P as quite the solution, as this involves each user installing software on their own computer.  My proposed web based model lacks this requirement and thus has quite a few advantages, without precluding the idea of programs users can install that integrate searches from various sites.  Also, the above proposed model precludes the ability for malicious links to be injected, as webmasters who index such sites will unlikely be trusted by others, and users will likely not use engines that fail to exclude such links.

Reported a few years ago so some of you may have seen this. I hadn’t so I’m guessing a lot of you hadn’t either, so I’m posting the link…

[www.sciencedaily.com]

How do writers of extreme books, lyrics and other things feel when their words go out in the world? I can tolerate every writing because i feel that words are a part of reality and imagination. Maybe extreme writing releases the negative energy in words instead of actually hurting people. But not everyone shares my opinion and some people get easily offended by words. I also get offended sometimes but it lasts only for minutes (maybe sometimes longer) and i clear it away with meditation. There was a time when i was trapped in the world of words and i suffered all the time. It was horrible. Keeping that in mind i try to be careful what i write or speak, to avoid hurt or being hurt. Not everyone is prepared for total freedom of words. Even if everyone is prepared for it, there should be some universal culture code.

I bought "A Short History Of Nearly Everything" by Bill Bryson in the airport bookshop, on my way to Poland before Easter this year. It was almost "accidental" choice and it turned to be one of these books that change my attitude to reality.

Around two months ago I had discussion with my boyfriend about the modern world in general. We noticed that in our childhood we both felt as though the world was full of mysteries awaiting you "in every street corner" but with time there were less and less mysteries left until we started realising that there is no much more to be discovered.

I even think that my obsession with paranormal world and UFO was coming form the deep need to find the world, I saw when I was a child.

And now I found this book, which in very simple words describes the history of science, lives of scientists and summarises the whole of the modern knowledge about the world. Almost every chapter ends with a conclusion that we barely know anything. We don’t understand universe, we don’t know what is really hidden inside our planet’s core and its oceans, we don’t know why we are here and how long it would take before people can become extinct. The whole of our world is built of bacteria and our existence depends on them, but more than 90 percent of them were never seen or classified. We don’t know how the genes work and why they exist at all… Suddenly I feel as though I opened the door to some magical land which hides more mysteries that I could ever wish to discover!

The book lead me also to some reflection about magic and occult practices. As I mentioned previously, the book describes the history of science and many scientists. It turns out that many of them had strong interest in occult, including for example Newton who was fascinated in alchemy… ( I didn’t know that). It is also surprising how many of them could be almost geniuses in one subject and at the same time complete ignorants in the other…

Many people criticize Crowley for some of his writings and they assume that he couldn’t that smart after all. My theory now is that if science made so many mistakes in its quest to define the world, surely the same would apply to the occult world which by its nature deals with much more subtle matters. Humans are allowed mistakes and we shouldn’t discredit all of their work just because they were wrong is some places. At the same time we should be able to question things and if something doesn’t make much sense to us it is worth to explore it further. Maybe we can not see the whole picture of it or maybe it is all wrong. That’s why conservatism in magic is as bad as conservatism in science or religion. :S

I am not sure if I mentioned that before, but for a long time I was really obsessed with UFO and conspiracy theories. There were countless hours and days in my childhood when I tried to call aliens, I was going to famous "UFO seeing" spots and I have never seen a UFO in my life

Last month, actually during the Beltane weekend my uni friends went to Loch Ness to celebrate the end of the academic year. I didn’t go with them as I had another plan for Beltane (see my Beltane 2010 post).

And guess what? They have seen UFOs in there!! Here is the link to the newspaper that published an artlicle about them:

[www.highland-news.co.uk] Search for UFO news on the website ::o

Now I am really convinced that aliens don’t like me…

Diaspora – the privacy aware, personally controlled, do-it-all distributed open source social network

We are four talented young programmers from NYU’s Courant Institute trying to raise money so we can spend the summer building Diaspora; an open source personal web server that will put individuals in control of their data.

What is it?

Enter your Diaspora “seed,” a personal web server that stores all of your information and shares it with your friends. Diaspora knows how to securely share (using GPG) your pictures, videos, and more. When you have a Diaspora seed of your own, you own your social graph, you have access to your information however you want, whenever you want, and you have full control of your online identity. Once we have built a solid foundation, we will make Diaspora easy to extend to facilitate any type of communication, and the possibilities will be endless.

For a little more detailed explanation, checkout this blog post.

What is the project about?

We believe that privacy and connectedness do not have to be mutually exclusive. With Diaspora, we are reclaiming our data, securing our social connections, and making it easy to share on your own terms. We think we can replace today’s centralized social web with a more secure and convenient decentralized network. Diaspora will be easy to use, and it will be centered on you instead of a faceless hub.

Why are we building it?

This February, Eben Moglen, Columbia law professor and author of the latest GPL, gave a talk on Internet privacy. As more and more of our lives and identities become digitized, Moglen explains, the convenience of putting all of our information in the hands of companies on “the cloud” is training us to casually sacrifice our privacy and fragment our online identities.

But why is centralization so much more convenient, even in an age where relatively powerful computers are ubiquitous? Why is there no good alternative to centralized services that, as Moglen pointed out, comes with “spying for free?” Why do we keep our personal data in a thousand places? We have the technology, someone just needs to take the time to figure out how we can communicate smoothly and intuitively, without the hidden costs of “the cloud”. As good programmers, when we noticed that the application we need doesn’t exist, we set out to fill the hole in our digital lives.

Why do we need money?

We have a plan, a bunch of ideas and the programming chops to build Diaspora. What we need is the time it takes to iron out a powerful, secure, and elegant piece of software. Daniel, Ilya, Raphael, and Maxwell are all ready to trade our internships and summer jobs for three months totally focused on building Diaspora. We want to write code all the time, everyday. Once we have made our first solid iteration, we are going to release our code as free software so everyone can make Diaspora even better. $10,000 buys the software for everyone who wants to use it, forever. We think it can change the way people communicate and empower individuals to permanently take control of their online identities.

After we open source our source code, we hope to also provide a paid turnkey hosted service in the vein of WordPress.com to make it easy for people who want to use Diaspora, but don’t want to deal with the fuss of setting it up.* We will make it easy to export your data and configuration, so if you decide you want to graduate and host your seed yourself, you are free to do so at anytime.

Our goal is for everyone to have full control over their data and to empower people in to become responsible, secure, and social Internet dwellers. We believe offering this service will be helpful to non-technical users who are also worried about their data and privacy online.

Our Promise.
We promise to you that Diaspora will be aGPL software which will released at the end of the summer.

Want more info?
Check out our website for project updates, blog posts, pictures, and plans. More information is being added every day! www.joindiaspora.com

Check out more videos here.

Follow us at @joindiaspora Twitter or identi.ca

Want get Diaspora updates via email? Sign up here!
* This service will be available a few months after the end of the summer.

Project location: New York, NY

A few interesting thoughts from an article found at

http://jacksonville.com/opinion/blog/401574/abel-harding/2010-05-10/oil-spill-clean-doing-more-harm-good

“It is important to understand that oil spill dispersants do not in any way reduce the amount of oil spewing from its source nor do they eliminate oil from the environment. What these chemical agents are designed to do is alter the physical and chemical properties of the oil allowing it to sink further down into the water column.” 

As ProPublica.org points out, there are significant concerns that the treatment could severely harm the Gulf’s ecosystem, leaving dead fish in its wake.

The exact makeup of the dispersants is kept secret under competitive trade laws, but a worker safety sheet for one product, called Corexit, says it includes 2-butoxyethanol, a compound associated with headaches, vomiting and reproductive problems at high doses.

“There is a chemical toxicity to the dispersant compound that in many ways is worse than oil,” said Richard Charter, a foremost expert on marine biology and oil spills who is a senior policy advisor for Marine Programs for Defenders of Wildlife and is chairman of the Gulf of the Farallones National Marine Sanctuary Advisory Council. “It’s a trade-off – you’re damned if you do, damned if you don’t — of trying to minimize the damage coming to shore, but in so doing you may be more seriously damaging the ecosystem offshore.”

http://jacksonville.com/opinion/blog/401574/abel-harding/2010-05-10/oil-spill-clean-doing-more-harm-good

other interesting articles . . .

Chemicals Meant To Break Up BP Oil Spill Present New Environmental Concerns
http://www.propublica.org/article/bp-gulf-oil-spill-dispersants-0430

BP’s Oil-Dispersant Use Veers Into Uncharted Science (Update1)
http://www.businessweek.com/news/2010-05-12/bp-s-oil-dispersant-use-veers-into-uncharted-science-update1-.html

Is the BP Clean-Up Creating A Toxic Soup in the Gulf?
http://motherjones.com/blue-marble/2010/05/toxic-soup-gulf

What are we dumping into the Gulf to ‘fix’ the oil spill?
http://www.grist.org/article/2010-05-03-how-risky-is-the-dispersant-strategy-for-addressing-the-gulf-spi/

Find out more by doing your own search, I used “2-butoxyethanol gulf oil spill” to find these articles.

via http://itsgettinghotinhere.org/2010/05/08/national-day-of-action-night-of-mourning-against-offshore-drilling-friday-may-14/

Once again the fossil fuel industry has brought crisis to the Gulf Coast. Devastation of untold proportions spews non-stop from BP’s oil well as politicians try to save face with empty promises, and oil companies preserve their profits with PR campaigns. This catastrophic spill comes on the heels of Obama’s plan to expand offshore drilling. The price of burning fossil fuels is too high. From combustion to extraction the oil industry poisons our communities, destroys ecosystems, and destabilizes the climate. Now is the time to stop offshore drilling dead in its tracks and drive another nail into the fossil fuel industry’s coffin.

Take action Friday May 14 to demand:

-An immediate ban on all offshore drilling

-A rapid and just transition away from fossil fuels

-No bailouts for the oil industry. All recovery costs must be paid for by BP, Halliburton, Transocean and other implicated companies.

-The federal government must remove any caps on liability for oil companies.

-BP provides full compensation for impacted communities and small businesses.

-BP provides full funding for long-term ecosystem restoration for impacted areas.

-Oil companies operating in the Gulf fully fund restoration of coastal ecosystems damaged by canals, pipelines, and other industry activities.

Take action at:

-BP gas stations and offices

-Halliburton and Transocean offices

-Federal buildings

-Offices of members of Congress

-State government officials in states affected by Obama’s offshore drilling proposal.

-Critical Mass bike rides

-Vigils to mourn the unspeakable loss brought by this spill

-Get creative!