How hard can it be?
Well for a corporation very hard. But fortunately my solution to this problem that seems to be vexing everyone lately doesn’t rely on a corporate sponsor, and places the future of web search in OUR hands.

My vision of search is that each website takes a small share of responsibility, indexing sites that website finds relevant and/or useful and sharing its results with other search engines that request the info via a standard API, much like blogs share feeds using standards such as RSS and ATOM. Each webmaster can control how much weight to give to each set of imported results in comparison to sites it indexes itself and how the results are merged. I have the following basic vision, but as different developers get involved in ironing out the final details of the standard things may change. None the less the basic principles of my idea are as follows:

API:
Each search engine should accept requests in either/both http and/or https, probably with search parameters passed in the query string. The results then get sent back in an standard XML format, which may look something like:

<results source="http://example-search-engine.com">
	<site weight="200" url="http://example-result.org" title="Example Results">
		<favicon url="http://example-result.org/favicon.ico" />
		<page weight="150" url="http://example-result.org/page1.php" title="">
			<match><title>This title contains the search terms</title></match>
			<match><p>This paragraph contains one of the search terms</p></match>
			<match><img src="someimage.jpg" alt="This image description contains one of the search terms" /></match>
		</page>
		<page weight="150" url="http://example-result.org/page2.php">
			<match><p>This paragraph contains one of the search terms</p></match>
		</page>
	</site>
	<site ...>
		...
	</site>
</results>

CMS:
With the API open, different CMS systems can be developed that both index selected sites and process results from other search engines that implement the API. Each CMS can process the results in their own way, and allow the webmaster to configure various options in their own way also. For example, they can either integrate imported results, show them separately in a side panel or some mixture of the two depending on source. They can determine whether to show multiple results for one site or whether to group them. They may be configured to show country of the webserver of each result, or warnings if they appear on a malware list, or any other number of options. They may have options for doing image search, or map results etc, linking to whatever service they choose for these specialist searches.

A basic CMS may involve no login at all, with configuration handled via a file, whilst others may implement the API as a plugin to existing CMS systems such as Wordpress, Elgg, MediaWiki or whatever. Some may allow each logged in user to configure their own preferences and weightings, suggest relevant sites to index, report sites as spam, etc.

Plugins:
More complex dedicated search systems may themselves allow plugins or feature advanced options. This may include scraping results from existing search engines such as Google, Bing and Yahoo, in the manner that <a href=”https://scroogle”>Scroogle</a> currently does with Google results. Again these results may have weightings applied to them by the webmasters.

Conclusion:
A distributed solution to search means we don’t have to have massive datacenters under our control in order to compete with Google, each of us can take responsibility for indexing portions of the web important to ourselves, and share that data with each other freely for the benefit of all. To make a distributed system work we don’t need to abandon the huge stores of data provided by google until we are ready, we can in the mean time scrape from them, but ultimately a distributed system, based on qualified trust, should prove better at promoting sites that are useful over sites with good SEO but useless content.

Google is becoming less and less useful as each search tends to produce 100 sites all with the same article scraped from wikipedia, interspersed with adfarms, and I can’t help think such sites wouldn’t make it to prominence in the distributed model, as it won’t be in any webmasters interest to index useless sites. Unlike google who actually profit from many of the adfarms that host google advertisements. But yet despite this I fail to see why Microsoft or Yahoo would prove better, as both also suffer from corrupt corporate agendas. Rather than looking for another closed top-down corporate ’saviour’ to rescue us from the latest tyrant, I believe a bottom up emergent approach can prove not only successful, but also something that will become the dominant means of doing search.

Doesn’t this exist already?
Often when I come up with a good idea, I’m not surprised to find that something like it already exists.  However I can’t find anything quite like this idea at present, although some similar Peer-to-Peer based systems exist, such as FAROO and YaCy.  However I don’t see P2P as quite the solution, as this involves each user installing software on their own computer.  My proposed web based model lacks this requirement and thus has quite a few advantages, without precluding the idea of programs users can install that integrate searches from various sites.  Also, the above proposed model precludes the ability for malicious links to be injected, as webmasters who index such sites will unlikely be trusted by others, and users will likely not use engines that fail to exclude such links.

After writing my recent note concerning the 15 Books that I am most aware of having influenced me (not the same as my favourite 15 books incidentally), I have decided to write deeper reviews of some of those books, and to start with I have chosen The Apophenion: A Chaos Magic Paradigm by Peter J Carroll, mainly because I have just finished reading it.

I have to admit I wasn’t sure I’d enjoy this book, as I have found Carroll’s work a little too reductionist at times.  On occasion I did find myself unhappy with this book for precisely this reason, a prominent example being his insistence that magic tends to be about enchantment and divination, with invocation, evocation and illumination only serving to aid these two goals.  With these defined as obtaining information or encouraging events to happen ‘by non-ordinary means’, one could be tempted to reduce things further and define magic as the idea that one can get things by non-ordinary means (whether events, information or anything else), but this could imply a tendency to control.    These abilities can cause disaster for an individual that fails to use them wisely.

Not that I’m implying such a lack of wisdom in Carroll, he has hinted at its importance in earlier works (such as the combined power-wisdom catastrophe charts in the back of Liber Kaos), just pointing out that he seems to take it as read in this work.

I didn’t find the balance between physics and magic problematic, indeed mostly inspirational, although I did find myself skimming one of the more science based appendices for lack of obvious application for such ideas in my present situation.

These minor criticisms aside I have found great inspiration from this book, to the extent I feel a renewed connection to the Chaos Magic current after having abandoned it for a few years.  I once again understand things I had for a time forgotten or misunderstood.  For example I had always identified with Carroll for favouring an ‘animist’ perspective over either a ‘materialist’ or ‘transcendental’ one, but during my disillusionment with Chaos Magic, tended to see it as ‘too materialist’.  Carroll’s writings on Pan-psychism in this book have renewed the understanding and identification, and built upon them.

However the real gem for me, perhaps demonstrating a tendency of mine towards the Pantheistic end of the Magic spectrum (as opposed the to Science end), has been the discovery of the twin Goddesses of Apophenia and Pareidolia, the former of which gave her name to this book.

This work should appeal to all those in the chaos current that can stomach, understand or gloss over the science bits, but could enjoy broader appeal for those that practise magic of whatever label. Indeed I would include this work in a recommended reading list of how to practise magic if I chose to create one.

This is a response to Seani Fool over at PagAnarchy.net who wrote a blog listing his 15 books that most shaped who he is today.  Amongst others, he has asked me to share the 15 that most influence me currently.

1. The Apophenion: A Chaos Magic Paradigm by Peter J Carroll.  I’m only half way through this book so I’m not sure if I should count it yet, but since this is about what is influencing me right now, and through my current action of reading it and enjoying it, it is occupying a lot of my thoughts.  So far I think this is Carroll’s best book to date.  It seems to expand a lot upon some of the concepts hinted at in PsyberMagick, and present them in a much more lucid and coherent arguement.  It also ties various threads together in a way that appeals to the way I think.

2. Promethea by Alan Moore.  I don’t care if it is a graphic novel (or more a series of graphic novels), it continues to inspire.

3. The Invisibles by Grant Morrison.  Same applies to this one.  I don’t care if Grant and Alan apparently don’t care much for each other, I’ve never met either of them, so its just about appreciating their respective creations.

4. Visual Magick and Seidways by Jan Fries.  Okay, I know that’s two so I’m cheating a bit.  But I think these two books compliment each other so well that I think they could be combined into one big volume.  Possibly an even bigger one with Hellrunar thrown in for good measure.  Reading both these books changed the way I performed and thought about magick forever, and helped liberate me from thinking of myself exclusively as a Chaos Magician.

5. PsyberMagick by Peter J Carroll.  If I hadn’t read this book, I probably wouldn’t have signed up to the Illuminates of Thanateros all those years ago.  As a result, reading it shaped the pretty much all the path of magic I have tread ever since, including the formation of KIA.  It was the first Peter Carroll book I read to, didn’t read the others until after I was in the IoT.  I loved the humour in it, and enjoyed the format even though I was yet to read Crowley’s Book of Lies at the time.

6. BLAST your way to MegaBuck$ with my SECRET sex power formula! and other reflections upon the spiritual path by Ramsey Dukes.  This book was my introduction to Duke’s theories, including Johnstone’s Paradox.  My life would never be the same again.  I still think every so often about recreating an old command line program I made for doing magick by hacking into the computer that controls our universe.  It worked a bloody treat.

7. Paradoxical Emblems of D A Freher.  Reading this book caused me to make a choice.  That choice changed my life and freed me from certain bad decisions I’d made in the past and certain undesirable associations I’d made.  But not in the way that it made me think about those things.  No.  In the way that reading that book is an act of magick and the choice you make when reading it plants a seed.

8. Prometheus Rising by Robert Anton Wilson.  Doesn’t guarantee you won’t get brainwashed again, but makes it a lot harder for someone to do so over a sustained length of time.  See through the bullshit of those who would control you.  Whoever they are.

9. The Devil and the Goddess by Gyrus.  Proving that a book doesn’t need to have many pages to be brilliant.  This book combined with Seidways by Jan Fries to really inspire me thoughts on the importance of serpent imagery in magick.

10. Anarchism, arguments for and against by Albert Meltzer.  Another little booklet that has greatly inspired.

11. Spectacular Times.  A series of booklets.  About seeing through the lies and propaganda.

12. The Origin of Consciousness in the Breakdown of the Bicameral Mind by Julian Jaynes.  Not the definition of consciousness I feel comfortable, but other than getting into an arguement about what consciousness ‘is’, a damn good hypothesis about the way certain mind phenomena came to be.

13. Snowcrash by Neal Stephenson.  A virtual reality internet, a hacker hero with deadly martial arts skills, futuristic skaters that harpoon cars to hitch rides and a bad guy who has a nuclear warhead in the sidecar of his motorbike.  Strange religious cults, babalonian mythology, glossolalia, the Mafia and an America so taken over by corporations the state barely exists.  Could a be kick arse movie if they did a good job of it.  A rollercoaster of a read.  Damn good author.

14. Principlia Discordia.  Fnord.

15. The Way of the Sacred by Francis Huxley.  The common currents and themes of the worlds religions and mystery traditions exposed.  Another book with a lot to say about serpents in symbolic, mystical and mythological senses.

I have just finished reading my first ever on-line book from the Project Gutenburg library. Well, more like a pamphlet really, but inspiring none the less. It was Open Source Democracy by Douglas Rushkoff. I came across it searching on the title. I’d never actually heard of it, and although the authors name is familiar to me, I can’t actually remember where I’ve come across it in the past. Probably several places.

His argument is about how new knowledge emergent, or bottom-up, organisation, such as one might find in chaos mathematics, the behaviour of coral colonies and perhaps more importantly considering the pamphlet title the Open Source Software movement, will eventually impact and revitalise democracy, returning power and participation to the people. This knowledge will in largely be experiential. In that we as individuals have slowly gained control over the spectacle of media. With the advent of of television, we were passive viewers of a strange ‘magic’. Remote controls, video recorders, camcorders all played their part in giving us more control and power over what we watched. Finally video games, computers and the internet played their share in reducing our attention to corporate and government controlled centralised programming and allowed us to use media as a means of communication. As people get used to the participatory nature of the web, they will start to realise how unresponsive the current democratic system is. This, he argues, is why participation in elections are falling. Apparently though, people will start to demand a more participatory democratic experience, rebuilding it in such a way as to make it more responsive.

The actual argument is slightly more complex than my brief summary, and well worth a read in full if you have time. It is not my intent to reproduce it here. I have my own argument to add…

The reason I was searching on the term was because I had an idea. And the term ‘Open Source Democracy’ summed up that idea. I wanted to see if anyone had already thought of it. Rushkoff is heading in the right direction, but falls short of the idea that came to mind. Although I might be one of the people he mentions that will rewrite the rules. That’s not a role I object to playing! So I shall put my idea out there right now. The sooner people start building towards the realisation of this idea the better really…

My idea was at first a reaction against the take over of democracy by corporate closed source electronic voting systems, with no paper trail, verifiability or accountability. Of course, being a software engineer, I realised the problem was not with electronic voting, but with trusting the building and running of electronic voting systems to closed source and unaccountable corporations. After all, with old paper voting systems, we wouldn’t have trusted a private company to count the votes in secret. Yet this is effectively what we allow electronic voting companies to do today. Whether they have actually abused their position yet is besides the point. They shouldn’t be given the power to do so in the first place.

Why would an Open Source e-voting system be different? For one thing, it would allow the public to see the source code and submit improvements, meaning better security, ensured anonymity of voters, and guaranteed accuracy. Any flaws in the system would be spotted and corrected without vested interests hiding the flaws for fear of losing profits.

But why stop there? Such a system has the potential to do so much more than simply recreate the paper based system. We could rethink democracy and make it much more responsive and participatory. Here is my idea for a new democratic system:

1. Every member of society can register as a voter, and must be uniquely identifiable to prevent voting fraud from duplicate voting counts.

2. Voters can also register as a candidate, with their own personal manifesto, and if they have them details of party allegiances.

3. Voters can register and change their support for candidates at any time. Via a secure website systems, local public booths or where available via a traditional paper ballot.

4. The position of every candidate is clear at every stage of the voting process, so voters can see whether they feel the need to back a different horse.

5. At the beginning of each month, week, day or even hour depending on the model required, the top n candidates are awarded seats, where n is the number of seats.
6. Seat holder with the most backers gets to propose the agenda for the next debate, and the other seat holders are given a set amount of time to prepare counter arguments before the debate starts.

7. After all the debate, amendments to the proposal by the lead seat holder must be declared, and each successive seat holder may propose an alternative option to the proposal.

8. Seat holders then vote on the proposal and the alternative options, using a single transferable vote.

9. Each seat holder represents the total backers they currently hold, which may be different from the total they had when they were first selected. In this way, everyone gets to vote on each proposal, not just the seat holders.

Well, at the moment it is little more than a pipe dream. And I would add some caveats to prevent the system from being taken over by tyrants. This reflects that I do not stop at support for Open Source, but take things one step further by supporting the principles of the Free Software Foundation.

First, all truly democratic systems have limits on the power of the government over its people. Most people recognise freedom of speech as an important cornerstone of democratic liberty. However, I extend this to suggest complete individual sovereignty in matters of mind, body and soul. Not only should it be beyond the remit of government to violate an individuals sovereignty, it should be considered treason against democracy for them to make an attempt to do so, resulting in an immediate ban from candidacy at the very least.

Of course, it would be good to see this system developed and tested in experimental communities before attempting to introduce it for a real government, so that any flaws in my proposal or the initial implementations of it, can be ironed out. But I think I have come up with a basic blue print for advancing on the road to a better and freer society that early democratic efforts set us upon. Later I would like to see the system tested in a real governmental situation. With all the controversy over the House of Lords reforms in the UK, our parliaments second house might well prove to be the ideal testing ground.

The UK government has already started the Open Source Academy to promote the use of cost saving Open Source Software in local government and with the Conservative Party already promising a level playing field for Open Source Software in the UK, the early signs of political momentum building in this direction are already there. But the nature of emergent behaviour politics is such that we have to build it from the bottom up rather than waiting for the powers that be to do it for us.