MP3-Files from www and distributed link database

[ormm]

Open letter to the developer and supporters of free p2p technologies.

I am developing a search engine spidering www for ogg and mp3 files, at the time I am writing this is c. 100 000 files indexed. The project is named "It is Music" see: http://www.itismusic.org

This is a brief discussion of ways to distribute this index in an unhierarchical way built over existing p2p networks, it is submitted here as a request for opinions on how it could be implemented without violating and/or disturbing protocol specifications and usability of the networks.

The motivation for implementing this is that many unsigned artists releases their music on the internet but those files are rarely available at p2p networks. This decentralized design is good in contrast to central servers for ensuring it will remain free. It will take too much computer power to provide such service to the whole worlds p2p-networks. Only a big company concerning more about money than actual usability have funds to realisticly set up such central server system.

The spidering and updating of index is centrally made by our servers, with a program released under GNU/GPL at sourceforge. Spidering could be done in an unhierarchical way but we insist that p2p-clients must remain free from such addons and only implementing things that directly benefits the user. Therefore do we indent to do so in the future but the result and the tools for spidering will be available under GNU/GPL and free for any one to copy or modify.

An conceptual approach on the problem:
The index is split in many small files (ex. 100 which will take c. 300kb each in compressed state), the index is ordered by the artist name and the client searching for an artist downloads the meta-file containing an url to it and additional redundant information about other artists. The client shares that meta file and makes in this way it available for more users. The wanted targetfile is downloaded via http. This layout with many small files ordered by artist name and distributed randomly makes it impossible to search for a particular song without knowing the artist but I consider it appropriate since in most cases is the artist known and it is also the most common subject in querys over p2p networks.

Inconvenience
This might cause a dissonance with clients that not is designed to handle this distribution of meta data, since these files containing the index will be viewed by people browsing the host expecting to find "real" files and not a set of meta data.

Vulnerability
This system is vulnerable to erroneous data and spam, but it is more a question about the p2p concept than this way of using it. A correct implementation of these ideas will not effect client security since it is built over and not within the networks.

Worth to consider implementing or not?

Sincerely Johan Mattsson

You can find a part of the index here: http://www.itismusic.org