|
|
Bitmunk Transaction Platform Architecture OverviewFrom DBWiki[edit] IntroductionThis paper introduces a Secure File Distribution Network (SFDN) for the sale of digital media on the Internet by any participant of the network. The core architecture and techniques described in this paper have been implemented for Bitmunk - a legal, copyright-aware P2P marketplace for music, television and movies. The long-term goal of the SFDN is to catalog every human creation in existence that can be expressed by a digital medium. The SFDN is a file distribution system that appropriates royalties to copyright holders while allowing consumers to legally buy and then resell copyrighted works. By providing a system that rewards content creators, distributors and fans/resellers we plan to severely reduce content piracy for content creators while providing significant fair-use advantages to consumers. The distribution and protection mechanism is fundamentally different from every approach taken thus-far for digital media distribution. Unlike current solutions, the SFDN rewards all parties involved in the end-to-end media creation and consumption process. [edit] Bitmunk Network Technology Overview
 Bitmunk Transaction Platform High-level Network Diagram [edit] Bitmunk Network Technology Capabilities
[edit] BackgroundWhy do we need a legal, copyright-aware P2P distribution system? Digital e-commerce is a fairly new economic medium and is currently threatened by software piracy. Recent legislative changes, such as the Digital Millennium Copyright Act of 1998 [1], the extension of copyright protection, and the increasing support for digital rights management, all underscore the importance of digital media sales in our future economy. The greatest obstacle to the digital media industry is the development of a secure, online, open marketplace that protects the creator's intellectual property while rewarding a culture that supports the creation of art. Wide-spread piracy is a side-effect of the state of digital distribution today - making exact copies of any digital media is effortless and requires no attention to appropriate royalty reimbursement. The solution is a network that simultaneously protects and remunerates the intellectual property owners and acknowledges the rights of producers to be fairly compensated in a primary market, yet allows consumers the equal right to benefit from sales in a secondary market. It is estimated that the U.S. film industry loses more than $3B in revenue per year due to piracy [2], while the U.S. music industry loses more than $4B annually [3]. Clearly, piracy is a significant concern to content producers. The introduction of digital media and high-speed networks has aided the spread of piracy from an underground culture to a pop-culture, with an average of three million people illegally trading copyrighted works at any given time [4]. This proliferation of digital music piracy resulted in damages in excess of $1B during 2002 [5]. Since the creation of file-sharing networks such as Napster, Kazaa, Morpheus, Gnutella, and LimeWire, digital content creators have been fighting an increasingly difficult battle against digital piracy [6]. The Recording Industry Association of America (RIAA) and the Motion Picture Association of America (MPAA) represent content creators who are concerned about the protection of their creative works, while various corporations such as Napster, Inc., Kazaa BV, and StreamCast Networks, Inc. represent file-sharing systems that enable customers to share any digital media they wish. Both sides are desperately struggling for their respective positions with no meaningful solution to the piracy problem in sight. Meanwhile, consumers are harmed by recent legislation passed in response to pervasive digital piracy. Increasingly restrictive fair-use rights, digital rights management initiatives, and trusted computing initiatives restrict a consumer's option to truly utilize all the privileges of ownership to a purchased digital device or work. Most parties involved in digital media and the entertainment industries suggest that the flashpoint for digital piracy was the invention of the MP3 file format and the Napster file-sharing network. In the early 1990s, when the Internet was just beginning its meteoric rise into mainstream culture, transferring media files from computer to computer was cost prohibitive. Streaming media compression formats for multi-media had not reached widespread use; most media files were too large for the delivery medium. Equally important was the quality of most compression formats, which were not as robust as formats available today. The MP3 file format was invented in 1987 and patented in 1989 by Fraunhofer-Gesellschaft and Thompson Multimedia. It did not gain widespread acceptance as a streaming sound file format until 1997. At the same time, CDs were becoming the most popular sales medium for music, providing crystal clear playback in a format that would not degrade over time. These digital audio files were often quite large (e.g., a 5 minute song could require more than 40 megabytes), far too large to download in a reasonable amount of time even using today's standards. The subsequent MP3 format allowed a consumer to encode the 40-megabyte CD file into an MP3, which might be only 5 megabytes in size. The MP3 could then be decoded using a media player, producing a sound almost acoustically indistinguishable from the original. It was this breakthrough that made audio transfer and piracy feasible using low-bandwidth Internet connections. Many computer-savvy music listeners started digitizing their entire music collections, which offered the convenience of accessing their entire music collection without having to search through a stack of plastic CDs. Some listeners started to illegally trade their music over the Internet. However, the digital music piracy problem became epidemic when the first generation of file sharing applications started operating in late 1997. The most notorious of these was Napster. Napster is reviled by the professional digital content industry due to the mass-scale piracy movement that it started. While file-sharing networks fundamentally enhance freedom of expression, they also provide a mass conduit for piracy due to the anonymous nature of transactions. Napster was well aware that its networks were allegedly being used for piracy, and could have stopped it, yet did not attempt to do so. At its height, the file trading network had over 70 million unique users and facilitated more than 3 billion file downloads per month, most of which have been alleged as pirated material [7]. So, what made Napster such an effective application? The core benefit of any file-sharing network is that it enables relatively fast searches across the contents on the sharing network. A file-sharing network usually consists of millions of computers with a combined file pool of billions of files. The summed storage space and bandwidth availability of the combined systems are far greater than most large corporations could ever support. A user of the Napster system could search for a song encoded in the MP3 format and, within minutes, download it anonymously without paying a fee for the service or royalties for the downloaded file. Users could share their entire music collection and anonymously trade with others, resulting in the single largest, publicly-accessible database for music ever created. The fruits of this new file-sharing network have been countless lawsuits over intellectual property rights, freedom of expression, digital piracy and consumer content ownership [4,7,8,9]. These battles continue today, where the second generation of file-sharing networks, such as Kazaa, Grokster, Morpheus and others, have subsequently failed due to mass piracy on their networks, resulting in litigation by the RIAA and the MPAA. The digital content industries have chosen to solve the problem in several ways: litigation threats and action [9], consumer copyright education, digital guerilla warfare, intellectual property (IP) protection legislation, digital rights management, and more-secure computing initiatives. While some of these initiatives are needed and on target, the industry's main focus in curbing digital piracy maybe misguided, in that some of these initiatives have done more harm than good. Consumers have become increasingly concerned with the strategies of both the RIAA and MPAA, with some consumers reacting quite negatively to the infringement of their fair-use rights. The technology protection that content companies (i.e. those who produce digital media) are seeking is not likely to stop piracy or to be accepted by mainstream consumers. In fact, there has never been a digital rights management system that could not be bypassed. Piracy on this scale is a social problem just as much as it is a technological and financial problem. Thus any solution must be approached from social as well as financial and technological perspectives. [edit] Problems with current P2P technologyWhat is needed to create a secure P2P digital goods marketplace? There are currently four discrete entities involved in the digital media lifecycle: artists, publishers, distributors and fans. The core motivations for their behavior can be summarized like so: Artists: I would love to get as much exposure and sales as I can, but I want my work to be protected. Afterall, it is my creation and I need to feed my family. If there is a solution out there that builds my fan-base and allows me to make a living doing what I love, I'm on board. Publishers/Labels: We want to make all of our artists wildly successful - but more often than not we have 10 failures to every success and that success has to subsidize all of our failures. We want the artist to be protected as well as ourselves. Furthermore, making an artist succeed costs a great deal of money. Most of the money goes into publicity and promotion, without which, an artist cannot become a huge success. If we can build a system that lowers our risk and helps our bottom line while helping us decide who has a good chance of becoming successful, we would have no problem migrating to that system. Distributors: The markup that we are allowed to capture on each sale must pay for all of our failed merchandise. If there is a better way to maximize our profit, we're interested - especially if it lets us know what we need to be selling without spending a fortune on marketing research. Fans: I want to support the artist, but I'm not going to pay ridiculous prices for something I don't even know if I will like. P2P lets me try before I buy - so I don't get screwed over by a one hit wonder. I want really good value for the money I spend... and I want to try stuff out before I buy it. I want most of my money to go to the artist. The relationships described above are symbiotic, they are not top down or bottom up. Each player is intricately linked to each other - the proper solution, thus, cannot be top down or bottom up - it must be peer-to-peer. Before a peer-to-peer digital media distribution system can be developed, however, several problems with current P2P technology must be addressed: [edit] Problem: All transactions are anonymous.Would you give $5 to someone on the street that is selling a CD-ROM, claiming that it has 100 MP3 songs on it? More than likely not - mostly because you don't know who you are dealing with. This is okay, as long as there isn't any financial risk involved on your part - but put money into the equation, and P2P falls apart. There are certain places on the Internet where complete anonymity is preferable - financial transactions are not one of them. The SFDN replaces anonymous transactions with verified entity transactions. Both buyers and sellers on the system have real-world identities associated with their accounts; thus reducing fraud or making fraud/copyright infringements a more directly accountable event. Note, we are describing trusted anonymity - nobody knows your address or name when dealing with you on the SFDN. However, you know that they went through the same verification process that you did, and if something bad happens, you can identify them in the real world. [edit] Problem: Transactions do not allow royalty disbursements to be made.Non-trust metric, copyright-unaware P2P networks (Kazaa, Limewire, eDonkey, etc) operate on the simple notion of sharing. If you download, you should share - it is a very simple economy. While the concept of sharing for a greater good is a noble idea, it is quickly struck down when others are harmed by it. It has recently been shown that file sharing can have a positive impact on CD sales, but this is a rather round-about way of getting money to the artist (it goes through many middle-men, thus reducing the artist or label's take and giving the fan less value for their money). Thus a system that allows new intellectual property to be distributed must have a way of reimbursing the creator - if it doesn't, the creator has no incentive to create. The system must be balanced and self-sufficient. The SFDN maintains a database of all available creative works as well as records of copyright, royalty schedules, and applicable financial information for all those involved in digital media creation and distribution; thereby facilitating appropriate and timely reimbursement to all concerned parties. [edit] Problem: There is no social or financial cost for downloading material and not reimbursing the creator.To most people between the ages of 13 to 40, downloading copyrighted content feels as natural as breathing. In fact, most don't know what copyright law is, why it is needed, or how an artist makes a living in the first place. P2P downloaders know that they want something, its available on a network, and they can have it by performing a short download (instant gratification is a powerful motivator). Most take pride in their large collection of digital content, sharing it with peers to turn them onto new music, movies or TV. The largest problem here is not only that the artist is not getting paid, but that it has become socially acceptable to do so. So, what measures are needed to make it socially unacceptable to not reimburse the artist? What does this micro-economy need to sustain itself indefinitely? What are the fan and artists' motivations?
The solution lies in a non-restrictive method that lets people share, but doesn't lock content down via Digital Rights Management technology. What is needed is an open watermarking standard on top of an open media standard, such as MP3, Ogg Vorbis, MPEG video, etc. The fans must be able to decide which file formats they want to work with. Therefore, the SFDN tags all media that is purchased by consumers through the network with a watermark enclosing the buyer's identification information using non-invasive/non-restrictive methods. Note that this is fundamentally different from digital rights management technology, as the buyer's fair-use rights are not infringed upon. However, with these expanded use rights come expanded responsibilities. Exercising fair use rights, anybody can share their content with friends that they trust. If digital media is found on an illegal file trading network, the consumer/owner of that file could be identified and could be held liable for the copyright violation, thus, giving consumers a social and financial incentive to protect their media from piracy, but promote sharing between trusted friends. [edit] Problem: There is no social or financial advantage for not pirating material.There is one very important individual that is left out in the creation of art - that is the end user - the fan. Traditionally, the sale of art has gone from:
The artist and fan are the ones that pay for the middle-men, a process which is always inefficient. In fact, the middle-men could capture a much higher ROI if the process was made more efficient. The question is: How do you make this distribution process more efficient and increase ROI? How do you cut piracy? The answer lies in the forgotten individual. The SFDN discourages piracy by providing social and financial advantages for the buyer, namely the ability to re-sell the digital media via the network to other buyers on the system. Many view the exchange of digital media as a cost-less transaction; it is not. Bandwidth, transaction processing and disk storage are all expenses to the provider, therefore all sellers on a network should be allowed to charge distribution fees, separate from the creator's royalty fees. This network also allows any buyer to become a seller, thus creating social and financial micro-economies. Users of the network have no incentive to pirate the material, if they can sell the same material somewhere else. Ultimately this should create an immense distribution network of inexpensively produced, high-quality, widely available, legal digital media. There is a clear reason for fans to use this system over one that just treats them as a consumer. [edit] Problem: True micro-payment and distribution systems are not readily available.There is a high cognitive load placed on anybody purchasing something that costs a few cents. "Do I really want to click on this button, enter my username and password, and deal with the hassle of spending 10 cents?". The counter point is that you cannot charge $1 for every song - some are worth more than others, and it is the market that should decide and not the corporation. Therefore, any system that distributes digital content must have a very flexible micro-payment system at it's core, but not concentrate on selling items for 10-20 cents. It is much easier to sell a bundle of 10 songs for $2, than to sell each one individually for 20 cents. At the core of the sale should exist a very flexible micro-payment system, but the end user should see bundles of items - as it is much easier to decide to buy an album, than each particular song inside that album. This realization is the reason most micro-payment systems have failed thus far - the item to be purchased must be worth the trouble of spending your money in the first place. The SFDN creates a micro-payment-based accounting system that should compensate for the fact that digital media files will never sell for the same amount as their physical counterparts. For example, music selections might garner $0.15-$0.75 per sale, a television episode around $2.00, or a movie as much as $8.00 per sale. The SFDN will support, as part of its core architecture, a cost-effective system for processing these small monetary amounts as well as a way of bundling transactions together to reduce the cognitive load on the end user. [edit] Problem: Technology for secure financial transactions with unknown individuals is under-developed.Most financial transactions on the Internet are based on real-world banking transactions - systems developed during a time where computers did not exist. It is now possible to have far more complex transactions between multiple legal entities - however, the financial markets have such a large deal of cruft that building such a system into the current one, or migrating to a new system is nearly impossible. Thus a new system must be layered on top of the old one - one that allows mass-distribution of small monetary amounts to persons that are not aware of each others existence. One where you do not have to be a full time merchant to accept electronic financial payments from others. It is possible to create a very secure, open standards, communication mechanism coupled with a trust-metric-based system to achieve a trustworthy financial transaction system for independent sellers and buyers that have never met each other. The SFDN system employs a secure transaction protocol so that both parties are ensured of a legitimate and legal transaction, since there is the potential for dishonest buyers and sellers on the system. A method of rating buyers and sellers is also provided giving both parties the power to rate each other in accordance with their transaction experience. Sellers or buyers that have excessive negative feedback will be removed from the network. Since each participant in the SFDN would be associated with a real-world identity, everyone would then have an incentive to act responsibly when transacting business on the network. [edit] System Architecture OverviewEnough talk, how does this thing work? The entire SFDN system is designed to be platform and operating system agnostic. The current system is written in Java to guarantee compatibility with all systems on launch. Platform-specific C/C++/MFC/C#/etc clients and sales servers are being developed. The communication mechanism between system components is done via XML/SOAP at the moment and will grow to encompass XML-RPC and other open communication methods in the future. You can download the WSDL specification for the SVA as well as the WSDL specification for the Sales Server. Please do not begin developing clients and sales servers until the API has stabilized and we have released documentation - while we're comfortable with the current API, things might need to change slightly once the system has been under considerable load for extended periods of time. The entire back-end for the SVA is implemented using open-source technology - Apache, PHP, Python, MySQL, Lustre FS, and Java running on top of Debian Linux. A very sincere thank you to those developers that have contributed to these and other open source projects - we are here today because we have stood on the shoulders of giants. Our plan is to reciprocate in turn by releasing several open source, license and royalty-free libraries and tools for inter-operating with the SFDN. Understanding the Core Operations of the SFDN Where does it all begin? Creators must first register their creative work with the network, claim ownership of it and clear it for sale on Bitmunk before anybody can buy or sell the creative work. This entire process is handled via the Bitmunk website. The registration and end-user experience is explained a bit more in depth in the following articles (which you may want to familiarize yourself with):
[edit] The SVA Databases (registration, cataloging, and search)Central indexing for fast searches and security management A distributed database system has been implemented for distributed registration, cataloging and fast searching. The current database holds close to 4,240,000 unique creative works (mostly music), but will eventually encompass any creative medium that can be digitized (movies, television, books, etc). The distributed database system can also index every seller and buyer on the network so that they may find each other more easily. A little math will show that we are talking about rather large distributed database: (2,000,000 active creative works) * (1,000,000 sellers, each selling 0.001% of the total available catalog) * (1.5, average ware permutations per catalog) = 3,000,000,000 (3 billion) catalogued works for sale (roughly 6TB of searchable information). The system is designed to scale almost linearly to a value that is far greater than this calculation. The databases also hold encrypted security information for the parties that are buying and selling from each other on the network, including trust metrics for everyone. Unfortunately, it is impossible to open the system up without creating security vulnerabilities - thus we cannot discuss the security measures or the internal SVA architecture in great detail. However, the communications protocol and functionality of buyer and seller clients on the network are open and will be discussed in the next section. [edit] Encrypt All CommunicationSecurity is of top-most concern in a financial system All communication between players on the SFDN is encrypted using the industry standard Secure Sockets Layer protocol. All messages are digitally signed and verified (using standard X.509 public/private key technology). This allows everyone on the network two very important securities:
[edit] The Contract Creation, Negotiation, and Validation PhasePlaying fair is a complex process We assume that all software is setup and running on the network. Another assumption made for the rest of the white paper is that the SVA is operating with many buyers, sellers and creative works authorized for sale. [edit] Seller Catalog Registration Seller uploads catalog to the SVA. A catalog is composed of every digital file that a seller wants to sell on the SFDN, pricing information and descriptions of each ware is included in the catalog. Every seller has one catalog which is composed of one or more manifests. Each manifest may include one or more wares. When a seller uploads a catalog to the SVA, all of the items are listed for sale immediately. Whenever anybody does a search for a particular creative work that is contained in the catalog, the sales server and pricing information is displayed to the buyer. The catalog is in XML format and is digitally signed by the seller. [edit] Intent to Purchase Buyer notifies seller of intent to purchase. After somebody on the network has found something that a seller has listed (via the search service), they will send an intent to purchase. This is the beginning of the contract negotiation phase. After consulting the SVA, a sales server will fill out a purchase contract and send it back to the buyer. The purchase contract is in XML format and is digitally signed by the sender. [edit] Contract Negotiation - Agreement to Contract Terms and Contract UploadThe buyer reads the purchase contract sent from the sales server. If everything in the contract is agreeable, both the buyer and seller will upload the purchase contract to the SVA. The SVA will check to see if both contracts match and then store the contract in it's database. [edit] Upload the Digital File to the Buyer and Decryption key to SVA First phase of a typical Bitmunk transaction. Once the final contract has been uploaded to the SVA by both the buyer and seller, the buyer will request the file from the sales server. The sales server will lock (standard DES-based block encryption) the file and send the locked file to the buyer. It will also send the unlock key (standard DES-based block encryption password string) to the SVA. The SVA will only give the key to the buyer after the buyer has paid the seller and all payees listed in the transaction contract. [edit] Payment Distribution Second phase of a typical Bitmunk transaction. Once the buyer has downloaded the file, they authorize payment to everyone in the contract. They get the decryption key in return for their authorization of payment and unlock the file that was sold to them. The SVA distributes payment from the buyer's SFDN financial account to all the payee SFDN financial accounts (as well as everyone else that was listed in the transaction contract). This method ensures that only the people that have been authorized by the buyer will get paid. [edit] Parting wordsWhere to go from here... Developer API and documentation will be released once we are certain of the interfaces into the SVA that we want. Security is the number one priority - thus it is difficult to put a time frame on it. The SFDN is in beta currently and has downloadable software for those that are interested in using the system. You must be a registered and verified user on the system to buy or sell, it is a requirement of every individual on Bitmunk. For further discussion, there are forums available - please post there if you have any concerns or comments about the system. [edit] References and Related Work[1] Additional Provisions of the Digital Millennium Copyright Act, U.S. Copyright Office, 8 January 2003, http://www.copyright.gov/title17/appendixv.pdf [2] Motion Picture Association of America Anti-Piracy Page, 1 June 2003, http://www.mpaa.org/anti-piracy [3] Effects of Music Piracy, Recording Industry Association of America, 25 May 2003, http://www.riaa.org/Protect-Campaign-3.cfm [4] International Federation of the Phonographic Industry Music Piracy Report, June 2002, http://www.ifpi.org/site-content/library/piracy2002.pdf [5] Borland, John , Music industry: Piracy is choking sales , CNET News.com, 9 April 2003, http://zdnet.com.com/2100-1105-996205.html [6] Gundersen, Edna, Record industry faces its doom , Gannett News Service, 12 May 2003, http://www.floridatoday.com/!NEWSROOM/peoplestoryA52301A.htm [7] King, Brad, Napster Is Alive, Alive , Wired Magazine Online, 26 June 2001, mp3/0,1285,44809,00.html">http://www.wired.com/news/mp3/0,1285,44809,00.html [8] Naraine, Ryan, RIAA Sues College File-Swappers , Internet.com, 4 April 2003, http://boston.internet.com/news/article.php/2175901 [9] Borland, John, RIAA Sues Campus file-swappers , News.com, 3 April 2003, http://news.com.com/2100-1027-995429.html [10] Legal Issues Don't Hinder American Downloaders , TEMPO 1st Quarter 2003, TEMPO: Keeping Pace with Online Music Distribution, 14 March 2003, http://www.ipsos-pa.com/gotous.cfm?id=1763&Region=us&PDF_name=mr030314-2revis.pdf [11] Gaither, Chris, Another spin on digital music , Boston Globe, 26 May 2003, http://www.boston.com/dailyglobe2/146/business/Another_spin_on_digital_musicP.shtml [12] Texas A&M Rules For Responsible Computing, Texas A&M University Rules, 27 August 1997, http://www.net.tamu.edu/network/policy/dsl-staff.html [13] Verizon Terms of Service, Verizon Online, 23 May 2003, http://www2.verizon.net/policies/tos.asp [14] AT&T Broadband Internet Subscriber Agreement, AT&T Broadband Internet, 1 June 2003, http://help.broadband.att.com/faq.jsp?content_id=973&category_id=34 [15] Spanier, Graham B., Testimony to the Subcommittee on Courts, the Internet, and Intellectual Property, Committee on the Judiciary United States House of Representatives , 26 February 2003, http://www.psu.edu/ur/GSpanier/testimony/p2p.html [16] Corbett Broad, Molly, Testimony to the Subcommittee on Courts, the Internet, and Intellectual Property, Committee on the Judiciary United States House of Representatives , 26 February 2003, http://www.house.gov/judiciary/broad022603.htm [17] U.S. Music Downloaders Prefer A Pay-Per-Download Transaction Over Current Subscription- Based Offerings , TEMPO 4th Quarter 2003, TEMPO: Keeping Pace with Online Music Distribution, 25 September 2002, http://www.ipsos-pa.com/gotous.cfm?id=1631&Region=us&PDF_name=mr020925-1.pdf [18] iTunes Music Store Tops Two Million Song Downloads , Apple Press Releases, 14 May 2003, http://www.apple.com/pr/library/2003/may/14musicstore.html [19] McCord, Alan, EDUCAUSE 2001 Bandwidth Management Roundtable Pre-Conference Survey Results , University of Michigan, 25 October 2001, http://www-personal.umich.edu/~amccord/bandwidth-survey-results.pdf [20] Carlson, Scott, A President Tries to Settle the Controversy Over File Sharing , The Chronicle, 23 May 2003, http://chronicle.com/free/v49/i37/37a02701.htm [21] Post-Secondary Education Information , Digest of Education Statistics, March 2002, http://nces.ed.gov/pubs2002/digest2001/ch3.asp#1 [22] The Research Division at Virginia Tech, Virginia Tech, 23 May 2003, http://www.research.vt.edu/ [23] University at a Glance, Virginia Tech, 21 June 2002, http://www.vt.edu/aboutvt/universityataglance.html [24] File-Sharing And CD Burning Remain Steady In 2002 , TEMPO: Keeping Pace with Digital Music Behavior, Ipsos-Reid, 20 February 2003, http://www.ipsos-reid.com/pdf/media/mr030220-1.pdf |
