The Wayback Machine is a digital archive of the World Wide Web and other information on the Internet created by the Internet Archive, a nonprofit organization, based in San Francisco, California, United States.
Video Wayback Machine
History
The Internet Archive launched the Wayback Machine in October 2001. It was set up by Brewster Kahle and Bruce Gilliat, and is maintained with content from Alexa Internet. The service enables users to see archived versions of web pages across time, which the archive calls a "three dimensional index".
Since 1996, the Wayback Machine has been archiving cached pages of websites onto its large cluster of Linux nodes. It revisits sites on occasion (see technical details below) and archives a new version. Sites can also be captured on the fly by visitors who enter the site's URL into a search box. The intent is to capture and archive content that otherwise would be lost whenever a site is changed or closed down. The overall vision of the machine's creators is to archive the entire Internet.
Information had been kept on digital tape for five years, with Kahle occasionally allowing researchers and scientists to tap into the clunky database. When the archive reached its fifth anniversary, in 2001, it was unveiled and opened to the public in a ceremony at the University of California, Berkeley.
The name Wayback Machine was chosen as a reference to the "WABAC machine" (pronounced way-back), a time-traveling device used by the characters Mr. Peabody and Sherman in The Rocky and Bullwinkle Show, an animated cartoon. In one of the animated cartoon's component segments, Peabody's Improbable History, the characters routinely used the machine to witness, participate in, and, more often than not, alter famous events in history.
Maps Wayback Machine
Technical details
Software has been developed to "crawl" the web and download all publicly accessible World Wide Web pages, the Gopher hierarchy, the Netnews (Usenet) bulletin board system, and downloadable software. The information collected by these "crawlers" does not include all the information available on the Internet, since much of the data is restricted by the publisher or stored in databases that are not accessible. To overcome inconsistencies in partially cached websites, Archive-It.org was developed in 2005 by the Internet Archive as a means of allowing institutions and content creators to voluntarily harvest and preserve collections of digital content, and create digital archives.
Crawls are contributed from various sources, some imported from third parties and others generated internally by the Archive. For example crawls are contributed by the Sloan Foundation and Alexa, crawls run by IA on behalf of NARA and the Internet Memory Foundation, mirrors of Common Crawl. The "Worldwide Web Crawls" have been running since 2010 and capture the global Web.
The frequency of snapshot captures varies per website. Websites in the "Worldwide Web Crawls" are included in a "crawl list", with the site archived once per crawl. A crawl can take months or even years to complete depending on size. For example "Wide Crawl Number 13" started on January 9, 2015, and completed on July 11, 2016. However there may be multiple crawls ongoing at any one time, and a site might be included in more than one crawl list, so how often a site is crawled varies widely.
Storage capabilities
As of 2009, the Wayback Machine contained approximately three petabytes of data and was growing at a rate of 100 terabytes each month; the growth rate reported in 2003 was 12 terabytes/month. The data is stored on PetaBox rack systems manufactured by Capricorn Technologies.
In 2009, the Internet Archive migrated its customized storage architecture to Sun Open Storage, and hosts a new data center in a Sun Modular Datacenter on Sun Microsystems' California campus.
In 2011 a new, improved version of the Wayback Machine, with an updated interface and fresher index of archived content, was made available for public testing.
In March 2011, it was said on the Wayback Machine forum that, "the Beta of the new Wayback Machine has a more complete and up-to-date index of all crawled materials into 2010, and will continue to be updated regularly. The index driving the classic Wayback Machine only has a little bit of material past 2008, and no further index updates are planned, as it will be phased out this year".
In January 2013, the company announced a ground-breaking milestone of 240 billion URLs.
In October 2013, the company announced the "Save a Page" feature which allows any Internet user to archive the contents of a URL. This became a threat of abuse by the service for hosting malicious binaries.
As of December 2014, the Wayback Machine contained almost nine petabytes of data and was growing at a rate of about 20 terabytes each week.
As of July 2016, the Wayback Machine reportedly contained around 15 petabytes of data.
Growth
Between October 2013 and March 2015 the website's global Alexa rank changed from 162 to 208.
Website exclusion policy
Historically, Wayback Machine respected the robots exclusion standard (robots.txt) in determining if a website would be crawled or not; or if already crawled, if its archives would be publicly viewable. Website owners had the option to opt-out of Wayback Machine through the use of robots.txt. It applied robots.txt rules retroactively; if a site blocked the Internet Archive, any previously archived pages from the domain were immediately rendered unavailable as well. In addition the Internet Archive stated, "Sometimes a website owner will contact us directly and ask us to stop crawling or archiving a site. We comply with these requests." In addition, the website says: "The Internet Archive is not interested in preserving or offering access to Web sites or other Internet documents of persons who do not want their materials in the collection."
Oakland Archive Policy
Wayback's retroactive exclusion policy is based in part upon Recommendations for Managing Removal Requests and Preserving Archival Integrity published by the School of Information Management and Systems at University of California, Berkeley in 2002, which gives a website owner the right to block access to the site's archives. Wayback has complied with this policy to help avoid expensive litigation.
The Wayback retroactive exclusion policy began to relax in 2017, when it stopped honoring robots.txt on U.S. government and military web sites for both crawling and displaying web pages. As of April 2017, Wayback is exploring ignoring robots.txt more broadly, not just for U.S. government websites.
Uses
The site is frequently used by journalists and citizens to review dead websites, dated news reports or changes to website contents. Its content has been used to hold politicians accountable and expose battlefield lies.
In 2014 an archived social media page of separatist rebel leader in Ukraine Igor Girkin showed him boasting about his troops having shot down a suspected Ukrainian military airplane before it became known that the plane actually was a civilian Malaysian Airlines jet after which he deleted the post and blamed Ukraine's military.
In 2017 the March for Science originated from a discussion on reddit that indicated someone had visited Archive.org and discovered that all references to climate change had been deleted from the White House website. In response, a user commented, "There needs to be a Scientists' March on Washington".
Furthermore, the site is used heavily for verification, providing access to references and content creation by Wikipedia editors.
In legal evidence
Civil litigation
Netbula LLC v. Chordiant Software Inc.
In a 2009 case, Netbula, LLC v. Chordiant Software Inc., defendant Chordiant filed a motion to compel Netbula to disable the robots.txt file on its website that was causing the Wayback Machine to retroactively remove access to previous versions of pages it had archived from Netbula's site, pages that Chordiant believed would support its case.
Netbula objected to the motion on the ground that defendants were asking to alter Netbula's website and that they should have subpoenaed Internet Archive for the pages directly. An employee of Internet Archive filed a sworn statement supporting Chordiant's motion, however, stating that it could not produce the web pages by any other means "without considerable burden, expense and disruption to its operations."
Magistrate Judge Howard Lloyd in the Northern District of California, San Jose Division, rejected Netbula's arguments and ordered them to disable the robots.txt blockage temporarily in order to allow Chordiant to retrieve the archived pages that they sought.
Telewizja Polska
In an October 2004 case, Telewizja Polska USA, Inc. v. Echostar Satellite, No. 02 C 3293, 65 Fed. R. Evid. Serv. 673 (N.D. Ill. Oct. 15, 2004), a litigant attempted to use the Wayback Machine archives as a source of admissible evidence, perhaps for the first time. Telewizja Polska is the provider of TVP Polonia and EchoStar operates the Dish Network. Prior to the trial proceedings, EchoStar indicated that it intended to offer Wayback Machine snapshots as proof of the past content of Telewizja Polska's website. Telewizja Polska brought a motion in limine to suppress the snapshots on the grounds of hearsay and unauthenticated source, but Magistrate Judge Arlander Keys rejected Telewizja Polska's assertion of hearsay and denied TVP's motion in limine to exclude the evidence at trial. At the trial, however, district Court Judge Ronald Guzman, the trial judge, overruled Magistrate Keys' findings, and held that neither the affidavit of the Internet Archive employee nor the underlying pages (i.e., the Telewizja Polska website) were admissible as evidence. Judge Guzman reasoned that the employee's affidavit contained both hearsay and inconclusive supporting statements, and the purported web page printouts were not self-authenticating.
Patent law
Provided some additional requirements are met (e.g., providing an authoritative statement of the archivist), the United States patent office and the European Patent Office will accept date stamps from the Internet Archive as evidence of when a given Web page was accessible to the public. These dates are used to determine if a Web page is available as prior art for instance in examining a patent application.
Limitations of utility
There are technical limitations to archiving a website, and as a consequence, it is possible for opposing parties in litigation to misuse the results provided by website archives. This problem can be exacerbated by the practice of submitting screen shots of web pages in complaints, answers, or expert witness reports, when the underlying links are not exposed and therefore, can contain errors. For example, archives such as the Wayback Machine do not fill out forms and therefore, do not include the contents of non-RESTful e-commerce databases in their archives.
Legal status
In Europe the Wayback Machine could be interpreted as violating copyright laws. Only the content creator can decide where their content is published or duplicated, so the Archive would have to delete pages from its system upon request of the creator. The exclusion policies for the Wayback Machine may be found in the FAQ section of the site.
Archived content legal issues
A number of cases have been brought against the Internet Archive specifically for its Wayback Machine archiving efforts.
Scientology
In late 2002, the Internet Archive removed various sites that were critical of Scientology from the Wayback Machine. An error message stated that this was in response to a "request by the site owner". Later, it was clarified that lawyers from the Church of Scientology had demanded the removal and that the site owners did not want their material removed.
Healthcare Advocates, Inc.
In 2003, Harding Earley Follmer & Frailey defended a client from a trademark dispute using the Archive's Wayback Machine. The attorneys were able to demonstrate that the claims made by the plaintiff were invalid, based on the content of their website from several years prior. The plaintiff, Healthcare Advocates, then amended their complaint to include the Internet Archive, accusing the organization of copyright infringement as well as violations of the DMCA and the Computer Fraud and Abuse Act. Healthcare Advocates claimed that, since they had installed a robots.txt file on their website, even if after the initial lawsuit was filed, the Archive should have removed all previous copies of the plaintiff website from the Wayback Machine, however, some material continued to be publicly visible on Wayback. The lawsuit was settled out of court, after Wayback fixed the problem.
Suzanne Shell
In December 2005, activist Suzanne Shell filed suit demanding Internet Archive pay her US $100,000 for archiving her website profane-justice.org between 1999 and 2004. Internet Archive filed a declaratory judgment action in the United States District Court for the Northern District of California on January 20, 2006, seeking a judicial determination that Internet Archive did not violate Shell's copyright. Shell responded and brought a countersuit against Internet Archive for archiving her site, which she alleges is in violation of her terms of service. On February 13, 2007, a judge for the United States District Court for the District of Colorado dismissed all counterclaims except breach of contract. The Internet Archive did not move to dismiss copyright infringement claims Shell asserted arising out of its copying activities, which would also go forward.
On April 25, 2007, Internet Archive and Suzanne Shell jointly announced the settlement of their lawsuit. The Internet Archive said it "...has no interest in including materials in the Wayback Machine of persons who do not wish to have their Web content archived. We recognize that Ms. Shell has a valid and enforceable copyright in her Web site and we regret that the inclusion of her Web site in the Wayback Machine resulted in this litigation." Shell said, "I respect the historical value of Internet Archive's goal. I never intended to interfere with that goal nor cause it any harm."
Daniel Davydiuk
In 2013-2016, a pornographic actor tried to remove archived images of himself from the WayBack Machine's archive, first by sending multiple DMCA requests to the archive, and then by appealing to the Federal Court of Canada.
Censorship and other threats
Archive.org is currently blocked in China. After the site enabled the encrypted HTTPS protocol, the Internet Archive was blocked in its entirety in Russia in 2015.
Alison Macrina, director of the Library Freedom Project, notes that "while librarians deeply value individual privacy, we also strongly oppose censorship".
There are known rare cases where online access to content which "for nothing" has put people in danger was disabled.
Other threats include natural disasters, destruction (remote or physical), manipulation of the archive's contents (see also: cyberattack, backup), problematic copyright laws and surveillance of the site's users.
Kevin Vaughan suspects that in the long-term of multiple generations "next to nothing" will survive in a useful way besides "if we have continuity in our technological civilization" by which "a lot of the bare data will remain findable and searchable".
Some find the Internet Archive, which describes itself to be built for the long-term, to be working furiously to capture data before it disappears without any long-term infrastructure to speak of.
See also
- Collective memory
- National memory
- Deep web
- Heritrix
- Library Genesis
- The Memory Hole
- Web archiving
- WebCite
References
External links
- Official website
- Official mirror of the Wayback Machine at the Bibliotheca Alexandrina
- Tool to retrieve a backup from the Wayback Machine
- Wayback Machine Downloader Online
Source of article : Wikipedia
0 Comments