==Phrack Inc.== Volume Four, Issue Thirty-Eight, File 8 of 15 Wide Area Information Servers How Do I Use It and Why Should I Care? by Mycroft mycroft@gnu.ai.mit.edu Introduction ~~~~~~~~~~~~ This file serves as an introduction to "information servers," and in particular to the WAIS system from Thinking Machines Corp. Overview ~~~~~~~~ The Wide Area Information Server (or WAIS) system provides a way for people ("providers") to make information sources ("sources") accessible via a network, with a very simple interface to search for and retrieve particular pieces of information ("documents"). Essentially, you pick a source and specify a few keywords, and the WAIS search engine tries to find documents that match those specific keywords. Each document is scored, and the highest scoring documents are listed first. In addition, there is a mechanism ("relevance feedback") for feeding information back to the server about which documents are most interesting to you, and having it narrow the search based on this. To summarize: WAIS gives you a fast and easy way to search vast amounts of information, and to provide access to it to other users on a network. Why Should I Care? ~~~~~~~~~~~~~~~~~~ You should care because I, through the goodness of my heart, have made all the issues of Phrack Inc. available through WAIS. :-) I'll soon be adding issues of the LOD/H TJ, NARC, NIA, Worldview, and a lot of other files. If anyone would care to donate files, I'd appreciate it. There are also many other sources currently available that will probably be of interest to you. Step 1: Compiling A Client ~~~~~~~~~~~~~~~~~~~~~~~~~~~ To use WAIS, you need a client program. There are currently 4 available that I know of: Xwais - for the X Window System SWAIS - terminal-based Mac WAIStation NeXT WAIStation (I vaguely recall something about a Windows client.) Xwais and SWAIS both come in the standard distribution, with the search and index engines. You can FTP any of the above from think.com, in directory /wais. The relevant files are: wais-8-b4.tar.Z - contains the search and index engines, as well Xwais and SWAIS WAIStation-0-63.sit.hqx - the Mac WAIStation WAIStation-NeXT-1.0.tar.Z - the NeXT WAIStation After you choose a client and get the source, compile it. There are decent directions on how to do this in each package. Step 2: Finding An Information Source ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To find a source, you just do a search in the "directory of servers" -- a source containing pointers to all the registered WAIS sources on the net. For example, if you're using Xwais: (I am *not* going to go into the details of how to use the scrollbars and whatnot. If you're stuck, ask a Mac weenie for help.) Tell me about: .----------------------------------------------------------------. .------. |phrack | |Search| `----------------------------------------------------------------' `------' In Sources: Similar to: .------------------------. .----------------------------------------------. |directory-of-servers.src| | | `------------------------' `----------------------------------------------' .----------. .-------------. .------------+ .---------------. .----. .----. |Add Source| |Delete Source| |Add Document| |Delete Document| |Help| |Done| `----------' `-------------' `------------' `---------------' `----' `----' .+------------------------------------------------------------. Resulting || 1000 551 phrack.src /proj/wais/wais-sources/ | documents: || | .----. || | |View| || | `----' `+------------------------------------------------------------' .-----------------------------------------------------------------. Status: |Found 1 document. | `-----------------------------------------------------------------' The lines in the "Resulting documents:" window break down into three parts: Score -- How well it matched your query, as compared to other documents. Size -- of the document. Headline -- The "headline" is generated while building the index. For source files, it's broken down by filename and path. For the p/h/c/a server, it's the title of the article, the authors, and the issue and file number. So double-click on the document, and you'll get another window (shortened a bit): Source Edit Name: phrack.src Server: hal.gnu.ai.mit.edu Service: 8000 Database: /src/wais/wais-sources/phrack Cost: 0 Units: :free Maintainer: mycroft@hal.gnu.ai.mit.edu Description: .+------------------------------------------------------------------------. ||Server created with WAIS release 8 b3.1 on Jan 31 12:30:28 1992 by mycro| || | ||Here are all the issues of Phrack for your edification. | || | ||Phrack is an old hacking, cracking, phreaking, and general anarchy | ||newsletter. Articles range from how the phone system works to making | ||------------------------------------------------------------------------| `+------------------------------------------------------------------------' .----. .------. |Save| |Cancel| `----' `------' The fields work like this: Name: Filename to store this source under on *your* machine. Server, Service, Database: Where the source lives (my machine). Cost, Units: How much it will cost you to access the information. Maintainer: Me! Description: What is there. You really want this one, so just click the "Save" button. This will create a "source file" on your machine, which you can then access with the "Add Source" button of the question window. This setup is sort of a lose, because your copy could get out of date and not work. I've proposed a way to fix this problem, but so far it hasn't been implemented. This bit me once when I moved the files to their current location. Step 3: A Query ~~~~~~~~~~~~~~~~ Now, let's make another query. I can't remember where I saw this, so: Tell me about: .----------------------------------------------------------------. .------. |that night with tuc | |Search| `----------------------------------------------------------------' `------' In Sources: Similar to: .------------------------. .----------------------------------------------. |phrack.src | | | `------------------------' `----------------------------------------------' .----------. .-------------. .------------+ .---------------. .----. .----. |Add Source| |Delete Source| |Add Document| |Delete Document| |Help| |Done| `----------' `-------------' `------------' `---------------' `----' `----' .+------------------------------------------------------------. Resulting || 1000 24.9K "Phrack World News Issue XIV, Part 2", compiled | documents: || 967 29.9K "Phrack World News Special Edition III", compile| .----. || 800 74.9K "Phrack World News Special Edition II", compiled| |View| || 467 6.1K "Phrack Pro-Phile V: Tuc", by Taran King (issue | `----' `+------------------------------------------------------------' .-----------------------------------------------------------------. Status: |Found 40 documents. | `-----------------------------------------------------------------' All you have to do is double-click on one of the documents. After a while you'll get another window: .+------------------------------------------------------------------------. || | ||"Phrack World News Issue XIV, Part 2", compiled by Knight Lightning (iss| || | || | || PWN ^*^ PWN ^*^ PWN { SummerCon '87 } PWN ^*^ PWN ^*^ PWN | || ^*^ ^*^ | || PWN Phrack World News PWN | || ^*^ Issue XIV/2 ^*^ | || PWN PWN | || ^*^ "SummerCon Strikes" ^*^ | || PWN PWN | ||------------------------------------------------------------------------| `+------------------------------------------------------------------------' .-----------. .--------. .----. .--------. .------------. .----. |Add Section| |Find Key| |Next| |Previous| |Save To File| |Done| `-----------' `--------' `----' `--------' `------------' `----' Status: The "Add Section" button is used for relevance feedback. You select a region of text and press "Add Section" and it will show up in the "Similar to:" box in the question window. "Find Key," "Next," and "Previous" are used to search for the keywords in the document. The rest is pretty obvious. What Else? ~~~~~~~~~~ There are more powerful ways to use WAIS. For example, using the "waisq" and "waisretrieve" programs, you could query the directory of servers nightly to get the latest copy of phrack.src. This would ensure that yours is never more than a day out of date. (I recommend subscribing to the wais-discussion list and/or reading alt.wais instead, though, since it's more interesting and won't put a load on the directory of servers.) Or if you keep an archive of your mail, you could use it to index that. (I know several people who do this, including Brewster.) Or whatever. Take a look at some of the existing sources to get an idea. Conclusion ~~~~~~~~~~ WAIS is a very useful tool for finding information. It is still under development, though, and there a few rough edges that need to be worked out. In particular: * Source files getting out of date. * Multiple servers for a single source (for reliability and speed). * Multiple indices for the same source on a given server (for transient information). * Index overhead. (The Phrack index, for example, is currently larger than the text itself!) _______________________________________________________________________________