QWKPRUNE 1.12 ----- A utility to remove unwanted messages from QWK-format mail packets. QWKPRUNE.EXE, KILLMSG.EXE, ZIPPYZAP.EXE, this documentation, and all associated documentation are Copyright (c) 1992-1994. SAVEQWK.EXE and MKPRN.EXE, are Copyright (c) 1992, 1993. KMSG-D2.EXE and VIDGRAB.EXE are Copyright (c) 1993, 1994. FILGRAB.EXE is Copyright (c) 1994 by David Harden. All rights reserved. PARM.EXE is hereby released into the public domain. The code used to qualify filenames is based on QFN.C by Ray Duncan. The original code was published in Vol. 7, No. 14 of PC Magazine, and is copyrighted by Ziff-Davis. Autodetection of LHA and ZOO files is based on information from GUS (General Unpack Shell) version 1.70, by Johan Zwiekhorst. All brand or product names mentioned in this documentation or any associated files are trademarks or registered trademarks of their respective owners. All products mentioned are copyrighted by their respective owners. My use of these trademarks does not imply that their owners have endorsed QWKPRUNE, nor does it imply that I have endorsed their products. QWKPRUNE should not be confused with QWKMerge, which is a trademark and copyrighted product of Mike King. QWKPRUNE and David Harden are not affiliated with Mike King. See LICENSE.DOC for license and registration information. Disclaimer of Liability ----- There is no warranty of any kind, either express or implied, covering this software or any associated materials. This software and all associated materials are provided as is, use them at your own risk. The copyright holder is in no way responsible or liable for any losses or damages of any kind that may result from the use of this software or any associated materials, or from inability to use them. System requirements DOS 3.x or higher is preferred. I see no reason why this program shouldn't work with DOS 2.x, but this hasn't been tested. How much memory is required depends on how many messages are to be processed. Approximately 211K free RAM is needed to process 400 messages. If either the /T or /TQ command line switches are used, this drops to 147K, as the 64K write buffer won't be allocated. QWKPRUNE will attempt to allocate enough memory to store data from all the message headers. 400 messages require 42K of RAM for storage. If there's not enough memory to store all the header data, and an overflow directory has been defined, QWKPRUNE will store the data in that directory. In this case, there must be enough free RAM to store the header data from all the messages in the largest conference. What does it do? ----- Removes unwanted messages from QWK-format mail packets based on the contents of their message headers. The modified packet can either be repacked or fed into a mail reader or other QWK utility. See READER.DOC for more details. An "unkill" option allows flexibility in deciding which messages are pruned out. Removes (R) and Re: prefixes, as well as leading whitespace, from the subject field of messages. Up to ten additional user-defined prefixes can also be stripped from subject lines. Can tag messages for processing by programs such as KingQWK and QWKMerge. Transparently repairs conference numbers in packets from older doors that pad the conference number with a space. Repairs "garbage" conference numbers in packets from Virtual QWK. Fixes null-terminated string fields in message headers. Also fixes headers in which the To: and From: fields are terminated with linefeed characters. "Untrashes" QWK packets altered by EZ-RDR. How to install it ----- Place QWKPRUNE.EXE in a directory on your DOS path. Use your editor to create the needed control files (see below). That's all. Files ----- QWKPRUNE's configuration and prune files are ASCII text files that may be created and modified by any text editor, or by a word processor in nondocument mode. Lines may be up to 99 characters long, but usually shouldn't need to be that long. Any line that begins with a semicolon will be treated as a comment, and its contents will be ignored. The configuration file ----- The configuration file is named QWKPRUNE.CFG, and is located in the same directory as QWKPRUNE.EXE. A configuration file may look something like this: ZIP = pkzip UNZIP = pkunzip LHA = lha a /m UNLHA = lha e ARJ = arj a -e UNARJ = arj e ZOO = zoo -add %q * UNZOO = zoo -extract %q * USER_PACK = pkpak -a USER_UNPACK = pkunpak QWK = c:\download\qwk WORK = c:\ PRUNE = c:\prune GLOBAL = global.prn STRIP = Re :, Re(2):, Re , SPILL = d:\ MAX_QWK_AGE = 0 FORCE_REPACK = ON USE_MESSAGE_COUNTS = OFF KEEP_NETSTATUS = OFF QWKPRUNE will automatically detect ZIP, LHA, ARJ, and ZOO files. USER_PACK and USER_UNPACK are commands to pack and unpack files compressed with another archiver. In these entries, %q may be used as a substitute for the QWK file name. If %q isn't used, QWKPRUNE assumes that the QWK file name is at the end of the command. QWK is the directory where you want QWKPRUNE to look for QWK files. WORK is the directory where QWKPRUNE will create its work directory. A directory named QP$WORK will be created under this directory. PRUNE is the default directory where prune files are located. GLOBAL is the name of a global prune file, which will be used for all QWK packets. If GLOBAL is not specified, a global prune file will not be used. (See the section on prune files for more details on this file.) SPILL is the directory where QWKPRUNE will create its overflow directory. If necessary, a directory named QP$SPILL will be created under this directory. If there's not enough space in the work directory to rebuild MESSAGES.DAT, it will be rebuilt there. The new MESSAGES.DAT will then copied to the work directory. If there's not enough free RAM to store the data from all the message headers, QWKPRUNE will store this data data in the QP$SPILL directory. If SPILL hasn't been set in the config file, or if it's set to a nonexistent directory, QWKPRUNE will read the TEMP environment variable. STRIP is the list of prefixes, other than (R) and Re:, that you want to strip from subject lines. Up to ten prefixes may be specified, and individual prefixes may be up to twelve characters long. Prefixes may have trailing spaces. The list may end with a comma, but doesn't have to unless the last prefix has trailing spaces. Case is not significant. If FORCE_REPACK is set to either ON or YES, QWKPRUNE will always repack QWKs, even if they haven't been changed. If MAX_QWK_AGE is set to a number, QWKs more than that many days old won't be processed if wildcards are used in the QWK file name or the default (*.Q*) is used. For example, MAX_QWK_AGE = 0 limits processing to packets with the current date. If USE_MESSAGE_COUNTS is set to ON or YES, message counts will be added to MESSAGES.DAT and CONTROL.DAT if MESSAGES.DAT is rebuilt. If it's set to OFF or NO, and a message count is in CONTROL.DAT, that count will be set to zero if MESSAGES.DAT is rebuilt. If KEEP_NETSTATUS is set to OFF or NO, Net Status blocks at the end of MESSAGES.DAT will be discarded if MESSAGES.DAT is rebuilt. The default is to keep Net Status blocks. QWKPRUNE preserves Net Status blocks by copying everything past the end of the last indexed message to the new MESSAGES.DAT file. If some .NDX files have been deleted to reduce the apparent size of a large packet, some messages at the end of the packet may end up past the last indexed message. Setting KEEP_NETSTATUS to OFF will keep these "orphaned" messages from being copied to the new packet. A packer and unpacker for at least one archive type must be specified. All other options may be omitted, though you should at least specify the QWK and WORK directories. If QWK or WORK is left unspecified, the current working directory will be used for that option. If PRUNE is not specified, it's assumed that prune files are located in the same directory as QWKPRUNE.EXE. Options may appear in any order, and case is not significant. If a line is not a valid option, it should be ignored. If the work directory exists and is empty, QWKPRUNE will use it without comment. If it isn't empty, you'll be asked if you want to abort the program or delete all the files in the work directory. If you use a batch file to call a packer or unpacker, or if you want to silence a packer or unpacker by redirecting its output to NUL, place COMMAND /C in front of the command, or use PARM.EXE. A sample configuration file is included in the distribution archive. Prune files ----- Prune files control which messages will be removed from the QWK packet. By default, they are named BBSID.PRN, and are located in the same directory as QWKPRUNE.EXE. Normally, BBSID is the name the board's mail door gives to QWK packets. For example, if a BBS creates packets named ABC.QWK, the default prune file name for that board should be ABC.PRN. QWKPRUNE takes the BBSID from CONTROL.DAT, so it doesn't matter what the QWK packet is named on disk. A line should have one of the following formats: [!]conference number,switch[option]:text &,switch[option]:text |,switch[option]:text Z,text Not counting Z,text lines, a prune file may have up to 75 entries. A global prune file (see below) may have up to 25 entries. If a line has an invalid format, it will be ignored. If a global prune file is defined in the configuration file, it will be used with all QWK packets. When scanning a QWK packet, entries in the global prune file will be processed before those in the regular prune file. With the following exceptions, it acts like any other prune file. Entries in the global prune file may only scan the entire packet or conference 0. Pruning based on message numbers is not allowed within this file. Z,text lines may not be used in this file. The conference number is the conference to prune messages from, and must be followed by a comma or semicolon. Switch values are F,T,B,S,D,N,P and A. Option may be either P or B, or it may be omitted. If an invalid option is used, it will be ignored. The switch (or option, if used) must be followed by a colon or tilde (~). Text is the text to search for. Case is not significant, and invalid lines should be ignored. To search the entire packet, enter an 'A' as the conference number. If this option is used with the N switch, the line will be ignored. Using Z,text allows you to delete ("zap") certain files from the packet. If "text" is SESSION, SESSION.TXT will be deleted. If it's FILES, NEWFILES.DAT will be deleted. If "text" is HELLO, NEWS, or GOODBYE, the logon screen, BBS news file, or logoff screen will be deleted. QWKPRUNE gets the names of these files from CONTROL.DAT. Otherwise, "text" will be treated as a filename, and that file will be deleted. However, the .NDX files, MESSAGES.DAT, CONTROL.DAT, and DOOR.ID will not be deleted. Unless "text" starts with BLT (for bulletins), it may not contain wildcards. You may not specify a drive or directory in "text". These entries do not count against the maximum number of entries allowed in a prune file. This format may not be used in a global prune file. F: assumes that "text" is a user name, and will delete all messages from that user. T: works similarly, except that it kills messages to a user. B: finds messages that are both from and to a user. If "text" is @ME@, it'll be replaced with your user name from CONTROL.DAT. S: prunes out all messages that have "text" as their subject line. If any of these switches are followed by P, all messages that contain "text" within the appropriate field(s) will be removed. If the S switch is followed by a B, all messages whose subject line begins with "text" will be deleted. D: treats "text" as a number of days before the current system date. Messages older than that will be removed from the packet. N: assumes that "text" is a message number, and deletes that message. If the colon is followed by a > (greater than) or < (less than) sign, messages with numbers greater than or less than "text" will be removed. When deleting messages greater than or less than a number, "text" may be given as HIGH - nnn and LOW + nnn, where HIGH and LOW are the highest and lowest message number in the specified conference. Note that some BBS software uses a single numbering system for all messages, rather than a separate numbering system for each conference. This option probably won't work too well with packets from such systems. P: assumes that "text" is a user name, and will cause all messages to that user to be marked as personal. Blue Wave and RoboMail ignore PERSONAL.NDX, so this option is useless with those readers. A: prunes out all messages within a conference. Anything following the 'A' will be ignored. If a line begins with & or |, QWKPRUNE assumes that that line is to be grouped with the previous entry. Lines beginning with & or | must not contain a conference number. For example, "|20,F:Joe Blow" would be incorrect. All entries in a group search the same set of messages - either a specific conference or the entire QWK. Each entry in a group counts against the maximum number of entries allowed in a prune file. Any non-comment line that doesn't start with & or | ends a group. Zap-file commands, lines that personalize messages, and lines that kill (or "unkill") all messages in a conference (or packet) cannot be part of a group. Entries in a group are processed from the top down, For example: 20,F:John Doe |,F:Joe Blow &,T:Richard Roe first determines if a message in conference 20 is from either "John Doe" or "Joe Blow". It then checks if the message is to "Richard Roe". If both those conditions are met, the message will be pruned out. This is different from: 20,F:John Doe 20,F:Joe Blow &,T:Richard Roe which first deletes all messages from "John Doe" in conference 20, then removes all messages in conference 20 that are from "Joe Blow" to "Richard Roe". If all the entries in a group start with &, messages that meet all the search specifications in the group will be killed or "unkilled". See SAMPLE.PRN for more examples of grouping. If a line begins with an exclamation point, messages meeting the search specification will be "unkilled". This allows you to do things such as killing all messages in a conference, then "unkilling" those you want to keep. If the switch (or option, if used) is followed by a tilde (~), QWKPRUNE will look for messages that do not meet the search specification. If this is used by itself, it has the same effect as killing all messages in a conference, then unkilling those that meet the search specification. In the example below, the first two lines could be replaced by 10,N~100. This also works with the D switch. For example, A,D~30 will delete all messages that are 30 or less days old. Of course, it'll also work with the unkill option. These features do not work with the P switch, or when zapping files. Prune files are read from the top down. For example: 10,A: !10,N:100 !10,SP:Interesting subject A,B:Joe Schmuck A,D:30 will kill all messages in conference 10, except for message number 100 and all messages with "interesting subject" in their subject fields. It will then kill all messages either to or from "Joe Schmuck". Finally, it will delete all messages more than 30 days old. Command line syntax ----- The command line syntax is: QWKPRUNE [d:][dir][QWK_file] [d:][dir][prune_file] [options] If you give a drive and/or directory with the prune file, QWKPRUNE will still look for the global prune file in the default prune directory specified in QWKPRUNE.CFG. DOS wildcards (* and ?) may be used in the QWK file name. If no extension is given, .Q* will be appended to the QWK file name, and .PRN will be appended to the prune file name. The QWK and prune directories can be specified on the command line by ending the appropriate entry with a colon, backslash, or period. This will override the QWK and/or PRUNE entries in the configuration file. If you wish to specify the prune file or prune directory, you must enter something for the QWK file, unless you're using the /M option. The options are: /M[d:][dir] - If this switch is set, QWKPRUNE assumes that the QWK file has already been unpacked. If a drive and/or directory is given, it'll look there for it, otherwise the current directory will be used. It will not create its work directory or repack the QWK file. Note: When using this option, only give the name of the prune file on the command line. When giving a drive and/or directory, there must not be a space between the M and the drive/directory. Use this switch if you want to feed the output to a mail reader. /B - "Batch" mode. If this switch is set, the work directory won't be removed. /C[d:][dir]filename.ext - This option lets you use a configuration file other than the default QWKPRUNE.CFG. There must not be a space between the C and the filename to use. /T - Use this switch to tag messages to be deleted later. /TQ - Use this option to tag messages for processing by KingQWK and QWKMerge. /T and /TQ will be ignored if /N has appeared earlier in the command line. /N - Use this option if you don't want to use a prune file. Messages that have been tagged for deletion by KingQWK, KILLMSG, ZIPPYZAP, or QWKPRUNE /T will still be pruned out. Also, (R) and Re: prefixes, user-defined prefixes, and leading whitespace will still be stripped from the subject lines. This will be ignored if either /T or /TQ appeared earlier in the command line. /NG - If a global prune file is defined, this switch keeps it from being used. Regular prune files will still be used. /Dnn - If wildcards are used in the QWK file name, or the default (*.Q*) is used, packets older than nn days will not be pruned. There must not be a space between the D and the number. If this is given as /D- or /DA, QWKPRUNE will process all packets that match the QWK file name, no matter what MAX_QWK_AGE is set to. /P - If this option is used, QWKPRUNE will always repack QWKs, even if they haven't been changed. If this option is entered as /P-, QWKs will only be repacked if they've been changed, even iF FORCE_REPACK is ON. Things to watch out for ----- QWKPRUNE hasn't been tested with DOS 2.x, but I see no reason why it shouldn't work. QWKPRUNE.EXE won't be able to find itself under DOS 2.x, so it'll assume that its configuration file and prune files are in the current directory. If your mail reader keeps "bookmarks", you may want to mark packets as unread after pruning them. Otherwise, they'll no longer be accurate. If your reader can't mark packets as unread, you can manually delete the "bookmarks". QWKPRUNE will delete bookmarks from EZ-RDR, KingQWK, DeLuxeý, Session Manager, Blue Wave, and MikeMayl if any messages are pruned out. Don't use the /M option on a QWK while you're reading it, unless you're also using the /T option. Some readers may leave MESSAGES.DAT open while running outside programs. Rewriting MESSAGES.DAT under such conditions may garble the file. Even if your reader closes MESSAGES.DAT, it may not reload the changed .NDX files generated by QWKPRUNE. Users on networks or using multitaskers should be sure that the QWK file isn't being read or modified by any other process while it's being pruned. Error Messages ----- Prune file(s) empty: Neither the regular prune file nor the global prune file (if defined) contained any valid message-procession entries. There may have been file deletion entries. (R) and Re: prefixes, user-defined prefixes, and leading whitespace were still stripped from the subject lines. No messages pruned: No messages were found to prune. Not enough free disk space to rebuild MESSAGES.DAT: There wasn't enough free space on the work directory to build the new MESSAGES.DAT, and QWKPRUNE was unable to use the overflow directory for this purpose. Not enough memory to process QWK file: A check of the far heap didn't find enough memory to store all the message headers, and QWKPRUNE was unable to use the overflow directory to store the header information. Memory allocation error: An error occurred when allocating dynamic memory. Probably caused by attempting to allocate more memory than was available. Since QWKPRUNE checks to see how much far memory is available before allocating any, this shouldn't happen. Please let me know if it does. Unable to open config file: QWKPRUNE was unable to open its configuration file. Unsupported archive type: Either a packer or unpacker wasn't defined for the archive type of the QWK file. Unable to locate QWK file(s): No .QWK packets matching the search specifications (filespec and/or age) could be found. Unable to open CONTROL.DAT: QWKPRUNE was unable to open the CONTROL.DAT file from the mail packet. If this error occurs, the mail packet is probably not a QWK packet. If the packet was compressed with LHA or ZOO, it also may indicate that there wasn't enough disk space to unpack CONTROL.DAT. If a batch file was used to call the unpacker, there may not have been enough free memory to unpack it. Error in CONTROL.DAT: An error occurred when reading CONTROL.DAT. Either it's garbled, or it isn't from a QWK file. Unable to open MESSAGES.DAT: QWKPRUNE was unable to open the MESSAGES.DAT file from the QWK packet. If the packet was compressed with LHA or ZOO, this may indicate that there wasn't enough disk space to unpack it. If you used a batch file to call the unpacker, it may also indicate there wasn't enough free memory to unpack it. MESSAGES.DAT is corrupt: An entry in one of the .NDX files doesn't point to a valid message header. No .NDX files found: QWKPRUNE was unable to locate the .NDX files from the mail packet. Either the mail packet isn't a QWK, or the mail door didn't create .NDX files, or there are no messages in the packet. If the QWK door that produced the packet was configured not to create .NDX files, you'll need to rebuild them before pruning the packet. [QWK/prune/work/overflow] directory not found: The QWK, prune, work, or overflow directory does not exist. DOS error levels ---- 4: Not enough memory to process QWK file. 5: Unsupported archive type. 6: Unable to open CONTROL.DAT. 7: Error in CONTROL.DAT. 8: Unable to open MESSAGES.DAT. 9: No .NDX files found. 10: An error occurred when unpacking the QWK file. 11: Not enough free disk space to rebuild MESSAGES.DAT. 12: Not enough free disk space to rebuild the .NDX files. 13: MESSAGES.DAT is corrupt. 14: Invalid command line option. 15: Unable to locate QWK file(s). 16: Error changing to the drive the work directory is on. Error changing to the work directory. 17: An error occurred when attempting to spawn a program. 18: An error occurred when repacking the QWK file. 19: File open error. 20: File write error. 21: Memory allocation error. 22: Unable to open config file. 23: The work, QWK, prune, or overflow directory does not exist. 24: Control-break exit. 25: The program was aborted by the user because the work directory wasn't empty. Acknowledgements ----- Thanks to Mark May and Patrick Lee for gathering and distributing information on the QWK specification, and to all those they collected this data from. Belated, though sincere, thanks to Mark "Sparky" Herring for creating the QWK format in the first place. Thanks to Jack Hudgions for providing information from GUS on detecting archive types. Thanks to Diane Merriam, Bill Peek, and Darrell Kitchen for helping me get started on this project, and to Joe and Pat McNeal for their continued support while I've been working on it. How to reach me ----- Please send comments, suggestions, bug reports, etc. to me at: RIME Shareware or Offline conference R/O capable at site ->1066 RIME Common conference Routed R/O ONLY at site ->1066 Ilink Shareware or Offline conference Fido Offline conference. Fido netmail 1:2320/110 WWIVnet email #360@8262 CompuServe mail >INTERNET:david.harden@tfd.iglou.com Internet Email david.harden@tfd.iglou.com Direct BBS Conf. #55 on the Valley PC Clone BBS (502) 937-6143 or (502) 933-7424 Log in as QWKPRUNE USER;QPRUNE for immediate access Or send mail to: David Harden P.O. Box 197086 Louisville, KY 40259