┌───────────────────┐
│ │ ║ UpLoadProcessor
│ ╥ ╥ ╥ │ ║
│ ║ ║ ║ ╓──╖ │ ║ Version 0.99ß
│ ║ ║ ║ ║ ║ │ ║
│ ╙───╜ ╨ ║──╜ │ ║ (c) Copyright 1992-1993 - Stacy Smith
│ ╨ │ ║
└───────────────────┘ ║
════════════════════╝
Courtesy of:
The Bloom Beacon-Picayune BBS
Node 1: (804) 525-9760 (USRobotics Courier Dual Standard)
Node 2: (804) 525-5372 (USRobotics Courier V.32bis)
FidoNet 1:276/112
Intelec
RoseNet (ID: BB-P)
Stacy Smith
Route 6 Box 189
Forest, Virginia 24551
┌────────────────────┐
│ 1. Introduction: │
└────────────────────┘
This system was born out of a need for a universal upload processor. There are
many alternative systems available, but they are limited to the ZIP format and
perhaps one or two others. Few are able to handle self-extracting archives.
Most are limited in the number of levels of archive nesting allowed in a file
to be tested. All require the use of a third-party duplicate file checking
system if you want to screen your uploads for duplicates.
Tired of waiting for PKZIP 2.something-or-another, I converted my BBS files
over to the ARJ compression format, due to its superior compression ratio over
PKZip and its features over LHA. I do not intend to debate the merits of one
archive format over another, but due to my decision, the need for a universal
upload processor became apparent, so off I went.
While I was at it, I decided to incorporate other technologies, such as
duplicate checking, archive format conversion, information lines, support for
internal description files, etc., into a single package. This software is the
result of my efforts to allow my BBS to handle any archive that my users can
throw at it.
┌──────────────────────────────────────────────┐
│ 2. Features of the UpLoadProcessor System: │
└──────────────────────────────────────────────┘
∙ Identifies and processes ARC, ARJ, HYP, LZH, PAK, SQZ, ZIP, ZOO and
GIF files, regardless of their file extensions (ideal for software
distribution networks, like .SDN files).
∙ Identifies and processes ARJ, LZH, PAK, SQZ and ZIP self-extracting
(SFX) archives.
∙ Scans ARC, PAK, ZIP and ZIP SFX archives for DOS reserved keywords to
prevent hacking by hex-editing. (ARJ and LHA are resistant to these
type of hacking attempts).
∙ Detects ARJ security envelopes and ZIP version 1.x and 2.x authenticity
verification (-AV) stamps, and can be left alone after processing
(configurable).
∙ Detects and processes ARJ and ZIP archives with imbedded paths.
∙ Detects and rejects encrypted ARJ and ZIP archives.
∙ Selected uncompressed files uploaded can be processed and compressed
using your default format.
∙ Removes known BBS ads from archives; includes a BBS ads maintenance
utility so sysops can update their BBS ads databases in real time. ULP
can also insert a BBS ad, if desired.
∙ Updates the PCBoard DOWNLOAD.TXT file, if desired, with the correct
archive extension to reflect the conversion process.
∙ Allows the use of up to 9 different archiving programs, all user-
configurable. Any archiving program used that is not listed above will
be identified using the file extension only, until it's signature is
determined and incorporated into the program.
∙ Uses a recursive processing routine that will allow (theoretically)
unlimited nested archives (the only limit is imposed by the DOS path).
This routine has been tested to 5 levels deep as of this writing.
∙ Allows the use of up to 3 different file-checking programs, all user-
configurable, for virus and trojan checking, 3rd party utilities, etc.
∙ Allows the use of a GIF file checking program, completely configurable.
∙ Rejects GIF files based upon image width, height, colors and
compression.
∙ User-definable disposition (rename or delete) of corrupted, duplicate
or other archives (not virus-related).
∙ User-definable disposition (rename or delete) of virus-infected
archives.
∙ Incorporates it's own CRC-32 duplicate checking system, as well as the
associated database processing software. No more executing (and
registering!) third-party duplication-checking software, sacrificing
speed and memory. This system is extremely fast and it's database is
much smaller than other systems. Despite it's size, the probability of
a false duplication is more than 4 trillion to 1! The database is
self-validating, to quickly determine if a database has been corrupted
or altered.
∙ The duplication determination system uses two filters to determine an
archive's level of duplication, preventing false rejection by simply
counting up the number of blatantly duplicate files.
∙ Converts all uploads into a default archive format of your choosing,
or they may be re-archived in their original format (user-defined).
Nested archives can also be converted to your default format, or
re-archived using their original format. SFX archives can be archived
using your default format, or optionally left alone after verification.
∙ Can utilize a user-defined time window (in months) for acceptance of
new upload files, based on the most recent file date in the archive.
∙ Changes all archive file attributes to 'read/write' from 'read-only'.
∙ Supports the use of private and public upload directories. Moves files
and upload descriptions from the private directory to the public
directory. Single directory operation is also available.
∙ Honors the '/' identifier in the description marking the file as a
private upload for the sysop by processing the file, but not making
it public.
∙ Supports the use of DESC.SDI and FILE-ID.DIZ description files in an
archive, user-configurable, for those not running PCBoard 14.5a+ and
PCBDESC.
∙ Smart word-wrapping word-wraps descriptions or leaves them as entered,
depending upon the presence of boxes, etc.
∙ Can optionally insert an archive or GIF information line in the file
description that contains various information about the archive or GIF
files.
∙ Two modes of online testing are available: normal mode, which fully
unpacks the archive and tests each file individually; and fast mode,
which scans a ZIP or ARJ archive directly for file CRCs and dates, and
uses the archiving program's internal integrity testing.
∙ The online tester will accept a redirected ARJ or PKUNZIP archive
listing file to pre-verify the duplication and age limits before a user
uploads the actual archive, saving him wailing and gnashing of teeth.
∙ ULPTEST can generate COM port status output to inform the online user
of the progress of testing. ULPTEST supports IRQ2-7, non-standard
port addresses and baud rates to 115K in direct mode, and also supports
FOSSIL drivers. The port information can be defined on the command-line
or can be read directly from PCBOARD.DAT and PCBOARD.SYS. ULP can also
be implemented in a "seamless" mode (when registered) to look like a
part of PCBoard.
∙ Import of FWKCS CRC database is supported to ease transition to the ULP
duplication system. No more rebuilding databases!
∙ Archives failed for exceeding duplication limits can be viewed using
ULPDB (the database program), to allow easy determination of manual
archive acceptance.
∙ User-selectable process logging to a disk file. Two logging modes are
available: terse and verbose.
∙ Written mostly in C (and a little assembler) for optimal speed, using
Microsoft C 7.0, Borland C++ 3.1 and Turbo Assembler 3.0.
∙ Lifetime registration; pay ONCE and your registration number will work
on all future versions!
┌─────────────────────────────────────────────────────┐
│ 3. Files Included in the ULP Distribution Archive │
└─────────────────────────────────────────────────────┘
ULP.EXE Event upload processing program.
ULPTEST.EXE On-line testing program for use in PCBTEST.BAT.
ULPDB.EXE CRC-32 database creation and maintenance program.
ULPCFG.EXE Configuration program for the ULP system.
ULPADS.EXE BBS ads database maintenance program.
ULPADS.DB BBS ads database file.
ULP.CFG Sample text configuration file for the ULP system.
EXCLUDE.LST Sample exclusion list for archivers.
PCBTEST.BAT Sample PCBTEST.BAT file for using ULPTEST.
ULP.DOC This file.
HISTORY.TXT ULP revision history in reverse order.
ULPBLT Archive verification bulletin for sysops to post.
REGISTER.FRM Registration form for ULP.
COMMENT.TXT Sample comment file for inclusion in archive headers.
FILE_ID.DIZ Internal description file.
When you unzip the distribution archive, you should see my PKZIP authenticity
verification stamp, and a '-AV' after every file in the archive:
# SSU301 The Bloom Beacon-Picayune BBS
If there are any files missing or added, or the -AV stamp is missing, the
archive has been tampered with. It would be advisable to call my BBS (listed at
the top of this document) for the latest version of ULP.
┌───────────────────────────┐
│ 4. Program Requirements │
└───────────────────────────┘
To the best of my knowledge, this program will run on most any machine capable
of running PCBoard 14.5+. My BBS setup is DESQview on a LANtastic network, but
other sysops that I have been in contact with have successfully implemented ULP
on setups with CD-ROMs and other varying hardware and software.
ULP has been developed and tested using the following archiving and file
checking software packages:
ARJ 2.10 through 2.39f (by Robert Jung)
HYPER 2.5 (by P. Sawatzki and K. P. Nischke)
LHA 2.12 and 2.13 (by Haruyasu Yoshizaki)
LHarc 1.13c (by Haruyasu Yoshizaki)
PAK 2.51 (by NoGate Consulting)
PKPAK 3.61 (by PKWare)
PKZIP 1.10 through 2.04g (by PKWare)
SQZ 1.08.2 (by Jonas Hammarberg)
ZOO 2.01 through 2.10 (by Rahul Deshi)
SCAN V82 through V104 (by McAfee Associates)
GIFTEST 4.0 Beta 10 through Beta 14 (by Dave Navarro)
The ULP system requires DOS 3.x or later, as it uses DOS SHARE-compatible file
reads and writes, and can use the DOS PATH to find the archiving and other
utilities.
ULP's memory requirements are relatively small (about 80K or so, plus the disk
buffer allocated), but all programs are spawned or shelled, which reduces the
free memory for the program being executed. It would be a good idea to have as
much free conventional memory as possible (ULP itself cannot use EMS or XMS
memory), especially if you use the ARJ compression system, which requires in
the neighborhood of 280K itself to run.
ULPDB also should have as much conventional memory as can be afforded, as it
performs its sort in memory for optimal speed; it may use from 128K to 576K of
memory depending upon the operation being performed. If your duplication
database is larger than 500K (over 50,000 file entries), ULPDB may utilize a
segmented sort algorithm, which is slower but capable of completely compiling a
database up to 8MB in size (over 800,000 file entries)! The memory sort will
probably be fine for many systems, as 50,000 files would probably average out
to be about 4,000 to 5,000 typical archives.
Obviously, the 286+ and 386+ versions of these programs that is available to
registered users will not run on a 8088, 8086 or 80186 (huh?) based computer.
So don't try it (I haven't, so I don't have a clue what will happen...probably
spontaneous combustion...).
┌───────────────────┐
│ 5. Registration │
└───────────────────┘
The ULP system is not free; nor is ULP is crippled to force registration. ULP
is fully functional, and will always remain so. The only variation with the
registered copies is no time delay and beg message.
Why register? Besides a clean conscience, you will get a registration code that
will work for all future versions of ULP, and will remove the delay and message
at the end of execution of each program. In addition, registration will allow
you to run ULPTEST in a "seamless" mode so that it looks like a part of
PCBoard to the user.
In addition, the registation code will allow use of the 286 or 386 specific
versions of ULP that are made available only to registered users of ULP. These
versions are slightly smaller and faster than the shareware 086+ version.
The registration fee for your unique code is $20 for non-commercial BBSes. The
registration fee for commercial BBSes, defined as running your BBS in the
course of a commercial business or for profit, is $40. Other variations are
available; refer to the file REGISTER.FRM for all registration options. Please
print the file REGISTER.FRM and fill it out. You can print out the form by
issuing the following command from the DOS prompt:
TYPE REGISTER.FRM > PRN
┌───────────────────────────────────────┐
│ 6. License, Warranty and Disclaimer │
└───────────────────────────────────────┘
I'll keep this part short and sweet, and dispense with the legal-ese:
License: You are allowed to use ULP for 30 days, after which you must
either register ULP or stop using it completely. ULP registration is
a license for your use of ULP; I retain ownership of the software. A
single registration applies to a single BBS system, regardless of
the number of computers used in the system. If you run two or more
distinct BBS systems on the same computer or network (with different
names), you require two or more ULP registrations. Refer to the
registration form for the currect pricing structure.
Warranty: There isn't one. The only thing I'll guarantee is that ULP
will take up disk space, and will disappear when deleted.
Disclaimer: I'm not responsible for anything bad that happens. ULP
works here, but I cannot be held responsible for it not working on
your computer or doing any damage to hardware or software.
If these aren't agreeable with you, then the best thing to do is delete ULP
right now. I'll do my best to help any user (registered or not) that wants to
use ULP, and I'll act on bug reports quickly, but I simply cannot and will not
be responsible for anything bad, like lost data, disk crashes, or whatever
else you can think of.
┌──────────────────────────────┐
│ 7. Principles of Operation │
└──────────────────────────────┘
Since the ULP system is made up of several inter-operating programs, this
overview and concept explanation should help you understand how ULP is
designed to be used.
*******************************************************************************
READ THIS SECTION VERY CAREFULLY, AS IT WILL MAKE LIFE MUCH EASIER!!!
*******************************************************************************
I believe that all responsible BBS sysops verify all of their uploads prior to
posting them, in order to protect both themselves and their users. ULP is
designed with idea in mind. Most, if not all, sysops process uploads in one of
two ways (listed with benefits and liabilities as I see them):
1) Make all uploads private, processing them during a system event.
BENEFITS:
∙ Takes up very little on-line time on the user's part to
process archives.
∙ Allows the conversion of all archives to a default format,
so that the BBS archives are consistent.
∙ Allows the BBS to accept any archive format...face it, it's
hard enough to get some of these weenies to upload, much
less compress them the same way.
LIABILITIES:
∙ Files are not available immediately for download.
∙ The 'U' subcommand of PCBoard's (F)ile Directories command
is no longer available to the user.
∙ Does not catch duplicates or aged archives until after the
user has uploaded them, and perhaps leads to abuse by
clever (?) users. (It is assumed that these sysops still
use the venerable 'PKUNZIP -T' in their PCBTEST.BAT...)
2) Process (test) each upload online after the user uploads them, and
making them available for immediate download.
BENEFITS:
∙ Catches duplicate, defective and aged archives while the
user is online, denying him upload credits.
∙ Files are available immediately for download if they are not
made private in the PCBoard setup.
LIABILITIES:
∙ Takes up on-line time for a user, potentially adding to his
long-distance phone bill, discouraging further uploading;
this process is typically quite slow for large archives.
∙ The archive format cannot be converted (a limitation of
PCBoard), leaving many different archive formats in your
BBS directories. Because of this, many sysops force users
to use a specific format, which may or may not be consistent
with other BBS's the user calls.
The ULP system takes the best of both of these methods of testing. ULP.EXE is
an event-mode processor which will fully process every new archive found in the
source upload directory, including archive format conversion, and optionally
move the file and description to a destination directory. In the simplest
ULP system setup, you MUST run ULP.EXE in your event to completely process all
uploads to your system.
ULPTEST.EXE is an on-line processor, with a twist: it doesn't recompress the
archive. It does little more than thoroughly test the upload; this will greatly
reduce the amount of on-line time the user is forced to waste waiting for file
processing. The conversion process is handled by ULP during the system event.
Pay attention to this part:
PCBoard normally has two upload directories for each conference: a private and
a public directory. When PCBoard invokes PCBTEST.BAT, the upload is in the
private directory. If the archive fails the testing, it will remain there.
However, if it passes, one of two things will occur depending upon your system
setup; if you have made all uploads private, the file will remain in the
private directory; if you have not made uploads private, it will be moved BY
PCBOARD (not ULPTEST) to the public directory.
If you have made all uploads private, then setup and configuration of ULP is a
snap: the source directory is the private upload directory, and the destination
is the public directory. However, if you want to allow users access to untested
uploads, then your source directory is the public upload directory, and the
destination information is left blank. To illustrate the operation:
MAKE ALL UPLOADS PRIVATE │ ALL UPLOADS AVAILABLE AFTER TESTING
───────────────────────────────────┼─────────────────────────────────────
2 directories: C:\PRIVATE │ 2 directories: C:\PRIVATE
C:\PUBLIC │ C:\PUBLIC
│
User uploads a file, gets placed │ User uploads a file, gets placed
in C:\PRIVATE by PCBoard │ in C:\PRIVATE by PCBoard
│
ULPTEST.EXE tests it │ ULPTEST.EXE tests it
│
PCBoard leaves file in C:\PRIVATE │ If it passes, PCBoard moves it to
│ C:\PUBLIC; if it fails, PCBoard
│ leaves it in C:\PRIVATE
│
ULP.EXE processes all new uploads │ ULP.EXE processes all *new* uploads
found in C:\PRIVATE since last │ found in C:\PUBLIC since last event
event and moves all good uploads │
to C:\PUBLIC │
The following are PCBSETUP (14.5a) screen snapshots of what the Main Conference
should look like when implemented as described above:
MAKE ALL UPLOADS PRIVATE:
╔═════════════════════════════════════════════════════════════════════════════╗
║ Main Board Configuration ║
║ ║
║ Make All Uploads Private : Y ║
║ Make All Messages Private : N ║
║ ║
║ Number of Message Blocks : 2 ║
║ Name/Loc of MSGS File : C:\PCB\MAIN\MSGS ║
║ ║
║ Name/Loc of Users Menu : C:\PCB\GEN\BRDM ║
║ Name/Loc of Sysops Menu : C:\PCB\GEN\BRDS ║
║ Name/Loc of NEWS File : C:\PCB\GEN\NEWS ║
║ ║
║ Sort Name/Loc Upload DIR File Location of Uploads ║
║ Public Upld: 0 : : ║
║ Private Upld: 2 : C:\PCB\GEN\PRIVDIR : C:\PRIVATE\ ║
║ ║
║ Menu Listing Path/Name List File ║
║ Doors : C:\DOORS\DOORS : C:\DOORS\DOORS.LST ║
║ Bulletins : C:\PCB\GEN\BLT : C:\PCB\GEN\BLT.LST ║
║ Scripts : : ║
║ Directories : C:\PCB\GEN\DLFILES : C:\PCB\GEN\DLFILES.LST ║
║ Download Paths (listing only) : C:\PCB\GEN\DLPATH.LST ║
║ ║
╚══ 10:13:03 ═══ 09-30-92 ═════ F1 ═ help ════ caps: OFF num: OFF ins: OFF ═╝
MAKE ALL UPLOADS AVAILABLE IMMEDIATELY AFTER TESTING:
╔═════════════════════════════════════════════════════════════════════════════╗
║ Main Board Configuration ║
║ ║
║ Make All Uploads Private : N ║
║ Make All Messages Private : N ║
║ ║
║ Number of Message Blocks : 2 ║
║ Name/Loc of MSGS File : C:\PCB\MAIN\MSGS ║
║ ║
║ Name/Loc of Users Menu : C:\PCB\GEN\BRDM ║
║ Name/Loc of Sysops Menu : C:\PCB\GEN\BRDS ║
║ Name/Loc of NEWS File : C:\PCB\GEN\NEWS ║
║ ║
║ Sort Name/Loc Upload DIR File Location of Uploads ║
║ Public Upld: 2 : C:\PCB\GEN\PUBLDIR : C:\PUBLIC\ ║
║ Private Upld: 2 : C:\PCB\GEN\PRIVDIR : C:\PRIVATE\ ║
║ ║
║ Menu Listing Path/Name List File ║
║ Doors : C:\DOORS\DOORS : C:\DOORS\DOORS.LST ║
║ Bulletins : C:\PCB\GEN\BLT : C:\PCB\GEN\BLT.LST ║
║ Scripts : : ║
║ Directories : C:\PCB\GEN\DLFILES : C:\PCB\GEN\DLFILES.LST ║
║ Download Paths (listing only) : C:\PCB\GEN\DLPATH.LST ║
║ ║
╚══ 10:13:03 ═══ 09-30-92 ═════ F1 ═ help ════ caps: OFF num: OFF ins: OFF ═╝
ULPTEST.EXE also has two modes of operation: normal and fast. The normal mode
decompresses the files, performs file, duplication and age checking, and then
deletes the extracted files and returns to PCBoard, informing PCBoard of the
test results. Fast mode DOES NOT decompress the file; it firsts performs an
archive integrity check, then scans ARJ and ZIP archives directly for duplicate
and age information, and then returns to PCBoard (if the archive is not ARJ or
ZIP, then normal mode is forced). In fast mode, file checking (viruses, etc.)
is left for ULP to do (which is why the above discussion regarding
private/public directories is important).
ULPTEST.EXE will also accept a redirected ARJ or PKUNZIP listing text file with
the special name VERIFY.ULP as input to pre-verify an upload for a user, before
the user actually spends his time uploading the file only to find out it won't
pass the limits you set.
┌───────────────────┐
│ 8. Installation │
└───────────────────┘
GENERAL (REQUIRED!):
───────────────────
Make a subdirectory on your hard drive. For the purposes of this document,
we'll call it "C:\ULP". Unarchive the ULP distribution archive into this
subdirectory. You've more than likely already made it this far, if you're
reading this file. <g>
The ULP system opens several files at once for various reasons. I would
recommend that you have a minimum of FILES=30 per node in your system
CONFIG.SYS file, since ULPTEST is run in conjunction with PCBoard.
If you are running PCBoard under a network or a multitasking operating system,
you should already have DOS's SHARE.EXE loaded. You must have SHARE loaded in
order to take advantage of the file sharing and locking methods used by the ULP
programs to prevent data loss. (If you are running a single-node system without
a multitasker, SHARE is not needed).
Edit the configuration file to suit your needs. Proper configuration will
require you to refer to this section, the previous section and the section
titled "Configuration", which has an in-depth explanation of each configuration
parameter and its function. It would probably be wise to fire up your printer
for this document.
For this document, we'll assume the name of your text configuration file is
"ULP.CFG". After editing the configuration file, generate the binary version
(called "ULP.BIN" in this document) using the ULPCFG.EXE program as follows:
ULPCFG -TULP.CFG -CULP.BIN
Note that for any of the ULP programs, you can obtain a limited program syntax
screen simply by executing the program with no arguments.
FOR EVENT-MODE UPLOAD PROCESSING (REQUIRED!):
────────────────────────────────────────────
As a minimum setup, you MUST run ULP as an event-mode processor, as ULP.EXE
handles all of the CRC database updating, archive conversion, file and
description moving, archive information line computation, and other features.
Further, ULP requires that you define the upload disk directories and upload
DIRxx files. They can be either a private/public setup or public setup, as
described below (and in the 'principles of operation'section):
- PRIVATE/PUBLIC SETUP: This setup will not allow users to get to any
archive that has not been fully processed. In PCBSETUP, under Main
Board Configuration (and any other conferences you want ULP to
process), make all uploads private. Define a private upload
directory and a private DIRxx file (which should have already been
done, since all sysop-only uploads are placed here by PCBoard). Also
create a public upload directory and public DIRxx file, and add it into
the DLPATH files. This is how I have my system setup.
- PUBLIC SETUP: This setup will allow users to download archives from
the public upload directory, including partially testing archives. In
PCBSETUP, under Main Board Configuration (and any other conferences you
want ULP to process), define a private upload directory and a private
DIRxx file, and a public upload directory and a public DIRxx file (this
is where PCBoard will place all new non-private uploads since you did
not make them private).
To process your uploads during your system event, add the following lines to
your EVENT.SYS file (be sure to change the drive and path as necessary):
C:
CD \ULP
ULP -CULP.BIN
ULP will lock the duplication database, preventing any other program from
accessing them (ULPTEST and ULPDB, specifically). Therefore, I strongly suggest
you have all nodes down when running ULP.
Once you have ULP configured, before you run ULP for the first time, the
process data file must be initialized using the -I parameter. The syntax is as
follows, but please refer to the manual operation section below for more
information:
ULP -CULP.BIN -I
This is only done ONCE, and never needs to be done again; ULP maintains this
file as a course of action during normal operation.
FOR THE INTEGRATED CRC-32 DUPLICATION CHECKER (HIGHLY RECOMMENDED):
───────────────────────────────────────────────────────────────────
Initially, you must create the CRC database from scratch. If you have mostly
ZIP and ARJ files, then this should be very quick (on the order of 10 minutes
per 1000 archives for my 10MHz AT clone on a hard disk; a CD-ROM will be
somewhat slower). After you edit the configuration file, go to the ULP
directory and this command will create and compile your database of your
current downloadable file base:
ULPDB -CULP.BIN -A@C:\PCB\GEN\DLPATH.LST -S
The '@' symbol (a '!' may also be used) indicates to ULPDB that the argument is
a list file, not a directory. The list file must have one path per line, and
each path must end with a trailing backslash. In general, you may simply use
your PCBoard DLPATH.LST file, unless you index your downloads.
If you would like to include any off-line files in your CRC database (or have
to add files originally not included in the database) you can temporarily copy
your offline files to a temp directory (for the sake of argumentation, let's
call it "C:\TEMP\"; note the trailing backslash). You can then add and compile
them into the CRC database with the following command (no list symbol!):
ULPDB -CULP.BIN -AC:\TEMP\ -S
After you've added the offline files, just delete them from the disk, and if
someone uploads a file that you already have, it will be rejected.
* NOTE: The default method of database generation will be the fastest mode
since ULPDB will scan ARJ and ZIP archives for CRCs without unpacking.
This is not as accurate as recursively unpacking the archives since
ULPDB can only 'see' one level deep into an archive; nested information
will not be obtained. If this poses a problem, add the -U parameter to
the end of the command lines above; this will force ULPDB to unpack all
archives and compute the CRCs. This will be significantly slower than
the default method, but 100% accurate.
If you already have a FWKCS duplication database, you can import and translate
it into the ULP database by using the -I command line parameter. ULPDB will
prompt you for which database format it is, and will then proceed to import it
into the ULP database defined in ULP.BIN:
ULPDB -CULP.BIN -IC:\FWKCS\CSLIST -S
Once you have your database built, you should also perform maintenance on the
duplication database files regularly. This will compile any new data from the
day's uploads into the main database, and remove any added temporary data from
ULPTEST. This is not required to be done every day, but it will speed the
duplicate checking in ULP and ULPTEST. To perform database maintenance, add the
following line to your PCBoard EVENT.SYS after the event-mode processor has
run:
C: ──────┐ These lines are repeated
CD \ULP ├───── for clarity.
ULP -CULP.BIN ──────┘
ULPDB -CULP.BIN -S
ULPDB also locks the CRC-32 database, preventing any other program from
accessing them. I strongly suggest you have all nodes down when running ULP.
IF YOU USE PCBOARD'S 'TESTFILE.EXE' UTILITY:
────────────────────────────────────────────
* NOTE: If you are unfamiliar with the operation of PCBTEST.BAT, you will need
to refer to the PCBoard manual for an explanation of PCBTEST.BAT and
TESTFILE.EXE.
In general, the use of TESTFILE.EXE is not necessary with ULPTEST. ULPTEST is
fully capable of handling most any situation that arises. However, if you must
use TESTFILE.EXE for some reason, in order to take advantage of the upload
pre-verification abilities of ULPTEST, you must add the extension 'ULP' to the
TESTFILE.EXE extension list. This will allow the file 'VERIFY.ULP' to pass
through the filter created by TESTFILE so that ULPTEST may scan it. You will
also have to add 'EXE' to your extension list if you use TESTFILE.EXE, to allow
SFX archives to be processed by ULPTEST.
FOR THE ONLINE UPLOAD TESTING (OPTIONAL):
─────────────────────────────────────────
Online testing of archives can greatly reduce the amount of garbage your BBS
can pick up from user's uploading defective archives, ancient files, and
duplicates. The theory that ULPTEST uses is to simply check the archive AS
FAST AS IT CAN for archive integrity, duplication, and age, and then return
the user to greater and better things. ULP.EXE will handle everything else
during the system event.
* NOTE: In order to use ULPTEST, you must set PCBoard to swap by adding the
/SWAP switch to the SET PCB= environment variable in BOARD.BAT (in
PCBoard 15.0, this is in PCBSETUP, not the environment variable).
Otherwise, there will not be enough memory for ULPTEST to execute
programs like ARJ, SCAN, etc.
To use ULPTEST.EXE for on-line testing of archives, use the following command
line in PCBTEST.BAT (refer to the sample PCBTEST.BAT included in the ULP
distribution archive, and the PCBoard manual, to get the general logic of
PCBTEST.BAT's operation). Note the explicit paths to the ULPTEST executable and
the ULP configuration file:
C:\ULP\ULPTEST -CC:\ULP\ULP.BIN -F%1 -M%2 -D%3
ULPTEST will garner any other information that is required for operation from
the PCBOARD.DAT and PCBOARD.SYS files. In most installations, ULPTEST will
locate these two system files via the environment variables PCBDAT, PCBDRIVE
and PCBDIR. If the environment variables are used, PCBDIR and PCBDAT are
required; PCBDRIVE is required only if the BBS drop files exist on a different
drive than PCBTEST.BAT is being executed from.
* NOTE: Don't rely on PCBoard to create these variables; it only does so prior
to executing a door. As a last resort, ULPTEST will look in the current
directory for the files it requires.
If you have the environment variables disabled in PCBoard (using the /NOENV
switch), you can define the location of these files by adding the -P parameter:
C:\ULP\ULPTEST -CC:\ULP\ULP.BIN -F%1 -M%2 -D%3 -PC:\PCB\NODE1\
Or, or you can provide the port and node number information directly to ULPTEST
by using the -N, -I, -A and -B command switches:
C:\ULP\ULPTEST -CC:\ULP\ULP.BIN -F%1 -M%2 -D%3 -N5 -I4 -A3F8 -B38400
where -N is the node number, -I is the port IRQ line, -A is the port base
address (in hex) and -B is the baud rate of the port. If you are using the
DSZPORT environment variable to define the port IRQ and address to DSZ, ULPTEST
will use this information as well.
In addition, regardless of the method used to inform ULPTEST of the port
information, ULPTEST defaults to using hardware handshaking. If you want to use
software (XON/XOFF) handshaking or both, use the -H command switch with either
HARDWARE, SOFTWARE or BOTH as the argument:
C:\ULP\ULPTEST -CC:\ULP\ULP.BIN -F%1 -M%2 -D%3 -Hboth
ULPTEST is also capable of using a FOSSIL driver, and automatically detects if
you are running PCBoard /M and uses the appropriate FOSSIL port. However, if
you wish to force ULPTEST to use a FOSSIL driver (for example, to access high
IRQs), use the -X command switch with the FOSSIL port number as the argument:
C:\ULP\ULPTEST -CC:\ULP\ULP.BIN -F%1 -M%2 -D%3 -X1
During the course of testing, ULPTEST will output the current testing status to
the user online. Note that this information is not displayed locally; it is
being transmitted to the online user only. The program header can be disabled
in registered versions of ULP, making ULPTEST seamless to look like a part of
PCBoard itself. A sample follows:
Verifying FILENAME.ZIP...
┌────────────────────────────────┐
│ UpLoadProcessor 1.23 │ Registered to: Bloom Beacon-Picayune
│ Copyright 1992-93 Stacy Smith │ Serial number: 1000
└────────────────────────────────┘
Identifying file archive format...ZIP
Screening archive for corruption...OK
Unpacking archive...OK
Testing files for viruses, etc...OK
Checking for duplicate files...OK
Checking average age of files...OK
Archive passed.
ULPTEST will return a DOS errorlevel of 0 if the archive is OK, and an
errorlevel of 1-12 if there was an error (depending on the error). This allows
the PCBTEST.BAT batch file to branch based upon the results of the ULPTEST. The
appendix to this document lists all of the DOS errorlevels set by various ULP
programs.
ULPTEST also creates the requires PCBPASS.TXT and PCBFAIL.TXT for PCBoard to
determine if the archive passed or not.
┌────────────────────┐
│ 9. Configuration │
└────────────────────┘
The sample configuration file included in the distribution archive is heavily
commented, but some additional information is provided below:
GENERAL PARAMETERS:
───────────────────
REG_CODE
The registration code EXACTLY as provided to you on your registration
letter. If this is an unregistered version, comment out this line. Note
that this is case-sensitive, as the serial number, BBS name and sysop name
are all encrypted into this code.
BBS_NAME
The BBS name EXACTLY as provided to you on your registration letter. If
this is an unregistered version, comment out this lines. Note that this is
case-sensitive!
SYSOP_NAME
The sysop name EXACTLY as provided to you on your registration letter. If
this is an unregistered version, comment out this line. You get the picture
by now...
SWAP
If you wish the ULP programs to swap themselves out of memory prior to
executing external programs, set this parameter to YES. The ULP programs
will attempt to swap to EMS, XMS and lastly, the hard disk. If you do not
want this feature, enter NO or comment the line out. This feature will be
of most benefit to users using ARJ to compress archives, networked systems
and small DESQview windows. Use this feature ONLY if you have a memory
crunch!
INTERNAL_DEL
If you wish the ULP programs to delete the extracted files themselves,
versus using DOS, set this parameter to YES. If not, enter NO or comment
the line out. In some cases, letting DOS do the work is faster, but foreign
language implementations of DOS may prevent this ability.
DISK_BUFFER
The size of the disk I/O buffer you wish to allocate in bytes (valid values
are between 2048 bytes and 16384 bytes). Optimal performance is obtained
with a value that is a multiple of 2048 bytes, since this is the size of a
typical hard disk cluster. 4096 seems to be a good number for me, although
a larger size will improve performance somewhat. If you find yourself
running out of memory when executing archivers or file checking programs,
reduce this number to increase your available memory. If no value is
specified, the disk buffer will default to 4096 bytes.
LOG_FILE
All of the ULP programs will write process and error information to a log
file, if desired. This is the path/filename of the log file for ULP to
record its activities. Comment out if you don't want any disk logging.
LOG_MODE
If you want ULP and ULPTEST to record their statistics and finding for each
archive processed to the log file, use the VERBOSE keyword. If you only
want errors and summary statistics logged to disk, use the TERSE keyword or
comment the line out. VERBOSE mode will quickly produce a large log file,
but gives much insight into what ULP is doing and why a particular upload
failed.
DIR_CFG_FILE
The path/filename of the upload directory configuration file for ULP. This
file is automatically created by ULPCFG.EXE from data contained within the
ULP.CFG text file, and must be defined for ULP to operate.
PROC_DAT_FILE
The path/filename of the ULP process data file. This file contains
information on previous ULP archive processing and is maintained
automatically by ULP. This file must exist, or ULP will not operate; to
intitialize the process data file, use the -I command line switch in
ULP.EXE.
WORK_DIR
If you want ULP, ULPTEST or ULPDB to use a specific directory for storage
while processing, enter the full pathname (including the trailing
backslash). The drive letter MUST be included. If this directory does not
exist at runtime, the ULP programs will attempt to create it. This can be
set to a RAM disk for greater speed. If you do not desire to use a specific
directory, comment it out (ULPDB will extract at its current location on
disk). If a RAM disk is used (which will greatly speed processing), be sure
that largest expected archives will fit on the RAM disk decompressed. If an
archive will not fit onto a RAM disk, ULPDB will flag it as a bad archive
in the log file and not record its CRCs.
ERR_DISP
This defines how ULP dispositions archives found to have an error other
than a file check failure. These kind of errors take the form of ZIP CRCs,
etc., and are not potentially system damaging. Under some conditions, these
archives can be restored. If you want ULP to delete the archive and
description, enter DELETE; if you want ULP to rename the archive to a .ERR
extension, type RENAME or comment the line out.
VIRUS_DISP
This defines how ULP dispositions archives that fail a file check, such as
a virus scan. These errors are (obviously) potentially system-damaging. (I
wouldn't keep them if I were you.) If you want ULP to delete the archive
and description, type DELETE; if you want ULP to rename the archive to a
.CHK extension, type RENAME or comment the line out.
VIEWER
The command line for an ASCII file viewing (or editing) utility, such as
Vernon Buerg's venerable LIST utility. This will be called by ULPDB to
facilitate viewing of an archive information file generated for the
purposes of archive review. Place the @FILES@ variable where the filename
is to be entered on the utility's command line.
UPLOAD DIRECTORIES:
───────────────────
DIR_CFG_FILE
The path/filename of the upload directory configuration file for ULP. This
file is automatically created by ULPCFG.EXE from data contained within the
ULP.CFG text file, and must be defined for ULP to operate.
Each pair of upload directories (one source, and one destination), is denoted
by UL_DIR x, where x is a sequential number. An unlimited number of directory
pairs can be processed by ULP. The next 5 keywords are REQUIRED after the
UL_DIR statement; do not comment any of them out or insert other parameters or
comments in between them:
UL_DIR = denotes the beginning of a pair of upload directories
definition.
SOURCE_PATH = the directory where the uploads are placed by PCBoard.
This must include the trailing backslash! NOTE: if you
are using the 2 public directory setup, this parameter
will be the INITIAL public upload directory; the same
goes for the next two parameters).
SOURCE_LIST = the source upload directory list file that contains the
archive descriptions, including the path.
SOURCE_HDR = the number of lines to ignore at the beginning of the
source upload directory list file (for headers,
detachable header filespecs, etc.)
DEST_PATH = the destination directory where the uploads can be placed
by ULP after successful processing. This must include the
trailing backslash! (Leave blank if all uploads are
public).
DEST_LIST = the upload directory list file that contains the archive
descriptions, including the path. ULP will append this
file with the descriptions of successfully processed and
move archives. (Leave blank if all uploads are public).
For example, if you want all uploads to be private, you would define the
upload directory as (noting that the SOURCE_HDR should be set to the
appropriate value):
UL_DIR 1
SOURCE_PATH c:\private\
SOURCE_LIST c:\pcb\gen\privdir
SOURCE_HDR 1
DEST_PATH c:\public\
DEST_LIST c:\pcb\gen\publdir
If you want all uploads to be public, you would define it as:
UL_DIR 1
SOURCE_PATH c:\public\
SOURCE_LIST c:\pcb\gen\publdir
SOURCE_HDR 1
DEST_PATH
DEST_LIST
AGE-LIMIT CHECKING:
───────────────────
AGE_CHK
If you want ULP to fail an archive based on the average age of the files
contained within, enter YES; otherwise, enter NO or comment out the line.
AGE_LIMIT
The maximum average age in months, that you will accept for an upload. (If
the ERR_DISP is set to RENAME, it will be renamed to a .ERR extension).
ARCHIVING CONTROL FLAGS:
────────────────────────
ARC_CVT
If you want ULP to convert the archive to a default archiving format of
your choice, type YES; if not, type NO or comment the line out. This only
affects the 'outside' level of the archive. This flag has no affect on the
conversion of internal, nested archives.
ARC_NEST_CVT
If you want ULP to convert any internal archives found inside the archive
to your default archiving format, type YES; if not, type NO or comment the
line out, and they will be rearchived using the original archiving method
(using the original extension, even if it isn't that archiver's default
extension). The benefit of converting archives is it may save some disk
space; on the down side, it will potentially hose automatic installation
programs/batch files that may be contained within the archive.
SFX_CVT
If you want ULP to convert a self-extracting (SFX) archive to your default
archiving format, type YES; if not, type NO or comment the line out. If NO
is selected, then the archive will be processed, and then left alone if it
passes (NOT re-archived!). The reasoning for this is the LHA autostart
batch files and other similar features; they are usually used for novice
users and for automatic installation. By converting SFX archives, this may
save some disk space, but can cause inconveniences. For example, if you
don't have PKZ110.EXE on your BBS, a novice user has no way of obtaining
the PKZIP utilities from your BBS.
* NOTE: If there is a BBS ad in a SFX archive, and SFX_CVT is set to 'no',
the ad will remain in the archive. Can't be helped...
KEEP_AV
If you want ULP to keep ZIP archives with authenticity verification stamps
(-AV) regardless of the other archiving control flags, then set this to
'yes', or comment it out. If you set this flag to 'no', then the archive
will be recompressed per the other control flags.
* NOTE: If there is a BBS ad in a -AV stamped archive, and KEEP_AV is set
to 'yes', the ad will remain in the archive.
KEEP_ENV
If you want ULP to keep ARJ archives with security envelopes (similar to a
ZIP -AV) regardless of the other archiving control flags, then set this to
'yes' or comment it out. If you set this flag to 'no', then the archive
will be recompressed per the other control flags.
* NOTE: If there is a BBS ad in a secured archive, and KEEP_ENV is set to
'yes', the ad will remain in the archive.
ARCHIVER DEFINITIONS:
─────────────────────
Each archiver is denoted by ARCHIVER x, where x is a sequential number. Up to 9
archivers can be executed by ULP. ULP is capable of automatically detecting
ARC, ARJ, HYP, LZH, PAK, SQZ, ZIP and ZOO archives. The next 7 keywords are
required after the ARCHIVER statement; do not comment any of them out or insert
other parameters or comments in between them. I would also advise against
changing the command options I have set, unless you have need to do so. Some
are there to ensure uninterrupted processing (e.g. /m1 in LHA). Any other
archiver can be added, if desired; the only requirement is that the archiver
must return a DOS errorlevel so that ULP can determine if it ran properly. If
you do not explicitly define the path to your archiver(s), it must in a
directory included in the DOS PATH environment variable.
* NOTE: The ARJ and ZIP formats MUST be included for ULP to function properly!
ARCHIVER = denotes the beginning of an archiving program
definition.
ARC_EXT = the extension for this particular archiving format.
ARC_PRIMARY = if this is the default archiving format you use on your
BBS, type YES; if not, type NO.
ARC_PACK = the command line to pack archives, including all
command-line switches desired. Place the variable
@ARCHIVE@ where the filename is to be inserted in the
command line and @FILES@ where the filespec is to be
inserted. If you want a comment to be inserted, you must
include it in the command-line switches. This cannot
be a batch file! (DOS re-direction is allowed)
ARC_COMMENT = the command line to comment archives. Place the variable
@ARCHIVE@ where the filename is to be inserted in the
command line. If the archiver does not support comments,
or if you do not wish to add comments, leave the line
blank.
ARC_UNPACK = the command line to unpack archives, including all
command-line switches desired. Place the variable
@ARCHIVE@ where the filename is to be inserted in the
command line. This cannot be a batch file! (No DOS
re-direction allowed)
ARC_TEST = the command line to test archives, including all
command-line switches desired. Place the variable
@ARCHIVE@ where the filename is to be inserted in the
command line. This cannot be a batch file! (No DOS
re-direction allowed)
ARC_ERR = the errorlevel that the archiver returns upon success.
This information should be available in the program's
documentation.
UNCOMPRESSED FILES TO PROCESS:
──────────────────────────────
PROC_EXT
ULP can process selected uncompressed uploads, using this option. Up to 10
uncompressed file types can be specified by extension. Do not use any
compression format extensions (ARJ, ZIP, etc.)! Any files that are uploaded
with one of these extensions and could not be identified by signature will
be file-checked, duplicate-checked, and compressed using your default
archiving method. If you do not want to use this feature, simply remove or
comment out these lines. It probably is not a good idea to have EXE as an
extension to compress, since ULP is not able to detect ARC, HYP or ZOO
self-extractors, and they would be packed as an uncompressed archive.
FILES TO NEVER PROCESS:
───────────────────────
NO_PROC_EXT
ULP can be forced to NOT process selected uploads, using the NO_PROC_EXT
keyword. Up to 10 file types can be specified by extension. THESE FILES
WILL BE AUTOMATICALLY PASSED; be sure this is desired. This is intended for
use with non-DOS uploads, such as the Unix Tar Z format, or Macintosh
files. Note that these files will NOT be uncompressed, tested for viruses,
duplication, age, etc.. Please note this fact to your users of these files.
FILE-CHECKING (VIRUS SCANNING, ETC.):
─────────────────────────────────────
Each file checker is denoted by FILE_CHECKER x, where x is a sequential number.
Up to 3 file checking programs can be executed by ULP. The next 2 keywords are
required after the FILE_CHECKER statement; do not comment them them out or
insert other parameters or comments in between them. Any kind of file checker
may be used, such as a virus checker, or a 3rd party CRC duplication checker,
if desired. If you do not explicitly define the path to your file checking
program(s), it must in a directory included in the DOS PATH environment
variable. The only requirement is that it must return a DOS errorlevel,
otherwise ULP has no clue if there was an error or not. Note that if you are
running a network, the use of network-specific scanners may be necessary.
* NOTE: If you are using McAfee's SCAN (most sysops do), be sure you are using
the correct program. SCAN is fine for single node and multitasker
systems and scanning a local drive from the network server, but NETSCAN
is required to scan a network drive from a workstation.
FILE_CHECKER = denotes the beginning of a file-checker definition.
FILE_CHK_CMD = the command line to execute each file checker, including
all command-line switches. Place the variable @FILES@
where the filespec is to be placed on the command line.
This cannot be a batch file! (No DOS re-direction
allowed)
FILE_CHK_ERR = the errorlevel that the file checking program returns
upon success. This information should be available in
the program's documentation.
GIF FILE CHECKING
─────────────────
MIN_WIDTH
If ULP is to reject GIF files based upon picture width, enter the minimum
acceptable width in pixels. If no width limit is desired, set it to zero or
comment out the line.
MIN_HEIGHT
If ULP is to reject GIF files based upon picture height, enter the minimum
acceptable height in pixels. If no height limit is desired, set it to zero
or comment out the line.
MIN_COLORS
If ULP is to reject GIF files based upon the number of colors in the
picture, enter the minimum acceptable number of colors. If no color limit
is desired, set it to zero or comment out the line.
KEEP_GIFLITE
If ULP is to keep uploaded GIF files if they have been compressed with
GIFLITE, set this to YES. If not, set it to NO or comment it out. At this
time, if you use GIFTEST, this parameter will have no effect.
* NOTE: Accepting GIFLITEd GIF files can compromise the duplication system
due to its variable compression ratio! I strongly suggest you do
not accept GIFLITEd GIF files!
Each external GIF file checker is denoted by GIF_CHECKER x, where x is a
sequential number. Up to 3 GIF file checking program(s) can be executed by ULP.
In addition, do not use the GIF file checking program's internal duplicate
checking or file handling abilities, if any. ULP and ULPTEST will take care of
these operations. The next 2 keywords are required after the GIF_CHECKER
statement; do not comment them out or insert other parameters or comments in
between them.
GIF_CHECKER = denotes a new file checking/virus program definition.
GIF_CHK_CMD = the command line to execute each GIF file checker,
including all command-line switches. Place the variable
@FILES@ where the filespec is to be placed on the command
line. This cannot be a batch file! (No DOS re-direction
allowed)
GIF_CHK_ERR = the errorlevel that the GIF file checking program returns
upon success. This information should be available in the
program's documentation.
DESCRIPTION PROCESSING:
───────────────────────
WORD_WRAP
If you want ULP to word-wrap inserted descriptions, enter YES for smart
word-wrapping, otherwise, type NO or comment the line out. Note that ULP
now checks for boxes, etc. before attempting a word-wrap.
INT_DESC
ULP can extract and insert FILE_ID.DIZ and DESC.SDI internal description
files into the DIRxx files. If you want ULP to insert internal description
files, type YES; if not, type NO or comment the line out.
DESC_LIMIT
If ULP is to insert internal description files, the following parameter is
the maximum number of lines that ULP will insert from the description file.
This defaults to 10 if the line is commented out. Clark Development Company
(the developer of PCBoard and FILE_ID.DIZ) has defined that a FILE_ID.DIZ
file should have no more than 10 lines, but ULP will insert up to 15 lines.
INFO_LINE
ULP can add a line of archive information for the benefit of your users.
The format is completely defined by you. If you want ULP to add a single
line to the archive description, type YES; if not, type NO or comment the
line out. Note that the statistics generated by ULP will not include any
FILE_ID.DIZ or DESC.SDI files found in the archive.
INFO_INDENT
The column where the | marker goes in your directory listing setup. The
standard PCBoard setup indents to column 32.
ARC_INFO_FMT
The format that you would like for the archive information line (do not
include the '|' character or the space; ULP will add those). This format is
completely configuration by the user; any ASCII character can be used. To
include specific information compiled by ULP, just add the variables. The
legal variables are (must be in all caps):
@#@ = Number of files in the archive (and nested archives)
@NEWEST@ = Date of the newest file in the archive
@OLDEST@ = Date of the oldest file in the archive
@BYTES@ = The total number of bytes (uncompressed) in the archive
@DATE@ = Today's date
@CR@ = Inserts a line break for multiline capability
GIF_INFO_FMT
The format that you would like for the GIF information line (do not include
the '|' character or the space; ULP will add those). The legal variables
are (must be in all caps):
@WIDTH@ = Horizontal resolution (number of columns)
@HEIGHT@ = Vertical resolution (number of rows)
@COLORS@ = Number of colors in GIF
@COMPRESS@ = Inserts "GIFLITE" or "None", depending upon compression
@DATE@ = Today's date
@CR@ = Inserts a line break for multiline capability
DOWNLOAD.TXT UPDATING:
──────────────────────
UPDATE_DLTXT
If you want ULP to scan your DOWNLOAD.TXT file and update the file
extension of the upload to reflect the correct archiving method used, enter
YES. If not, enter NO or comment the line out. This feature will allow
utilities that scan DOWNLOAD.TXT for file information to reflect correct
information.
* NOTE: If you are using a utility that creates an independent database
from the DOWNLOAD.TXT file, such as PCBFX version 2.x, be sure
ULP.EXE is run in the event BEFORE that utility so the independent
database reflects the changes that ULP makes to DOWNLOAD.TXT.
DLTXT_FILE
The full path and name for the PCBoard DOWNLOAD.TXT file.
ULPTEST TESTING:
────────────────
TEST_MODE
This defines how ULPTEST processes an archive. NORMAL indicates a full test
is to be performed, identical to the ULP event processing except that the
archive formats are not converted or recompressed (this is a limitation of
PCBoard). FAST indicates a fast test in which the archive is scanned for
archive integrity, duplicates, and age without unpacking the archive (ARJ
and ZIP files only). If this mode is used, it should be held for later
processing by ULP so that it can be file-checked for viruses, etc. Also,
FAST mode can only 'see' one level deep in the archive. If you have a slow
computer and/or hard disk, FAST mode would be preferable for your users
when large archives are uploaded. If you comment this line out, NORMAL mode
is assumed. I recommend you use NORMAL mode, and then set a limit with the
next parameter for when FAST mode will be utilized.
TEST_LIMIT
If NORMAL mode is used, the archive's file size will first be compared
against this limit (in kilobytes). If it is larger than this number of
kilobytes, it will be forced into FAST mode to speed the process and
prevent wasting the user's online time. Entering 0, or commenting the line
out will disable this feature, resulting in NORMAL mode being used at all
times. If TEST_MODE is set to FAST, this parameter has no effect. For my
system, a good value is 300 kilobytes, as anything much bigger than this
takes quite awhile to decompress, test, and delete the files extracted.
COMM_IO
If you want ULPTEST to output status information to the user on-line, set
it to YES; if you do not want ULPTEST to output status information, enter
NO or comment the line out. ULPTEST supports both direct interrupt-driven
I/O and FOSSIL driver I/O.
SUPPRESS_HDR
If you want ULPTEST to suppress the program header output to the user via
the COM port (the program name, copyright notice, and registration
information). This gives a seamless look for ULP into PCBoard. If this is
desired, set this to YES; otherwise, set to NO or comment out the line. * *
NOTE: This will work for registered users only!
BBS AD HANDLING:
────────────────
ADS_CHK
If you want ULP to remove known BBS ads from new uploads during processing,
type YES; if not, type NO or comment the line out.
ADS_DB
The directory path and name for the BBS ads database. This database can be
updated by the sysop by using the ULPADS utility (see section 11 of this
document), freeing the sysop from being reliant on me to maintain the ads
database. If you find a large number of new BBS ad files, please upload
them to my board for inclusion into my master ads database.
BBS_AD_FILE
The directory path and filename for your BBS ad to be added to each and
every archive processed by ULP. Note that you must provide a path, or it
will not be copied. I STRONGLY recommend against them; BBS ads are
aggravating to other sysops, so please be considerate and use the ARJ or
ZIP comment instead. BBS ad files are the scourge of the BBS world. This
parameter is commented out by default.
CRC-32 DUPLICATE FILE CHECKING:
───────────────────────────────
DUPE_CHK
If you want ULP to use its internal CRC-32 duplication checking, type YES;
if not, type NO or comment the line out. If you want to use a third-party
duplication checker, such as ZDCS, refer to Appendix B for installation
information. I would strongly suggest using ULP's, as it is fast and
compact, not to mention integrated, eliminating the time to shell to disk
for each file in each archive to be tested.
DUPE_LIMIT
The minimum percentage of duplication allowed for a file to pass. Anything
smaller will pass regardless of the executable duplication limit. I would
suggest this number be set to around 50, as some program authors don't
modify every file, especially with maintenance releases.
EXE_DUPE_LIMIT
The maximum percentage of duplication allowed for executable files and
overlays to pass. This number should be higher than DUPE_LIMIT; I would
recommend about 80. This will catch archives that are duplicates, but
someone has changed enough files to normally pass it when uploaded (such as
configs, high score files, etc.). This will also pass files where the
author has changed nothing but the executables; they would have failed
before. Executables are determined by extension; any file with one of the
following extensions will be counted as an executable: EXE, COM, SYS, OVL,
OVR, DLL.
DUPE_DB
The path and name for the CRC-32 database file. It's a good idea to keep
the database, index, and new data files all together, just to make things
easier.
DUPE_IDX
The path and name for the CRC-32 database file index.
┌────────────────────────┐
│ 10. Manual Operation │
└────────────────────────┘
While the ULP system is mostly automatic, there are occasions where some
operations may have to be done manually:
ULP:
────
During the course of operation, ULP will rename archives that have been found
to be defective in some manner according to the following convention:
.UNK Unknown archive format
.DOS DOS reserved keyword found in archive
.ERR Error occurred while unpacking archive (archive integrity failure)
.CHK Error found while file checking archive file (virus, etc.)
.DUP Excessive duplicate files contained in archive
.PCK Error occurred while repacking archive file
.AGE Age limit exceeded by archive file
.ENC Encrypted file found in archive
.BOM Bomb file detected in archive
.BAD Error found while testing GIF file
.RES Unacceptable GIF resolution
.GLT GIF compressed with GIFLITE
I would enter these extensions in your UPSEC file, as ULP will automatically
pass over any file with these extensions, presuming that they are defective
archives. Doing so will prevent a user from uploading a file with one of these
"reserved" extensions.
If you feel that these files are acceptable after reviewing them, you can force
them to be accepted by adding the command-line switch -O (for override) to the
ULP command line, e.g.:
ULP -Culp.bin -O
This will accept the archives regardless of duplication or age limits. It will
not override unpacking, packing, integrity and virus errors, however, for the
safety of your board and your users.
To initialize the ULP process database, use the -I command line switch:
ULP -Culp.bin -I
ULP will scan your source upload directories as configured and generate a
process data file. From then on, ULP will maintain this data file
automatically.
Three other command-line parameters are available in ULP. '-R' will retest all
archives found in the subdirectory passed to it. It will not use duplication
checking or age checking, nor will it repack the archive; it was installed to
allow the periodic retesting of archives for viruses, etc.:
ULP -Culp.bin -Rc:\temp\
Another, '-M' will retest and convert all archives found in the subdirectory
indicated; this can be used for mass archive conversion:
ULP -Culp.bin -Mc:\temp\
Note that the '-R' and '-M' switches cannot be used on a CD-ROM or other
write-protected media at this time.
ULPDB:
──────
If a file has been failed for some reason, you can view the contents of the
archive using the -V command switch with ULPDB. This will scan the archive, and
generate a report of the results, which are indicative of why it failed. For
example:
ULPDB -Culp.bin -Vc:\privul\archive.dup
This will produce a report of the archive ARCHIVE.DUP located in the specified
path. This can be useful in determining if an archive that failed duplication
should be removed or should be accepted anyway, and similar reasons.
In addition, duplication databases can be merged using the -M command-line
option. In order to merge a second database into your main database:
ULPDB -Culp.bin -Msecond.db
where 'second.db' is the full path and filename for the second duplication
database to be merged with the main CRC-32 database. This will allow pre-built
databases to be constructed for CD-ROM disks for easy implementation.
Finally, database entries can be purged from the database using the file date
contained in the database record using the -P command switch:
ULPDB -Culp.bin -P60
where '60' is the maximum age (in months) for a file to remain in the database.
Removal is based upon the file date represented by the entry, NOT the date the
file entry was made into the database; these are not one and the same. In the
example above, all records for files older than 60 months (5 years) will be
removed from the database. As a protective measure, ULPDB will not perform a
purge if the value specified is less than 6 months.
┌──────────────────────────┐
│ 11. BBS Ads and ULPADS │
└──────────────────────────┘
The ULP system includes a BBS ad removal feature based on CRC-32 calculation of
the file contents and other data. In this fashion, ULP can detect a known ad
file despite the file name and date.
In order for sysops to be able to 'keep up' with new ads produced by the weenie
sysops who insert the @!&*#%$ things, I have included a utility called ULPADS
that can scan a BBS ad file, and update the BBS ads with it's information.
Don't worry about duplication, as part of the process is to purge duplicate BBS
ad info. The command-line syntax for the ULPADS.EXE program is simply:
ULPADS -Ddatabase.ad -Fbbs.ad
where 'database.ad' is the full path and filename for the BBS ads database, and
'bbs.ad' is the path and filename for the BBS ad file to be added to the ads
database.
If you have a subdirectory full of BBS ads, the following batch file provided
by Ian Singer may be of help:
rem DELADS.BAT
rem this batch file written by Ian Singer 1 Jul 92 to take all
rem the bbs ads in c:\ulp\ad and run them against the database
rem in c:\ulp
rem
for %%f in (c:\ulp\ad\*.*) do ULPADS -Dc:\ulp\ulpads.db -F%%f
I would greatly appreciate your uploading of any new BBS ad files that you may
collect over time to my BBS so I can update the master listing that I include
with the ULP distribution archive. Please refer to the top of this document for
my BBS number.
The latest version of my BBS ads database is included in the distribution
archive. In order to add whatever new BBS ads information that may be
contained, you can merge the two BBS databases:
ULPADS -Dmaster.db -Mother.db
where 'master.db' is the full path and filename for the master (your) BBS ads
database, and 'other.db' is the path and filename for the other BBS ads
datatbase to be merged with the master ads database.
┌───────────────┐
│ 12. Support │
└───────────────┘
If you require support for ULP, I can be contacted by any of the following
means:
∙ Fidonet netmail (node: 1:276/112)
∙ Intelec PCBoard conference
∙ RoseNet routed mail (system ID: BB-P)
∙ The ULP support conference (#42) on Salt Air (PCBoard support BBS)
∙ The Support conference (#2) on my BBS (listed at the top of this
document).
┌─────────────────────────┐
│ 13. The Future of ULP │
└─────────────────────────┘
ULP will be supported as long as I'm in the BBSing business (which will be
quite a while...once it's in your blood, you can never shake it <grin>). The
ULP system will be rapidly expanding it's features so it will be your first
choice in BBS upload processors. Some current plans:
∙ Add the ability to preprocess files prior to file checking them;
for example, decompress executables that have been PKLite'd.
∙ Support other BBS software directory listing formats.
If you have any other suggestions, or want other archivers supported, please
contact me via Email, U.S. snail-mail or on my BBS at the number at the top of
this document.
Thanks for giving ULP a try!
┌────────────────────────────────┐
│ Appendix A: DOS Errorlevels │
└────────────────────────────────┘
The errorlevels returned to DOS by the ULP system programs are consistent
among all of the programs (primarily with the error conditions); the following
is a list of the errorlevels than can be returned by the ULP programs, along
with which programs return the codes:
0 Successful execution (ULPTEST, ULPDB, ULPADS)
0 Successful execution, archives were processed (ULP)
1 Successful execution, nothing to do (ULP)
1 Unknown archive format (ULPTEST)
2 DOS reserved keyword found in archive (ULPTEST)
3 Error unpacking archive (archive integrity) (ULPTEST)
4 Error file checking archive files (virus, etc.) (ULPTEST)
5 Error duplicate checking archive files (ULPTEST)
7 Error commenting archive (ULPTEST)
8 Age limit exceeded by archive files (ULPTEST)
9 Encrypted file found in archive files (ULPTEST)
10 Bomb archive detected (ULPTEST)
11 Bad GIF file (ULPTEST)
12 Unacceptable GIF resolution (ULPTEST)
13 GIF compressed with GIFLITE (ULPTEST)
99 Help screen (executing a program with no or an insufficient
number of arguments) (All)
100 Could not close all open files (ULP, ULPTEST, ULPDB)
101 Unknown command line parameter (All)
102 Config file not found (All)
103 Invalid config file format (All)
104 Excessive number of command line parameters defined for
file checker (maximum number of parameters is 10) (ULP)
105 Excessive number of command line parameters defined for
archiver (max is 10) (ULP, ULPTEST, ULPDB)
106 Excessive number of words defined in archive information
line (max is 10) (ULP)
107 No primary archiver defined (ULP)
108 Invalid disk buffer size (All)
109 Upload directory list file(s) not found (ULP, ULPTEST)
110 Unable to allocate heap memory (All)
111 Unable to open or lock disk logging file (ULP, ULPTEST, ULPDB)
112 Invalid registration code or attempting to run 286 or 386
versions without registering (ULP, ULPTEST, ULPDB)
113 Unable to open/lock CRC database file(s) (ULP, ULPTEST, ULPDB)
114 Unable to open BBS ads database file (ULP, ULPADS)
115 Invalid filename provided by user on command-line (ULP, ULPTEST)
116 Invalid path provided by user on command-line (ULPDB)
117 Invalid or no node number (valid values are 1 to 256) (ULPTEST)
118 CRC database usage not enabled (configuration file parameter
DUPE_CHK is set to 'no') (ULPDB)
120 Invalid or old database format (All)
121 Unable to process com port/invalid port parameter (ULPTEST)
122 Unable to access PCBOARD.DAT/PCBOARD.SYS (ULPTEST)
123 CRC database failed validation test (ULPDB)
200 Undefined error (internal to ULP system program(s))
┌────────────────────────────────────────────────────────┐
│ Appendix B: Using ZDCS vs. ULP's Duplication System │
└────────────────────────────────────────────────────────┘
The following message should provide the information required to integrate ULP
and ZDCS:
===========================================================================
Packet: SALTAIR
Date: 05-11-93 (08:42) Number: 44
From: STACY SMITH Refer#: 1187
To: ALL Recvd: NO
Subj: ULP/ZDCS PCBTEST.BAT 1/2 Conf: (44) ZDCS
---------------------------------------------------------------------------
Between myself and another ZDCS user, we have come up with a simple way to
use ZDCS and ULP together by running ZDCS after ULPTEST in PCBTEST.BAT.
While I will continue to work on internal support for ZDCS within ULP, this
will make the combination quite functional in the interim.
===========================================================================
Packet: BB-P
Date: 05-11-93 (02:55) Number: 1187
From: JIM CONDON Refer#: NONE
To: ALL Recvd: NO
Subj: ULP/ZDCS PCBTEST.BAT Conf: (2) Support
---------------------------------------------------------------------------
After trying other Upload testing programs for PCBoard I think I have found
one that suits my needs and is reliable. I like the ZDCS dupe checking
system better then the one that comes with ULP. If you like what you got
stop reading! If you would like to use ZDCS along with ULP below is my
pcbtest.bat. It will not test file attaches (in PCBoard 15.0's message
base). If you want to do that remove line 6. It will use ULPTEST via
PCBoard's TEST command to check the integrity of files that are already
online. It will also test uploads using ULPTEST first. If it fails for any
reason it will go back to PCBoard without adding the bad upload to the ZDCS
database. If ULPTEST passes the file then ZDCSFC is called to add it to the
database. It will also test zdcstest.chk files without loading ULPTEST. You
will have to edit it slightly to reflect where to look for the file being
tested. Also your paths to ZDCSFC and ULP maybe different. Feel free to
use or modify (like I could stop you if I wanted to!) it to suit your needs.
I hope this helps.
@echo off
cls
IF EXIST PCBFAIL.TXT DEL PCBFAIL.TXT
IF EXIST PCBPASS.TXT DEL PCBPASS.TXT
IF EXIST ZDCS-DEL.LST DEL ZDCS-DEL.LST
if '%2 == 'ATTACH goto end
if %1 == D:\PRIVATE\ZDCSTEST.CHK goto zdcs
c:\ulp\ulptest -Cc:\ulp\ulp.bin -F%1 -M%2 -D%3
if exist pcbfail.txt goto end
if %2 == TEST goto end
goto ZDCS
:zdcs
c:\zdcs\zdcsfc %1 %2 %3
if exist d:\private\zdcstest.chk del d:\private\zdcstest.chk
:end
Line 1 and 2 are self explanatory I think!
Lines 3, 4 and 5 are just house keeping (extra precaution) that was in the
sample pcbtest.bat that came with ZDCS 2.0.
Line 6 is a line David Terry (CDC) suggests people running 15.0 add to pass
file attaches without testing.
Line 7 Checks to see if this is a ZDCSTEST.CHK file and if it is skips past
ULP and just runs ZDCS.
Line 8 is the ULPTEST command line from the docs (or was it the included
pcbtest.bat?) Anyway Stacy Smith's line.
Line 9 tests to see if ULPTEST failed the upload for ANY reason. I think
this way is better then errorlevel testing. If pcbfail.txt is found it
exits back to PCBoard and NEVER runs ZDCSFC. I feel if the upload
failed it shouldn't be added to the ZDCS database since someone else may
try to upload a good copy that would then probably fail the dupe check.
Line 10 checks to see if pcbtest.bat was invoked via the TEST command in
PCBoard and if it is skip ZDCSFC and brings PCBoard backup without
wasting time loading ZDCSFC just to have it pass control back to
PCBoard.
Line 11 Takes you to ZDCSFC. If it got this far it should be a valid upload
and ZDCSFC should test it and add it to the database.
Line 12 just a DOS label.
Line 13 the actual ZDCSFC command. I believe it is from a sample
pcbtest.bat that came with ZDCS 2.0.
Line 14 Checks for the zdcstest.chk file and deletes it. For some reason I
was sometimes getting one left over and callers couldn't upload a test
file till I killed it off. It probably isn't even needed.
Line 15 is just a Dos label.
P.S. Don't forget to configure ULP and ULPTEST not to use ULPDB
if you are using ZDCSFC instead. One dupe checker is enough!
===========================================================================
Some comments of my own regarding the above message:
Lines 3 and 4 are superfluous, but don't hurt anything. ULPTEST, as a matter
of design, cleans up any stray PCBPASS.TXT and PCBFAIL.TXT files.
Line 6 is not necessary when version 0.99ß is released in a week or two, as
ULPTEST is/will be fully compatible with PCBoard 15.0, including attach
mode. ULPTEST will test and log the results of testing an attachment,
while always passing the attached file (there's no reason to reject an
attachment, although it might be interesting to know if a virus is in
one).
Line 13 doesn't require the %3 parameter on the ZDCS command line. At this
point, ULPTEST has already inserted the FILE_ID.DIZ and DESC.SDI file
and ZDCS is just duplicating effort.
Also note that you cannot use the VERIFY.ULP method of pre-verifying
uploads; you will have to tell your users to continue using the ZDCSTEST.CHK
method. If there are any questions, please don't hesitate to ask. I can't
help with ZDCS questions, but I can help with everything else.
