FTP Mirror Tracker design
- Robot - The function of this component is to gather raw directory listings
from FTP servers for later processing.
- Summariser - This is the heart of the FTP Mirror Tracker. Its job is to parse the directory listings of the FTP servers which are being tracked, analyzes them, and creates MD5 digests based on this analysis.
- Digest exchange - We parse, for example, 100 MB of compressed
listings from FTP servers , create a 50 MB digest file and a 25 MB
links file for feeding into the Mirror Tracker database. After
compression, these files are 7 Mb and 2 Mb respectively. These files
can be shared among FTP Mirror Tracker servers in different
domains.
- Front end programs:
- Users (content providers) can query FTP Mirror Tracker directly for the
specific URLs using simple CGI programs.
- Web Cache and Mirror service integration
Copyright © 2000, Martin Hamilton and Alexei Novikov. All rights
reserved. This is free slideware, you can redistribute it under the
terms of the GNU General
Public License, version 2 or
later.