Next Generation FTP

OSI Telecommunications Model FTP has some fundamental design flaws that make it poorly suited for the ever evolving internet.

FTP is a session layer protocol in the OSI model.

The current fatal flaws of FTP

Structural


Implementation

Protocol
  1. Non-existent support for uploading large files from dialup accounts, the reverse scheme of what BitTorrent does for downloading.
  2. You can't restart an FTP upload or download session after your dialup circuit goes down, and if your client does support such an FTP feature it often does not work well.. BitTorrent handles stateless uploading and downloading with no problems at all. This lack of uplink and downlink self healing is a problem that needs to be fixed in FTP.
  3. FTP has a somewhat suboptimal bandwidth utilization with respect to transmitting files of mixed sizes.
  4. For files at websites that are designed to be downloaded by the general public, there is no BitTorrent like mechanism within FTP to oversee the transfers -- these FTP transfers would however be "Trackerless".
  5. Lack of protocol security: FTP streams can be decoded at all points along the transmission path.
  6. Secure FTP does not really fix (in a simplified way) the problems of point to point FTP link security.
  7. There is no way to guarantee file integrity using the current FTP protocol, although workarounds involving file date, time and size comparisons currently work. Still, requesting a file's CRC32 checksum (or MD5 sum) can't be done.
  8. There is no standardized (modular and upgradeable, using XML) way of transmitting file system meta-data, making Unix, Windows and Mac OS file system integration very difficult.
  9. Most FTP state machines are very non-standardized -- changes to the protocol must indeed standardize all FTP state machines. A modular protocol core is absolutely necessary for the future of FTP.
  10. FTP application program data structures vary all over the map, a public domain FTP kernel (with support for POSTIX threads) needs to be created.
  11. Extended Passive Mode: it is almost not used in current FTP; with FTP-NG it should be mandatory for all communications as it is more BitTorrent like.
  12. If a file is accidentally named in a non-compliant manner, current FTP methods for deleting or renaming it either don't work at all or fail to work depending upon the mix of client and server software. Files must be accessible by not only their name but their hash value or CRC. Hash values (and checksums)  must exist for the filename and the file contents as separate entities.

Proposed modifications of FTP for FTP-NG

Feature "FEAT" Code
 Record Structure
 Meaning
NGP0
 Announces that FTP-NG functions are available
 Informational, announces protocol is available; used when client requests features from server via "FEAT" operand: Answer back with AFPV=xxx, CRCV=xxx, HSHV=xxx, NGPV=xxx, VRST=xxx;
NGPV
 xxx (0000...9999)
 Announces FTP-NG protocol version number at the server



Hashes
 Record structure
 Meaning
HSHV
 xxx (000 ... 999)
 Hash support version number
SHA1
 Hex reply: [Filename-Hash] [File-Hash] Filename reply hash should be MD5
WRPL
 Hex reply: [Filename-Hash] [File-Hash] Filename hash should be Whirlpool



Checksums
 Record structure Meaning
CRCV
 xxx (000 ... 999)
 CRC version number
CRC1
 [Filename] [CRC-32-Castagoli]
 CRC-32-Castagoli
CRC2
 CRC-64-ECMA (Filename-Hex(UTF7)) Always client requests to the server
CRIV
 {0,1,2 ...9}{space} xxxx{:}xxxx
 Override CRC "Initial Value" (bi-directional)
CRFV
 {0,1,2 ...9}{space} xxxx{:}xxxx Override CRC "Final Value" (bi-directional)



Advanced file properties (meta-data)
 Record structure Meaning
AFPV
 xxx (0000...9999)
 What protocol version do you support for transfer of file system meta-data?
AFPQ
 AFPQ [Filename-SHA1]
 Request advanced file properties for filename with SHA-1 hash for verification
AFPR
 UUCP encoded XML-DB record
 The FTP server should maintain a shadow XML-DB record of file system properties for all accounts

It is assumed that these commands will be implemented after the "MODE" command.

Suggested FTP state machine modifications
Technical references

Created by
Max Power

Initial proposal
16 MAR 2007

Last Modified
28 SEP 2009