Welcome to pyftpdlib’s documentation¶

If you’re in a hurry just skip to the Tutorial.

Install¶

By using pip:

$ pip install pyftpdlib

From sources:

$ git clone git@github.com:giampaolo/pyftpdlib.git
$ cd pyftpdlib
$ python setup.py install

You might want to run tests to make sure pyftpdlib works:

$ make test
$ make test-contrib

Additional dependencies¶

PyOpenSSL, to support FTPS:

$ pip install PyOpenSSL

pysendfile, if you’re on UNIX, in order to speedup uploads (from server to client):

$ pip install pysendfile

Tutorial ¶

Table of Contents

Tutorial

Below is a set of example scripts showing some of the possible customizations that can be done with pyftpdlib. Some of them are included in demo directory of pyftpdlib source distribution.

The script below uses a basic configuration and it’s probably the best starting point to understand how things work. It uses the base DummyAuthorizer for adding a bunch of “virtual” users, sets a limit for incoming connections and a range of passive ports.

source code

import os

from pyftpdlib.authorizers import DummyAuthorizer
from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer

def main():
    # Instantiate a dummy authorizer for managing 'virtual' users
    authorizer = DummyAuthorizer()

    # Define a new user having full r/w permissions and a read-only
    # anonymous user
    authorizer.add_user('user', '12345', '.', perm='elradfmwMT')
    authorizer.add_anonymous(os.getcwd())

    # Instantiate FTP handler class
    handler = FTPHandler
    handler.authorizer = authorizer

    # Define a customized banner (string returned when client connects)
    handler.banner = "pyftpdlib based ftpd ready."

    # Specify a masquerade address and the range of ports to use for
    # passive connections.  Decomment in case you're behind a NAT.
    #handler.masquerade_address = '151.25.42.11'
    #handler.passive_ports = range(60000, 65535)

    # Instantiate FTP server class and listen on 0.0.0.0:2121
    address = ('', 2121)
    server = FTPServer(address, handler)

    # set a limit for connections
    server.max_cons = 256
    server.max_cons_per_ip = 5

    # start ftp server
    server.serve_forever()

if __name__ == '__main__':
    main()

Logging management ¶

pyftpdlib uses the logging module to handle logging. If you don’t configure logging pyftpdlib will write logs to stderr. In order to configure logging you should do it before calling serve_forever(). Example logging to a file:

import logging

from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import DummyAuthorizer

authorizer = DummyAuthorizer()
authorizer.add_user('user', '12345', '.', perm='elradfmwMT')
handler = FTPHandler
handler.authorizer = authorizer

logging.basicConfig(filename='/var/log/pyftpd.log', level=logging.INFO)

server = FTPServer(('', 2121), handler)
server.serve_forever()

DEBUG logging ¶

You may want to enable DEBUG logging to observe commands and responses exchanged by client and server. DEBUG logging will also log internal errors which may occur on socket related calls such as send() and recv(). To enable DEBUG logging from code use:

logging.basicConfig(level=logging.DEBUG)

To enable DEBUG logging from command line use:

python -m pyftpdlib -D

DEBUG logs look like this:

[I 2017-11-07 12:03:44] >>> starting FTP server on 0.0.0.0:2121, pid=22991 <<<
[I 2017-11-07 12:03:44] concurrency model: async
[I 2017-11-07 12:03:44] masquerade (NAT) address: None
[I 2017-11-07 12:03:44] passive ports: None
[D 2017-11-07 12:03:44] poller: 'pyftpdlib.ioloop.Epoll'
[D 2017-11-07 12:03:44] authorizer: 'pyftpdlib.authorizers.DummyAuthorizer'
[D 2017-11-07 12:03:44] use sendfile(2): True
[D 2017-11-07 12:03:44] handler: 'pyftpdlib.handlers.FTPHandler'
[D 2017-11-07 12:03:44] max connections: 512
[D 2017-11-07 12:03:44] max connections per ip: unlimited
[D 2017-11-07 12:03:44] timeout: 300
[D 2017-11-07 12:03:44] banner: 'pyftpdlib 1.5.4 ready.'
[D 2017-11-07 12:03:44] max login attempts: 3
[I 2017-11-07 12:03:44] 127.0.0.1:37303-[] FTP session opened (connect)
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[] -> 220 pyftpdlib 1.0.0 ready.
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[] <- USER user
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[] -> 331 Username ok, send password.
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] <- PASS ******
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] -> 230 Login successful.
[I 2017-11-07 12:03:44] 127.0.0.1:37303-[user] USER 'user' logged in.
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] <- TYPE I
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] -> 200 Type set to: Binary.
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] <- PASV
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] -> 227 Entering passive mode (127,0,0,1,233,208).
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] <- RETR tmp-pyftpdlib
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] -> 125 Data connection already open. Transfer starting.
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] -> 226 Transfer complete.
[I 2017-11-07 12:03:44] 127.0.0.1:37303-[user] RETR /home/giampaolo/IMG29312.JPG completed=1 bytes=1205012 seconds=0.003
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] <- QUIT
[D 2017-11-07 12:03:44] 127.0.0.1:37303-[user] -> 221 Goodbye.
[I 2017-11-07 12:03:44] 127.0.0.1:37303-[user] FTP session closed (disconnect).

Changing log line prefix ¶

handler = FTPHandler
handler.log_prefix = 'XXX [%(username)s]@%(remote_ip)s'
server = FTPServer(('localhost', 2121), handler)
server.serve_forever()

Logs will now look like this:

[I 13-02-01 19:12:26] XXX []@127.0.0.1 FTP session opened (connect)
[I 13-02-01 19:12:26] XXX [user]@127.0.0.1 USER 'user' logged in.

Storing passwords as hash digests ¶

Using FTP server library with the default DummyAuthorizer means that passwords will be stored in clear-text. An end-user ftpd using the default dummy authorizer would typically require a configuration file for authenticating users and their passwords but storing clear-text passwords is of course undesirable. The most common way to do things in such case would be first creating new users and then storing their usernames + passwords as hash digests into a file or wherever you find it convenient. The example below shows how to storage passwords as one-way hashes by using md5 algorithm.

source code

import os
import sys
from hashlib import md5

from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import DummyAuthorizer, AuthenticationFailed


class DummyMD5Authorizer(DummyAuthorizer):

    def validate_authentication(self, username, password, handler):
        if sys.version_info >= (3, 0):
            password = md5(password.encode('latin1'))
        hash = md5(password).hexdigest()
        try:
            if self.user_table[username]['pwd'] != hash:
                raise KeyError
        except KeyError:
            raise AuthenticationFailed


def main():
    # get a hash digest from a clear-text password
    hash = md5('12345').hexdigest()
    authorizer = DummyMD5Authorizer()
    authorizer.add_user('user', hash, os.getcwd(), perm='elradfmwMT')
    authorizer.add_anonymous(os.getcwd())
    handler = FTPHandler
    handler.authorizer = authorizer
    server = FTPServer(('', 2121), handler)
    server.serve_forever()

if __name__ == "__main__":
    main()

Unix FTP Server ¶

If you’re running a Unix system you may want to configure your ftpd to include support for “real” users existing on the system and navigate the real filesystem. The example below uses UnixAuthorizer and UnixFilesystem classes to do so.

from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import UnixAuthorizer
from pyftpdlib.filesystems import UnixFilesystem

def main():
    authorizer = UnixAuthorizer(rejected_users=["root"], require_valid_shell=True)
    handler = FTPHandler
    handler.authorizer = authorizer
    handler.abstracted_fs = UnixFilesystem
    server = FTPServer(('', 21), handler)
    server.serve_forever()

if __name__ == "__main__":
    main()

Windows FTP Server ¶

The following code shows how to implement a basic authorizer for a Windows NT workstation to authenticate against existing Windows user accounts. This code requires Mark Hammond’s pywin32 extension to be installed.

source code

from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import WindowsAuthorizer

def main():
    authorizer = WindowsAuthorizer()
    # Use Guest user with empty password to handle anonymous sessions.
    # Guest user must be enabled first, empty password set and profile
    # directory specified.
    #authorizer = WindowsAuthorizer(anonymous_user="Guest", anonymous_password="")
    handler = FTPHandler
    handler.authorizer = authorizer
    server = FTPServer(('', 2121), handler)
    server.serve_forever()

if __name__ == "__main__":
    main()

Changing the concurrency model ¶

By nature pyftpdlib is asynchronous. This means it uses a single process/thread to handle multiple client connections and file transfers. This is why it is so fast, lightweight and scalable (see benchmarks). The async model has one big drawback though: the code cannot contain instructions which blocks for a long period of time, otherwise the whole FTP server will hang. As such the user should avoid calls such as time.sleep(3), heavy db queries, etc. Moreover, there are cases where the async model is not appropriate, and that is when you’re dealing with a particularly slow filesystem (say a network filesystem such as samba). If the filesystem is slow (say, a open(file, 'r').read(8192) takes 2 secs to complete) then you are stuck. Starting from version 1.0.0 pyftpdlib supports 2 new classes which changes the default concurrency model by introducing multiple threads or processes. In technical terms this means that every time a client connects a separate thread/process is spawned and internally it will run its own IO loop. In practical terms this means that you can block as long as you want. Changing the concurrency module is easy: you just need to import a substitute for FTPServer. class:

Thread-based example:

from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import ThreadedFTPServer  # <-
from pyftpdlib.authorizers import DummyAuthorizer


def main():
    authorizer = DummyAuthorizer()
    authorizer.add_user('user', '12345', '.')
    handler = FTPHandler
    handler.authorizer = authorizer
    server = ThreadedFTPServer(('', 2121), handler)
    server.serve_forever()

if __name__ == "__main__":
    main()

Multiple process example:

from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import MultiprocessFTPServer  # <-
from pyftpdlib.authorizers import DummyAuthorizer


def main():
    authorizer = DummyAuthorizer()
    authorizer.add_user('user', '12345', '.')
    handler = FTPHandler
    handler.authorizer = authorizer
    server = MultiprocessFTPServer(('', 2121), handler)
    server.serve_forever()

if __name__ == "__main__":
    main()

Throttle bandwidth ¶

An important feature for an ftpd is limiting the speed for downloads and uploads affecting the data channel. ThrottledDTPHandler.banner can be used to set such limits. The basic idea behind ThrottledDTPHandler is to wrap sending and receiving in a data counter and temporary “sleep” the data channel so that you burst to no more than x Kb/sec average. When it realizes that more than x Kb in a second are being transmitted it temporary blocks the transfer for a certain number of seconds.

import os

from pyftpdlib.handlers import FTPHandler, ThrottledDTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import DummyAuthorizer


def main():
    authorizer = DummyAuthorizer()
    authorizer.add_user('user', '12345', os.getcwd(), perm='elradfmwMT')
    authorizer.add_anonymous(os.getcwd())

    dtp_handler = ThrottledDTPHandler
    dtp_handler.read_limit = 30720  # 30 Kb/sec (30 * 1024)
    dtp_handler.write_limit = 30720  # 30 Kb/sec (30 * 1024)

    ftp_handler = FTPHandler
    ftp_handler.authorizer = authorizer
    # have the ftp handler use the alternative dtp handler class
    ftp_handler.dtp_handler = dtp_handler

    server = FTPServer(('', 2121), ftp_handler)
    server.serve_forever()

if __name__ == '__main__':
    main()

FTPS (FTP over TLS/SSL) server ¶

Starting from version 0.6.0 pyftpdlib finally includes full FTPS support implementing both TLS and SSL protocols and AUTH, PBSZ and PROT commands as defined in RFC-4217. This has been implemented by using PyOpenSSL module, which is required in order to run the code below. TLS_FTPHandler class requires at least a certfile to be specified and optionally a keyfile. Apache FAQs provide instructions on how to generate them. If you don’t care about having your personal self-signed certificates you can use the one in the demo directory which include both and is available here.

source code

"""
An RFC-4217 asynchronous FTPS server supporting both SSL and TLS.
Requires PyOpenSSL module (http://pypi.python.org/pypi/pyOpenSSL).
"""

from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import DummyAuthorizer
from pyftpdlib.handlers import TLS_FTPHandler


def main():
    authorizer = DummyAuthorizer()
    authorizer.add_user('user', '12345', '.', perm='elradfmwMT')
    authorizer.add_anonymous('.')
    handler = TLS_FTPHandler
    handler.certfile = 'keycert.pem'
    handler.authorizer = authorizer
    # requires SSL for both control and data channel
    #handler.tls_control_required = True
    #handler.tls_data_required = True
    server = FTPServer(('', 21), handler)
    server.serve_forever()

if __name__ == '__main__':
    main()

Event callbacks ¶

A small example which shows how to use callback methods via FTPHandler subclassing:

from pyftpdlib.handlers import FTPHandler
from pyftpdlib.servers import FTPServer
from pyftpdlib.authorizers import DummyAuthorizer


class MyHandler(FTPHandler):

    def on_connect(self):
        print "%s:%s connected" % (self.remote_ip, self.remote_port)

    def on_disconnect(self):
        # do something when client disconnects
        pass

    def on_login(self, username):
        # do something when user login
        pass

    def on_logout(self, username):
        # do something when user logs out
        pass

    def on_file_sent(self, file):
        # do something when a file has been sent
        pass

    def on_file_received(self, file):
        # do something when a file has been received
        pass

    def on_incomplete_file_sent(self, file):
        # do something when a file is partially sent
        pass

    def on_incomplete_file_received(self, file):
        # remove partially uploaded files
        import os
        os.remove(file)


def main():
    authorizer = DummyAuthorizer()
    authorizer.add_user('user', '12345', homedir='.', perm='elradfmwMT')
    authorizer.add_anonymous(homedir='.')

    handler = MyHandler
    handler.authorizer = authorizer
    server = FTPServer(('', 2121), handler)
    server.serve_forever()

if __name__ == "__main__":
    main()

Command line usage ¶

Starting from version 0.6.0 pyftpdlib can be run as a simple stand-alone server via Python’s -m option, which is particularly useful when you want to quickly share a directory. Some examples. Anonymous FTPd sharing current directory:

$ python -m pyftpdlib
[I 13-04-09 17:55:18] >>> starting FTP server on 0.0.0.0:2121, pid=6412 <<<
[I 13-04-09 17:55:18] poller: <class 'pyftpdlib.ioloop.Epoll'>
[I 13-04-09 17:55:18] masquerade (NAT) address: None
[I 13-04-09 17:55:18] passive ports: None
[I 13-04-09 17:55:18] use sendfile(2): True

Anonymous FTPd with write permission:

$ python -m pyftpdlib -w

Set a different address/port and home directory:

$ python -m pyftpdlib -i localhost -p 8021 -d /home/someone

See python -m pyftpdlib -h for a complete list of options.

API reference ¶

Table of Contents

API reference

pyftpdlib implements the server side of the FTP protocol as defined in RFC-959. This document is intended to serve as a simple API reference of most important classes and functions. After reading this you will probably want to read the tutorial including customization through the use of some example scripts.

Modules and classes hierarchy ¶

pyftpdlib.authorizers
pyftpdlib.authorizers.AuthenticationFailed
pyftpdlib.authorizers.DummyAuthorizer
pyftpdlib.authorizers.UnixAuthorizer
pyftpdlib.authorizers.WindowsAuthorizer
pyftpdlib.handlers
pyftpdlib.handlers.FTPHandler
pyftpdlib.handlers.TLS_FTPHandler
pyftpdlib.handlers.DTPHandler
pyftpdlib.handlers.TLS_DTPHandler
pyftpdlib.handlers.ThrottledDTPHandler
pyftpdlib.filesystems
pyftpdlib.filesystems.FilesystemError
pyftpdlib.filesystems.AbstractedFS
pyftpdlib.filesystems.UnixFilesystem
pyftpdlib.servers
pyftpdlib.servers.FTPServer
pyftpdlib.servers.ThreadedFTPServer
pyftpdlib.servers.MultiprocessFTPServer
pyftpdlib.ioloop
pyftpdlib.ioloop.IOLoop
pyftpdlib.ioloop.Connector
pyftpdlib.ioloop.Acceptor
pyftpdlib.ioloop.AsyncChat

Users ¶

class pyftpdlib.authorizers.DummyAuthorizer¶

Basic “dummy” authorizer class, suitable for subclassing to create your own custom authorizers. An “authorizer” is a class handling authentications and permissions of the FTP server. It is used inside pyftpdlib.handlers.FTPHandler class for verifying user’s password, getting users home directory, checking user permissions when a filesystem read/write event occurs and changing user before accessing the filesystem. DummyAuthorizer is the base authorizer, providing a platform independent interface for managing “virtual” FTP users. Typically the first thing you have to do is create an instance of this class and start adding ftp users:

>>> from pyftpdlib.authorizers import DummyAuthorizer
>>> authorizer = DummyAuthorizer()
>>> authorizer.add_user('user', 'password', '/home/user', perm='elradfmwMT')
>>> authorizer.add_anonymous('/home/nobody')

add_user(username, password, homedir, perm="elr", msg_login="Login successful.", msg_quit="Goodbye.")¶

Add a user to the virtual users table. AuthorizerError exception is raised on error conditions such as insufficient permissions or duplicate usernames. Optional perm argument is a set of letters referencing the user’s permissions. Every letter is used to indicate that the access rights the current FTP user has over the following specific actions are granted. The available permissions are the following listed below:

Read permissions:

"e" = change directory (CWD, CDUP commands)
"l" = list files (LIST, NLST, STAT, MLSD, MLST, SIZE commands)
"r" = retrieve file from the server (RETR command)

Write permissions:

"a" = append data to an existing file (APPE command)
"d" = delete file or directory (DELE, RMD commands)
"f" = rename file or directory (RNFR, RNTO commands)
"m" = create directory (MKD command)
"w" = store a file to the server (STOR, STOU commands)
"M" = change file mode / permission (SITE CHMOD command) New in 0.7.0
"T" = change file modification time (SITE MFMT command) New in 1.5.3

Optional msg_login and msg_quit arguments can be specified to provide customized response strings when user log-in and quit. The perm argument of the add_user() method refers to user’s permissions. Every letter is used to indicate that the access rights the current FTP user has over the following specific actions are granted.

add_anonymous(homedir, **kwargs)¶: Add an anonymous user to the virtual users table. AuthorizerError exception is raised on error conditions such as insufficient permissions, missing home directory, or duplicate anonymous users. The keyword arguments in kwargs are the same expected by add_user() method: perm, msg_login and msg_quit. The optional perm keyword argument is a string defaulting to “elr” referencing “read-only” anonymous user’s permission. Using a “write” value results in a RuntimeWarning.

override_perm(username, directory, perm, recursive=False)¶: Override user permissions for a given directory.

validate_authentication(username, password, handler)¶

Raises pyftpdlib.authorizers.AuthenticationFailed if the supplied username and password doesn’t match the stored credentials.

Changed in 1.0.0: new handler parameter.

Changed in 1.0.0: an exception is now raised for signaling a failed authenticaiton as opposed to returning a bool.

impersonate_user(username, password)¶: Impersonate another user (noop). It is always called before accessing the filesystem. By default it does nothing. The subclass overriding this method is expected to provide a mechanism to change the current user.

terminate_impersonation(username)¶: Terminate impersonation (noop). It is always called after having accessed the filesystem. By default it does nothing. The subclass overriding this method is expected to provide a mechanism to switch back to the original user.

remove_user(username)¶: Remove a user from the virtual user table.

Control connection ¶

class pyftpdlib.handlers.FTPHandler(conn, server)¶

This class implements the FTP server Protocol Interpreter (see RFC-959), handling commands received from the client on the control channel by calling the command’s corresponding method (e.g. for received command “MKD pathname”, ftp_MKD() method is called with pathname as the argument). All relevant session information are stored in instance variables. conn is the underlying socket object instance of the newly established connection, server is the pyftpdlib.servers.FTPServer class instance. Basic usage simply requires creating an instance of FTPHandler class and specify which authorizer instance it will going to use:

>>> from pyftpdlib.handlers import FTPHandler
>>> handler = FTPHandler
>>> handler.authorizer = authorizer

All relevant session information is stored in class attributes reproduced below and can be modified before instantiating this class:

timeout¶

The timeout which is the maximum time a remote client may spend between FTP commands. If the timeout triggers, the remote client will be kicked off (defaults to 300 seconds).

New in version 5.0

banner¶: String sent when client connects (default "pyftpdlib %s ready." %__ver__).

max_login_attempts¶: Maximum number of wrong authentications before disconnecting (default 3).

permit_foreign_addresses¶: Whether enable FXP feature (default False).

permit_privileged_ports¶: Set to True if you want to permit active connections (PORT) over privileged ports (not recommended, default False).

masquerade_address¶: The “masqueraded” IP address to provide along PASV reply when pyftpdlib is running behind a NAT or other types of gateways. When configured pyftpdlib will hide its local address and instead use the public address of your NAT (default None).

masquerade_address_map¶: In case the server has multiple IP addresses which are all behind a NAT router, you may wish to specify individual masquerade_addresses for each of them. The map expects a dictionary containing private IP addresses as keys, and their corresponding public (masquerade) addresses as values (defaults to {}). New in version 0.6.0

passive_ports¶: What ports ftpd will use for its passive data transfers. Value expected is a list of integers (e.g. range(60000, 65535)). When configured pyftpdlib will no longer use kernel-assigned random ports (default None).

use_gmt_times¶: When True causes the server to report all ls and MDTM times in GMT and not local time (default True). New in version 0.6.0

tcp_no_delay¶: Controls the use of the TCP_NODELAY socket option which disables the Nagle algorithm resulting in significantly better performances (default True on all platforms where it is supported). New in version 0.6.0

use_sendfile¶

When True uses sendfile(2) system call to send a file resulting in faster uploads (from server to client). Works on UNIX only and requires pysendfile module to be installed separately.

New in version 0.7.0

auth_failed_timeout¶

The amount of time the server waits before sending a response in case of failed authentication.

New in version 1.5.0

Follows a list of callback methods that can be overridden in a subclass. For blocking operations read the FAQ on how to run time consuming tasks.

on_connect()¶

Called when client connects.

New in version 1.0.0

on_disconnect()¶

Called when connection is closed.

New in version 1.0.0

on_login(username)¶

Called on user login.

New in version 0.6.0

on_login_failed(username, password)¶

Called on failed user login.

New in version 0.7.0

on_logout(username)¶

Called when user logs out due to QUIT or USER issued twice. This is not called if client just disconnects without issuing QUIT first.

New in version 0.6.0

on_file_sent(file)¶: Called every time a file has been successfully sent. file is the absolute name of that file.

on_file_received(file)¶: Called every time a file has been successfully received. file is the absolute name of that file.

on_incomplete_file_sent(file)¶

Called every time a file has not been entirely sent (e.g. transfer aborted by client). file is the absolute name of that file.

New in version 0.6.0

on_incomplete_file_received(file)¶: Called every time a file has not been entirely received (e.g. transfer aborted by client). file is the absolute name of that file. New in version 0.6.0

Data connection ¶

class pyftpdlib.handlers.DTPHandler(sock_obj, cmd_channel)¶

This class handles the server-data-transfer-process (server-DTP, see RFC-959) managing all transfer operations regarding the data channel. sock_obj is the underlying socket object instance of the newly established connection, cmd_channel is the pyftpdlib.handlers.FTPHandler class instance.

Changed in version 1.0.0: added ioloop argument.

timeout¶: The timeout which roughly is the maximum time we permit data transfers to stall for with no progress. If the timeout triggers, the remote client will be kicked off (default 300 seconds).

ac_in_buffer_size¶

ac_out_buffer_size¶: The buffer sizes to use when receiving and sending data (both defaulting to 65536 bytes). For LANs you may want this to be fairly large. Depending on available memory and number of connected clients setting them to a lower value can result in better performances.

class pyftpdlib.handlers.ThrottledDTPHandler(sock_obj, cmd_channel)¶

A pyftpdlib.handlers.DTPHandler subclass which wraps sending and receiving in a data counter and temporarily “sleeps” the channel so that you burst to no more than x Kb/sec average. Use it instead of pyftpdlib.handlers.DTPHandler to set transfer rates limits for both downloads and/or uploads (see the demo script showing the example usage).

read_limit¶: The maximum number of bytes to read (receive) in one second (defaults to 0 == no limit)

write_limit¶: The maximum number of bytes to write (send) in one second (defaults to 0 == no limit).

Server (acceptor)¶

class pyftpdlib.servers.FTPServer(address_or_socket, handler, ioloop=None, backlog=100)¶

Creates a socket listening on address (an (host, port) tuple) or a pre- existing socket object, dispatching the requests to handler (typically pyftpdlib.handlers.FTPHandler class). Also, starts the asynchronous IO loop. backlog is the maximum number of queued connections passed to socket.listen(). If a connection request arrives when the queue is full the client may raise ECONNRESET.

Changed in version 1.0.0: added ioloop argument.

Changed in version 1.2.0: address can also be a pre-existing socket object.

Changed in version 1.2.0: Added backlog argument.

Changed in version 1.5.4: Support for the context manager protocol was added. Exiting the context manager is equivalent to calling :meth:`close_all`.

>>> from pyftpdlib.servers import FTPServer
>>> address = ('127.0.0.1', 21)
>>> server = FTPServer(address, handler)
>>> server.serve_forever()

It can also be used as a context manager. Exiting the context manager is equivalent to calling close_all().

>>> with FTPServer(address, handler) as server:
...     server.serve_forever()

max_cons¶: Number of maximum simultaneous connections accepted (default 512).

max_cons_per_ip¶: Number of maximum connections accepted for the same IP address (default 0 == no limit).

serve_forever(timeout=None, blocking=True, handle_exit=True)¶

Starts the asynchronous IO loop.

Changed in version 1.0.0: no longer a classmethod; ‘use_poll’ and ‘count’ *parameters were removed. ‘blocking’ and ‘handle_exit’ parameters were *added

close()¶: Stop accepting connections without disconnecting currently connected clients. server_forever() loop will automatically stop when there are no more connected clients.

close_all()¶

Disconnect all clients, tell server_forever() loop to stop and wait until it does.

Changed in version 1.0.0: ‘map’ and ‘ignore_all’ parameters were removed.

Filesystem ¶

class pyftpdlib.filesystems.FilesystemError¶: Exception class which can be raised from within pyftpdlib.filesystems.AbstractedFS in order to send custom error messages to client. New in version 1.0.0

class pyftpdlib.filesystems.AbstractedFS(root, cmd_channel)¶

A class used to interact with the file system, providing a cross-platform interface compatible with both Windows and UNIX style filesystems where all paths use "/" separator. AbstractedFS distinguishes between “real” filesystem paths and “virtual” ftp paths emulating a UNIX chroot jail where the user can not escape its home directory (example: real “/home/user” path will be seen as “/” by the client). It also provides some utility methods and wraps around all os.* calls involving operations against the filesystem like creating files or removing directories. The contructor accepts two arguments: root which is the user “real” home directory (e.g. ‘/home/user’) and cmd_channel which is the pyftpdlib.handlers.FTPHandler class instance.

Changed in version 0.6.0: root and cmd_channel arguments were added.

root¶: User’s home directory (“real”). Changed in version 0.7.0: support setattr()

cwd¶

User’s current working directory (“virtual”).

Changed in version 0.7.0: support setattr()

ftpnorm(ftppath)¶: Normalize a “virtual” ftp pathname depending on the current working directory (e.g. having "/foo" as current working directory "bar" becomes "/foo/bar").

ftp2fs(ftppath)¶: Translate a “virtual” ftp pathname into equivalent absolute “real” filesystem pathname (e.g. having "/home/user" as root directory "foo" becomes "/home/user/foo").

fs2ftp(fspath)¶: Translate a “real” filesystem pathname into equivalent absolute “virtual” ftp pathname depending on the user’s root directory (e.g. having "/home/user" as root directory "/home/user/foo" becomes "/foo".

validpath(path)¶: Check whether the path belongs to user’s home directory. Expected argument is a “real” filesystem path. If path is a symbolic link it is resolved to check its real destination. Pathnames escaping from user’s root directory are considered not valid (return False).

open(filename, mode)¶: Wrapper around open() builtin.

mkdir(path)¶

chdir(path)¶

rmdir(path)¶

remove(path)¶

rename(src, dst)¶

chmod(path, mode)¶

stat(path)¶

lstat(path)¶

readlink(path)¶: Wrappers around corresponding os module functions.

isfile(path)¶

islink(path)¶

isdir(path)¶

getsize(path)¶

getmtime(path)¶

realpath(path)¶

lexists(path)¶: Wrappers around corresponding os.path module functions.

mkstemp(suffix='', prefix='', dir=None, mode='wb')¶: Wrapper around tempfile.mkstemp.

listdir(path)¶: Wrapper around os.listdir. It is expected to return a list of unicode strings or a generator yielding unicode strings.

Changed in version 1.6.0: can also return a generator.

Extended classes ¶

We are about to introduces are extensions (subclasses) of the ones explained so far. They usually require third-party modules to be installed separately or are specific for a given Python version or operating system.

Extended handlers ¶

class pyftpdlib.handlers.TLS_FTPHandler(conn, server)¶

A pyftpdlib.handlers.FTPHandler subclass implementing FTPS (FTP over SSL/TLS) as described in RFC-4217 implementing AUTH, PBSZ and PROT commands. PyOpenSSL module is required to be installed. Example below shows how to setup an FTPS server. Configurable attributes:

certfile¶: The path to a file which contains a certificate to be used to identify the local side of the connection. This must always be specified, unless context is provided instead.

keyfile¶: The path of the file containing the private RSA key; can be omittetted if certfile already contains the private key (defaults: None).

ssl_protocol¶: The desired SSL protocol version to use. This defaults to SSL.SSLv23_METHOD which will negotiate the highest protocol that both the server and your installation of OpenSSL support.

ssl_options¶: specific OpenSSL options. These default to: SSL.OP_NO_SSLv2 | SSL.OP_NO_SSLv3 | SSL.OP_NO_COMPRESSION disabling SSLv2 and SSLv3 versions and SSL compression algorithm which are considered insecure. Can be set to None in order to improve compatibilty with older (insecure) FTP clients.

New in version 1.6.0.

ssl_context¶: A SSL.Context instance which was previously configured. If specified ssl_protocol and ssl_options parameters will be ignored.

tls_control_required¶: When True requires SSL/TLS to be established on the control channel, before logging in. This means the user will have to issue AUTH before USER/PASS (default False).

tls_data_required¶: When True requires SSL/TLS to be established on the data channel. This means the user will have to issue PROT before PASV or PORT (default False).

Extended authorizers ¶

class pyftpdlib.authorizers.UnixAuthorizer(global_perm="elradfmwMT", allowed_users=None, rejected_users=None, require_valid_shell=True, anonymous_user=None, msg_login="Login successful.", msg_quit="Goodbye.")¶

Authorizer which interacts with the UNIX password database. Users are no longer supposed to be explicitly added as when using pyftpdlib.authorizers.DummyAuthorizer. All FTP users are the same defined on the UNIX system so if you access on your system by using "john" as username and "12345" as password those same credentials can be used for accessing the FTP server as well. The user home directories will be automatically determined when user logins. Every time a filesystem operation occurs (e.g. a file is created or deleted) the id of the process is temporarily changed to the effective user id and whether the operation will succeed depends on user and file permissions. This is why full read and write permissions are granted by default in the class constructors.

global_perm is a series of letters referencing the users permissions; defaults to “elradfmwMT” which means full read and write access for everybody (except anonymous). allowed_users and rejected_users options expect a list of users which are accepted or rejected for authenticating against the FTP server; defaults both to [] (no restrictions). require_valid_shell denies access for those users which do not have a valid shell binary listed in /etc/shells. If /etc/shells cannot be found this is a no-op. anonymous user is not subject to this option, and is free to not have a valid shell defined. Defaults to True (a valid shell is required for login). anonymous_user can be specified if you intend to provide anonymous access. The value expected is a string representing the system user to use for managing anonymous sessions; defaults to None (anonymous access disabled). Note that in order to use this class super user privileges are required.

New in version 0.6.0

override_user(username=None, password=None, homedir=None, perm=None, anonymous_user=None, msg_login=None, msg_quit=None)¶

Overrides one or more options specified in the class constructor for a specific user. Example:

>>> from pyftpdlib.authorizers import UnixAuthorizer
>>> auth = UnixAuthorizer(rejected_users=["root"])
>>> auth = UnixAuthorizer(allowed_users=["matt", "jay"])
>>> auth = UnixAuthorizer(require_valid_shell=False)
>>> auth.override_user("matt", password="foo", perm="elr")

class pyftpdlib.authorizers.WindowsAuthorizer(global_perm="elradfmwMT", allowed_users=None, rejected_users=None, anonymous_user=None, anonymous_password="", msg_login="Login successful.", msg_quit="Goodbye.")¶

Same as pyftpdlib.authorizers.UnixAuthorizer except for anonymous_password argument which must be specified when defining the anonymous_user. Also requires_valid_shell option is not available. In order to use this class pywin32 extension must be installed.

New in version 0.6.0

Extended filesystems ¶

class pyftpdlib.filesystems.UnixFilesystem(root, cmd_channel)¶

Represents the real UNIX filesystem. Differently from pyftpdlib.filesystems.AbstractedFS the client will login into /home/<username> and will be able to escape its home directory and navigate the real filesystem. Use it in conjuction with pyftpdlib.authorizers.UnixAuthorizer to implement a “real” UNIX FTP server (see demo/unix_ftpd.py).

New in version 0.6.0

Extended servers ¶

class pyftpdlib.servers.ThreadedFTPServer(address_or_socket, handler, ioloop=None, backlog=5)¶

A modified version of base pyftpdlib.servers.FTPServer class which spawns a thread every time a new connection is established. Differently from base FTPServer class, the handler will be free to block without hanging the whole IO loop.

New in version 1.0.0

Changed in 1.2.0: added ioloop parameter; address can also be a pre-existing *socket.

class pyftpdlib.servers.MultiprocessFTPServer(address_or_socket, handler, ioloop=None, backlog=5)¶

A modified version of base pyftpdlib.servers.FTPServer class which spawns a process every time a new connection is established. Differently from base FTPServer class, the handler will be free to block without hanging the whole IO loop.

New in version 1.0.0

Changed in 1.2.0: added ioloop parameter; address can also be a pre-existing socket.

Availability: POSIX + Python >= 2.6

FAQs ¶

Table of Contents

FAQs

Introduction ¶

What is pyftpdlib?¶

pyftpdlib is a high-level library to easily write asynchronous portable FTP servers with Python.

What is Python?¶

Python is an interpreted, interactive, object-oriented, easy-to-learn programming language. It is often compared to Tcl, Perl, Scheme or Java.

I’m not a python programmer. Can I use it anyway?¶

Yes. pyftpdlib is a fully working FTP server implementation that can be run “as is”. For example you could run an anonymous ftp server from cmd-line by running:

$ sudo python -m pyftpdlib
[I 13-02-20 14:16:36] >>> starting FTP server on 0.0.0.0:8021 <<<
[I 13-02-20 14:16:36] poller: <class 'pyftpdlib.ioloop.Epoll'>
[I 13-02-20 14:16:36] masquerade (NAT) address: None
[I 13-02-20 14:16:36] passive ports: None
[I 13-02-20 14:16:36] use sendfile(2): True

This is useful in case you want a quick and dirty way to share a directory without, say, installing and configuring samba. Starting from version 0.6.0 options can be passed to the command line (see python -m pyftpdlib --help to see all available options). Examples:

Anonymous FTP server with write access:

$ sudo python -m pyftpdlib -w
~pyftpdlib-1.3.1-py2.7.egg/pyftpdlib/authorizers.py:265: RuntimeWarning: write permissions assigned to anonymous user.
[I 13-02-20 14:16:36] >>> starting FTP server on 0.0.0.0:8021 <<<
[I 13-02-20 14:16:36] poller: <class 'pyftpdlib.ioloop.Epoll'>
[I 13-02-20 14:16:36] masquerade (NAT) address: None
[I 13-02-20 14:16:36] passive ports: None
[I 13-02-20 14:16:36] use sendfile(2): True

Listen on a different ip/port:

$ python -m pyftpdlib -i 127.0.0.1 -p 8021
[I 13-02-20 14:16:36] >>> starting FTP server on 0.0.0.0:8021 <<<
[I 13-02-20 14:16:36] poller: <class 'pyftpdlib.ioloop.Epoll'>
[I 13-02-20 14:16:36] masquerade (NAT) address: None
[I 13-02-20 14:16:36] passive ports: None
[I 13-02-20 14:16:36] use sendfile(2): True

Customizing ftpd for basic tasks like adding users or deciding where log file should be placed is mostly simply editing variables. This is basically like learning how to edit a common unix ftpd.conf file and doesn’t really require Python knowledge. Customizing ftpd more deeply requires a python script which imports pyftpdlib to be written separately. An example about how this could be done are the scripts contained in the demo directory.

Getting help ¶

There’s a mailing list available at: http://groups.google.com/group/pyftpdlib/topics

Installing and compatibility ¶

How do I install pyftpdlib?¶

If you are not new to Python you probably don’t need that, otherwise follow the install instructions.

Which Python versions are compatible?¶

From 2.6 to 3.4. Python 2.4 and 2.5 support has been removed starting from version 0.6.0. The latest version supporting Python 2.3 is pyftpdlib 1.4.0. Python 2.3 support has been removed starting from version 0.6.0. The latest version supporting Python 2.3 is pyftpdlib 0.5.2.

On which platforms can pyftpdlib be used?¶

pyftpdlib should work on any platform where select(), poll(), epoll() or kqueue() system calls are available and on any Python implementation which refers to cPython 2.6 or superior. The development team has mainly tested it under various Linux, Windows, OSX and FreeBSD systems. For FreeBSD is also available a pre-compiled package maintained by Li-Wen Hsu (lwhsu@freebsd.org). Other Python implementation like PythonCE are known to work with pyftpdlib and every new version is usually tested against it. pyftpdlib currently does not work on Jython since the latest Jython release refers to CPython 2.2.x serie. The best way to know whether pyftpdlib works on your platform is installing it and running its test suite.

Usage ¶

How can I run long-running tasks without blocking the server?¶

pyftpdlib is an asynchronous FTP server. That means that if you need to run a time consuming task you have to use a separate Python process or thread for the actual processing work otherwise the entire asynchronous loop will be blocked.

Let’s suppose you want to implement a long-running task every time the server receives a file. The code snippet below shows the correct way to do it by using a thread.

With self.del_channel() we temporarily “sleep” the connection handler which will be removed from the async IO poller loop and won’t be able to send or receive any more data. It won’t be closed (disconnected) as long as we don’t invoke self.add_channel(). This is fundamental when working with threads to avoid race conditions, dead locks etc.

class MyHandler(FTPHandler):

    def on_file_received(self, file):
        def blocking_task():
            time.sleep(5)
            self.add_channel()

        self.del_channel()
        threading.Thread(target=blocking_task).start()

Another possibility is to change the default concurrency model.

Why do I get socket.error “Permission denied” error on ftpd starting?¶

Probably because you’re on a Unix system and you’re trying to start ftpd as an unprivileged user. FTP servers bind on port 21 by default and only super-user account (e.g. root) can bind sockets on such ports. If you want to bind ftpd as non-privileged user you should set a port higher than 1024.

How can I prevent the server version from being displayed?¶

Just modify FTPHandler.banner.

Can control upload/download ratios?¶

Yes. Starting from version 0.5.2 pyftpdlib provides a new class called ThrottledDTPHandler. You can set speed limits by modifying read_limit and write_limit class attributes as it is shown in throttled_ftpd.py demo script.

Are there ways to limit connections?¶

FTPServer. class comes with two overridable attributes defaulting to zero (no limit): max_cons, which sets a limit for maximum simultaneous connection to handle by ftpd and max_cons_per_ip which set a limit for connections from the same IP address. Overriding these variables is always recommended to avoid DoS attacks.

I’m behind a NAT / gateway ¶

When behind a NAT a ftp server needs to replace the IP local address displayed in PASV replies and instead use the public address of the NAT to allow client to connect. By overriding masquerade_address attribute of FTPHandler class you will force pyftpdlib to do such replacement. However, one problem still exists. The passive FTP connections will use ports from 1024 and up, which means that you must forward all ports 1024-65535 from the NAT to the FTP server! And you have to allow many (possibly) dangerous ports in your firewalling rules! To resolve this, simply override passive_ports attribute of FTPHandler class to control what ports pyftpdlib will use for its passive data transfers. Value expected by passive_ports attribute is a list of integers (e.g. range(60000, 65535)) indicating which ports will be used for initializing the passive data channel. In case you run a FTP server with multiple private IP addresses behind a NAT firewall with multiple public IP addresses you can use passive_ports option which allows you to define multiple 1 to 1 mappings (New in 0.6.0).

What is FXP?¶

FXP is part of the name of a popular Windows FTP client: http://www.flashfxp.com. This client has made the name “FXP” commonly used as a synonym for site-to-site FTP transfers, for transferring a file between two remote FTP servers without the transfer going through the client’s host. Sometimes “FXP” is referred to as a protocol; in fact, it is not. The site-to-site transfer capability was deliberately designed into RFC-959. More info can be found here: http://www.proftpd.org/docs/howto/FXP.html.

Does pyftpdlib support FXP?¶

Yes. It is disabled by default for security reasons (see RFC-2257 and FTP bounce attack description) but in case you want to enable it just set to True the permit_foreign_addresses attribute of FTPHandler class.

Why timestamps shown by MDTM and ls commands (LIST, MLSD, MLST) are wrong?¶

If by “wrong” you mean “different from the timestamp of that file on my client machine”, then that is the expected behavior. Starting from version 0.6.0 pyftpdlib uses GMT times as recommended in RFC-3659. In case you want such commands to report local times instead just set the use_gmt_times attribute to False. For further information you might want to take a look at this

Implementation ¶

sendfile()¶

Starting from version 0.7.0 if pysendfile module is installed sendfile(2) system call be used when uploading files (from server to client) via RETR command. Using sendfile(2) usually results in transfer rates from 2x to 3x faster and less CPU usage. Note: use of sendfile() might introduce some unexpected issues with “non regular filesystems” such as NFS, SMBFS/Samba, CIFS and network mounts in general, see: http://www.proftpd.org/docs/howto/Sendfile.html. If you bump into one this problems the fix consists in disabling sendfile() usage via FTPHandler.use_sendfile option:

from pyftpdlib.handlers import FTPHandler
handler = FTPHandler
handler.use_senfile = False
...

Globbing / STAT command implementation ¶

Globbing is a common Unix shell mechanism for expanding wildcard patterns to match multiple filenames. When an argument is provided to the STAT command, ftpd should return directory listing over the command channel. RFC-959 does not explicitly mention globbing; this means that FTP servers are not required to support globbing in order to be compliant. However, many FTP servers do support globbing as a measure of convenience for FTP clients and users. In order to search for and match the given globbing expression, the code has to search (possibly) many directories, examine each contained filename, and build a list of matching files in memory. Since this operation can be quite intensive, both CPU- and memory-wise, pyftpdlib does not support globbing.

ASCII transfers / SIZE command implementation ¶

Properly handling the SIZE command when TYPE ASCII is used would require to scan the entire file to perform the ASCII translation logic (file.read().replace(os.linesep, ‘rn’)) and then calculating the len of such data which may be different than the actual size of the file on the server. Considering that calculating such result could be very resource-intensive it could be easy for a malicious client to try a DoS attack, thus pyftpdlib rejects SIZE when the current TYPE is ASCII. However, clients in general should not be resuming downloads in ASCII mode. Resuming downloads in binary mode is the recommended way as specified in RFC-3659.

IPv6 support ¶

Starting from version 0.4.0 pyftpdlib supports IPv6 (RFC-2428). If you use IPv6 and want your FTP daemon to do so just pass a valid IPv6 address to the FTPServer class constructor. Example:

>>> from pyftpdlib.servers import FTPServer
>>> address = ("::1", 21)  # listen on localhost, port 21
>>> ftpd = FTPServer(address, FTPHandler)
>>> ftpd.serve_forever()
Serving FTP on ::1:21

If your OS (for example: all recent UNIX systems) have an hybrid dual-stack IPv6/IPv4 implementation the code above will listen on both IPv4 and IPv6 by using a single IPv6 socket (New in 0.6.0).

How do I install IPv6 support on my system?¶

If you want to install IPv6 support on Linux run “modprobe ipv6”, then “ifconfig”. This should display the loopback adapter, with the address “::1”. You should then be able to listen the server on that address, and connect to it. On Windows (XP SP2 and higher) run “netsh int ipv6 install”. Again, you should be able to use IPv6 loopback afterwards.

Can pyftpdlib be integrated with “real” users existing on the system?¶

Yes. Starting from version 0.6.0 pyftpdlib provides the new UnixAuthorizer and WindowsAuthorizer classes. By using them pyftpdlib can look into the system account database to authenticate users. They also assume the id of real users every time the FTP server is going to access the filesystem (e.g. for creating or renaming a file) the authorizer will temporarily impersonate the currently logged on user, execute the filesystem call and then switch back to the user who originally started the server. Example UNIX and Windows FTP servers contained in the demo directory shows how to use UnixAuthorizer and WindowsAuthorizer classes.

Does pyftpdlib support FTP over TLS/SSL (FTPS)?¶

Yes, starting from version 0.6.0, see: Does pyftpdlib support FTP over TLS/SSL (FTPS)?

What about SITE commands?¶

The only supported SITE command is SITE CHMOD (change file mode). The user willing to add support for other specific SITE commands has to define a new ftp_SITE_CMD method in the FTPHandler subclass and add a new entry in proto_cmds dictionary. Example:

from pyftpdlib.handlers import FTPHandler

proto_cmds = FTPHandler.proto_cmds.copy()
proto_cmds.update(
    {'SITE RMTREE': dict(perm='R', auth=True, arg=True,
      help='Syntax: SITE <SP> RMTREE <SP> path (remove directory tree).')}
)

class CustomizedFTPHandler(FTPHandler):
    proto_cmds = proto_cmds

def ftp_SITE_RMTREE(self, line):
    """Recursively remove a directory tree."""
    # implementation here
    # ...

Benchmarks¶

pyftpdlib 0.7.0 vs. pyftpdlib 1.0.0¶

benchmark type	0.7.0	1.0.0	speedup
STOR (client -> server)	528.63 MB/sec	585.90 MB/sec	+0.1x
RETR (server -> client)	1702.07 MB/sec	1652.72 MB/sec	-0.02x
300 concurrent clients (connect, login)	1.70 secs	0.19 secs	+8x
STOR (1 file with 300 idle clients)	60.77 MB/sec	585.59 MB/sec	+8.6x
RETR (1 file with 300 idle clients)	63.46 MB/sec	1497.58 MB/sec	+22.5x
300 concurrent clients (RETR 10M file)	4.68 secs	3.41 secs	+0.3x
300 concurrent clients (STOR 10M file)	10.13 secs	8.78 secs	+0.1x
300 concurrent clients (QUIT)	0.02 secs	0.02 secs	0x

pyftpdlib vs. proftpd 1.3.4¶

benchmark type	pyftpdlib	proftpd	speedup
STOR (client -> server)	585.90 MB/sec	600.49 MB/sec	-0.02x
RETR (server -> client)	1652.72 MB/sec	1524.05 MB/sec	+0.08
300 concurrent clients (connect, login)	0.19 secs	9.98 secs	+51x
STOR (1 file with 300 idle clients)	585.59 MB/sec	518.55 MB/sec	+0.1x
RETR (1 file with 300 idle clients)	1497.58 MB/sec	1478.19 MB/sec	0x
300 concurrent clients (RETR 10M file)	3.41 secs	3.60 secs	+0.05x
300 concurrent clients (STOR 10M file)	8.60 secs	11.56 secs	+0.3x
300 concurrent clients (QUIT)	0.03 secs	0.39 secs	+12x

pyftpdlib vs. vsftpd 2.3.5¶

benchmark type	pyftpdlib	vsftpd	speedup
STOR (client -> server)	585.90 MB/sec	611.73 MB/sec	-0.04x
RETR (server -> client)	1652.72 MB/sec	1512.92 MB/sec	+0.09
300 concurrent clients (connect, login)	0.19 secs	20.39 secs	+106x
STOR (1 file with 300 idle clients)	585.59 MB/sec	610.23 MB/sec	-0.04x
RETR (1 file with 300 idle clients)	1497.58 MB/sec	1493.01 MB/sec	0x
300 concurrent clients (RETR 10M file)	3.41 secs	3.67 secs	+0.07x
300 concurrent clients (STOR 10M file)	8.60 secs	9.82 secs	+0.07x
300 concurrent clients (QUIT)	0.03 secs	0.01 secs	+0.14x

pyftpdlib vs. Twisted 12.3¶

By using sendfile() (Twisted does not support sendfile()):

benchmark type	pyftpdlib \| twisted \| speedup
STOR (client -> server)	585.90 MB/sec \| 496.44 MB/sec \| +0.01x
RETR (server -> client)	1652.72 MB/sec	283.24 MB/sec	+4.8x
300 concurrent clients (connect, login)	0.19 secs	0.19 secs	+0x
STOR (1 file with 300 idle clients)	585.59 MB/sec	506.55 MB/sec	+0.16x
RETR (1 file with 300 idle clients)	1497.58 MB/sec	280.63 MB/sec	+4.3x
300 concurrent clients (RETR 10M file)	3.41 secs	11.40 secs	+2.3x
300 concurrent clients (STOR 10M file)	8.60 secs	9.22 secs	+0.07x
300 concurrent clients (QUIT)	0.03 secs	0.09 secs	+2x

By using plain send():

benchmark type	tpdlib*	twisted	speedup
RETR (server -> client)	894.29 MB/sec	283.24 MB/sec	+2.1x
RETR (1 file with 300 idle clients)	900.98 MB/sec	280.63 MB/sec	+2.1x

Memory usage¶

Values on UNIX are calculated as (rss - shared).

benchmark type	pyftpdlib	proftpd 1.3.4	vsftpd 2.3.5	twisted 12.3
Starting with	6.7M	1.4M	352.0K	13.4M
STOR (1 client)	6.7M	8.5M	816.0K	13.5M
RETR (1 client)	6.8M	8.5M	816.0K	13.5M
300 concurrent clients (connect, login)	8.8M	568.6M	140.9M	13.5M
STOR (1 file with 300 idle clients)	8.8M	570.6M	141.4M	13.5M
RETR (1 file with 300 idle clients)	8.8M	570.6M	141.4M	13.5M
300 concurrent clients (RETR 10.0M file)	10.8M	568.6M	140.9M	24.5M
300 concurrent clients (STOR 10.0M file)	12.6	568.7M	140.9M	24.7M

Interpreting the results¶

pyftpdlib and proftpd / vsftpd look pretty much equally fast. The huge difference is noticeable in scalability though, because of the concurrency model adopted. Both proftpd and vsftpd spawn a new process for every connected client, where pyftpdlib doesn’t (see the C10k problem). The outcome is well noticeable on connect/login benchmarks and memory benchmarks.

The huge differences between 0.7.0 and 1.0.0 versions of pyftpdlib are due to fix of issue 203. On Linux we now use epoll() which scales considerably better than select(). The fact that we’re downloading a file with 300 idle clients doesn’t make any difference for epoll(). We might as well had 5000 idle clients and the result would have been the same. On Windows, where we still use select(), 1.0.0 still wins hands down as the asyncore loop was reimplemented from scratch in order to support fd un/registration and modification (see issue 203). All the benchmarks were conducted on a Linux Ubuntu 12.04 Intel core duo - 3.1 Ghz box.

Setup¶

The following setup was used before running every benchmark:

proftpd¶

# /etc/proftpd/proftpd.conf

MaxInstances        2000

…followed by:

$ sudo service proftpd restart

vsftpd¶

# /etc/vsftpd.conf

local_enable=YES
write_enable=YES
max_clients=2000
max_per_ip=2000

…followed by:

$ sudo service vsftpd restart

twisted FTP server¶

from twisted.protocols.ftp import FTPFactory, FTPRealm
from twisted.cred.portal import Portal
from twisted.cred.checkers import AllowAnonymousAccess, FilePasswordDB
from twisted.internet import reactor
import resource

soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
resource.setrlimit(resource.RLIMIT_NOFILE, (hard, hard))
open('pass.dat', 'w').write('user:some-passwd')
p = Portal(FTPRealm('./'),
[AllowAnonymousAccess(), FilePasswordDB("pass.dat")])
f = FTPFactory(p)
reactor.listenTCP(21, f)
reactor.run()

…followed by:

$ sudo python twist_ftpd.py

pyftpdlib¶

The following patch was applied first:

Index: pyftpdlib/servers.py
===================================================================
--- pyftpdlib/servers.py    (revisione 1154)
+++ pyftpdlib/servers.py    (copia locale)
@@ -494,3 +494,10 @@

def _map_len(self):
return len(multiprocessing.active_children())
+
+import resource
+soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
+resource.setrlimit(resource.RLIMIT_NOFILE, (hard, hard))
+FTPServer.max_cons = 0

…followed by:

$ sudo python demo/unix_daemon.py

The benchmark script was run as:

python scripts/ftpbench -u USERNAME -p PASSWORD -b all -n 300

…and for the memory test:

python scripts/ftpbench -u USERNAME -p PASSWORD -b all -n 300 -k FTP_SERVER_PID

pyftpdlib RFC compliance ¶

Table of Contents

pyftpdlib RFC compliance

Introduction ¶

This page lists current standard Internet RFCs that define the FTP protocol.

pyftpdlib conforms to the FTP protocol standard as defined in RFC-959 and RFC-1123 implementing all the fundamental commands and features described in them. It also implements some more recent features such as OPTS and FEAT commands (RFC-2398), EPRT and EPSV commands covering the IPv6 support (RFC-2428) and MDTM, MLSD, MLST and SIZE commands defined in RFC-3659.

Future plans for pyftpdlib include the gradual implementation of other standards track RFCs.

Some of the features like ACCT or SMNT commands will never be implemented deliberately. Other features described in more recent RFCs like the TLS/SSL support for securing FTP (RFC-4217) are now implemented as a demo script, waiting to reach the proper level of stability to be then included in the standard code base.

RFC-959 - File Transfer Protocol ¶

The base specification of the current File Transfer Protocol.

Issued: October 1985
Status: STANDARD
Obsoletes: RFC-765
Updated by: RFC-1123, RFC-2228, RFC-2640, RFC-2773
Link

Command	Implemented	Milestone	Description	Notes
ABOR	YES	0.1.0	Abort data transfer.
ACCT	NO	—	Specify account information.	It will never be implemented (useless).
ALLO	YES	0.1.0	Ask for server to allocate enough storage space.	Treated as a NOOP (no operation).
APPE	YES	0.1.0	Append data to an existing file.
CDUP	YES	0.1.0	Go to parent directory.
CWD	YES	0.1.0	Change current working directory.
DELE	YES	0.1.0	Delete file.
HELP	YES	0.1.0	Show help.	Accept also arguments.
LIST	YES	0.1.0	List files.	Accept also bad arguments like “-ls”, “-la”, …
MKD	YES	0.1.0	Create directory.
MODE	YES	0.1.0	Set data transfer mode.	“STREAM” mode is supported, “Block” and “Compressed” aren’t.
NLST	YES	0.1.0	List files in a compact form.	Globbing of wildcards is not supported (for example, `NLST *.txt` will not work)
NOOP	YES	0.1.0	NOOP (no operation), just do nothing.
PASS	YES	0.1.0	Set user password.
PASV	YES	0.1.0	Set server in passive connection mode.
PORT	YES	0.1.0	Set server in active connection mode.
PWD	YES	0.1.0	Get current working directory.
QUIT	YES	0.1.0	Quit session.	If file transfer is in progress, the connection will remain open until it is finished.
REIN	YES	0.1.0	Reinitialize user’s current session.
REST	YES	0.1.0	Restart file position.
RETR	YES	0.1.0	Retrieve a file (client’s download).
RMD	YES	0.1.0	Remove directory.
RNFR	YES	0.1.0	File renaming (source)
RNTO	YES	0.1.0	File renaming (destination)
SITE	YES	0.5.1	Site specific server services.	No SITE commands aside from “SITE HELP” are implemented by default. The user willing to add support for a specific SITE command has to define a new `ftp_SITE_CMD` method in the `FTPHandler` subclass.
SMNT	NO	—	Mount file-system structure.	Will never be implemented (too much system-dependent and almost never used).
STAT	YES	0.1.0	Server’s status information / File LIST
STOR	YES	0.1.0	Store a file (client’s upload).
STOU	YES	0.1.0	Store a file with a unique name.
STRU	YES	0.1.0	Set file structure.	Supports only File type structure by doing a NOOP (no operation). Other structure types (Record and Page) are not implemented.
SYST	YES	0.1.0	Get system type.	Always return “UNIX Type: L8” because of the LIST output provided.
TYPE	YES	0.1.0	Set current type (Binary/ASCII).	Accept only Binary and ASII TYPEs. Other TYPEs such as EBCDIC are obsoleted, system-dependent and thus not implemented.
USER	YES	0.1.0	Set user.	A new USER command could be entered at any point in order to change the access control flushing any user, password, and account information already supplied and beginning the login sequence again.

RFC-1123 - Requirements for Internet Hosts ¶

Extends and clarifies some aspects of RFC-959. Introduces new response codes 554 and 555.

Issued: October 1989
Status: STANDARD
Link

Feature	Implemented	Milestone	Description	Notes
TYPE L 8 as synonym of TYPE I	YES	0.2.0	TYPE L 8 command should be treated as synonym of TYPE I (“IMAGE” or binary type).
PASV is per-transfer	YES	0.1.0	PASV must be used for a unique transfer.	If PASV is issued twice data-channel is restarted.
Implied type for LIST and NLST	YES	0.1.0	The data returned by a LIST or NLST command SHOULD use an implied TYPE AN.
STOU format output	YES	0.2.0	Defined the exact format output which STOU response must respect (“125/150 FILE filename”).
Avoid 250 response type on STOU	YES	0.2.0	The 250 positive response indicated in RFC-959 has been declared incorrect in RFC-1123 which requires 125/150 instead.
Handle “Experimental” directory cmds	YES	0.1.0	The server should support XCUP, XCWD, XMKD, XPWD and XRMD obsoleted commands and treat them as synonyms for CDUP, CWD, MKD, LIST and RMD commands.
Idle timeout	YES	0.5.0	A Server-FTP process SHOULD have a configurable idle timeout of 5 minutes, which will terminate the process and close the control connection if the server is inactive (i.e., no command or data transfer in progress) for a long period of time.
Concurrency of data and control	YES	0.1.0	Server-FTP should be able to process STAT or ABOR while a data transfer is in progress	Feature granted natively for ALL commands since we’re in an asynchronous environment.
554 response on wrong REST	YES	0.2.0	Return a 554 reply may for a command that follows a REST command. The reply indicates that the existing file at the Server-FTP cannot be repositioned as specified in the REST.

RFC-2228 - FTP Security Extensions ¶

Specifies several security extensions to the base FTP protocol defined in RFC-959. New commands: AUTH, ADAT, PROT, PBSZ, CCC, MIC, CONF, and ENC. New response codes: 232, 234, 235, 334, 335, 336, 431, 533, 534, 535, 536, 537, 631, 632, and 633.

Issued: October 1997
Status: PROPOSED STANDARD
Updates: RFC-959
Link

Command	Implemented	Milestone	Description	Notes
AUTH	NO	—	Authentication/Security Mechanism.	Implemented as demo script by following the RFC=4217 guide line.
CCC	NO	—	Clear Command Channel.
CONF	NO	—	Confidentiality Protected Command.	Somewhat obsoleted by RFC-4217.
EENC	NO	—	Privacy Protected Command.	Somewhat obsoleted by RFC-4217.
MIC	NO	—	Integrity Protected Command.	Somewhat obsoleted by RFC-4217.
PBSZ	NO	—	Protection Buffer Size.	Implemented as demo script by following the RFC-4217 guide line as a no-op command.
PROT	NO	—	Data Channel Protection Level.	Implemented as demo script by following the RFC-4217 guide line supporting only “P” and “C” protection levels.

RFC-2389 - Feature negotiation mechanism for the File Transfer Protocol ¶

Introduces the new FEAT and OPTS commands.

Issued: August 1998
Status: PROPOSED STANDARD
Link

Command	Implemented	Milestone	Description	Notes
FEAT	YES	0.3.0	List new supported commands subsequent RFC-959
OPTS	YES	0.3.0	Set options for certain commands.	MLST is the only command which could be used with OPTS.

RFC-2428 - FTP Extensions for IPv6 and NATs ¶

Introduces the new commands EPRT and EPSV extending FTP to enable its use over various network protocols, and the new response codes 522 and 229.

Issued: September 1998
Status: PROPOSED STANDARD
Link

Command	Implemented	Milestone	Description	Notes
EPRT	YES	0.4.0	Set active data connection over IPv4 or IPv6
EPSV	YES	0.4.0	Set passive data connection over IPv4 or IPv6

RFC-2577 - FTP Security Considerations ¶

Provides several configuration and implementation suggestions to mitigate some security concerns, including limiting failed password attempts and third-party “proxy FTP” transfers, which can be used in “bounce attacks”.

Issued: May 1999
Status: INFORMATIONAL
Link

Feature	Implemented	Milestone	Description	Notes
FTP bounce protection	YES	0.2.0	Reject PORT if IP address specified in it does not match client IP address. Drop the incoming (PASV) data connection for the same reason.	Configurable.
Restrict PASV/PORT to non privileged ports	YES	0.2.0	Reject connections to privileged ports.	Configurable.
Brute force protection (1)	YES	0.1.0	Disconnect client after a certain number (3 or 5) of wrong authentications.	Configurable.
Brute force protection (2)	YES	0.5.0	Impose a 5 second delay before replying to an invalid “PASS” command to diminish the efficiency of a brute force attack.
Per-source-IP limit	YES	0.2.0	Limit the total number of per-ip control connections to avoid parallel brute-force attack attempts.	Configurable.
Do not reject wrong usernames	YES	0.1.0	Always return 331 to the USER command to prevent client from determining valid usernames on the server.
Port stealing protection	YES	0.1.1	Use random-assigned local ports for data connections.

RFC-2640 - Internationalization of the File Transfer Protocol ¶

Extends the FTP protocol to support multiple character sets, in addition to the original 7-bit ASCII. Introduces the new LANG command.

Issued: July 1999
Status: PROPOSED STANDARD
Updates: RFC-959
Link

Feature	Implemented	Milestone	Description	Notes
LANG command	NO	—	Set current response’s language.
Support for UNICODE	YES	1.0.0	For support of global compatibility it is rencommended that clients and servers use UTF-8 encoding when exchanging pathnames.

RFC-3659 - Extensions to FTP ¶

Four new commands are added: “SIZE”, “MDTM”, “MLST”, and “MLSD”. The existing command “REST” is modified.

Issued: March 2007
Status: PROPOSED STANDARD
Updates: RFC-959
Link

Feature	Implemented	Milestone	Description	Notes
MDTM command	YES	0.1.0	Get file’s last modification time
MLSD command	YES	0.3.0	Get directory list in a standardized form.
MLST command	YES	0.3.0	Get file information in a standardized form.
SIZE command	YES	0.1.0	Get file size.	In case of ASCII TYPE it does not perform the ASCII conversion to avoid DoS conditions (see FAQs for more details).
TVSF mechanism	YES	0.1.0	Provide a file system naming conventions modeled loosely upon those of the Unix file system supporting relative and absolute path names.
Minimum required set of MLST facts	YES	0.3.0	If conceivably possible, support at least the type, perm, size, unique, and modify MLSX command facts.
GMT should be used for timestamps	YES	0.6.0	All times reported by MDTM, LIST, MLSD and MLST commands must be in GMT times	Possibility to change time display between GMT and local time provided as “use_gmt_times” attribute

RFC-4217 - Securing FTP with TLS ¶

Provides a description on how to implement TLS as a security mechanism to secure FTP clients and/or servers.

Issued: October 2005
Status: STANDARD
Updates: RFC-959, RFC-2246, RFC-2228
Link

Command	Implemented	Milestone	Description	Notes
AUTH	YES	—	Authentication/Security Mechanism.
CCC	NO	—	Clear Command Channel.
PBSZ	YES	—	Protection Buffer Size.	Implemented as as a no-op as recommended.
PROT	YES	—	Data Channel Protection Level.	Support only “P” and “C” protection levels.

Unofficial commands ¶

These are commands not officialy included in any RFC but many FTP servers implement them.

Command	Implemented	Milestone	Description	Notes
SITE CHMOD	YES	0.7.0	Change file mode.

Adoptions ¶

Table of Contents

Here comes a list of softwares and systems using pyftpdlib. In case you want to add your software to such list add a comment below. Please help us in keeping such list updated.

Packages ¶

Following lists the packages of pyftpdlib from various platforms.

Debian ¶

A .deb packaged version of pyftpdlib is available.

Fedora ¶

A RPM packaged version is available.

FreeBSD ¶

A freshport is available.

GNU Darwin ¶

GNU Darwin is a Unix distribution which focuses on the porting of free software to Darwin and Mac OS X. pyftpdlib has been recently included in the official repositories to make users can easily install and use it on GNU Darwin systems.

Softwares ¶

Following lists the softwares adopting pyftpdlib.

Google Chrome ¶

http://pyftpdlib.googlecode.com/svn-history/wiki/images/chrome.jpg

Google Chrome is the new free and open source web browser developed by Google. Google Chromium, the open source project behind Google Chrome, included pyftpdlib in the code base to develop Google Chrome’s FTP client unit tests.

Smartfile ¶

Smartfile is a market leader in FTP and online file storage that has a robust and easy-to-use web interface. We utilize pyftpdlib as the underpinnings of our FTP service. Pyftpdlib gives us the flexibility we require to integrate FTP with the rest of our application.

Bazaar ¶

Bazaar is a distributed version control system similar to Subversion which supports different protocols among which FTP. As for Google Chrome, Bazaar recently adopted pyftpdlib as base FTP server to implement internal FTP unit tests.

Python for OpenVMS ¶

http://pyftpdlib.googlecode.com/svn-history/wiki/images/pyopenvms.png

OpenVMS is an operating system that runs on the VAX and Alpha families of computers, now owned by Hewlett-Packard. vmspython is a porting of the original cPython interpreter that runs on OpenVMS platforms. pyftpdlib recently became a standard library module installed by default on every new vmspython installation.

http://www.vmspython.org/DownloadAndInstallationPython

OpenERP ¶

OpenERP is an Open Source enterprise management software. It covers and integrates most enterprise needs and processes: accounting, hr, sales, crm, purchase, stock, production, services management, project management, marketing campaign, management by affairs. OpenERP recently included pyftpdlib as plug in to serve documents via FTP.

Plumi ¶

Plumi is a video sharing Content Management System based on Plone that enables you to create your own sophisticated video sharing site. pyftpdlib has been included in Plumi to allow resumable large video file uploads into Zope.

put.io FTP connector ¶

http://pyftpdlib.googlecode.com/svn-history/wiki/images/putio.png

A proof of concept FTP server that proxies FTP clients requests to putio via HTTP, or in other words an FTP interface to put.io Put.io is a storage service that fetches media files remotely and lets you stream them immediately. More info can be found here. See https://github.com/ybrs/putio-ftp-connector blog entry

Rackspace Cloud’s FTP ¶

http://pyftpdlib.googlecode.com/svn-history/wiki/images/rackspace-cloud-hosting.jpg

ftp-cloudfs is a ftp server acting as a proxy to Rackspace Cloud Files. It allows you to connect via any FTP client to do upload/download or create containers.

Far Manager ¶

Far Manager is a program for managing files and archives in Windows operating systems. Far Manager recently included pyftpdlib as a plug-in for making the current directory accessible through FTP. Convenient for exchanging files with virtual machines.

Google Pages FTPd ¶

http://pyftpdlib.googlecode.com/svn-history/wiki/images/google-pages.gif

gpftpd is a pyftpdlib based FTP server you can connect to using your Google e-mail account. It redirects you to all files hosted on your Google Pages account giving you access to download them and upload new ones.

Peerscape ¶

Peerscape is an experimental peer-to-peer social network implemented as an extension to the Firefox web browser. It implements a kind of serverless read-write web supporting third-party AJAX application development. Under the hood, your computer stores copies of your data, the data of your friends and the groups you have joined, and some data about, e.g., friends of friends. It also caches copies of other data that you navigate to. Computers that store the same data establish connections among themselves to keep it in sync.

feitp-server ¶

An extra layer on top of pyftpdlib introducing multi processing capabilities and overall higher performances.

Symbian Python FTP server ¶

http://pyftpdlib.googlecode.com/svn-history/wiki/images/symbianftp.png

An FTP server for Symbian OS: http://code.google.com/p/sypftp/

ftp-cloudfs ¶

An FTP server acting as a proxy to Rackspace Cloud Files or to OpenStack Swift. It allow you to connect via any FTP client to do upload/download or create containers: https://github.com/chmouel/ftp-cloudfs

Sierramobilepos ¶

The goal of this project is to extend Openbravo POS to use Windows Mobile Professional or Standard devices. It will import the data from Ob POS (originally in Postgres, later MySql). This data will reside in a database using sqlite3. Later a program will allow to sync by FTP or using a USB cable connected to the WinMob device. link

Faetus ¶

Faetus is a FTP server that translates FTP commands into Amazon S3 API calls providing an FTP interface on top of Amazon S3 storage.

Pyfilesystem ¶

Pyfilesystem is a Python module that provides a common interface to many types of filesystem, and provides some powerful features such as exposing filesystems over an internet connection, or to the native filesystem. It uses pyftpdlib as a backend for testing its FTP component.

Manent ¶

Manent is an algorithmically strong backup and archival program which can offer remote backup via a pyftpdlib-based S/FTP server.

Aksy ¶

Aksy is a Python module to control S5000/S6000, Z4/Z8 and MPC4000 Akai sampler models with System Exclusive over USB. Aksy introduced the possibility to mount samplers as web folders and manage files on the sampler via FTP.

Imgserve ¶

Imgserve is a python image processing server designed to provide image processing service. It can utilize modern multicore CPU to achieve higher throughput and possibly better performance. It uses pyftpdlib to permit image downloading/uploading through FTP/FTPS.

Shareme ¶

Ever needed to share a directory between two computers? Usually this is done using NFS, FTP or Samba, which could be a pain to setup when you just want to move some files around. Shareme is a small FTP server that, without configuration files or manuals to learn, will publish your directory, and users can download from it and upload files and directory. Just open a shell and run shareme -d ~/incoming/ …and that’s it!

Welcome to pyftpdlib’s documentation¶

Install¶

Additional dependencies¶

Benchmarks¶

pyftpdlib 0.7.0 vs. pyftpdlib 1.0.0¶

pyftpdlib vs. proftpd 1.3.4¶

pyftpdlib vs. vsftpd 2.3.5¶

pyftpdlib vs. Twisted 12.3¶

Memory usage¶

Interpreting the results¶

Setup¶

proftpd¶

vsftpd¶

twisted FTP server¶

pyftpdlib¶

Indices and tables¶