NcFTPd: Quotas; page 2 of 2

Enabling NcFTPd quota support

The general.cf option u-vquota must be set to yes. This toggle controls whether NcFTPd honors individual user quotas it finds, and can therefore be used to turn off NcFTPd quotas system wide. In fact, if the option is omitted entirely, it defaults to yes since user quotas are only enforced when NcFTPd finds that a user has a quota configuration.

Creating quota-restricted users using ncftpd_passwd

The real work is accomplished by setting the individual quota configurations for each user. Quota settings are stored in NcFTPd password databases. One way to do this is to use ncftpd_passwd utility program and set the seven quota fields in the user record. These seven fields are:

disk usage kilobytes, hard limit
disk usage kilobytes, soft limit
number of files, hard limit
number of files, soft limit
quota options
download bandwidth limit, in kilobytes per second
upload bandwidth limit, in kilobytes per second

Note that the soft limits and quota options fields are reserved for future use. The NcFTPd quotas are modeled after the OS-level quotas which feature both soft and hard limits.

These seven fields are all integer fields that follow the shell field in the password database. A zero in any field (or missing the field entirely) means no limit. For example, a user record without quotas might look like:

mmgleason:unUuvgMaz/h76:696:33,31:Michelle:/home/mmgleason:/bin/ksh

Adding a 3 megabyte (3072 kB) disk usage hard limit, a 2800 kB soft limit, a 500 file hard limit, a 450 file soft limit, a zero for quota options, a zero for no download bandwidth limit, and a zero for no upload bandwidth limit to the record would cause the user record to appear like:

mmgleason:unUuvgMaz/h76:696:33,31:Michelle:/home/mmgleason:/bin/ksh:3072:2800:500:450:0:0:0

Essentially all you need to do when creating your user records is to append :hardKB:softKB:hardNum:softNum:0:downBW:upBW to your record. Unfortunately this becomes cumbersome when you want to change an existing quota for a particular user, since you would have to query the record first, copy and paste the first 7 fields, then add the 7 changed quota fields, and use that string to update the record.

Setting quotas using ncftpd_edquota

As mentioned above, using ncftpd_passwd to create a user and set initial quotas is reasonable, but using ncftpd_passwd to change an existing user's quota becomes tedious. As an alternative, you can use the ncftpd_edquota utility program to change quotas, or to add initial quotas for users that exist in your password database but do not have the quota fields set.

There are two modes to the program. The first is setting a single user's quota by explicitly inputting the values. Here's an example that sets the quota for the waldo user in the password database local.db. I typed "7600" and "800" as new values for the quotas:

# ./ncftpd_edquota -f local.db waldo
    Hard quota, kB [7500] : 7600
    Hard quota, files [500] : 800
    Download bandwidth limit, in kB/sec [0 (no limit)] : 12
    Upload bandwidth limit, in kB/sec [0 (no limit)] : 0
    Quotas changed for user waldo.

As with OS quotas (and the "edquota" program), a value of zero means no limit. The values shown in brackets are the existing settings, and if you just hit return, that value will not be changed.

Note: This example also shows how to set indivudal bandwidth limits for users, in kilobytes per second. Although a bandwidth limit is a slightly different concept than a disk quota, this configuration information is managed as a part of the quota system. If you want only bandwidth limits and no quotas, you would simply leave the quota fields at 0.

The second mode is modeled after the "edquota" program's prototype mode, where you can use one user's quota settings as a template and copy those settings to other users. For example, if you wanted to replicate waldo's configuration for the users huey, duey, and louie, you could do that like this:

# ./ncftpd_edquota -f local.db -p waldo huey duey louie
    Quotas changed for user huey.
    Quotas changed for user duey.
    Quotas changed for user louie.

Auditing all users quotas using ncftpd_repquota

Like the OS quotas "repquota" program, NcFTPd supplies a program which can produce an audit report for users, aptly named "ncftpd_repquota". You simply run it and tell it which password database to iterate users from, and let it produce the summary report on standard output. An example run might look like "ncftpd_repquota -f /path/to/the.db". You can also use the optional "-v" flag to check disk usage for users in the database that do not have quotas.

Since a "du -k -s" is done for each user in the database, be aware that this could impact system performance.

How users can view their quota status

The most reliable way to let users view their quota status is to take advantage of the special cookies NcFTPd supports in the login message (specified by u-login-message). A "cookie" is a special character sequence that is expanded each time by the server when it prints the message. This makes it possible to create a single text file containing the message contents, but is updated each time it is printed. The cookies for quota support are:

%qbh - expands to disk usage in bytes, hard limit
%qbu - expands to current disk usage in bytes
%qfh - expands to number of files, hard limit
%qfu - expands to current number of files
%qd - expands to the number of seconds NcFTPd spent calculating the disk quotas for this user.
%bdn - expands to the downloading bandwidth limit
%bup - expands to the uploading bandwidth limit

The %qd cookie can be useful for testing, to see how long a user is waiting while the server was traversing the directory structure.

Here is some sample text that could be used as a login message:

Welcome to ftp.example.com!  All actions are logged; if you don't like this, disconnect now.
Your disk usage quota is %qbh bytes.  The maximum number of files you are allotted is %qfh.

Notice how this example did not use the %qbu or %qfu. While these are useful, those cookies would require that the disk usage calculations be performed at login time, instead of being postponed until really needed. The quota-related cookies can be used anywhere NcFTPd expands cookies, which includes the change-of-directory .message files and u-goodbye-message. Unfortunately there is no standard behavior which requires FTP clients to display any of these messages to the user.

Another way a user can find out their quota status is to have the FTP client program issue a "SITE QUOTA" command. Here's an example using the standard bare-bones command-line FTP client:

ftp> site quota
200-Quota report:
200-  Bytes: 6660649  Limit: 7680000
200-  Files: 647  Limit: 650
200 Done (0.03 seconds used initially to compute quota).
ftp>

What a user sees when reaching their quota

Again, the user may not see any message from their FTP client program aside from the fact that the upload failed. The infamous "server returned extended information" message that Internet Explorer displays is one such example where the message being sent by the server would be helpful to the user.

Here is what happens when using the bare-bones command line FTP client, to illustrate the error returned:

ftp> put myfile.zip
local: myfile.zip remote: myfile.zip
200 PORT command successful.
450 Disk quota exceeded (7682249 bytes;  limit: 7680000).
ftp>

(This example also shows that the hard limit is really not a firm, to-the-exact-byte limit. In fact, the user could exceed the limit by as much as 4095 bytes. This behavior is required for optimal performance at the expense of a small amount of bytes.)

Significant differences between OS-level quotas and NcFTPd quotas

NcFTPd can consume a significant amount of time and system resources in order to calculate the initial disk usage; OS quotas are calculated efficiently in real time after system-startup.
OS quotas have both soft and hard limits; NcFTPd currently only supports hard limits.
OS quotas have time limits used in conjunction with the soft limits; NcFTPd currently does not support soft limits (and hence time limits).
NcFTPd quotas work on every OS that it runs on; OS quotas are not standard on each platform.
NcFTPd quota changes are only detected in the current FTP session; OS quotas effect every process running concurrently.
NcFTPd disk usage limit may be exceeded by a slight amount; OS quota hard limits cannot be exceeded by any amount.
NcFTPd quotas only work in conjunction with virtual users from NcFTPd password databases; OS quotas work with users from /etc/passwd.
NcFTPd stores user quota information along side the user's account information in a password database; OS quotas store password and quota information in different files.

Quota Mini-FAQ

Q. I *really* need NcFTPd quotas to work with users from /etc/passwd. How can I do this?

A. You will need to create a NcFTPd password database by importing your existing /etc/passwd. If you only want quotas for a few users, you can create an "override" database, where the users you want to have quotas exist in both /etc/passwd and a password database, and users that don't have quotas only exist in /etc/passwd. You would then set "passwd=/path/to/the/override.db,/etc/passwd" in the domain.cf to specify that both a database and /etc/passwd are checked for users.

Q. Users complain that their FTP client times out when they login (or try and upload).

A. The user may have a very deep and complex directory structure. NcFTPd may need a significant amount of time to calculate the disk quota, and the FTP client may have gotten impatient and assumed the server connection had been lost. Have the user try a different FTP client.

Q. Users complain that uploads fail.

A. By design, the quota system doesn't allow any more bytes uploaded when the user reaches their quota. Show the user how to check their quota and perhaps suggest a different FTP client which can show the quota statistics in your login or goodbye message. Ideally this client program should also display the error message from the server, so when this happens, the user will see "disk quota exceeded" message text.

Q. Users complain that uploads are corrupted.

A. Some FTP clients don't report that the upload was not successful! Others display an error, but not prominently enough for the user to notice.

Q. Does NcFTPd track when the disk usage goes down in a session, or only up?

A. File and directory removals are reflected in the disk usage, so amount of disk used should go down when the user deletes files in the session.

Q. Do directories and links count as "files" in the file limit?

A. Yes. They do not, however, count towards the byte usage limit.

Q. Can I set a system-wide defaults for quotas and bandwidth limits?

A. Disk quotas must be set per-user, but you can set a domain-wide bandwidth limit which would let you use bandwidth limits without using NcFTPd password databases. See the documentation for download-bandwidth-per-user.

Working with password databases

See the documentation for the ncftpd_passwd utility for information about how you can create a database, add/remove users, etc. This utility allows you to set user quotas since the user record contains the quota fields.

Configuration Summary

General configuration options:

u-vquota