Facebook
Digg
StumbleUpon
del.icio.usThanks to massive amounts of legal torrent downloads, I always find disk space lacking. Over the years, I've gone from 40GB in my fileserver to 940, and I plan on buying another 320GB disk shortly. This page has a few notes on setting up disks in FreeBSD and some ideas on manageable filesystem layout. I have found it effective for my low-load home server with ATA disks.
Partition Layout
I have found the following layout very effective in keeping things organized. Obviously, different people have different needs, but this should lend a hand in coming up with something decent.
First and foremost, I like to avoid over-partitioning. The problem with having many partitions is that it is not flexible; one could have huge amounts of free space while another is almost completely full. Further, copying or moving files from one partition to another on the same physical disk is very slow. On the other hand, having everything on one giant partition takes away reliability; any damage to that partition will take everything with it. Also, if a partition that the OS uses for itself runs out of space, problems could occur. It sure would be lousy to have your website or other important service stop working because you filled up the disk with street racing videos.
Base System
With the above in mind, I use the following as a starting point:
Mounted Size / 400M swap Min: amount of ram + 100MB; Max: 2G /var 2G // allows log files to swell way up. /usr 9-15G // Depends on num apps installed.
The root filesystem, /, does not need to be very big
in FreeBSD. It is read-mostly, and uses a consistent, predictible
amount of space. In fact, it could be scaled down to 256MB without
issue, I've only gone bigger because I can.
Make the swap partition somewhere between the amount
of RAM the system has plus a few megs, to a gig or two. I find the
more common formula of 2 x RAM to be a bit weird. A
system with 32MB RAM gets 64MB that way, whereas a system with 4GB
RAM gets 8GB swap. Why?
The /var partiton needs to be pretty big to be able to
store swelling log files. The only time that log files are much use
is when something bad is happening, and in this cruel world, that
usually means events such as DDoS attacks, and so forth. If
/var fills up, you will never be able to see those
last few critical lines of information. Best to just make it
huge, unless it's a desktop system that isn't doing anything important.
The /usr filesystem is a bit of a grey area. I don't
host my web content on it, whereas most people do (more on that later),
so I don't need to consider that as a factor. However, the partition
is still used for compiling ports and the base system, application
installations and configurations, and who-knows-what else. Systems
that are only running a handful of services can have small /usr
partitions, but if there are plans for tons of installed applications and
so forth, best to make it huge.
You'll notice the lack of a /tmp partition. As I mentioned
a while ago, I don't like to over-partition. When I need a temporary
storage directory, I usually need huge amounts of space, but only for a while;
the rest of the time, I need very little. I have addressed this by removing
the /tmp partition altogether, and replacing it with a symlink
to /var/tmp. I never use that directory myself, the only things
that use it are applications programmed to do so.
Home Partition
Choosing a partition size for /home is easy: make it as big as possible.
The default in FreeBSD is to have /home be a symlink to /usr/home,
but I don't like that. I prefer to have /home be in complete isolation from
any of the base system partitons, because it is by far the most important. Luckily,
the entries in the passwd database in FreeBSD point to /home/<user>
by default, so this change is easy; it disturbs nothing.
Everything Else
Now that a base system has been setup, it's time for everything else. This is where few partitions is certainly better than many. Specific disk space needs from here are very unpredictable.
Because the /home partiton is so important, I like to keep unimportant
things like all of my TV show downloads and so forth away from it (just like the
base system). Further, I have a considerable collection of videos and pictures and
the like, so a lot of space needs to go to this. As of this writing, the magic
number is 1.12TB, which is more space than I currently own in disks.
To do this, I have a directory named /fs, which is just short for
"filesystem". It may be unoriginal, but it's easy to type! Within it, I have the following
subdirectories:
Directory Purpose
---------------------
incoming Incoming data: new downloads, etc.
local Contains pictures, videos, projects,
etc that I consider to be non-private.
The images section of this website is just
a link to local/media.
public The Big One. All videos and whatnot
that I've downloaded off of the internet
goes here. This is where the 1.12TB
figure comes from.
tmp A temp directory.
Having all of the above directories in one place allows them to take
up space as they need it. If I don't bother to sort out my downloads
in the incoming directory for a long while, it doesn't
matter, because all of that data is using the same space as it would be
if it was sitting in the correct subfolder of /fs/public.
Also, when I move things between incoming and public,
the disk doesn't need to grind away, it just renames them in a fraction
of a second.
Disk Utilities
In this section, I'll talk about what utilites do what, and some optimisations that make things quicker or more elite. I will also mention some handy ways of moving and verifying data between partitions.
Fdisk
fdisk is very powerful tool, and luckily has a powerful manpage.
To save some time, here are a few common commands for it:
fdisk -I /dev/ad6 # Create a DOS partition ("slice")
# that spans the entire disk (ad6)
# and is not bootable
fdisk -BI /dev/ad6 # Same thing, but bootable
When running one of the above commands on a brand new disk, an error about an invalid partition table being found will be displayed. This is fine because the disk is new and therefore has no partition table at all! To see for yourself, run the command twice; the error will not be there the second time.
BSDlabel
I'll update this when I learn how to use bsdlabel. I just
cheat and use sysinstall.
Newfs
The newfs utility has many options, some of which I
consider obsolete, while the rest need not be changed from default anyways.
In fact, usually a simple newfs /dev/node is all that
is required for a proper filesystem. However, some of the defaults
are (in my opinion) out of date, and can be changed.
One such option is the -m option, which specifies the
percentage of the filesystem to reserve from non-root users. It defaults to
8%. On a 250GB disk, that is a colossal waste of space, for no visible
benefit. I use -m 5 on volumes such as /usr and
/var, and others that are critical for system operation.
On volumes that can stand to fill up without causing problems, I use
-m 0, and turn a blissfully ignorant eye to the warnings
is produces. The kernel will then optimize the filesystem for space
instead of time, which causes a write performance hit. This can be circumvented
by changing the MINFREE constant to 0, defined in
/usr/src/sys/ufs/ffs/fs.h on line ~161. For this to take effect,
the kernel will need to be recompiled.
Another option that I almost always change is the value for -i,
the number of bytes per inode. Fewer inodes means faster fsck times,
to a certain degree. It defaults to 4096, which creates
far too many inodes in my case. I have turned it up as high as 65536
in the past. You need to be careful to not go too high with this value, or too
few inodes will be created, causing you to run out of inodes, while still having
free space on the volume. Not cool.
If you are putting nothing but large files (such as mp3s, DivX, etc) on the
volume, you can greatly reduce the number of inodes the filesystem needs.
The number of bytes per inode (which is what
-i is), must be less than the average file size of the
filesystem by a significant margin.
To be clear:
Large value for -i:
fewer inodes created
filesystem is same size,
but less number of files will fit on
faster fsck times
Small value for -i:
exact opposite
Max number of files on partition is equal to:
[partition size in bytes] / [-i value]
Therefore:
[-i value] < [average file size]
....to avoid running out of inodes before disk space.
Inode usage can be seen with:
df -i
[G]Vinum for Striping
In light of the fact that I need so much space all in one place, I've had to fuse multiple disks into one partition. Currently, I have 2x 250GB disks in this configuration, which after formatting and factoring in the base10-base2 marketing ripoff factor, gives me a single 464GB partition to play with. This is known as a "RAID-0" configuration, or "striping".
The way I did this is with gvinum, the FreeBSD 5
replacement for the old vinum utility. If you are
wondering, the g is short for GEOM,
which simply means that this new utility is GEOM-aware.
Vinum/Gvinum is a software-raid utility.
To get the two disks setup in RAID-0 with a single partition spanning both of them, I did the following:
On my installation of FreeBSD 5.4, the rc system
had no understanding of gvnium, so I had to patch it myself.
# in /etc/rc.conf: start_gvinum="YES" # create a file named "/etc/rc.d/gvinum" # and in it, put this: #!/bin/sh # PROVIDE: disks # KEYWORD: nojail . /etc/rc.subr name="gvinum" rcvar="start_gvinum" start_cmd="gvinum start" stop_cmd=":" load_rc_config $name run_rc_command "$1"
After that headache, I setup gvinum itself to recognise my particular disks,
and config them as a RAID-0 partition. To do this, I ran fdisk -I
on both disks, and then bsdlabel -we on both. This gives me
/dev/ad4s1a and /dev/ad6s1a nodes to represent each
disk.
# in /etc/vinum:
drive d1 device /dev/ad4s1a
drive d2 device /dev/ad6s1a
volume fs
plex org striped 1m
sd length 0 drive d1
sd length 0 drive d2
The vinum partition is named 'fs', to match its mountpoint. This isn't necessary, but I find it makes things easier. "length 0" means "auto size". Back in the day, the size of each partition had to be determined manually. The "1m" is the stripe size. Bigger stripe sizes make reads and writes of large files faster, and conversely, smaller files a bit slower. I can't tell much difference between different settings.
# Running newfs on a vinum volume needs -v: newfs -v -[other options here] /dev/gvinum/fs # in /etc/fstab: /dev/gvinum/fs /fs ufs rw,nodev,noatime 2 2
I'm pretty sure I've forgotten some significant parts of setting up a gvinum RAID-0 volume. It was a long time ago, and I've forgotten a lot of it. I do remember having a hard time with it though. It's probably best to read the handbook section on vinum as well.
Copying Data
This might just be the easiest part of the whole operation. Whenever I'm copying
a whole partition's data, i use dump. dump is the top
dog for backups, and for tasks like these. This isn't opinion; it's fact. Using
dump for this application is simple. Make sure to be the superuser
(root).
# safety handle: make the source partition readonly.
mount -u -o ro /dev/node_for_source_part
# Do the transfer
dump -0uan -f - /dev/node_for_source_part \
| (cd /destdir && restore rf -)
The dump can take a very long time. It displays its progress as it goes,
albeit not very often. You can press control-T to make it
display its progress at that moment. Many userland applications respond
to the SIGINFO signal that ^T sends, but you knew that already.
When I want to copy a specific directory (as in, not a whole partition), I
use pax. pax is good at preserving things like
permissions and ownership, and so on. Its error messages are also more
useful than most apps out there. pax is in the base system.
To copy a directory from one place to another, use the command options
below:
pax -rw -p e /src_dir /dest_dir
Verifying Data
The simplest way (that I know of) to verify that a directory is the same
as another is by using the good old diff utility, as
shown below.
diff -r /dir1 /dir2
Conclusion
Setting up disks is a very complex thing, and depends a lot on the needs of the users. The important things to remember is to plan ahead and allow for changes in needs as time goes on.
