Introduction

Ghost4Linux (or more accurately, GhostInLinux) is a collection of scripts, programs and kernel modules for network booting Norton Ghost on a given target machine and backing up or restoring that machine. It is built from a net booting Linux, which runs DosEmu, the DOS emulator for Linux, which in turn runs Norton Ghost with the appropriate options. It then automatically chooses the first drive and writes out its contents to an image. The image directory is a separate NFS file system, including a different machine. The overall result is a backup/restore system for NTFS and FAT file systems that can backup to/restore from a NFS file server.

What is required

1. The client machines must be PXE boot capable, with modern on board NICs this is not normally a problem as the boot rom is a bios option.

2. The server side of the above, a dhcp server and tftp server. The backup trigger script must have access to (and the ability to restart) the dhcpd server.

3. Need to prepare a Linux system that can be remotely booted. In my case a copy of my firewall machine was a good starting point.

4. Need to prepare a working bootable Ghost floppy, as I can't supply you with one. Norton Ghost 2003 is known to work. Ghost V8 is known to die shortly after starting ghost. Linux is used for the networking, so this only needs to know about hard drives, not dos network stacks. This is modified to take advantage of it running under emulation.

5. Minor but very essential programs like ether-wake and ethtool.

General idea

The overall idea is to use Linux for its network stack and general scripting capabilities to run a complete diskless system. It is only really diskless from the point of view of Linux, any harddisk in the system is a data sink or source that Linux itself does not care about. Once booted, we can then have a script run DosEmu, which can then be setup to autorun Ghost and backup the drive to image files.

The devil here is in the details: changing the dhcpd config so the system normally boots its HD based OS, but then loads the remote Linux system when required; checking the machine has started, and then changing the dhcpd config back to allow the user to boot into the native OS.

1.1. PXE on the client side

The client needs to have PXE capable boot rom, which allows it to boot without local data on the HD, or even a HD at all. If the motherboard has an on-board NIC, chances are there is a 'Boot ROM' option next to the 'Enable LAN' option. The boot order also needs changing so LAN is first, and a local harddisk is second.

1.2. dhcpd

The dhcpd configuration is all in /etc/dhcpd.conf, using the standard RedHat distro DHCP server (Written by Ted Lemon for Vixie Labs, ISC). The layout follows the form:
default-lease-time 86400;
max-lease-time 86400;
option routers 192.168.7.1;
option domain-name-servers 192.168.7.1;
option domain-name "flint";
option subnet-mask 255.255.255.128;
option broadcast-address 192.168.7.127;

subnet 192.168.7.0 netmask 255.255.255.128 { range 192.168.7.64 192.168.7.126 ; }
host john { fixed-address 192.168.7.100; hardware ethernet 00:01:80:5C:37:D4; filename "nobackup"; }
host joe { fixed-address 192.168.7.15; hardware ethernet 00:20:ED:6F:30:61; filename "nobackup"; }
host linuxbox { fixed-address 192.168.7.10; hardware ethernet 00:01:80:53:FA:A6; filename "nobackup"; }
host fakebox { fixed-address 192.168.7.10; hardware ethernet 01:80:53:FA:A6:00; filename "pxelinux.0"; }

The layout of the last four lines is important, the backup script changes the "nobackup" filename to "pxelinux.0" as the first step into booting Linux remotely. The host name, IP, MAC address and filename must be valid as the backup script uses all these details at some point. The other lines are free format as far as the backup is concerned, and obviously must be adapted to your scenario.

Under normal operation, the client machine will boot up, DHCP would reply with the "nobackup" name, and the PXE boot would fail and then boot the local HD. When the backup script is run, it changes the dhcpd.conf, restarts the dhcpd daemon and sends a wake up packet to the MAC. The machine will then power on, boot the pxelinux.0 image (the next step), and meanwhile the script changes the dhcpd server back to "nobackup". The script creates a lock file so there are no race conditions when changing the daemon configuration.

1.3. tftp

PXE uses the tftp protocol to fetch files from the server. The setup here used the HP tftp server (tftp-hpa), not the RedHat supplied server. The backup script assumes the base path is /tftpboot , which is normally the default.

1.4. SYSLinux

The PXE boot process needs a loader to boot the Linux kernel, which is also an opportunity to pass arguments to the kernel. We use the same options for all machines (/tftpboot/pxelinux.cfg/default):
DEFAULT vmlinuz IPAPPEND 1
APPEND root=/dev/nfs nfsroot=192.168.7.1:/home/pxe/,wsize=8192,rsize=8192,hard,intr ip=dhcp

PXELinux (part of SYSLinux) passes these options to the kernel found at /tfptboot/vmlinuz, so just remember when building a new kernel to update this copy of the kernel too.

2. Setting up a bootable Linux system.

When each backup client boots, its root directory is a directory on the server. The root needs to contain all the usual startup software that will allow it to boot up, startup networking (which is somehow separate from the NFS root that will already be running) and mount the backup image NFS mount. For debugging, a working ssh daemon might also be useful.

For my base system I used my firewall machine, which is a RedHat 7.3 install with many small additions and a kernel.org kernel. This was simply copied to a subdirectory of the same machine, /home/pxe, which is the NFS root. The bash was something like:
cd /
mkdir /home/pxe
find / | grep -v "^/home/pxe/" | cpio -padmu /home/pxe/

Some of the large and unnecessary installed software, such as the browser and Java were then deleted. How much time you want to spend removing software is up to you, storage costs being what they are, there is no real point in deleting anything.

At this point it might be worth changing the root (chroot /home/pxe ; passwd), and setting the run level to 3 (vi /etc/inittab). Note, 'chroot /home/pxe' is a handy way of working on the NFS root.

3 Preparing a bootable Linux

3.1

Look at the NFS ROOT HOWTO for the details. In short, configure the kernel for an i386 processor (or the lowest common denominator) with IP auto configuration and NFS ROOT enabled and built into the kernel. Also, build into the kernel the NFS client options and the network cards. Modules are a bad idea for anything but optional extras. Your new kernel should be copied to this directory in /home/pxe/boot and to the /tftpboot directory.

3.2

Setup /etc/fstab to use the NFS drives as root and as the /backup mount point, which will later be used by Ghost. In my case this looks like:
PXEADDR:PXEBASE    /                nfs     wsize=8192,rsize=8192,hard,intr     0 0
none               /dev/pts         devpts  gid=5,mode=620  0 0
proc               /proc            proc    defaults        0 0
tmpfs              /tmp             tmpfs   size=128m       0 0
tmpfs              /var/tmp         tmpfs   size=128m       0 0
GHSTNFS            /backup          nfs  rsize=8192,wsize=8192,hard,intr,tcp  0 0

Where PXEADDR:PXEBASE refers to the NFS server and where on that machine the root can be found. GHSTNFS refers to the NFS server for backup images, it can be different to the PXE server. In my case, PXEADDR is 192.168.7.1 (my firewall and dhcp server) and PXEBASE is /home/pxe. As we need to modify the dhcpd configuration and restart dhcpd, the backup scripts need to be on that machine. GHSTNFS (192.168.7.9:/home/backup) is the file server, so it has storage space but nothing else.

Note also the use of tmpfs for temporary storage. The key words PXEADDR, PXEBASE and GHSTNFS are edited by the server side scripts, the actual values from the configuration file.

3.3

When a backup operation is started, separate directories are created for /etc and /var, this is the minimum necessary change. But, as we need those directories at boot time, it is a little awkward to create extra mount points and have separate /etc and /var mounted after / has been mounted. The solution is to use the --bind option of mount. The separate directories created by the backup script are in a directory accessible via the NFS mount of /home/pxe and called <client ip>/etc and <client ip>/var.

Consequently, the moment /etc/rc.sysinit is read, it is reading the original /home/pxe/etc/rc.sysinit. But, we add the lines:
echo "Binding custom /var and /etc"
ADDR=`/sbin/ifconfig eth0 | grep "inet addr:" | sed "s/.*inet addr: *\([0-9\.][0-9\.]*\) .*/\1/"`
mount --bind /${ADDR}etc /etc
mount --bind /${ADDR}var /var
just after / is remounted read/write.

So by the time rc.sysinit runs anything that will daemonise etc, it will be running from a now distinct /etc and /var.

3.4

The net effect of these steps should be a Linux system that operates entirely over NFS and uses around 50 megabytes for its individual /etc and /var. You should also have /backup available over NFS. You should check it all works as expected, Ghost needs all this to be setup.

4 Preparing Ghost

4.1 Setup DOSEmu

Download a copy of DOSEmu from http://www.dosemu.org/, I used version 1.2.1 (stable) and then version 1.2.2 (stable), both without problems. It should be installed as root, for root, into /home/pxe (chroot into /home/pxe before building it).

4.2 Make a template boot disk

My boot disk uses MS-DOS as created by a Windows 98 machine. I've had reports of FreeDOS working. Format a bootable (with system files) floppy in Win98. Once you have a bootable disk (floppies being what they are, check that it works), create an image file of it with:
dd if=/dev/fd0 bs=512 of=/home/pxe/root/.dosemu/ghost.img

The correct path is important as all the scripts and DOSEmu asssume the image will be there.

4.3 Edit the boot disk

Mount the boot disk with:
mount -o loop /home/pxe/root/.dosemu/ghost.img /mnt/floppy
and edit AUTOEXEC.BAT to look like:
@echo off
SET TZ=GHO+00:00
rem MOUSE.COM
echo Adding NFS dir as drive
lredir z: LINUX\FS\backup\BackUpName\
echo Loading...
CD GHOST
echo > z:\BackUpName.sta
BackUpCommand
type a:ghosterr.txt >> z:\BackUpName.end
echo Ghost exited >> z:\BackUpName.end
a:\exitemu

The backup script creates individual boot disk images, and so BackUpCommand and BackUpName are replaced with the real command and name when needed.

You also need to copy some files to the boot disk, namely those used above (exitemu, lredir) to /mnt/floppy and GHOST.EXE (I used Norton Ghost 2003) to /mnt/floppy/ghost/.

When done, unmount the image with:
umount /mnt/floppy

4.4 Edit the DOSEmu config file

DOSEmu needs some configuration options to be set. These are:
$_cpu = "80586"
$_rdtsc = (on)
disk {wholedisk "/dev/hda"}

Although I've not tested it much, I suspect the $_cpu should match your kernel. With a 386 kernel, $_cpu = "80386" was stable, but then compiling a 586 kernel I had to also use $_cpu = "80586" to remain stable. The only symptom was hanging at one of a random few points during an image dump.

The _rdtsc line was added while testing for the processor issue. As we only use 586 or more processors, it seems like a good idea.

The wholedisk option lets Ghost see the real HD as a real HD. Contrary to the docs, this option is supported, they just consider it such a security risk that it is left undocumented. We want unrestricted access to the harddisk, so their security risk is our essential feature.

5.0 Other bits

5.1 Kernel module

When Ghost loads, it normally ask "Mark these drives as Ghost?". It does this even when run in batch mode, so is a small problem for our purposes as nobody will be there to select "Don't mark as Ghost". DOSEmu provides some options for sending key presses to the running program, but unfortunately they don't work with Ghost. This kernel module does the job instead, it sends key presses as if they were pressed on the keyboard, not far above the level of keyboard interrupt.

The source for the module is sendscancode.c. It can be compiled with:
chroot /home/pxe
cd /root
gcc -nostdinc -I/usr/src/linux/include -I/usr/lib/gcc-lib/i686-pc-linux-gnu/3.2.3/include sendscancode.c -c -o sendscancode.o
cp sendscancode.o /lib/modules/2.4.27/kernel/drivers/input/
depmod -ra
# and ignore the symbols msg

insmod sendscancode thechar=0xf ; sends a tab

It will probably be obvious that I don't really know how to write kernel modules. This is good enough, as it works and doesn't do anything bad for your health. If somebody knows how to program a module and wants to give me a better solution, I'd be happy to accept it.

5.2 Screen dump

As the backup/restore process will be left unattended, I thought it would be a good idea to have some log. The best way forward here was a screen dump of the running Ghost. The program rdvga.c does just that and is used by the client side scripts to keep a short recent history of what the screen looked like. It is in black and white, but it gives the general idea. Compile with:
gcc -Wall rdvga.c -o /home/pxe/root/rdvga

It will also be obvious that this is a massive hack and only works because ancient backward compatibility is still supporting an MGA screen (I /think/ thats why it works). It has been tested with recent Nvidia graphics cards and old cards of various makers.

A better solution would be using a kernel module to save out the real video memory, as it exists outside of user mode address space. If you know how to do that and send me a solution, I'd be happy to accept it.

5.3 etherwake and miitools

You also need the etherwake program (for sending magic wake up packets) and the miitools package, which contains ethtool that is essential for setting up a NIC to remote wake up. etherwake is used on the server side, so needs to be accessible there. ethtool is used on the client side, so needs to be accessible to /home/pxe.

5.4 netmon

This program monitors the rate of traffic flow ever a network connection. The main client side script uses this to detect problems during the Ghosting process. Its source is available here. As this is run by the client, it needs to be in (say) /home/pxe/bin.

6 The controlling bash scripts

The last few sections have described the programs and configuration that need to be present. This section talks about the high level scripts that pull it together.

6.1 Client side

There are two additional scripts on the client side, a modified /home/pxe/etc/rc.local which has this appended to the end:
# enable WOL for next time
/usr/local/sbin/ethtool -s eth0 wol g

# identify the name of the machine, as found in /etc/dhcpd.conf
MAC=`ifconfig eth0 | grep HWaddr | head -n 1 | sed "s/.*HWaddr //"`
BACKUP=`grep -i $MAC /etc/dhcpd.conf | head -n 1 | sed "s/^host  *\([^ ]*\) .*$/\1/"`

# start the script that runs and then monitors Ghost
export BACKUP
(
sleep 10
export BACKUP
/etc/init.d/ghost start
) &

Also on the client side is the main script that runs and then monitors Ghost. Obviously it needs to go in /home/pxe/etc/init.d because the above script expects it to be there. This script does all the real work of supporting Ghost, so chances are, if you miss something out then it will break here. It is this script that:
1. Copies /root/.dosemu/ghost.img to /root/.dosemu/ghost-<client hostname>.img and modifies the copy for use on this client.
2. Runs DOSEmu and points it at the new boot disk image.
3. Runs the key press simulation to skip the "Mark drives as Ghost?" message.
4. Monitors the network and hard drive access statistics for signs of a trouble, which can be implied by there being very little data flow on the network card or hard drive.
5. Takes screen shots at 1 minute intervals.
And finally,
6. If Ghost quits, for whatever reason, or the statistics indicate trouble, the machine is shutdown.

6.2 Server side

The server side scripts are fairly simple, as their main goals are:

1. Modify /etc/dhcpd.conf to boot Linux remotely rather than whatever is on the local harddisk.
2. Create copies of /home/pxe/{etc,var} called /home/pxe/<client IP>{etc,var}
3. Create the directories for the backup images and screen dumps.
4. Wake up the machine.
5. Undo the changes made to /etc/dhcpd.conf, so the machine now boots locally.
6. Make sure the machine came up, and then wait for it to shut down.

The server side scripts go in /home/pxe/root, but that is just to keep the ghost4linux system together in one place. The scripts are backup.sh, verify.sh, test.sh ghost.sh and restore.sh. But for test.sh and ghost.sh, the meaning of each is obvious. test.sh uses all the same mechanisms to bring up the machine, but doesn't run Ghost. Instead the machine is left running for an hour, giving you the chance to ssh into it and do whatever you like. Killing the running 'sleep 3600' will then shut down the machine within 20 seconds.

ghost.sh also brings up the machine using all the same mechanisms, but runs Ghost with no parameters. This gives you the chance to fiddle with the user interface (ie do a disk to disk copy) without using any boot media.

All the scripts and programs are available here: ghost4linux-1.1.tar.gz

The scripts use a config file /etc/ghostrc to find the right paths. The client also uses this file, so it is copied into /home/pxe/etc by the server side scripts. The config file looks like this:
#!/bin/bash

# IP and path to NFS mounted Linux client
PXEADDR=192.168.7.1
PXEBASE=/home/pxe/

# These two must be the same place or bad things will happen
GHSTBASE=/net/backup/    # image path rel to server
GHSTNFS=192.168.7.9:/home/backup/   # image path when mounted over NFS

# paths
DHCPD=/etc/init.d/dhcpd
ETHERWAKE=/sbin/ether-wake

and obviously needs to be edited for your environment.

To use, you would run:
cd /home/pxe/root
./backup.sh john
./verify.sh john

which should do everything required to backup the machine called john to the file server and then shut down john. The final line would use john to verify the image on the file server and then shut down john.

7.0 Other issues

Note that some ditributions will use different paths, or slightly different names for executables. ether-wake is the standard name, but under Debian it is etherwake. Also, RedHat's dhcpd startup script is /etc/init.d/dhcpd, but under Debian it is /etc/init.d/dhcp. The config file /etc/ghostrc is there to account for these differences.

The End

That should be all you need. If you want more explanation or have ideas, email me on greg at csc liv ac uk.

Ideally all this messing about could be reduced to a patch set agaisnt a small linux distribution. As this is a reasonably large amount of extra work, I'll only do it if several people ask.

History

23/8/2005
  Initial release of complete HOWTO.

26/8/2005
  Fixed some small typos. Added history and license sections.

17/11/2005
  DOS limitations meant using hostnames as filenames broke too easily. As far as DOS is concerned, have now avoided all use dynamic filenames.
  Fixed broken links.
  Added link to tar ball containing all the scripts, calling it V1.1

22/2/2008
  Clayton Craft pointed out a typo, Ghost V8 does not work with DosEmu.

License

This documentation is released under GNU FDL Version 1.2 or later. The programs are released under the GNU GPL.


Last changed 17/11/2005.