Recreate ISO Linking Boot File
Obviously had to recreate the ISO file using mkisofs. Had some trouble deciding on settings here; using some internet sources and common sense lead me to the following command:
mkisofs -r -R -l -J -joliet-long -no-cache-inodes -iso-level 2 -no-emul-boot -b BSCRIPT/BSCRIPT.BIN -c boot/boot.catalog -boot-load-seg 0x0 -boot-load-size 0x0 -o nrmeaio_en.iso /home/lukeoconnell/Desktop/NRMEAIO_EN/cdrom/
- -no-cache-inodes: Stops hardlinks being detected. The option seems to be needed whenever a filesystem does not have unique inode numbers. Cygwin apparently creates fake inode numbers using a hash algorithm which is not 100% correct, so if mkisofs were to follow these inodes, it would believe that some files are identical even if they were not. Hmm, on second thoughts (after revisiting this option to find out what it does exactly), I've decided that this option was not needed unless using mkisofs under a Microsoft operating system.
- -R: Creates SUSP and RR records using the Rock Ridge protocol to provide additional file structure information on an iso9660 filesystem. To my understanding, this allows for backward compatibility, as when folders violate depth (6 seems to be limit), they are recreated on root under RR_MOVED and Rock Ridge points to them instead.
- -r: Option is "like" the -R option, but file ownership and premissions are set to more useful values. UIDs and GIDs are zeroed as they are only useful on authoring system. All file read permissions are set to allow and if executable files exist, all execute permissions are set to allow. Additionally all write permissions are set to deny as will be CDFS. All special mode permissions are also cleared.
- -l: Allows 31 character filenames. By default ISO9660 filesnames will be in an 8.3 format (XXXXXXXX.XXX) which is compatible with MS-DOS. Although the ISO standard allows filename s up to 31 characters, the disk may be difficult to read on MS-DOS systems when using this option.
- -J: Creates Joliet directory records. Mostly useful when disc is to be used on Microsoft OS. Filenames are specified in Unicode and each path component can be up to 64 characters. CDs that use only Joliet extensions but no Rock Ridge extensions can usually only be used with Microsoft OSes.
- -joliet-long: Allows filenames to be up to 103 characters, breaking the Joliet specification, but "appears to work". 103 is comprised of maximum Directory Record Length (254), minus length of Directory Record (33), minus CD-ROM XA System Use Extension Information (14), divided by UTF-16 character size (2). Sort of understand that, but an area of further learning. Hmm... second thoughts on this one as well, do I actually need this extension?
- -iso-level: Sets ISO9660 conformance level. Options are 1, 2, 3 and 4.
- Level 1: Files may only consist of one section and filename are limited to 8.3 characters.
- Level 2: Files may only consist of one section.
- Level 3: No restrictions (other than ISO-9660:1988) apply.
- With level 1, 2, and 3, filenames are restricted to upper case letters, numbers and the underscore. Maximum filename length is restricted to 31 characters and directory nesting level (depth) is restricted to 8. Maximum path length is limited to 255 characters.
- Level 4: Does not officially exist. There is no level 4. Stop reading... mkisofs maps it to ISO-9660:1999 which is ISO-9660 version 2. This version uses an enhanced volume descriptor. Nesting more than 8 directories is AOK, there is no requirement for periods in filenames and the period has no specical meaning/reservation. Filenames do not have version numbers and the maximum length for files and directory is raised to 207 (unless using RR in which case reduced to 197). The grass really is greener with level 4!
(I'll be opting for level 2, as it seems to be the norm for MS OS based discs (as 8.3 limit is raised))
- -no-emul-boot: Determines that the El Torito boot image used is a no emulation image. System will therefore load image without any emulation.
- -c: Specifies where boot catalog should be placed when creating an El Torito bootable CD. Stopped using this as boot catalog is automatically created on the root as boot.catalog when not specified.
- -boot-load-seg: Specifies load segment address for the boot image (for no-emu images). This option seems to be more prevalent with MS based burning apps; mkisofs seems to determine automatically. Stopped using this.
- -boot-load-size: Specifies virtual (512-byte) sectors to load in no-emu mode. Default is to load enture boot file but some BIOSes apparently have problems if not multiple of 4, so in essence, padding image file.
- -o: Specifies ISO output file.
Result: No go. Just blank screen on boot. Hmm, decided to try loader.bin instead... no go also... that same blank scree.
ISOLINUX Boot Attempt
To further explore the boot process, I decided to try and bring in ISOLINUX which was already included on the CD. Initially tried this with:
mkisofs -r -R -l -J -joliet-long -no-cache-
inodes -iso-level 2 -hard-disk-boot -b isolinux.bin -c boot/boot.catalog -no-emul-boot -boot-load-size 4 -v -o nrmeaio_en.iso /home/lukeoconnell/Desktop/NRMEAIO_EN/cdrom/
Result: Error in booting. Decided to cut down on params, using only those from an Internet guide.
mkisofs -o bootcd.iso -b isolinux.bin -c boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table -J -hide-rr-moved -R /home/lukeoconnell/Desktop/NRMEAIO_EN/cdrom/
Some new extensions appeared here, maybe some of them may be useful?
- -boot-info-table: Specifies 56-byte tabel with information of CD-ROM layout. This is patched in an offset 8 in the bootfile (almost like a map?). The bootfile seems to be modified in the source filesystem, so must be backed up.
- -hide-rr-moved: Simply hides RR folders created on root. Seems like a good idea!
So much simpler syntax, but l
acking ISO level... I guess that the default level is 1?
Result: ISOLINUX booted OK, but tried to load linux kernel instead of the MEMDISK kernel I specified (MEMDISK seemed to be used when loading .img files).
After much investigating I discovered that
isolinux.cfg (which I was using to direct to MEMDISK) was very specific about location, I then discovered that the format I was attempting to use:
label dos
kernel /BSCRIPT/MODULES/ISOLINUX/MEMDISK
append initrd=/BSCRIPT/BOOT_MSD.IMG
Was not enough, and ISOLINUX was therefore ignoring config completely. Changed to the following which loaded AOK:
DEFAULT /BSCRIPT/MODULES/ISOLINUX/MEMDISK
APPEND initrd=/BSCRIPT/BOOT_MSD.IMG
During this investigation I tried using the simpler mkisofs params for bscript/loader.bin with no luck.
Working with the LOADER
Determined that the boot loader was a third party application, used by the creator of the DVD. This application, called BootScriptor was responsible for the menu front end (the menu front end that was not loading). The documentation specified the following mkisofs syntax:
mkisofs -o output.iso -b bscript/loader.bin -no-emul-boot -boot-load-size 4 root-of-iso-tree
Result: So I retried by rebuilding the bscript completely. No luck... grr! I then tried to go back to basics and create a new ISO with just the bscript loader. This worked AOK, so tried to recreate the CD from scrach.
This process was incredibly frustrat
ing. Something was happening structure-wise as boot would not occur when there were files in the root folder. Boot would also not occur when I moved all files to data directory, but when I renamed that directory to temp, it worked! What on earth? Then discovered that previously booting .img files were not, so decided to burn current work to CD and attempted real time boot (not VirtualBox boot as had previously been using).
Result: Booted OK! Rebooted back in to OS and attempted again, with success also. What happened? Did restart clear some kind of VM cache? Had VB just become too confused?
Whatever the reason, I then recreated file structure to include all data.
Result: New ISO appeared to be working for the mostpart; .img files seemed to be working, but no OS installs would begin with message NTLDR missing.
Much investigation later, and an attempt to put NTLDR in all manner of places, I was still no closer, I even fully investigated how the boot loader was calling the DAT files (and how they subsequently booted) and the editing process required to achieve that.
Still no closer. At this point I decided that there was a fundamental issue with the DVD filestructure (come on... what else could it be?!) and decided to use the guide at MSFN (http://flyakite.msfn.org/) to recreate the DVD from scratch. To do this I had to actually use the Windows interface to obtain the ISO files and then work from there. At the same time I also added in W2000 Professional, so there was some silver lining after all. The process of adding the structures in to the AIO DVD and hex editing the files was certainly a learning curve!
Slipstreaming W2000Pro
While slipstreaming SP4 with W2000 Professional, I decided to refine my mkisofs syntax. I used the following to create the ISO:
mkisofs -o w2000p.iso -r -R -l -J -iso-level 2 -hide-rr-moved -V W2PFPP_EN -no-emul-boot -b /home/lukeoconnell/Desktop/boot.bin -boot-load-size 4 /media/disk/windows2000/
- -V: Being the only new parameter. This simply sets the volume label, which is E2PFPP_EN for Windows 2000 Professional (English).
Result: Hmm... CDBOOT: Couldn't find NTLDR. How strange, seems familiar. Will have to revisit this, but will include on AIO DVD.
Mounting .vdi Images
After spending some time recreating the AIO DVD in my VirtualBox WXP installation, I was left with an ISO file of many gigabytes and source files of even more gigabytes... I needed to find a way of mounting my virtual hard disk image. After some searching I came across this post (http://forums.virtualbox.org/viewtopic.php?t=52) which suggested that an NTFS or FAT32 .vdi file could be mounted with the following command:
mount -o loop,offset=x,umask=000 virtualdiskimage.vdi /mountpoint/
0000073728 ëR.NTFS ..........ø..?.ÿ.?............å?.............Yþ......
I understand from further reading that this would read FAT32 for FAT32 partitions. I've also heard that you can set the volume label to enable easier location of this line, however I'm guessing this only applies to FAT32 as I had a volume label set with this NTFS partition.
After creating an empty directory in /media I then simply ran:
mount -o loop,offset=73728,umask=000 virtualdiskimage.vdi /media/vdifs/
I must admit that it was one of those moments that I just didn't expect to work, and when it did was a little overwhelmed! It mounted flawlessly in the mount point and was ready for access. If nothing else, this exercise has taught be a good deal already.
Investigation Continues
So I now had access to both the newly created AIO ISO file (created with CDIMAGE.EXE) and the source files. The advantage to CDIMAGE.EXE is that it reuses data when creating ISO files, almost like the Linux process of creating softlinks except that this process happens transparently within the ISO file. Very interesting. I quickly saw the difference after I had created my own ISO using mkisofs. The CDIMAGE.EXE ISO was 1.7GB while the mkisofs ISO totaled 3.0GB. Ouch! I'm wondering if there is a way to have mkisofs do the same?
Regardless, I now tested both DVDs with the obvious thinking that they would both be identical in operation. Good news was that the CDIMAGE.EXE ISO worked AOK. Bad news was that the mkisofs ISO file generated the infamous CDBOOT: Couldn't find NTLDR message. What the heck? Maybe I am getting closer to the holy grail of answers here. I started by ensuring I had followed the guide at MSFN to the letter, and even compared the hex data from both sources (see left). After confirming that I had probably followed the guide properly, I decided to look back to the CDIMAGE.EXE command: start cdimage.exe -lAIODVD -t08/23/2001,09:00:00 -b\AIO-DVD\BOOT\loader.bin -h -n -o -m \AIO-DVD C:\AIODVD.iso
Now CDIMAGE.EXE is a Microsoft internal application, so documentation is limited. But by running without any switches it revealed a list of all switches.
- -lLABEL: As we would have guessed, the volume label.
- -tTIME: The timestamp to be applied to every file/folder in the ISO.
- -bPATH: Path to the boot loader.
- -h: Includes hidden files and directories.
- -n: Allows for long filenames (longer than 8.3).
- -o: Optimizes storage by encoding duplicate files only once (thus the reduced file sizes).
- -m: Ignores maximum image size of 681,984,000 bytes (650MB).
The only really interesting switch here is the timestamp. Why would it matter specifically when it was encoded? Maybe this is causing NTLDR to fail? Also, as a side thought, is the -n switch similar to the -l mkisofs switch? Or maybe it is more similar to the standard breaking Joliet switch in mkisofs? At this time however I am going to stay on track and try to find a way to master a timestamped ISO in mkisofs.
Touch!
After some digging it seems like the touch command can be used (I hope recursively) to change file timestampt. I will therefore try to set the CDIMAGE.EXE reference 08/23/2001,09:00:00:
touch -t 200108230900 *
Only downside is that I have to copy all files from vdifs as vdifs does not support writing to (NTFS). Although I wonder if it would be writable if it were FAT32? Something for future follow up I guess!
Due to the fact that touch has no recursive option, I actually ended up running the following. I could only traverse 5 deep though before touch complained with Argument list too long:
touch -t 200108230900 */*/*/*/*
Result: No luck... hmm...
Just Relax
My final try will be to use the -D option of mkisofs. The -D option disables deep directory relocation and leaves as-is. Although this violates ISO9660 specification, apparently it "happens to work on most systems".
mkisofs -o aiodvd.iso -r -R -l -J -iso-level 2 -D -V AIODVD -no-emul-boot -b boot/loader.bin -boot-load-size 4 /home/lukeoconnell/aio-dvd/
Result: Still no go. Just one more attempt, this time I'll relax to ISO level 3 and include -joliet-long:
mkisofs -o aiodvd.iso -r -R -l -J -joliet-long -iso-level 3 -D -V AIODVD -no-emul-boot -b boot/loader.bin -boot-load-size 4 /home/lukeoconnell/aio-dvd/
Result: Nope.
After searching the Internet (thank Google) I came across the following article (http://www.g-loaded.eu/2007/04/25/how-to-create-a-windows-bootable-cd-with-mkisofs/). Who would have guessed that someone was here before. In this guide the author George Notaras suggested the following command:
mkisofs -b cdboot/msboot.img -no-emul-boot -boot-load-seg 1984 -boot-load-size 4 -iso-level 2 -J -l -D -N -joliet-long -relaxed-filenames -V "WINSP" -o winsp.iso /pathspec/
Having very little faith at this point it seemed far from plausible. The only missing switches were:
- -N: Which omits the version numbers from ISO9660 file names. Breaking the ISO9660 standard, but apparently the versions numbers are no longer used.
- -relaxed-filenames: Which allows ISO9660 filenames to include digits, upper case chars and other 7 bit ASCII characters (except lower case characters). Again violates the ISO9660 standard.
Another difference is the -boot-load-seg address of 1984.
mkisofs -b boot/loader.bin -no-emul-boot -boot-load-seg 1984 -boot-load-size 4 -iso-level 2 -J -l -D -N -joliet-long -relaxed-filenames -V "AIODVD" -o aiodvd.iso /home/lukeoconnell/aio-dvd/
Result: Phew! Boot AOK! The holy grail has been answered. The case was a mkisofs switch all along. Frustratingly, this could have caused the issues from the get go, as some of my very early attempts with BootScriptor resulted in this, as did my W2000 slipstreaming.
I'm wondering which was the exact answer however, so will omit the -boot-load-seg address... as I believe this to be the most likely solution.
mkisofs -b boot/loader.bin -no-emul-boot -boot-load-size 4 -iso-level 2 -J -l -D -N -joliet-long -relaxed-filenames -V "AIODVD" -o aiodvd.iso /home/lukeoconnell/aio-dvd/
Result: Pass! So it has to be in the -relaxed-filenames switch. Incredible. Maybe (should have seen this coming), the fact that NTLDR has no extension prohibits its proper existence without the switch.
I can now rest easy. I've found the answer. I think that I'll use George's suggestion without the -boot-load-seg switch. I'm sure that other combinations would work, and that the ISO does not have to be that relaxed, but it works.
mkisofs -b boot/loader.bin -no-emul-boot -boot-load-size 4 -iso-level 2 -J -l -D -N -joliet-long -relaxed-filenames -V "AIODVD" -o aiodvd.iso /home/lukeoconnell/aio-dvd/
Windows XP x64
Slightly more challenging than the other installs. The guide as MSFN did not include an x64 section. Searching the web lead me back to MSFN with this post (http://www.msfn.org/board/Automatically-create-multi-boot-folders-files-t58446.html). Problems included the fact that there were two data directories (i386 and AMD64) and that the setupldr.bin had a checksum which prevented editing (see http://www.msfn.org/board/Solution-multibooting-Win-XP-2k3-64-Bit-Win2k3-SP1-WinPE-t58410.html for checksum solution). Also the install could not be started within Windows as others could, so boot folder generation was hampered. Overall solution was a script file which not only created all data directories for me, but also edited the setupldr.bin file accordingly. Phew!
Conclusion
Wow, what a marathon. The business of computing... but specifically Linux, takes you through some wild and wonderful turns. From making images to exploring boot processes, today has been an eye opener. I've learned about:
- mkisofs: Inside and out; its interface and various switches. Its limitations.
- CDIMAGE.EXE: Its use in duplicate file optimization and ultimately, the only way I currently know how to create an AIO DVD.
- Bootable media: A little bit more knowledge on how bootable media, specifically CDFS media boots.
- Boot scripts: Like BootScriptor and CDShell, and their varying uses.
- Mounting VDI: Something which will definitely prove invaluable in the future.
- Making HEX Adjustments: Linux HEX editors such as KHexEdit and BVI and Windows variants.
My next challenge will be to try and port this AIO DVD to a USB device... but that really is another story...
0 comments:
Post a Comment