Can't start X in regular AB installation on a USB drive.

SpawnHappyJake · 2012-06-09 19:05:21

I have ArchBang 64-bit installed to a thumb drive. (Yes, actually installed) I get the great idea of updating the kernel. So I do. It suggests that I update pacman as well, so I do.

First I forget to add "usb" to the HOOKS in mkinitcpio.conf, so I couldn't boot back in. I copy over a working kernel and a working initrd. Surprisingly, I failed at that simple task a few times (wrong kernel version/ having errors when making the initrd/ maybe some Live installer issues of not actually writing to the drive). But I got the right ones on now.

I get the "Id: 'x' is respawning too fast" message, and could only get to a commandline. No desktop. At first I was even more uneducated about this error than I am now, and actually thought that by "x" it meant Xorg, so I updated all my xorg packages. Ha! Now I know it means "x" in the algebraic sense where some program with an id number, an unknown id number, hence the "x", is respawning too fast, perhaps Slim or something.

I am not a complete stranger to this problem. When ever I try to EFI boot ArchBang off my thumb drive on my MacBook, I get that message, so I have to boot into Fedora. Sure, installing the nvidia driver (MacBook uses NVIDIA) might fix that (?), but then I loose the mobility to be able to boot my thumb drive into an ArchBang desktop on any computer (such as ATI machines and Intel graphics machines). Fedora boots to a Desktop on my BIOS machines and my MacBook just fine.

Anyways, now I'm getting the error on my BIOS machine.

I have been researching for a solution. First I found some (I think wrong) information saying that it's caused by booting into runlevel 3 instead of 5. I found suggestions to modify /etc/inittab. Yet my working installation starts out booting into runlevel 3 (grub line) and winds up in 5, so I know its not anything wrong with /etc/inittab in that regard. Plus, /etc/inittab in my working installation has the same md5 hash as the one in my "non-working" installation. So again, nothing apparently wrong with /etc/inittab. Maybe it has to be edited to work with new stuff?

I found a thread talking about getting the error due to NVIDIA driver issues, but I was just trying to get vesa to work! Now I'm taking a shot at the intel driver.

I also found this: http://bbs.archbang.org/viewtopic.php?pid=791#p791

I try to follow Ircaballero's solution. Remember that I can only get to a commandline.

I suceed in doing these two steps:
#pacman -R xf86-video-vesa
#pacman -S xf86-video-intel

I do this:
#nano /etc/mkinitcpio.conf and add...intel_agp i915 in the MODULES line...
#MODULES="intel_agp i915"

Note that's the MODULES line, not the HOOKS line. I messed up and tried to put it in HOOKS first, but I fixed that.

I know that editing /etc/mkinitcpio.conf doesn't do any good unless I build a new initrd with mkinitcpio, so I do that.

I get errors saying that the intel_agp and i915 modules are missing.

Ircaballero's instructions say to edit xorg.conf through the OpenBox menu. Well, I have no OpenBox menu because I'm in commandline. I take a guess that I'm supposed to edit /etc/X11/xorg.conf.d/20-gpudriver.conf . I change that from "vesa" to intel. Going into a working installation of ArchBang, that file doesn't even exist in that installation (odd). Also, I don't see any xorg.conf editing option in the OpenBox menu whilst booted into the working installation.

I try booting with the new (the one I just made) and the old initrd without the "nomodeset" parameter. Still didn't get me to a desktop.

I have tried updating the kernel again to see if that would smooth out any missing modules or whatever.

I build the initrd again, and I get a bunch of errors. It says it makes it succesfully, but I don't have a warm fuzzy, because it still doesn't boot to a desktop after using that initrd.

mkinitcpio gives me a bunch of "WARNING: Module acpi:PNP...: not found."s. Also complains of some pci modules not being found. Also, could not open /tmp/mkinitcpio/HmnHjp/root/lib/modules/3.3.7-1-ARCH/modules.order: No such file or directory. Same with .../builtin: No such file or directory.

I don't see any complaints about the intel_agp or i915 module missing this time durning the making of the initrd, even though they are still in /etc/mkinitcpio.conf.

In trying different initrds, the one that I supposedly put intel_agp and i915 into gets me the error:

[ 0.281260] [drm:i915_init] *ERROR* drm/i915 can't work without intel_agp module!

ERROR: Unable to determine major/minor number of root device.....
You are being dropped to a recovery shell.

But with some fallback initrd, I at least can get to a normal terminal.

I saw what I thought was a sticky, but was actually just a post right under a sticky, and got here: http://bbs.archbang.org/viewtopic.php?id=2889 (see 7th post). I tried deleting /etc/X11/xorg.conf.d/20-gpu-driver.conf. Still no success. I move on to the 8th post.

Don't know how to download an image and intall it, if it's talking about kernel images. Then it talks about doing a system upgrade. Can't do that because I need libgl 7.11. So I try to install libgl. Can't do that because to install libgl I need libgl 7.11. So I search for packages with the string "libgl". I install "libgles' and "libglapi". Still can't do a system upgrade. Still says I need libgl=7.11. I try installing curl. Nope. Still can't do a system upgrade.

If I install ArchBang over my existing installation without formatting, will it keep my programs I installed and all my stuff?

Any help would be rather appreciated.

I know that I could just reinstall ArchBang, but it would be nice to save it (unless I can install over it without getting rid of my programs, then I'll just do that). One reason why is so that I'll know how to fix this issue in the future. I plan on installing ArchBang to my main machine as the only OS, and would need to know how to fix this should it happen on that. But I'll have plenty of bootable USB stuff in case I can't.

Also, I'm hopeful that once I know how to fix it, I can get it to EFI boot to a Desktop on the MacBook. Then it would be as versitle as Fedora in that regard.

I have been at this for days.

Please help me,
Jake

P.S. If this "Id: 'x" is respawning too fast...disabling for 5 minutes" problem ever gets solved for me, this really, _really_ needs a sticky somewhere. Somewhere obvious. Maybe even dedicate a whole super-thread for it under "Index". Lots of people seem to be getting it, and apparently, there is no one fix (other than reinstalling).

Last edited by SpawnHappyJake (2012-06-12 00:49:30)

Ugo2 · 2012-06-09 19:46:54

I had the exact smae problem after the last updates and unfortunatly every post i found had different or half solutions.
What i did to fix this was to install nvidia drivers and i could bot into DE and it went away but came back!!
I then edited /etc/inittab and changed x:5:respawn:/usr/bin/slim >/dev/null 2>&1
to x:5:once:/usr/bin/slim >/dev/null 2>&1

that fixed it for me

oliver · 2012-06-09 19:53:20

SpawnHappyJake - you have an incredibly esoteric setup so it's hard for someone to re-create your exact problems but I I would be working along the same lines as Ugo2.

Since you're booting off a USB device, do you even need slim installed? Seems like extra overhead when the goal should be to limit file-system writes as much as possible

SpawnHappyJake · 2012-06-09 23:22:27

Thanks for the care and efforts folks! I tried replacing "respawn" with "once" as Ugo2 recommended.

I still can't get to a Desktop. Instead of Getting the "Id 'x' is respawning too fast...disabling for 5 minutes", I get right to a commandline.

I do "startx" and I get a fatal error:

[ 19.014] (EE) Screen(s) found, but none have a usable configuration.
[ 19.014]
Fatal server error:
[ 19.014] no screens found

Any ideas? Do I "just" (hehe - "just") go in and edit/make some conf file(s) for my screen?

Also, since I uninstalled vesa, it's tring to use fbdev instead. Xorg's log file shows it trying to load vesa, failing (because it doesn't exist), and then resorting to fbdev. So apparently I haven't thouroghly gotten all the references to vesa removed.

It's funny, because on my external hard drive I installed ArchBang, updated the kernel and pacman, and it didn't have any of these issues.

About Slim - I didn't know it increased writes to the drive. I thought it was just a GUI thing.

Jake

Mr Green · 2012-06-09 23:59:59

https://bbs.archlinux.org/viewtopic.php?id=132172

Post your xorg config?

SpawnHappyJake · 2012-06-10 00:37:57

Thanks for the effort Mr. Green, but I have no /etc/X11/xorg.conf file. In /etc/X11/xorg.conf.d I had a 20-gpudriver.conf. I heard that deleting it could help, so I did. Well, it didn't help. I put it back. This is what it has:

Section "Device"
Identifier "Card0"
Driver "intel"
EndSection

I tried with "vesa" instead of "intel" as well.

And I noticed something hyper-weird: my ~/.xinitrc had the DE as xfce! What the heck? _I_ did _not_ set that! I put it back to "openbox" as it is in my working installation. (as far as I know I don't even have xfce installed)

Even more funny, it still had the line "exec ck-launch-session dbus-launch openbox-session", even though it didn't have "openbox" as the DE anymore. (Why would it try to start an openbox-session without openbox?)

It looks like it can't detect any screens. Right now I'm using QEMU hardware (-vga std) so that I can be booted into the thumb drive and working on it and post stuff here at the same time. As well as hand-copy errors it throws to the screen (you didn't think my memory was _that_ good did you?).

But the hardware _shouldn't_ be an issue because I got the problem on my real hardware as well.

Stumped,
Jake

Mr Green · 2012-06-10 02:26:37

Does X work if you run a normal livecd?, is your system fully up to date?

intel driver can be a problem but there is plenty of information in wiki

https://wiki.archlinux.org/index.php/Intel

oliver · 2012-06-10 06:04:28

SpawnHappyJake wrote:

And I noticed something hyper-weird: my ~/.xinitrc had the DE as xfce! What the heck? _I_ did _not_ set that! I put it back to "openbox" as it is in my working installation. (as far as I know I don't even have xfce installed)
Even more funny, it still had the line "exec ck-launch-session dbus-launch openbox-session", even though it didn't have "openbox" as the DE anymore. (Why would it try to start an openbox-session without openbox?)

~/.xinitrc is not dynamic - those entries are there because thats how they're set up by (ArchBang) default. Removing or adding WMs/DEs will not change it.

I always assumed that the DE entry is because some apps need *something* to be set and since Openbox is a window manager rather than desktop environment it was the best fit

Mr Green · 2012-06-10 06:38:54

More recent isos .xinitrc was changed, but it should not stop X working

oliver · 2012-06-10 08:18:03

Mr Green wrote:

More recent isos .xinitrc was changed, but it should not stop X working

Oops - thats the problem with using an older ISO and a rolling release :-)

SpawnHappyJake · 2012-06-10 09:04:18

I can't do a system-wide upgrade because it says I need "libgl=7.11". So no, I suppose it's not up-to-date. Though the kernel is 3.3-7-ARCH, I believe (going off memory, away from machine).

I can boot the Live CD into QEMU as well as my real hardware just fine.

I know several things have to be wrong with my "broke" installation.

It complains about i915 being missing (even though I attempted to add it to the initrd).

It can't find any screens when I do "startx".

And I get lots of complaints from mkinitcpio when I try to build an initrd.

Jake

SpawnHappyJake · 2012-06-10 09:08:45

Wait a minute - that initrd might _not_ have i915. I was using a fallback. I did try to add intel_agp and i915 to an initrd, but that one only gets me to a recovery shell. So I'm not using that one anymore.

Mr Green · 2012-06-10 09:25:59

If you are going to run a live usb that will work on a number of different machines it may be worth using vesa instead of intel.

Would love to have a go at this on my system see if I can recreate your problem but I need clear steps, not just your installed it to usb

SpawnHappyJake · 2012-06-10 23:54:01

Thanks again Mr. Green! As far as telling you step-by-step how to recreate my situation, I think I can do better than that.

How about I just image the partition (only 5 GiB, not all of it's used. Maybe 3.5 GBs used?) with Clonezilla (and it will be compressed at that point), upload the compressed partition-only image to the public folder of my Dropbox, then post the link here?

Though at the moment I just got some pretty bad filesystem errors on the partition (something to do with the issue?). I'll try and fsck them out and then I'll image.

Then you can create a hard drive image and blow the partition-only image onto one of the hard drive image's partitions, and boot the hard drive image you just made in qemu. Of course you'll have to insall a bootloader, but that's easy. I'll post the grub2 entry after I upload it.

Let me know if you need any help with Qemu. Basically:

//Insert kvm modules (adjust if you have amd)
sudo modprobe kvm
sudo modprobe kvm-intel

//supposed to give you access to kvm, if not, run qemu as root, or use your magic ArchBang moderator powers to outsmart it:
sudo usermod -a -G kvm [your user]

//Run it! (giveing name of 64-bit binary, adjust if 32-bit)
[possibly need a sudo] qemu-system-x86_64 --enable-kvm -vga std -hda [path to hard drive image (not parition-only image)] -m [insert a decent amount of RAM here in MB] -smp [insert how many cores you're willing to share with the VM. You can share all of them and be ok]

And possibly tack on "--boot menu=on" if you need to choose what you want to boot into, such as if you are booting into a Clonezilla iso to blow on the image rather than just decompressing and using dd.

Thanks again,
Jake

SpawnHappyJake · 2012-06-11 00:00:00

[shjake@archbang ~]$ sudo fsck -f -c /dev/sdc4
fsck from util-linux 2.21.1
e2fsck 1.42.2 (27-Mar-2012)
Superblock last write time is in the future.
(by less than a day, probably due to the hardware clock being incorrectly set). Fix<y>? yes
Checking for bad blocks (read-only test): 23.70% done, 0:35 elapsed. (0/0/0 errors)

Accidentally killed it when trying to copy it a couple times. Man I must be hard on filesystems. Don't think I'll have them mounted in my "real" OS and in QEMU at the same time anymore. Shh. Don't tell anyone I do that.

Jake

Mr Green · 2012-06-11 00:15:31

Moving to Arch Discussions do not want anyone to get confused with Installation problems relating to a normal install.

SpawnHappyJake · 2012-06-11 00:32:06

If you are going to run a live usb...

Moving to Arch Discussions do not want anyone to get confused with Installation problems relating to a normal install.

Sorry, I forgot to re-clarify. This is not a Live Install. This is ArchBang actually installed to the USB drive, just like how one would install to an internal hard drive. (See parenthetical statement after first sentence of first post).

Jake

SpawnHappyJake · 2012-06-11 00:46:52

Ok. Got past scary filesystem errors problems. I ran fsck a few times just to make sure. Said "Updating bad block inode." twice. Figured it'd just keep saying that, so I went ahead and booted it. It booted into a terminal login again - no filesystem complaints. Phew!

Here's some stuff from termial when I was running fsck:

[cropped a bunch of stuff out]
[...]
Fix<y>? yes
Free inodes count wrong for group #34 (2656, counted=2645).
Fix<y>? yes
Free inodes count wrong for group #36 (5248, counted=5247).
Fix<y>? yes
Free inodes count wrong (185744, counted=185740).
Fix<y>? yes

ArchBang64: ***** FILE SYSTEM WAS MODIFIED *****
ArchBang64: 141940/327680 files (1.2% non-contiguous), 1278933/1310720 blocks
[shjake@archbang ~]$ sudo fsck -h
fsck from util-linux 2.21.1
fsck.ext4: invalid option -- 'h'
Usage: fsck.ext4 [-panyrcdfvtDFV] [-b superblock] [-B blocksize]
[-I inode_buffer_blocks] [-P process_inode_size]
[-l|-L bad_blocks_file] [-C fd] [-j external_journal]
[-E extended-options] device

Emergency help:
-p Automatic repair (no questions)
-n Make no changes to the filesystem
-y Assume "yes" to all questions
-c Check for bad blocks and add them to the badblock list
-f Force checking even if filesystem is marked clean
-v Be verbose
-b superblock Use alternative superblock
-B blocksize Force blocksize when looking for superblock
-j external_journal Set location of the external journal
-l bad_blocks_file Add to badblocks list
-L bad_blocks_file Set badblocks list
[shjake@archbang ~]$ sudo fsck -f -c -y /dev/sdc4
fsck from util-linux 2.21.1
e2fsck 1.42.2 (27-Mar-2012)
Checking for bad blocks (read-only test): 4.67% done, 0:07 elapsed. (0/0/0 errdone
ArchBang64: Updating bad block inode.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

ArchBang64: ***** FILE SYSTEM WAS MODIFIED *****
ArchBang64: 141940/327680 files (1.2% non-contiguous), 1278933/1310720 blocks
[shjake@archbang ~]$ sudo fsck -f -c -y /dev/sdc4
fsck from util-linux 2.21.1
e2fsck 1.42.2 (27-Mar-2012)
Checking for bad blocks (read-only test): 0.00% done, 0:00 elapsed. (0/0/0 errdone
ArchBang64: Updating bad block inode.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

ArchBang64: ***** FILE SYSTEM WAS MODIFIED *****
ArchBang64: 141940/327680 files (1.2% non-contiguous), 1278933/1310720 blocks

Cheers,
Jake

SpawnHappyJake · 2012-06-11 14:27:55

Ok. It's been uploading for several hours now. I have to say that that was an ordeal in and of itself.

I go to Dropbox.com, login, and try to upload the file through the web interface. I can't because the web interface has a 300MB file size limit. But it says that if I install the Dropbox app (which I normally have installed), I can upload it though the app.

There's no arch package to download for Dropbox, so I have to compile from source. I try to compile it, and it says that it can't find "pygtk". I install pygtk. Still can't find pygtk (and the dev headers for it are supposed to be included because this is Arch).

I find a thread about it on the Dropbox forums. I try the posted fix of making a link to the python binary like they say. Still can't find pygtk. I try installing stuff they're saying to install. Still can't find pygtk.

So I resort to the AUR. makepkbuild can't resolve all dependencies.

So finally, I pop the thumb drive out, stick it in my MacBook booted into Mac OS X with the Mac OS X version of Dropbox installed and start uploading it with that. Sheesh.

Well, at least it's uploading now.

I'm hoping it will be uploaded by the end of the day, but it might take as much as 31 more hours.

Cheers,
Jake

oliver · 2012-06-11 14:55:06

SpawnHappyJake wrote:

So I resort to the AUR. makepkbuild can't resolve all dependencies.

Make it easy on yourself:

$ packer -S dropbox

It will take care of all the dependencies

SpawnHappyJake · 2012-06-11 19:15:27

I did "sudo packer -S dropbox". That didn't install dropbox. All it did was print to the screen in terminal what looks like the contents of a PKGBUILD file.

I copy and pasted it out of terminal to a file and ran "makepkg [path to file]" and got "==> ERROR: PKGBUILD does not exist."

Any other ideas?

BTW, it's still uploading.

Cheers,
Jake

SpawnHappyJake · 2012-06-11 19:53:30

I'm sorry to say this, but ArchBang has been nothing but problem after problem after problem. I'm trying to switch to it, and everything I try to do has to be troubleshooted.

First, install wouldn't work because I renamed the archisobasedir. That's somewhat excusable, but caused a lot of frustrations.

Already frustrated, I try to do a few updates. X breaks. In the working installation: Printer problem. Can't install Dopbox. I have sound and then it goes away. And today, I got "Failed to execute login command" when I right-click the Desktop (to get to exit) and Ctrl+Alt+F4 doesn't get me to a pure terminal ("pure" as in no GUI - not in a window).

I've been reading wikis. I've been working on my move to ArchBang for over a week. I'm hoping that I'll eventually have all the problems I'll ever have and know how to work past all of them. But there's no end in sight. I think this rolling release distro has square wheels.

That said, I would love to use it if I could. It's fast, and it has more up-to-date programs available from the remote repositories than I had in Mint. I like the OpenBox GUI. I like the fluff being gone (for the most part).

My question is, do you folks have this many problems with ArchBang?

mike4ca · 2012-06-11 20:24:40

SpawnHappyJake wrote:

My question is, do you folks have this many problems with ArchBang?

I'm sorry to say, no. Sometimes you just need to go with what works so if AB is not working for you it may be time to look around more. If you like Openbox, have you tried Crunchbang?

SpawnHappyJake · 2012-06-12 00:12:37

Well I haven't given up yet. I've looked at CrunchBang, but I was actually thinking about Fedora if AB didn't work out.

Again, I haven't given up yet. My Dropbox just ran out of space, so I'll have to think of something. Maybe I can clear some space.

Cheers,
Jake

SpawnHappyJake · 2012-06-12 00:43:57

Ok. I renamed by Dropbox folder (on my Mac on the Mac OS X partition) to "Dropbox_original", so all my Dropbox stuff is still locally on that hard drive.

I made a new Dropbox folder and put the image in there. Now it ought to fit. I of course had to start the upload all over again, so it will be a while.

When this is done, I'll just rename "Dropbox_original" back to "Dropbox", and I'll be back.

Cheers,
Jake

oliver · 2012-06-12 07:13:40

SpawnHappyJake wrote:

I did "sudo packer -S dropbox". That didn't install dropbox. All it did was print to the screen in terminal what looks like the contents of a PKGBUILD file.
I copy and pasted it out of terminal to a file and ran "makepkg [path to file]" and got "==> ERROR: PKGBUILD does not exist."
Any other ideas?
BTW, it's still uploading.
Cheers,
Jake

You saw the contents of the PKGBUILD file because you either hit <return> (to take the default option) or hit "Y" when asked the question "Do you want to edit the PKGBUILD?"

Once you finish that step (or just say 'no' when asked) it goes out and downloads the source, compiles and installs. If any dependencies are also in AUR you'll be asked the same questions for those too

If you're *sure* you will always want to say "no" to editing, there is a flag for that.

SpawnHappyJake · 2012-06-12 14:09:05

Thanks Oliver! That worked! Yey! As Maxwell Smart would say, I "missed it by that much". But you got me on the right path. I must have been over-eger and over-hitting the enter button...every single attempt.

Ok, it uploaded...working on next post in leafpad....

SpawnHappyJake · 2012-06-12 20:35:09

*Note, the following requires a 64-bit processor - unless you want to emulate one.
*Also, this assumes you are following along in a 64-bit Arch-based distro of Linux with a functioning GUI.

Ok. Miracuously, my Internet connection decided to stop being slow! It's uploaded in its entirety now!

Go here: https://www.dropbox.com/sh/5ftlky0kdap0c6h/D0aVc1-gOB

Following the link you will be brought to a web page resembling a file browser. You will be in my Public folder.

In the Public folder is a folder named "helpjake-img". "helpjake-img" is a Clonezilla partition-only image. Yes, Clonezilla images are folders, not files. The folder contains several files, but really all you need is the sda4.ext3-ptcl-img.bz2.aa file. That's the actual image file (the meat and potatoes of the whole thing).

Download at least the actual image file. Possibly even make a folder on your hard drive named "helpjake-img" and download _all_ the files into that folder. Then you will have the Clonezilla image.

Once you do that, open a terminal and do "openssl md5 [path to actual image file archive]"

After a while it should spit out the md5 of the image. It should be:
757a4e53bb2313d078574459327c24fe

If it's not that, something went wrong. (An md5 hash is a funtion of the contents of the file and can be used to tell if it got corrupted by seeing if the before and after hashes match. No, I'm not saying that for you moderators, I know you already know that - just trying to make my post half-way n00b friendly for Googlers.)

The MD5SUMS file in the helpjake-img folder contains the md5 for every file in the image except for MD5SUMS.
If you're feeling OCD, I'm not stopping you from checking other files.

You're going to need a few tools.

Optional: Download a Clonezilla iso
Highly recommended: gdisk (can use fdisk, but I prefer gdisk)
qemu
kpartx
autogen - needed for Grub2
python2 - needed for Grub2
bzr - needed for Grub2
(and obviously we'll be getting Grub2)

Downloading the Clonezilla iso is pretty straight-forward. Just Google "Clonezilla", go to their website, click "Downloads", and you can figure it out from there. I usually go for the latest testing iso, because, as you can tell, I kinda live on the edge.

Here's a link to the testing download links: http://clonezilla.org/downloads/testing … -files.php

Again, you can find the md5 hash of the iso. The hashes are also called checksums. You can find what the checksum of the download you choose is supposed to be by going here (for "testing" versions): http://clonezilla.org/downloads/testing/checksums.php or here: http://clonezilla.org/downloads.php and clicking "checksums" under "extra info", for your download.

Clonezilla-amd64 is for 64-bit processors (even if they are Intel processors - don't let the name fool you)
Clonezilla-i686 is for use with more than one 32-bit CPU core
Clonezilla-i486 is for use with only one 32-bit CPU core

The zip files are for if you want to make a thumb drive or something bootable into Clonezilla, and the iso is a optical disc image you can burn to an optical disc. With Qemu, you can boot the iso as though it were an optical disc in a drive without actually burning a disc. So we'll use the iso, if you choose to use Clonezilla.

To obtain gdisk in Arch, simply do "sudo pacman -S gdisk"

To obtain qemu in Arch, simply do "sudo pacman -S qemu"

To obtain kpartx in Arch, simply do "sudo pacman -S multipath-tools" - ha! Got you that time!

To obtain autogen in Arch, do "sudo pacman -S autogen"

To obtain python2 in Arch, do "sudo pacman -S python2"

To obtain bzr in Arch, do "sudo pacman -S bzr"

Once you have bzr, we will use that to download the Grub2 source:
open a terminal. Where ever it is cd'ed to, that's where bzr will download the "grub" folder to.
So make sure you're cd'ed somewhere that doesn't already have a folder named "grub", or else it will override it.

Run this to download the grub source code to wherever the terminal is cd'ed:
bzr branch http://bzr.savannah.gnu.org/r/grub/trunk/grub

Now make a copy of the grub folder where the original is named "grub_fresh_from_bzr" and the copy is named "grub_BIOS".

This is done by making a new folder next to the grub folder, copying the grub folder into the new folder, renaming the grub folder to "grub_fresh_from_bzr", renaming the grub folder copy to grub_BIOS, then cutting and pasting the grub_BIOS folder out of the new folder next to the grub_fresh_from_bzr folder, then deleting the "New" folder.

You have to do it like that because you can't have two folders with the same name in the same directory at the same time.

Leave the fresh one untouched, so that you can always go back to it.

We named the other one "grub_BIOS" because we will compile Grub for BIOS in there (as opposed to compiling it for EFI).

Cd a terminal to grub_BIOS.

Do "sh ./autogen.sh"
Now do "./configure --help" or "./configure -h"
Take a moment to marvel all the choices.
I do "./configure --disable-werror --enable-efiemu --enable-cache-stats" when I compile Grub for BIOS.

*Note "--disable-werror" is often the secret to compiling Grub. Often times if not specified, you won't get past make.

After running configure with the options you choose run "make".

Now, before running "sudo make install" let me let you know about this:

1. the command "sudo make install" will NOT, I repeat, NOT, install the Grub bootloader to your hard drive. It is merely installing the utilities to do so into Linux.

2. "sudo make install" merely copies binaries and stuff out of the folder you compiled them in to where programs are typically installed to (or where you specified) for use with Linux. So you might think that you can skip this step and run them out of the folder. Last time I tried that it didn't work because stuff wasn't where the utilities expected the stuff to be. So you may have to install them. At least that would be the path of least fuss. Perhaps the right configure option would let you run it out of the folder. Installing Grub2 stuff may override existing Grub-Legacy-installing utilities. I'm sure pacman or something can install Grub Legacy stuff back in afterwards if you insist on using Grub Legacy. But honestly, Grub2 is vastly superior and you may just want to keep the new one. If "sudo make install" overrid Grub Legacy stuff, and you put Grub Legacy stuff back, now Grub2 stuff that has been installed might get broke. That's ok. You can just cd back to the folder and run "sudo make install" again if you want it back.

Check to make sure Grub2 is installed by doing (in a _new_ terminal):
grub-install --version

It should say at least "grub 2.00~beta6"

The above command is very safe to do as non-root. That way if your fingers slip and you hit enter before you type the "--version" part, you don't have to worry about accidentally installing to your main hard drive.

Now burn off a raw image file with dd:

dd if=/dev/zero of=~/vitual_hdd bs=1048576 count=5150

What potentially needs to happen now is a change in philosophy. When it comes to hard drives, ditch the meteric system of kilobytes, megabytes, and gigabytes etc. And replace it with the "ibi" system. Don't use the meteric system for measuring partition size or for measuring overall hard drive size. Instead use kibibytes, mebibytes, gibibytes, etc.

The exact size of a megabyte or terabyte is subjective. The strictest of us use literal 1000's: a kilobyte is a 1000 bytes, a megabyte is 1000 kilobytes, a gi....etc. Some use 1024's: a kilobyte is 1024 bytes, a megabyte is 1024 kilobytes... Some even mix and match saying that a megabyte is (1000 * 1024) bytes.

Not only is that ambiguity removed with the "ibi" system, it works better with hard drives. A kibibyte is 1024 bytes, a mebibyte is 1024 kibibytes, and a gibibyte is 1024 mebibytes. None of it is up for interpretation. They are abbrivieated as KiB, MiB, GiB, TiB, etc.

Usually hard drives have 512-byte sectors (a sector being the smallest addressable unit). Thus a kibibyte (1024 bytes) is exactly 2 sectors. A whole number! You can't have a whole number of sectors make up exactly one 1000-byte kilobyte. So the ibi system works better with hard drives.

Also, it is common for hard drives to have 2048 sectors from one sector boundary to the next. 2048 / 2 = 1024, the number of the "ibi" system. Since 1 mebibyte is 1024 kibibytes, and each kibibyte is 2 (512-byte) sectors, 2048 (512-byte) sectors is exactly 1 mebibyte. So it works out again. You want partitions to begin and end right on these boundaries for performance reasons. Also for easier math.

So in the above dd command, 1048576 bytes is exactly one mebibyte (1048576 = 1024 * 1024). And it would be mighty convinient to have a raw hard drive image exactly 5150 mebibytes. Here's why:

1 MiB for the primary GPT (only needs 34 sectors, but giving it 1 MiB (2048 secotors) to round to 2048-sector boundaries. Remember, one mebibyte is exactly from one boundary to another)

5120 MiB for the partition where my messed-up installation of ArchBang 64-bit will go. 5120 MiB is exactly 5 GiB, a nice, round number that works. (5 * 1024 = 5120)

1 MiB spacer between the above partition and the next. These are good to have. It is exactly one boundary-to-boundary region.

1 MiB BIOS_BOOT partition. Should only need 62 sectors at a maximum, but we're giving it 2048 sectors (exactly one MiB) to align on these boundaries. Plus it gives more space should Grub get much bigger in the future.

26 MiB EFI system partition (ESP). Even though we won't use this, I'm saying to do this out of good practice. It's good to get in the habit of making one every time you carve up a drive. Why 26 MiB? Well, most importantly, it's a whole number of mebibytes. It quite comfortably can hold EFI Grub for both 32-bit and 64-bit. It has just historically been a good size for me. I didn't pick 26 to come out to the grand total of a nice 5150. It's what I've historically used.

1 MiB for the secondary GPT
_________________________________

Total that up: (1 + 5120 + 1 + 1 + 26 + 1) MiB = 5150 MiB

So we want to make a 5150 MiB image. That's 5150 1048576's. Thus the following command:
dd if=/dev/zero of=~/virtual_hdd bs=1048576 count=5150

bs is the block size and count says how many blocks to do.
"if" stands for "input file". Here we use /dev/zero, a file produced by the kernel that is an endless supply of zeros.
"of" stands for "output file". Here we will output the desired image file.

Now that we have the image file that will serve as the virtual hard disk, it's time to scheme it and partition it.

I recommend using gdisk over fdisk, for one, because you don't have to do as much math. We know how big we want each partition to be in mebibytes. You can just use the "+" syntax to say you want the partition to be so big and have such a distance from the previous partition rather than trying to figure out what sectors the start and end would be, as you have to do in fdisk (at least that's what I remember).

Another reason why I recommend gdisk over fdisk is because gdisk does GPT, and fdisk doesn't. I prefer GPT. For one, it lets you have unlimited partitions without having to deal with an extended partition. Not having to deal with an extended partition means a little less setup work, and it's more malleable when you go in and start removing/adding/growing/shrinking partitions.

And something I think is quite a benifit over MBR scheming is that GPT scheming allows for a BIOS bootloader to have its own dedicated partition. The MBR standard did not define any use for the space between the MBR and the first partition. This space was usually 62 secotors in size, because the first partition usually didn't start until the second track, and each track is usually 63 sectors in size, and the MBR was the first secotor, leaving 62 sectors of unclaimed space.

That means anything can rightfully write to that space. It wasn't reserved for anyone use. In other words, a program would not be in the wrong to write to there, because it was "owned" by nothing. A no-man's land. The second stage bootloader of Grub2 goes there on MBR-schemed disks. Although I've never seen it happen, techniqually some other program could legally try to use that space for something, overriding the bootloader.

But with the GPT standard, the bootloader is "protected" in that nothing is supposed to write to "its" parition. It's reserved for the bootloader. Might be kind of like making a law that it's illegal for a weed to grow in a garden in attempts to stop weeds, but at least it's in the standard.

Also, vital disk structures (partition table and such) are backed up. There's the primary GPT, and a backup copy, the secondary GPT. One's at the front of the disk, the other on the end. gdisk makes it really easy to restore the primary GPT using the secondary GPT and visa-versa, from the recovery and transformation options menu.

So sick gdisk on the image file:
gdisk ~/virtual_hdd

Now you are in gdisk. Notice you didn't need root priveliges because you own the file. You would need root privleges to work on an abstraction file going to a real drive.

Press "?" for help if you want to know what the options are.

Press "o" to make a new empty GPT partition table. This will scheme the image as GPT, as opposed to MBR.

Press "w" to write to the image file. Press "y" to say you're sure.

You're now kicked out of gdisk, back to a plain terminal prompt. Hit the up arrow key to get the last command (use gdisk on the image file), and press enter to get back in.

Now that the image is schemed, it is time to make partitions.

Press "n" to make a partition.

The default partition number to assign the first partition is "1", which is what we want, so go ahead and press enter.

We want the first partition to start on sector 2048, which is the default, so just press enter. We want it to start on 2048 to start on a "sector boundary".

We want it to be a total of 5120 MiB, so type in "+5120M".

We want it's partition type id to be 8300, code for "Linux filesystem", because that's what it will be, and that's the default, so just press enter.

Do "w" and then "y" to write this partition to the image. You'll be kicked out of gdisk. Hit the up arrow key, then enter to get back into gdisk.

Why am I having you write each step of the way instead of all at once? So that if you mess up you won't have to start all over. Something I learned the hard way. Passing the tip on. It's like hitting Ctrl+s every now and then in LibreOffice.

Once you're back in, hit "p" to see that your labor is actually doing something. It will print the partition, it's info, and above that, even some info about the drive in general, including the UUID (GUID) for the entire drive. GPT gives the drive as a whole a UUID. Once formatted, a partition will have its own UUID. So both the entire drive and individual partitions can have a UUID.

Hit "n" to make the next partition.

We want it to be partition #2, which will be the default, so hit enter.

Now pause a moment. Don't just hit enter. We want the second partition to start 1 MiB after the the previous partition ended to have a 1 MiB spacer. So type in "+1M" to tell it to start the partition 1 MiB after the previous.

Now type "+1M" in again and hit enter to tell it to make it a total size of 1 MiB. The last time we said "+1M" was because we wanted a 1 MiB spacer, now this "+1M" is to actually make a 1 MiB partition.

Specify to use a partition type id of "EF02" to mark the partition as a BIOS_BOOT partition.

Now do "w", "y", up arrow key, enter as before to write and go back into the program.

Press "p" again to see that there are 2 partitions.

Now press "n" to make a new partition. Yes, hit enter because we want it to be the 3rd partition, which will be the default.

Again, specify a 1 MiB spacing by saying "+1M" for the start of the partition.

Now, since this is the last partition, instead of having to do "+26M", you can just press enter, because the default is to go to the end (but not override the secondary GPT, which goes on the very end. The default will stop at the second-to-last "sector boundary".)

Specify a partition type of EF00 to mark the partition as an EFI system partition (ESP).

Do the w,y,up thing again.

Press "c" to change the name of a partition. Enter "1" to change the name of the first partition to "ArchBang64".

Likewise, go ahead and name partition 3 to "ESP" if you wish.

Do w,y,up thing. Hit p, notice the partitions are now named.

Time for an important distinction: The GPT gives each partition a name. If a partition is formatted, the filesystem of the partition also gives the partition a name (a label). So if someone says "the label of a partition" that's not enough info to know which he or she is talking about. It can give you a warm fuzzy if the two always match and can make you feel like you prepared a drive really well. But then there's reality.

Honestly, if you will never execute a program on a partition, never store a file larger than 4 gigs (2^32 bytes - another example of a gigabyte being subjective) on it, and never use any advanced filesystem stuff on it, and is on a thumb drive or external, it should probably be FAT32, unless it's too small, then it should be FAT16. That way it's compatible between OSes.

But if you format a partition as FAT32 or FAT16, those filesystems don't give you many characters to name the partition.

So often the name of my partition according the GPT will be fully spelled out/more descriptive, and the name of my partition according to the filesystem will be spelled with "liscence plate spelling"/less descriptive.

Ok, now we have a hard drive image that is schemed and partitioned. Now one of the parititons should be formatted.

To make the image available for use, do the following:

losetup -a

Look to see which loop abstraction files are used.

Now do:

ls /dev/loop*

Look to see what loop abstraction files you have to choose from. If there aren't any unused ones available (/dev/loop-control doesn't count, you need a /dev/loopX where "X" is a number), do the following:

"sudo nano /etc/modprobe.d/modprobe.conf" or "sudo nano /etc/modprobe.conf", whichever exists. If neither exists, I suppose you ought to make one (in Arch, the first one exists in my install, so probably make that one if you don't have it and you're running Arch.).

Add the following line:
options loop max_loop=n, where "n" is how many loop devices you want to have. All you need for this is 1 or 2 (depending on whether you take the Clonezilla route or not), but 8 is common.
Now restart the loop module:
sudo modprobe loop

Now do "ls /dev/loop*", and you should have some loopback devices available.

Now do:

"sudo losetup /dev/loopX ~/virtual_hdd", replacing "X" with the number of an unused loop abstraction file.

This links /dev/loopX to ~/virtual_hdd.

Notice you can do "sudo gdisk /dev/loopX" and hit "p" to show the partitions, just like when it was directly pointed at the image file, but now you need root privileges because it's using an abstraction file.

Now we want an abstraction file for each partition.

/dev/loopX is the abstraction file for the whole image file (whole "virtual hard drive"), but to get abstraction files for each partition do:

"sudo kpartx -a /dev/loopX"

Now do:
ls /dev/mapper

That will show abstraction files for each partition.

You should see:
loopXp1, loopXp2, and loopXp3.

Now you can do:
sudo mkdosfs -F 16 -n ESP /dev/mapper/loopXp3

to format the third partition as Fat16 with a label of "ESP".

Now to restore the partition-only image to the first partition of the hard drive image.

You have two choices (at least).

One, just extract the image out of the archive and dd it over yourself:
sudo dd if=[path to _extracted_ image, _not_ the archive image file downloaded from Dropbox] of=/dev/mapper/loopXp1
bs=[something, the bigger the number, the faster, but more likely to error. 512 is a common choice.]

If you want to Google for it, there is a way to pipe a decompressor into dd, allowing you to restore right out of the archive, rather than having to extract it first.

Two, use Clonezilla:
Make the helpjake-img folder, put all the files that go in it in it.
Make another raw hard drive image, scheme it, partition it, and format the partition; you will need to have Linux make another loopback abstraction file if you only told it to make 1.
Mount the partition you just made:
sudo mkdir /media/archive_holder
sudo mount -o users /dev/mapper/loopXpx /media/archive_holder
Copy the archive to /media/archive_holder
unmount the partition of the hard drive image:
sudo umount /dev/mapper/loopXpx
or:
sudo umount /media/archive_holder
Start qemu:
sudo modprobe kvm
sudo modprobe kvm-intel (or whatever for amd)
//Give yourself rights to kvm
sudo usermod -a -G kvm [your user]
//Actually start qemu:
[may need a sudo if you couldn't get rights to kvm] qemu-system-x86_64 --enable-kvm -vga std -m [insert how much RAM you're willing to share to it here] -smp [insert how many cores you're willing to share to it here, can share all] -hda [insert path to image we will be restoring to here] -hdb [insert path to image holding the helpjake-img folder here] -cdrom [insert path to Clonezilla iso here] --boot menu=on

Now chose to boot into the DVD drive.

Now use Clonezilla to restore the helpjake-img image to the first partition of the virtual hard drive we made.

After restoring, shutdown Qemu if that was used.

Now mount the partition we blew the image onto:

sudo mkdir /media/ArchBang64
sudo mount -o users /dev/mapper/loopXp1 /media/ArchBang64

Now it's time to install Grub:
sudo grub-install --boot-directory=/media/ArchBang64 /dev/loopX

That will tell it to install to /dev/loopX, the target virtual_hdd, and copy the files it needs to /media/ArchBang64, which is the first partition of that virtual_hdd. Since the abstraction files to refer to individual partitions were set up by kpartx, grub-install can query the kernel about where they come from and figure out that /dev/loopX is GPT-schemed.

Go to /media/ArchBang64/grub and make a file called "grub.cfg" and paste the following into it:

# BIOS Grub2 config file

set timeout=-1
search --no-floppy --fs-uuid --set=root 0609b258-e4b3-4741-b77d-862f781ca45d
set debug=video
insmod vbe
loadfont /grub/unicode.pf2
set gfxmode=3488x2228
set gfxmode=keep
insmod gfxterm
terminal_output gfxterm
terminal gfxterm
insmod gettext
set menu_color_normal=white/black
set menu_color_highlight=black/light-gray
insmod jpeg
background_image /grub/[picture of choice]

menuentry "ArchBang 64-Bit (actually installed)"{
insmod ext2
#[I like to put a comment here telling me which partition it's on, such as "This is on the 4th partition, BTW."]
search --no-floppy --fs-uuid --set=root 0609b258-e4b3-4741-b77d-862f781ca45d
linux /boot/vmlinuz-linux root=/dev/disk/by-uuid/0609b258-e4b3-4741-b77d-862f781ca45d loglevel=3 ro quiet
initrd /boot/initramfs-linux-fallback.img
}

Check that you put in the UUID that the partition actually has:
sudo blkid /dev/mapper/loopXp1

Is the UUID actually 0609b258-e4b3-4741-b77d-862f781ca45d? If not, make the appropriate adjustments above.

In my Public Dropbox folder there is a unicode.pf2 file. Download it and put it in the same directory as grub.cfg.

Also in my Public Dropbox folder are three pictures: butterfly.jpg, CH.jpg (a picture of a C&H Pure Cane Sugar building), and fish.jpg. CH.jpg looks pretty cool, but once you need to go into Grub command line, it's almost impossible to read what you type. I'm using butterfly right now, and it works pretty well. Fish looks like it will work quite well because of the big blue part.

Feel free to download any or all of those pictures, or use your own. Put the picture in the /media/ArchBang64/grub directory (along with grub.cfg and unicode.pf2). Edit the background_image line acordingly. If the image you choose is not a jpg, change "insmod jpeg" to the appropriate.

Now boot the virtual hard drive image we made:
sudo modprobe kvm
sudo modprobe kvm-intel (or whatever for amd)
//Give yourself access to kvm:
sudo usermod -a -G kvm [you user]
[may need a sudo if you couldn't get access to kvm] qemu-system-x86-64 --enable-kvm -vga std -m [insert how much memory you want to share to it here] -smp [insert how many cores you want to share to it here, can share all] -hda [path to the virtual hard drive image we have been laboring on]

Now it should boot into Grub. Selecet "ArchBang 64-Bit (actually installed)". Watch it not get to a GUI.

This bootable thumb drive is actually something I was working on for my mom, so the username is "mom". I made the password be "helpjake" both for root and for "mom".

There are multiple initrds to choose from in /boot. You can hit "e" at the grub menu and edit to change which one you want to use.

Now you're free to troubleshoot. Thank you very much.

Cheers,
Jake

SpawnHappyJake · 2012-06-14 22:50:01

Anyone start to try it yet? Or think about trying it?

For the record, "sudo packer -S dropbox" and "sudo packer -S vlc" installed those two programs, the printer issue can possibly be blamed on the printer because I couldn't print to it from Fedora either, and the "Failed to execute login command" I think may be due to bumping the data and/or power cord of my Western Digital external, from which I am booted. It has pretty poor quality connectors. It seems like every external hard drive has one issue or another. To the point I'm tempted to recommend buying an internal hard drive and an enclosure rather than buy an external hard drive, especially when I see this seemingly nice 3TB drive for $169.99: http://www.newegg.com/Product/Product.a … 6822148844

Cheers,
Jake

Mr Green · 2012-06-14 23:33:25

Not sure I could run a distro from usb drive on a daily basis, the read writes , slow transfer and weak connection would put me off

Depending on how much ram you have load up os into it and run from there, still not the safest route... but if speed is the essence.

I personally would not buy a spinning hard drive for os now as ssds are so much faster, storage then yes more the better.

3TBs, I have an internal 1TB and I have used < 100gb!

SpawnHappyJake · 2012-06-15 16:28:57

The installation on my external hard drive will not be my main installation. I do intend to eventually install to my internal hard drive.

The thumb drive is just meant to be a handy thing. It is bootable into ArchBang 64-bit and 32-bit (actually installed), Fedora 32-bit and Fedora 64-bit (actually installed), and is also bootable into the installers for those four things.

It is also bootable into Clonezilla 64-bit, Clonezilla i686, and Clonezilla i486.

And memtest.

I am working on my external hard drive to have those same things.

Having ArchBang installed to my external gives me something to work from as I copy stuff off my internal, as well as transfer stuff from one external to another. Once I have done all my saving and moving and whatnot, I can nuke my internal and install ArchBang to it, which will be my main installation.

The external will be a great rescue device (just as the thumb drive is, but with more space) for fixing friends' computers as well. I can instantly boot into an OS with a functioning GUI and with plenty of tools (that I installed) at my disposal. Also Clonezilla and memtest will be there for me. From within Linux booted off the external I can copy stuff off that they need to keep, restore an image, install Linux from booting off the external into a Linux Live installer, or install an OS to the main drive via Qemu.

Basically I'll be able to pull a rabbit out of a hat.

Also, the reads and writes won't wear on my external hard drive like on the thumb drive. The negatives of the external compared to the thumb drive are the need of a power outlet and a delicate power connection.

And as far as SSDs go, I totally recommend going SSD. If buying parts for a new computer, I would only recommend getting that 3TB drive _after_ already having an SSD and needing additional space/a backup drive _and_ not being able to afford the needed amount of extra space as SSD space. I found a 120 GB SSD cheaper than many spindle hard drives, but if you need much more space than that, and depending on your budget, you stand a high chance of having to buy a large capacity spindle drive. It's at that point that I would recommend that 3TB drive and possibly an enclosure.

Here is a pretty fast 120 GB SSD for $95.99:http://www.newegg.com/Product/Product.aspx?Item=N82E16820226236 . It is the most bang-for-buck/affordable SSD I have found.

Anyways, have you started working on re-creating my issue? Let me know if you run into any difficulties. (reminder: I posted instructions on how to do this all with a hard drive image and Qemu - you won't need a USB drive, nor will you need to install Linux or Grub to your "real machine".)

Thank again,
Jake

Last edited by SpawnHappyJake (2012-06-15 16:38:36)

SpawnHappyJake · 2012-06-18 17:33:20

Anyone take a stab at it?

oliver · 2012-06-18 20:04:37

I'm reading the directions now... During each paragraph break I've been converting Proust's "In Search Of Lost Time" into binary and I'm nearly done!

I'll add some comments as I go if you don't mind:

Now make a copy of the grub folder where the original is named "grub_fresh_from_bzr" and the copy is named "grub_BIOS".

This is done by making a new folder next to the grub folder, copying the grub folder into the new folder, renaming the grub folder to "grub_fresh_from_bzr", renaming the grub folder copy to grub_BIOS, then cutting and pasting the grub_BIOS folder out of the new folder next to the grub_fresh_from_bzr folder, then deleting the "New" folder.

You have to do it like that because you can't have two folders with the same name in the same directory at the same time.

Leave the fresh one untouched, so that you can always go back to it.

This totally confused me... is it not just this (with my filenames, not yours)?:

$ cp -pR grub grub.vanilla

I have a dir called ~/tmp/helpjake... so for this bit I added --prefix=~/tmp/helpjake
That way, my 'make install' doesn't have to be done as root and it's easy to wipe out when I done

Do "sh ./autogen.sh"
Now do "./configure --help" or "./configure -h"
Take a moment to marvel all the choices.
I do "./configure --disable-werror --enable-efiemu --enable-cache-stats" when I compile Grub for BIOS.

*Note "--disable-werror" is often the secret to compiling Grub. Often times if not specified, you won't get past make.

OK... first real problem

I dd'd the image

 bunzip2 -c ./sda4.ext3-ptcl-img.bz2 | dd of=/dev/mapper/loop0p1 bs=512k
1+1274100 records in
1+1274100 records out
5244899188 bytes (5.2 GB) copied, 852.572 s, 6.2 MB/s

But I get a superblock error when trying to mount

# mount -o users /dev/mapper/loop0p1 /mnt/ArchBang64
mount: wrong fs type, bad option, bad superblock on /dev/mapper/loop0p1,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail or so

(to be continued tomorrow)

Mr Green · 2012-06-19 00:24:25

Sorry not had time to test the is out, do not see any advantage over simply carrying live iso around on a usb drive.

If you go for poor mans install on drive you can have ArchBang plus any files or configs you like.

There may be a slight speed improvement but we are still working over usb.

This not taking anything away from what you are trying to do by any means, I am still trying to figure out efibooting

oliver · 2012-06-19 07:33:22

I uncompressed the sda4 image file (not sure why it had a .aa extension but the checksum matched) and am unable to mount it on a loopback device (this is without dd'ing it to the pseudo file-system)

I'm not using the clonezilla method - maybe thats the cause

SpawnHappyJake · 2012-06-19 17:10:34

Thanks for being there guys : ) .

About the image, the file extension is just what Clonezilla gave it. That it can't be mounted is weird...I'll look into that. I'm fairly sure it is a raw image. But at least the Clonezilla method should work.

As far as a regular install vs a "poor man's install", it's a little less work and possibly is a little faster, assuming that by "poor man's install" you mean live + a persistence file. And honestly, I don't remember how to do a persistence file setup off the top of my head, though I'm sure at least one wiki is all over it.

Also, this is a USB 3.0 thumb drive which can be read from and written to at the same time. It has a 200 MB read speed.

Also don't have to deal with fragmentation of the persistence file itself in addition to fragmentation of files in the filesystem in the persistence file.

Ultimately, I want certain programs available without having to re-install them every boot, which I understand can be done with a persistence file, but I just went the easy way and actually installed it.

I imagine things might get interesting if you try to update the kernel if you have live + persistence.

I have tried persistence files before (with Mint) and remember them being painfully slow and picky. I couldn't have more than one distro on one partition, each in a different folder each with their own persistence file because I had no way of specifying which persistence file I wanted a kernel to use.

Cheers,
Jake

SpawnHappyJake · 2012-06-19 18:11:34

I couldn't get the extracted archive to mount either. Let's stick to the Clonezilla method. Perhaps the Clonezilla image isn't a plain raw filesystem image after all.

Cheers,
Jake

oliver · 2012-06-21 06:40:52

still can't get this to work even using clonezilla method but it's entirely possible I'm messing something up

I created a second virtual disk, ran losetup/kpartx on it, mounted it and created a dir called helpjake-img with the 16 files from the dropbox folder. Total space used was about 2.5GB (including f/s overhead)

I unmounted the disk, fired up qemu, booted clonezilla and took all defaults until it asked where to mount /home/partimage. I selected /dev/sdb1 (my second virtual disk.)

It mounts it (and I see the space used matches what I expect) and I try the "restore image to disk" option (in "beginners") but it errors out and says no image was found.

oliver · 2012-06-21 18:58:04

OK... so I got the restore done (my problem was that my virtual disk had three partitions (sda1/sda2/sda3) and the image has to restore to sda4. To get around it, I renamed the image file and edited the 'parts' file

So now I can boot the image and see that it doesn't boot into X. However, I also see that you're calling the vesa driver in /etc/X11/xorg.conf.d/20-gpudriver.conf but you don't have xf86-video-vesa installed

When I install it, I don't get the Xorg error but I only get a blank screen (i.e. no openbox) but I'm not sure if thats because qemu doesn't know how to render.

If I play around with other drivers I get a plain red screen with no cursor etc but Xorg.log doesn't show any major errors anymore.

Can you try installing the vesa driver and running on real hardware?

SpawnHappyJake · 2012-06-22 10:55:02

Did you tell Qemu to use "-vga std"? Also, friendly reminder to use kvm if you can.

Thank you Oliver and Mr. Green for your continued help!

Oliver, I'll give that a shot, though I will say I've tried installing vesa, uninstalling vesa, installing the intel driver, deleting 20-gpudriver.conf, and putting 20-gpudriver.conf back. Maybe I just haven't hit the right combo yet.

scjet · 2012-06-22 16:42:48

I've used Clonezilla, but not to any expert extent, but Clonezilla is there to "make a clone" of your entire hard-drive,..., mmmm ?
...and that's all I use it for. Clonezilla will NOT work on weird partitions/USB's.... possibly?
but you guys are way ahead on working this out. I just wanted to warn (anyone) about what Clonezilla can, and canNOT do, with my experience-thus far.

Last edited by scjet (2012-06-22 16:45:35)

oliver · 2012-06-22 18:36:11

SpawnHappyJake wrote:

Did you tell Qemu to use "-vga std"? Also, friendly reminder to use kvm if you can.

I already had qemu-kvm installed so I didn't use the "-enable-kvm" option - not sure if that made a difference.

I did use "-vga std" but after starting X I got a plain black screen (and no Xorg errors)... I then installed the vmware driver and changed it to "-vga vmware" and got a plain red screen

Before I did anything I was getting a "no screens found" error

If anyone else wants to try this, you will need to edit out the extra file-systems in /etc/fstab :-)

SpawnHappyJake · 2012-06-22 20:31:02

I've used Clonezilla, but not to any expert extent, but Clonezilla is there to "make a clone" of your entire hard-drive,..., mmmm ?
...and that's all I use it for. Clonezilla will NOT work on weird partitions/USB's.... possibly?
but you guys are way ahead on working this out. I just wanted to warn (anyone) about what Clonezilla can, and canNOT do, with my experience-thus far.

Clonezilla can either make an image of only a partition, or the entire hard drive. Acronis has both modes as well. Don't worry, this is a common concept. One of the questions Clonezilla asks you is whether you want to deal with entire hard drive stuff or partition-only stuff. If I remember right, it asks you that fairly early on.

Also, there is only one thing wierd about USB media, and it is a non-issue in Linux (Clonezilla runs in Linux).

Other than that one thing, a hard drive is a hard drive. A flash drive, external USB drive, FireWire drive, internal hard drive, eSATA external drive...they are all hard drives.

Besides, in Qemu, the drive appears to be IDE.

You can forget about USB being an issue when it comes to scheming (as in choosing the partitioning scheme), partitioning, and formatting, unless you're in Windows.

WARNING: THE FOLLOWING IS A RABBIT TRAIL TO BE ENTERED ONLY BY THE CURIOUS.

In case you're curious, that "one thing" that's wierd about USB flash drives is that they falsely report that they are removable media readers, whereas a "regular" external USB hard drive correctly reports itself as a hard drive.

Removable media is media that can be removed from a reader and include CDs, DVDs, SD cards, casette tapes, and floppy diskettes. Not flash drives.

When Windows detects a removable media reader, it sets up a policy for it where no caching is used, allowing uneducated computer users to safely yank out the media without ejecting the media first (well, safe as long as no programs are reading or writing to it).

If thumb drives reported themselves as the hard drives that they are, Windows would establish a caching policy with them and when uneducated computer users yank out their thumb drive without ejecting there will be mass destruction because Windows would be doing caching with that thumb drive.

So, flash drive manufacturers program their flash drives to lie and say that they are some kind of removable media reader (as if it was a USB SD card reader or something). Of course the user could just go in and turn write caching off for that thumb drive, but again, this is for uneducated computer users who wouldn't know to do that.

The driver Windows uses with removable media does not support more than one partition. This means Windows will typically only let you use the first partition of a thumb drive and will not let you make more than one partition on a thumb drive. You can make Windows use the regular hard drive driver with a thumb drive by installing the Hitachi Filter Driver.

That is the only time USB drives are wierd compared to other drives.

But in Linux, there is no such issue. Partition away!

It is worth noting that some thumb drives are shipped with a single filesystem spaning the entire drive (no partition table). This lack of scheming is called "superfloppy". I would be surpized if Clonezilla can handle superfloppies, since Clonezilla looks for a partition table.

But superfloppies can't be used as an arguement for USB drives being weird because that's not intrinsic to it's USBness and it can be re-schemed as MBR or GPT by the user; note that an internal hard drive could be non-schemed as superfloppy.

Cheers,
Jake

SpawnHappyJake · 2012-06-22 20:34:46

I already had qemu-kvm installed so I didn't use the "-enable-kvm" option - not sure if that made a difference.

Yes, do specify "--enable-kvm". It will make the VM much faster. But you have to insert the "kvm" module and the "kvm-intel" module first (or whatever module appropriate for your CPU). Your CPU must have a ring -1 and use of it must be enabled in the CPU's MSR to do that though.

What version of qemu are you using? (Do qemu-system-x86_64 --version or whatever.)

Cheers,
Jake

SpawnHappyJake · 2012-06-23 00:05:15

Ok, I installed vesa and ran "startx" on my real hardware. I got a black screen with a non-blinking white underscore in the upper-left corner.

I restarted into my real hardware and uninstalled the intel driver (in case it was competing) and ran "startx". Same black screen with underscore.

Then I booted the thumb drive in Qemu and got a totally black screen after running "startx".

I have not encountered any red screens in either Qemu or real hardware (probably because I didn't install the vmware driver). Running Qemu version 1.1.0.

The "no screens found" error happened to me before with this project.

Thank again folks. Would be pretty nice to know how to fix this.

SpawnHappyJake · 2012-06-25 09:58:40

Sorry folks, I'll be unavailable this week. How about picking it up the next? Have a good one!
Jake

SpawnHappyJake · 2012-07-23 13:05:30

Ok. I tried a whole bunch of crazy stuff to finally get a system-wide update to go through on it (hoping that would fix it).

After doing a system-wide update and getting a functional initrd to go with the new kernel, which required installing udev, I still get the x respawning too fast error.

I tried with and without "nomodeset", and I tried with and without /etc/X11/xorg.conf.d/10-gpudriver.conf.

I also explicitly reinstalled xf86-video-intel and xf86-video-vesa.

To get the system-wide upgrade to work, I booted into another installation of ArchBang, moved to target installation's /lib folder to /libold, and used "sudo pacman -Su -r [path to where target installation was mounted]".

But to even get there, I had to uninstall mach64-dri, mga-dri, r128-dri, savage-dri, sis-dri, tdfx-dri, ati-dri, intel-dri, and xf86-video-ati. Because they conflicted with something. Maybe it was libgl or glibc. Then I installed what they were conflicting with, then I installed them all back. That got those things up-to-date.

I had to delete /usr/share/hwdata/pci.ids and /usr/hwdata/usb.ids for the system-wide upgrade to go through.

But it still didn't go through because there wasn't enough space on the partition.

So I deleted all the packages, made a folder in my home folder of the installation I was booted into called "trickdir", then "mount -o bind"ed that folder to where the target installation downloads packages, and that got around the lack of space problem.

Also, to download udev, which I needed to make a functional initrd, I had to actually Google for it and download it with a web browser and save the package to where pacman downloads packages, because pacman could not find mirror that had it.

Also, now when I try to log in, it says:

Magnum login:

Which is normal...so far.

I type in "mom" and hit enter, and the blinking white underscore goes down a line, and it never asks for the password.

Eventually it just goes back to:

Magnum login:

Pretty weird.

Jake

Last edited by SpawnHappyJake (2012-07-23 13:06:11)

SpawnHappyJake · 2012-07-23 22:09:08

Does the above error when I deliberately boot into runlevel 3 by modifying inittab as well.

This might be a permanent mystery.

If I deleted everything but /home and /usr and then re-installed without formatting, would that keep all my programs and settings? What would that do? Would it be a "repair install"?

Jake

oliver · 2012-07-24 07:52:46

SpawnHappyJake wrote:

If I deleted everything but /home and /usr and then re-installed without formatting, would that keep all my programs and settings? What would that do? Would it be a "repair install"?
Jake

Is /home a separate file-system?

Why would you *want* to keep /usr (unless you have stuff in /usr/local that you compiled from source)

SpawnHappyJake · 2012-07-24 18:52:11

No, /home is not on a separate filesystem.

I have compiled a few things from source such as gparted and libparted.

I just want to know how to fix an installation if it breaks. A system-wide update didn't fix it. So I'm wondering if I can do a repair install.

I want to know how to fix an installation of ArchBang in case I use it as my main OS and it goes down.

The thumb drive is like a practice run.

Jake

Arch Linux

Announcement

#1 2012-06-09 19:05:21

Can't start X in regular AB installation on a USB drive.

#2 2012-06-09 19:46:54

Re: Can't start X in regular AB installation on a USB drive.

#3 2012-06-09 19:53:20

Re: Can't start X in regular AB installation on a USB drive.

#4 2012-06-09 23:22:27

Re: Can't start X in regular AB installation on a USB drive.

#5 2012-06-09 23:59:59

Re: Can't start X in regular AB installation on a USB drive.

#6 2012-06-10 00:37:57

Re: Can't start X in regular AB installation on a USB drive.

#7 2012-06-10 02:26:37

Re: Can't start X in regular AB installation on a USB drive.

#8 2012-06-10 06:04:28

Re: Can't start X in regular AB installation on a USB drive.

#9 2012-06-10 06:38:54

Re: Can't start X in regular AB installation on a USB drive.

#10 2012-06-10 08:18:03

Re: Can't start X in regular AB installation on a USB drive.

#11 2012-06-10 09:04:18

Re: Can't start X in regular AB installation on a USB drive.

#12 2012-06-10 09:08:45

Re: Can't start X in regular AB installation on a USB drive.

#13 2012-06-10 09:25:59

Re: Can't start X in regular AB installation on a USB drive.

#14 2012-06-10 23:54:01

Re: Can't start X in regular AB installation on a USB drive.

#15 2012-06-11 00:00:00

Re: Can't start X in regular AB installation on a USB drive.

#16 2012-06-11 00:15:31

Re: Can't start X in regular AB installation on a USB drive.

#17 2012-06-11 00:32:06

Re: Can't start X in regular AB installation on a USB drive.

#18 2012-06-11 00:46:52

Re: Can't start X in regular AB installation on a USB drive.

#19 2012-06-11 14:27:55

Re: Can't start X in regular AB installation on a USB drive.

#20 2012-06-11 14:55:06

Re: Can't start X in regular AB installation on a USB drive.

#21 2012-06-11 19:15:27

Re: Can't start X in regular AB installation on a USB drive.

#22 2012-06-11 19:53:30

Re: Can't start X in regular AB installation on a USB drive.

#23 2012-06-11 20:24:40

Re: Can't start X in regular AB installation on a USB drive.

#24 2012-06-12 00:12:37

Re: Can't start X in regular AB installation on a USB drive.

#25 2012-06-12 00:43:57

Re: Can't start X in regular AB installation on a USB drive.

#26 2012-06-12 07:13:40

Re: Can't start X in regular AB installation on a USB drive.

#27 2012-06-12 14:09:05

Re: Can't start X in regular AB installation on a USB drive.

#28 2012-06-12 20:35:09

Re: Can't start X in regular AB installation on a USB drive.

#29 2012-06-14 22:50:01

Re: Can't start X in regular AB installation on a USB drive.

#30 2012-06-14 23:33:25

Re: Can't start X in regular AB installation on a USB drive.

#31 2012-06-15 16:28:57

Re: Can't start X in regular AB installation on a USB drive.

#32 2012-06-18 17:33:20

Re: Can't start X in regular AB installation on a USB drive.

#33 2012-06-18 20:04:37

Re: Can't start X in regular AB installation on a USB drive.

#34 2012-06-19 00:24:25

Re: Can't start X in regular AB installation on a USB drive.

#35 2012-06-19 07:33:22

Re: Can't start X in regular AB installation on a USB drive.

#36 2012-06-19 17:10:34

Re: Can't start X in regular AB installation on a USB drive.

#37 2012-06-19 18:11:34

Re: Can't start X in regular AB installation on a USB drive.

#38 2012-06-21 06:40:52

Re: Can't start X in regular AB installation on a USB drive.

#39 2012-06-21 18:58:04

Re: Can't start X in regular AB installation on a USB drive.