Blogs

How to Convert a GPT disk layout to a MS-DOS/MBR layout without data loss (and Gigabyte Hybrid EFI)

If you're coming here from Google searching for how to convert a GPT disk layout to MS-DOS/MBR and don't want to read through my (probably boring) story, click here ;)

Adventures with Hybrid EFI

My gaming PC has been long overdue due for a reformat. I naively allocated only 30GB to the Windows partition (and the other 120GB to 3 flavours of Linux) thinking I wouldn't use Windows for much other than Starcraft 2, but a few months back I had the urge to play Battlefield 2 again. Ever since installing and fully patching it disk space has been running pretty tight. I had to disable sleep, hibernation as well as system restore and still only had 4GB of free space, so my filesystem became fragmented easily. With the release of Windows 8 Customer Preview (download it free here), I figured it was a good time to reformat my disk and reinstall all my OSs from scratch.

I figured while I'm at it, I would make all of the big changes at once and enabled EFI booting on my Gigabyte GA-Z68A-D3H board. Little did I know that when the BIOS says "EFI," it really means Gigabyte's "Hybrid EFI" implementation and not UEFI (although in retrospect, the fact that I made the change in the BIOS should have been enough of a hint, right?). With Hybrid EFI enabled, Windows 7 and Windows 8CP installed perfectly and even created a nice GPT disk layout so reinstalled my games and activated Windows 7. Then I rebooted to play around in Windows 8CP for a bit (I do not like it, btw).

I then tried installing Fedora 16. To my surprise EFI booting failed every time, despite the all of the Fedora 16 installation media being EFI-capable. When attempting to boot from my Fedora 16 Live (x86_64) USB key I just would get a black screen with "........." printed one dot a time and then it would proceed to fall back to the next boot device (Windows boot manager on the hard disk). Upon re-examining my BIOS settings, I was disappointed to find that the setting was actually called "CD/DVD EFI Boot Option" indicating that perhaps USB EFI booting was not supported. Fair enough, I burnt the same F16 image I was using on the USB key to a CD and tried again. The same "........." text appeared.

It was then as I went back to boot Windows 7 that I discovered my attempts to set it as the default OS from Windows 8CP removed my capability of booting Windows 7 somehow. At this point it was 2AM and I was fed up with this stupid Hybrid EFI. I looked for a way to revert to a good old MS-DOS/MBR partition layout. After some Googling I stumbled across Rod Smith's website. He has extensive documentation on EFI booting, including with Gigabyte's implementation of Hybrid EFI. He says that it shares a large amount of code with EFI DUET (tianocore) and although it does work natively with Windows 7, it is not a full UEFI implementation. That would explain the problems I was having with Fedora, then.

The actual GPT to MBR conversion

Through the Rod Smith's guidance and a few dirty tricks, I was successfully able to convert my GPT partition - without data loss or deleting any partitions - and then boot Windows 7 in legacy/MBR mode. In order to do this you'll need your Windows installation media at hand as well as a copy of the Fedora 16 Live media. If you don't have a copy of Fedora 16 Live handy, you can download the Live media ISO (64-bit) from a local mirror here. See the Fedora 16 Installation Guide for details on burning this image to a CD or on creating a bootable USB key.

Keep in mind that at this point I only had 3 partitions and a bunch of unpartitioned space on the disk, so conversion was a rather straightforward process (all GPT partitions mapped directly to primary partitions). Although it is theoretically possible to convert GPT partitions with >4 partitions by defining which ones are to be logical partitions after conversion, I have not tested this.

  1. Boot your Fedora 16 Live media and wait for your session to start. If you're having troubles booting, press Tab at the boot loader screen and try booting with the nomodeset parameter added.
  2. Depending on your graphics card, you'll either be presented with the new Gnome 3 Shell or with the traditional interface. Start a terminal session by putting your mouse in the top right corner of the screen and typing "terminal" in the search (Gnome Shell) or by selecting Applications > System Tools > Terminal (traditional interface)
  3. Install gdisk:
    su -
    yum -y install gdisk
    This may take a few moments.
  4. Make a backup of your current GPT scheme:
    gdisk -b sda-preconvert.gpt /dev/sda
  5. Now we will attempt to convert your GPT disk layout to MS-DOS/MBR. Start gdisk:
    gdisk /dev/sda

    You should be prompted with:
    Command (? for help):
  6. Press r to start recovery/transformation.
  7. Press g to convert GPT to MBR.
  8. Press p to preview the converted MBR partition table.
  9. Make any modification necessary to the partition layout. See Rod Smith's Converting to or from GPT page for more details on this.
  10. When you're happy with the MS-DOS/MBR layout, press w to write changes to the disk.
  11. Shutdown Fedora 16 and boot from the Windows 7 installation media
  12. Enter your language & keyboard layout and then select the option to repair your computer in the bottom left corner.
  13. From the available options, select Startup Repair. Windows will ask for a reboot.
  14. Follow the previous three steps again to boot the Windows 7 installation and run startup repair
  15. Once again, boot the Windows 7 installation media but this time opt to open a command prompt instead of choosing startup repair. Type:
    bootrec /scanos
    bootrec /rebuildbcd
    bootrec /fixmbr
    bootrec /fixboot
  16. Close the command prompt and run Startup Repair one last time.

That's it! You should now have a bootable installation of Windows 7 on a MBR partition layout.

References

Rating: 

New module: uc_cano forked as uc_conditional_attributes

I have been using uc_cano on a few Ubercart sites and found that it is a great module for restricting attributes combinations when one attribute only makes sense if a certain option is selected on another attribute. Unfortunately, the module is very minimally maintained and there are many bugs present in the most recent release. Several community patches have shown up to fix some of the issues, but they need to be mixed and matched to get a working module. Furthermore, out of the box uc_cano is not compatible with the uc_node_checkout nor uc_aac modules.

In order to future centralize development efforts, facilitate releases and maintain a more up-to-date code base, I have collected the various community contributions plus written several fixes of my own and forked uc_cano as uc_conditional_attributes, now available on Drupal.org at http://drupal.org/project/uc_conditional_attributes. I hope you enjoy the new module! A dev release is available and I am hoping to add the first stable release soon after two known bugs are taken care of.

Some the changes I have made to cano include:

  • Rewrite of the JavaScript code and the addition of hooks
  • Addition of a new hooks & preliminary API for other modules to use
  • Compatibility with uc_aac (patch also required to uc_aac until it gets accepted upstream)
  • Compatibility with uc_node_checkout
Rating: 

Drupal module & patch work

I have been working closely with a company in the United States called Grindflow Management LLC (grindflow.com) on several Web projects including a few Drupal sites. I'm excited to say that over the holidays I had a chance to finally put together some of the patches and modules we worked on! More details on each below.

Patches

  • Add support for simultaneous display of multiple product nodes on one page. (#1377310)
    This patch for uc_option_images resolves the problem where uc_option_image would cease to function if multiple nodes were displayed on one page (for example, if promoted to the front page or if displayed in a View). The issue here was that drupal_add_js() in Drupal 6.x has an annoying bug (feature?) where array_merge_recursive() is used when the same JavaScript settings data is added multiple times. The code that generates the settings in uc_option_image (validly) has no idea if you're going to be displaying one node or ten, so each time the settings data for a node is generated is must be merged and the merging caused all sorts of weird duplicate data with invalid offsets. The workaround to the drupal_add-js() problem was to use a static array to ensure that the global settings only get added once, and to use string key indexes for the node settings data so that array_merge_recursive() would not attempt to merge them as numerical indexes.
  • UC Option Image doesn't work with required attributes (#1377310)
    This bug, also in uc_option_images, was due to the fact that the uc_option_image attempts to analyst the first attribute and preload the set option image for its default value if applicable. The problem with required attributes is that they don't have a default value, so an attempt to generate the default image would always fail. This failure meant that there was no HTML img element generated, so future calls to switch the option image also failed since there was no img element to operate on in the first place. Fixed by defaulting to the 'no image' if analyzing the first attribute fails.
  • Compatibility with Ubercart Ajax Attribute Calculations module (#1377310)
    Yet another one for uc_option_image, this patch adds uc_aac compatibility by changing uc_option_image's JavaScript code to work with Drupal Behaviours. Previously, the code would only run once upon page load so when uc_aac changed the DOM, uc_option_image would break. Using Drupal Behaviours also means paying attention to the 'context' variable, which also played a role in making multiple products on a single page work.
  • Some hooks and form handlers have not been updated for Drupal 6 (#1377310)
    The uc_invite_discount was originally written for 5.x and many portions were not updated for Drupal 6.x, leaving it rather broken. This patch fixed these issues and made it confirm to 6.x code style standards.
  • Preserve attributes from product form when redirecting anonymous users to node checkout form after logging in (#1376864)
    This bug in uc_node_checkout makes it so that anonymous users logging in get redirected back to the product configuration form with the attribute data preserved. Prior to this patch, users would get redirected to the node successfully but their selection of attributes was erased and they would have to start over.

Modules

  • Ubercart Attribute Per Role: http://drupal.org/project/uc_attribute_per_role
    This project allows site administrators to choose attributes that they would like hidden from display based on the user's active roles. For example, a 'gift wrap' attribute could be shown to retail (anonymous/authenticated) users but a definition could be added to hide this attribute for members of the 'wholesaler' role.

  • Clean Module List: http://drupal.org/project/clean_module_list
    This module is a very simple module that with the help from a bit of JavaScript, hides the dependency information (Depends on/Requires/Required by) in the module list and provides a controls to dynamically show/hide the information without a page reload. It can be used in conjunction with module_filter to provide an even cleaner and more searchable module list.
  • Ubercart Chained Attributes and Options (CANO): http://drupal.org/sandbox/firewing1/1374824
    This sandbox is the temporary home for the uc_cano project until I promote it to a full project. It is the continued development of the original module posted by Vizteck Solutions here. The code base is rather stagnant at the moment and I have a few bug fixes and feature improvements planned, so I am in touch with the original sponsor and developers of the module to ensure that all parties agree to move the new code to Drupal.org.
  • Ubercart Pay After Checkout: http://drupal.org/sandbox/firewing1/1229572
    I have written about this one before, uc_payafter allows users to pay for a product after checkout. It supports a configurable set of checkout panes just like the original checkout process does, so you can choose which panes appear during the initial checkout and during the payment (payafter) checkout processes. I am hoping to get the chance to refactor some of the code soon to reduce code duplication. Once that's done, I will promote it to a full project.
Rating: 

Exams are done!

Hurray, another semester of school finished. Sorry for being such a hermit lately... Second half of the semester didn't leave me with much spare time between assignments and work.

That said, please do check back in a few days for a new post! I have a bit of a queue to work through for the holiday break, but I have a several new Drupal modules written and several more patches that I will be uploading to drupal.org and writing about here.

Rating: 

Avoiding kernel crashes when under DDoS attacks with CentOS 5

iWeb Technologies has recently been the victim of several [1 2 3 4] distributed denial of service (DDoS) attacks over the past three or so weeks and it's become a rather irritating issue for iWeb's customers. Not all of their server or their customer's co-located servers are affected in each attack, but often during the attacks there is a general slowdown on their network and a minority of the servers (both hosted and co-located) experience some packet loss.

The server hosting this website was, unfortunately, was one the servers that went completely offline during the DDoS attack on October 14th. iWeb has been a great host so far and their staff looked into the issue immediately after I submitted a support ticket, so within 30 minutes I had a KVM/IP attached to my server.

After an hour or so iWeb had the DDoS attacks under control but my server's kernel would panic within roughly 10 minutes of the network going up. The time between each kernel panic was inconsistent, but would it would crash every time the network went up given a bit of time. I found this message repeated in my server logs every time before a hang:

ipt_hook: happy cracking.

That led me to this post dating back to November 2003 on the Linux Kernel Mailing Lists (LKML) about the same message. I jumped on irc.freenode.net and joined #netfilter to ask what they thought about the message. While I waited for a response from #netfilter, I looked into installing kexec in order to get a backtrace since the kernel oops messages were not being logged (This tutorial proved especially handy).

The backtrace revealed that the problem was indeed in the netfilter/iptables kernel modules, but seemed to be triggered by QEMU-KVM:

Process qemu-kvm (pid: 3911, threadinfo ffff8101fc74c000, task ffff8101fe4ba100)
Stack:  0001d7000001d600 0001d9000001d800 0001db000001da00 0001dd000001dc00
0001df000001de00 0001e1000001e000 0001e3000001e200 0001e5000001e400
0001e7000001e600 0001e9000001e800 ffff81020001ea00 ffff8101fe5d8bc0
Call Trace:
<IRQ>  [<ffffffff80236459>] dev_hard_start_xmit+0x1b7/0x28a
[<ffffffff88665bab>] :ip_tables:ipt_do_table+0x295/0x2fa
[<ffffffff886c7b2c>] :bridge:br_nf_post_routing+0x17c/0x197
[<ffffffff80034077>] nf_iterate+0x41/0x7d
[<ffffffff886c7816>] :bridge:br_nf_local_out_finish+0x0/0x9b
[<ffffffff800565d5>] nf_hook_slow+0x58/0xbc
[<ffffffff886c7816>] :bridge:br_nf_local_out_finish+0x0/0x9b
[<ffffffff8000f470>] __alloc_pages+0x78/0x308
[<ffffffff886c85f9>] :bridge:br_nf_local_out+0x23f/0x25e
[<ffffffff80034077>] nf_iterate+0x41/0x7d
[<ffffffff886c3192>] :bridge:br_forward_finish+0x0/0x51
[<ffffffff800565d5>] nf_hook_slow+0x58/0xbc
[<ffffffff886c3192>] :bridge:br_forward_finish+0x0/0x51
[<ffffffff8025042d>] rt_intern_hash+0x474/0x4a0
[<ffffffff886c3367>] :bridge:__br_deliver+0xb4/0xfc
[<ffffffff886c2294>] :bridge:br_dev_xmit+0xc7/0xdb
[<ffffffff80236459>] dev_hard_start_xmit+0x1b7/0x28a
[<ffffffff8002f76f>] dev_queue_xmit+0x1f3/0x2a3
[<ffffffff80031e19>] ip_output+0x2ae/0x2dd
[<ffffffff8025359a>] ip_forward+0x24f/0x2bd
[<ffffffff8003587a>] ip_rcv+0x539/0x57c
[<ffffffff80020c21>] netif_receive_skb+0x470/0x49f
[<ffffffff886c3ef9>] :bridge:br_handle_frame_finish+0x1bc/0x1d3
[<ffffffff886c801b>] :bridge:br_nf_pre_routing_finish+0x2e9/0x2f8
[<ffffffff886c7d32>] :bridge:br_nf_pre_routing_finish+0x0/0x2f8
[<ffffffff800565d5>] nf_hook_slow+0x58/0xbc
[<ffffffff886c7d32>] :bridge:br_nf_pre_routing_finish+0x0/0x2f8
[<ffffffff886c8c18>] :bridge:br_nf_pre_routing+0x600/0x61c
[<ffffffff80034077>] nf_iterate+0x41/0x7d
[<ffffffff886c3d3d>] :bridge:br_handle_frame_finish+0x0/0x1d3
[<ffffffff800565d5>] nf_hook_slow+0x58/0xbc
[<ffffffff886c3d3d>] :bridge:br_handle_frame_finish+0x0/0x1d3
[<ffffffff886c407e>] :bridge:br_handle_frame+0x16e/0x1a4
[<ffffffff800a4e3f>] ktime_get_ts+0x1a/0x4e
[<ffffffff80020b34>] netif_receive_skb+0x383/0x49f
[<ffffffff8003055c>] process_backlog+0x89/0xe7
[<ffffffff8000ca51>] net_rx_action+0xac/0x1b1
[<ffffffff80012562>] __do_softirq+0x89/0x133
[<ffffffff8005e2fc>] call_softirq+0x1c/0x28
<EOI>  [<ffffffff8006d636>] do_softirq+0x2c/0x7d
[<ffffffff8004de57>] netif_rx_ni+0x19/0x1d
[<ffffffff887b951d>] :tun:tun_chr_writev+0x3b4/0x402
[<ffffffff887b956b>] :tun:tun_chr_write+0x0/0x1f
[<ffffffff800e3307>] do_readv_writev+0x172/0x291
[<ffffffff887b956b>] :tun:tun_chr_write+0x0/0x1f
[<ffffffff80041ef3>] do_ioctl+0x21/0x6b
[<ffffffff8002fff5>] vfs_ioctl+0x457/0x4b9
[<ffffffff800b9c60>] audit_syscall_entry+0x1a8/0x1d3
[<ffffffff800e34b0>] sys_writev+0x45/0x93
[<ffffffff8005d28d>] tracesys+0xd5/0xe0

Code: c3 41 56 41 55 41 54 55 48 89 fd 53 8b 87 88 00 00 00 89 c2
RIP  [<ffffffff80268427>] icmp_send+0x5bf/0x5c0
RSP <ffff810107bb7830>

This was interesting as I do have several KVM virtual machines running on the server, some with bridged networking and others with shared networking. The #netfilter guys confirmed that the happy cracking message was due to the attempted creation of malformed packets by root. My guest guess was that some of the packets from the DDoS attacks were hitting my server and so the bridge was faithfully attempting to forward those invalid packets to one of the virtual machine's network interfaces, causing problems in icmp_send().

The LKML message hinted the REJECT policy could be at fault, so I opened up /etc/sysconfig/iptables and switched to a DROP policy:

*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -m physdev  --physdev-is-bridged -j ACCEPT
# ... a bunch of forwarding rules for the shared network VMs
-A FORWARD -o virbr0 -j DROP # <-- THIS ONE
-A FORWARD -i virbr0 -j DROP # <-- THIS ONE
-A FORWARD -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp -m icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
# ... a bunch of ACCEPT rules for the server
-A RH-Firewall-1-INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
-A RH-Firewall-1-INPUT -j DROP # <-- THIS ONE
COMMIT

And that was all it took! I did not experience a hang after that. According to the guys in #netfilter, newer kernels reassemble packets from scratch when forwarding them, so if a machine is sent an invalid packet this problem is averted. However, CentOS 5 uses an older kernel and I guess this hasn't been backported.

TL;DR: If you using CentOS 5 (or any other distro with an older kernel), use a DROP policy in your iptables configuration instead of REJECT!

Rating: 

RIP Steve Jobs

 

 

 

 

Rating: 
Tags: 

An update on the CentOS 5 server setup series

I posted back in May about the start of my the CentOS 5 Server Setup series and I wanted to give a quick update on it. Since writing the first guide back in May, I have made a few changes and additions to the original getting started and mail server guides in addition to posting several new guides:

Enjoy! If you have any questions or comments I would be happy to hear some feedback. You can reach me at s.adam@diffingo.com.

Rating: 

Downlink/uplink frequencies listed in the iPhone Field Test

I have been attempting to debug a poor reception problem with my iPhone 4 near my house. It has been confirmed that I reside in a dead zone (I am right in the middle of where 2 towers overlap, so I have the weakest signal from both locations) and there's no incentive from my cell provider to fix the problem since the dead zone is very small.

In an attempt to help document the problem, I wanted to capture the exact cell IDs and frequencies used when I experience reception problems. Activating the iPhone's Field Test Mode is easy enough (dial *3001#12345#*), but I quickly realized that something was off with the frequencies listed under by the UMTS/GSM RR Info panes. It was displaying download/upload frequencies of 1037/812 respectively, which is reasonable, but at other times it would show frequencies like 437/37 which made no sense at all.

After a bit of research, it looks like the the label for that data value is wrong; it should be channel and not frequency. Wikipedia has a list of UMTS (3G) frequency bands and the corresponding channel codes as well as the corresponding list for GSM (2G/EDGE). Channels numbers 1037/812 correspond to the 850MHz frequency band which I know Rogers, Telus & Bell all use in their new 3G network deployments. The other popular GSM band in use in Canada is 1900MHz PCS, and sure enough that's what 437/37 corresponds to. Problem solved!

Rating: 

Waiting for this too...

I went to see Feed Me at the beginning of August and was really happy that he played this:

I had heard the track before from people's footage of him playing at EDC, but wasn't sure what it was actually called/who it was by... After some research it's apparently called "Dill The Noise" (by Dillon Francis and Kill The Noise). Francis said on his twitter feed (@dillonfrancis) that it should be coming out on his next EP!

Update 2011/08/03: It has been leaked! youtube.com/watch?v=ZmAdoAysK-w

Rating: 

AMD A-Series Llano APUs out now

This weekend AMD's Llano APUs started shipping this week and Newegg has started offering select models! While I won't be picking one up myself (sorry, I still like my discrete Radeon HD 3850 and will most probably continue to buy discrete PCIe cards), the release of this product makes me really happy because it's been too long that PC manufacturers have been skimping on graphics to lower the cost of PCs and it's really the consumer that pays the price in the end. Yes, true, you can argue "well they should have read the specs" but let's face it, your average consumer has no idea what the difference between the Intel GMA X3150, GeForce 310M or a Radeon HD 6670M is even though choosing one or the other will dramatically affect how they perceive their computer's performance. It is all too common that I see OEMs advertise their beefy and overpriced PCs as fast and all-capable and then when you look at the specs it's got something terribly outdated and underpowered like an Intel GMA X4500 or nVidia GeForce 7100 graphics processor. Seriously, why even bother with the powerful CPU if it can't handle any sort of graphics properly?

The situation is even worse for laptops... In the past few months I've found that it is nearly impossible to find a laptop that has a nice balance between CPU and GPU power for the average consumer. OEMs seem to have their computers fall into one of three builds, and those builds are (a) low-power computing with cheap CPUs and integrated graphics (b) high performance computing with i5s/i7s and an overkill on RAM or (c) gaming computers that weigh lots, have a short battery life and are very expensive. In my opinion they are completely missing the mark on what the "average consumer" actually needs, which is a mediocre CPU with a entry level discrete graphics card and a healthy amount of RAM (3 or 4GB is acceptable) in the $600-700 range. You can think of it as targeting "basic gaming" or "media PC". Samsung has been very good at this actually, offering laptops the higher-end Core 2 Duo CPUs or i3s with the low-class nVidia GeForce 310M/320M series GPUs (some even in the $550-range). Computers like that are well priced, perfect for web & office work but won't choke if you throw 1080p at it, and you could even play Starcraft 2 or WoW on low settings.

Anyways, all that goes to say that this integrated graphics madness should now come to an end. I can't wait until the OEMs start using the AMD chips in desktop in laptops, as it will provide the consumer with a cost-effective computer that can actually process a decent graphic payload - and that's worth a lot. Not to mention that the battery life should improve considerably, too.

Rating: 
Tags: