bug#41792: Acknowledgement (dd function – message: "No boot sector on USB device"")

[Bob Proulx]

close 41792
thanks

Since the discussion has moved away from anything GNU Coreutils
related and doesn't seem to be reporting any bugs in any of the
utilities I am going to close the bug ticket.  But discussion may
continue here regardless.  If we see a dd bug we can re-open the
ticket.

Ricky Tigg wrote:
> The difference of device path is due to the fact that the USB media was
> plugged out after the write-operation was achieved on the Linux computer
> then plugged into a computer –Asus– whose Windows OS has to be restored,
> then plugged back to the same computer but to a *different* USB port. It's
> safe to open the present issue-ticket.

Hmm...  There is no reason that the Linux kernel would renumber the
device simply because it was removed and inserted again.  Therefore me
thinks that it was not cleanly removed.  Me thinks that something in
the system had mounted it keeping it busy preventing it from cleanly
being ejected.  This "something" may have been an automatic mounting
of it as many Desktop Environments unfortunately default to doing.
IMNHO automated mounting is a bad idea and should never be enabled by
default.

> *Source media*:
> https://www.microsoft.com/en-us/software-download/windows10ISO

The source media doesn't matter to GNU utilities.  The 'dd' utility
treats files as raw bytes and does not treat MS-Windows-10 ISO images
any differently than any other raw data.  It might be that or pictures
of your dog or random cosmic noise recorded from your radio.  It
doesn't matter.  It's just data.

Your Desktop Environment may take action however.  It is possible that
your DE will probe the device, detect that it is an ISO image, and
automatically mount that ISO image.  That's bad.  But that's your
Desktop Environment and unrelated to 'dd'.  But it always been a bad
idea.  Regardless of how many people do it.

> *Rufus v4.1.4* – I couldn't use it since The Windows OS installed is
> missing some system's files. Will convert it to fit on Fedora at release of
> version 33 which will update the uniformly mingw component and thus
> mingw64-headers which is old and is the cause of a known issue.
>
> I wrote the disc image as well using those tools then booted the USB device
> having the disc image written on.:
>
> *Fedora Media Writer v4.1.4* – Officially does not support Microsoft
> Windows disc images. I did not know that before writing.

My first thought was, huh?  Why would Fedora Media Writer not treat
files as raw files?  My second thought was that the question was for a
Fedora Media Writer mailing list as this bug ticket is not the place
to be discussing other random projects.

> *Unetbootin v677* – It writes partially the disc image thus the installer
> is operational partially. Issue was already reported by someone on Git.
>
> *Woeusb v3.3.1* – Installer is operational on BIOS but not on EFI systems.
> Issue was already reported by someone on Git.
>
> *Balena Etcher v1.5.9*8 x64 as AppImage format – The device is not listed
> at boot.

Gosh.  Reading your report makes MS-Windows seem like such a terrible
system!  I read about all of your pain of working on it.  You have
tried all of these tools and nothing is working for you.  It is
reading these types of reports that I am thankful I am working on a
Free(dom) Software operating system where things Just Work!

Meanwhile...  Let's get back to your information about 'dd'.

> $ file -b Win10_2004_Finnish_x64.iso
> ISO 9660 CD-ROM filesystem data 'CCCOMA_X64FRE_FI-FI_DV9' (bootable)

That looks like you were successfully able to write the ISO image to
the device.  Looks okay.

> *Component*: coreutils.x86_64  8.32-4.fc32.; *OS*: Linux Fedora

Good.

> Source of file:
> https://www.microsoft.com/en-us/software-download/windows10ISO
>
> Disc image file
> - checked against its SHA-256 checksum was correct
> - written successfully with that command:
> # dd if=Win10_2004_Finnish_x64.iso of=/dev/sdc bs=4M oflag=direct 
> status=progress && sync

I don't see any error messages.  That's good.

The oflag=direct should use direct I/O.  Which means that the 'sync'
shouldn't matter since there should be no file system buffer to flush.
It will simply flush other unrelated buffers.  Won't hurt though.

The bs size seems very small at 4M to me.  Especially for use with a
NAND flash USB storage device.  I would select a much larger size.  I
would probably use 64M which is likely to be an integral size of your
original ISO image but that should be verified.

> Once written, the partition is as follows:
> $ mount | fgrep /run/media/$USER
> /dev/sdb on /run/media/yk/CCCOMA_X64FRE_FI-FI_DV9 type udf
> (ro,nosuid,nodev,relatime,uid=1000,gid=1000,iocharset=utf8,uhelper=udisks2)

WHY is this mounted?  That seems like a problem.

You said that the device was removed and replaced and went from sdc to
sdb?!  Probably because it was mounted.

This feels like the root cause of all of your problems.  It feels to
me that something is automatically mounting the device.  That's bad.
Then the device is written to the base partition under the file
system.  Then I think likely the sync is flying data to the file
system on top of it which would corrupt the resulting image.

If it were me I would seek and stop all automated mounting.  It's
always bad and particularly a problem due to these types of things.

It does not make sense to me that the device would be written to as
sdc and then removing and inserting it again would rotate it to sdb, a
lower letter.  And it makes no sense to me that it would become
mounted.

If I were writing a USB device with data and then wanted to verify
that the data was written correctly I would read the data back and
verify that the sha256sum matched the data that was written.  If the
check digest matched then I would know the data on the device was
correct.

The image being written is likely smaller than the physical device.
Therefore one can't simply read the device and get a checksum.  One
must read and check only the size of the original image.  Otherwise
the extra data on the end will produce a different digest hash.

There are many ways to do this.  I might note down the size of the ISO
being written as N.  Then use "head --bytes=N /dev/sdb | sha256sum" to
produce the digest hash and then compare what was written to what was
read back.  Or there are probably better ways too.

> While the USB device was booted in BIOS, the following message was
> produced  "*No boot sector on USB device*"".

I think this result was likely due to automatic mounting by the
Desktop Environment.  But without debugging to root cause that is
simply a guess.

> Using *Gparted*, while the device's partition was unmounted, the flags
> could not be managed thus no boot flag was added. The following warning was
> present:
> *2.66 GiB of unallocated space within the partition.*

I think this is not important.  I think this is due to the ISO image
being smaller than the size of your physical device.  Which it must be
in order to hold the image.  I think this is normal and expected.

Bob