Re: Interface for SCSI transactions ?

[Thomas Schmitt]

Hi,

Olaf Buddenhagen:
> Unless there is some practical difficulty, I *strongly* suggest to
> describe the individual struct members with MIG, rather than doing the
> marshalling (serialisation) by hand.

Regrettably MIG explicitely refuses to model struct entrails.
mig.ps page 3:
"There is no way to specify the fields of a C structure to MIG."
(which is quite a disappointment)


> This means the RPC will be really SCSI-specific. It is a bit ugly, as it
> will add a meaningless entry for all other device classes;

As it is planned currently, it will be usable for any device oriented
transaction, that takes a struct as input and delivers another
struct as output.
In 
  gnumach/include/device/device.defs
i plan to add

  routine device_transact_native(
             device          : device_t; 
       in    function_code   : unsigned int;
       in    in_data         : ^ array[] of unsigned char;
       out   out_data        : ^ array[] of unsigned char
       );

The one function_code, which i plan to define for my needs, will apply
to SCSI disks, SCSI CDs, and IDE/ATAPI CDs.
As i do not know much about SCSI disks, i currently tend to restrict it
to the two CD drive classes.

Other function_code values may be defined to support other device type
specific transactions with other in/out structs.


> the transaction RPC isn't actually
> likely ever to be used by anything else anyways.

Not necessarily. Who ever has the plan to make a specialized driver in
userspace, will at least find an existing RPC gateway.

A further function_code will need serializers and de-serializers for
both structs, and a case in its appropriate implementation of the new
  gnumach/device/device_emul.h
  struct device_emulation_ops
method
  io_return_t (*transact_native) (void *, unsigned int function_code,
                                  unsigned char *in_data,
                                  mach_msg_type_number_t in_len,
                                  unsigned char *out_data,
                                  mach_msg_type_number_t *out_len);

My function_code needs a method implementation in
  gnumach/linux/dev/glue/block.c
as function

  static io_return_t
  device_transact_native(void *d, unsigned int function_code,
                      unsigned char *in_data, mach_msg_type_number_t in_len,
                      unsigned char *out_data, mach_msg_type_number_t *out_len)
  {
    struct block_data *bd = d;
  
    switch (function_code)
      {
      case DEVICE_TRANSACT_NATIVE_SCSI_PKT: 

        /* >>> my upcomming SCSI transaction */

        break;
      default:
        return D_INVALID_OPERATION;
      }
    return D_SUCCESS;
  }

(Possibly it becomes DEVICE_TRANSACT_CD_PKT rather than
 DEVICE_TRANSACT_SCSI_PKT. Well, these all are just work names. I will
 gladly adopt any particular bike shed design.)

The following source modules have instances of device_emulation_ops with
initializers which should get a NULL for the new member .transact_native :
  gnumach/xen/net.c
  gnumach/xen/block.c
  gnumach/linux/dev/glue/net.c
  gnumach/linux/pcmcia-cs/glue/ds.c
  gnumach/device/ds_routines.c
A special case is
  gnumach/i386/i386/io_perm.c
which only initializes a single element by 
  .no_senders = no_senders
So this one can stay as is.

This list of instances matches the "/* List of emulations.  */"
in gnumach/device/ds_routines.c . So it should be complete.

---------------------------------------------------------------------

>From here on i plan to dive into 
  gnumach/linux/src/drivers/block/ide.c
  gnumach/linux/src/drivers/block/ide-cd.c
  gnumach/linux/src/drivers/scsi/scsi.c
to finally use their command transaction facilities.
  ide_do_drive_cmd() or cdrom_queue_packet_command()
  scsi_ioctl_send_command()


> > I plan to assess the structs of FreeBSD CAM and Solaris uscsi
> > [...]
> > the userspace interface should already now be wide enough to express
> > what the other OSes can express.

I meanwhile compared the transaction structs of Linux SG_IO,
Linux CDROM_SEND_PACKET, FreeBSD CAM, and Solaris uscsi.
Mostly Linux SG_IO is appealing. With a touch of CAM and slightly extended.
So my structs are designed now, with much room for improvements in the
communication with underlying SCSI or IDE drivers.

Olaf:
> Agreed. While I'm a strong proponent of the YAGNI principle, in this
> case it's indeed pragmatic to use something based on the tried structure
> from some popular modern system, so we can minimise Hurd-specific
> effort.

It is quite interesting to see what other programmers deemed worth to
be exposed during the last 20 years.
Astounding that Linux SG_IO totally hides the auto sense mechanism,
which issues a REQUEST SENSE command after the payload command yielded
CHECK_CONDTION. CAM and uscsi both expose some info about this fetching
of sense data. I plan to allow generous info about it (provided one can
get it from the kernel).

---------------------------------------------------------------------

> (Doing manual marshalling is extra effort; and it will have to change
> again when moving to userspace drivers -- while using a proper hurdish
> RPC from the setout means it can be directly reused for the userspace
> driver interface.)

Should i know the term "manual marshalling" ?
(Google ... but still half clueless.)

Do you mean my plan to have serializer and de-serializer ?
My hairs are raised by the idea to transfer plain struct memory through
the RPC. Sure, it works as long as client and server run on the same
architecture and were compiled in a compatible way ... but ... urgh !

So i think it is appropriate to prescribe an explizit representation
layer for the structs and their components.

I believe that my RPC proposal fits well into the existing method panel of
  struct device_emulation_ops
in
  gnumach/device/device_emul.h 

/* Each emulation layer provides these operations.  */
struct device_emulation_ops
{
  void (*reference) (void *);
  void (*dealloc) (void *);
  ipc_port_t (*dev_to_port) (void *);
  io_return_t (*open) (ipc_port_t, mach_msg_type_name_t,
                       dev_mode_t, char *, device_t *);
  io_return_t (*close) (void *);
  io_return_t (*write) (void *, ipc_port_t, mach_msg_type_name_t,
                        dev_mode_t, recnum_t, io_buf_ptr_t, unsigned, int *);
  io_return_t (*write_inband) (void *, ipc_port_t, mach_msg_type_name_t,
                               dev_mode_t, recnum_t, io_buf_ptr_inband_t,
                               unsigned, int *);
  io_return_t (*read) (void *, ipc_port_t, mach_msg_type_name_t,
                       dev_mode_t, recnum_t, int, io_buf_ptr_t *, unsigned *);
  io_return_t (*read_inband) (void *, ipc_port_t, mach_msg_type_name_t,
                              dev_mode_t, recnum_t, int, char *, unsigned *);
  io_return_t (*set_status) (void *, dev_flavor_t, dev_status_t,
                             mach_msg_type_number_t);
  io_return_t (*get_status) (void *, dev_flavor_t, dev_status_t,
                             mach_msg_type_number_t *);
  io_return_t (*set_filter) (void *, ipc_port_t, int, filter_t [], unsigned);
  io_return_t (*map) (void *, vm_prot_t, vm_offset_t,
                      vm_size_t, ipc_port_t *, boolean_t);
  void (*no_senders) (mach_no_senders_notification_t *);
  io_return_t (*write_trap) (void *, dev_mode_t,
                             recnum_t, vm_offset_t, vm_size_t);
  io_return_t (*writev_trap) (void *, dev_mode_t,
                              recnum_t, io_buf_vec_t *, vm_size_t);
}

Except, possibly, for the basic types in my proposal:

  io_return_t (*transact_native) (void *, unsigned int function_code,
                                  unsigned char *in_data,
                                  mach_msg_type_number_t in_len,
                                  unsigned char *out_data,
                                  mach_msg_type_number_t *out_len);

Do i need some bling_t here ?
Proposals ? How to define in C and MIG ?

---------------------------------------------------------------------

I'm slowly making progress with my plan. (But currently i am distracted
by Debian GNU/Linux happiness to remove /dev/dvdrw while my program is
using it for burning a CD or DVD.)

Open questions concern the wiring from glue/block.c to ide and scsi,
and the implementation of the counterpart functions in userspace.
(Where to implement the serializer for the input struct , and the
 de-serializer for the output ? Where to define the function_code value ?)

Still lots to learn.

---------------------------------------------------------------------

> > i had to learn that a kvm emulated CD-ROM appears as /dev/hd2.

> Is there no option to control this? In such things qemu often offers a
> choice...

I meanwhile know how to cause an emulated SCSI CD, but did not yet find
out how to make kvm emulate an SCSI controller that is supported by gnumach.


> OTOH, machines able to run Mach natively are indeed more likely to have
> an ATAPI writer (including mine), so it's probably more useful to focus
> on these...

Really old CD burners were attached to really old SCSI controllers, back
in the 1990s. I got my last SCSI burner in 2000. It died 2003.


Have a nice day :)

Thomas