Re: [Gluster-devel] [SOLVED] "failed to fetch volume file (key:dpkvol)" error , when tried as non-root (vdsm) user

[Deepak C Shetty]

After doing many experiments and inputs I got from vijay on IRC, this is 
what finally worked.

1) volume set <volname> server.allow-insecure on
2) option rpc-auth-allow-insecure on in glusterd.vol

If either of the above is not present, i get the same "unable to get 
volfile" error.
I have tried setting virsh (actually libvirt) to spawn qemu process as 
'root', 'vdsm' and 'qemu' user
and it fails for all the 3 if either of the above is NOT set.

Only when both of the above is set, it works for me and I am able to 
start a VM via virsh.

Strangely enuf, on Bharata's machine, none of the above options are set, 
but it still works for him !
How why is the difference is  not clear.

But for now, i think i am good to go for my vdsm work, as this part of 
the stack is now working.

thanx,
deepak



On 12/13/2012 12:21 PM, Vijay Bellur wrote:
> On 12/13/2012 11:50 AM, Deepak C Shetty wrote:
> > On 12/13/2012 11:26 AM, Vijay Bellur wrote:
> > > On 12/13/2012 10:20 AM, Deepak C Shetty wrote:
> > > > Here is the qemu process spawned by virsh ( via libvirt )...
> > > >
> > > > root      2912     1  1 10:16 ?        00:00:00
> > > > /usr/local/bin/qemu-system-x86_64 -name virsh-vm-backed-by-gluster 
> > > > -S -M
> > > > pc -cpu qemu64,-svm -enable-kvm -m 1024 -smp
> > > > 1,sockets=1,cores=1,threads=1 -uuid 
> > > > bdbae806-c272-4b87-ae69-274fd4d57c5f
> > > > -smbios type=1,manufacturer=oVirt,product=oVirt
> > > > Node,version=17-1,serial=762589AD-3D52-42C3-6F65-D682277D5B37_52:54:00:c7:66:ec,uuid=bdbae806-c272-4b87-ae69-274fd4d57c5f 
> > > >
> > > >
> > > > -no-user-config -nodefaults -chardev
> > > > socket,id=charmonitor,path=/var/lib/libvirt/qemu/virsh-vm-backed-by-gluster.monitor,server,nowait 
> > > >
> > > >
> > > > -mon chardev=charmonitor,id=monitor,mode=control -rtc
> > > > base=2012-12-13T04:46:48,driftfix=slew -no-shutdown -device
> > > > piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
> > > > virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x3
> > > > -drive
> > > > file=gluster+tcp://vm-vdsm-de-1/dpkvol/debian_lenny_i386_standard.qcow2,if=none,id=drive-ide0-0-0 
> > > >
> > > >
> > > > -device
> > > > ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1
> > > > -chardev pty,id=charconsole0 -device
> > > > virtconsole,chardev=charconsole0,id=console0 -vnc 0:0,password -vga
> > > > cirrus
> > > >
> > > > With this also, i still get the same error which says "unabel to fetch
> > > > volfile"
> > >
> > > Did you happen to restart glusterd after setting allow-insecure to ON
> > > in glusterd.vol? Were the results same before and after setting the
> > > option?
> > Yes, today i came in and started glusterd from scratch so that should
> > have covered your point.
> > Also setting allow-insecure in glsuterd.vol ( as you suggested
> > yesterday) doesn't help either, in fact i get a warning msg in the
> > glusterd log saying that option is incorrect and hence ignoring.
>
> Please set rpc-auth-allow-insecure to on. You can always lookup the 
> relevant keys of translator options by looking at the definition of 
> struct volume_options within the translator code.
>
> > Yes in short, with and without that insecure option, i see the same 
> > result.
>
> Since the option has not been recognized, you haven't still seen it 
> being effective.
>
> > I am curious to know how insecure option can be related to glusterd not
> > able to find volfile ?
>
> The problem is about not being able to fetch volume file. This can 
> happen mostly due to:
>
> 1. A rpc failure between the client and glusterd
>
> 2. glusterd unable to locate the volume
>
> In your case, I suspect it is a rpc failure because of an unprivileged 
> client trying to access glusterd.
>
> > Can you help list all the possible reason why glusterd can ever complain
> > abt not able to fetch volfile ? Maybe that can help me debug my env
>
> It would be primarily one of the above. You can follow the code path 
> from server_getspec in xlators/mgmt/glusterd/src/glusterd-handshake.c 
> to determine where it is failing after the unprivileged issue is 
> sorted out.
>
> Regards,
> Vijay