Re: [Discuss-gnuradio] File sink file size mismatch problem.

[Josh Stevens]

Hey Tom,
  
    I thought i was replying to the list.(Apologies....) I was thinking about it on similar lines as you have. If the fllow graph is stopped by hand the latency when i hit stop and when the flow graph stops it should put some more data but i tried over running the flowgraph for a time span of atleast 15 more seconds after the image-sink(fyi the image sink in this case is quite different from the gr_file_sink because the image sink displays the image as soon as the last bit that completes the image is received.)  and that is when i stop the flow graph which brings me to my next question 
 
Can 1 second bring about a 11KB file change??

Also the flowgraph that i was talking about is as follows.
http://i.imgur.com/1lJzD.png 

The reason why i convert a float to char is because when i connect 
Image Source (Float)  ------->  Image Sink (Float)
                                    |
                                    |
                                    |----> File SInk (Float)  { Tjhe size of the output file was 336KB (6 times the size of the input image).
 as opposed to a float to char which got me closer to the actual size of the input file. What i was aiming for was to have 2 files of equal size (the output of the file sink in the above mentioned flow graph and the output of the packet decoder at the receiver end and convert those values into binaries and do a BER .)

Thank you so much again for your help,

Josh.

On Mon, Jun 4, 2012 at 8:56 PM, Tom Rondeau <address@hidden> wrote:

On Mon, Jun 4, 2012 at 11:12 AM, Josh Stevens <address@hidden> wrote:
> Tom,
>
>     What i was also able to find was that when an image source (type =
> float) is connected to a file sink(type float) the size of the received file
> is around 700 KB while the size of the transmitted file is 65 KB and when i
> use a float2char converter the size of the received file is found to be 76
> KB.
>
>  Is there a way around this or would i have to use a char type conversion on
> every file i receive and convert it into a readable file to then calculate
> the error rate ? Seems like a lengthy process.
>
> Thanks,
> Josh.

Josh,

Some of this might just be a mismatch of data types. I'm not sure I
properly followed how everything worked. One thing that wasn't clear
was how the flowgraph is stopped. Does the image source just keep
running until you stop it by hand? If that's the case, then what is
the image source reading after it's read the entire file? Does it loop
around? And if you're just stopping it by hand, then there's a latency
a) between the program and the display b) between your eye and when
you hit stop and c) from the time you hit stop and when the flowgraph
actually stops. The blocks are all in threads and likely in the middle
of a work function that's handling some number of samples; this will
finish before the program closes. Basically, it will flush what's left
to the file before closing.

Could this be what's happening?

Also, please reply on-list. I think any conversation about GNU Radio
should be held publicly for the records for anyone else looking at
similar problems. Thanks.

Tom

>
> On Sun, Jun 3, 2012 at 11:53 AM, Josh Stevens <address@hidden>
> wrote:
>>
>> Hey Tom,
>>    Thank you for replying .
>>
>> So here is how i do the conversion. The packet decoder is connected to the
>> image sink and file sink. The moment the output is displayed on the screen i
>> stop the flowgraph at the receiver side and the size of the
>> "received_file.dat" is achieved to be 64 KB (which is 1KB smaller in size
>> and still not exact but is considerably better as opposed to a 11KB
>> difference). I then use python command to convert this file into a readable
>> format by using numpy.fromfile and throw in the name of the file as the
>> first argument to the same but the dtype i choose is int8 . The received
>> file contains values ranging from 0-127 since the image is choose is a Black
>> and White image which when converted to binary would be 8 bits which also
>> explains the range(0-127) .
>>
>> About the question that you asked , the extra bits that are added , are
>> added to the end of the file , for example in this case the received file
>> contains 65536 ( 64*1024) and these bytes match the first 65536 bytes of the
>> numpy int8 converted transmitted file but the other 10,000 or so bytes are
>> just different numbers but all within the 0-127 range .
>>
>>
>> Thanks again and Kind regards,
>> Josh.
>>
>> On Sun, Jun 3, 2012 at 10:41 AM, Tom Rondeau <address@hidden> wrote:
>>>
>>> On Wed, May 30, 2012 at 3:04 PM, Josh Stevens <address@hidden>
>>> wrote:
>>> > Hello All,
>>> >
>>> >       A couple of days ago i had installed a GNURadio digital image
>>> > processing block that makes an image source and sink block available as
>>> > displayed in the image below.
>>> >  Resource : https://github.com/a-w-s/GNURadio-DIP
>>> >  Flowgraph : http://i.imgur.com/1lJzD.png
>>> >
>>> > The output of the image source and the input to the image sink are
>>> > "floats"
>>> > only and nothing else. I tried to collect pixel information into a file
>>> > sink
>>> > and i am successful at that but the problem comes in when the size of
>>> > the
>>> > input file size is not the same as the size of the file sink output.
>>> >
>>> > The image is a basic black and white test image called lena.bmp whose
>>> > file
>>> > size is 65.0 KB. The link to which is also provided below ,
>>> > Resource to Image :
>>> >
>>> > https://www.google.com/search?q=lena+256+x+256&hl=en&prmd=imvns&source=lnms&tbm=isch&ei=_G7GT7vkC4rs8wSG99SaBg&sa=X&oi=mode_link&ct=mode&cd=2&sqi=2&ved=0CD8Q_AUoAQ&biw=1280&bih=827
>>> >
>>> > The file size of the received file (file sink) is 76.0 KB.
>>> >
>>> > The reason why I pay more attention to this is because i intend to
>>> > calculate
>>> > BER / Pixel Error Rate which would take into account a reference stream
>>> > which in this case would be the file with the extra bits , and Image
>>> > sink
>>> > output ( or the demodulated output) .
>>> >
>>> > Please feel free to ask me any questions that one might have .
>>> >
>>> > Thanks and regards,
>>> > Josh.
>>>
>>> Hi Josh,
>>>
>>> In general, when doing things like this, there is often some extra
>>> "stuff" that happens because the scheduler works on a per-chunk basis.
>>> So sometimes the size of what's been processed can be confused a bit,
>>> but if you're file is only so large, I wouldn't think this would
>>> happen. Also, 11 KB is a pretty substantial difference in file size.
>>>
>>> Do you know where your image is actually located in the output file?
>>> That is, what's the byte offset? (You could try to read this into
>>> Python with scipy or Matlab and do a correlation to see where in the
>>> file the image exists). I'm just trying to help determine if there's
>>> some initial fill or if the extra 11 KB are produced at the end of the
>>> file.
>>>
>>> Tom
>>
>>
>