[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Question re: floor of integer division
From: |
Rhys Ulerich |
Subject: |
Question re: floor of integer division |
Date: |
Mon, 24 Feb 2014 08:42:39 -0600 |
I must be missing something, but I don't understand why floor(i/N)
behaves differently than floor(i/int32(N)) for N = 128 in the sample
below...
octave:1> i = 0:639;
octave:2> S = 5;
octave:3> N = 128;
octave:4> qinv = mod(i,N)*S + floor(i/N); min(qinv), max(qinv)
ans = 0
ans = 639
octave:5> qinv = mod(i,N)*S + floor(i/int32(N)); min(qinv), max(qinv)
ans = 0
ans = 640
This is atop vection 3.6.4.
Thanks,
Rhys
- Question re: floor of integer division,
Rhys Ulerich <=