Re: [igraph] reading large, sparse graph with read.graph

[Gábor Csárdi]

On Wed, Apr 28, 2010 at 12:42 AM, David Hunkins <address@hidden> wrote:
> So I am trying both of your suggestions (thanks):
>
> Tamas: I created a test file that's a subset of the very large file I'm
> working with. I had already converted it to Pajek format. The read.graph
> function reads it into memory in about 15 seconds. But it is taking a very
> long time to write out the lgl file (now thirty minutes already). The memory
> footprint doesn't change, and the cpu is maxed at 100%. I would think that
> the more 'compact' format could be written out quite fast, so I think
> something is wrong.

You are right, this is very slow if you have edge weights. I am not
sure why, we just call back to R to convert the numbers to strings and
then print them, but the conversion is very slow. I'll check this.

> system.time(g <- ba.game(1000, m=3))
   user  system elapsed
  0.000   0.000   0.002
> system.time(write.graph(g, file="/tmp/ba.lgl", format="lgl"))
   user  system elapsed
  0.004   0.000   0.003
> system.time(E(g)$weight <- runif(ecount(g)))
   user  system elapsed
  0.004   0.000   0.002
> system.time(write.graph(g, file="/tmp/ba.lgl", format="lgl"))
   user  system elapsed
 26.801   0.036  26.839

> ---------
>
> g <- read.graph("email_1m.csv_dg", format="pajek")
> write(proc.time(),file="")
> write.graph(g, "email_1m_lgl", format="lgl")  <- Hangs here
> ----------
>
> Gabor, I tried using graph.edgelist to create a graph from a matrix that was
> scanned from a file, and (consistent with the docs), edgelist doesn't want
> any weights; it only accepts two-column input. What did I miss?

What I mean is something like:

el <- matrix(scan("/tmp/edgelist"), nc=3, byrow=TRUE)
g <- graph( t(el[,1:2]) )
E(g)$weight <- el[,3]

Where the "/tmp/edgelist" file looks like this:
0 1 1.5
1 2 2.2
2 1 0.1
3 4 0.4
...

If you have an edge list format in your Pajek files, then it is very
easy to convert it to a file that you can just read in with 'scan', as
above.

Don't forget that vertex ids start with zero in igraph, so you might
need to subtract one from first two columns of the 'el' matrix.

Best,
Gabor

[...]


-- 
Gabor Csardi <address@hidden>     UNIL DGM