[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Maposmatic-dev] Features from the Code Sprint?

From: Jeroen van Rijn
Subject: Re: [Maposmatic-dev] Features from the Code Sprint?
Date: Wed, 15 Sep 2010 18:19:38 +0200

On Wed, Sep 15, 2010 at 17:10, Maxime Petazzoni <address@hidden> wrote:
Another solution would be read-only access to a kept up-to-date OSM

Not sure how this would be done nor its impact, just throwing the idea
out there for discussion.

This is just my thinking 'aloud', so to speak.

1. pgsql stores the database in e.g. /var/pgsql/ocitysmap
2. at 00:00, finish any current job if running, then shut down postgres
3. mv /var/pqsql/ocitysmap /var/pgsql/ocityold
4. mv /var/pgsql/ocitynew /var/pgsql/ocitysmap
5. restart pgsql and start processing jobs again
6. meanwhile at low io prio: rsync address@hidden:/somewhere /var/pgsql/ocityold
7. mv /var/pgsql/ocityold /var/pgsql/ocitynew

The other machine 'somehost' can keep the database up to date, just making sure it's in a known good state by the time the public server comes for its rsync.

That or you could have this other server be the initiator of the rsync update, putting a semaphore file on the maposmatic server when it's done; this public server would then upon seeing this file finish any running job, move those directories around, and restart postgres.

Then again, I'm not a postgres wizard, so perhaps postgres' native hot copy, failover, replication (et al) functionality would give similar or less load on the machine and be preferred. It could well be that doing the database upgrade on a 2nd machine and streaming replication to maposmatic as it's running is less of a load than it doing the update itself? For one the other machine would be running all of the queries an update requires, the maposmatic machine would be fielding mostly updates and inserts.

Perhaps some performance can be eked out of map rendering being primarily read only queries.

Another thought is to ask the authors of the update tool if it can delay the actual updating to near the end of the process and write all the updates at once in a single transaction. The database would be doing mostly reads until then and keep more rows around in cache that any rendering job at the time might make use of as well.

For all I know i'm talking absolute bollocks, but if someone better versed in postgres and mapnik comes away inspired, it's been a good use of 5 minutes for me. ;)

Also... have you thought to ask the guys at geofabrik how they keep things up to date? Is it solely throwing hardware at the problem, or did they do something clever with configuring both postgres and the update jobs too.


I got a fever, and the only prescription is more cowbell
- Bruce Dickinson

reply via email to

[Prev in Thread] Current Thread [Next in Thread]