Build your own OpenStreetMap Server - Ubuntu 10.04 Lucid Lynx
Submitted by rw on Fri, 03/26/2010 - 19:13.
Build your own OpenStreetMap server.
Build your own what?
OpenStreetMap is the editable World map of everything. It is the Wikipedia of maps. It is to other on-line maps as Wikipedia is to Britannica. And it is awesome in every possible way.
This article updates an earlier version, archived here. This current version was written in March 2010.The current version of this article is based on Ubuntu 10.04 Lucid Lynx. Lucid will be released on 29 April 2010 and be in beta until then. Operating systems in beta are not suitable for production use.
OpenStreetMap is a massive project that started as Steve Coast's, frame-breaking idea in 2004. If I make a map of my neighborhood and give it away, and you make a map of your neighborhood and give it away, then we both have better maps.
As of March 2009 there are over 100,000 contributors making maps of their neighborhoods
and contributing them to this World wide effort.
OpenStreetMap makes the data and the software available to you with Free Software and free data licenses so that you can use, learn from, teach with, improve upon and share with others what you gain from OpenStreetMap. And you can build your own local copy of OpenStreetMap for your business, school, community group or personal interests.
The project operates on a massive scale as there is an incredible amount of data, there is more data every day, and there are more people using the data every day. OpenStreetMap has to run on several servers, including a handful of API servers and separate database, development, web and tile servers. This article does not cover the creation of a complete OSM datacentre.
It does cover creating a single server
from a typical PC circa 2006. These instructions build what OpenStreetMap calls a tile server
. That is, a computer that uses the OSM data set to create map images that are suitable for a web site. Not every OpenStreetMap function is supported, but you will be able to create a local map, keep it up to date and customize it for your own purposes.
Why would I build my own?
Why indeed? OpenStreetMap.org is already freely available on the internet. Why not just use that? You can and you should. Eventually you may come up with an idea. You might want to make the map work a little differently for you. You might want a map for a special purpose.
Perhaps for cycling, http://www.opencyclemap.org/ OpenCycleMap is a wonderful example of what you can do with the tools and data of OpenStreetMap catalyzed by an idea. Created by Andy Allan and Dave Stubbs, OpenCycleMap uses OpenStreetMap data, then displays it in a way that is useful to cyclists with the emphasis placed on cycle trails, bike shops and bike parking. They've also added elevation contours and hill-shading as cyclists care about hills. Sometimes they are looking for a good challenging climb, and other times they just want to get home with the groceries. The brilliant work of the OpenCycleMap team was recognized with a Commendation from the British Cartographic Society as well as the prestigious Lolcat of Awesomness from the OpenStreetMap community at large.
Or maybe you need to have access to your map even when your internet provider is down. Or when the power is off. Or both. It won't take much for you to see the benefit of having your own piece of OpenStreetMap infrastructure. All you need to start is an idea and the thirst for knowledge.
There are a lot of moving parts to OpenStreetMap. I hope that these steps will make it easier for you to get your first map working. This article is intended to get from bare iron to a working local OpenStreetMap tile server. This one will collect OpenStreetMap data and allow you to render that data into images suitable for use on a web site. Future articles will cover how to use your new server and several customizations that you'll want to try.
Let's get started, shall we?
Prerequisites
Hardware requirements
Minimum hardware requirements for your tile server are modest by modern computer standards. You may find that you can run your tile server from a ~2005 desktop computer with more than 1GB of memory. But that is not recommended. The operations on large files and large databases suggest that you provide as much RAM as you can. Some OpenStreetMap developers use machines with 32GB or more of RAM. Many import and update operations will be disk-bound, so get the fastest disks possible for production use, and be aware that testing on typical disks will be substantially slower.
- Bare minimum hardware requirements
- RAM: 1 GB
- Disk: 250 GB
- Recommended minimum hardware requirements
- RAM: 4 GB
- Disk: 1 TB
- potential production hardware requirements
- RAM: 64 GB
- Disk: Many TB of 15,000 RPM RAID
Software configuration
Start with an installation of Ubuntu (10.04) Lucid Lynx Server for your hardware architecture. Add the LAMP server and SSH server during installation. Once installed, this can be run as a headless server with neither monitor nor keyboard. So ssh to the new box, and let's get started.
Update operating system
Get Ubuntu updates. Yes, you just installed Ubuntu, but it has been getting security and bug fixes. Use this to bring your system up to data as of right now.
sudo apt-get update
sudo apt-get upgrade
Get some system tools
We'll need subversion to get the latest updates from OpenStreetMap and other places. Munin makes pretty pictures of activity on the server. I like screen. htop is neat-o.
sudo apt-get install subversion autoconf screen munin-node munin htop
Organize the file system a bit
cd ~
mkdir src bin planet
Get the latest planet
- OpenStreetMap data file
A new planet
file is published approximately each week. The mirror and archives of the planet
files are here. In March 2010 the planet file was about 8.2GB in length. If you are not interested in the entire planet you can choose to download an extract file instead.
Facing an 8.2GB or larger download, this is an excellent time to consider using screen if you haven't used it before. Screen allows you to operate several terminal windows through one ssh connection. And more. Have a look this screen tutorial [Link removed.] if you haven't used it before. Or wait for your download to complete, or use another ssh session to continue.
cd planet
wget http://planet.openstreetmap.org/planet-latest.osm.bz2
Prepare the postGIS database
Use the PostGIS extensions to postgresql for all sorts of geographical goodness. Install the postGIS and prerequisites.
sudo apt-get install postgresql-8.4-postgis postgresql-contrib-8.4
sudo apt-get install postgresql-server-dev-8.4
sudo apt-get install build-essential libxml2-dev libtool
sudo apt-get install libgeos-dev libpq-dev libbz2-dev proj
Install osm2pgsql from the repository
The latest version of osm2pgsql has the most goodies, so we'll use that rather than a package.
cd ~/bin
svn co http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/
cd osm2pgsql
./autogen.sh
./configure
make
Configure the PostGIS database
edit /etc/postgresql/8.4/main/postgresql.conf in four places. These changes help with the large quantities of data that we are using.
shared_buffers = 128MB # 16384 for 8.1 and earlier
checkpoint_segments = 20
maintenance_work_mem = 256MB # 256000 for 8.1 and earlier
autovacuum = off
Edit kernel parameter shmmax to increase maximum size of shared memory.
sudo sysctl -w kernel.shmmax=268435456
sudo sysctl -p /etc/sysctl.conf
Restart postgres to enable the changes
sudo /etc/init.d/postgresql-8.4 restart
It should restart as above.
* Restarting PostgreSQL 8.4 database server ...done.
Create a database called gis
. Some of our future tools presume that you will use this database name. Substitute your username for username
in two places below. This should be the username that will render maps with mapnik.
sudo -u postgres -i
createuser username # answer yes for superuser
createdb -E UTF8 -O username gis
createlang plpgsql gis
exit
Set up PostGIS on the postresql database.
psql -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -d gis
This should respond with many lines ending with
... CREATE FUNCTION COMMIT ... DROP FUNCTION
Note: recent change requiring a new path to postgis.sql, is now reflected above.
Substitute your username for username
in two places in the next line. This should be the username that will render maps with mapnik.
echo "ALTER TABLE geometry_columns OWNER TO username; ALTER TABLE spatial_ref_sys OWNER TO username;" | psql -d gis
# Should reply with
ALTER TABLE ALTER TABLE
Set the Spatial Reference Identifier (SRID) on the new database.
psql -f ~/bin/osm2pgsql/900913.sql -d gis
Should reply with
INSERT 0 1
Load planet into the database with osm2pgsql
Your planet file will have a different date. Use an extract file if your interest is limited to a smaller portion of the planet. This operation will take 30 hours or longer. It is very I/O intensive and you can speed it up with very fast disks.
The osm2pgsql command that follows has several parameters. Because this process takes a long time to complete, it is worthwhile to take a moment to consider before starting.
- -S
- style file name.
- --slim
- Use slim mode. Do this. Trust me. Slim mode is required if you can't hold the complete node data in RAM during the import. Even if you have enough RAM and can hold all the node data, you probably want to use slim mode. Slim mode is required if you plan to later apply updates to your data base rather than reloading from scratch.
- -d
- data base name.
- -C
- RAM cache size in MB. Make this as large as you can.
- ~/filename.osm.bz2
- Location of your planet file.
cd ~/bin/osm2pgsql
./osm2pgsql -S default.style --slim -d gis -C 2048 --number-processes=1 --cache-strategy=dense ~/planet/planet-100217.osm.bz2
[note from 26 October 2011] - osm2pgsql development is currently aiming for performance improvement. The new parameters above, for --number-processes=1 --cache-strategy=dense
should be safe.
Loading the planet file will take some time. How much time it will take depends primarily on the speed of your hard drive system and on the configuration of your system and your available memory. For more details on tuning your Mapnik stack for better performance, see the SotM 2010 session and follow up by Frederik Ramm of GEOFABRIK.
In the interim, let's have a look at your planet import. The first part of the osm2pgsql output looks scary, but is normal.
Using projection SRS 900913 (Spherical Mercator) Setting up table: planet_osm_point NOTICE: table "planet_osm_point" does not exist, skipping NOTICE: table "planet_osm_point_tmp" does not exist, skipping Setting up table: planet_osm_line NOTICE: table "planet_osm_line" does not exist, skipping NOTICE: table "planet_osm_line_tmp" does not exist, skipping Setting up table: planet_osm_polygon NOTICE: table "planet_osm_polygon" does not exist, skipping NOTICE: table "planet_osm_polygon_tmp" does not exist, skipping Setting up table: planet_osm_roads NOTICE: table "planet_osm_roads" does not exist, skipping NOTICE: table "planet_osm_roads_tmp" does not exist, skipping Mid: pgsql, scale=100, cache=4096MB, maxblocks=524289*8192 Setting up table: planet_osm_nodes NOTICE: table "planet_osm_nodes" does not exist, skipping NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_nodes_pkey" for table "planet_osm_nodes" Setting up table: planet_osm_ways NOTICE: table "planet_osm_ways" does not exist, skipping NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_ways_pkey" for table "planet_osm_ways" Setting up table: planet_osm_rels NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_rels_pkey" for table "planet_osm_rels"
Don't be concerned by the NOTICE:
entries above. All normal.
Next, osm2pgsql will start reading the compressed planet file.
Reading in file: /home/nerd/planet/planet-100217.osm.bz2
As osm2pgsql reads the planet file it will give progress reports. The line below will refresh every few seconds and update the numbers in brackets. This part of the import takes a long time. Depending on your server, it will take between hours and days.
Processing: Node(10140k) Way(0k) Relation(0k)
As the import proceeds, the Node
number will update a couple of times per second until complete, then the Way
number will update not quite so quickly, roughly every second or two. Finally the Relation
number will update but at a slower rate, roughly once per minute. As long as you can see these numbers advancing the import process is still operating normally for your server. Do not interrupt the import process unless you have decided to start over again from the beginning.
Processing: Node(593072k) Way(45376k) Relation(87k) Exception caught processing way id=110802 Exception caught processing way id=110803 Processing: Node(593072k) Way(45376k) Relation(474k)
The exceptions shown above are due to minor errors in the planet file. The planet import is still proceeding normally.
The next stage of the osm2pgsql planet import process also will take between hours and days, depending on your hardware. It begins like this.
Node stats: total(593072533), max(696096737) Way stats: total(45376969), max(55410575) Relation stats: total(484528), max(555276) Going over pending ways processing way (752k)
The processing way
number should update approximately each second.
Going over pending relations node cache: stored: 515463899(86.91%), storage efficiency: 96.01%, hit rate: 85.97% Committing transaction for planet_osm_roads Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Sorting data and creating indexes for planet_osm_line Sorting data and creating indexes for planet_osm_roads Sorting data and creating indexes for planet_osm_polygon Committing transaction for planet_osm_point Sorting data and creating indexes for planet_osm_point Stopping table: planet_osm_nodes Stopping table: planet_osm_ways Stopping table: planet_osm_rels Building index on table: planet_osm_rels Stopped table: planet_osm_nodes Building index on table: planet_osm_ways Stopped table: planet_osm_rels Completed planet_osm_point Completed planet_osm_roads Completed planet_osm_polygon Completed planet_osm_line Stopped table: planet_osm_ways
This should mean that you import is complete and successful.
Install Mapnik library
The Mapnik library is the first of two items sometimes called Mapnik
. The other item is a collection of tools that OpenStreetMap uses to invoke Mapnik.
The official and up-to-date Mapnik Installation Instructions are here.
You might find that this procedure works as well.
Get some dependencies for building the Mapnik library.
sudo apt-get install libltdl3-dev libpng12-dev libtiff4-dev libicu-dev
sudo apt-get install libboost-python1.40-dev python-cairo-dev python-nose
sudo apt-get install libboost1.40-dev libboost-filesystem1.40-dev
sudo apt-get install libboost-iostreams1.40-dev libboost-regex1.40-dev libboost-thread1.40-dev
sudo apt-get install libboost-program-options1.40-dev libboost-python1.40-dev
sudo apt-get install libfreetype6-dev libcairo2-dev libcairomm-1.0-dev
sudo apt-get install libgeotiff-dev libtiff4 libtiff4-dev libtiffxx0c2
sudo apt-get install libsigc++-dev libsigc++0c2 libsigx-2.0-2 libsigx-2.0-dev
sudo apt-get install libgdal1-dev python-gdal
sudo apt-get install imagemagick ttf-dejavu
Build Mapnik library from source.
cd ~/src
svn co http://svn.mapnik.org/tags/release-0.7.1/ mapnik
cd mapnik
python scons/scons.py configure INPUT_PLUGINS=all OPTIMIZATION=3 SYSTEM_FONTS=/usr/share/fonts/truetype/
python scons/scons.py
sudo python scons/scons.py install
sudo ldconfig
Confirm that Mapnik library is installed.
python
>>> import mapnik
>>>
If python replies with the second chevron prompt >>>
and without errors, then Mapnik library was found by Python. Congratulations.
Install Mapnik tools
The Mapnik tools are the second item sometimes called mapnik
. This is a collection of tools from OpenStreetMap for making effective use of the Mapnik library.
cd ~/bin
svn co http://svn.openstreetmap.org/applications/rendering/mapnik
Install prepared world boundary data
Mapnik uses prepared files to generate coastlines and ocean for small scale maps. This is faster than reading the entire database to render zoom levels from zero to nine.
This section now includes the additional shape files that were added to OpenStreetMap default styles in mid-2010. Beware of the long, strange looking links with the repeated http. They are unlikely to copy / paste directly. Use copy link location or equivalent.
cd ~/bin/mapnik
mkdir world_boundaries
wget http://tile.openstreetmap.org/world_boundaries-spherical.tgz
tar xvzf world_boundaries-spherical.tgz
wget http://tile.openstreetmap.org/processed_p.tar.bz2
tar xvjf processed_p.tar.bz2 -C world_boundaries
wget http://tile.openstreetmap.org/shoreline_300.tar.bz2
tar xjf shoreline_300.tar.bz2 -C world_boundaries
wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/...
unzip 10m-populated-places.zip -d world_boundaries
wget http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/...
unzip 110m-admin-0-boundary-lines.zip -d world_boundaries
Render your first map
The database is loaded and the tools are installed. Let's test everything together. Remember to replace username
with your username.
cd ~/bin/mapnik
./generate_xml.py --dbname gis --user username --accept-none
./generate_image.py
View image.png to confirm that you have rendered a map of England. Congratulations.
References and credits
This tutorial is one of a series of tutorials for OpenStreetMap beginners. Find more tutorials here.
Thank you to the developers who wrote all of these Free Software tools.
Thanks to:
All of the helpful folks on #osm on irc.oftc.net especially jburgess, Ldp and springmeyer
All of the helpful folks on the OSM mailing list http://lists.openstreetmap.org/
http://wiki.openstreetmap.org/wiki/Mapnik
http://mapnik.org/
Thanks dbaker and acant for copy editing help on the previous version of this article.
Thanks nelson, codebrainz, Ldp, balrog-k1n and kW for more updates on the previous version of this article.
Thanks wnoronha for additional patches.
Lynx photo CCBYND Tambako the Jaguar on Flickr.
Articles © 2010 
Map images and data © 2010
help needed
sudo apt-get install postgresql-8.4-postgis postgresql-contrib-8.4, the script always fail at this portion
Hi marco, What was the error
Hi marco,
What was the error message? Which distro are you using? Which repositories have you enabled?
I do have the same problem,
I do have the same problem, it seems that the package does not exist. My error message is:
"E: Konnte Paket postgresql-8.4-postgis nicht finden"
Which means "could not find this package"
Same questions to you,
Same questions to you, Bikeman2000,
Which distro are you using?
Type this,
cat /etc/issue.netIt should reply,
Which repositories have you enabled?
Type this,
grep '^deb .* universe$' /etc/apt/sources.listIt should reply,
How about you?
postgresql-8.4
hi rw,
thanks for replying, I was kind of mess up, I was on ubuntu 9.10 and trying to following the instruction made for ubuntu 10, so I guess it was the reason it was not working, now I got the ubuntu 10 installed and following the procedure to run again, I will let you know how it comes out.
later
Marco
Hi marco, Even the previous
Hi marco,
Even the previous article was not for 9.10, but for 8.04, so you may find some differences.
Good luck,
I am using Ubuntu 9.10 and
I am using Ubuntu 9.10 and am surprised that this specific Postgresql 8.4 package is for 10.04 only. Currently I am trying to install postgresql with postgis.
I found postgresql 8.4 with postgis but the file "postgis.sql" ist missing.
Anybody knows a working command to install postgresql with postgis in Ubuntu 9.10 (I do not care if it is 8.3 or 8.4 or a developer version)?
Hi Bikeman2000, I don't have
Hi Bikeman2000,
I don't have a tutorial specific to building an OpenStreetMap tile server on Karmic 9.10, but I did find this link for you. It looks like Tom Wardill has a solution for you
It suggests that what you are seeing in Karmic is:- postgresql 8.4 is default
- postgis was not built for postgresql 8.4 in Karmic
- an automated installation won't work without help
http://blog.isotoma.com/2010/03/installing-postgis-on-ubuntu-karmic-koal...
If it works for you on Karmic, you should thank Tom.
Thanks this helped a lot.
Thanks this helped a lot. The psql-command worked but i don't have the file "_int.sql".
Forget the last comment, I
Forget the last comment, I reinstalled the postgis-packages from this tutorial and after that I had all the .sql files.
Currently I am importing the .osm file for my city into the database
Wonderful! Congratulations.
Wonderful! Congratulations.
update postgis data
I wish there is a article like this to go through to procedures of sync current database with latest data. I really have problem to do it when I followed instruction on http://wiki.openstreetmap.org/wiki/Minutely_Mapnik.
Please help
Thanks
This tutorial is very
This tutorial is very precise and comfortable since the reader only has to copy/paste the commands into the terminal.
But in two places the reader does not get the results because even a superuser does not have the rights to modify the files "/etc/postgresql/8.3/main/postgresql.conf" and "/etc/sysctl.conf"
First I started gedit and opened the files. Saving my edits was denied due to my userrights. After that I started gedit as a superuser and was surprised that I still did not have the rights to save the file.
Then I opened the folder with Nautilus and opened the file properties. I saw that I am not the owner of the file and I was not able to change the ownershipt there.
So I had to search the web for a solution and finally I found the "chown" command to change it.
Could you add the following lines to "edit /etc/postgresql/8.3/main/postgresql.conf in four places...":
sudo chown root /etc/postgresql/8.3/main/postgresql.confsudo gedit /etc/postgresql/8.3/main/postgresql.conf
and give the hint that the user has to close gedit or else he cannot continue in the terminal?
The same to "edit /etc/sysctl.conf":
sudo chown /etc/sysctl.confsudo gedit /etc/sysctl.conf
I'm not able to reproduce
I'm not able to reproduce your bug regarding gedit with either 9.04, 9.10 or 10.04 Beta1. I do see a couple of similar reports after a search, but those appeared to a) relate to a version in 8.04, or b) be fixed with
Check the following
- make sure your operating system is up to date. sudo apt-get upgrade; sudo apt-get update
- try another editor. Perhaps nano? sudo nano /etc/postgresql/8.3/main/postgresql.conf
Good luck
Finally it seems I made
Finally it seems I made it.
Only at "Render your first map" I got an error message:
PostGIS: SRID warning, using srid=-1Traceback (most recent call last):
File "./generate_image.py", line 37, in
mapnik.load_map(m,mapfile)
RuntimeError: PSQL error:
FEHLER: Relation »planet_osm_polygon« existiert nicht
LINE 3: from planet_osm_polygon
^
Full sql was: 'select * from
(select way,aeroway,amenity,landuse,leisure,man_made,military,"natural",power,shop,tourism,name
from planet_osm_polygon
where landuse is not null
or leisure is not null
or shop is not null
or aeroway in ('apron','aerodrome')
or amenity in ('parking','university','college','school','hospital','kindergarten','grave_yard')
or military in ('barracks','danger_area')
or "natural" in ('field','beach','heath','mud','wood')
or power in ('station','sub_station')
or tourism in ('attraction','camp_site','caravan_site','picnic_site','zoo')
order by z_order,way_area desc
) as leisure
limit 0'
(encountered during parsing of layer 'leisure')
I did not import the whole planet.osm and used an extract for my region. Did this cause the error?
hmmmm. "PostGIS: SRID
hmmmm.
"PostGIS: SRID warning, using srid=-1"
Make me wonder if you have missed this step?
psql -f ~/bin/osm2pgsql/900913.sql -d gis
"FEHLER: Relation »planet_osm_polygon« existiert nicht"
Makes me wonder about your extract import. Did it seem to import without error? How long did the import take?
You did not load data
I make same error. Forget to load data. Try this:
./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/YOUR_OSM_FILE.osm.bz2
Feedback
Hello !
Thanks for this very nice tutorial !
I did not achieve to generate properly a postgre/postgis database from a regional OSM extract last saturday, on 9.10.
This was mainly because I couldn't setup Postgis in my database
The day after, I found your blog :-)
I figured out that my initial choice of Postgis version (1.5.1) was wrong.
I reinstalled postgreSQL & Postgis packages per your tutorial and I got better results !
I have few comments though :
Same issue as for some comments above, I am new to linux / Ubuntu, and editing the conf files as root user was not that easy for me.
I finally used the command "sudo nano /etc/postgresql/8.3/main/postgresql.conf" and that allowed to save the modifications.
Use of osm2pgsql returned an error stating that postgreSQL was maybe not listening on port 5432.
I guess this is because access rights setting for localhost was not set to "trust", but "md5" instead.
(in /var/lib/pgsql/data/pg_hba.conf)
This is now fixed, and I can generate my database and perform SQL queries with pgadmin that return coherent results.
Thanks again !
Finally I did it under
Finally I did it under Ubuntu 9.10. I created the image of England and made a copy of the script which generated an image of my city. I will post my solution later.
THANK YOU VERY MUCH FOR THIS TUTORIAL
Yay! \o/
Yay! \o/
osm2pgsql uses autoconf now.
On my first try I only had 500 MB of ram so I built a new virtual server, this time with 8GB ram and also I used Ubuntu 10.04. I was following these new instructions when I came to building osm2pgsql. There was no configure file. Someone on the IRC said to use the autogen.sh, but that calls autoconf, so I had to install that.
sudo apt-get install autoconf
./autogen.sh
./configure
make
So far it's been great and I'm importing the OSM data now.
Thanks!
Thank you. Updated for
Thank you. Updated for autoconf in the article.
stuck?
Hi, I'm stuck on this step and am receiving the following error:
Can anyone help?
sudo apt-get install libtool
sudo apt-get install libtool
Thanks! libtool is now
Thanks! libtool is now added in the tutorial.
libboost woes
While the DB was importing I switched to another 'screen' (thanks for showing me that one) and started working on Mapnik prerequisites.
I cut and pasted from the libboost section and got:
E: Couldn't find package libboost1.41-dev
and more.
I checked the repositories as suggested above, and they were as shown.
IRC members suggested just using 1.40 so I'm working on it.
This would be a great item for a BitNami stack!
Thanks for all your help :)
Hmm, my installation showed
Hmm, my installation showed that libboost1.41-dev was up to date. When I remove boost and re-installed, I see your error. I've reverted this tutorial to boost 1.40 until I learn what is up with 1.41.
Thanks!
Great guide and right to the point. Miles better than anything else I've found so far. Thank you!
Thanks!
Just finished the first render of the UK - seems to work very well. Thanks again! Just a quick correction to the above - the generate_image.py produces an image.png, not image.py.
Thank you for the kind
Thank you for the kind words, and for the correction.
Installation with Ubuntu 8.04 and Postgre 8.3
Hi, everythign works fine until psql -f /usr/share/postgresql/8.3/contrib/postgis.sql -d gis
The file postgis.sql is just missing...
Everyting before went jsut as written in this tutorial. Does anyone have a clue what I could have been missing?
Thank you.
MW
Hi MW, Thanks for stopping
Hi MW,
Thanks for stopping by and having a look at this article.
You find that the previous version of this article is for installing an OpenStreetMap tile server on Ubuntu 8.04, the current version is for Ubuntu Lucid lynx, 10.04.
The postgis file name has changed since 8.04. From the archived article, try this
Set up PostGIS on the [Ubuntu 8.04] postresql database.
psql -f /usr/share/postgresql-8.3-postgis/lwpostgis.sql -d gisAnd that should move you along to the next step. Consider following the archived article. Lucid will be in beta for another couple of days as of this writing.
Thank you for answering so
Thank you for answering so quickly.
In fact I have found this other page: http://www.ioexception.de/2010/02/16/openstreetmap-rendering-mit-mapnik/
Which guided me to this other sql file.
Unfortunately I saw too late that I have to stop the maintenance script for postgresql as it kills the server process...
I am now trying again to import the osm into the db... wish me luck ;-)
PS: I have found this page for mod_tile
http://trac.openstreetmap.org/browser/applications/utils/mod_tile/readme...
Isn't there an easier way of having osm on the webserver?
Thx
MW
Still many errors
Hi,
unfortunately there are quite a few errors occuring while trying to insert the data into the database with this command:
./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/planet-100217.osm.bz2
2010-04-18 01:58:48 CEST HINWEIS: Tabelle »planet_osm_point« existiert nicht, wird übersprungen
2010-04-18 01:58:48 CEST HINWEIS: Tabelle »planet_osm_point_tmp« existiert nicht, wird übersprungen
2010-04-18 01:58:49 CEST HINWEIS: Tabelle »planet_osm_line« existiert nicht, wird übersprungen
2010-04-18 01:58:49 CEST HINWEIS: Tabelle »planet_osm_line_tmp« existiert nicht, wird übersprungen
2010-04-18 01:58:49 CEST HINWEIS: Tabelle »planet_osm_polygon« existiert nicht, wird übersprungen
2010-04-18 01:58:49 CEST HINWEIS: Tabelle »planet_osm_polygon_tmp« existiert nicht, wird übersprungen
2010-04-18 01:58:49 CEST HINWEIS: Tabelle »planet_osm_roads« existiert nicht, wird übersprungen
2010-04-18 01:58:49 CEST HINWEIS: Tabelle »planet_osm_roads_tmp« existiert nicht, wird übersprungen
2010-04-18 01:58:49 CEST HINWEIS: Tabelle »planet_osm_nodes« existiert nicht, wird übersprungen
2010-04-18 01:58:49 CEST HINWEIS: CREATE TABLE / PRIMARY KEY erstellt implizit einen Index »planet_osm_nodes_pkey« für Tabelle »planet_osm_nodes«
2010-04-18 01:58:49 CEST HINWEIS: Tabelle »planet_osm_ways« existiert nicht, wird übersprungen
2010-04-18 01:58:49 CEST HINWEIS: CREATE TABLE / PRIMARY KEY erstellt implizit einen Index »planet_osm_ways_pkey« für Tabelle »planet_osm_ways«
2010-04-18 01:58:49 CEST HINWEIS: Tabelle »planet_osm_rels« existiert nicht, wird übersprungen
2010-04-18 01:58:49 CEST HINWEIS: CREATE TABLE / PRIMARY KEY erstellt implizit einen Index »planet_osm_rels_pkey« für Tabelle »planet_osm_rels«
In the postgre logs there are many error messages saying that some columns are missing. Then after hours of importing I get error message and the import stops.
2010-04-18 03:22:26 CEST LOG: Background-Writer-Prozess (PID 31586) wurde von Signal 9 beendet: Killed
2010-04-18 03:22:29 CEST LOG: aktive Serverprozesse werden abgebrochen
2010-04-18 03:22:48 CEST WARNUNG: breche Verbindung ab wegen Absturz eines anderen Serverprozesses
2010-04-18 03:22:52 CEST DETAIL: Der Postmaster hat diesen Serverprozess angewiesen, die aktuelle Transaktion zurückzurollen und die Sitzung zu beenden, weil ein anderer Serverprozess abnormal beendet wurde und möglicherweise das Shared Memory verfälscht hat.
2010-04-18 03:22:53 CEST TIPP: In einem Moment sollten Sie wieder mit der Datenbank verbinden und Ihren Befehl wiederholen können.
2010-04-18 03:22:53 CEST ZUSAMMENHANG: COPY planet_osm_roads, Zeile 1
2010-04-18 03:22:53 CEST WARNUNG: breche Verbindung ab wegen Absturz eines anderen Serverprozesses
2010-04-18 03:22:53 CEST DETAIL: Der Postmaster hat diesen Serverprozess angewiesen, die aktuelle Transaktion zurückzurollen und die Sitzung zu beenden, weil ein anderer Serverprozess abnormal beendet wurde und möglicherweise das Shared Memory verfälscht hat.
2010-04-18 03:22:53 CEST TIPP: In einem Moment sollten Sie wieder mit der Datenbank verbinden und Ihren Befehl wiederholen können.
2010-04-18 03:22:53 CEST ZUSAMMENHANG: COPY planet_osm_rels, Zeile 1
2010-04-18 03:22:45 CEST WARNUNG: breche Verbindung ab wegen Absturz eines anderen Serverprozesses
2010-04-18 03:22:53 CEST DETAIL: Der Postmaster hat diesen Serverprozess angewiesen, die aktuelle Transaktion zurückzurollen und die Sitzung zu beenden, weil ein anderer Serverprozess abnormal beendet wurde und möglicherweise das Shared Memory verfälscht hat.
2010-04-18 03:22:54 CEST TIPP: In einem Moment sollten Sie wieder mit der Datenbank verbinden und Ihren Befehl wiederholen können.
2010-04-18 03:22:54 CEST ZUSAMMENHANG: COPY planet_osm_line, Zeile 1
2010-04-18 03:22:53 CEST WARNUNG: breche Verbindung ab wegen Absturz eines anderen Serverprozesses
2010-04-18 03:22:54 CEST DETAIL: Der Postmaster hat diesen Serverprozess angewiesen, die aktuelle Transaktion zurückzurollen und die Sitzung zu beenden, weil ein anderer Serverprozess abnormal beendet wurde und möglicherweise das Shared Memory verfälscht hat.
2010-04-18 03:22:54 CEST TIPP: In einem Moment sollten Sie wieder mit der Datenbank verbinden und Ihren Befehl wiederholen können.
2010-04-18 03:22:54 CEST ZUSAMMENHANG: COPY planet_osm_point, Zeile 1001077
2010-04-18 03:22:54 CEST WARNUNG: breche Verbindung ab wegen Absturz eines anderen Serverprozesses
2010-04-18 03:22:54 CEST DETAIL: Der Postmaster hat diesen Serverprozess angewiesen, die aktuelle Transaktion zurückzurollen und die Sitzung zu beenden, weil ein anderer Serverprozess abnormal beendet wurde und möglicherweise das Shared Memory verfälscht hat.
2010-04-18 03:22:54 CEST TIPP: In einem Moment sollten Sie wieder mit der Datenbank verbinden und Ihren Befehl wiederholen können.
2010-04-18 03:22:54 CEST ZUSAMMENHANG: COPY planet_osm_nodes, Zeile 207502220: »239224827 209015140 -739239709 \N«
2010-04-18 03:22:46 CEST WARNUNG: breche Verbindung ab wegen Absturz eines anderen Serverprozesses
2010-04-18 03:22:52 CEST DETAIL: Der Postmaster hat diesen Serverprozess angewiesen, die aktuelle Transaktion zurückzurollen und die Sitzung zu beenden, weil ein anderer Serverprozess abnormal beendet wurde und möglicherweise das Shared Memory verfälscht hat.
2010-04-18 03:22:55 CEST TIPP: In einem Moment sollten Sie wieder mit der Datenbank verbinden und Ihren Befehl wiederholen können.
2010-04-18 03:22:55 CEST ZUSAMMENHANG: COPY planet_osm_ways, Zeile 1
2010-04-18 03:23:41 CEST WARNUNG: breche Verbindung ab wegen Absturz eines anderen Serverprozesses
2010-04-18 03:24:02 CEST DETAIL: Der Postmaster hat diesen Serverprozess angewiesen, die aktuelle Transaktion zurückzurollen und die Sitzung zu beenden, weil ein anderer Serverprozess abnormal beendet wurde und möglicherweise das Shared Memory verfälscht hat.
2010-04-18 03:24:03 CEST TIPP: In einem Moment sollten Sie wieder mit der Datenbank verbinden und Ihren Befehl wiederholen können.
2010-04-18 03:24:04 CEST ZUSAMMENHANG: COPY planet_osm_polygon, Zeile 1
2010-04-18 03:24:05 CEST LOG: could not send data to client: Broken pipe
2010-04-18 03:24:05 CEST CONTEXT: COPY planet_osm_polygon, line 1
2010-04-18 03:24:05 CEST STATEMENT: COPY planet_osm_polygon (osm_id,"access","addr:flats","addr:housenumber","addr:interpolation","admin_level","aerialway","aeroway","amenity","area","barrier","bicycle","bridge","boundary","building","construction","cutting","disused","embankment","foot","highway","historic","horse","junction","landuse","layer","learning","leisure","lock","man_made","military","motorcar","name","natural","oneway","operator","power","power_source","place","railway","ref","religion","residence","route","service","shop","sport","tourism","tracktype","tunnel","waterway","width","wood","z_order","way_area",way) FROM STDIN
2010-04-18 03:24:06 CEST LOG: alle Serverprozesse beendet; initialisiere neu
2010-04-18 03:24:07 CEST LOG: Datenbanksystem wurde unterbrochen; letzte bekannte Aktion am 2010-04-18 03:20:05 CEST
I know it is full of german, but I guess that many terms in english explain what the issue is...
I will start all over again, just not sure what could have gine wrong.
The installation of Mapnik was successfull though.
Thx for helping.
PS: unfortunately it is not possible to reply or comment on the Ubuntu 8.04 install howto...
MW
All worked well!
Thanks for this - I got it all up and running very easily. The only part I got stuck was when I wanted to generate all the tiles, when I ran ./generates_tiles.py I kept getting the error that /home/alex/svn.openstreetmap.org/applications/rendering/mapnik/osm-local.xml couldn't be found. But I got this fixed by editing the mapfile =... line in generate_tiles.py to point to /home/alex/osm/bin/mapnik/osm.xml instead and then all worked fine. Is there a better way of setting 'MAPNIK_MAP_FILE' variable? (I'm very new to python).
Hi Alex, You probably sussed
Hi Alex,
You probably sussed it by now, but here goes...
MAPNIK_MAP_FILE="osm.xml" MAPNIK_TILE_DIR="tiles/" ./generate_tiles.py
Found it here: http://wiki.openstreetmap.org/wiki/Mapnik
Cheers
Where am I to create dirs ?
Thanks for this tutorial I will use.
Just a little question... I will put mapnik on my computer. And I don't know where I must create the folders with the command :
mkdir src bin planetThank you for your reply that, I'm shure, will come soon.
I've been putting all of
I've been putting all of this in the /home/username directory. I'll add a hint in the article.
mod_tile
Great blog post, so much easier to follow than the wiki page and I got everything up and running without a hitch,
Are you going to do another post on getting mod_tile running? :)
xml2-config missing
Under Karmic Koala, I had the error message
"Exiting... the following required dependencies were not found:
- xml2-config (xml2-config program | try setting XML2_CONFIG SCons option)"
when building mapnik.
I solved it by installing libxml2-dev :
sudo aptitude install libxml2-dev
Best regards
openstreetmap server for Red hat enterpise linux or Suze
Hello Friends,
Could anybody please tell me whether it is possible to configure openstreetmap tile server on Red Hat Enterprise Linux or Suze O.S?
Any help would be greatly appreciated.
Thanks a lots!
Balram
Hello Balram, Sure, it
Hello Balram,
Sure, it should be possible. I did a partial installation on RHCE 5.4 but went back to Debian / Ubuntu. Too many prerequisite packages were only available via external repositories. Not sure about SUSE. YMMV.
FYI.
you can combine lines like:
bunzip2 shoreline_300.tar.bz2
tar xvf shoreline_300.tar
into: tar xvfj shoreline_300.tar.bz2
no reason to run 2 different commands.
Problem
HI,
I find your blog very usefull since I'm trying to accomplish the exact same thing. However I do get an error.
I want to import the data with the latlong option, since I need to see the coordinates, but when I add the -l I get the following error:
Using projection SRS: 4326(LatLong)
table planet_osm_point does not exist, skipping
table planet_osm_point_tmp does not exist, skipping
SELECT AddGeometryColumn('planet_osm_point', 'way', 4326, 'POINT', 2 );
failed: ERROR: AddGeometryColumns() - invalid SRID
CONTEXT: SQL statement "SELECT AddGeometryColumn('','', $1 , $2 , $3 ,
$4 , $5
)"
PL/pgSQL function "addgeometrycolumn" line 5 at SQL statement
When i don't use the -l everything works fine. Any idea's on how to fix that?
DejaVu fonts
Thanks a lot for the tutorial
I had to install the ttf-dejavu package for rendering the image
Only ttf-dejavu-core was installed
And how do we implement
And how do we implement search on the osm db?
Hi anonymous, thanks for
Hi anonymous, thanks for coming by. Weren't you here before? Maybe it was somebody else with the same name. :-)
How do you search on the OSM database? That depends on what you are doing. As a user, I often just use the OpenStreetMap search box, on the left side of the page at http://openstreetmap.org/
On my local tile server I usually write a postgres or PostGIS query.
I might use XAPI.
For .osm files I use grep or sometimes sed if I'm feeling a bit saucy.
If I had a bunch of clients wanting location search that 'just works', I suppose I would set up a local instance of the amazing Nominatim, by Twain.
But really it depends on too many things to just say, "Do this specific thing." Tell me more?
Load planet into database
On "Load planet into the database with osm2pgsql" I got an error message along the lines of
Connection to database failed... Is the server running locally and accepting connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"
Diagnosis, run ls -la /var/run/postgresql and you see a line like .s.PGSQL.5433. Eeek, did you notice it complains about 5432 but has 5433?
Solution, specify the port by changing the command to be ./osm2pgsql -S default.style --slim -d gis -C 2048 -P 5433 ~/planet/planet-100217.osm.bz2.
These instructions are great Richard, thanks.
I'm glad you got that sorted
I'm glad you got that sorted out, Gregory! Any idea why you were running postgres on port 5433? Did you have two postgres servers running or something?
I also run into this
I also run into this socket-domain problem and you provided me with the right solution, so thanks! However, I also had to add -H localhost to the code.
And also thanks for the writer of this article, very helpful!
GEOS via SVN results in OSM2PGSQL-Make-error
First let me thank you for this profound description, which is straight forward to a working result.
I am new to the OSM subject and doing my first steps so far.
For several reasons I would like to use both the GEOS library and the OSM2PGSQL library via SVN. However when I start MAKE for OSM2PGSQL it results into one error (obviously raised by the GEOS library). It might be a missing INCLUDE-header or something different, which is easy to fix by somebody who is familiar with compiling from the source - but difficult for a newbie.
Is there any chance to use the GEOS-SVN library rather than the (older) package? Did anybody already try?
Kind appreciation,
Kartograph
Oops, my last comment got
Oops, my last comment got stuck (unless it's in moderation?).
I had problems with "Load planet into the database with osm2pgsql" and so changed the command to specify the port.
./osm2pgsql -S default.style --slim -d gis -C 2048 -P 5433 ~/planet/planet-100217.osm.bz2
generate_image.py will also fail. I'm working out if I need to edit that or if there is a way I can allow the server to accept port 5432.
Port problems in postgres?
Port problems in postgres? Interesting. Gregory reported a similar problem above. I wonder what has changed?
What do you get from this?
grep -R "postgres" /etc/servicesI have:
And from postgresql.conf?
grep "port" /etc/postgresql/8.4/main/postgresql.confI have:
How do Updates work? I do
How do Updates work? I do have an openstreetmap installation of Europe and would like to upgrade now to world. Is it enough to just import the planet with osm2pgsql?
Yes. If you use osm2pgsql
Yes. If you use osm2pgsql to import the planet file you'll drop the old Europe database tables and start from zero with the planet file.
Ideas?
Thanks for the very well done tutorial. I am looking to implement an api to querry tiles from my local server. I'd also like to implement a similar functionallity to the OSM server where a user can querry my tile server via an http request. Do you have any suggestions? Thanks much!
XAPI
How can you attach XAPI to this local server?
SRID - psql -f ~/bin/osm2pgsql/900913.sql -d gis
Sir
I have run all steps successfully except the following one,which in turns has stopped me in installing osm server,
Set the Spatial Reference Identifier (SRID) on the new database.
#psql -f ~/bin/osm2pgsql/900913.sql -d gis
This command is giving the following problem:
/var/lib/postgresql/bin/osm2pgsql/900913.sql: No such file or directory
what is the reason of it..only this is not working..
Help me as soon as possile...
Thank's in advance.
Hi Parveen, Welcome. Use
Hi Parveen,
Welcome.
Use locate 900913.sql to find your location for the SRID file. Then use your location in the psql command.
mod_tile and update/ expiry?
Hi Richard,
this is a really neat tutorial and I often like to refer people to it to explain how to set up a tile server as it is the best documentation I have seen so far!
Now it would be really cool, if you could extend the tutorial to also include the setting up of mod_tile and renderd (potentially in a separate article). The iceing on top of the cake would of cause be if it even included an explanation of keeping the db up to date and expire updated tiles.... :-)
Thanks again
P.S. it would also be good if you could rename the article to "Build your own openstreetmap _tile_ server" to distinguish it from the rails_port server.
Hi apmon, and thanks! I
Hi apmon, and thanks!
I like the idea of adding an article on mod_tile, renderd and tirex. Plus minutely mapnik and expiry. One of these days! Anybody with a supply of extra 'Round 'Tuits would be nice. ;-)
Your suggestion on renaming for clarity is a good one. I'll put that renaming off until the next revision. I fear breaking incoming links.
generate_image.py creates bad png file
We setup OSM accrding to this article... when we run generate_image.py the resulting png wont open. Any ideas what might be wrong?
Thanks
Hi Kent, What error message
Hi Kent,
What error message do you see? Is any .png generated, or just a 0-length file?
update and thanks
*update needed*
It seems that now two more files are needed:
http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/...
http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/...
Done that...
... it works! Thanks for this guide :-)
Ciao
RuntimeError: 110m_admin_0_boundary_lines_land does not exist
RuntimeError: d:/SVN/mapnik/world_boundaries/110m_admin_0_boundary_lines_land does not exist (encoun
tered during parsing of layer 'necountries')
Cause: two required shapefiles are missing.
Workaround: edit layer-shapefiles.xml.inc and remove the necountries layer
OR
Download the shapes :-) ( thnx to VSK http://forum.openstreetmap.org/viewtopic.php?pid=85606 )
10m-populated-places.zip
110m-admin-0-boundary-lines.zip
I have updated the article
I have updated the article to include the additional shape files.
Let's leave your work around here, just in case.
The urls there got
The urls there got obfuscated some way. would be nice if you can fix this, so it's possible to use c&p
no need to make generate_xml.py executable now
fyi: http://trac.openstreetmap.org/changeset/22069
Error on last command
Thank you first for this superbly written guide.
I was able to follow it right up to the very last step, then error hits:
mn@IOSOSM:~/bin/mapnik$ ./generate_image.py
Traceback (most recent call last):
File "./generate_image.py", line 37, in
mapnik.load_map(m,mapfile)
RuntimeError: /home/mn/bin/mapnik/world_boundaries/110m_admin_0_boundary_lines_land does not exist (encountered during parsing of layer 'necountries')
Maybe you could give me a hint on what might have gone wrong.
Thank you.
Thank you, streitberg.
Thank you, streitberg.
Check the "Install prepared world boundary data" section again. I have updated it to include the additional shape files.
thanx
Thanx a lot for the excellent manual.
I was able to generate my own cyclemap for Belgium.
Thanx to Holland! http://git.openstreet.nl/ (hartelijk dank dat jullie dit ter beschikking stellen!)
I incorporated their overlay into osm.xml.
I managed to get tile-server up and running on localhost.
Copied it with Tangogps for off-line use :)
Thanks for the tutorial
Thanks for these steps! I believe this is the first Linux tutorial that I followed which actually works! Too bad I'm using a virtual machine with slow disks, so it's been importing the data for the last two days now ;-)
Anyone remember how many ways and relations it has to import? It's at "Node(704211k) Way(18843k) Relation(0k)" at the moment...
Don't find osm2pgsql
if i have the instruction ./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/planet-100217.osm.bz2 it doesn't work, say's me "it doesn't exist osm2pgsql, why happen this, i follow all step's right.
if you can help me im very thnxs.
Hi jcervantes and thank you
Hi jcervantes and thank you for writing.
The says to use the osm2pgsql command and to find the osm2pgsql command in the . So this will only work if you are in the directory.
Alternatively, you could specify the full path to osm2pgsql, and the default.style, like this
~/bin/osm2pgsql -S ~/bin/osm2pgsql/default.style --slim -d gis -C 2048 ~/planet/planet-100217.osm.bz2Error Osm_polygon
Hi thanks for answer but i have another fail
when i'm follow the next step
( ./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/planet-100217.osm.bz2 )
send me this error
Using projection SRS 900913 (Spherical Mercator)
Setting up table: planet_osm_point
NOTICE: la tabla «planet_osm_point» no existe, ignorando
NOTICE: la tabla «planet_osm_point_tmp» no existe, ignorando
Setting up table: planet_osm_line
NOTICE: la tabla «planet_osm_line» no existe, ignorando
NOTICE: la tabla «planet_osm_line_tmp» no existe, ignorando
Setting up table: planet_osm_polygon
NOTICE: la tabla «planet_osm_polygon» no existe, ignorando
NOTICE: la tabla «planet_osm_polygon_tmp» no existe, ignorando
Setting up table: planet_osm_roads
NOTICE: la tabla «planet_osm_roads» no existe, ignorando
NOTICE: la tabla «planet_osm_roads_tmp» no existe, ignorando
Mid: pgsql, scale=100, cache=2048MB, maxblocks=262145*8192
Setting up table: planet_osm_nodes
NOTICE: la tabla «planet_osm_nodes» no existe, ignorando
NOTICE: CREATE TABLE / PRIMARY KEY creará el Ãndice implÃcito «planet_osm_nodes_pkey» para la tabla «planet_osm_nodes»
Setting up table: planet_osm_ways
NOTICE: la tabla «planet_osm_ways» no existe, ignorando
NOTICE: CREATE TABLE / PRIMARY KEY creará el Ãndice implÃcito «planet_osm_ways_pkey» para la tabla «planet_osm_ways»
Setting up table: planet_osm_rels
NOTICE: la tabla «planet_osm_rels» no existe, ignorando
NOTICE: CREATE TABLE / PRIMARY KEY creará el Ãndice implÃcito «planet_osm_rels_pkey» para la tabla «planet_osm_rels»
then when i generate image send me the next error
PostGIS: SRID warning, using srid=-1
Traceback (most recent call last):
File "./generate_image.py", line 37, in
mapnik.load_map(m,mapfile)
RuntimeError: PSQL error:
FEHLER: Relation »planet_osm_polygon« existiert nicht
LINE 3: from planet_osm_polygon
^
Full sql was: 'select * from
(select way,aeroway,amenity,landuse,leisure,man_made,military,"natural",power,shop,tourism,name
from planet_osm_polygon
where landuse is not null
or leisure is not null
or shop is not null
or aeroway in ('apron','aerodrome')
or amenity in ('parking','university','college','school','hospital','kindergarten','grave_yard')
or military in ('barracks','danger_area')
or "natural" in ('field','beach','heath','mud','wood')
or power in ('station','sub_station')
or tourism in ('attraction','camp_site','caravan_site','picnic_site','zoo')
order by z_order,way_area desc
) as leisure
limit 0'
(encountered during parsing of layer 'leisure')
I think thas is because the first error, but i don't now. please if you can tell me y have very thnks.
Hi jcervantes, I've added
Hi jcervantes,
I've added more detail to the portion of the tutorial that deals with importing the planet file. Are you allowing the planet import step to complete?
Como acceder desde mi navegador al servicio montado de OSM?
Saludos...
Segui el tutorial para montar el servidor OSM, ya me genera el mapa de ejemplo: imagen.png, pero ahora quiero probarlo para que me aparezca el mapa desde el navegador... Como hago esto?
How to access my server OSM from the browser web?
Hi!
I followed the tutorial for mounting the OSM server, I generated the map example: imagen.png, but now I try to bring up the map from the browser ... As I do this?
Hi. Thanks you for writing.
Hi. Thanks you for writing. This tutorial does not address putting OSM on the web. With some requests and enough time, I'll write a tutorial for that.
Disk Space?
Hi,
Do you have any idea how much diskspace the import will use? I've just found that my previous attempt stopped because of a lack of diskspace... I'm using a Linux VM under Hyper-V so I can only address 137GB.
Dear Jelmer, Ah, you have a
Dear Jelmer,
Ah, you have a potential problem then. The planet download is around 10GB now (August 2010). The database will be on the order of 100 - 120GB depending on your import settings in osm2pgsql. That will be close enough to your limits without considering the size of your OS image and the rendering stack.
This gets worse for you as the database will be larger almost every week.
And if you implement database updates your local database will grow faster than the raw planet file version of the database.
Good luck.
Hi and thanks for the great
Hi and thanks for the great Tutorial, however i have a question, what did you mean the next Parameters?
z = 5
imgx = 100 * z
imgy = 100 * z
I need build diferent size of my map but i don't understand how it's the dimensions right to generate the images for upload my maps to WSM.
generate_image.py is for
generate_image.py is for individual images of arbitrary dimensions and scale. Use generate_tiles.py to generate tiles.
Great tutorial ..... Bombed
Great tutorial ..... Bombed out after 1 day as my disk was only 80GB ..... I see the comment now about needing at leave 120GB
Perhaps you should add a disk size pre-requisit to the top of the article?
Sorry to hear about the set
Sorry to hear about the set back.
Thank you for the suggestion. I've added some hardware requirements.
Hi, I installed the whole
Hi,
I installed the whole server and it seems to work.
But now, how can i generate specific maps of given region ?
Will there way a way too access an generate data on localhost ?
thanks
PROBLEM: Server worked fine, but now is generating empty dirs
Hi,
your tutorial is great and worked fine for the first week, but now i can only generate tiles from level 1 - 9,
for level 10 - 18 i got only empty directories
This are my rendering options:
#-------------------------------------------------------------------------
#
# Change the following for different bounding boxes and zoom levels
#
# Start with an overview
# World
bbox = (16.26, 48.25, 16.48, 48.14)
minZoom = 1
maxZoom = 18
render_tiles(bbox, mapfile, tile_dir, minZoom, maxZoom, "MAP")
i read in comments about some problems with bbox, but i double checked, my bbox is correct
i also generated an image with ./generate_image.py for the same bbox, and everything worked fine
here is the output i got from, ./generate_tiles.py:
standard@mapper:~/bin/mapnik$ ./generate_tiles.py
render_tiles( (16.260000000000002, 48.25, 16.48, 48.140000000000001) /home/standard/bin/mapnik/osm.xml /home/standard/bin/mapnik/tiles/ 1 18 MAP )
Started render thread Thread-1
Started render thread Thread-2
Started render thread Thread-3
Started render thread Thread-4
MAP : 4 8 5
MAP : 3 4 2
MAP : 2 2 1
MAP : 5 17 11
MAP : 1 1 0
MAP : 6 34 22
MAP : 9 279 177
MAP : 8 139 88
MAP : 7 69 44
standard@mapper:~/bin/mapnik$
i hope somebody can help me
BR
Sky
Your bounding box looks a
Your bounding box looks a little strange to me. It appears to be this area? http://www.openstreetmap.org/?lat=16.26&lon=48.25&zoom=10&layers=M
Also, I've used this order for bounding boxes in Mapnik with success. (West, South, East, North)
Did it all work ?
Did it all work ?
So, after over a week processing (on a fairly old PC) my setup is complete.
When running the processing the counts finished at:
Processing: Node(746179k) Way(60926k) Relation(699k)
... Followed by.....
Going over pending ways
processing way (259144k)
I would have thought that the 2nd way count would have matched the first one ?
Anyway... on to the final step : Rendering the map :
I'm getting the folowing error, having tried all the different user's i can think of :
ubuntu@devserver:~/bin/mapnik$ ./generate_xml.py --dbname gis --user ubuntu --accept-none
Include files written successfully! Pass the osm.xml file as an argument if you want to serialize a new version or test reading the XML
ubuntu@devserver:~/bin/mapnik$ ./generate_image.py
Traceback (most recent call last):
File "./generate_image.py", line 37, in
mapnik.load_map(m,mapfile)
RuntimeError: PSQL error:
FATAL: Ident authentication failed for user "ubuntu" (encountered during parsing of layer 'leisure')
Any hints greatly appreciated !
Ro
Not able to run _int.sql script.
Hi to All.
Thank to all who have developed such a bunch of tools and put them together to work.
I need Help here.
We are trying to build OSM server on Debian Squeeze.
After successful installation and configuration of postgis 1.5 after postgresql 8.4.
I am configuring it all as per given here in document and all things were fine, but.
I can run postgis.sql script successfully from the postgres user, then when i try to run _int.sql script it gives me following errors.
I tried to run this command from root with -U postgres options, that too gives same errors. In postgis.sql script same variable $libdir is used that works but in _int.sql script that $libdir does not work. I have pg_config from /usr/bin.
Using
pg_config --libdir , i get /usr/lib
and
pg_config --pkglibdir, i get /usr/lib/postgresql/8.4/lib
Please suggest us solution, Thank you very much.
$libdir problem in _int.sql can be resolved by trick
Hello All
Today we have solved errors by a simple trick.
As we can see errors are related to some shared library starts with _int, actually _int.so is exact one.
we knew that /usr/lib/postgresql/8.4/lib does contain that shared libraries ,
but _int.sql script is not able to read it from $libdir.
our $libdir , shown by pg_config --libdir shows /usr/lib which does not contain such shared libraries which are required
so I replaced $libdir inside /usr/share/postgres/8.4/contrib/_int.sql script with /usr/lib/postgresql/8.4/lib
Following link give us hint to solve the error from postgresql official site.
http://www.postgresql.org/docs/8.4/static/xfunc-c.html
Thank you to all for support.
Hello
ya i am just sucessfull to produce the england png image...
but when i run "./osm2pgsql -S default.style --slim -d gis -C 2048 ~/planet/planet-100217.osm.bz2" this command again.....
there is an error produced every time when i run "./generate_inage.py"...
the error is ----->>>>>
Traceback (most recent call last):
File "./generate_image.py", line 37, in
mapnik.load_map(m,mapfile)
RuntimeError: PSQL error:
ERROR: relation "planet_osm_polygon" does not exist
LINE 4: from planet_osm_polygon
^
Full sql was: 'select * from
(select way,aeroway,amenity,landuse,leisure,man_made,military,"natural",power,tourism,name,
case when religion in ('christian','jewish') then religion else 'INT-generic'::text end as religion
from planet_osm_polygon
where landuse is not null
or leisure is not null
or aeroway in ('apron','aerodrome')
or amenity in ('parking','university','college','school','hospital','kindergarten','grave_yard')
or military in ('barracks','danger_area')
or "natural" in ('field','beach','heath','mud','wood')
or power in ('station','sub_station','generator')
or tourism in ('attraction','camp_site','caravan_site','picnic_site','zoo')
order by z_order,way_area desc
) as leisure
limit 0'
(encountered during parsing of layer 'leisure')
i completely remove & install all the files again but there is same problem occurs......
plz help meeeeeeeeeeeeeeeeeeeeeee..
thanks you in advance
update openstreetmap postgres database
Hi,
I used ubuntu 10.4 server to installed the open streetmap. I used planet-100811.osm.bz2 to build the database. and it was working great. however, I was not able to create a script to sync this database with latest data.
I used this link http://wiki.openstreetmap.org/wiki/Minutely_Mapnik to guide me to the update procedures and I failed.
would you please kindly provide a step by step procedures to help me sync my database with latest open street map?
Your helps are greatly appreciated.
Marco
what about a smaller map?
Is there any way to restrict the imported area? I don't really need the entire planet, just some 150 sq kms. I can obviously delete the objects in excess from postgis after loading them, but maybe there's a way to filter the import.
Welcome, Bèrto. There are
Welcome, Bèrto.
There are several ways to select a smaller portion of the planet. Some folks publish that correspond to a country or region. Find out more about extracts here: http://weait.com/content/openstreetmap-extracts
For smaller, rectangular areas, you can export the data from the tab on the OpenStreetMap web site. See this article and choose OpenStreetMap XML Data, then save. http://weait.com/content/map-tiles-and-bounding-boxes
Good luck!
autogen.sh error
I'm not sure if it was just my installation (Ubuntu 10.10 running in a VM), but I received errors about AM_PROG_LIBTOOL not being found. However, installing libtool fixed this.
On to the next step!
problem with ./generate-image.py
i have this error :
Traceback (most recent call last):
File "./generate_image.py", line 37, in
mapnik.load_map(m,mapfile)
RuntimeError: PSQL error:
FATAL: authentification Ident ?chou?e pour l'utilisateur << postgres >> (encountered during parsing of layer 'leisure')
somone know what happen ?
postgis.sql not there
Hi
Thanks for all your work. I'm trying to set up a server on Ubuntu 10.04 and got this far...
Following your directions and no problems until the step:
psql -f /usr/share/postgresql/8.4/contrib/postgis.sql -d gis
I found a file named postgis.sql in /usr/share/postgresql/8.4/contrib/postgis-1.5/ and tried that but it did not end with
CREATE FUNCTION
COMMIT
It does end with CREATE FUNCTION, COMMIT, DROP AGGREGATE, DROP AGGREGATE, DROP AGGREGATE, DROP AGGREGATE, and about 20 DROP FUNCTIONs. So I'm wondering if this is the wrong flavor of postgis.sql? It may have been left around from an earlier install attempt that failed/aborted, not sure.
Thanks
Eric
Thanks! I've revised the
Thanks! I've revised the location of postgis.sql in the main article.
Were you right the first time?
On my setup (Ubuntu 10.04.2), the file postgis.sql is located at /usr/share/postgresql/8.4/contrib/postgis.sql, which judging by Eric's comment is how the tutorial had it originally. This location is defined by the package postgresql-8.4-postgis (checking with dpkg -L postgresql-8.4-postgis).
When I run the psql line, I get a lot of output ending in
...
CREATE FUNCTION
COMMIT
(not DROP FUNCTION as mentioned in the tutorial)
but it seems it is still all OK and working fine.
Another minor comment is that it isn't all that clear in the tutorial which commands have to be run by postgres (or a privileged postgres user?), and what to do if your username isn't a postgresql user. I found I had to run the above psql -f command as user postgres and for the next command (echo "ALTER TABLE...), I had to make my username (alex) a postgresql user. So I did (as user postgres)
createuser -s alex
echo "ALTER TABLE geometry_columns OWNER TO alex; ALTER TABLE spatial_ref_sys OWNER TO alex;" | psql -d gis
But these things are not very important and your tutorial is really good, clear and detailed. It would have taken me ten times longer with a normal style of tutorial. Thank you.
mod_tile
Hi,
I liked your tutorial ...its very helpful...But its 95 % ...to add another 5% please please write a tutorial for mod_tile.I have spent over a week in installing/configuring mod_tile but no success..please write a tutorial for mod_tile for Ubuntu.
Thank you so much.
waiting for a good tutorial.
Is procps service really needed
I have problem with starting the procps service on lucid in VMWare Server 1.10 guest. Job not started. The syslog gives only a vague explanation "procps main process (some number) terminated with status 255". I really cannot find the reason.
I was wondering whether the procps is really needed?
Wonderful, works great. This
Wonderful, works great. This will save hundreds of hours of downloads. I would love to do something like opencyclemap but for aviation maps..
this is what the Internet is all about.
mod_tile
I don't know if this is helpful, but some people above were asking about mod_tile, so here are some comments about how to get mod_tile working. (Or at least, how I got it working. I think what I did was a bit messy and probably not good in various ways, but hopefully still helpful. Eventually I decided that I would prefer to do what I wanted to do by modifying the program ~/bin/mapnik/generate_tiles_multiprocess.py instead of using mod_tile.) This is written with the same file arrangement as in the main tutorial, with everything at the top level of your user directory.
sudo apt-get install apache2-threaded-dev
cd
svn co http://svn.openstreetmap.org/applications/utils/mod_tile
cd mod_tile
make
(You need apache2-threaded-dev or apache2-prefork-dev to get apxs2.)
At this point the best approach would probably be to build a debian package using the information in the directory debian (after adjusting a few things). This would cause useful things to happen, like file ownerships to be changed, the apache2 module mod_tile to be set up, some directories and a suitable /etc/init.d file to be created etc., but I don't know how to do this, and I was just trying out mod_tile as a one-off, so I've proceeded manually.
Edit renderd.conf to be
[renderd]
socketname=/var/run/renderd/renderd.sock
num_threads=4
tile_dir=/var/lib/mod_tile ; DOES NOT WORK YET
stats_file=/var/run/renderd/renderd.stats
[mapnik]
plugins_dir=/usr/local/lib/mapnik/input; lib would be lib64 in a 64 bit environment
font_dir=/usr/share/fonts/truetype/ttf-dejavu
font_dir_recurse=1
[default]
URI=/osm/
XML=/home/<username>/bin/mapnik/osm.xml
HOST=localhost
replacing <username> with your username. Then
sudo make install
changing /usr/local/lib to /usr/local/lib64 if you are in a 64-bit environment.
Note that if you decide to alter renderd.conf after doing make install you will need to delete /etc/renderd.conf before doing another make install (or just copy the new renderd.conf to /etc/renderd.conf directly), because there is a missing dependency in the Makefile. (This had me confused for a while.)
Edit mod_tile.conf: (i) change the ServerName to localhost, (ii) delete the ServerAlias lines, (iii) delete the LoadModule line (we'll put it in a separate file according to the usual conventions). Then (replacing <username>)
sudo cp debian/tile.load /etc/apache2/mods-available/
sudo cp mod_tile.conf /etc/apache2/mods-available/tile.conf
sudo a2enmod tile
sudo mkdir /var/run/renderd /var/lib/mod_tile
sudo chown <username>:<username> /var/run/renderd /var/lib/mod_tile
sudo service apache2 restart
./renderd -f 2> renderlogfile &
(In the end you'd probably want to run renderd as a daemon, and also not as <username>, but this way is convenient as you can see the error messages.)
To check mod_tile (not renderd): point your browser at http://localhost/mod_tile and see that it gives you some statistics.
To check renderd: go to http://localhost/osm/0/0/0.png (or whatever tile you want that it is in the database).
Useful logfiles are those in /var/log/apache2, ~/mod_tile/renderlogfile and /var/run/renderd/renderd.stats. renderd also appears to dump lots of stuff in /var/log/syslog, /var/log/debug, /var/log/daemon.log
Useful references are bits of ~/mod_tile/readme.txt and http://wiki.openstreetmap.org/wiki/Mod_tile
These (German) describe an alternative method, using mod_tile/Tirex (instead of mod_tile/renderd): http://wiki.openstreetmap.org/wiki/DE:HowTo_minutely_hstore and http://wiki.openstreetmap.org/wiki/DE:HowTo_Mapnik_%26_Tirex
Problem with Set Up PostGIS on the postresql database
Hi,
I have a problem with this line
psql -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -d gis
I did every step like you described it before that ;) but know I become the whole time this error
psql: FATAL: Ident-Authentifizierung für Benutzer >>blitzortung<< fehlgeschlagen
Thanx for your help
Problem with Set Up PostGIS on the postresql database
Hi,
I did everything you said in the manual. But know I have the problem with the Set Up PostGIS on the postgresql database
when I punch in this line
psql -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -d gis
at first it said that I dont have any Authentification and know it doesn't even exist anymore.
Thanx for your help
Error on final command
I was able to follow your tutorial successfully until the very last command. When I run ./generate_image.py I get the following error.
Traceback (most recent call last):
File "./generate_image.py", line 37, in
mapnik.load_map(m,mapfile)
RuntimeError: PSQL error:
FATAL: Ident authentication failed for user "mapsuser" (encountered during parsing of layer 'landcover')
Any ideas on what I did wrong and how to fix it?
Thanks for a great tutorial. It's been extremely helpful.
A suggestion
Hi,
Running
sudo /etc/init.d/postgresql-8.4 restart
results in a 'command not found'. I needed to remove the '-8.4' to make it work.
Also, running
psql -f /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql -d gis
gives a ''permission denied". I fixed it by doing a
sudo -u postgres -i
I think this needs to be added to the box of commands above, before the 'exit'.
Actually, I think the following commands all need to be run as user postgres...
Thanks,
Peter.
More corrections
Set the Spatial Reference Identifier (SRID) on the new database.
psql -f ~/bin/osm2pgsql/900913.sql -d gis
>> No such file or directory
Changed back to my normal user, but now:
psql: FATAL: Ident authentication failed for user "peter"
Solution: change path to absolute.
Wow! Thanks!
Hi
I just wanted to thank you for this, ran beautifully!
I had one tiny issue with having to install unzip using apt-get (fresh EC2 instance, nothing installed yet). Otherwise I don't think I had a single issue.
Also: "sudo apt-get install python-mapnik" was the least painful option to get mapnik set up.
For anyone on the interwebs who sees this: I had success doing this using ubuntu oneiric 11.04 on a free EC2 micro instance using a small osm excerpt (South Africa's Western Cape). I didn't have to wait for the import to finish for more than 15 minutes.
Thanks again, this was really satisfying. :)
Rainer
Mod_tile
Perhaps you want to set up proper tileserver also - I just created small how-to for mod_tile:
http://wiki.openstreetmap.org/wiki/HowTo_mod_tile
How i can use this installed OSM Server
Hi!
I successfully installed the OSM (postgreSQL, Mapnik, e.t.c.) server, as it says in the manual.
Do I need to start generating all tiles (by [MAPNIK_MAP_FILE="osm.xml" MAPNIK_TILE_DIR="tiles/" ./generate_tiles.py] command) or tiles that can be created automatically on the first request?
How I can use map tails in my own windows map client application?
Do I need to configure Apache2 service (may be necessary to install special plugin?)? If so, where can I find a how to guide for configuring the Apache?
F.Y.I.: After upgrading to
F.Y.I.: After upgrading to Ubuntu 11.04 everything seemed broken. It seems all libboost libs are upgraded to 1.42, from 1.40
apt-get install libboost-iostreams1.40-dev libboost-regex1.40-dev libboost-thread1.40-dev
becomes
apt-get install libboost-iostreams1.42-dev libboost-regex1.42-dev libboost-thread1.42-dev
Location of database?
Dear rw and other readers / contributors,
Thanks for this magnificent article. I followed the instructions and got no severe issues at all. Only thing worth mentioning is the fact that for Ubuntu 11.04 I needed version 1.42 rather than 1.40 in the following commands:
$ sudo apt-get install libboost-python1.42-dev python-cairo-dev python-nose
$ sudo apt-get install libboost1.42-dev libboost-filesystem1.42-dev
$ sudo apt-get install libboost-iostreams1.42-dev libboost-regex1.42-dev libboost-thread1.42-dev
$ sudo apt-get install libboost-program-options1.42-dev libboost-python1.42-dev
Only questions I have:
1. is it possible to manipulate the location of the database? I have a separate home partition and even an empty 2nd hard drive on the dedicated server. So the / partition has limited space. I downloaded netherlands.osm, had it processed with osm2pgsql but I could only watch the / partition grow until it was full and then it reversed all actions. I may assume it is in the /usr/share/postgresql/8.4/contrib/postgis-1.5/ folder? Maybe a separate /usr/share/postgresql/8.4/contrib/postgis-1.5/ partition on that 2nd drive?
2. It is my intention to update existing tiles that I gathered over time from tile servers with Oziexplorer and tangogps. is it possible to create new tiles based on the tiles that already exist on my laptop, rather than creating (too many) tiles in a bounding box?
stuck at
Everything works up to here:
cd ~/bin
svn co http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/
cd osm2pgsql
./autogen.sh
./configure
make
.configure ends with "does not exist or is not an executable file"
make ends with "*** no tarkets specified and no makefile found. Stop"
Any ideas? All precedeing steps appeared to work correctly.
Thx!
JB
load planet file - Unknown node type 8
When I load my planet file I get the responses regarding setting up tables, and that NOTICES as shown below, as expected, but then after that I see:
Reading in file: /home/jbrosowsky/planet/planet-latest.osm.bz2
Unknown node type 8
Processing: Node(1145363k) Way(68098k) Relation(0)
... and the loading process is still running now. What does "unknown node type 8" mean and should I be worried. Also should my node count be higher, given that it has now moved on to processing ways?
Using projection SRS 900913 (Spherical Mercator)
Setting up table: planet_osm_point
NOTICE: table "planet_osm_point" does not exist, skipping
NOTICE: table "planet_osm_point_tmp" does not exist, skipping
Setting up table: planet_osm_line
NOTICE: table "planet_osm_line" does not exist, skipping
NOTICE: table "planet_osm_line_tmp" does not exist, skipping
Setting up table: planet_osm_polygon
NOTICE: table "planet_osm_polygon" does not exist, skipping
NOTICE: table "planet_osm_polygon_tmp" does not exist, skipping
Setting up table: planet_osm_roads
NOTICE: table "planet_osm_roads" does not exist, skipping
NOTICE: table "planet_osm_roads_tmp" does not exist, skipping
Mid: pgsql, scale=100, cache=4096MB, maxblocks=524289*8192
Setting up table: planet_osm_nodes
NOTICE: table "planet_osm_nodes" does not exist, skipping
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_nodes_pkey" for table "planet_osm_nodes"
Setting up table: planet_osm_ways
NOTICE: table "planet_osm_ways" does not exist, skipping
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_ways_pkey" for table "planet_osm_ways"
Setting up table: planet_osm_rels
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "planet_osm_rels_pkey" for table "planet_osm_rels"
Reading in file: /home/jbrosowsky/planet/planet-latest.osm.bz2
Unknown node type 8
Processing: Node(1145363k) Way(68098k) Relation(0)
Intarray no longer necessary
Hello,
Love the tutorial and the option to have my own map server!
I installed today another server, but it failed to load the planet-file. It seems a change was made in osm2psql:
The target database has the intarray contrib module loaded.
While required for earlier versions of osm2pgsql, intarray
is now unnecessary and will interfere with osm2pgsql's array
handling. Please use a database without intarray.
It seems that this part is no longer necessary:
Enable intarray
psql -f /usr/share/postgresql/8.4/contrib/_int.sql -d gis
Thanks for the tutorial!
I've removed the intarray
I've removed the intarray portion of the tutorial now that the new array handling is in the mainstream code.
excellent guide, thanks very much - couple of minor updates tho'
Since this guide was by far the best I've found, I used it on a new ubuntu 10.04 VM I just built this morning, still importing the data (british isles only) - it's just going through the tables index builds at the end after about 90 minutes.
2 problems I found:
Generating image: error
Thank you for the excellent tutorial. Two questions:
1) When calling ./generate_image.py I get this error: /home/username/bin/mapnik/world_boundaries/10m_populated_places does not exist although I've extracted both the 10m and the 110m file to the directory world_boundaries. Any idea what went wrong?
2) Your last link to more tutorials does not work. Do you have any recommendations for further readings how to render tiles for specific regions and zoom levels. Say, I want to render tiles for Brittany in France and tiles for Sicily in Italy, how would I start? By extracting those regions from an OSM file or by importing the world and then rendering for specific regions? Any advice?
As a side note, I was only able to install the libboost libraries with version 1.42.
Thanks!
i faced the same problem
i faced the same problem today - and i compared the world-boundaries-directory to a system i installed a few weeks ago:
on the old system:
-rw-rw-rw- 1 root root 30158632 2011-01-24 02:09 10m_populated_places.dbf -rw-rw-rw- 1 root root 145 2011-01-24 01:47 10m_populated_places.prj -rw-rw-rw- 1 root root 204892 2011-01-24 02:09 10m_populated_places.shp -rw-rw-rw- 1 root root 58612 2011-01-24 02:09 10m_populated_places.shx ~/osm/bin/mapnik# ls -la 10m-populated-places.zip -rw-r--r-- 1 root root 1465293 2011-01-27 10:04 10m-populated-places.zip ~/osm/bin/mapnik# unzip -t 10m-populated-places.zip Archive: 10m-populated-places.zip testing: 10m_populated_places.dbf OK testing: 10m_populated_places.prj OK testing: 10m_populated_places.shp OK testing: 10m_populated_places.shx OK No errors detected in compressed data of 10m-populated-places.zip.on the new system:
-rw-rw-rw- 1 root root 30154509 2011-08-08 03:06 ne_10m_populated_places.dbf -rw-rw-rw- 1 root root 145 2011-07-24 22:13 ne_10m_populated_places.prj -rw-rw-rw- 1 root root 204864 2011-08-08 03:06 ne_10m_populated_places.shp -rw-rw-rw- 1 root root 58604 2011-08-08 03:06 ne_10m_populated_places.shx ~/osm/bin/mapnik# ls -la 10m-populated-places.zip -rw-r--r-- 1 root root 1501274 2011-08-20 00:03 10m-populated-places.zip ~/osm/bin/mapnik# unzip -t 10m-populated-places.zip Archive: 10m-populated-places.zip testing: ne_10m_populated_places.dbf OK testing: ne_10m_populated_places.prj OK testing: ne_10m_populated_places.shp OK testing: ne_10m_populated_places.shx OK No errors detected in compressed data of 10m-populated-places.zip.looks like if there is a "bug" in the naming of the content of the zip-File - but i did not try to rename the unpacked files by now
ok, renaming the new files
ok, renaming the new files to "old names" doesn't work...
~/osm/bin/mapnik/world_boundaries# cp ne_10m_populated_places.dbf 10m_populated_places.dbf ~/osm/bin/mapnik/world_boundaries# cp ne_10m_populated_places.prj 10m_populated_places.shp ~/osm/bin/mapnik/world_boundaries# cp ne_10m_populated_places.prj 10m_populated_places.prj ~/osm/bin/mapnik/world_boundaries# cp ne_10m_populated_places.shx 10m_populated_places.shx ~/osm/bin/mapnik/world_boundaries# cd .. ~/osm/bin/mapnik# ./generate_xml.py --dbname gis --user username --accept-none Include files written successfully! Pass the osm.xml file as an argument if you want to serialize a new version or test reading the XML ~/osm/bin/mapnik# ./generate_image.py wrong file code : 1195724615 Traceback (most recent call last): File "./generate_image.py", line 42, in mapnik.load_map(m,mapfile) RuntimeError: wrong file code : 1195724615 (encountered during parsing of layer 'nepopulated')I have the same problem!
I have the same problem! does anyone know how to solve this problem?
Re: Generating image: error
I symlinked the ne_10m_populated_places.* files to 10m_populated_places.* and it seems to have worked. Don't know if that's the right solution, or if it will break something down the line.
Having issues loading the planet-latest.osm.bz2 file
I have gone through this guide 3 times now and I always end up with the following:
While reading and loading the osm file I get one error: Unknown node type 8
Towards the end of loading the huge file I get more errors, all similar to:
Processing: Node(477100k 55.5k/s) Way(0k 0.00k/s) Relation(0 0.00/s)Entity: line 670327483: parser error : attributes construct error
69-09:::66:5Z" version="073:.T1852.97Z" version="07" lon="-09108" timestamp="15"
At the end I get /home/administrator/planet/planet-latest.osm.bz2 : failed to parse
Error occurred, cleaning up
Any idea what I need to do or stop doing ;-) ?
Thanks
generate_tiles.py at higher zoom levels
Great tutorial!
I have one issue with the look of the generated tiles. Everything works fine, up to zoom level 9, the tiles (png) files are PERFECT. However, at zoom level 10 an onwards the land has the same color as water areas #b5d0d0. This color is listed as bgcolor in my osm.xml. However if I change this to #f1eee8 which is the color for land on openstreetmap than the see(oceans) color is missing i.e. it is set to #f1eee8. What am I doing wrong?
The same is with generate_image.py. If I set wider extent boundaries then the colors are perfect. If I narrow it down, then the same issue appears.
Thanks!
Are you using Mapnik2 ?
Mapnik2 is awesome and just released in September 2011. This tutorial uses and older version of Mapnik. There are several style sheet changes in Mapnik2, including ...
Which may well cause the problem that you are having (and others that you will run into later). Mapnik2 includes a stylesheet updater to automatically update your stylesheets. Try
upgrade_map_xml.pyUsage: upgrade_map_xml.py input_file output_file
10m_populated
STRANGE ERROR!! I have following error
/home/bin/mapnik/world_boundaries/10m_populated_places does not exist (encountered during parsing of layer 'nepopulated')
root@ks356390:/home/bin/mapnik# cd world_boundaries
root@ks356390:/home/bin/mapnik/world_boundaries# ls
110m_admin_0_boundary_lines_land.dbf places.shx
110m_admin_0_boundary_lines_land.prj processed_p.dbf
110m_admin_0_boundary_lines_land.sbn processed_p.index
110m_admin_0_boundary_lines_land.sbx processed_p.shp
110m_admin_0_boundary_lines_land.shp processed_p.shx
110m_admin_0_boundary_lines_land.shp.xml shoreline_300.dbf
110m_admin_0_boundary_lines_land.shx shoreline_300.index
builtup_area.dbf shoreline_300.shp
builtup_area.index shoreline_300.shx
builtup_area.prj world_bnd_m.dbf
builtup_area.shp world_bnd_m.index
builtup_area.shx world_bnd_m.prj
ne_10m_populated_places.dbf world_bnd_m.shp
ne_10m_populated_places.prj world_bnd_m.shx
ne_10m_populated_places.shp world_boundaries_m.dbf
ne_10m_populated_places.shx world_boundaries_m.index
places.dbf world_boundaries_m.prj
places.prj world_boundaries_m.shp
places.shp world_boundaries_m.shx
root@ks356390:/home/bin/mapnik/world_boundaries# mkdir 10m_populated_places
root@ks356390:/home/bin/mapnik/world_boundaries# ls
10m_populated_places places.shx
110m_admin_0_boundary_lines_land.dbf processed_p.dbf
110m_admin_0_boundary_lines_land.prj processed_p.index
110m_admin_0_boundary_lines_land.sbn processed_p.shp
110m_admin_0_boundary_lines_land.sbx processed_p.shx
110m_admin_0_boundary_lines_land.shp shoreline_300.dbf
110m_admin_0_boundary_lines_land.shp.xml shoreline_300.index
110m_admin_0_boundary_lines_land.shx shoreline_300.shp
builtup_area.dbf shoreline_300.shx
builtup_area.index world_bnd_m.dbf
builtup_area.prj world_bnd_m.index
builtup_area.shp world_bnd_m.prj
builtup_area.shx world_bnd_m.shp
ne_10m_populated_places.dbf world_bnd_m.shx
ne_10m_populated_places.prj world_boundaries_m.dbf
ne_10m_populated_places.shp world_boundaries_m.index
ne_10m_populated_places.shx world_boundaries_m.prj
places.dbf world_boundaries_m.shp
places.prj world_boundaries_m.shx
places.shp
root@ks356390:/home/bin/mapnik/world_boundaries# cd ..
root@ks356390:/home/bin/mapnik# unzip 10m-populated-places.zip -d world_boundaries/10m_populated_places
Archive: 10m-populated-places.zip
inflating: world_boundaries/10m_populated_places/ne_10m_populated_places.dbf
inflating: world_boundaries/10m_populated_places/ne_10m_populated_places.prj
inflating: world_boundaries/10m_populated_places/ne_10m_populated_places.shp
inflating: world_boundaries/10m_populated_places/ne_10m_populated_places.shx
Sie haben neue Post in /var/mail/root.
root@ks356390:/home/bin/mapnik# ./generate_image.py
Traceback (most recent call last):
File "./generate_image.py", line 42, in
mapnik.load_map(m,mapfile)
RuntimeError: /home/bin/mapnik/world_boundaries/10m_populated_places does not exist (encountered during parsing of layer 'nepopulated')
there is a folder, there are files inside, but it doesn't see it...How to overcome this?????
P.S. Thanks for great tutorial
Update for Ubuntu 11.10 ?
Thank you for this great tutorial.
It was working on ubuntu Oneiric so far but the "Package 'postgresql-8.4-postgis' has no installation candidate" anymore.
What is the best way to make it work on the lastest Ubuntu ? Grab on an old postgresql-8.4-postgis package somewhere or switch to postgresql 9.1 ?
Thanks !
12.04 is the next scheduled rewrite of the tutorial.
I've written these tutorials for the last two LTS versions of Ubuntu. I don't have plans to increase that but I suppose a sponsor could change my mind.
I cannot sponsor you but I
I cannot sponsor you but I hope you'll get one :) This tutorial is definitely the kind of contribution that help openstreetmap go in the right direction.
10.11 differences
Differences compared to this tutorial are roughly:
You can use postgresql9.1:
sudo apt-get install postgresql-9.1-postgis postgresql-contrib-9.1
sudo apt-get install postgresql-server-dev-9.1
To get osm2pgsql to run/compile you need:
sudo apt-get install bzip2
sudo apt-get install libbz2-dev
sudo apt-get install libgeos-dev
sudo apt-get install proj
Use
gksudo gedit postgresql.conf
to edit postgresql.conf
I also had to manually edit /etc/sysctl.conf because I couldn't get the changes to be saved:
gksudo gedit /etc/sysctl.conf
add to the end
kernel.shmmax=...
Command to restart postgreql has no version number:
sudo /etc/init.d/postgresql restart
With osm2psql I used the "-C 1024" parameter because it reported running out of memory otherwise.
I didn't follow the instructions for compiling mapnik from source. I used the prepared packages for ubuntu as described on: http://trac.mapnik.org/wiki/UbuntuInstallation
Everything worked without a problem.
Post new comment