Index
1.-Introduction
2.-Let’s start
3.-Preparing the database
4.-Install Mapnik library
5.-Install prepared world boundary data
6.-Render your first map
7.-Troubleshooting
1.-Introduction
Firstly I woudl like to thank “rw”, the author of the post this post is about. This post is based (95%) on this one (currently not available, but you can check it out here)
My main intention is spread this post to all my viewers and save here this knowledge about OSM platform to have at least an online reference for everyone (and of course, for me). This is neither an introduction nor post information about OSM, but just technical info about building up your own OSM server. I also would like to improve my knowledge about this server (I don’t want just to download, install and run it, it would be nice trying to understand it).
I won’t just copy and paste the same content of there, because (as an ordinary large installation) I had problems installing this platform and I solved then, so I will write here everything I saw, I solved, and my personal conclusions. I also will go in depth in some issues like explaining some commands and stuff.
2.-Let’s start
Upgrade your system
Just for security issues, it is recommended to update and upgrade your system, but not necessary.
sudo apt-get update
sudo apt-get upgrade
Get some system tools
sudo apt-get install subversion autoconf screen munin-node munin htop
Subversion: we’ll use it to donwload files from some repositories.
Autoconf: software to generate autoconfig scripts.
Screen: Screen is a full-screen window manager that multiplexes a physical terminal between several processes, typically interactive shells.
Munin-node: software to monitoring our server from the client.
Munin: client of munin-node.
Htop: software to check and monitoring the system used resources.
Organize your filesystem a bit
cd ~
mkdir src bin planet
- src: source of mapnik.
- bin: here we’ll have the scripts to generate the image and load the database.
- planet: planet (or piece) database we’re gonna build up.
Get the latest planet (OSM file)
Well, in this part we can download the entire world wide database (nowadays [20 dec 2012] is 24GBs) or maybe we can download not the world but a piece. Just for testing I will choose download a piece.
cd planet
wget http://planet.openstreetmap.org/planet-latest.osm.bz2
3.-Preparing the database
Here we’re gonna download postgresql to deploy our own database there and some required libraries to use it.
sudo apt-get install postgresql-8.4-postgis postgresql-contrib-8.4
sudo apt-get install postgresql-server-dev-8.4
sudo apt-get install build-essential libxml2-dev
sudo apt-get install libgeos-dev libpq-dev libbz2-dev proj
Build-essential: This package contains an informational list of packages which are considered essential for building Debian packages.
Libxml2-dev: Very useful if you wish to develop your own programs using the GNOME XML library.
Libgeos-dev: GEOS provides a spatial object model and fundamental geometric functions. It implements the geometry model defined in the OpenGIS Consortium Simple Features Specification for SQL.
Libpq-dev: Header files and static library for compiling C programs to link with the libpq library in order to communicate with a PostgreSQL database backend.
Libbz2-dev: Static libraries and include files for the bzip2 compressor library.
Proj: Standard Unix filter function which converts geographic longitude and latitude coordinates into cartesian coordinates, by means of a wide variety of cartographic projection functions.
Install osm2pgsql from the repository
We firstly download osm2pgsql from the repository and after that, configurate it. osm2pgsql is a utility program that converts OpenStreetMap data to PostgreSQL databases.
cd ~/bin
svn co http://svn.openstreetmap.org/applications/utils/export/osm2pgsql/
cd osm2pgsql
./autogen.sh (Error?)
./configure
make
Configure the PostGIS database
edit /etc/postgresql/8.4/main/postgresql.conf in four places. These changes help with the large quantities of data that we are using.
107: shared_buffers = 128MB # 16384 for 8.1 and earlier
170: checkpoint_segments = 20
(new line): maintenance_work_mem = 256MB # 256000 for 8.1 and earlier
387: autovacuum = off
edit /etc/sysctl.conf
(new line) kernel.shmmax=268435456
Remind that the above only takes effect after a reboot. Making this work immediately requires the following.
sudo sysctl kernel.shmmax=268435456
Restart postgres to enable the changes.
sudo /etc/init.d/postgresql-8.4 restart
Create a database called “gis”. Some of our future tools presume that you will use this database name. Substitute your username for “username” in two places below. This should be the username that will render maps with mapnik.
Note from delanover author: I had problems with the username so I decided to use the same username that in the computer server system.
sudo -u postgres -i
createuser username # answer yes for superuser
createdb -E UTF8 -O username gis
createlang plpgsql gis
exit
Set up PostGIS on the postresql database.
psql -f /usr/share/postgresql/8.4/contrib/postgis.sql -d gis
This should respond with many lines ending with
…
CREATE FUNCTION
COMMIT
Substitute your username for “username” in two places in the next line. This should be the username that will render maps with mapnik.
echo “ALTER TABLE geometry_columns OWNER TO username; ALTER TABLE spatial_ref_sys OWNER TO username;” | psql -d gis
Should reply with
ALTER TABLE
ALTER TABLE
Enable intarray
psql -f /usr/share/postgresql/8.4/contrib/_int.sql -d gis
Replies with many lines ending with
…
CREATE FUNCTION
CREATE OPERATOR CLASS
Set the Spatial Reference Identifier (SRID) on the new database.
psql -f ~/bin/osm2pgsql/900913.sql -d gis
Should reply with
INSERT 0 1
Load planet into the database with osm2pgsql
cd ~/bin/osm2pgsql
./osm2pgsql -S default.style –slim -d gis -C 2048 ~/planet/YourFileMap.osm.bz2 (Error?)
Note: load 8.4GB take 30 hours, and particularly 1.4GB took me 3-4 hours (it depends of the I/O), so be patient.
Completed planet_osm_roads
Copying planet_osm_polygon to cluster by geometry finished
Copying planet_osm_line to cluster by geometry finished
Creating indexes on planet_osm_polygon finished
All indexes on planet_osm_polygon created in 4257s
Completed planet_osm_polygon
Creating indexes on planet_osm_line finished
All indexes on planet_osm_line created in 5053s
Completed planet_osm_line
Stopped table: planet_osm_ways in 16095s
Osm2pgsql took 29436s overall
4.-Install Mapnik library
sudo apt-get install libltdl3-dev libpng12-dev libtiff4-dev libicu-dev
sudo apt-get install libboost-python1.40-dev python-cairo-dev python-nose
sudo apt-get install libboost1.40-dev libboost-filesystem1.40-dev
sudo apt-get install libboost-iostreams1.40-dev libboost-regex1.40-dev libboost-thread1.40-dev
sudo apt-get install libboost-program-options1.40-dev libboost-python1.40-dev
sudo apt-get install libfreetype6-dev libcairo2-dev libcairomm-1.0-dev
sudo apt-get install libgeotiff-dev libtiff4 libtiff4-dev libtiffxx0c2
sudo apt-get install libsigc++-dev libsigc++0c2 libsigx-2.0-2 libsigx-2.0-dev
sudo apt-get install libgdal1-dev python-gdal
sudo apt-get install imagemagick
Libltdl3-dev: A small library that aims at hiding the various difficulties of dlopening libraries from programmers. It is a system independent dlopen wrapper for GNU libtool.
Libpng12-dev: Library implementing an interface for reading and writing PNG format files.
Libtiff4-dev: Library providing support for the Tag Image File Format (TIFF), a widely used format for storing image data. This package includes the development files, static library, and header files.
Libicu-dev: ICU is a C++ and C library that provides robust and full-featured Unicode and locale support. This package contains the development files for ICU along with programs used to manipulate data files found in the ICU sources.
Libboost libraries: Boost provides free peer-reviewed portable C++ source libraries.
Python libraries: So obvious.
Libfreetype6-dev: This package contains all supplementary files (static library, headers and documentation) you need to develop your own programs using the FreeType 2 library (digital typography library).
Libcairo2-dev: Cairo is a multi-platform library providing anti-aliased vector-based rendering for multiple target backends.
Libgeotiff-dev: The GeoTIFF standard has been developed for reading, and writing geographic meta-information tags on top of TIFF raster.
Tiff libraries: So obvious, to work with tiff format.
Libsigc libraries: implement a typesafe callback system for standard C++. It allows you to define signals and to connect those signals to any callback function, either global or a member function, regardless of whether it is static or virtual.
Libgdal1-dev: GDAL is a translator library for raster geospatial data formats. As a library, it presents a single abstract data model to the calling application for all supported formats.
Imagemagick: software suite to create, edit, compose, or convert bitmap images.
Build Mapnik library from source.
cd ~/src
svn co http://svn.mapnik.org/tags/release-0.7.1/ mapnik (Error? offline)
cd mapnik
python scons/scons.py configure INPUT_PLUGINS=all OPTIMIZATION=3 SYSTEM_FONTS=/usr/share/fonts/truetype/ (Error? boost version)
python scons/scons.py #be patient here
sudo python scons/scons.py install
sudo ldconfig
Confirm that Mapnik library is installed.
python
>>> import mapnik
>>>
Install Mapnik tools
cd ~/bin
svn co http://svn.openstreetmap.org/applications/rendering/mapnik
5.-Install prepared world boundary data
cd ~/bin/mapnik
mkdir world_boundaries
wget http://tile.openstreetmap.org/world_boundaries-spherical.tgz
tar xvzf world_boundaries-spherical.tgz
cd world_boundaries
wget http://tile.openstreetmap.org/processed_p.tar.bz2
bunzip2 processed_p.tar.bz2
tar xvf processed_p.tar
wget http://tile.openstreetmap.org/shoreline_300.tar.bz2
bunzip2 shoreline_300.tar.bz2
tar xvf shoreline_300.tar
6.-Render your first map
cd ~/bin/mapnik
chmod +x generate_xml.py
./generate_xml.py –dbname gis –accept-none [IMPORTANT NOTE]
IMPORTANT NOTE: I don’t know why but the double hyphen is not seen. Remember is “–” (two “-“) and not only one “-“.
Include files written successfully! Pass the osm.xml file as an argument if you want to serialize a new version or test reading the XML
Now we finally generate the image.
./generate_image.py (Error?)
The result of this is having a 10000x5000px PNG image of England en your mapnik folder. Remember, be patient.
Hope you enjoy this tutorial!
7.-Troubleshooting
Error 1: Configuring at installing osm2pgsql
Output error at performing ./configure when installing osm2pgsql:
Use of uninitialized value in pattern match (m//) at /usr/bin/autoreconf line 189.
autoreconf: Entering directory `.’
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal –force -I m4
configure.ac:41: warning: macro `AM_PROG_LIBTOOL’ not found in library
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: running: /usr/bin/autoconf –force
configure.ac:41: error: possibly undefined macro: AM_PROG_LIBTOOL
If this token and others are legitimate, please use m4_pattern_allow.
See the Autoconf documentation.
autoreconf: /usr/bin/autoconf failed with exit status: 1
Solution
You can’t exec libtoolize, so you must install libtool before continue, writting down:
sudo apt-get install libtool
Error 2: Loading planet into the database
Maybe you get this error while loading the data:
While required for earlier versions of osm2pgsql, intarray
is now unnecessary and will interfere with osm2pgsql’s array
handling. Please use a database without intarray.
Error occurred, cleaning up
To fix it, we’ll do some fix located in the official documentation.
sudo -u postgres -i -H
dropdb ptgis ### here the old database is dropped
createdb -E UTF8 -O ptuser ptgis
createlang plpgsql ptgis
# for Ubuntu <=9.10:
# psql -d ptgis -f /usr/share/postgresql/8.3/contrib/_int.sql ### (do not execute this line)
psql -d ptgis -f /usr/share/postgresql-8.3-postgis/lwpostgis.sql
# for Ubuntu >=10.04:
#psql -d ptgis -f /usr/share/postgresql/8.4/contrib/_int.sql ### (do not execute this line)
psql -d ptgis -f /usr/share/postgresql/8.4/contrib/postgis.sql
psql ptgis -c “ALTER TABLE geometry_columns OWNER TO ptuser”
psql ptgis -c “ALTER TABLE spatial_ref_sys OWNER TO ptuser”
exit
# for Ubuntu <=9.10:
sudo /etc/init.d/postgresql-8.3 reload
# for Ubuntu >=10.04:
sudo /etc/init.d/postgresql-8.4 reload
psql ptgis ptuser -f osm2pgsql/900913.sql
Now, we try to load the data again.
Error 3: Offline mapnik repository
I set this up as an error because who knows, maybe someday is online again. But now it is not, so let’s find another place to download it.
We need to download git software in order to download from a git repository, and after that, we’ll be able to download mapnik from here
sudo apt-get install git-core
git clone https://github.com/mapnik/mapnik.git
Error 4: Boost version >= 1.47 required
We can actually check our boost version looking at line 30 in /usr/include/boost/version.hpp file to realize that we don’t have 1.47 version or greater, so, we have to upgrade it. This is not that simple than apt-get install, but is this simple: download and install it.
We need to download the last version (http://www.boost.org/) (nowadays, 1.52, from http://sourceforge.net/projects/boost/files/boost/1.52.0/). We select tar.bz2 file, unzip it, configure it and so on.
bunzip2 boost_1_52_0.tar.bz2
tar xvf boost_1_52_0.tar
./bootstrap.sh
./b2
sudo ./b2 install
Error 5: Needed files don’t exist!
After trying to generate the image (what a bad luck, failing in the last step…) this error might be returned to you.
The only problem here is we need many files that we don’t have, so let’s download and move them to fix it into world_boundaries folder.
I found that files here. We have to donwload this below:
- 110m_admin_0_boundary_lines_land.shp
- 110m_admin_0_boundary_lines_land.dbf
- 10m_populated_places.shp (*)
- 10m_populated_places.dbf (*)
(*): We have to rename them to “ne_10m…”
One Reply to “Build an OpenStreetMap server on Ubuntu 10.04”