INSA/be-graphes-map-generator
4
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Update README.

Browse Source
This commit is contained in:
Mikaël Capelle 2018-03-14 11:11:54 +01:00
parent 019ec58e2f
commit c6247a9ae2
3 changed files with 90 additions and 83 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

113
README.md
View File

@ -29,80 +29,131 @@ The two main folders are `inputs` and `outputs`:
- `inputs` mainly contains: - `inputs` mainly contains:
- a script `coastline-fix.sh` to merge land/sea polygon (see below) and PBF files; - a script `coastline-fix.sh` to merge land/sea polygon (see below) and PBF files;
- a directory containing boundinx-box (`bbox`) for specific regions; - multiple polygon (`.poly`) and bounding-box (`.bbox`) files for specific regions;
- a file containing highway filters;
- PBF files downloaded organized in folder `$REGION/$REGION.pbf`. - PBF files downloaded organized in folder `$REGION/$REGION.pbf`.
- `outputs` is the default folder for generated output. - `outputs` is the default folder for generated output.
### Generating maps ### Generating maps
This project can be used to generate two types of files:
- Graph file (`.mapgr`) containing graph information (see
[OSM2Graph](https://github.com/Holt59/OSM2Graph));
- Mapsforge file (`.mapfg`) containing [mapsforge](https://github.com/mapsforge/mapsforge)
data to display graph in a more visual way.
The process to generate the Mapsforge file is more complex than the one to generate graph file,
and it will often take much more time. Fortunately, it is often **not necessary** to generate
the Mapsforge file, because pre-generated files exist for most countries and regions.
Before generating the Mapsforge file, check if a file corresponding to the region you want
exists on the following server:
[http://download.mapsforge.org/maps/v4/](http://download.mapsforge.org/maps/v4/)
The hierarchy of this server is similar to [Geofabrik](http://download.geofabrik.de/), which
is the main source for our PBF files.
#### Step 1 - Obtaining a PBF file. #### Step 1 - Obtaining a PBF file.
You should create a PBF files containining the region you want: The first step is to a PBF files containining the region you want:
- Download a complete region from Geofabrik: [http://download.geofabrik.de/](http://download.geofabrik.de/) - Download a complete region from Geofabrik: [http://download.geofabrik.de/](http://download.geofabrik.de/)
- Chop a region previously downloaded: - If necessary, chop a region previously downloaded:
*Using a bounding box:* *Using a bounding box:*
```bash ```bash
bin/osmosis --rb inputs/bretagne/bretagne-latest.osm.pbf \ # Specify your input PBF bin/osmosis --rb inputs/europe/france/bretagne/bretagne.osm.pbf \ # Specify your input PBF
--bounding-box bottom=47.160000 left=-3.855000 top=48.330000 right=-1.915000 completeWays=yes \ # Specify the bounding bo --bounding-box bottom=47.160000 left=-3.855000 top=48.330000 right=-1.915000 completeWays=yes \ # Specify the bounding bo
--write-pbf inputs/bretagne/morbihan.pbf # Specify the output file --write-pbf inputs/europe/france/bretagne/morbihan.pbf # Specify the output file
``` ```
*Using a polygon:* *Using a polygon:*
```bash ```bash
osmosis --rb inputs/bretagne/bretagne-latest.osm.pbf \ # Specify your input PBF osmosis --rb inputs/europe/france/bretagne/bretagne.osm.pbf \ # Specify your input PBF
--bounding-polygon file=inputs/bbox/morbihan.poly completeWays=yes \ # Specify your bounding polygon --bounding-polygon file=inputs/bbox/morbihan.poly completeWays=yes \ # Specify your bounding polygon
--write-pbf file=inputs/bretagne/morbihan.pbf # Specify your output file --write-pbf file=inputs/europe/france/bretagne/morbihan.pbf # Specify your output file
``` ```
The `inputs/bbox` directory contains bounding-box (`.bbox`) and bounding polygon (`.poly`) for some regions. You can find polygon for a lots of region here: [http://polygons.openstreetmap.fr/index.py](http://polygons.openstreetmap.fr/index.py) (to find the id of the relation, search it on openstreetmap, e.g. [https://www.openstreetmap.org/relation/7447](https://www.openstreetmap.org/relation/7447)). The `inputs` directory contains bounding-box (`.bbox`) and bounding polygon (`.poly`) for some
regions. You can find polygon for a lots of region here:
[http://polygons.openstreetmap.fr/index.py](http://polygons.openstreetmap.fr/index.py)
(to find the id of the relation, search it on openstreetmap, e.g.
[https://www.openstreetmap.org/relation/7447](https://www.openstreetmap.org/relation/7447)).
#### Step 1 Bis - Fixing coastlines and sea. #### Step 1 Bis - Fixing coastlines and sea.
Skip this step if your region does not contain sea or big inner lakes, or if you do not want to generate mapsforge file. **Important**: You can skip this step if your region does not contain sea or big inner
lakes, or if you do not want to generate the corresponding Mapsforge file.
We need to add sea information to the PBF file to generate a correct mapsforge map. To do so, a script `coastline-fix.sh` is provided. Run it simply: We need to add sea information to the PBF file to generate a correct mapsforge map. To do so,
a script `coastline-fix.sh` is provided. Run it simply:
```bash ```bash
./coastline-fix.sh bretagne/morbihan.pbf ./coastline-fix.sh bretagne/morbihan.pbf
``` ```
To apply the fix, you need a polygon file (`morbihan.poly` — not another name), either in the same directory as the PBF or in the `inputs/bbox` directory (recommended). To apply the fix, you need a polygon file (`morbihan.poly` — not another name), in the
same directory as the PBF or in the `inputs/bbox` directory (recommended).
This script will generate multiple intermediate files, but only two are importants: This script will generate multiple intermediate files, but only two are importants:
- `inputs/bbox/morbihan.bbox`, actual bounding-box of the PBF file — This will overwrite the old bounding-box if there was one. - `inputs/europe/france/bretagne/morbihan.bbox`, actual bounding-box of the PBF file —
- `inputs/bretagne/morbihan_merge.pbf`, merged file that you should use as input for mapsforge. This will overwrite the old bounding-box if there was one.
- `inputs/europe/france/bretagne/morbihan_merge.pbf`, merged file that you should use as input
for mapsforge.
#### Step 2 - Binary graph files #### Step 2 - Generate graph & Mapsforge files
To generate both the graph file and the Mapsforge file corresponding to your input PBF,
you simply need to run the `generate.sh` script:
```bash ```bash
# You can use either "insa2016" or "insa2018" writers, and you should change the map id to your need. bash generate.sh inputs/europe/france/bretagne/bretagne.osm.pbf \
bin/osmosis --rb inputs/bretagne/morbihan.pbf \ --id FR-E --name "Bretagne" \
--used-node \ --output outputs/bretagne
--tf reject-relations \
--tf accept-ways highway=(cat inputs/highway-filter.cmd) natural=coastline \
--osm2graph file=outputs/morbihan.mapgr writer=insa2018 map-id=0x235
``` ```
#### Step 3 - Mapsforge files This command will generate two files: `outputs/bretagne.mapgr` and `outputs/bretagne.mapfg`,
you should not add the file extension when specifying `--output`.
You can get help about `generate.sh` by running:
```bash ```bash
# You must specify a bounding-box if you used the coastline fix. bash generate.sh --help
# You can use type=ram to speed-up the process for small maps.
bin/osmosis --rb inputs/bretagne/morbihan_merge.pbf \
--mapfile-writer file=outputs/morbihan.mapfg bbox=$(cat inputs/bbox/morbihan.bbox) type=hd threads=2
``` ```
#### Extra informations: If you use a merged file (`_merge.pbf`) file as input, you need to specify the corresponding
bounding-box:
- Using a polygon to crop your region is preferred if you plan on applying the coastline fix. ```bash
- I do not recommend using the merged file as input for `osm2graph` because coastlines are often very long OpenStreetMap ways, and the plugin will not cut them (typically, you could reach Spain when using the merged Morbihan file). bash generate.sh inputs/europe/france/bretagne/morbihan_merge.pbf \
- I do not recommend using `osmosis` bounding-box option with `osm2graph` and/or `mapfile-writer` unless your bounding-box is small and does not contain sea or coastline. It is always preferred to generate an intermediate PBF file containing only your region (using `--write-pbf`). --bounding-box inputs/europe/france/bretagne/morbihan.bbox \
--id FR-E --name "Bretagne" \
--output outputs/bretagne
```
The script will automatically find the original file (`morbihan.pbf`) and use it to generate
the graph file (using the merged file can generate unexpected result... ).
If the input PBF file correspond to a region that has been cropped, it is better to specify
the polygon or bounding-box used for the generation of the PBF:
```bash
bash generate.sh inputs/europe/france/bretagne/morbihan_merge.pbf \
--bounding-box inputs/europe/france/bretagne/morbihan.bbox \
--bounding-polygon inputs/europe/france/bretagne/morbihan.poly \
--id FR-E --name "Bretagne" \
--output outputs/bretagne
```
If both the `--bounding-box` and `--bounding-polygon` option are specified, the bounding-box
will be ignored for the generation of the graph file (the `--bounding-polygon` option has no
effect for the generation of the Mapsforge file).
You can generate only the graph file (`--graph-only`) or the Mapsforge file
(`--mapsforge-only`). With `--mapsforge-only`, the `--id` and `--name` options are not
required.

10
generate.sh
View File

@ -39,6 +39,8 @@ function help_long() {
printf "\t-d,--debug\tEnable debug mode.\n" printf "\t-d,--debug\tEnable debug mode.\n"
printf "\t-i,--id\tMap ID.\n" printf "\t-i,--id\tMap ID.\n"
printf "\t-n,--name\tMap name.\n" printf "\t-n,--name\tMap name.\n"
printf "\t--graph-only\tDo not generate Mapsforge file.\n"
printf "\t--mapsforge-only\tDo not generate graph file.\n"
printf "\t-b,--bounding-box\tSpecify a bounding-box, either via a string or a file.\n" printf "\t-b,--bounding-box\tSpecify a bounding-box, either via a string or a file.\n"
printf "\t-p,--bounding-polygon\tSpecify a bounding-polygon, only used for OSM2Graph.\n" printf "\t-p,--bounding-polygon\tSpecify a bounding-polygon, only used for OSM2Graph.\n"
printf "\t-h,--help\tPrint this help.\n" printf "\t-h,--help\tPrint this help.\n"
@ -209,18 +211,18 @@ fi
if ${mapsforge_only}; then if ${mapsforge_only}; then
info "Skipping graph generation... " info "Skipping graph generation... "
else else
# Merge file as input... # Merge file as input...
if echo ${input_file} | grep "merge"; then if echo ${input_file} | grep "merge"; then
# Try to find the originl (.osm.pbf)... # Try to find the originl (.osm.pbf)...
input_file=$(echo ${input_file} | sed 's/_merge.pbf/.osm.pbf/') input_file=$(echo ${input_file} | sed 's/_merge.pbf/.osm.pbf/')
# ...or a generated .pbf... # ...or a generated .pbf...
if [ ! -e ${input_file} ]; then if [ ! -e ${input_file} ]; then
input_file=$(echo ${input_file} | sed 's/.osm//') input_file=$(echo ${input_file} | sed 's/.osm//')
fi fi
# If nothing is found, warn the user... # If nothing is found, warn the user...
if [ ! -e ${input_file} ]; then if [ ! -e ${input_file} ]; then
warning "Merge file specified but the original file was not found, the graph file may not be generated correctly from the merged file." warning "Merge file specified but the original file was not found, the graph file may not be generated correctly from the merged file."

50
inputs/coastline-fix.sh
View File

@ -20,6 +20,7 @@ region_dir=$(dirname $(realpath ${inpbf}))
region_name=$(basename ${inpbf} .pbf) region_name=$(basename ${inpbf} .pbf)
region_name=$(basename ${region_name} .osm) region_name=$(basename ${region_name} .osm)
poly_file=${region_dir}/${region_name}.poly poly_file=${region_dir}/${region_name}.poly
bbox_file=${region_dir}/${region_name}.bbox
if [ ! -e "${inpbf}" ]; then if [ ! -e "${inpbf}" ]; then
echo "${inpbf} does not exist, you may want to download it from https://download.geofabrik.de/index.html." echo "${inpbf} does not exist, you may want to download it from https://download.geofabrik.de/index.html."
@ -46,7 +47,7 @@ olddir=$(pwd)
cd ${INPUTS}/mapsforge-creator cd ${INPUTS}/mapsforge-creator
BBOX=$(perl poly2bb.pl "${poly_file}") BBOX=$(perl poly2bb.pl "${poly_file}")
echo "$BBOX" > ${INPUTS}/bbox/${region_name}.bbox echo "$BBOX" > ${bbox_file}
BBOX=(${BBOX//,/ }) BBOX=(${BBOX//,/ })
BOTTOM=${BBOX[0]} BOTTOM=${BBOX[0]}
@ -86,50 +87,3 @@ done
CMD="$CMD --wb file=${region_dir}/${region_name}_merge.pbf omitmetadata=true" CMD="$CMD --wb file=${region_dir}/${region_name}_merge.pbf omitmetadata=true"
echo $CMD echo $CMD
$CMD $CMD
exit 0
# Map
if [ "$SKIP_MAP_CREATION" != "true" ]; then
CMD="$OSMOSIS_HOME/bin/osmosis --rb file=$WORK_PATH/merge.pbf"
[ $MAP_TRANSFORM_FILE ] && CMD="$CMD --tt file=$MAP_TRANSFORM_FILE"
CMD="$CMD --mw file=$WORK_PATH/$NAME.map \
type=$2 \
bbox=$BOTTOM,$LEFT,$TOP,$RIGHT \
map-start-position=$LAT,$LON \
map-start-zoom=8 \
tag-values=$TAG_VALUES \
threads=$THREADS"
[ $3 ] && CMD="$CMD preferred-languages=$3"
[ $MAP_TAG_CONF_FILE ] && CMD="$CMD tag-conf-file=$MAP_TAG_CONF_FILE"
echo $CMD
$CMD || exit 1
# Check map size
if [ -f "$MAPS_PATH/$NAME.map" ]; then
OLD_SIZE=$(wc -c < "$MAPS_PATH/$NAME.map")
NEW_SIZE=$(wc -c < "$WORK_PATH/$NAME.map")
if [ $NEW_SIZE -lt $(($OLD_SIZE * 70 / 100)) ]; then
echo "$WORK_PATH/$NAME.map creation is significantly smaller."
exit 1
fi
fi
mv "$WORK_PATH/$NAME.map" "$MAPS_PATH/$NAME.map"
fi
# POI
if [ "$SKIP_POI_CREATION" != "true" ]; then
CMD="$OSMOSIS_HOME/bin/osmosis --rb file=$WORK_PATH/$NAME-latest.osm.pbf \
--pw file=$WORK_PATH/$NAME.poi"
[ $POI_TAG_CONF_FILE ] && CMD="$CMD tag-conf-file=$POI_TAG_CONF_FILE"
echo $CMD
$CMD || exit 1
mv "$WORK_PATH/$NAME.poi" "$POIS_PATH/$NAME.poi"
fi
# Post-process
rm -rf "$WORK_PATH"