Strabon
view README @ 1291:4787717f0ac1
[maven-release-plugin] prepare for next development iteration
| author | constant <Konstantina.Bereta@di.uoa.gr> |
|---|---|
| date | Sun Oct 20 20:10:12 2013 +0300 (2013-10-20) |
| parents | 3ec1c2e8f6d3 |
| children | 4dff2f1163d8 |
line source
1 Strabon
2 -----------
4 Introduction
5 ============
6 Strabon is a fully implemented semantic geospatial database system that can be
7 used to store linked geospatial data expressed in RDF and query them using an
8 extension of SPARQL. Strabon supports spatial selections, spatial joins, a rich
9 set of spatial functions similar to those offered by geospatial relational
10 database systems, support for multiple Coordinate Reference Systems and widely
11 used serializations for geometric objects such as WKT and GML. Strabon is built
12 on top of the well-known RDF store Sesame and extends Sesame’s components to be
13 able to manage thematic and spatial data that are stored in PostGIS.
15 The development of Strabon started in the context of European FP7 project
16 SemsorGrid4Env (Semantic Sensor Grids for Rapid Application Development for
17 Environmental Management) [http://www.semsorgrid4env.eu/]. Starting September
18 2011, Strabon is being utilized and extended with new functionalities in the
19 FP7 project TELEIOS (Virtual Observatory Infrastructure for Earth Observation
20 Data) [http://www.earthobservatory.eu/] which our group leads.
22 The query language of Strabon is called stSPARQL. stSPARQL can be used to query
23 data represented in an extension of RDF called stRDF. stRDF and stSPARQL have
24 been designed for representing and querying geospatial data that changes over
25 time (e.g., the growth of a city over the years due to new developments).
27 Given the very close relationship between stSPARQL and GeoSPARQL which is a
28 recent OGC standard for an extension of SPARQL for querying geospatial metadata,
29 we recently provided support for the Core, Geometry and Geometry Topology
30 extension of GeoSPARQL.
33 Conformance to GeoSPARQL
34 ========================
35 Strabon implements the `Core', the `Topology Vocabulary Extension', the
36 `Geometry Extension', the `Geometry Topology Extension', and the `RDFS Entailment
37 Extension' except for Req. 25
38 (http://www.opengis.net/spec/geosparql/1.0/req/rdfs-entailment-extension/bgp-rdfs-ent).
40 With respect to GML, Strabon supports the GML Profile corresponding to Simple
41 Features, that is, GML Simple Features Profile 2.0.
44 Strabon Homepage
45 ================
46 The homepage of Strabon is at http://www.strabon.di.uoa.gr/.
49 Demo
50 ====
51 You can find a demo of the system Strabon at http://test.strabon.di.uoa.gr/NOA/.
54 How to build and run Strabon from command line
55 ==============================================
56 Assuming you have already downloaded Strabon and you are in the top-level
57 directory of Strabon, issue the following command to build it from command line:
59 $ mvn clean package
61 In order to run automatically the JUnit tests, pass the option `-DskipTests=false'
62 to the above command. The complete command is the following:
64 $ mvn -DskipTests=false clean package
66 In case of an error during building of Strabon and assuming that the error does
67 not come from the JUnit tests, please have a look at the `Known Issues' section
68 below. If none of the known issues of that section applies, please contact the
69 developers through the Strabon mailing-list or submit a corresponding bug
70 (see `Bugs' section below).
72 After you have successfully built Strabon, you have access to the following
73 components:
75 * Strabon Endpoint
77 This is a SPARQL endpoint for Strabon. It is distributed as a war file so
78 you may deploy it in a Tomcat container. You may find the war file under
79 directory `endpoint/target'.
81 * Strabon Endpoint (standalone)
83 This is a SPARQL endpoint for Strabon like the above one, but it differs
84 only in that it does not require the user to have already set up a Tomcat
85 container. The standalone Strabon Endpoint may be run by issuing the
86 following command:
88 $ java -jar endpoint-exec/target/strabon-endpoint-executable-${version}.jar
90 After issuing the above command, you may access the Strabon Endpoint at
91 the following URL: <http://localhost:8080/>.
93 Please see the page at <http://hg.strabon.di.uoa.gr/Strabon/rev/674f8f91162b>
94 to find out other options that you may pass to the Tomcat container that
95 will run by the above command.
97 SPECIAL NOTE: if you need to configure the connection details to the
98 underlying database, you may do so in two ways:
99 1. By modifying file `endpoint/WebContent/WEB-INF/connection.properties'
100 before building Strabon and executing the above command.
101 2. After executing the above command, by visiting the following page by a
102 browser: <http://localhost:8080/ChangeConnection>
104 * Strabon Endpoint Client
106 This is a Java client for interacting with Strabon Endpoint or any other
107 SPARQL endpoint. It is packaged as a jar file and may be found under
108 directory `endpoint-client/target/' with name
109 `strabon-endpoint-client-${version}.jar'. This jar contains any dependencies
110 to other code, so may copy and paste it to your project and start playing
111 with the code immediately.
113 * Strabon script
115 The `strabon' script is located under the `scripts/' directory and it is the
116 main command-line tool for interacting with Strabon. You may use it to store
117 RDF data with geospatial information or query/update it using one of
118 stSPARQL or GeoSPARQL query languages.
120 * Endpoint script
122 The `endpoint' script is located under the `scripts' directory and it is the
123 main command-line tool for interacting with a `Strabon Endpoint'. You may
124 use it to do any operation you would like to do with the `strabon' script
125 above, but in contrast to the `strabon' script you need to have access to a
126 Strabon endpoint. Of course, the `Strabon Endpoint Client' component above
127 can be used as well as a command-line tool. At the time of writing, the
128 `Strabon Endpoint Client' component supports only querying of RDF data with
129 geospatial information.
132 Getting Started
133 ===============
134 To get started with Strabon please have a look at the tutorial for the stRDF
135 data model and stSPARQL query language, the User Guide, and the Developer Guide.
137 stRDF and stSPARQL tutorial
138 http://www.strabon.di.uoa.gr/files/stSPARQL_tutorial.pdf
141 stSPARQL Reference
142 ~~~~~~~~~~~~~~~~~~
143 The reference for the spatial and temporal extension functions defined in
144 stSPARQL can be found at http://www.strabon.di.uoa.gr/stSPARQL.
147 User Guide
148 ~~~~~~~~~~
149 Assuming that you are familiar with Maven, the following steps need to be
150 followed in order to use Strabon using Eclipse:
152 1. Install PostgreSQL from http://www.postgresql.org/download/. At the time of
153 this writing the latest PostgreSQL version is 9.1.
154 2. Install PostGIS from http://postgis.refractions.net/download/. At the time of
155 this writing we have tested Strabon with PostGIS 1.5.3. For the
156 temporal features of stSPARQL download the PostgreSQL temporal extension
157 from here: https://github.com/jeff-davis/PostgreSQL-Temporal and follow
158 the installation guide.
159 3. Install Maven from http://maven.apache.org/download.html. At the time of this
160 writing the latest Maven version is 3.0.4.
161 4. Install Eclipse from http://www.eclipse.org/downloads/. At the time of this
162 writing the latest Eclipse version is 3.7.2.
163 5. Install the m2e plugin for Eclipse from http://www.eclipse.org/m2e/.
164 6. Install the MercurialEclipse plugin for Eclipse from
165 http://javaforge.com/project/HGE .
166 7. From Eclipse, go to File --> Import --> Mercurial --> Clone Existing
167 Mercurial Repository --> Next. In the URL textarea paste the following
168 URL: http://hg.strabon.di.uoa.gr/StrabonUser and then press Next --> Next -->
169 Finish. If you used the default settings, you should have a new project named
170 StrabonMain. Right click on the project and select Configure --> Convert to
171 Maven project. Eclipse will enable Maven dependency management for the
172 project, download any dependencies and build the project.
175 Storing stRDF graphs and evaluating stSPARQL queries
176 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
177 You can see some examples in the classes gr.uoa.di.strabon.example.PostgisExample
178 and gr.uoa.di.strabon.example.PostgisExample2.
180 Tuning PostgreSQL
181 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
182 The default settings of Postgres are rather conservative. As a result, parameter
183 tuning is neccessary for speeding up Postgres, therefore Strabon. If you are
184 using Strabon to compare its performance against your implementation of
185 stSPARQL/GeoSPARQL, you are *strongly* encouraged to contact us using the Strabon
186 Users mailing list for assistance on tuning Postgres.
188 You can follow the instructions below for tuning a Postgres server running on an
189 Ubuntu machine that is dedicated to PostgreSQL and Strabon.
191 1. Append the following text at the end of postgresql.conf.
192 *Uncomment* the appropriate lines.
194 ### RAM
195 ## 4 GB of RAM
196 #shared_buffers = 3GB
197 #effective_cache_size = 3GB
198 #maintenance_work_mem = 1GB
199 #work_mem = 2GB
200 ## 8 GB of RAM
201 #shared_buffers = 5GB
202 #effective_cache_size = 6GB
203 #maintenance_work_mem = 2GB
204 #work_mem = 5GB
205 ## 16 GB of RAM
206 #shared_buffers = 10GB
207 #effective_cache_size = 14GB
208 #maintenance_work_mem = 4GB
209 #work_mem = 10GB
210 ## 24 GB of RAM
211 #shared_buffers = 16GB
212 #effective_cache_size = 22GB
213 #maintenance_work_mem = 6GB
214 #work_mem = 15GB
215 ## 48 GB of RAM
216 #shared_buffers = 32GB
217 #effective_cache_size = 46GB
218 #maintenance_work_mem = 8GB
219 #work_mem = 30GB
220 ## 64 GB of RAM
221 # contact us to find out!
222 ### HD
223 ## RAID with ordinary 7.200 disks
224 #random_page_cost = 3.5 #3.0-3.5
225 ## High-End NAS/SAN
226 #random_page_cost = 2 #1.5-2.5
227 ## Amazon EBS/Heroku
228 #random_page_cost = 1.3 #1.1-2.0
229 ## SSD array
230 #random_page_cost = 2.0 #1.5-2.5
231 ### Planner options
232 # Increase the following values in order to avoid using the GEQO planner.
233 # Small values (<8) reduce planning time but may produce inferior query plans
234 #
235 geqo_threshold = 15 # keep this value larger that the following two parameters
236 from_collapse_limit = 14
237 join_collapse_limit = 14
238 ### Misc
239 default_statistics_target = 10000
240 constraint_exclusion = on
241 checkpoint_completion_target = 0.9
242 wal_buffers = 32MB
243 checkpoint_segments = 64
244 ### Connections
245 max_connections = 10
247 2. Append the following lines at the end of /etc/sysctl.conf
248 *Uncomment* the appropriate lines.
250 ## 4 GB of RAM
251 #kernel.shmmax = 3758096384
252 #kernel.shmall = 3758096384
253 #kernel.shmmni = 4096
254 ## 8 GB of RAM
255 #kernel.shmmax = 5905580032
256 #kernel.shmall = 5905580032
257 #kernel.shmmni = 4096
258 ## 16 GB of RAM
259 #kernel.shmmax = 11274289152
260 #kernel.shmall = 11274289152
261 #kernel.shmmni = 4096
262 ## 24 GB of RAM
263 #kernel.shmmax = 17716740096
264 #kernel.shmall = 17716740096
265 #kernel.shmmni = 4096
266 ## 48 GB of RAM
267 #kernel.shmmax = 35433480192
268 #kernel.shmall = 35433480192
269 #kernel.shmmni = 4224
270 ## 64 GB of RAM
271 # contact us to find out!
273 3. Apply all changes by executing
275 $ sudo sysctl -p
276 $ sudo /etc/init.d/postgresql restart
278 4. Prepare for the next run by issuing the command
280 $ sudo -u postgres psql -c 'VACUUM ANALYZE;' db
282 or
284 $ psql -c 'VACUUM ANALYZE;' db
286 where db is the name of the Postgres database that Strabon will use.
289 Developer Guide
290 ~~~~~~~~~~~~~~~
291 Assuming that you are familiar with Maven, the following steps need to be
292 followed in order to use Strabon using Eclipse:
294 1. Install PostgreSQL from http://www.postgresql.org/download/. At the time of
295 this writing the latest PostgreSQL version is 9.1.
296 2. Install PostGIS from http://postgis.refractions.net/download/. At the time of
297 this writing we have tested Strabon with PostGIS 1.5.3. For the
298 temporal features of stSPARQL download the PostgreSQL temporal extension
299 from here: https://github.com/jeff-davis/PostgreSQL-Temporal and follow
300 the installation guide.
301 3. Install Maven from http://maven.apache.org/download.html. At the time of this
302 writing the latest Maven version is 3.0.4.
303 4. Install Eclipse from http://www.eclipse.org/downloads/. At the time of this
304 writing the latest Eclipse version is 3.7.2.
305 5. Install the m2e plugin for Eclipse from http://www.eclipse.org/m2e/.
306 6. Install the MercurialEclipse plugin for Eclipse from
307 http://javaforge.com/project/HGE.
308 7. From Eclipse, go to File --> Import --> Mercurial --> Clone Existing
309 Mercurial Repository --> Next. In the URL textarea paste the following
310 URL: http://hg.strabon.di.uoa.gr/Strabon and then press Next --> Next -->
311 Finish. If you used the default settings, you should have a new project named
312 StrabonMain. Right click on the project and select Configure --> Convert to
313 Maven project. Eclipse will enable Maven dependency management for the
314 project, download any dependencies and build the project.
317 Tester Guide
318 ~~~~~~~~~~~~
319 Assuming again that you are familiar with Maven and Junit these are the steps
320 you need to follow to test the functionality of Strabon:
323 * Using Eclipse
325 If you want to create a new test:
327 1. Import Strabon into Eclipse as explained in the Developer Guide.
328 2. Go to strabon-testsuite project.
329 3. Create a new folder (Recommended folder name: <test's name>) and place inside the following files:
330 3.1. An ntriples or nquads file with the test dataset (with .nt or .nq extension).
331 3.2. Pairs of files with sparql test queries and expected test results in xml format.
332 Notice that each pair must have the same name and .rq extension for the queryFile and .srx extension for the resultsFile.
333 4. Create a test class that extends TemplateTest class.
334 5. If you have followed the recommendations the test is ready. If you have different names or location for your
335 files, insert them explicitly in the constructor of the class. WARNING: All prefixes must be placed in file "prefixes" so that
336 every time a namespace changes, we have to change it just once.
338 If you want to run a test:
340 1. Right-click on the test class.
341 2. Select "Run as JUnit Test".
342 3. Database properties are retrieved from database.properties file. If you want, you can change a property
343 "on the fly" with an environment variable.
346 * Command Line
348 If you want to run all the tests:
350 1. Go to Strabon directory (root directory of all the subprojects).
351 2. Run "mvn test -DskipTests=false".
352 3. Optionally you can pass an environment variable with "-DvariableName=variableValue".
355 Storing stRDF graphs and evaluating stSPARQL queries
356 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
357 You can see some examples in the classes
358 eu.earthobservatory.runtime.postgis.StoreOp and
359 eu.earthobservatory.runtime.postgis.QueryOp.
362 Download
363 ========
364 You can download the source code of the latest version of Strabon by accessing
365 our public mercurial repository located at http://hg.strabon.di.uoa.gr/Strabon.
366 You can find more information on how to use and extend Strabon at the Getting
367 Started section.
370 Publications
371 ============
372 You can learn about stRDF data model and stSPARQL query language employed in
373 Strabon by reading our tutorial under the Getting Started section and/or the
374 publications given on this page.
376 The current versions of stRDF and stSPARQL which are based on OGC standards are
377 presented in the following documents:
378 * K. Kyzirakos, M. Karpathiotakis, and M. Koubarakis. Strabon: A Semantic
379 Geospatial DBMS. In Internatioanl Semantic Web Conference (ISWC'12). Boston,
380 USA, November 11-15, 2012.
381 [pdf: http://strabon.di.uoa.gr/files/strabon-iswc.pdf]
383 * Manolis Koubarakis, Kostis Kyzirakos, Babis Nikolaou, Michael Sioutis, and
384 Stavros Vassos. A data model and query language for an extension of RDF with
385 time and space. Deliverable D2.1, European ICT project TELEIOS, 2011.
386 [pdf: http://strabon.di.uoa.gr/files/deliv2-1-re-revised.pdf]
388 The temporal features of stRDF and stSPARQL are presented in the following
389 document:
390 * K. Bereta, P. Smeros and M. Koubarakis. Representing and Querying the
391 Valid Time of Triples for Linked Geospatial Data. In the 10th Extended
392 Semantic Web Conference (ESWC 2013). Montpellier, France. May 26-30, 2013.
393 [pdf: http://www.strabon.di.uoa.gr/files/eswc2013.pdf]
395 The initial versions of stRDF and stSPARQL that are based on constraint
396 databases are presented in the following publications:
397 * Manolis Koubarakis and Kostis Kyzirakos. Modeling and Querying Metadata in
398 the Semantic Sensor Web: the Model stRDF and the Query Language stSPARQL.
399 In 7th Extended Semantic Web Conference (ESWC 2010). Heraklion, Crete,
400 30 May - 03 June, 2010.
401 [pdf: http://strabon.di.uoa.gr/files/stSPARQL.pdf]
403 * Kostis Kyzirakos, Manos Karpathiotakis and Manolis Koubarakis. Developing
404 Registries for the Semantic Sensor Web using stRDF and stSPARQL (short
405 paper).
406 In Proceedings of 3rd International workshop on Semantic Sensor Networks
407 2010, in conjunction with ISWC 2010, November 2010, Shanghai, China.
408 [pdf: http://strabon.di.uoa.gr/files/strabon.pdf]
410 Applications of stRDF, stSPARQL, and the system Strabon are described here:
411 * Alasdair J. G. Gray, Raúl García-Castro, Kostis Kyzirakos, Manos
412 Karpathiotakis, Jean-Paul Calbimonte, Kevin Page, Jason Sadler, Alex
413 Frazer, Ixent Galpin, Alvaro A. A. Fernandes, Norman W. Paton, Oscar
414 Corcho, Manolis Koubarakis, David De Roure, Kirk Martinez and Asunción
415 Gómez-Pérez. A Semantically Enabled Service Architecture for Mashups over
416 Streaming and Stored Data. In 8th Extended Semantic Web Conference (ESWC
417 2011). Heraklion, Crete, May 20 - June 2, 2011.
418 [pdf: http://strabon.di.uoa.gr/files/Gray2011Architecture.pdf]
420 * A.J.G. Gray, J. Sadler, O. Kit, K. Kyzirakos, M. Karpathiotakis, J.-P.
421 Calbimonte, K. Page, R. García-Castro, A. Frazer, I. Galpin, A.A.A.
422 Fernandes, N.W. Paton, O. Corcho, M. Koubarakis, D.D. Roure, K. Martinez,
423 A. Gómez-Pérez. A Semantic Sensor Web for Environmental Decision Support
424 Applications. Sensors. 11, 8855-8887.
425 [pdf: http://strabon.di.uoa.gr/files/sensors-11-08855.pdf]
427 Coming up soon:
428 Strabon will soon support an extension of RDF for incomplete geospatial
429 information. The following publication gives a preview of the relevant research
430 problems:
431 * M. Koubarakis, K. Kyzirakos, M. Karpathiotakis, C. Nikolaou, M. Sioutis,
432 S. Vassos, D. Michail, T. Herekakis, C. Kontoes and I. Papoutsis. Challenges
433 for Qualitative Spatial Reasoning in Linked Geospatial Data. In Proceedings
434 of IJCAI 2011 Workshop on Benchmarks and Applications of Spatial Reasoning,
435 Barcelona, Spain.
436 [pdf: http://www.earthobservatory.eu/publications/SciQL_ADASS2011.pdf]
438 * C. Nikolaou and M. Koubarakis: "Querying Linked Geospatial Data with
439 Incomplete Information". In 5th International Terra Cognita Workshop -
440 Foundations, Technologies and Applications of the Geospatial Web. In
441 conjunction with the 11th International Semantic Web Conference, Boston,
442 USA, November, 2012.
443 [http://www.earthobservatory.eu/publications/iswc-workshop.pdf]
446 Contributors
447 ============
448 The system Strabon has been developed by the following members of our team:
450 * Manos Karpathiotakis <mk@di.uoa.gr>
451 * Kostis Kyzirakos <kkyzir@di.uoa.gr>
452 * Manolis Koubarakis <koubarak@di.uoa.gr>
453 * Giorgos Garbis <ggarbis@di.uoa.gr>
454 * Konstantina Bereta <konstantina.bereta@di.uoa.gr>
455 * Charalampos Nikolaou <charnik@di.uoa.gr>
456 * Stella Gianakopoulou <sgian@di.uoa.gr>
457 * Panagiotis Smeros <psmeros@di.uoa.gr>
458 * Kallirroi Dogani <kallirroi@di.uoa.gr>
461 Mailing-list
462 ============
463 Currently, we maintain the following mailing lists:
465 * Strabon-users, is used as a communication channel for Strabon users.
466 To subscribe to the mailing-list, please visit page
467 http://cgi.di.uoa.gr/~mailman/listinfo/strabon-users. To post e-mails
468 to Strabon-users mailing-list, write to strabon-users@di.uoa.gr.
470 * Strabon-devel, is used as a communication channel with the developers
471 of Strabon. To subscribe to the mailing-list, please visit page
472 http://cgi.di.uoa.gr/~mailman/listinfo/strabon-devel. To post e-mails
473 to Strabon-devel mailing-list, write to strabon-devel@di.uoa.gr.
476 Bugs
477 ====
478 Please report bugs to http://bug.strabon.di.uoa.gr/report or
479 the Strabon-devel mailing-list Strabon-devel@di.uoa.gr.
482 Known Issues
483 ============
484 * By default, Tomcat uses ISO-8859-1 character encoding when decoding URLs received
485 from a browser. This can cause problems when encoding is UTF-8, and you are using
486 international characters. In order to fix this, edit conf/server.xml and find the
487 line where the Connector is defined. Add the parameter URIEncoding and set it to
488 UTF-8. For example:
490 <Connector port="8080" protocol="HTTP/1.1"
491 connectionTimeout="20000"
492 URIEncoding="UTF-8"
493 redirectPort="8443" />
495 * Building and executing any maven goals fails for maven versions <3.0 due to a
496 dependency to the `shade' plugin that is available only for maven version 3.0
497 (http://maven.apache.org/plugins/maven-shade-plugin/). In such systems, you may
498 disable execution of this plugin by setting the environmental variable
499 `shade.skip'. For example, to build Strabon using maven version 2.0 you may
500 execute the following command:
502 $ mvn clean package -Dshade.skip
504 * When using MonetDB as a backend, the following source code of MonetDB must be
505 used:
506 https://hg.strabon.di.uoa.gr/MonetDB/
509 License
510 =======
511 This Source Code Form is subject to the terms of the Mozilla Public
512 License, v. 2.0. If a copy of the MPL was not distributed with this
513 file, You can obtain one at http://mozilla.org/MPL/2.0/.
515 Copyright (C) 2010, 2011, 2012, 2013 Pyravlos Team
517 http://www.strabon.di.uoa.gr/
520 How to apply the license
521 ========================
522 * In the beginning of script files (after the shell directive) paste the
523 following statement:
524 #
525 # This Source Code Form is subject to the terms of the Mozilla Public
526 # License, v. 2.0. If a copy of the MPL was not distributed with this
527 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
528 #
529 # Copyright (C) 2010, 2011, 2012, 2013 Pyravlos Team
530 #
531 # http://www.strabon.di.uoa.gr/
532 #
534 * In the beginning of Java source code files paste the following statement:
535 /**
536 * This Source Code Form is subject to the terms of the Mozilla Public
537 * License, v. 2.0. If a copy of the MPL was not distributed with this
538 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
539 *
540 * Copyright (C) 2010, 2011, 2012, 2013 Pyravlos Team
541 *
542 * http://www.strabon.di.uoa.gr/
543 */
545 * In the beginning of HTML/XML files paste the following statement:
546 <!-- This Source Code Form is subject to the terms of the Mozilla Public
547 - License, v. 2.0. If a copy of the MPL was not distributed with this
548 - file, You can obtain one at http://mozilla.org/MPL/2.0/.
549 -
550 - Copyright (C) 2010, 2011, 2012, 2013 Pyravlos Team
551 -
552 - http://www.strabon.di.uoa.gr/
553 -->