Strabon
view README @ 1246:da14a303bf2c
merge with default
| author | Konstantina Bereta <Konstantina.Bereta@di.uoa.gr> |
|---|---|
| date | Thu Aug 01 14:40:30 2013 +0300 (2013-08-01) |
| parents | 3803532cc513 079711dfc125 |
| children | 3ec1c2e8f6d3 |
line source
1 Strabon
2 -----------
4 Introduction
5 ============
6 Strabon is a fully implemented semantic geospatial database system that can be
7 used to store linked geospatial data expressed in RDF and query them using an
8 extension of SPARQL. Strabon supports spatial selections, spatial joins, a rich
9 set of spatial functions similar to those offered by geospatial relational
10 database systems, support for multiple Coordinate Reference Systems and widely
11 used serializations for geometric objects such as WKT and GML. Strabon is built
12 on top of the well-known RDF store Sesame and extends Sesame’s components to be
13 able to manage thematic and spatial data that are stored in PostGIS.
15 The development of Strabon started in the context of European FP7 project
16 SemsorGrid4Env (Semantic Sensor Grids for Rapid Application Development for
17 Environmental Management) [http://www.semsorgrid4env.eu/]. Starting September
18 2011, Strabon is being utilized and extended with new functionalities in the
19 FP7 project TELEIOS (Virtual Observatory Infrastructure for Earth Observation
20 Data) [http://www.earthobservatory.eu/] which our group leads.
22 The query language of Strabon is called stSPARQL. stSPARQL can be used to query
23 data represented in an extension of RDF called stRDF. stRDF and stSPARQL have
24 been designed for representing and querying geospatial data that changes over
25 time (e.g., the growth of a city over the years due to new developments).
27 Currently, only the geospatial features of stSPARQL have been implemented fully.
28 The temporal features have been implemented under the "temporals" branch of the repository.
30 Given the very close relationship between stSPARQL and GeoSPARQL which is a
31 recent OGC standard for an extension of SPARQL for querying geospatial metadata,
32 we recently provided support for the Core, Geometry and Geometry Topology
33 extension of GeoSPARQL.
36 Conformance to GeoSPARQL
37 ========================
38 Strabon implements the `Core', the `Topology Vocabulary Extension', the
39 `Geometry Extension', the `Geometry Topology Extension', and the `RDFS Entailment
40 Extension' except for Req. 25
41 (http://www.opengis.net/spec/geosparql/1.0/req/rdfs-entailment-extension/bgp-rdfs-ent).
43 With respect to GML, Strabon supports the GML Profile corresponding to Simple
44 Features, that is, GML Simple Features Profile 2.0.
47 Strabon Homepage
48 ================
49 The homepage of Strabon is at http://www.strabon.di.uoa.gr/.
52 Demo
53 ====
54 You can find a demo of the system Strabon at http://test.strabon.di.uoa.gr/NOA/.
57 How to build and run Strabon from command line
58 ==============================================
59 Assuming you have already downloaded Strabon and you are in the top-level
60 directory of Strabon, issue the following command to build it from command line:
62 $ mvn clean package
64 In order to run automatically the JUnit tests, pass the option `-DskipTests=false'
65 to the above command. The complete command is the following:
67 $ mvn -DskipTests=false clean package
69 In case of an error during building of Strabon and assuming that the error does
70 not come from the JUnit tests, please have a look at the `Known Issues' section
71 below. If none of the known issues of that section applies, please contact the
72 developers through the Strabon mailing-list or submit a corresponding bug
73 (see `Bugs' section below).
75 After you have successfully built Strabon, you have access to the following
76 components:
78 * Strabon Endpoint
80 This is a SPARQL endpoint for Strabon. It is distributed as a war file so
81 you may deploy it in a Tomcat container. You may find the war file under
82 directory `endpoint/target'.
84 * Strabon Endpoint (standalone)
86 This is a SPARQL endpoint for Strabon like the above one, but it differs
87 only in that it does not require the user to have already set up a Tomcat
88 container. The standalone Strabon Endpoint may be run by issuing the
89 following command:
91 $ java -jar endpoint-exec/target/strabon-endpoint-executable-${version}.jar
93 After issuing the above command, you may access the Strabon Endpoint at
94 the following URL: <http://localhost:8080/>.
96 Please see the page at <http://hg.strabon.di.uoa.gr/Strabon/rev/674f8f91162b>
97 to find out other options that you may pass to the Tomcat container that
98 will run by the above command.
100 SPECIAL NOTE: if you need to configure the connection details to the
101 underlying database, you may do so in two ways:
102 1. By modifying file `endpoint/WebContent/WEB-INF/connection.properties'
103 before building Strabon and executing the above command.
104 2. After executing the above command, by visiting the following page by a
105 browser: <http://localhost:8080/ChangeConnection>
107 * Strabon Endpoint Client
109 This is a Java client for interacting with Strabon Endpoint or any other
110 SPARQL endpoint. It is packaged as a jar file and may be found under
111 directory `endpoint-client/target/' with name
112 `strabon-endpoint-client-${version}.jar'. This jar contains any dependencies
113 to other code, so may copy and paste it to your project and start playing
114 with the code immediately.
116 * Strabon script
118 The `strabon' script is located under the `scripts/' directory and it is the
119 main command-line tool for interacting with Strabon. You may use it to store
120 RDF data with geospatial information or query/update it using one of
121 stSPARQL or GeoSPARQL query languages.
123 * Endpoint script
125 The `endpoint' script is located under the `scripts' directory and it is the
126 main command-line tool for interacting with a `Strabon Endpoint'. You may
127 use it to do any operation you would like to do with the `strabon' script
128 above, but in contrast to the `strabon' script you need to have access to a
129 Strabon endpoint. Of course, the `Strabon Endpoint Client' component above
130 can be used as well as a command-line tool. At the time of writing, the
131 `Strabon Endpoint Client' component supports only querying of RDF data with
132 geospatial information.
135 Getting Started
136 ===============
137 To get started with Strabon please have a look at the tutorial for the stRDF
138 data model and stSPARQL query language, the User Guide, and the Developer Guide.
140 stRDF and stSPARQL tutorial
141 http://www.strabon.di.uoa.gr/files/stSPARQL_tutorial.pdf
144 stSPARQL Reference
145 ~~~~~~~~~~~~~~~~~~
146 The reference for the spatial and temporal extension functions defined in
147 stSPARQL can be found at http://www.strabon.di.uoa.gr/stSPARQL#spatial and
148 http://www.strabon.di.uoa.gr/stSPARQL#temporals respectively.
151 User Guide
152 ~~~~~~~~~~
153 Assuming that you are familiar with Maven, the following steps need to be
154 followed in order to use Strabon using Eclipse:
156 1. Install PostgreSQL from http://www.postgresql.org/download/. At the time of
157 this writing the latest PostgreSQL version is 9.1.
158 2. Install PostGIS from http://postgis.refractions.net/download/. At the time of
159 this writing we have tested Strabon with PostGIS 1.5.3.
160 3. Install Maven from http://maven.apache.org/download.html. At the time of this
161 writing the latest Maven version is 3.0.4.
162 4. Install Eclipse from http://www.eclipse.org/downloads/. At the time of this
163 writing the latest Eclipse version is 3.7.2.
164 5. Install the m2e plugin for Eclipse from http://www.eclipse.org/m2e/.
165 6. Install the MercurialEclipse plugin for Eclipse from
166 http://javaforge.com/project/HGE .
167 7. From Eclipse, go to File --> Import --> Mercurial --> Clone Existing
168 Mercurial Repository --> Next. In the URL textarea paste the following
169 URL: http://hg.strabon.di.uoa.gr/StrabonUser and then press Next --> Next -->
170 Finish. If you used the default settings, you should have a new project named
171 StrabonMain. Right click on the project and select Configure --> Convert to
172 Maven project. Eclipse will enable Maven dependency management for the
173 project, download any dependencies and build the project.
176 Storing stRDF graphs and evaluating stSPARQL queries
177 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
178 You can see some examples in the classes gr.uoa.di.strabon.example.PostgisExample
179 and gr.uoa.di.strabon.example.PostgisExample2.
181 Tuning PostgreSQL
182 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
183 The default settings of Postgres are rather conservative. As a result, parameter
184 tuning is neccessary for speeding up Postgres, therefore Strabon. If you are
185 using Strabon to compare its performance against your implementation of
186 stSPARQL/GeoSPARQL, you are *strongly* encouraged to contact us using the Strabon
187 Users mailing list for assistance on tuning Postgres.
189 You can follow the instructions below for tuning a Postgres server running on an
190 Ubuntu machine that is dedicated to PostgreSQL and Strabon.
192 1. Append the following text at the end of postgresql.conf.
193 *Uncomment* the appropriate lines.
195 ### RAM
196 ## 4 GB of RAM
197 #shared_buffers = 3GB
198 #effective_cache_size = 3GB
199 #maintenance_work_mem = 1GB
200 #work_mem = 2GB
201 ## 8 GB of RAM
202 #shared_buffers = 5GB
203 #effective_cache_size = 6GB
204 #maintenance_work_mem = 2GB
205 #work_mem = 5GB
206 ## 16 GB of RAM
207 #shared_buffers = 10GB
208 #effective_cache_size = 14GB
209 #maintenance_work_mem = 4GB
210 #work_mem = 10GB
211 ## 24 GB of RAM
212 #shared_buffers = 16GB
213 #effective_cache_size = 22GB
214 #maintenance_work_mem = 6GB
215 #work_mem = 15GB
216 ## 48 GB of RAM
217 #shared_buffers = 32GB
218 #effective_cache_size = 46GB
219 #maintenance_work_mem = 8GB
220 #work_mem = 30GB
221 ## 64 GB of RAM
222 # contact us to find out!
223 ### HD
224 ## RAID with ordinary 7.200 disks
225 #random_page_cost = 3.5 #3.0-3.5
226 ## High-End NAS/SAN
227 #random_page_cost = 2 #1.5-2.5
228 ## Amazon EBS/Heroku
229 #random_page_cost = 1.3 #1.1-2.0
230 ## SSD array
231 #random_page_cost = 2.0 #1.5-2.5
232 ### Planner options
233 # Increase the following values in order to avoid using the GEQO planner.
234 # Small values (<8) reduce planning time but may produce inferior query plans
235 #
236 geqo_threshold = 15 # keep this value larger that the following two parameters
237 from_collapse_limit = 14
238 join_collapse_limit = 14
239 ### Misc
240 default_statistics_target = 10000
241 constraint_exclusion = on
242 checkpoint_completion_target = 0.9
243 wal_buffers = 32MB
244 checkpoint_segments = 64
245 ### Connections
246 max_connections = 10
248 2. Append the following lines at the end of /etc/sysctl.conf
249 *Uncomment* the appropriate lines.
251 ## 4 GB of RAM
252 #kernel.shmmax = 3758096384
253 #kernel.shmall = 3758096384
254 #kernel.shmmni = 4096
255 ## 8 GB of RAM
256 #kernel.shmmax = 5905580032
257 #kernel.shmall = 5905580032
258 #kernel.shmmni = 4096
259 ## 16 GB of RAM
260 #kernel.shmmax = 11274289152
261 #kernel.shmall = 11274289152
262 #kernel.shmmni = 4096
263 ## 24 GB of RAM
264 #kernel.shmmax = 17716740096
265 #kernel.shmall = 17716740096
266 #kernel.shmmni = 4096
267 ## 48 GB of RAM
268 #kernel.shmmax = 35433480192
269 #kernel.shmall = 35433480192
270 #kernel.shmmni = 4224
271 ## 64 GB of RAM
272 # contact us to find out!
274 3. Apply all changes by executing
276 $ sudo sysctl -p
277 $ sudo /etc/init.d/postgresql restart
279 4. Prepare for the next run by issuing the command
281 $ sudo -u postgres psql -c 'VACUUM ANALYZE;' db
283 or
285 $ psql -c 'VACUUM ANALYZE;' db
287 where db is the name of the Postgres database that Strabon will use.
290 Developer Guide
291 ~~~~~~~~~~~~~~~
292 Assuming that you are familiar with Maven, the following steps need to be
293 followed in order to use Strabon using Eclipse:
295 1. Install PostgreSQL from http://www.postgresql.org/download/. At the time of
296 this writing the latest PostgreSQL version is 9.1.
297 2. Install PostGIS from http://postgis.refractions.net/download/. At the time of
298 this writing we have tested Strabon with PostGIS 1.5.3.
299 3. Install Maven from http://maven.apache.org/download.html. At the time of this
300 writing the latest Maven version is 3.0.4.
301 4. Install Eclipse from http://www.eclipse.org/downloads/. At the time of this
302 writing the latest Eclipse version is 3.7.2.
303 5. Install the m2e plugin for Eclipse from http://www.eclipse.org/m2e/.
304 6. Install the MercurialEclipse plugin for Eclipse from
305 http://javaforge.com/project/HGE.
306 7. From Eclipse, go to File --> Import --> Mercurial --> Clone Existing
307 Mercurial Repository --> Next. In the URL textarea paste the following
308 URL: http://hg.strabon.di.uoa.gr/Strabon and then press Next --> Next -->
309 Finish. If you used the default settings, you should have a new project named
310 StrabonMain. Right click on the project and select Configure --> Convert to
311 Maven project. Eclipse will enable Maven dependency management for the
312 project, download any dependencies and build the project.
315 Tester Guide
316 ~~~~~~~~~~~~
317 Assuming again that you are familiar with Maven and Junit these are the steps
318 you need to follow to test the functionality of Strabon:
321 * Using Eclipse
323 If you want to create a new test:
325 1. Import Strabon into Eclipse as explained in the Developer Guide.
326 2. Go to strabon-testsuite project.
327 3. Create a new folder (Recommended folder name: <test's name>) and place inside the following files:
328 3.1. An ntriples or nquads file with the test dataset (with .nt or .nq extension).
329 3.2. Pairs of files with sparql test queries and expected test results in xml format.
330 Notice that each pair must have the same name and .rq extension for the queryFile and .srx extension for the resultsFile.
331 4. Create a test class that extends TemplateTest class.
332 5. If you have followed the recommendations the test is ready. If you have different names or location for your
333 files, insert them explicitly in the constructor of the class. WARNING: All prefixes must be placed in file "prefixes" so that
334 every time a namespace changes, we have to change it just once.
336 If you want to run a test:
338 1. Right-click on the test class.
339 2. Select "Run as JUnit Test".
340 3. Database properties are retrieved from database.properties file. If you want, you can change a property
341 "on the fly" with an environment variable.
344 * Command Line
346 If you want to run all the tests:
348 1. Go to Strabon directory (root directory of all the subprojects).
349 2. Run "mvn test -DskipTests=false".
350 3. Optionally you can pass an environment variable with "-DvariableName=variableValue".
353 Storing stRDF graphs and evaluating stSPARQL queries
354 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
355 You can see some examples in the classes
356 eu.earthobservatory.runtime.postgis.StoreOp and
357 eu.earthobservatory.runtime.postgis.QueryOp.
360 Download
361 ========
362 You can download the source code of the latest version of Strabon by accessing
363 our public mercurial repository located at http://hg.strabon.di.uoa.gr/Strabon.
364 You can find more information on how to use and extend Strabon at the Getting
365 Started section.
368 Publications
369 ============
370 You can learn about stRDF data model and stSPARQL query language employed in
371 Strabon by reading our tutorial under the Getting Started section and/or the
372 publications given on this page.
374 The current versions of stRDF and stSPARQL which are based on OGC standards are
375 presented in the following documents:
376 * K. Kyzirakos, M. Karpathiotakis, and M. Koubarakis. Strabon: A Semantic
377 Geospatial DBMS. In Internatioanl Semantic Web Conference (ISWC'12). Boston,
378 USA, November 11-15, 2012.
379 [pdf: http://strabon.di.uoa.gr/files/strabon-iswc.pdf]
381 * Manolis Koubarakis, Kostis Kyzirakos, Babis Nikolaou, Michael Sioutis, and
382 Stavros Vassos. A data model and query language for an extension of RDF with
383 time and space. Deliverable D2.1, European ICT project TELEIOS, 2011.
384 [pdf: http://strabon.di.uoa.gr/files/deliv2-1-re-revised.pdf]
386 The temporal features of stRDF and stSPARQL are presented in the following
387 document:
388 * K. Bereta, P. Smeros and M. Koubarakis. Representing and Querying the
389 Valid Time of Triples for Linked Geospatial Data. In the 10th Extended
390 Semantic Web Conference (ESWC 2013). Montpellier, France. May 26-30, 2013.
391 [pdf: http://www.strabon.di.uoa.gr/files/eswc2013.pdf]
393 The initial versions of stRDF and stSPARQL that are based on constraint
394 databases are presented in the following publications:
395 * Manolis Koubarakis and Kostis Kyzirakos. Modeling and Querying Metadata in
396 the Semantic Sensor Web: the Model stRDF and the Query Language stSPARQL.
397 In 7th Extended Semantic Web Conference (ESWC 2010). Heraklion, Crete,
398 30 May - 03 June, 2010.
399 [pdf: http://strabon.di.uoa.gr/files/stSPARQL.pdf]
401 * Kostis Kyzirakos, Manos Karpathiotakis and Manolis Koubarakis. Developing
402 Registries for the Semantic Sensor Web using stRDF and stSPARQL (short
403 paper).
404 In Proceedings of 3rd International workshop on Semantic Sensor Networks
405 2010, in conjunction with ISWC 2010, November 2010, Shanghai, China.
406 [pdf: http://strabon.di.uoa.gr/files/strabon.pdf]
408 Applications of stRDF, stSPARQL, and the system Strabon are described here:
409 * Alasdair J. G. Gray, Raúl García-Castro, Kostis Kyzirakos, Manos
410 Karpathiotakis, Jean-Paul Calbimonte, Kevin Page, Jason Sadler, Alex
411 Frazer, Ixent Galpin, Alvaro A. A. Fernandes, Norman W. Paton, Oscar
412 Corcho, Manolis Koubarakis, David De Roure, Kirk Martinez and Asunción
413 Gómez-Pérez. A Semantically Enabled Service Architecture for Mashups over
414 Streaming and Stored Data. In 8th Extended Semantic Web Conference (ESWC
415 2011). Heraklion, Crete, May 20 - June 2, 2011.
416 [pdf: http://strabon.di.uoa.gr/files/Gray2011Architecture.pdf]
418 * A.J.G. Gray, J. Sadler, O. Kit, K. Kyzirakos, M. Karpathiotakis, J.-P.
419 Calbimonte, K. Page, R. García-Castro, A. Frazer, I. Galpin, A.A.A.
420 Fernandes, N.W. Paton, O. Corcho, M. Koubarakis, D.D. Roure, K. Martinez,
421 A. Gómez-Pérez. A Semantic Sensor Web for Environmental Decision Support
422 Applications. Sensors. 11, 8855-8887.
423 [pdf: http://strabon.di.uoa.gr/files/sensors-11-08855.pdf]
425 Coming up soon:
426 Strabon will soon support an extension of RDF for incomplete geospatial
427 information. The following publication gives a preview of the relevant research
428 problems:
429 * M. Koubarakis, K. Kyzirakos, M. Karpathiotakis, C. Nikolaou, M. Sioutis,
430 S. Vassos, D. Michail, T. Herekakis, C. Kontoes and I. Papoutsis. Challenges
431 for Qualitative Spatial Reasoning in Linked Geospatial Data. In Proceedings
432 of IJCAI 2011 Workshop on Benchmarks and Applications of Spatial Reasoning,
433 Barcelona, Spain.
434 [pdf: http://www.earthobservatory.eu/publications/SciQL_ADASS2011.pdf]
436 * C. Nikolaou and M. Koubarakis: "Querying Linked Geospatial Data with
437 Incomplete Information". In 5th International Terra Cognita Workshop -
438 Foundations, Technologies and Applications of the Geospatial Web. In
439 conjunction with the 11th International Semantic Web Conference, Boston,
440 USA, November, 2012.
441 [http://www.earthobservatory.eu/publications/iswc-workshop.pdf]
444 Contributors
445 ============
446 The system Strabon has been developed by the following members of our team:
448 * Manos Karpathiotakis <mk@di.uoa.gr>
449 * Kostis Kyzirakos <kkyzir@di.uoa.gr>
450 * Manolis Koubarakis <koubarak@di.uoa.gr>
451 * Giorgos Garbis <ggarbis@di.uoa.gr>
452 * Konstantina Bereta <konstantina.bereta@di.uoa.gr>
453 * Charalampos Nikolaou <charnik@di.uoa.gr>
454 * Stella Gianakopoulou <sgian@di.uoa.gr>
455 * Panagiotis Smeros <psmeros@di.uoa.gr>
456 * Kallirroi Dogani <kallirroi@di.uoa.gr>
459 Mailing-list
460 ============
461 Currently, we maintain the following mailing lists:
463 * Strabon-users, is used as a communication channel for Strabon users.
464 To subscribe to the mailing-list, please visit page
465 http://cgi.di.uoa.gr/~mailman/listinfo/strabon-users. To post e-mails
466 to Strabon-users mailing-list, write to strabon-users@di.uoa.gr.
468 * Strabon-devel, is used as a communication channel with the developers
469 of Strabon. To subscribe to the mailing-list, please visit page
470 http://cgi.di.uoa.gr/~mailman/listinfo/strabon-devel. To post e-mails
471 to Strabon-devel mailing-list, write to strabon-devel@di.uoa.gr.
474 Bugs
475 ====
476 Please report bugs to http://bug.strabon.di.uoa.gr/report or
477 the Strabon-devel mailing-list Strabon-devel@di.uoa.gr.
480 Known Issues
481 ============
482 * By default, Tomcat uses ISO-8859-1 character encoding when decoding URLs received
483 from a browser. This can cause problems when encoding is UTF-8, and you are using
484 international characters. In order to fix this, edit conf/server.xml and find the
485 line where the Connector is defined. Add the parameter URIEncoding and set it to
486 UTF-8. For example:
488 <Connector port="8080" protocol="HTTP/1.1"
489 connectionTimeout="20000"
490 URIEncoding="UTF-8"
491 redirectPort="8443" />
493 * Building and executing any maven goals fails for maven versions <3.0 due to a
494 dependency to the `shade' plugin that is available only for maven version 3.0
495 (http://maven.apache.org/plugins/maven-shade-plugin/). In such systems, you may
496 disable execution of this plugin by setting the environmental variable
497 `shade.skip'. For example, to build Strabon using maven version 2.0 you may
498 execute the following command:
500 $ mvn clean package -Dshade.skip
502 * When using MonetDB as a backend, the following source code of MonetDB must be
503 used:
504 https://hg.strabon.di.uoa.gr/MonetDB/
507 License
508 =======
509 This Source Code Form is subject to the terms of the Mozilla Public
510 License, v. 2.0. If a copy of the MPL was not distributed with this
511 file, You can obtain one at http://mozilla.org/MPL/2.0/.
513 Copyright (C) 2010, 2011, 2012, 2013 Pyravlos Team
515 http://www.strabon.di.uoa.gr/
518 How to apply the license
519 ========================
520 * In the beginning of script files (after the shell directive) paste the
521 following statement:
522 #
523 # This Source Code Form is subject to the terms of the Mozilla Public
524 # License, v. 2.0. If a copy of the MPL was not distributed with this
525 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
526 #
527 # Copyright (C) 2010, 2011, 2012, 2013 Pyravlos Team
528 #
529 # http://www.strabon.di.uoa.gr/
530 #
532 * In the beginning of Java source code files paste the following statement:
533 /**
534 * This Source Code Form is subject to the terms of the Mozilla Public
535 * License, v. 2.0. If a copy of the MPL was not distributed with this
536 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
537 *
538 * Copyright (C) 2010, 2011, 2012, 2013 Pyravlos Team
539 *
540 * http://www.strabon.di.uoa.gr/
541 */
543 * In the beginning of HTML/XML files paste the following statement:
544 <!-- This Source Code Form is subject to the terms of the Mozilla Public
545 - License, v. 2.0. If a copy of the MPL was not distributed with this
546 - file, You can obtain one at http://mozilla.org/MPL/2.0/.
547 -
548 - Copyright (C) 2010, 2011, 2012, 2013 Pyravlos Team
549 -
550 - http://www.strabon.di.uoa.gr/
551 -->