Strabon

view README @ 1031:baf571343765

Updated Tester Guide.
author Panayiotis Smeros <psmeros@di.uoa.gr>
date Fri Apr 12 16:44:00 2013 +0300 (2013-04-12)
parents 20ead981bfef
children 8b958fb34f48
line source
1 Strabon
2 -----------
4 Introduction
5 ============
6 Strabon is a fully implemented semantic geospatial database system that can be
7 used to store linked geospatial data expressed in RDF and query them using an
8 extension of SPARQL. Strabon supports spatial selections, spatial joins, a rich
9 set of spatial functions similar to those offered by geospatial relational
10 database systems, support for multiple Coordinate Reference Systems and widely
11 used serializations for geometric objects such as WKT and GML. Strabon is built
12 on top of the well-known RDF store Sesame and extends Sesame’s components to be
13 able to manage thematic and spatial data that are stored in PostGIS.
15 The development of Strabon started in the context of European FP7 project
16 SemsorGrid4Env (Semantic Sensor Grids for Rapid Application Development for
17 Environmental Management) [http://www.semsorgrid4env.eu/]. Starting September
18 2011, Strabon is being utilized and extended with new functionalities in the
19 FP7 project TELEIOS (Virtual Observatory Infrastructure for Earth Observation
20 Data) [http://www.earthobservatory.eu/] which our group leads.
22 The query language of Strabon is called stSPARQL. stSPARQL can be used to query
23 data represented in an extension of RDF called stRDF. stRDF and stSPARQL have
24 been designed for representing and querying geospatial data that changes over
25 time (e.g., the growth of a city over the years due to new developments).
27 Currently, only the geospatial features of stSPARQL have been implemented fully.
28 The temporal features are the subject of current work.
30 Given the very close relationship between stSPARQL and GeoSPARQL which is a
31 recent OGC standard for an extension of SPARQL for querying geospatial metadata,
32 we recently provided support for the Core, Geometry and Geometry Topology
33 extension of GeoSPARQL.
36 Conformance to GeoSPARQL
37 ========================
38 Strabon implements the `Core', the `Topology Vocabulary Extension', the
39 `Geometry Extension', the `Geometry Topology Extension', and the `RDFS Entailment
40 Extension' except for Req. 25
41 (http://www.opengis.net/spec/geosparql/1.0/req/rdfs-entailment-extension/bgp-rdfs-ent).
43 With respect to GML, Strabon supports the GML Profile corresponding to Simple
44 Features, that is, GML Simple Features Profile 2.0.
47 Strabon Homepage
48 ================
49 The homepage of Strabon is at http://www.strabon.di.uoa.gr/.
52 Demo
53 ====
54 You can find a demo of the system Strabon at http://test.strabon.di.uoa.gr/NOA/.
57 Getting Started
58 ===============
59 To get started with Strabon please have a look at the tutorial for the stRDF
60 data model and stSPARQL query language, the User Guide, and the Developer Guide.
62 stRDF and stSPARQL tutorial
63 http://www.strabon.di.uoa.gr/files/stSPARQL_tutorial.pdf
66 stSPARQL Reference
67 ~~~~~~~~~~~~~~~~~~
68 The reference for the spatial and temporal extension functions defined in
69 stSPARQL can be found at http://www.strabon.di.uoa.gr/stSPARQL#spatial and
70 http://www.strabon.di.uoa.gr/stSPARQL#temporals respectively.
73 User Guide
74 ~~~~~~~~~~
75 Assuming that you are familiar with Maven, the following steps need to be
76 followed in order to use Strabon using Eclipse:
78 1. Install PostgreSQL from http://www.postgresql.org/download/. At the time of
79 this writing the latest PostgreSQL version is 9.1.
80 2. Install PostGIS from http://postgis.refractions.net/download/. At the time of
81 this writing we have tested Strabon with PostGIS 1.5.3.
82 3. Install Maven from http://maven.apache.org/download.html. At the time of this
83 writing the latest Maven version is 3.0.4.
84 4. Install Eclipse from http://www.eclipse.org/downloads/. At the time of this
85 writing the latest Eclipse version is 3.7.2.
86 5. Install the m2e plugin for Eclipse from http://www.eclipse.org/m2e/.
87 6. Install the MercurialEclipse plugin for Eclipse from
88 http://javaforge.com/project/HGE .
89 7. From Eclipse, go to File --> Import --> Mercurial --> Clone Existing
90 Mercurial Repository --> Next. In the URL textarea paste the following
91 URL: http://hg.strabon.di.uoa.gr/StrabonUser and then press Next --> Next -->
92 Finish. If you used the default settings, you should have a new project named
93 StrabonMain. Right click on the project and select Configure --> Convert to
94 Maven project. Eclipse will enable Maven dependency management for the
95 project, download any dependencies and build the project.
98 Storing stRDF graphs and evaluating stSPARQL queries
99 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
100 You can see some examples in the classes gr.uoa.di.strabon.example.PostgisExample
101 and gr.uoa.di.strabon.example.PostgisExample2.
103 Tuning PostgreSQL
104 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
105 The default settings of Postgres are rather conservative. As a result, parameter
106 tuning is neccessary for speeding up Postgres, therefore Strabon. If you are
107 using Strabon to compare its performance against your implementation of
108 stSPARQL/GeoSPARQL, you are *strongly* encouraged to contact us using the Strabon
109 Users mailing list for assistance on tuning Postgres.
111 You can follow the instructions below for tuning a Postgres server running on an
112 Ubuntu machine that is dedicated to PostgreSQL and Strabon.
114 1. Append the following text at the end of postgresql.conf.
115 *Uncomment* the appropriate lines.
117 ### RAM
118 ## 4 GB of RAM
119 #shared_buffers = 3GB
120 #effective_cache_size = 3GB
121 #maintenance_work_mem = 1GB
122 #work_mem = 2GB
123 ## 8 GB of RAM
124 #shared_buffers = 5GB
125 #effective_cache_size = 6GB
126 #maintenance_work_mem = 2GB
127 #work_mem = 5GB
128 ## 16 GB of RAM
129 #shared_buffers = 10GB
130 #effective_cache_size = 14GB
131 #maintenance_work_mem = 4GB
132 #work_mem = 10GB
133 ## 24 GB of RAM
134 #shared_buffers = 16GB
135 #effective_cache_size = 22GB
136 #maintenance_work_mem = 6GB
137 #work_mem = 15GB
138 ## 48 GB of RAM
139 #shared_buffers = 32GB
140 #effective_cache_size = 46GB
141 #maintenance_work_mem = 8GB
142 #work_mem = 30GB
143 ## 64 GB of RAM
144 # contact us to find out!
145 ### HD
146 ## RAID with ordinary 7.200 disks
147 #random_page_cost = 3.5 #3.0-3.5
148 ## High-End NAS/SAN
149 #random_page_cost = 2 #1.5-2.5
150 ## Amazon EBS/Heroku
151 #random_page_cost = 1.3 #1.1-2.0
152 ## SSD array
153 #random_page_cost = 2.0 #1.5-2.5
154 ### Misc
155 default_statistics_target = 10000
156 constraint_exclusion = on
157 checkpoint_completion_target = 0.9
158 wal_buffers = 32MB
159 checkpoint_segments = 64
160 ### Connections
161 max_connections = 10
163 2. Append the following lines at the end of /etc/sysctl.conf
164 *Uncomment* the appropriate lines.
166 ## 4 GB of RAM
167 #kernel.shmmax = 3758096384
168 #kernel.shmall = 3758096384
169 #kernel.shmmni = 4096
170 ## 8 GB of RAM
171 #kernel.shmmax = 5905580032
172 #kernel.shmall = 5905580032
173 #kernel.shmmni = 4096
174 ## 16 GB of RAM
175 #kernel.shmmax = 11274289152
176 #kernel.shmall = 11274289152
177 #kernel.shmmni = 4096
178 ## 24 GB of RAM
179 #kernel.shmmax = 17716740096
180 #kernel.shmall = 17716740096
181 #kernel.shmmni = 4096
182 ## 48 GB of RAM
183 #kernel.shmmax = 35433480192
184 #kernel.shmall = 35433480192
185 #kernel.shmmni = 4224
186 ## 64 GB of RAM
187 # contact us to find out!
189 3. Apply all changes by executing
191 $ sudo sysctl -p
192 $ sudo /etc/init.d/postgresql restart
194 4. Prepare for the next run by issuing the command
196 $ sudo -u postgres psql -c 'VACUUM ANALYZE;' db
198 or
200 $ psql -c 'VACUUM ANALYZE;' db
202 where db is the name of the Postgres database that Strabon will use.
205 Developer Guide
206 ~~~~~~~~~~~~~~~
207 Assuming that you are familiar with Maven, the following steps need to be
208 followed in order to use Strabon using Eclipse:
210 1. Install PostgreSQL from http://www.postgresql.org/download/. At the time of
211 this writing the latest PostgreSQL version is 9.1.
212 2. Install PostGIS from http://postgis.refractions.net/download/. At the time of
213 this writing we have tested Strabon with PostGIS 1.5.3.
214 3. Install Maven from http://maven.apache.org/download.html. At the time of this
215 writing the latest Maven version is 3.0.4.
216 4. Install Eclipse from http://www.eclipse.org/downloads/. At the time of this
217 writing the latest Eclipse version is 3.7.2.
218 5. Install the m2e plugin for Eclipse from http://www.eclipse.org/m2e/.
219 6. Install the MercurialEclipse plugin for Eclipse from
220 http://javaforge.com/project/HGE.
221 7. From Eclipse, go to File --> Import --> Mercurial --> Clone Existing
222 Mercurial Repository --> Next. In the URL textarea paste the following
223 URL: http://hg.strabon.di.uoa.gr/Strabon and then press Next --> Next -->
224 Finish. If you used the default settings, you should have a new project named
225 StrabonMain. Right click on the project and select Configure --> Convert to
226 Maven project. Eclipse will enable Maven dependency management for the
227 project, download any dependencies and build the project.
230 Tester Guide
231 ~~~~~~~~~~~~
232 Assuming again that you are familiar with Maven and Junit these are the steps
233 you need to follow to test the functionality of Strabon:
236 * Using Eclipse
238 If you want to create a new test:
240 1. Import Strabon into Eclipse as explained in the Developer Guide.
241 2. Go to strabon-testsuite project.
242 3. Create a new folder (Recommended folder name: <test's name>) and place inside the following files:
243 3.1. An ntriples or nquads file with the test dataset (Recommended file name: <test's name>[.nt, .nq]).
244 3.2. A file with the sparql test query (Recommended file name: <test's name>.rq).
245 3.3. A file with the expected test results in xml format (Recommended file name: <test's name>.srx).
246 4. Create a test class that extends TemplateTest class.
247 5. If you have followed the recommendations the test is ready. If you have different names for your
248 files, insert them explicitly in the constructor of TemplateTest class.
250 If you want to run a test:
252 1. Right-click on the test class.
253 2. Select "Run as JUnit Test".
254 3. Database properties are retrieved from database.properties file. If you want, you can change a property
255 on the fly with an environment variable.
258 * Command Line
260 If you want to run all the tests:
262 1. Go to Strabon directory (root directory of all the subprojects).
263 2. Run "mvn test".
264 3. Optionally you can pass an environment variable with "-DvariableName=variableValue".
267 Storing stRDF graphs and evaluating stSPARQL queries
268 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
269 You can see some examples in the classes
270 eu.earthobservatory.runtime.postgis.StoreOp and
271 eu.earthobservatory.runtime.postgis.QueryOp.
274 Download
275 ========
276 You can download the source code of the latest version of Strabon by accessing
277 our public mercurial repository located at http://hg.strabon.di.uoa.gr/Strabon.
278 You can find more information on how to use and extend Strabon at the Getting
279 Started section.
282 Publications
283 ============
284 You can learn about stRDF data model and stSPARQL query language employed in
285 Strabon by reading our tutorial under the Getting Started section and/or the
286 publications given on this page.
288 The current versions of stRDF and stSPARQL which are based on OGC standards are
289 presented in the following document:
290 * K. Kyzirakos, M. Karpathiotakis, and M. Koubarakis. Strabon: A Semantic
291 Geospatial DBMS. In Internatioanl Semantic Web Conference (ISWC'12). Boston,
292 USA, November 11-15, 2012.
293 [pdf: http://strabon.di.uoa.gr/files/strabon-iswc.pdf]
295 * Manolis Koubarakis, Kostis Kyzirakos, Babis Nikolaou, Michael Sioutis, and
296 Stavros Vassos. A data model and query language for an extension of RDF with
297 time and space. Deliverable D2.1, European ICT project TELEIOS, 2011.
298 [pdf: http://strabon.di.uoa.gr/files/deliv2-1-re-revised.pdf]
300 The initial versions of stRDF and stSPARQL that are based on constraint
301 databases are presented in the following publications:
302 * Manolis Koubarakis and Kostis Kyzirakos. Modeling and Querying Metadata in
303 the Semantic Sensor Web: the Model stRDF and the Query Language stSPARQL.
304 In 7th Extended Semantic Web Conference (ESWC 2010). Heraklion, Crete,
305 30 May - 03 June, 2010.
306 [pdf: http://strabon.di.uoa.gr/files/stSPARQL.pdf]
308 * Kostis Kyzirakos, Manos Karpathiotakis and Manolis Koubarakis. Developing
309 Registries for the Semantic Sensor Web using stRDF and stSPARQL (short
310 paper).
311 In Proceedings of 3rd International workshop on Semantic Sensor Networks
312 2010, in conjunction with ISWC 2010, November 2010, Shanghai, China.
313 [pdf: http://strabon.di.uoa.gr/files/strabon.pdf]
315 Applications of stRDF, stSPARQL, and the system Strabon are described here:
316 * Alasdair J. G. Gray, Raúl García-Castro, Kostis Kyzirakos, Manos
317 Karpathiotakis, Jean-Paul Calbimonte, Kevin Page, Jason Sadler, Alex
318 Frazer, Ixent Galpin, Alvaro A. A. Fernandes, Norman W. Paton, Oscar
319 Corcho, Manolis Koubarakis, David De Roure, Kirk Martinez and Asunción
320 Gómez-Pérez. A Semantically Enabled Service Architecture for Mashups over
321 Streaming and Stored Data. In 8th Extended Semantic Web Conference (ESWC
322 2011). Heraklion, Crete, May 20 - June 2, 2011.
323 [pdf: http://strabon.di.uoa.gr/files/Gray2011Architecture.pdf]
325 * A.J.G. Gray, J. Sadler, O. Kit, K. Kyzirakos, M. Karpathiotakis, J.-P.
326 Calbimonte, K. Page, R. García-Castro, A. Frazer, I. Galpin, A.A.A.
327 Fernandes, N.W. Paton, O. Corcho, M. Koubarakis, D.D. Roure, K. Martinez,
328 A. Gómez-Pérez. A Semantic Sensor Web for Environmental Decision Support
329 Applications. Sensors. 11, 8855-8887.
330 [pdf: http://strabon.di.uoa.gr/files/sensors-11-08855.pdf]
332 Coming up soon:
333 Strabon will soon support an extension of RDF for incomplete geospatial
334 information. The following publication gives a preview of the relevant research
335 problems:
336 * M. Koubarakis, K. Kyzirakos, M. Karpathiotakis, C. Nikolaou, M. Sioutis,
337 S. Vassos, D. Michail, T. Herekakis, C. Kontoes and I. Papoutsis. Challenges
338 for Qualitative Spatial Reasoning in Linked Geospatial Data. In Proceedings
339 of IJCAI 2011 Workshop on Benchmarks and Applications of Spatial Reasoning,
340 Barcelona, Spain.
341 [pdf: http://www.earthobservatory.eu/publications/SciQL_ADASS2011.pdf]
343 * C. Nikolaou and M. Koubarakis: "Querying Linked Geospatial Data with
344 Incomplete Information". In 5th International Terra Cognita Workshop -
345 Foundations, Technologies and Applications of the Geospatial Web. In
346 conjunction with the 11th International Semantic Web Conference, Boston,
347 USA, November, 2012.
348 [http://www.earthobservatory.eu/publications/iswc-workshop.pdf]
351 Contributors
352 ============
353 The system Strabon has been developed by the following members of our team:
355 * Manos Karpathiotakis <mk@di.uoa.gr>
356 * Kostis Kyzirakos <kkyzir@di.uoa.gr>
357 * Manolis Koubarakis <koubarak@di.uoa.gr>
358 * Giorgos Garbis <ggarbis@di.uoa.gr>
359 * Konstantina Bereta <konstantina.bereta@di.uoa.gr>
360 * Charalampos Nikolaou <charnik@di.uoa.gr>
361 * Stella Gianakopoulou <sgian@di.uoa.gr>
362 * Panagiotis Smeros <psmeros@di.uoa.gr>
363 * Kallirroi Dogani <kallirroi@di.uoa.gr>
366 Mailing-list
367 ============
368 Currently, we maintain the following mailing lists:
370 * Strabon-users, is used as a communication channel for Strabon users.
371 To subscribe to the mailing-list, please visit page
372 http://cgi.di.uoa.gr/~mailman/listinfo/strabon-users. To post e-mails
373 to Strabon-users mailing-list, write to strabon-users@di.uoa.gr.
375 * Strabon-devel, is used as a communication channel with the developers
376 of Strabon. To subscribe to the mailing-list, please visit page
377 http://cgi.di.uoa.gr/~mailman/listinfo/strabon-devel. To post e-mails
378 to Strabon-devel mailing-list, write to strabon-devel@di.uoa.gr.
381 Bugs
382 ====
383 Please report bugs to http://bug.strabon.di.uoa.gr/report or
384 the Strabon-devel mailing-list Strabon-devel@di.uoa.gr.
387 Known Issues
388 ============
389 * By default, Tomcat uses ISO-8859-1 character encoding when decoding URLs received
390 from a browser. This can cause problems when encoding is UTF-8, and you are using
391 international characters. In order to fix this, edit conf/server.xml and find the
392 line where the Connector is defined. Add the parameter URIEncoding and set it to
393 UTF-8. For example:
395 <Connector port="8080" protocol="HTTP/1.1"
396 connectionTimeout="20000"
397 URIEncoding="UTF-8"
398 redirectPort="8443" />
400 * Building and executing any maven goals fails for maven versions <3.0 due to a
401 dependency to the `shade' plugin that is available only for maven version 3.0
402 (http://maven.apache.org/plugins/maven-shade-plugin/). In such systems, you may
403 disable execution of this plugin by setting the environmental variable
404 `shade.skip'. For example, to build Strabon using maven version 2.0 you may
405 execute the following command:
407 $ mvn clean package -Dshade.skip
409 * When using MonetDB as a backend, the following source code of MonetDB must be
410 used:
411 https://hg.strabon.di.uoa.gr/MonetDB/
414 License
415 =======
416 This Source Code Form is subject to the terms of the Mozilla Public
417 License, v. 2.0. If a copy of the MPL was not distributed with this
418 file, You can obtain one at http://mozilla.org/MPL/2.0/.
420 Copyright (C) 2010, 2011, 2012, Pyravlos Team
422 http://www.strabon.di.uoa.gr/
425 How to apply the license
426 ========================
427 * In the beginning of script files (after the shell directive) paste the
428 following statement:
429 #
430 # This Source Code Form is subject to the terms of the Mozilla Public
431 # License, v. 2.0. If a copy of the MPL was not distributed with this
432 # file, You can obtain one at http://mozilla.org/MPL/2.0/.
433 #
434 # Copyright (C) 2010, 2011, 2012, Pyravlos Team
435 #
436 # http://www.strabon.di.uoa.gr/
437 #
439 * In the beginning of Java source code files paste the following statement:
440 /**
441 * This Source Code Form is subject to the terms of the Mozilla Public
442 * License, v. 2.0. If a copy of the MPL was not distributed with this
443 * file, You can obtain one at http://mozilla.org/MPL/2.0/.
444 *
445 * Copyright (C) 2010, 2011, 2012, Pyravlos Team
446 *
447 * http://www.strabon.di.uoa.gr/
448 */
450 * In the beginning of HTML/XML files paste the following statement:
451 <!-- This Source Code Form is subject to the terms of the Mozilla Public
452 - License, v. 2.0. If a copy of the MPL was not distributed with this
453 - file, You can obtain one at http://mozilla.org/MPL/2.0/.
454 -
455 - Copyright (C) 2010, 2011, 2012, Pyravlos Team
456 -
457 - http://www.strabon.di.uoa.gr/
458 -->