Swiss ExperimentEPFL-LSIR Report
Hoyoung Jeung
SwissEx Annual Meeting, Zurich 15th June
Deliverable Overview
Framework Overview
capturedcaptured
queriedqueried
processedprocessed
Data Integration
Requires close collaboration with end-users
–BIG efforts to find appropriate solutions to very specific needs
–Continuous migrations of data along with hardware evolution and additionof new types of sensors.
in 2010
–SwissEx general (hosted by SLF) – http://montblanc.slf.ch
–LTE – http://ltepc.epfl.ch
–APUNCH – http://ifu-apunch.ethz.ch:22001/
–PermaSense – http://data.permasense.ch
Continuing in 2011
–CCES general portal - http://montblanc.slf.ch:22001
–CCES public data - http://montblanc.slf.ch:22002/
–Radar images and data from APUNCH: http:// ifu-apunch.ethz.ch:22001
–Disdrometer and radar data from Environmental Remote Sensing Laboratory: http://ltepc3.epfl.ch:22001
Data Repository/Mirroring Services
In addition to the aforementioned instances of GSN, mirrorsome of the data using additional machines at EPFL.
–Near real-time data duplication.
–Multipurpose: backup, query load distribution.
–Accessible only for authorized users
LSIR repository
–Mainly data from SLF, backup purpose
Lafouly repository
–Data from a catchment used by the EU project Hydrosys in Lafouly region.Mainly used for GEOtop model simulations.
PlanetData repository
– Mirroring Wannengrat public data at SLF for query load distribution.
Spatial Data Support
Extending GSN for handling geographical data
Support spatially-enabled processing
–Data types: point, polyline, polygon,…
–Spatial queries: e.g., average wind speed in Lausanne yesterday?
Two implementations
–Java topology suite (JTS) integration: used for any DB
–PostGIS integration
•Only for PostgreSQL DB
•Support spatial index
Output in GML format
Sensor Data Search
Processing in a federated network
–Wiki: central point, easy to use GUI based on Google Maps interface
–GSN instances: distributed, very fast data visualization
Ontology-based
–“temp=temperature=air temp”
–W3C SSN + NASA SWEET
–On-going work
Plan to scale up
–Search beyond swissex data
User-Interactive Data Cleaning
Model-based anomaly detection
–User can select model/parameters, parametric/non-parametric
original data stream
↓
approximation using user-selected models
↓
detecting anomalies
↓
user confirmation: anomaly is an actual error?
New Wrapper Developments
Grid wrapper
–Reads ESRI grids (e.g., digital elevation models) from files and stores as binary objects in DB, to be queried using a new web method (/griddata)
–Returns grids (ESRI ASCII format or rendered images) for a given time range or a stream of values for a single cell within a grid (as a time series).
JDBC wrapper
–Connects to remote DB directly, without going through standard GSNcommunication channels
–More efficient processing, used in PermaSense
Sensorscope wrapper
–The previous Sensorscope wrapper was extended to handle the new packet format and new sensor types (e.g. Decagon 10HS and Apogee SP-212).
Tweeter wrapper
–Enable data fusion between sensor data and Tweeter data in real time
Various Technical Support
Data access via web services
–A standard web service was developed for accessing data in GSN from external tools like Geotop and Alpine3D.
–A C++ client for data access using user’s existing programs.
Continuation of the access control component in GSN
–GSN discretionary access control for each virtual sensor.
–Access can be made through web interface (using interactive login) or through passing of parameters using REST web calls or standard web services.
–Direct passing of parameters without going through the interactive web sessions was needed for automatic tools such as Mathlab or R.
Install, configuration, patches
Continuous development of SwissEx wiki
Planned Work
Linking Swissex data
–e.g., Linked Open Data (LOD) data
–PlanetData (FP7 NoE) consortium
–OpenIoT (FP7 strep)
–Scalable solution necessary
Completion of ontology-based sensor data search
Cloud-based sensor data management
–Mirroring the current data with public/private cloud services
Questions ?