Tag Archives: data query


The development of TRANSFORMERS was co-funded by the HBP during the Ramp-up Phase. This page is kept for reference but will no longer be updated.

Spatial joins are becoming increasingly ubiquitous in many applications, particularly in the scientific domain. While several approaches have been proposed for joining spatial datasets, each of them has a strength for a particular type of density ratio among the joined datasets. More generally, no single proposed method can efficiently join two spatial datasets in a robust manner with respect to their data distributions. Some approaches do well for datasets with contrasting densities while others do better with similar densities. None of them does well when the datasets have locally divergent data distributions.

Therefore, we develop TRANSFORMERS, an efficient and robust spatial join approach that is indifferent to such variations of distribution among the joined data. TRANSFORMERS achieves this feat by departing from the state-of-the-art through adapting the join strategy and data layout to local density variations among the joined data. It employs a join method based on data-oriented partitioning when joining areas of substantially different local densities, whereas it uses big partitions (as in space-oriented partitioning) when the densities are similar, while seamlessly switching among these two strategies at runtime.

Date of release
Version of software
Version of documentation
Software available
ResponsibleEPFL-DIAS: Mirjana Pavlovic (mirjana.pavlovic@epfl.ch)
Requirements & dependencies
Target system(s)


The development of RUBIK was co-funded by the HBP during the Ramp-up Phase. This page is kept for reference but will no longer be updated.

An increasing number of applications from finance, meteorology, science and others are producing time series as output. The analysis of the vast amount of time series is key to understand the phenomena studied, particularly in the simulation sciences, where the analysis of time series resulting from simulation allows scientists to refine the model simulated. Existing approaches to query time series typically keep a compact representation in main memory, use it to answer queries approximately and then access the exact time series data on disk to validate the result. The more precise the in-memory representation, the fewer disk accesses are needed to validate the result. With the massive sizes of today’s datasets, however, current in-memory representations oftentimes no longer fit into main memory. To make them fit, their precision has to be reduced considerably resulting in substantial disk access which impedes query execution today and limits scalability for even bigger datasets in the future. In this paper we develop RUBIK, a novel approach to compressing and indexing time series. RUBIK exploits that time series in many applications and particularly in the simulation sciences are similar to each other. It compresses similar time series, i.e., observation values as well as time information, achieving better space efficiency and improved precision. RUBIK translates threshold queries into two dimensional spatial queries and efficiently executes them on the compressed time series by exploiting the pruning power of a tree structure to find the result, thereby outperforming the state-of-the-art by a factor of between 6 and 23. As our experiments further indicate, exploiting similarity within and between time series is crucial to make query execution scale and to ultimately decouple query execution time from the growth of the data (size and number of time series).

Date of release
Version of software
Version of documentation
Software available
DocumentationMore information: http://infoscience.epfl.ch/record/209731?ln=en
ResponsibleEPFL-DIAS: Eleni Tziritazacharatou (eleni.tziritazacharatou@epfl.ch)
Requirements & dependencies
Target system(s)Tested on Pico supercomputer (CINECA)


The development of MonetDB was co-funded by the HBP during the Ramp-up Phase. This page is kept for reference but will no longer be updated.

When a database grows into millions of records spread over many tables and business intelligence or science becomes the prevalent application domain, a column-store database management system (DBMS) is called for. Unlike traditional row-stores, such as MySQL and PostgreSQL, a column-store provides a modern and scalable solution without calling for substantial hardware investments.

monetDB logo

MonetDB pioneered column-store solutions for high-performance data warehouses for business intelligence and eScience since 1993. It achieves its goal by innovations at all layers of a DBMS, e.g. a storage model based on vertical fragmentation, modern CPU-tuned query execution architecture, automatic and adaptive indices, run-time query optimization, and a modular software architecture. It is based on the SQL 2003 standard with full support of foreign keys, joins, views, triggers, and stored procedures. It is fully ACID compliant and supports a rich spectrum of programming interfaces (JDBC, ODBC, PHP, Python, RoR, C/C++, Perl).

The current version provides the following new features as compared to the version that was part of the HBP-internal Platform Release in M18:

  • Python integration
  • Representation of arrays inside MonetDB
  • MonetDB as a standalone library (MonetDBLite)
Date of releaseOctober 2014, updated in July 2015
Version of software
Version of documentation
Software availablehttp://www.monetdb.org
ResponsibleCWI, Martin Kersten (martin.kersten@cwi.nl)
Requirements & dependencies
Target system(s)Fedora, Ubuntu, Windows, Mac, FreeBSD, CentOS, RHEL, Solaris


The development of SCOUT was co-funded by the HBP during the Ramp-up Phase. This page is kept for reference but will no longer be updated.

SCOUT is a structure-aware method for prefetching data along interactive spatial query sequences. Given the user input, which is a spatial range query sequence representing the structure explored (interactively) by the user, and the spatial dataset to be queried, SCOUT reduces the query response time by prefetching the data along the query sequence.

Similarly to FLAT, both the query ranges in the query sequence and the spatial objects should be represented using a minimum bounding rectangle.

SCOUT outperforms the related prefetching techniques (e.g., Straight Line Extrapolation or Hilbert prefetching) with high prefetching accuracy, which is translated to one order of magnitude speedup.

Date of releaseMarch 2015
Version of software1.0
Version of documentation1.0
Software availableCollaboratory, integrated in and part of BBP SDK tool set
ResponsibleEPFL-DIAS: Xuesong Lu (xuesong.lu@epfl.ch), Darius Sidlauskas (darius.sidlauskas@epfl.ch)
Requirements & dependenciesLinux, Boost library, BBP SDK
Target system(s)PICO supercomputer

Remote Connection Manager (RCM)

The development of Remote Connection Manager (RCM) was co-funded by the HBP during the Ramp-up Phase. This page is kept for reference but will no longer be updated.

The Remote Connection Manager (RCM) is an application that allows HPC users to perform remote visualisation on Cineca HPC clusters.

The tool offers to

  • Visualize the data produced on Cineca’s HPC systems (scientific visualization);
  • Analyse and inspect data directly on the systems;
  • Debug and profile parallel codes running on the HPC clusters.

The graphical interface of RCM allows the HPC users to easily create remote displays and to manage them (connect, kill, refresh).

Screenshot of RCM
Screenshot of Remote Connection Manager (RCM)
Screenshot of RCM
Screenshot of Remote Connection Manager (RCM)
Date of releaseApril 2015
Version of software1.2
Version of documentation1.2
Software availablehttp://www.hpc.cineca.it/content/remote-visualization-rcm
ResponsibleRoberto Mucci (superc@cineca.it)
Requirements & dependenciesThe “Remote Connection Manager” works on the following operating systems: Windows, Linux, Mac OSX
(OSX Mountain Lion users need to install XQuartz: http://xquartz.macosforge.org/landing/)
Target system(s)Notebooks, office computers