Data management.

T-Storm

,
, , , , ,

The development of T-Storm was co-funded by the HBP during the Ramp-up Phase. This page is kept for reference but will no longer be updated.

T-Storm is a platform for supporting scalable real-time analytics of massive sets of voluminous time-series. The platform is constructed over the Storm parallel dataflow engine, and supports both vertical scalability (fully utilizing high-end servers and multi-core systems) and horizontal scalability (scaling across a cluster of physical machines or even incorporating virtual cloud resources). The current version enables efficient maintenance of the highly correlated time-series in linear space and near-linear computational complexity (in practice, computational complexity depends on the input time-series). This functionality is, for example, useful to identify the pairs of neurons that fire in a correlated manner.

T-Storm is distributed as a prepared virtual machine. To use the platform, the user needs to 1) deploy and configure the required number of virtual machines, depending on the number of time series to monitor, and their velocity; 2) configure the virtual machines so that they have network access and can talk to each other; 3) provide the input in the documented format. Full instructions are provided with the virtual machine.

Date of releaseApril 2015
Version of software0.1
Version of documentation0.1
Software availablehttp://pan.softnet.tuc.gr/hbp/
Documentationhttp://pan.softnet.tuc.gr/hbp/
ResponsibleMinos Garofalakis (minos@acm.org)
Requirements & dependenciesJava (JDK>=7) (https://www.oracle.com/java/index.html)
Storm parallel dataflow engine (https://storm.apache.org)
Target system(s)Cluster

MonetDB

, , , ,
, , , , , , , , ,

The development of MonetDB was co-funded by the HBP during the Ramp-up Phase. This page is kept for reference but will no longer be updated.

When a database grows into millions of records spread over many tables and business intelligence or science becomes the prevalent application domain, a column-store database management system (DBMS) is called for. Unlike traditional row-stores, such as MySQL and PostgreSQL, a column-store provides a modern and scalable solution without calling for substantial hardware investments.

monetDB logo

MonetDB pioneered column-store solutions for high-performance data warehouses for business intelligence and eScience since 1993. It achieves its goal by innovations at all layers of a DBMS, e.g. a storage model based on vertical fragmentation, modern CPU-tuned query execution architecture, automatic and adaptive indices, run-time query optimization, and a modular software architecture. It is based on the SQL 2003 standard with full support of foreign keys, joins, views, triggers, and stored procedures. It is fully ACID compliant and supports a rich spectrum of programming interfaces (JDBC, ODBC, PHP, Python, RoR, C/C++, Perl).

The current version provides the following new features as compared to the version that was part of the HBP-internal Platform Release in M18:

  • Python integration
  • Representation of arrays inside MonetDB
  • MonetDB as a standalone library (MonetDBLite)
Date of releaseOctober 2014, updated in July 2015
Version of software
Version of documentation
Software availablehttp://www.monetdb.org
Documentationhttp://www.monetdb.org
ResponsibleCWI, Martin Kersten (martin.kersten@cwi.nl)
Requirements & dependencies
Target system(s)Fedora, Ubuntu, Windows, Mac, FreeBSD, CentOS, RHEL, Solaris